One of the most popular posts on this blog is Kelle’s reposting of Greenfield and Jedrzejewski’s IDL vs Python earlier this year. I am an avid Python user and have never installed IDL on my computer. Recently, I developed IDLSave, a package to read IDL ‘save’ files into Python, after being sent a few such files by collaborators. In this package, scalars are converted to their respective Python types, while arrays and structures are mapped to Numpy arrays and recarrays respectively. Most importantly, IDLSave does not require an installation of IDL.
With this package, there will be no need to bug your collaborators who send you IDL save files to resend them in a different format. This also means that if you are an IDL user holding back from switching over to Python because you have a large number of legacy IDL save files, you can now read these files directly into Python without first re-converting them to another format.
Using IDLSave is easy. Let’s say you (or a collaborator) writes a save file with
IDL> save, wavelength, flux, file='spectrum.sav'
you will now be able to open this file in Python using
>>> import idlsave
>>> s = idlsave.read('spectrum.sav')
after which you will be able to access the values using s.wavelength and s.flux.
The IDLSave homepage includes download and installation instructions. If you encounter problems reading in save files, don’t give up – please submit a bug report through the bug tracker, and I will look into it! One of the main limitations worth noting is that while it is possible to read save files into Python, it isn’t (currently) possible to write save files for reading back into IDL. It also is not possible to read special variables such as pointers or system variables from save files.
Enjoy!
Hi,
thanks a lot for the tool you’ve developed for reading IDL sav files.
I would like to ask you how do you manage to concatenate two or more output variables of the idlsave procedure preserving the field names. Using the numpy functions “r_” or “concatenate” the field names are lost.
Looking forward to your answer and thanks again.
Diego
@Diego: Do you mean specifically variables that are structured arrays? (i.e. structures from IDL). If so, then this might be helpful: http://stackoverflow.com/questions/5355744/numpy-joining-structured-arrays (in particular the last answer on that page)
@Tom: yes, I mean IDL structures. My question is about how to join 2 (or more) IDL structures, which have the same fields, so that the resulting variable keep the same fields (and their names) of the original IDL structure. The link you suggest could be useful, however, the functions in there try to add the fields of every IDL structure as new fields, instead of joining those fields which have the same name. The output sends an error message as: “the are two fields with the same name”.
Example:
What I have:
var1={field1 (string1), field2 (array1)}
var2={field1 (string2), field2 (array2)}
What I want:
var=[var1, var2]
var.field1=[string1, string2] ; var[0].field1= string1; var[1].field1=string2
var.field2=[array1, array2] ; var[0].field2= array1; var[1].field2=array2
What it seems to happen:
var={field1, field2, field1, field2} –> subsequent error: “field1 (first)” has the same name of “field1” (second).
Thanks for your help.
Diego
I just wrote a function to easily merge two structured arrays: https://gist.github.com/2552867 – can you try it out?
@Tom: that’s right, your function works perfect. I’m sorry the delay in my answer, but I had some vacation days in Spain and I didn’t check the mail. I’m new in Python and I might ask silly questions. I see from your function that the key was in the ‘resize’ function. Now, I’m going to learn how to create attributes, so that, given the structured array:
a = array(zip([1,2,3],[‘a’,’b’,’c’]), dtype=[(‘col1’, int), (‘col2’, ‘S1’)])
I can type: print a.col1
and get:
[1,2,3]
thanks a lot Tom.
Diego
@Diego – here’s how to convert a structured array into a record array (which allows attributes):
a = np.array(zip([1,2,3]), dtype=[(‘a’,int)])
b = a.view(np.recarray)
print b.a
@Tom: thanks for your very useful tutorial.
@Tom
thanks for your very useful tutorial. Diego
Tom,
Is there a way to be able to write to an IDL save file from Python?
No, there is no support for writing IDL save files, and I don’t have plans to implement it. However, the format is documented, so it should be possible if anyone wants to give it a shot (if you do, please let me know!)
Thanks a lot for this!
However. I’ve got an error when tried to read a *.sav file: UNKNOWN RECTYPE 20
After adding to idlsave.py RECTYPE_DICT[20] = “UNKNOWN” it works fine. It looks that I can read all the data I need.
There is also a SciPy module for this now: scipy.io.readsav. It creates a dictionary from the .sav file instead of an object like Tom’s code. Although we are all indebted to Tom for generously making his code available some five years ago, the SciPy option might be preferable just for portability.
http://docs.scipy.org/doc/scipy/reference/generated/scipy.io.readsav.html#scipy.io.readsav
How can we read huge binary files, when the size is in tens of GB?