One of the most popular posts on this blog is Kelle’s reposting of Greenfield and Jedrzejewski’s IDL vs Python earlier this year. I am an avid Python user and have never installed IDL on my computer. Recently, I developed IDLSave, a package to read IDL ‘save’ files into Python, after being sent a few such files by collaborators. In this package, scalars are converted to their respective Python types, while arrays and structures are mapped to Numpy arrays and recarrays respectively. Most importantly, IDLSave does not require an installation of IDL.

With this package, there will be no need to bug your collaborators who send you IDL save files to resend them in a different format. This also means that if you are an IDL user holding back from switching over to Python because you have a large number of legacy IDL save files, you can now read these files directly into Python without first re-converting them to another format.

Using IDLSave is easy. Let’s say you (or a collaborator) writes a save file with

IDL> save, wavelength, flux, file='spectrum.sav'

you will now be able to open this file in Python using

>>> import idlsave
>>> s = idlsave.read('spectrum.sav')

after which you will be able to access the values using s.wavelength and s.flux.

The IDLSave homepage includes download and installation instructions. If you encounter problems reading in save files, don’t give up – please submit a bug report through the bug tracker, and I will look into it! One of the main limitations worth noting is that while it is possible to read save files into Python, it isn’t (currently) possible to write save files for reading back into IDL. It also is not possible to read special variables such as pointers or system variables from save files.

Enjoy!

Here is a really nice listing of the pros and cons of IDL and Python for astronomers. It’s an Appendix of the Python tutorial Using Python for Interactive Data Analysis by Greenfield and Jedrzejewski at STScI. Data and scripts for the examples and exercises in the tutorial are available at scipy.org.

Update 28 Jan 2010: We’ve transferred this article to the wiki so that it can be updated as Python evolves.

Why would I switch from IDL to Python (or not)?

by Greenfield and Jedrzejewski

We do not claim that all, or even most, current IDL users should switch to using Python now. IDL suits many people’s needs very well and we recognize that there must be a strong motivation for starting to use Python over IDL. This appendix will present the pros and cons of each so that users can make a better informed decision about whether they should consider using Python. At the end we give a few cases where we feel users should give serious consideration to using Python over IDL.

Pros and Cons are addressed below in a comparative sense. Attributes that both share, e.g., that they are interpreted and relatively slow for very simple operations, are not listed.

Pros of IDL

• Mature many numerical and astronomical libraries available
• Wide astronomical user base
• Numerical aspect well integrated with language itself
• Many local users with deep experience
• Faster for small arrays
• Easier installation
• Good, unified documentation
• Standard GUI run/debug tool (IDLDE)
• Single widget system (no angst about which to choose or learn)
• SAVE/RESTORE capability
• Use of keyword arguments as flags more convenient

Cons of IDL

• Narrow applicability, not well suited to general programming
• Slower for large arrays
• Array functionality less powerful
• Table support poor
• Limited ability to extend using C or Fortran, such extensions hard to distribute and support
• Expensive, sometimes problem collaborating with others that don’t have or can’t afford licenses.
• Closed source (only RSI can fix bugs)
• Very awkward to integrate with IRAF tasks
• Memory management more awkward
• Single widget system (useless if working within another framework)
• Plotting:
  • Awkward support for symbols and math text
  • Many font systems, portability issues (v5.1 alleviates somewhat)
  • not as flexible or as extensible
  • plot windows not intrinsically interactive (e.g., pan & zoom)

Pros of Python

• Very general and powerful programming language, yet easy to learn. Strong, but optional, Object Oriented programming support
• Very large user and developer community, very extensive and broad library base
• Very extensible with C, C++, or Fortran, portable distribution mechanisms available
• Free; non-restrictive license; Open Source
• Becoming the standard scripting language for astronomy
• Easy to use with IRAF tasks
• Basis of STScI application efforts
• More general array capabilities
• Faster for large arrays, better support for memory mapping
• Many books and on-line documentation resources available (for the language and its libraries)
• Better support for table structures
• Plotting
  • framework (matplotlib) more extensible and general
  • Better font support and portability (only one way to do it too)
  • Usable within many windowing frameworks (GTK, Tk, WX, Qt…)
  • Standard plotting functionality independent of framework used
  • plots are embeddable within other GUIs
  • more powerful image handling (multiple simultaneous LUTS, optional resampling/rescaling, alpha blending, etc)
• Support for many widget systems
• Strong local influence over capabilities being developed for Python

Cons of Python

• More items to install separately
• Not as well accepted in astronomical community (but support clearly growing)
• Scientific libraries not as mature:
  • Documentation not as complete, not as unified
  • Not as deep in astronomical libraries and utilities
  • Not all IDL numerical library functions have corresponding functionality in Python
• Some numeric constructs not quite as consistent with language (or slightly less convenient than IDL)
• Array indexing convention “backwards”
• Small array performance slower
• No standard GUI run/debug tool
• Support for many widget systems (angst regarding which to choose)
• Current lack of function equivalent to SAVE/RESTORE in IDL
• matplotlib does not yet have equivalents for all IDL 2-D plotting capability (e.g., surface plots)
• Use of keyword arguments used as flags less convenient
• Plotting:
  • comparatively immature, still much development going on
  • missing some plot type (e.g., surface)
  • 3-d capability requires VTK (though matplotlib has some basic 3-d capability)

Specific cases where using Python provides strong advantages over IDL

• Your processing needs depend on running a few hard-to-replicate IRAF tasks, but you don’t want to do most of your data manipulation in IRAF, but would rather write your own IDL-style programs to do so (and soon other systems will be accessible from Python, e.g., MIDAS, ALMA, slang, etc)
• You have algorithms that cannot be efficiently coded in IDL. They likely won’t be efficiently coded in Python either, but you will find interfacing the needed C or Fortran code easier, more flexible, more portable, and distributable. (Question: how many distributed IDL libraries developed by 3rd parties include C or Fortran code?) Or you need to wrap existing C libraries (Python has many tools to make this easier to do).
• You do work on algorithms that may migrate into STSDAS packages. Using Python means that your work will be more easily adapted as a distributed and supported tool.
• You wish to integrate data processing with other significant non-numerical processing such as databases, web page generation, web services, text processing, process control, etc.
• You want to learn object-oriented programming and use it with your data analysis. (But you don’t need to learn object-oriented programming to do data analysis in Python.)
• You want to be able to use the same language you use for data analysis for most of your other scripting and programming tasks.
• Your boss makes you.
• You want to be a cool, with-it person.
• You are honked off at ITT Space Systems/RSI.

Obviously using a new language and libraries entails time spent learning. Despite what people say, it’s never that easy, especially if one has a lot of experience and code invested in an existing language. If you don’t have any strong motivations to switch, you should probably wait.

Reproduced with permission. Copyright 2007, Association of Universities for Research in Astronomy, Inc (AURA).

Update 28 Jan 2010: We’ve transferred this article to the wiki so that it can be updated as Python evolves.

We’ve all seen it before, illegible labels on a figure in a paper or talk. I don’t know how to remedy this problem in SuperMongo or other popular plotting programs, but I do in IDL. I’ve addressed the topic of postscript output on my IDL figures page but in this post I’m just going to focus on the issue of figure size and fonts. Symbols, line thickness, and colors will be discussed in future posts.

BAD: 18 cm across and vector font

BETTER: 9 cm across and Postscript font

Make the figure close to the actual printed size

First thing, size matters. That old manuscript preparation document that said figures should fill a letter-sized page is misguided. No matter what type of font you use, if the text size looks right when the figure is 8 inches across, it will be too small when printed as a single column figure only 9 cm across. The BAD figure above is designed at 18 cm across while the BETTER one is 9 cm. Notice the difference in font size. (Charsize was not specified for either plot.) Design the figures to be tiny from the beginning and no further adjustments to the font size will be necessary for either publications or talks.

When these “small from the start” figures are blown up to fill a letter-sized page (i.e., in Latex preprint mode), the fonts might seem too large and look funny to your eye but, since it ensures that nobody will ever have trouble reading your axis labels, you should just get used to it.
aspect_ratio=1.5 ;rectangle
device, xsize=9, ysize=xsize/aspect_ratio

Use Postscript instead of Vector fonts

Inevitably, when a figure is illegible during a talk, a vector (Hershey) font was used instead of a Postscript font. For the purposes of 2D figures, the relevant difference is that vector fonts are thin and have a tendency to disappear while Postscript fonts are thick. The above BAD figure uses the default vector font while the BETTER one uses a Postscript font.

It’s so simple to switch from vector to device fonts! All you need is !p.font=0. While you’re at it, you might as well specify Helvetica for good measure even though, as far as I can tell, it’s the default Postscript typeface.

set_plot, 'ps'
!p.font=0
device, /helvetica ; a classic sans-serif font

Everything you ever wanted to know about fonts in IDL is in the Reference Guide.

Putting it together

aspect_ratio=1.5
xsize=9
ysize=xsize/aspect_ratio
set_plot, 'ps'
!p.font=0
device, filename='fig_better.eps', encapsulated=1, /helvetica
device, xsize=xsize, ysize=ysize
plot, a, b, xtitle='X Title', ytitle='Y Title'
device, /close
set_plot, 'x'
!p.font=-1

Know how to get nice legible fonts in other plotting programs? Share it in the comments.