In June 2011, the announcement of a new Python package for Astronomy on the astropy mailing list prompted a long thread that started as a criticism of the proliferation of independently-developed Astronomy Python packages but quickly became the start of a common effort to develop a single core package for Astronomy. Many hundreds of emails, and one workshop later, the project is well underway! Development is actively ongoing, and it is likely that we will see an initial public release in the summer, if not sooner!
The Astropy project, as it is now called, brings together almost 90 developers from around the world. The project is coordinated by Perry Greenfield (STScI), Erik Tollerud (UC Irvine), and myself (Tom Robitaille, MPIA). We have written a long-term vision and have a code repository on GitHub that already contains thousands of lines of code, including not only ‘framework’ code (testing, configuration, etc.), but also functional astronomy submodules such as astropy.wcs and astropy.io.vo (ported over by Mike Droettboom, STScI, from pywcs and vo). PyFITS is almost ready for inclusion as astropy.io.fits (thanks to Erik Bray, STScI), and a number of other major components are being finalized.
Contributing to Astropy: Affiliated Packages
One of the key requirements of the Astropy project is that code that gets included in the package should be of a high standard, and with documentation and tests already included. This means that development of many sub-packages is not done directly in the main Astropy repository, but is done separately, and code is merged in once these criteria are met. To this end, we have converged on a system we refer to as ‘affiliated packages’. Essentially, these are self-contained packages that implement specific functionality (for example photometry, or spectrum-related functionality). Affiliated packages may be merged into the Astropy package once ready, although these can remain independent if they are too large, too specialized, have license conflicts with the core package, or if the author simply wants to maintain autonomy in their development cycle. An installation/configuration tool is in the works which should simplify installing affiliated packages, simplifying the process of gathering up the python astronomy tools you need.
If you are interested in developing an affiliated package for Astropy, first check here whether anyone is already working on a similar package. If so, then please avoid duplicating efforts, and contact the maintainer of the affiliated package to offer your help. Otherwise, read the README.rst file in the affiliated package template, then add your package to the list on the wiki page.
Other ways to contribute to Astropy
If you are interested in improving existing components of Astropy, you do not need to create an affiliated package. In that case, you can just fork the Astropy repository, work on the code, then submit a pull request which we can then review, and iterate on, before merging. Instructions for doing this are included here.
Once we start moving towards a first public release, we will ask for volunteers to try out the package, and read over and improve the documentation and write tutorials, so even if you don’t want to code, there are still ways in which you can help!
Finally, if you are interested in staying up to date with the development of Astropy, you can join the astropy-dev mailing list; the astropy mailing list will remain a general purpose Astronomy Python mailing list.
Nice, almost started a project like this as well. For example, the quite exhaustive idl-astro library should be ported to Python as well, is that being worked on?
Another question: How are dependencies to other libraries handled? For example, I wrote some Mars related stuff, that uses a lot osgeo’s gdal module. Is there a base distribution that everybody can rely on to exist? As Python suffers quite a bit of dependency hell, this is hopefully carefully managed. My private favorite for being nicely architecture/machine-independent is the Enthought Python distribution, where my code runs the same on Win/Lin/Mac, and I think it’s important to have good base distribution for scientists not having to manually install a lot of extra stuff; but I guess there are other good ones as well. Once I get my next PostDoc I certainly will join the community, shared development via Github is immensely productive and efficient … and fun! 😉
There’s been a lot of discussion about how to handle the idl astro library. I am personally not a fan of the idea of directly porting the whole thing, since the IDL library is itself a mishmash of routines without any real coherent API. There are individual algorithms that are great, without question, but I would rather see the community put its effort into developing a more Pythonic, object-based, unified system rather than just do a bulk port of all the existing IDL code.
That said, as a middle ground, there’s already been created an affiliated package for direct IDL ports, so that people who want to use & work on such can do so there (with the understanding that it’s probably a temporary solution on the way to developing a more coherent system in Python!)
@Michael: one of the affiliated packages is meant to be a port of the IDL astrolib routines – see here for more details.
Astropy internally allows dependencies to Numpy, Scipy, and Matplotlib, but other dependencies are allowed, with the caveat that if they are not present, the build or import should not fail (just disable those components). In some cases, if the license allows, we can include the dependency in Astropy (we’ve already done that for configobj, py.test, and expat).
Finally, you mention the Enthought distribution – once we have a released package, I know that there are people interested in developing an all-inclusive package like you mention. Like Numpy, Scipy, and Matplotlib, the primary released product for Astropy will be a normal Python package, and anyone is then free to develop a installer such as the Enthought distribution if they wish.
I think this initiative is most welcome. I have now essentially switched over to python and it has made my life much easier thanks to the power and the flexibility of the language and the available libraries. I still have IDL around for the occasional legacy script though. Hopefully astropy will develop into an extensive library with a consistent API. I will not waste the opportunity to advertise this library in the lab and to make a package for my favourite linux distribution to make the installation process totally trivial.
I also don’t think the IDL Astro Pro Library should be ported over, but I do think we do need a translation table to help he IDL-to-Python switchers quickly figure out how to do what. Here is a start, but it needs a lot of filling in and I suspect a lot will change with the new Astropy package: https://www.astrobetter.com/wiki/tiki-index.php?page=Python+Switchers+Guide
@Kelle Thanks, that’s a nice start. But: ergs? cm^2? Seriously? 😉 *just_teasing*