With the release of ATpy 0.9.4 and asciitable 0.2.4, it’s about time we told our readers how incredibly easy it is to create, read, write, and manipulate tables and databases of data in Python!
Most, if not all of us have to manage tabular data at one point or another, and it is safe to say that the number of different formats for tables is well beyond ridiculous. FITS, VO, HDF5, IRSA/IPAC, MRT/CDS, DAOphot, RDB, and CSV tables are some of the most common (and I’ll spare you the acronym definitions). In addition, most of us shy away from using databases, because the learning curve is steep, and often requires learning new tools. This is where asciitable and ATpy come in.
The goal of asciitable, developed by Tom Aldcroft, is to provide an extensible framework for reading in tables of ASCII data in arbitrary formats. If the table format you are using is not one of the presets, you can easily define your own!
But that is only half of the story. Manipulating tables (adding/removing/renaming columns, sorting, etc.) is still not easy, and asciitable does not help with non-ASCII formats. Enter ATpy, a tool that Eli Bressert and I have developed, which brings everything together in an easy-to-use package. ATpy uses asciitable to read ASCII tables, but also uses other packages to directly read and write from FITS, VO, and HDF5 tables, and can also talk to SQL databases, including MySQL, PostGreSQL, and SQLite. Reading tables is a one liner for each format:
>>> import atpy >>> t = atpy.Table('mytable.fits') # FITS table >>> t = atpy.Table('mytable.hdf5') # HDF5 table >>> t = atpy.Table('mytable.xml') # VO table >>> t = atpy.Table('mytable.tbl') # IRSA/IPAC table >>> t = atpy.Table('sqlite', 'mytable.db') # SQLite database >>> t = atpy.Table('mysql', db='python', user='bob', \ passwd='1(@N(!10))') # MySQL database >>> t = atpy.Table('postgres', database='python', \ user='alice', password='1(@N(!10))') # PostGreSQL database
and so on. Writing tables is equally easy:
>>> t.write('mynewtable.fits') # FITS >>> t.write('mynewtable.hdf5') # HDF5 >>> t.write('mynewtable.xml') # VO etc.
The file type is automatically recognized from the extension, but you can always override the default or define your own extensions. You can also create tables from scratch:
>>> import atpy >>> import numpy as np >>> ra = np.random.random(5) >>> dec = np.random.random(5) >>> flux = np.random.random(5) >>> t = atpy.Table() >>> t.add_column('ra', ra, unit='deg') >>> t.add_column('dec', dec, unit='deg') >>> t.add_column('flux', flux, unit='mJy')
Either way, accessing and modifying the data is simple and straightforward:
>>> t['ra'] array([ 0.2331109 , 0.23474117, 0.62186612, 0.65640893, 0.71596516]) >>> t['ra'] = 0.1 >>> t['ra'] array([ 0.2331109 , 0.1 , 0.62186612, 0.65640893, 0.71596516])
as is manipulating tables:
>>> t.sort('ra') >>> t.rename_column('ra','ra_j2000') >>> t.remove_column('flux')
Hopefully, if you are not already using ATpy, we’ve convinced you to at least give it a try! Installation instructions and a user-friendly manual are located here. We’ve recently created a new user discussion group at Google groups, which we encourage you to make use of! To read ASCII tables, you will also need to install asciitable. For more general information about using Python in Astronomy, we encourage you to visit the astropython.org web site!