Katharina Lutz is a postdoc at the Centre de Données astronomiques de Strasbourg (CDS). She works on gas and star formation in nearby galaxies, and the dissemination of the Virtual Observatory (VO) and the CDS services. This involves maintaining and developing tutorials, and mentoring at VO Schools. This post was written in collaboration with the CDS Python developers, who are currently working on further MOCpy and astroquery developments, and the CDS dissemination team, who are preparing for the next VO training events.
We are back with our miniseries focusing on accessing the CDS services with Python! In our first two articles we introduced ipyaladin to access (all-sky) imaging surveys and MOCs for simple mapping of complex regions on the sky (e.g., survey footprints). In this third article, we will show you how you can access all data stored in VizieR, and crossmatch local and VizieR tables.
VizieR is a database of tables and catalogues from published papers and big surveys and missions. In addition to tabular data, more images, spectra, cubes or light curves (which are associated with a table) become available in VizieR. Both the tabular data and the associated data can be searched and found with Python scripts.
Tables
There are currently three packages that you can use to query VizieR tables:
- the Astropy affiliated package astroquery (see Figure 1),
- through the TAP (Table Access Protocol, an IVOA standard) module of pyVO (see Figure 2), or
- within a MOC using MOCpy (see also their example notebooks).
The astroquery VizieR package enables you to search for certain catalogues, as well as query all VizieR catalogues in a specific region or for a certain object. An example of a basic search for all “SDSS DR9”-related VizieR catalogues is shown below.
Using the astronomy-specific version of SQL (called ADQL), you can query the table of all VizieR catalogues and single VizieR catalogues with pyVO. The first example below shows how you could search for all catalogues with the keywords “2MASS” and “Cutri” in their description. The second code cell shows an example of getting five entries of the 2MASS point source catalogue, which we identified with the first query.
The best thing about all of these queries is that they either directly return an Astropy Table or can be conveniently converted to an Astropy Table. This way you can load any table you need and immediately continue on with your analysis.
Associated Data
Since 2016, VizieR does not only host tables from papers and large surveys but also the associated data. If, for example, a table contained photometric measurements, the associated data could be the images from which the photometric measurements were taken. To search and find associated data that are in the VizieR database, you could go through the webpage. This post is, however, a little overview of what you can do with Python: in this case pyVO is our best friend (see Figure 3). Instead of asking the database of VizieR catalogues (called tables) for catalogues, we can ask the database of VizieR associated data (called obscore) for images at a certain location or associated with a certain catalogue.
In the example above, we asked for five entries in obscore, which belong to the image collection of the MASH Catalogues of Planetary Nebulae (known to VizieR as “V/127A”). The result is a table with a lot of metadata describing the images that are stored in the collection (e.g., the observatory and instrument), the wavelength range, and also a URL, where you can find the image (or data product in general).
Crossmatching of (big) tables with XMatch
When talking about tables, I often end up speaking of combining and crossmatching tables as well. One very efficient way of crossmatching two large tables is with the CDS XMatch service. And guess what? There is a neat astroquery package for the XMatch service as well. While you can go to the webpage, where it’s possible to upload your own tables and crossmatch them with each other or any available VizieR table, the XMatch module in astroquery allows you to do the same tasks from within a Python script or a Jupyter notebook (see Figure 4).
In this example, we show how you can crossmatch a local table (saved as “Data/2MASS_PSC.vot”) with the SDSS DR9 table in VizieR. The result is once again returned as an Astropy Table so that you can go straight ahead to the next analysis step.
If you want more examples, check out our tutorials on GitHub and if you like what you have seen about PyVO in this article, head over to our friends at NASA, where the team has compiled tutorials to show you the full power of PyVO.
As always, tell us how these tools work for you and contact our helpdesk for questions. We also welcome issues on GitHub, and are on Twitter, Facebook, and YouTube.
P.S. We not only love to hear your feedback but would also love to host your data!