Fun with Interactive Visuals in Astrophysics

This is a cross-post from Sudeep Das’s personal blog. Sudeep is a Data Scientist at OpenTable, where his main focus is on mining reviews and scrumptious restaurant data to extract actionable insights and enable a personalized dining experience. For most of this professional life, Sudeep has been an astrophysicist researching the properties of the early universe via the cosmic microwave background (CMB), where he led a study that culminated in the detection of CMB lensing. The original post may be found here.

Recently, I have been playing a lot with interactive data visualizations for my data science introspection work. While making these interactive plots, and experiencing their sheer power in expressing so many aspects of the data in clean and simple charts, I have often wondered how much more enriching such interactive visualizations would have been in my astrophysicist life. The most versatile tool for this purpose is undoubtedly the d3.js javascript library. There is a steep learning curve in mastering d3, and honestly, I am climbing it right now. So maybe, it would have been a bit too much to invest a lot of time in picking up this tool while I was busy writing code, papers and grant proposals.

Luckily, it turns out that there are high level libraries that have been built on top of d3.js, which are way simpler to use, and in as little as half an hour, one could have a rich interactive plot up and running!

So, here is my attempt to share that sliver of wisdom, so that you, my esteemed astrophysicist colleague, can make plots like this pretty easily. I show a case study here with a recent data release from the South Pole Telescope collaboration.

The 2500 square-degree SPT-SZ Cluster Sample

On September 2, 2014 the South Pole Telescope (SPT) collaboration published a paper on the arXiv entitled Galaxy Clusters Discovered via the Sunyaev-Zel’dovich Effect in the 2500-square-degree SPT-SZ survey (Bleem et al. 2014). Incidentally, the work was led by Lindsey Bleem whom I had the pleasure to interact with as a fellow postdoc at the Argonne National Laboratory.  This paper represents solid work of following up 677 Sunyaev-Zel’dovich (SZ) clusters detected (at more than 4.5-σ significance) by the temperature decrements they cause in the 95 and 150 GHz cosmic microwave background (CMB) maps. To confirm these detections, ground- and space-based imaging was used to find their optical and near-infrared (NIR) counterparts. In 516 of the 677 candidates, the counterparts were found and a redshift could be derived. Here is a plot from the paper showing the mass vs. redshift of these 516 clusters as black crosses. From each point on the figure, you can read off the the mass and the redshift of the cluster. But there is so much more information for each galaxy cluster:

  • the cluster ID
  • the detection significance (ξ)
  • whether the cluster has an X-ray compation (XRAY)
  • whether the cluster exhibits strong lensing (SL)
  • the integrated Comptonization parameter, YSK
  • the core radius (θc) (a measure of the size of the central core)

to name the most interesting ones.

So naturally, I was interested in if we could put all this rich information into a single plot!
I came up with two very quick versions of the visualization. They both have all the above info, but they are presented with different purposes in mind.

Here they are:

Visualization 1: I am more interested in the detection significance, and which clusters have strong lensing and/or an X-ray counterparts.

Click on the image below to see the interactive version on Sudeep’s bl.ocks.org page.

Click on the image to get to the interactive version on Sudeep's bl.ocks.org page.What I have done in the above plot is make the size of the bubble for each cluster proportional to its detection significance. You can immediately see that bubbles get bigger as you go higher in the plot (along the mass axis), showing that more massive clusters were detected much more easily (that is a characteristic of the SZ effect). Then, I also color coded the ones that have only SZ, SZ+SL, SZ+XRAY, and SZ+SL+XRAY so that you can quickly investigate the properties of any specific class of cluster (e.g. the strong lensing clusters). Note that all the other information are contained in the tooltip, as you hover over the cluster.

Click on the image below to see the interactive version on Sudeep’s bl.ocks.org page.

sudeep2In this case, I made the size of the cluster bubble proportional to, well, its size (θc), and its color a hue of orange that deepens as the Comptonization parameter YSK increases.

You can visually appreciate here the fact that YSK increases as the mass increases, and that it is fairly insensitive to redshift (this is a fairly typical scaling relation of the SZ effect). You can also see that the low mass, low redshift clusters have larger core radii. If you have spent a lot of time with SZ clusters, you can appreciated various other relations in this plot, which I am not going to belabor with here. Again, all other information is now neatly contained in the tooltip.

Do it yourself

So, how do you make this quickly yourself?

Here’s what I did:

  • For the data wrangling part I used Python. I read in the data from the FITS file using pyfits and then processed it using Pandas. The end result was a tab separated file with 516 rows.
  • For the visualization, I have used a library here called dimple.js that is a high level abstraction of the d3 library. I pretty much took the bubble chart example and started modifying it.

The code is essentially about 30 lines of HTML.

<div id="chartContainer">
<script src="http://cdnjs.cloudflare.com/ajax/libs/d3/3.4.11/d3.min.js"></script>
<script src="http://dimplejs.org/dist/dimple.v2.1.0.min.js"></script>
<script type="text/javascript">
    var svg = dimple.newSvg("#chartContainer", 750, 500);
    d3.tsv("spt_opt_labels.tsv", function (data) {
      var myChart = new dimple.chart(svg, data);
      myChart.setBounds(95, 25, 600, 445)
      var x = myChart.addMeasureAxis("x", "REDSHIFT");
      var y = myChart.addMeasureAxis("y", "M500 (1e14)");
      var z = myChart.addMeasureAxis("z", "XI");
      var s = myChart.addSeries(["SPT_ID","M500 (1e14)","THETA_CORE","YSZ","XI","LABEL"], dimple.plot.bubble);
      s.getTooltipText = function (e) {
                return [
                    "Cluster ID: " + e.aggField[0],
                    "Mass M500 (x 10^14) = " + parseFloat(e.aggField[1]).toFixed(2),
                    "Core radius: " + parseFloat(e.aggField[2]).toFixed(2),
                    "Y_SZ = " + parseFloat(e.aggField[3]).toExponential(2),
                    "Detection significance = "+parseFloat(e.aggField[4]).toFixed(2)
                ];
      };
 
      z.overrideMax = 150.0;
      var myLegend = myChart.addLegend(540, 30, 150, 50, "right");
      myLegend.fontSize = "10px";
      myChart.assignColor("SZ+SL+XRAY", "#a6611a",0.8);
      myChart.assignColor("SZ+XRAY", "#dfc27d");
      myChart.assignColor("SZ", "#80cdc1");
      myChart.assignColor("SZ+SL", "#018571");
      myChart.draw();
    });
 
   </script>
</div>

And that is all you need to produce this powerful graphic!

Learning by Example

To get started, I would suggest downloading the HTML file and tsv file that I have made publicly available. Here is my gist.

The steps.

  • Clone my gist using
 $ git clone https://gist.github.com/d353dd63e013448727b4.git
  • Inside the gist directory, you will find all necessary files. The main file to look at is index.html.
  • Fire up  server from inside that directory using
 $ python -m SimpleHTTPServer 9999
  • Point your browser (no IE8 please!) to localhost:9999
  • You should be able to see the viz on your browser now!
  • Get your own table and make your own viz.
  • Once you are happy with your results, you can post it to a Github Gist and point to it from bl.ocks.org as I have done here.

Any questions for Sudeep? Pointers or information for the rest of the community? Leave them in the comments!

1 comment… add one
  • Brian Hayden Sep 22, 2014 @ 11:37

    I am working on a web server for a current project I’m working on, and I am using mpld3 (http://mpld3.github.io/) along with flask (http://flask.pocoo.org/). The basic workflow is that you make your matplotlib plot however you want, and then you call mpld3.fig_to_html(figure). You can then pass that html to an html template using your favorite templating system (in the case of flask, it’s easiest to use jinja2), either through a post request or through the flask render_template method. A nice example package that got me started is here: https://github.com/nipunreddevil/mpld3-flask

Leave a Reply

Your email address will not be published. Required fields are marked *