How to Turn an ASCII Table into an SDSS spectrum? [Ask AstroBetter]

by Kelle on January 18, 2011

[title revised Jan 19]

Since Jane’s questions have proved to be very popular and useful posts, we’ve decided to make “Ask AstroBetter” a regular feature.

Continuing on the theme of spectral analysis, Eilat asks,

Is there a straightforward way to take a table of wavelength, flux, error spanning the same range as SDSS spectra and put it into SDSS spectrum format that is then digestible by SDSS-specific software. If anyone knows how to do this, I imagine it would be useful beyond my own specific needs, since there are lots of codes developed for analyzing SDSS spectra that best recognize that particular file format.

Looking forward to your responses in the comments!

Got a nagging question? Drop us a line at tips@astrobetter.com. Depending on how many questions we get, we may be motivated to get a proper discussion board started.

{ 15 comments… read them below or add one }

1 John O'Meara January 18, 2011 at 8:46 am

In principle, if you *only* care about the wavelength, flux, error of the spectrum, there’s no reason why you couldn’t do the following (I’m making the assumption here that your wavelength scale is identical, pixel by pixel, i.e. the same vales of CRVAL1, and the same logarithmic CDELT of 0.0001, same number of pixels) [IDL is assumed]

1) readcol,’yourfile.dat’,wave,flux,err,format=’f,f,f’

2) read in any sloan 1-d spectrum file with mrdfits, call it ‘foo’, and name the header file ‘hdr’

3) the sloan file is of the following format, [*,0] is the flux array, [*,2] is the sigma array, so
then foo[*,0] = flux
foo[*,2]=err

then write out the file
mwrfits,foo,’newfile.fits’,hdr,/create

now, if your wavelength scale is different, you can replace the values of CRVAL1 and CD1_1 in the header as needed to match your spectrum, but I’m not sure what badness this creates with the other codes you want to use.

all these steps can be easily wrapped up in a code and made scriptable.

Reply

2 Gus January 18, 2011 at 11:32 am

that looks pretty efficient to me and the algorithm would be equally easy in python. Would actual code snippets in either language be useful?

Since you might not know how the SDSS code you wish to use will respond, you may want to ensure that you map all the relevant keys. For example, you may need to update the SDSS special keys COEFF0 and COEFF1 in addition to CRVAL1 and CD1_1. from what i understand its COEFF0=CRVAL1, COEFF1=CD1_1. I do not know which others might factor in except maybe BUNIT (which is a standard key and SDSS uses ‘1.0E-17 erg/cm/s/Ang’ / units) and that may require manipulating your input data.

3 Jane Rigby January 18, 2011 at 11:41 am

“Ask AstroBetter” is a pithier name than “Jane’s innocent-sounding questions.”

Reply

4 John O'Meara January 18, 2011 at 12:20 pm

Good point on BUNIT and the COEFFS Gus!

Reply

5 Tanya January 18, 2011 at 9:28 pm

Using IRAF wspectext. You first need to convert it to 1D spectrum.

Reply

6 Tanya January 18, 2011 at 9:30 pm

Ohm and I used another wspectext for the error spectrum(as said above [*,0] is the flux and [*,2] is the error) and then just merged the tables.

Reply

7 Roy Williams January 19, 2011 at 10:51 am
8 Eilat January 19, 2011 at 11:23 am

I think there was some misunderstanding — I think the title of the post has it backwards — it is easy to get an ascii spectrum (table) from a fits file (e.g., wspectext or readfits). The question is going the other way. What if you have an ascii file with three columns: wavelength, flux, error. How might one convert that ascii file into a spSpec-XXXXX-XXX-XXX.fit file format with the appropriate header info and the proper array in the appropriate dimension?

Reply

9 Gus January 19, 2011 at 12:11 pm

I agree that the title was backwards but how did the first two comments do for answering the question? for a question like this would a snippet of code be best?

10 Tanya January 19, 2011 at 11:43 am

Well, with IRAF the opposite to “wspectext” is “rspectext”. However, you gotta be careful what value to give the parameter “dtype” (I honestly don’t have the parameters in front of my right now, but I think you just input “none” if you have the wavelength values in your table.

http://stsdas.stsci.edu/cgi-bin/gethelp.cgi?rspectext.hlp

You can only do this with x y tables. Once you have the two spectra, you can join them to be multidimensional, i.e. have also the error spectrum associated with it.

Reply

11 Eilat January 19, 2011 at 12:53 pm

Thanks, Gus, I will give it a try. I am worried that the code I am using (GANDALF) needs more than just arrays in the right place and proper dispersion headers. I worry that there are other header parameters that will make it balk, but I am going to try yours and John’s ideas and see where that takes me!

Reply

12 John O'Meara January 19, 2011 at 1:47 pm

@Eliat: That’s why I have you effectively copy the original sdss header over so that all the other ones are in place. Now certainly if the code need a certain header to match the data appropriately, you’ll have to chance that one manually.

Reply

13 Eilat January 21, 2011 at 3:55 pm

Success! I have incorporated your suggestions above with private communications with Adam Myers to produce a little piece of IDL code that should do just what I (and others might) need.

I am happy to share this code on the wiki. Kelle, et al., under what heading do you think I should file such a code?

Cheers,
Eilat

Reply

14 Tom Aldcroft January 24, 2011 at 9:26 pm

Does anybody know about a spec or documentation on the format for SDSS spectra? It must exist somewhere. I balked at trying to answer this question initially because the first thing I did was look at the HDU structure for a random SDSS spectrum I pulled from the archive:

Dataset: spSpec-51930-0285-500.fit
——————————————————————————–

Block Name Type Dimensions
——————————————————————————–
Block 1: PRIMARY Image Real4(3865×5)
Block 2: HDU2 Table 23 cols x 6 rows
Block 3: HDU3 Table 23 cols x 44 rows
Block 4: HDU4 Table 5 cols x 1 rows
Block 5: HDU5 Table 12 cols x 53 rows
Block 6: HDU6 Table 9 cols x 36 rows
Block 7: HDU7 Table 3 cols x 3865 rows

I don’t have a clue what’s in those HDUs 2-7 but presumably they are there for some reason and various “SDSS-specific software” may well use it. Without knowing more I’d be a little wary of just hacking a few header keywords and expecting everything to fly. E.g. for Chandra analysis there is a set of tightly integrated keywords and supporting HDUs — swapping in different data values would almost certainly give wrong answers even if the analysis tools apparently ran successfully.

Reply

15 Ben January 25, 2011 at 12:56 pm

The extra HDU tables in an SDSS spectrum are tables of parameters derived from the spectrum, like line indices. They aren’t metadata that directly influences the calibration of the spectrum (with possible exceptions like the mask values? but that’s really trying too hard). In optical data, calibration metadata is usually given (when it’s given) with header keywords. Descriptions of the SDSS spectrum data model are here:

http://www.sdss.org/dr2/products/spectra/read_spSpec.html
http://www.sdss.org/dr2/dm/flatFiles/spSpec.html

admittedly these pages aren’t that easy to find.

Reply

Leave a Comment

Previous post:

Next post: