Given the useful responses to Jane’s question about spectral line analysis, here’s another query for the community. This one is about making large image mosaics and it comes from Adam Ginsburg, a grad student at the University of Colorado, Boulder.
I want to make a large-scale mosaic of the Galactic Plane covering 90-180 degrees x a few degrees. There are a few ways to do this:
- IRAF’s imcombine with offset=wcs is a quick and dirty path to a mosaic. It is very fast, but very inaccurate over large scales and it sometimes does things flat-out wrong.
- IDL’s hastrom and related astrolib utils generally work, but for large arrays (96000×4000 pixels) I’ve seen IDL just crash.
- IRSA’s Montage works well, but is slow and consumes a lot of hard drive space in the process of making mosaics. The setup is also kind of a pain; you need to make a lot of subdirectories and .tbl files in the process.
- SSC’s mopex must work quite well, but I just discovered it today and, like Montage, it has a learning curve.
So, I’m curious, what do others use? For example, what did GigaGalaxy Zoom use? If you’re trying to build a large near-IR finder chart using 2MASS, do you just go to montage? Does anyone have convenient wrapper scripts written up? Let’s hear it in the comments.
{ 10 comments… read them below or add one }
SWarp?
http://astromatic.iap.fr/software/swarp
I haven’t used this for science per se, but to visualize what a field looked as I did analysis on several separate images that spanned a wider field.
swarp! for sure. i use it all the time and it works great. not a very steep learning curve — short manual. If you are familiar with SExtractor then it is even easier to pick up. If your images have standard WCS header keywords then it is a piece of cake.
I used to sit on the IRSA users group. If folks have comments on how to improve Montage, I would like to pass them on to IRSA.
I have used the Montage web interface to effortlessly make big 2MASS and SDSS mosaics. Perfect for my findercharts, and a 3rd color for Warm Spitzer pretty RGB pictures. Took me 5 min as a complete newbie.
Like Adam, I’ve casually tried running Montage on my own computer to make mosaics from my own data, since it’s extremely powerful and produces wonderful results. Like Adam, I was also frustrated by the setup complexity and lack of a start-to-finish cookbook. Can others comment on their use of Montage, or compare it to other mosaickers?
my experience is that swarp is designed for coadding and does not have the background matching capabilities of montage/mopex. montage seems to scale fine in big pipelines e.g. swire/glimpse.
what i’ve wanted is a python api to montage…
I use Montage all the time. I wrote a small python function that takes care of setting up the directories and cleaning up afterwards, so that all one needs to specify is a directory containing the input files to mosaic, the header to use (or let it choose the header automatically), and the output filename. Maybe I should make this available?
Thomas, based on what I’ve seen of APLpy, you should definitely make your montage wrappers available.
I was completely unaware of SWarp – thanks guys!
re: Jane – The quick cutouts are fine, and the big, large-scale projects are also fine, but I have very often been in situations where I just want to stitch stuff together quickly. Also, “big” is a relative term… 1 square degree isn’t much when you’re looking to compare to galactic plane surveys. I’ve successfully produced ~6 square degree 2MASS fields with MONTAGE but it took a couple days per image on my really fast computer. Also, if you want your output image bigger than the inputs, I’ve found you have to hack around it by passing a blank image of the desired size as one of the images to co-add.
A little follow-up….
swarp is actually more along the lines of what I’ve been hoping for: a single command-line program with sensible defaults but very extensible via configuration file. It also defaults to removing unnecessary temp files.
montage has some great features, but is split up into a lot of separate commands. I’ve particularly had problems (today!) with making a mosaic that matches the input header exactly (i.e. cropping), while in the past I’ve had difficulty but figured out how to do the opposite (make larger).
Are there more arguments in favor of montage over swarp, or is it just a matter of preference, perhaps based on side of the atlantic?
Sort of related – does either program work well with data cubes? Or are there any good data-cube-mosaicing tools? (STARLINK does OK on this for JCMT data, at least)
John Good (Montage architect) and myself (Montage project manager) are posting this note to clarify some of the comments made on this page. Montage is actively supported, and we welcome suggestions for upgrades and improvements. Please, contact us with requests and suggestions at the Montage Help desk at
http://irsa.ipac.caltech.edu/cgi-bin/Helpdesk/nph-genTicketForm?projname=Montage&projmail=montage@ipac.caltech.edu
On-request Mosaics
There is an on-request mosaic service at http://hachi.ipac.caltech.edu:8080/montage
This service returns mosaics of 2MASS, SDSS and DSS images. In the case of SDSS it is the currently the only place you can get photometrically-scaled Sloan date (including from the project itself). You simply select the parameters of the mosaic you want on a web form, and the service will notify you when the mosaic is done. You can enter a number of requests and monitor progress on a web page. In Spring 2010, we plan to extend this service to new data sets, as well as to extend functionality to, e.g., deliver 3-color images. We welcome feedback from users on new functionality and which new data sets may be most useful. For many users, this is the simplest way of getting mosaics.
Montage vs. Swarp
Swarp and Montage are very different products and satisfy different use cases. Swarp is a fine product and is best suited for a fast, browse quality mosaic, or if you want to a co-add a collection of images (as was pointed out on this web page). Beware that the output is not strictly flux-conserving. Montage, by design, preserves the flux and astrometric fidelity of the input images. And it rectifies the background radiation in the input images to a common level, which is essential interpreting the mosaics in regions with extended emission.
Speed
Part of the price that Montage pays for a “science grade” mosaic is speed – it takes a lot of crunching to generate such mosaics, especially if projections such as Aitoff are involved. They are computed by a general algorithm that applies to all WCS projections. We have sped up Montage by more than an order of magnitude in cases where only tangent plane projections are involved. Montage avoids costly plane-to-sky-to-plane deprojection/projection calculations in these cases using a fast “plane-to-plane library,” a modification on the algorithm that MOPEX uses.
Usability
Montage is designed as a toolkit rather than a single executable. This design has many benefits:
- it allows end users to adapt the code for their own use cases; a number of Spitzer Legacy teams have, for instance, installed it into their image processing pipelines.
- it makes Montage scalable across large-scale parallel systems
- the many files it produces are necessary to do proper background matching and assure photometric and astrometric accuracy. The files are a very powerful QA tool and we recommend users examine them.
Now, there is an “executive” (mExec) which you can point at an
existing set of image files and which will do the end-to-end
processing for you. See
http://montage.ipac.caltech.edu/docs/mExec.html
It tries to minimize storage, deleting unneeded files as it goes. This is a good solution for users who may want occasional mosaics. We do recommend that regular users examine the intermediate files and understand what the processing does to the images.
There is an end-to-end cookbook tutorial at
http://montage.ipac.caltech.edu/docs/m101tutorial.html
If this does not satisfy your needs, please let us know what further tutorials and documentation will be valuable.
- John Good and Bruce Berriman (IPAC, Caltech)
An important note on Montage is that it does not conserve flux when changing the image resolution. For example, if you want to rebin the image to a pixel size three times larger, the flux of all point sources will decrease by a factor of 9 (3^2). Multiplying by the ratio of the pixel areas, (new_pixel_size/old_pixel_size)^2, will get you back to the original flux. This can be done via the “-x” flag in Montage’s mProject. A “conserve flux” flag in Montage, as in MIDAS, would be nice though.
In response to Loren Anderson’s comment: Montage is very careful to conserve flux in how it redistributes pixel content (in fact the method used for the slower reprojection algorithm does so using spherical angles even at the pixel scale). The problem arises in not being able to know whether the units of the data represent a flux or a flux density. If the former, she is correct but most of the datasets we have encountered in practice have been flux densities, where such scaling would be inappropriate.
However, if there are enough people that need rescaling like this, we would be happy to include it. It’s a trivial addition; we just try to avoid too many extraneous flags. Anyone desiring additional functionality can write to me directly.
By the way, whenever possible you should try to use mProjectPP, as it is 25-30 times faster than mProject. The former can handle any projection pair; the latter is more restrictive but for small areas can often be used even with some of the odder projections through a little mathematical trickery (creating a “distorted” tangent-plane projection that fits the image locally).