Better ways to make large image mosaics?

by Kelle on October 21, 2009

Given the useful responses to Jane’s question about spectral line analysis, here’s another query for the community. This one is about making large image mosaics and it comes from Adam Ginsburg, a grad student at the University of Colorado, Boulder.

I want to make a large-scale mosaic of the Galactic Plane covering 90-180 degrees x a few degrees. There are a few ways to do this:

  • IRAF’s imcombine with offset=wcs is a quick and dirty path to a mosaic. It is very fast, but very inaccurate over large scales and it sometimes does things flat-out wrong.
  • IDL’s hastrom and related astrolib utils generally work, but for large arrays (96000×4000 pixels) I’ve seen IDL just crash.
  • IRSA’s Montage works well, but is slow and consumes a lot of hard drive space in the process of making mosaics. The setup is also kind of a pain; you need to make a lot of subdirectories and .tbl files in the process.
  • SSC’s mopex must work quite well, but I just discovered it today and, like Montage, it has a learning curve.

So, I’m curious, what do others use? For example, what did GigaGalaxy Zoom use? If you’re trying to build a large near-IR finder chart using 2MASS, do you just go to montage? Does anyone have convenient wrapper scripts written up? Let’s hear it in the comments.

{ 18 comments… read them below or add one }

1 Tim October 21, 2009 at 11:09 am

SWarp?

http://astromatic.iap.fr/software/swarp

I haven’t used this for science per se, but to visualize what a field looked as I did analysis on several separate images that spanned a wider field.

Reply

2 Eilat October 21, 2009 at 12:28 pm

swarp! for sure. i use it all the time and it works great. not a very steep learning curve — short manual. If you are familiar with SExtractor then it is even easier to pick up. If your images have standard WCS header keywords then it is a piece of cake.

Reply

3 Jane Rigby October 21, 2009 at 1:55 pm

I used to sit on the IRSA users group. If folks have comments on how to improve Montage, I would like to pass them on to IRSA.

I have used the Montage web interface to effortlessly make big 2MASS and SDSS mosaics. Perfect for my findercharts, and a 3rd color for Warm Spitzer pretty RGB pictures. Took me 5 min as a complete newbie.

Like Adam, I’ve casually tried running Montage on my own computer to make mosaics from my own data, since it’s extremely powerful and produces wonderful results. Like Adam, I was also frustrated by the setup complexity and lack of a start-to-finish cookbook. Can others comment on their use of Montage, or compare it to other mosaickers?

Reply

4 Gus Muench October 21, 2009 at 3:44 pm

my experience is that swarp is designed for coadding and does not have the background matching capabilities of montage/mopex. montage seems to scale fine in big pipelines e.g. swire/glimpse.

what i’ve wanted is a python api to montage…

Reply

5 Thomas Robitaille October 21, 2009 at 3:54 pm

I use Montage all the time. I wrote a small python function that takes care of setting up the directories and cleaning up afterwards, so that all one needs to specify is a directory containing the input files to mosaic, the header to use (or let it choose the header automatically), and the output filename. Maybe I should make this available?

Reply

6 Jessica Lu August 22, 2013 at 3:25 pm

Tom, we can put a link to a git repository on the Python wiki page.

7 Adam Ginsburg October 22, 2009 at 1:45 am

Thomas, based on what I’ve seen of APLpy, you should definitely make your montage wrappers available.

I was completely unaware of SWarp – thanks guys!

re: Jane – The quick cutouts are fine, and the big, large-scale projects are also fine, but I have very often been in situations where I just want to stitch stuff together quickly. Also, “big” is a relative term… 1 square degree isn’t much when you’re looking to compare to galactic plane surveys. I’ve successfully produced ~6 square degree 2MASS fields with MONTAGE but it took a couple days per image on my really fast computer. Also, if you want your output image bigger than the inputs, I’ve found you have to hack around it by passing a blank image of the desired size as one of the images to co-add.

Reply

8 Adam Ginsburg October 23, 2009 at 11:40 am

A little follow-up….
swarp is actually more along the lines of what I’ve been hoping for: a single command-line program with sensible defaults but very extensible via configuration file. It also defaults to removing unnecessary temp files.

montage has some great features, but is split up into a lot of separate commands. I’ve particularly had problems (today!) with making a mosaic that matches the input header exactly (i.e. cropping), while in the past I’ve had difficulty but figured out how to do the opposite (make larger).

Are there more arguments in favor of montage over swarp, or is it just a matter of preference, perhaps based on side of the atlantic?

Sort of related – does either program work well with data cubes? Or are there any good data-cube-mosaicing tools? (STARLINK does OK on this for JCMT data, at least)

Reply

9 Son August 20, 2013 at 5:12 pm

Hi,

I need some help here. I’m to start a mosaic scene at school of about 640 cm length and 13 feet high. I’hd like to create a large scale of the scene on the computer. How do I do this?

10 John Good August 20, 2013 at 11:42 pm

This is in reply to the question below. Montage can certainly make mosaics this size or larger and if the data source is 2MASS, WISE, SDSS or DSS it includes the data retrieval tools as well. However, it is unclear exactly what data is desired, what resolution, etc. I can’t really comment without more detail.

11 Emanuela Pompei August 26, 2014 at 11:19 am

Well, swarp is just perfect. Montage has a nice web pages, but no one will help you if you run into a problem. They list what kind of error you might get, but no reply if you ask help to them.

12 Bruce Berriman November 6, 2009 at 9:19 am

John Good (Montage architect) and myself (Montage project manager) are posting this note to clarify some of the comments made on this page. Montage is actively supported, and we welcome suggestions for upgrades and improvements. Please, contact us with requests and suggestions at the Montage Help desk at

http://irsa.ipac.caltech.edu/cgi-bin/Helpdesk/nph-genTicketForm?projname=Montage&projmail=montage@ipac.caltech.edu

On-request Mosaics
There is an on-request mosaic service at http://hachi.ipac.caltech.edu:8080/montage

This service returns mosaics of 2MASS, SDSS and DSS images. In the case of SDSS it is the currently the only place you can get photometrically-scaled Sloan date (including from the project itself). You simply select the parameters of the mosaic you want on a web form, and the service will notify you when the mosaic is done. You can enter a number of requests and monitor progress on a web page. In Spring 2010, we plan to extend this service to new data sets, as well as to extend functionality to, e.g., deliver 3-color images. We welcome feedback from users on new functionality and which new data sets may be most useful. For many users, this is the simplest way of getting mosaics.

Montage vs. Swarp
Swarp and Montage are very different products and satisfy different use cases. Swarp is a fine product and is best suited for a fast, browse quality mosaic, or if you want to a co-add a collection of images (as was pointed out on this web page). Beware that the output is not strictly flux-conserving. Montage, by design, preserves the flux and astrometric fidelity of the input images. And it rectifies the background radiation in the input images to a common level, which is essential interpreting the mosaics in regions with extended emission.

Speed
Part of the price that Montage pays for a “science grade” mosaic is speed – it takes a lot of crunching to generate such mosaics, especially if projections such as Aitoff are involved. They are computed by a general algorithm that applies to all WCS projections. We have sped up Montage by more than an order of magnitude in cases where only tangent plane projections are involved. Montage avoids costly plane-to-sky-to-plane deprojection/projection calculations in these cases using a fast “plane-to-plane library,” a modification on the algorithm that MOPEX uses.

Usability
Montage is designed as a toolkit rather than a single executable. This design has many benefits:
– it allows end users to adapt the code for their own use cases; a number of Spitzer Legacy teams have, for instance, installed it into their image processing pipelines.
– it makes Montage scalable across large-scale parallel systems
– the many files it produces are necessary to do proper background matching and assure photometric and astrometric accuracy. The files are a very powerful QA tool and we recommend users examine them.

Now, there is an “executive” (mExec) which you can point at an
existing set of image files and which will do the end-to-end
processing for you. See

http://montage.ipac.caltech.edu/docs/mExec.html

It tries to minimize storage, deleting unneeded files as it goes. This is a good solution for users who may want occasional mosaics. We do recommend that regular users examine the intermediate files and understand what the processing does to the images.

There is an end-to-end cookbook tutorial at

http://montage.ipac.caltech.edu/docs/m101tutorial.html

If this does not satisfy your needs, please let us know what further tutorials and documentation will be valuable.

– John Good and Bruce Berriman (IPAC, Caltech)

Reply

13 Loren Anderson December 4, 2009 at 4:35 am

An important note on Montage is that it does not conserve flux when changing the image resolution. For example, if you want to rebin the image to a pixel size three times larger, the flux of all point sources will decrease by a factor of 9 (3^2). Multiplying by the ratio of the pixel areas, (new_pixel_size/old_pixel_size)^2, will get you back to the original flux. This can be done via the “-x” flag in Montage’s mProject. A “conserve flux” flag in Montage, as in MIDAS, would be nice though.

Reply

14 John Good December 6, 2009 at 11:31 am

In response to Loren Anderson’s comment: Montage is very careful to conserve flux in how it redistributes pixel content (in fact the method used for the slower reprojection algorithm does so using spherical angles even at the pixel scale). The problem arises in not being able to know whether the units of the data represent a flux or a flux density. If the former, she is correct but most of the datasets we have encountered in practice have been flux densities, where such scaling would be inappropriate.

However, if there are enough people that need rescaling like this, we would be happy to include it. It’s a trivial addition; we just try to avoid too many extraneous flags. Anyone desiring additional functionality can write to me directly.

By the way, whenever possible you should try to use mProjectPP, as it is 25-30 times faster than mProject. The former can handle any projection pair; the latter is more restrictive but for small areas can often be used even with some of the odder projections through a little mathematical trickery (creating a “distorted” tangent-plane projection that fits the image locally).

Reply

15 Adam Ginsburg November 29, 2010 at 12:24 pm

This post (and comments on it) are mentioned in http://arxiv.org/abs/1011.5294.

Reply

16 Bruce Berriman December 31, 2010 at 6:51 pm

As a result of the comments in this thread, and subsequent e-mails, we have re-organized the Montage web site (http://montage.ipac.caltech.edu) to assist newcomers by making tutorials and scripts more visible. And we have released a blog at http://montageblog.wordpress.com to report news more quickly than we can do with the web page. We will be blogging about how Montage is used in scientific analysis and generating new science products, showing images derived with it, posting tips for getting the most out of the toolkit, entertaining suggestions for new functionality, and discussing image mosaics in general . We welcome comments and guest blog posts from users.

Reply

17 Jane Rigby March 10, 2011 at 1:44 pm

I’m thrilled by the Montage group’s response to this thread. They could have been defensive (“Jane you dummy, there is too a tutorial!”) but instead they took comments to heart, as well as clarifying what they felt we were misunderstanding (fluxing). I’m also pleased to see that there’s a Montage blog with examples and published results.

Reply

18 Bruce Berriman July 31, 2012 at 12:03 pm

Tom Robitaille and I are interested in finding out how Python users are using the Montage image mosaic engine (as this may impact future directions for the package).

[Note that if you use APLpy, you may be making use of Montage, so
please read on!]

If you indeed use or have used Montage before, please take a couple of
minutes to email thomas.robitaille AT gmail.com or gbb AT ipac.caltech.edu and let us know:

– whether you use Montage via Python scripts, and if so whether you
use it via the python-montage package, or via APLpy’s make_rgb_cube
and north=True functionality (which requires Montage), or whether you
have developed your wrappers or just call the commands directly with
subprocess or os.system (or other ways I haven’t mentioned!)

– what kind of projects you use Montage for (broadly)

– whether you have developed any packages that use Montage or python-montage

– whether you use Montage, but not from Python, and if so why not?

Your replies, however short, would be very helpful!

Thanks!
Tom and Bruce

Reply

Leave a Comment

Previous post:

Next post: