<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>AstroBetter &#187; python</title>
	<atom:link href="http://www.astrobetter.com/category/python/feed/" rel="self" type="application/rss+xml" />
	<link>http://www.astrobetter.com</link>
	<description>Tips and Tricks for Professional Astronomers</description>
	<lastBuildDate>Fri, 03 Feb 2012 14:05:00 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.2.1</generator>
		<item>
		<title>The Astropy Project: A core Python package for Astronomy</title>
		<link>http://www.astrobetter.com/the-astropy-project/</link>
		<comments>http://www.astrobetter.com/the-astropy-project/#comments</comments>
		<pubDate>Mon, 30 Jan 2012 12:30:59 +0000</pubDate>
		<dc:creator>Tom</dc:creator>
				<category><![CDATA[programming]]></category>
		<category><![CDATA[python]]></category>
		<category><![CDATA[astropy]]></category>

		<guid isPermaLink="false">http://www.astrobetter.com/?p=5520</guid>
		<description><![CDATA[In June 2011, the announcement of a new Python package for Astronomy on the astropy mailing list prompted a long thread that started as a criticism of the proliferation of independently-developed Astronomy Python packages but quickly became the start of a common effort to develop a single core package for Astronomy. Many hundreds of emails, [...]]]></description>
			<content:encoded><![CDATA[<p></p><p><img src="http://www.astrobetter.com/wp-content/uploads/2009/11/python-logo-new2.png" alt="python-logo-new2" title="python-logo-new2" width="150" height="44" class="alignright size-full wp-image-1154" />In June 2011, the announcement of a new Python package for Astronomy on the <a href="http://mail.scipy.org/mailman/listinfo/astropy">astropy</a> mailing list prompted a long <a href="http://mail.scipy.org/pipermail/astropy/2011-June/001380.html">thread</a> that started as a criticism of the proliferation of <a href="http://www.astropython.org/resources">independently-developed Astronomy Python packages</a> but quickly became the start of a common effort to develop a single core package for Astronomy. Many hundreds of emails, and one workshop later, the project is well underway! Development is actively ongoing, and it is likely that we will see an initial public release in the summer, if not sooner!</p>
<p>The <a href="http://www.astropy.org">Astropy</a> project, as it is now called, brings together almost 90 developers from around the world. The project is coordinated by Perry Greenfield (STScI), <a href="http://www.physics.uci.edu/~etolleru/">Erik Tollerud</a> (UC Irvine), and <a href="http://www.mpia-hd.mpg.de/homes/robitaille/index.html">myself</a> (Tom Robitaille, MPIA). We have written a <a href="http://astropy.readthedocs.org/en/latest/development/vision.html">long-term vision</a> and have a code <a href="http://www.github.com/astropy/astropy">repository</a> on GitHub that already contains thousands of lines of code, including not only &#8216;framework&#8217; code (testing, configuration, etc.), but also functional astronomy submodules such as <tt>astropy.wcs</tt> and <tt>astropy.io.vo</tt> (ported over by Mike Droettboom, STScI, from <tt>pywcs</tt> and <tt>vo</tt>). PyFITS is almost ready for inclusion as <tt>astropy.io.fits</tt> (thanks to Erik Bray, STScI), and a number of other major components are being finalized.</p>
<h3>Contributing to Astropy: Affiliated Packages</h3>
<p>One of the key requirements of the Astropy project is that code that gets included in the package should be of a high standard, and with documentation and tests already included. This means that development of many sub-packages is not done directly in the main Astropy repository, but is done separately, and code is merged in once these criteria are met. To this end, we have converged on a system we refer to as ‘affiliated packages’. Essentially, these are self-contained packages that implement specific functionality (for example photometry, or spectrum-related functionality). Affiliated packages may be merged into the Astropy package once ready, although these can remain independent if they are too large, too specialized, have license conflicts with the core package, or if the author simply wants to maintain autonomy in their development cycle. An installation/configuration tool is in the works which should simplify installing affiliated packages, simplifying the process of gathering up the python astronomy tools you need.</p>
<p>If you are interested in developing an affiliated package for Astropy, first check <a href="https://github.com/astropy/astropy/wiki/Affiliated-packages">here</a> whether anyone is already working on a similar package. If so, then please avoid duplicating efforts, and contact the maintainer of the affiliated package to offer your help. Otherwise, read the README.rst file in the <a href="https://github.com/astropy/package-template">affiliated package template</a>, then add your package to the list on the wiki page.</p>
<h3>Other ways to contribute to Astropy</h3>
<p>If you are interested in improving existing components of Astropy, you do not need to create an affiliated package. In that case, you can just fork the Astropy repository, work on the code, then submit a <em>pull request</em> which we can then review, and iterate on, before merging. Instructions for doing this are included <a href="http://astropy.readthedocs.org/en/latest/development/workflow/development_workflow_basic.html">here</a>.</p>
<p>Once we start moving towards a first public release, we will ask for volunteers to try out the package, and read over and improve the documentation and write tutorials, so even if you don’t want to code, there are still ways in which you can help!</p>
<p>Finally, if you are interested in staying up to date with the development of Astropy, you can join the <a href="http://groups.google.com/group/astropy-dev">astropy-dev mailing list</a>; the <a href="http://mail.scipy.org/mailman/listinfo/astropy">astropy</a> mailing list will remain a general purpose Astronomy Python mailing list.</p>
]]></content:encoded>
			<wfw:commentRss>http://www.astrobetter.com/the-astropy-project/feed/</wfw:commentRss>
		<slash:comments>6</slash:comments>
		</item>
		<item>
		<title>[Link] Statistics with Python</title>
		<link>http://www.astrobetter.com/link-statistics-with-python/</link>
		<comments>http://www.astrobetter.com/link-statistics-with-python/#comments</comments>
		<pubDate>Fri, 30 Sep 2011 19:10:55 +0000</pubDate>
		<dc:creator>Jessica</dc:creator>
				<category><![CDATA[programming]]></category>
		<category><![CDATA[python]]></category>
		<category><![CDATA[Uncategorized]]></category>

		<guid isPermaLink="false">http://www.astrobetter.com/?p=5190</guid>
		<description><![CDATA[Astronomers rely on statistical analysis all the time. The Python programming language has some excellent add-on packages for statistics. But sometimes the documentation isn&#8217;t as helpful in &#8220;real-world&#8221; scenarios. Prasanth Nair has posted a very nice tutorial on Simple Statistics with Python over at his blog. I highly recommend it for new users of python [...]]]></description>
			<content:encoded><![CDATA[<p></p><p>Astronomers rely on statistical analysis all the time. The Python programming language has some excellent add-on packages for statistics. But sometimes the documentation isn&#8217;t as helpful in &#8220;real-world&#8221; scenarios. Prasanth Nair has posted a very nice tutorial on Simple Statistics with Python over at his blog. I highly recommend it for new users of python and anyone learning how to use statistics in their astronomical research. While you are there, poke around a bit as Prasanth&#8217;s blog has several other useful astro-python related tutorials including some on ephemeris and angle calculations.</p>
<p><a href="http://oneau.wordpress.com/2011/02/28/simple-statistics-with-scipy/">Simple Statistics with Python</a> | <a href="http://oneau.wordpress.com/">Comfort at 1 AU</a> Blog</p>
<p>&nbsp;</p>
<p>&nbsp;</p>
]]></content:encoded>
			<wfw:commentRss>http://www.astrobetter.com/link-statistics-with-python/feed/</wfw:commentRss>
		<slash:comments>2</slash:comments>
		</item>
		<item>
		<title>Python 2 to 3 transition</title>
		<link>http://www.astrobetter.com/python-2-to-3-transition/</link>
		<comments>http://www.astrobetter.com/python-2-to-3-transition/#comments</comments>
		<pubDate>Wed, 31 Aug 2011 14:39:41 +0000</pubDate>
		<dc:creator>Jessica</dc:creator>
				<category><![CDATA[programming]]></category>
		<category><![CDATA[python]]></category>
		<category><![CDATA[Uncategorized]]></category>

		<guid isPermaLink="false">http://www.astrobetter.com/?p=5129</guid>
		<description><![CDATA[You can hardly be an astronomer today without knowing how to write code. As with programmers in all disciplines, we are at the mercy of the language we code with. Computer science is still a rapidly evolving field and new programming languages come along, existing programming languages get updated, and your favorite packages/routines might get [...]]]></description>
			<content:encoded><![CDATA[<p></p><p>You can hardly be an astronomer today without knowing how to write code. As with programmers in all disciplines, we are at the mercy of the language we code with. Computer science is still a rapidly evolving field and new programming languages come along, existing programming languages get updated, and your favorite packages/routines might get deprecated or de-supported all together. Whether you stick to your old favorites or jump-ship to the latest and greatest seems to be a matter of personal taste.</p>
<p>The python programming language is still fairly new to astronomers; but it has many advantages over C/Fortran/IDL (and some disadvantages, see here for our wiki page on <a title="IDL vs. Python" href="http://www.astrobetter.com/wiki/tiki-index.php?page=idl_vs_python">Python v. IDL</a>). In many scientific research fields, python has been adopted very rapidly. However, the python programming language has undergone a major overhaul going from 2.* versions to 3.* versions. The most common scientific python packages still use 2.* versions; but, rumblings have begun to migrate them to 3.* versions in the near future. I expect it will still be a a year or two before 3.* support in numpy, scipy, and matplotlib python packages are fully supported and de-bugged and I would imaging that 2.* support will continue for the next 5-10 years at least. But I still use codes written over 20 years ago, so long-range planning isn&#8217;t completely crazy.</p>
<p>For your early adopters out there, here is a useful <a href="http://www.astrobetter.com/wp-content/uploads/2011/08/python2python3.pdf" target="_blank">quick-switch guide</a> for python 2 to python 3 conversions. I should note that python 2.6 and 2.7 have transitional support for features found in python 3.*; but there are still some differences. Since python 2.7 comes installed with OS X Lion, if you get a new machine or upgrade, then consider tinkering with the <em>new</em> python features. And take a look at which of your coding habits will break things in the future. For instance:</p>
<pre>print 'I found %10d bugs' % bugCnt</pre>
<p>doesn&#8217;t work in python 3 and instead should be replaced with</p>
<pre>print('I found {:10d} bugs'.format(bugCnt))</pre>
<p>Here are a couple of other relevant links.</p>
<ul>
<li><a href="http://wiki.python.org/moin/Python2orPython3" target="_blank">Python 2 or Python 3?</a></li>
<li><a href="http://docs.python.org/library/2to3.html" target="_blank">Automated Python 2 to 3 code translation</a></li>
<li><a href="http://readthedocs.org/docs/python/en/latest/howto/pyporting.html" target="_blank">Porting Python 2 Code to Python 3</a></li>
</ul>
]]></content:encoded>
			<wfw:commentRss>http://www.astrobetter.com/python-2-to-3-transition/feed/</wfw:commentRss>
		<slash:comments>6</slash:comments>
		</item>
		<item>
		<title>Visualizing Simulated Data</title>
		<link>http://www.astrobetter.com/visualizing-simulated-data/</link>
		<comments>http://www.astrobetter.com/visualizing-simulated-data/#comments</comments>
		<pubDate>Thu, 09 Jun 2011 12:00:47 +0000</pubDate>
		<dc:creator>Jessica</dc:creator>
				<category><![CDATA[analysis]]></category>
		<category><![CDATA[python]]></category>
		<category><![CDATA[AMR]]></category>
		<category><![CDATA[yt]]></category>

		<guid isPermaLink="false">http://www.astrobetter.com/?p=4667</guid>
		<description><![CDATA[More and more astronomers are writing, running, and using massive computer simulations. The complexity of these simulations (often 3D space + time) means that visualization can be challenging. Over at Astropython, they have posted about the - yt - toolkit that allows you to analyze and visualize the results of adaptive mesh refinement simulation codes (AMR). AMR [...]]]></description>
			<content:encoded><![CDATA[<p></p><p><a href="http://www.astrobetter.com/wp-content/uploads/2011/05/s_projection_z_density.png"><img class="alignright size-medium wp-image-4668" title="Density Slice Produced by yt" src="http://www.astrobetter.com/wp-content/uploads/2011/05/s_projection_z_density-300x240.png" alt="" width="300" height="240" /></a>More and more astronomers are writing, running, and using massive computer simulations. The complexity of these simulations (often 3D space + time) means that visualization can be challenging. Over at <a href="http://www.astropython.org/">Astropython</a>, they have <a href="http://www.astropython.org/resource/2011/4/yt">posted</a> about the - <strong><a href="http://yt.enzotools.org/">yt</a></strong> - toolkit that allows you to analyze and visualize the results of <a href="http://astro.berkeley.edu/~cmckee/bafd/amr.html">adaptive mesh refinement simulation codes (AMR)</a>. AMR codes are used to simulate the formation of cosmological large scale structure, the collapse of gas clouds to form stars, fluid dynamics involving shocks, jets, etc. The <strong>yt</strong> toolkit is written in python and works with several AMR codes including <a href="http://lca.ucsd.edu/portal/software/enzo">Enzo</a>, Orion, <a href="http://flash.uchicago.edu/site/flashcode/">FLASH</a>, and preliminary support for <a href="http://irfu.cea.fr/Phocea/Vie_des_labos/Ast/ast_sstechnique.php?id_ast=904">RAMSES</a>, ART, <a href="https://seesar.lbl.gov/ANAG/chombo/" class="broken_link">Chombo</a>, <a href="https://ccse.lbl.gov/Research/CASTRO/index.html">CASTRO</a> and <a href="http://www.nersc.gov/research-and-development/benchmarking-and-workload-characterization/nersc-6-benchmarks/maestro/">MAESTRO</a>. Data from one of these AMR codes can be sliced, diced, profiled, binned, and volume-rendered.</p>
<p><strong>yt</strong> is described in <a href="http://adsabs.harvard.edu/abs/2011ApJS..192....9T">Turk et al. 2010</a> and can be <a href="http://yt.enzotools.org/">downloaded for free</a>. The documentation includes a nice <a href="http://yt.enzotools.org/doc/welcome_to_yt.html">summary</a> and <a href="http://yt.enzotools.org/doc/orientation.html">orientation</a>. I have updated our <a href="http://www.astrobetter.com/wiki/tiki-index.php?page=python">Python wiki page</a> to provide links.</p>
<p>Have you used <strong>yt</strong> and what was your experience like? What are the similar visualization packages for <a href="http://en.wikipedia.org/wiki/Smoothed-particle_hydrodynamics">smooth-particle hydrodynamics (SPH)</a> codes?</p>
<p><strong><span style="color: #c0c0c0;">Wiki: </span><a href="http://www.astrobetter.com/wiki/tiki-index.php?page=python">Python</a>, <a href="http://www.astrobetter.com/wiki/tiki-index.php?page=Simulations">Simulations</a></strong></p>
]]></content:encoded>
			<wfw:commentRss>http://www.astrobetter.com/visualizing-simulated-data/feed/</wfw:commentRss>
		<slash:comments>8</slash:comments>
		</item>
		<item>
		<title>Python Tutorial for Astronomers</title>
		<link>http://www.astrobetter.com/python4astronomers/</link>
		<comments>http://www.astrobetter.com/python4astronomers/#comments</comments>
		<pubDate>Wed, 25 May 2011 11:00:13 +0000</pubDate>
		<dc:creator>Tom</dc:creator>
				<category><![CDATA[python]]></category>
		<category><![CDATA[tutorial]]></category>

		<guid isPermaLink="false">http://www.astrobetter.com/?p=4765</guid>
		<description><![CDATA[On Monday, a group of CfA astronomers (Tom Aldcroft, Brian Refsdal, Gus Muench, and myself) announced the availability of a web tutorial aimed at teaching Python to astronomers through a series of interactive workshops: http://python4astronomers.github.com/ Practical Python for Astronomers is a series of hands-on workshops to explore the Python language and the analysis tools it [...]]]></description>
			<content:encoded><![CDATA[<p></p><p>On Monday, a group of CfA astronomers (Tom Aldcroft, Brian Refsdal, Gus Muench, and myself) announced the availability of a web tutorial aimed at teaching Python to astronomers through a series of interactive workshops:</p>
<p>  <a href="http://python4astronomers.github.com/">http://python4astronomers.github.com/</a></p>
<p>Practical Python for Astronomers is a series of hands-on workshops to explore the Python language and the analysis tools it provides. The emphasis is on using Python to solve real-world problems that astronomers are likely to encounter in research. Some features:</p>
<ul>
<li>Workshops immediately use plotting, analysis, and file reading tools.
<li>Along the way elements of the Python language are introduced.
<li>Workshops are interactive using examples run by participants on their laptops.
<li>Comprehensive instructions a given for installing a full Python environment.
</ul>
<p>There are two goals. First is to provide tutorials suitable for self-study by those wishing to learn Python for astronomy. The greater goal is for those knowledgable in Python to teach the workshop series at their local institutions, adapting the content as desired. To that end we have developed the content in Sphinx RestructuredText and hosted the source on github at <a href="https://github.com/python4astronomers/">https://github.com/python4astronomers/</a>. Anyone interested can clone the repository or download a tarball and make modifications needed to present the material locally. We would also welcome comments, fixes, or suggestions for improvement. This can be done as a Github issue or pull request, or by sending email to <a href="mailto:aldcroft@head.cfa.harvard.edu">Tom Aldcroft</a>.</p>
<p>The workshop material here was presented in the Spring of 2011 at the Harvard / Smithsonian Center for Astrophysics. A range of about 25 to 50 people participated in the different workshops, which were 1.5 hours in duration. One key accomplishment was installing a working Python with NumPy, SciPy, and IPython on over 50 laptops (MacOS, linux, and Windows) during a single session.</p>
<p>Feedback and suggestions are welcome!</p>
]]></content:encoded>
			<wfw:commentRss>http://www.astrobetter.com/python4astronomers/feed/</wfw:commentRss>
		<slash:comments>6</slash:comments>
		</item>
		<item>
		<title>PiCloud: Advantages of running python in the cloud</title>
		<link>http://www.astrobetter.com/picloud-advantages-of-running-python-in-the-cloud/</link>
		<comments>http://www.astrobetter.com/picloud-advantages-of-running-python-in-the-cloud/#comments</comments>
		<pubDate>Tue, 28 Dec 2010 09:54:43 +0000</pubDate>
		<dc:creator>Eli</dc:creator>
				<category><![CDATA[python]]></category>
		<category><![CDATA[analysis]]></category>
		<category><![CDATA[cloud computing]]></category>

		<guid isPermaLink="false">http://www.astrobetter.com/?p=3696</guid>
		<description><![CDATA[When we go to conferences, observing runs, collaborations and other travels the first thing that comes to mind for packing is our laptop (or iPad). The laptop is steadfast, resolute and is always there when we need to work or relax. But there&#8217;s one slight problem; when we need to do work on the run, [...]]]></description>
			<content:encoded><![CDATA[<p></p><p><a href="http://www.picloud.com/"><img class="alignright size-full wp-image-3936" title="picloud" src="http://www.astrobetter.com/wp-content/uploads/2010/12/picloud.png" alt="" width="296" height="58" /></a>When we go to conferences, observing runs, collaborations and other travels the first thing that comes to mind for packing is our laptop (or <a href="http://www.apple.com/ipad/" target="_blank">iPad</a>). The laptop is steadfast, resolute and is always there when we need to work or relax. But there&#8217;s one slight problem; when we need to do work on the run, which can computationally intensive, our laptop can be limiting.</p>
<p>Sometimes, even our desktop computers don&#8217;t have enough kick to get our computing problems done. If you&#8217;re a Python user, then you&#8217;re in luck. There is a new service called <a href="http://www.picloud.com/" target="_blank">PiCloud</a> that allows you to send algorithms and data on the fly for heavy computations. It works in a near seamless fashion with our native Python code and what could take hours of computation time will be done in a few minutes or less.<br />
<div class="toggle"><br />
Here&#8217;s an example of a simple and computationally intensive algorithm.</p>

<div class="wp_syntax"><div class="code"><pre class="python" style="font-family:monospace;"><span style="color: #ff7700;font-weight:bold;">def</span> fib_sequence<span style="color: black;">&#40;</span>nth<span style="color: black;">&#41;</span>:
	x = <span style="color: black;">&#91;</span><span style="color: #ff4500;">0</span>,<span style="color: #ff4500;">1</span><span style="color: black;">&#93;</span>
	<span style="color: #ff7700;font-weight:bold;">if</span> nth == <span style="color: #ff4500;">1</span>:
		<span style="color: #ff7700;font-weight:bold;">return</span> <span style="color: #ff4500;">1</span>
&nbsp;
	<span style="color: #ff7700;font-weight:bold;">for</span> i <span style="color: #ff7700;font-weight:bold;">in</span> <span style="color: #008000;">xrange</span><span style="color: black;">&#40;</span>nth-<span style="color: #ff4500;">1</span><span style="color: black;">&#41;</span>:
		x.<span style="color: black;">append</span><span style="color: black;">&#40;</span>x<span style="color: black;">&#91;</span>-<span style="color: #ff4500;">1</span><span style="color: black;">&#93;</span> + x<span style="color: black;">&#91;</span>-<span style="color: #ff4500;">2</span><span style="color: black;">&#93;</span><span style="color: black;">&#41;</span>
		x = <span style="color: black;">&#91;</span> x<span style="color: black;">&#91;</span>-<span style="color: #ff4500;">2</span><span style="color: black;">&#93;</span>, x<span style="color: black;">&#91;</span>-<span style="color: #ff4500;">1</span><span style="color: black;">&#93;</span><span style="color: black;">&#93;</span>
	<span style="color: #ff7700;font-weight:bold;">return</span> x<span style="color: black;">&#91;</span>-<span style="color: #ff4500;">1</span><span style="color: black;">&#93;</span></pre></div></div>

<p>The algorithm could be more efficient, but I&#8217;m leaving it as is to illustrate how fast PiCloud can be. Now let&#8217;s setup the PiCloud module and compare the speed to running it on my laptop (3.06 GHz Intel Core 2 Duo with 4 GB of RAM).</p>

<div class="wp_syntax"><div class="code"><pre class="python" style="font-family:monospace;"><span style="color: #808080; font-style: italic;">#Laptop time</span>
local = fib_sequence<span style="color: black;">&#40;</span><span style="color: #ff4500;">10000000</span><span style="color: black;">&#41;</span>
&nbsp;
<span style="color: #808080; font-style: italic;">#PiCloud time</span>
<span style="color: #ff7700;font-weight:bold;">import</span> cloud
<span style="color: #808080; font-style: italic;"># This has been anonymized, as the API keys are unique to</span>
<span style="color: #808080; font-style: italic;"># each personal account</span>
cloud.<span style="color: black;">setkey</span><span style="color: black;">&#40;</span>api_key=something1, api_secretkey=<span style="color: #483d8b;">'something2'</span><span style="color: black;">&#41;</span>
jid = cloud.<span style="color: black;">call</span><span style="color: black;">&#40;</span>fib_sequence, <span style="color: #ff4500;">10000000</span>, _high_cpu=<span style="color: #008000;">True</span><span style="color: black;">&#41;</span>
remote = cloud.<span style="color: black;">result</span><span style="color: black;">&#40;</span>jid<span style="color: black;">&#41;</span></pre></div></div>

<p>The time for my laptop to crunch and obtain the millionth element in the fibonacci sequence is 70 seconds, whereas PiCloud computed it in 48 seconds (including sending and receiving). The real power of PiCloud though is <a href="http://www.astrobetter.com/parallel-processing-in-python/">parallelism</a>, which we cannot readily use with the fibonacci sequence. So let&#8217;s assume that we need to know the millionth fibonacci number 10 times.</p>

<div class="wp_syntax"><div class="code"><pre class="python" style="font-family:monospace;">num_parallel = <span style="color: #ff4500;">10</span>
jids = cloud.<span style="color: #008000;">map</span><span style="color: black;">&#40;</span>fib_sequence, <span style="color: black;">&#91;</span><span style="color: #ff4500;">10000000</span><span style="color: black;">&#93;</span><span style="color: #66cc66;">*</span>num_parallel, _high_cpu=<span style="color: #008000;">True</span><span style="color: black;">&#41;</span>
remote = cloud.<span style="color: black;">result</span><span style="color: black;">&#40;</span>jids<span style="color: black;">&#41;</span>
<span style="color: #ff7700;font-weight:bold;">return</span> remote</pre></div></div>

<p>The parallel processing task took 60 seconds in total, which is roughly 12 times faster than my laptop. These examples above are very simplistic and ideal ways of utilizing PiCloud to highlight its usefulness, particularly with parallel processing.</p>
<p>PiCloud is not just limited to Python native modules. You can install <a href="http://numpy.scipy.org/" target="_blank">Numpy</a>, <a href="http://www.scipy.org/" target="_blank">Scipy</a> or other modules with ease on your PiCloud account. In the next post, I&#8217;ll give some examples of installing the modules and how much quicker such modules are. Note, this post is a simplified version of one of several examples on PiCloud&#8217;s <a href="http://docs.picloud.com/" target="_blank">documentation</a>.</p>
<p>PiCloud is a paid service and its <a href="http://www.picloud.com/pricing/" target="_blank">pricing</a> seems to be reasonable. If you&#8217;re keen on trying out <a href="http://www.picloud.com/" target="_blank">PiCloud</a> you will be happy to know that there&#8217;s free computing time available to test it. So why not give it a try and let us know what you think in the comments! The days of having the combination of a laptop and desktop may be irrelevant and all we need is a lightweight laptop to access services like PiCloud.</p>
</div>]]></content:encoded>
			<wfw:commentRss>http://www.astrobetter.com/picloud-advantages-of-running-python-in-the-cloud/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Making RGB images from FITS files with python/matplotlib.</title>
		<link>http://www.astrobetter.com/making-rgb-images-from-fits-files-with-pythonmatplotlib/</link>
		<comments>http://www.astrobetter.com/making-rgb-images-from-fits-files-with-pythonmatplotlib/#comments</comments>
		<pubDate>Fri, 22 Oct 2010 16:24:54 +0000</pubDate>
		<dc:creator>Jessica</dc:creator>
				<category><![CDATA[figures]]></category>
		<category><![CDATA[python]]></category>
		<category><![CDATA[analysis]]></category>

		<guid isPermaLink="false">http://www.astrobetter.com/?p=3221</guid>
		<description><![CDATA[Converting astronomical data taken in multiple filters into representative-color RGB images often provides one of the most visually appealing (and informative) views of a target. This can be done in ds9 very easily; however, if you want a little more control, then python/matplotlib can be used in a very similar fashion. I also note that [...]]]></description>
			<content:encoded><![CDATA[<p></p><p><a href="http://www.astrobetter.com/wp-content/uploads/2010/10/w51a_3color_ukidss.png"><img class="alignright size-medium wp-image-3353" title="w51a_3color_ukidss" src="http://www.astrobetter.com/wp-content/uploads/2010/10/w51a_3color_ukidss-300x300.png" alt="" width="300" height="300" /></a>Converting astronomical data taken in multiple filters into representative-color RGB images often provides one of the most visually appealing (and informative) views of a target. This can be done in ds9 very easily; however, if you want a little more control, then python/matplotlib can be used in a very similar fashion. I also note that <a href="http://aplpy.sourceforge.net/documentation/index.html">APLpy</a> has excellent capabilities for images with proper World Coordinate System headers already (see the <a href="http://aplpy.sourceforge.net/documentation/howto_rgb.html">APLpy RGB tutorial</a>). APLpy is far easier to use, but if you don&#8217;t have that choice, then you can use matplotlib as I am going to show below.</p>
<p>First, lets assume we are starting with three images, say J.fits, H.fits, and K.fits and that these images all have the same plate scale, the same angle, and the same position on the sky but lacks WCS information (as is often the case for ground-based OIR observations). Load up the images into numpy arrays using the pyfits module.</p>

<div class="wp_syntax"><div class="code"><pre class="python" style="font-family:monospace;"><span style="color: #ff7700;font-weight:bold;">import</span> pyfits
<span style="color: #ff7700;font-weight:bold;">import</span> numpy <span style="color: #ff7700;font-weight:bold;">as</span> np
<span style="color: #ff7700;font-weight:bold;">import</span> pylab <span style="color: #ff7700;font-weight:bold;">as</span> py
<span style="color: #ff7700;font-weight:bold;">import</span> img_scale
&nbsp;
j_img = pyfits.<span style="color: black;">getdata</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">'J.fits'</span><span style="color: black;">&#41;</span>
h_img = pyfits.<span style="color: black;">getdata</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">'H.fits'</span><span style="color: black;">&#41;</span>
k_img = pyfits.<span style="color: black;">getdata</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">'K.fits'</span><span style="color: black;">&#41;</span></pre></div></div>

<p><div class="toggle"><br />
You can use pylab.imshow on a NxMx3 array; however, each layer of the image must first be mapped onto the luminosity scale. This is done with a module called img_scale.py that I borrowed from Min-Su Shin at U. of Michigan who very generously freely distributed his code (thanks!). You can get img_scale.py from this <a href="http://www.astrobetter.com/wiki/tiki-index.php?page=RGB+Images+with+matplotlib">AstroBetter wiki page</a>. Using this module you can linear, log, sqrt, or asinh stretch each color channel independently and set minimum or maximum flux values.</p>

<div class="wp_syntax"><div class="code"><pre class="python" style="font-family:monospace;">img = np.<span style="color: black;">zeros</span><span style="color: black;">&#40;</span><span style="color: black;">&#40;</span>j_img.<span style="color: black;">shape</span><span style="color: black;">&#91;</span><span style="color: #ff4500;">0</span><span style="color: black;">&#93;</span>, j_img.<span style="color: black;">shape</span><span style="color: black;">&#91;</span><span style="color: #ff4500;">1</span><span style="color: black;">&#93;</span>, <span style="color: #ff4500;">3</span><span style="color: black;">&#41;</span>, dtype=<span style="color: #008000;">float</span><span style="color: black;">&#41;</span>
img<span style="color: black;">&#91;</span>:,:,<span style="color: #ff4500;">0</span><span style="color: black;">&#93;</span> = img_scale.<span style="color: black;">sqrt</span><span style="color: black;">&#40;</span>k_img, scale_min=<span style="color: #ff4500;">0</span>, scale_max=<span style="color: #ff4500;">10000</span><span style="color: black;">&#41;</span>
img<span style="color: black;">&#91;</span>:,:,<span style="color: #ff4500;">1</span><span style="color: black;">&#93;</span> = img_scale.<span style="color: black;">sqrt</span><span style="color: black;">&#40;</span>h_img, scale_min=<span style="color: #ff4500;">0</span>, scale_max=<span style="color: #ff4500;">10000</span><span style="color: black;">&#41;</span>
img<span style="color: black;">&#91;</span>:,:,<span style="color: #ff4500;">2</span><span style="color: black;">&#93;</span> = img_scale.<span style="color: black;">sqrt</span><span style="color: black;">&#40;</span>j_img, scale_min=<span style="color: #ff4500;">0</span>, scale_max=<span style="color: #ff4500;">10000</span><span style="color: black;">&#41;</span></pre></div></div>

<p>Finally, we can display the image:</p>

<div class="wp_syntax"><div class="code"><pre class="python" style="font-family:monospace;">py.<span style="color: black;">clf</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
py.<span style="color: black;">imshow</span><span style="color: black;">&#40;</span>img, aspect=<span style="color: #483d8b;">'equal'</span><span style="color: black;">&#41;</span>
py.<span style="color: black;">title</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">'Blue = J, Green = H, Red = K'</span><span style="color: black;">&#41;</span>
py.<span style="color: black;">savefig</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">'my_rgb_image.png'</span><span style="color: black;">&#41;</span></pre></div></div>

<p>If your input images don&#8217;t have the same rotation angles, plate scales, or positions, then IRAF&#8217;s handy <em>imlintran </em>can be used to rescale, rotate, and translate. This can still be done in python via pyraf. Here is an example of calling imlintran from within python for a J.fits image that I want to rotate, shift, and rescale to match H and K images. I will assume the H and K images have a position angle of zero.</p>

<div class="wp_syntax"><div class="code"><pre class="python" style="font-family:monospace;"><span style="color: #ff7700;font-weight:bold;">from</span> pyraf <span style="color: #ff7700;font-weight:bold;">import</span> iraf <span style="color: #ff7700;font-weight:bold;">as</span> ir
&nbsp;
h_img = pyfits.<span style="color: black;">getdata</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">'H.fits'</span><span style="color: black;">&#41;</span>
k_img = pyfits.<span style="color: black;">getdata</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">'K.fits'</span><span style="color: black;">&#41;</span>
scaleK = <span style="color: #ff4500;">0.1</span>   <span style="color: #808080; font-style: italic;"># arcsec per pixel</span>
scaleJ = <span style="color: #ff4500;">0.15</span> <span style="color: #808080; font-style: italic;"># arcsec per pixel)</span>
paJ = <span style="color: #ff4500;">30.0</span>  <span style="color: #808080; font-style: italic;"># degrees</span>
&nbsp;
ir.<span style="color: black;">unlearn</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">'imlintran'</span><span style="color: black;">&#41;</span>
&nbsp;
ir.<span style="color: black;">imlintran</span>.<span style="color: black;">boundary</span> = <span style="color: #483d8b;">'constant'</span>
ir.<span style="color: black;">imlintran</span>.<span style="color: black;">constant</span> = <span style="color: #ff4500;">0</span>
ir.<span style="color: black;">imlintran</span>.<span style="color: black;">interpolant</span> = <span style="color: #483d8b;">'spline3'</span>
ir.<span style="color: black;">imlintran</span>.<span style="color: black;">fluxconserve</span> = <span style="color: #483d8b;">'yes'</span>
&nbsp;
<span style="color: #808080; font-style: italic;"># the input pixel position of a reference source</span>
ir.<span style="color: black;">imlintran</span>.<span style="color: black;">xin</span> = <span style="color: #ff4500;">0</span>
ir.<span style="color: black;">imlintran</span>.<span style="color: black;">yin</span> = <span style="color: #ff4500;">0</span>
&nbsp;
<span style="color: #808080; font-style: italic;"># the output pixel position of a reference source</span>
ir.<span style="color: black;">imlintran</span>.<span style="color: black;">xout</span> = <span style="color: #ff4500;">10</span>
ir.<span style="color: black;">imlintran</span>.<span style="color: black;">yout</span> = <span style="color: #ff4500;">10</span>
&nbsp;
<span style="color: #808080; font-style: italic;"># Set the output size of the final image</span>
ir.<span style="color: black;">imlintran</span>.<span style="color: black;">ncols</span> = k_img.<span style="color: black;">shape</span><span style="color: black;">&#91;</span><span style="color: #ff4500;">1</span><span style="color: black;">&#93;</span>
ir.<span style="color: black;">imlintran</span>.<span style="color: black;">nlines</span> = k_img.<span style="color: black;">shape</span><span style="color: black;">&#91;</span><span style="color: #ff4500;">0</span><span style="color: black;">&#93;</span>
&nbsp;
ir.<span style="color: black;">imlintran</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">'j.fits'</span>, <span style="color: #483d8b;">'j_rot_scale.fits'</span>, angleJ, angleJ, scaleK/scaleJ, scaleK/scaleJ<span style="color: black;">&#41;</span>
&nbsp;
j_img = pyfits.<span style="color: black;">getdata</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">'j_rot_scale.fits'</span><span style="color: black;">&#41;</span></pre></div></div>

<p>Alternatively, you can use scipy.ndimage.interpolate to shift images.</p>

<div class="wp_syntax"><div class="code"><pre class="python" style="font-family:monospace;"><span style="color: #ff7700;font-weight:bold;">from</span> scipy.<span style="color: black;">ndimage</span> <span style="color: #ff7700;font-weight:bold;">import</span> interpolation <span style="color: #ff7700;font-weight:bold;">as</span> interp
&nbsp;
old_j = pyfits.<span style="color: black;">getdata</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">'j.fits'</span><span style="color: black;">&#41;</span>
new_j = interp.<span style="color: black;">shift</span><span style="color: black;">&#40;</span>old_j, <span style="color: black;">&#91;</span>shiftY, shiftX<span style="color: black;">&#93;</span><span style="color: black;">&#41;</span></pre></div></div>

<p>The image attached in this post was plotted using imshow as described above and is a JHK color composite from the UKIDSS galactic plane survey.</p>
</div>]]></content:encoded>
			<wfw:commentRss>http://www.astrobetter.com/making-rgb-images-from-fits-files-with-pythonmatplotlib/feed/</wfw:commentRss>
		<slash:comments>3</slash:comments>
		</item>
		<item>
		<title>Some drawbacks to Python: Python Provocation &#124; The e-Astronomer</title>
		<link>http://www.astrobetter.com/some-drawbacks-to-python-python-provocation-the-e-astronomer/</link>
		<comments>http://www.astrobetter.com/some-drawbacks-to-python-python-provocation-the-e-astronomer/#comments</comments>
		<pubDate>Sat, 02 Oct 2010 13:57:32 +0000</pubDate>
		<dc:creator>Kelle</dc:creator>
				<category><![CDATA[Links]]></category>
		<category><![CDATA[python]]></category>

		<guid isPermaLink="false">http://www.astrobetter.com/?p=3137</guid>
		<description><![CDATA[Python Provocation &#124; The e-Astronomer Python occupies a strange territory between the easy peasy world of “download this app and start clicking” and the stern world of “if you don’t know what a makefile is, you’d better look somewhere else mate”. At first I thought this was precisely its strength : grown up stuff for [...]]]></description>
			<content:encoded><![CDATA[<p></p><p><a href="http://andyxl.wordpress.com/2010/10/02/python-provocation/">Python Provocation</a> | <a href="http://andyxl.wordpress.com/">The e-Astronomer</a></p>
<blockquote><p>Python occupies a strange territory between the easy peasy world of “download this app and start clicking” and the stern world of “if you don’t know what a makefile is, you’d better look somewhere else mate”. At first I thought this was precisely its strength : grown up stuff for busy people. But now I ain’t so sure. Neither use nor ornament, as EG used to say.</p></blockquote>
<p>I&#8217;m neither here nor there about Python yet&#8230;all I know is that the tide is slowly but surely changing in its direction. I can also report that my method of having my students learn Python rather than IDL is working well. Without much effort, I&#8217;m learning how to use it (they bring me printouts of their code), I don&#8217;t/can&#8217;t do debugging for them and so they work it out as a team, and my IDL library is gradually being converted. And I feel better about them having Python on their CVs rather than IDL.</p>
<p>Other than the obvious problem with the packages strewn across everywhere and there not being a centralized astro library yet, what do you think the major drawbacks of Python are?</p>
]]></content:encoded>
			<wfw:commentRss>http://www.astrobetter.com/some-drawbacks-to-python-python-provocation-the-e-astronomer/feed/</wfw:commentRss>
		<slash:comments>11</slash:comments>
		</item>
		<item>
		<title>How to easily create, read, write and manipulate tables and databases in Python with ATpy and asciitable</title>
		<link>http://www.astrobetter.com/atpy-and-asciitable/</link>
		<comments>http://www.astrobetter.com/atpy-and-asciitable/#comments</comments>
		<pubDate>Mon, 06 Sep 2010 11:00:30 +0000</pubDate>
		<dc:creator>Tom</dc:creator>
				<category><![CDATA[python]]></category>
		<category><![CDATA[asciitable]]></category>
		<category><![CDATA[atpy]]></category>
		<category><![CDATA[databases]]></category>
		<category><![CDATA[tables]]></category>

		<guid isPermaLink="false">http://www.astrobetter.com/?p=2960</guid>
		<description><![CDATA[With the release of ATpy 0.9.4 and asciitable 0.2.4, it&#8217;s about time we told our readers how incredibly easy it is to create, read, write, and manipulate tables and databases of data in Python! &#160; Most, if not all of us have to manage tabular data at one point or another, and it is safe [...]]]></description>
			<content:encoded><![CDATA[<p></p><div><a href="http://www.astrobetter.com/wp-content/uploads/2010/09/atpy_logo.png"><img class="alignright size-full wp-image-2961" title="atpy_logo" src="http://www.astrobetter.com/wp-content/uploads/2010/09/atpy_logo.png" alt="" width="210" height="96" /></a>With the release of <a title="ATpy 0.9.4" href="http://atpy.sourceforge.net/" target="_blank">ATpy 0.9.4</a> and <a title="asciitable 0.2.4" href="http://cxc.harvard.edu/contrib/asciitable/" target="_blank">asciitable 0.2.4</a>, it&#8217;s about time we told our readers how incredibly easy it is to create, read, write, and manipulate tables and databases of data in Python!</div>
<div>&nbsp;</div>
<div>Most, if not all of us have to manage tabular data at one point or another, and it is safe to say that the number of different formats for tables is well beyond ridiculous. <a href="http://archive.stsci.edu/fits/fits_standard/">FITS</a>, <a href="http://www.ivoa.net/Documents/VOTable/">VO</a>, <a href="http://www.hdfgroup.org/HDF5/whatishdf5.html">HDF5</a>, <a href="http://irsa.ipac.caltech.edu/applications/DDGEN/Doc/ipac_tbl.html">IRSA/IPAC</a>, <a href="http://vizier.u-strasbg.fr/doc/catstd.htx">MRT/CDS</a>, DAOphot, <a href="http://hea-www.harvard.edu/MST/simul/software/docs/rdb.html">RDB</a>, and <a href="http://en.wikipedia.org/wiki/Comma-separated_values">CSV</a> tables are some of the most common (and I&#8217;ll spare you the acronym definitions). In addition, most of us shy away from using databases, because the learning curve is steep, and often requires learning new tools. This is where asciitable and ATpy come in.</div>
<p><div class="toggle"></p>
<div>The goal of <a href="http://cxc.harvard.edu/contrib/asciitable/">asciitable</a>, developed by Tom Aldcroft, is to provide an extensible framework for reading in tables of ASCII data in arbitrary formats. If the table format you are using is not one of the presets, you can easily define your own!</div>
<div>&nbsp;</div>
<div>But that is only half of the story. Manipulating tables (adding/removing/renaming columns, sorting, etc.) is still not easy, and asciitable does not help with non-ASCII formats. Enter <a href="http://atpy.sourceforge.net">ATpy</a>, a tool that <a href="http://www.astrobetter.com/about/about-eli/">Eli Bressert</a> and I have developed, which brings everything together in an easy-to-use package. ATpy uses asciitable to read ASCII tables, but also uses other packages to directly read and write from FITS, VO, and HDF5 tables, and can also talk to SQL databases, including <a href="http://www.mysql.com/">MySQL</a>, <a href="http://www.postgresql.org/">PostGreSQL</a>, and <a href="http://www.sqlite.org/">SQLite</a>. Reading tables is a one liner for each format:</div>
<div>&nbsp;</div>

<div class="wp_syntax"><div class="code"><pre class="python" style="font-family:monospace;"><span style="color: #66cc66;">&gt;&gt;&gt;</span> <span style="color: #ff7700;font-weight:bold;">import</span> atpy
<span style="color: #66cc66;">&gt;&gt;&gt;</span> t = atpy.<span style="color: black;">Table</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">'mytable.fits'</span><span style="color: black;">&#41;</span> <span style="color: #808080; font-style: italic;"># FITS table</span>
<span style="color: #66cc66;">&gt;&gt;&gt;</span> t = atpy.<span style="color: black;">Table</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">'mytable.hdf5'</span><span style="color: black;">&#41;</span> <span style="color: #808080; font-style: italic;"># HDF5 table</span>
<span style="color: #66cc66;">&gt;&gt;&gt;</span> t = atpy.<span style="color: black;">Table</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">'mytable.xml'</span><span style="color: black;">&#41;</span> <span style="color: #808080; font-style: italic;"># VO table</span>
<span style="color: #66cc66;">&gt;&gt;&gt;</span> t = atpy.<span style="color: black;">Table</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">'mytable.tbl'</span><span style="color: black;">&#41;</span> <span style="color: #808080; font-style: italic;"># IRSA/IPAC table</span>
<span style="color: #66cc66;">&gt;&gt;&gt;</span> t = atpy.<span style="color: black;">Table</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">'sqlite'</span>, <span style="color: #483d8b;">'mytable.db'</span><span style="color: black;">&#41;</span> <span style="color: #808080; font-style: italic;"># SQLite database</span>
<span style="color: #66cc66;">&gt;&gt;&gt;</span> t = atpy.<span style="color: black;">Table</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">'mysql'</span>, db=<span style="color: #483d8b;">'python'</span>, <span style="color: #dc143c;">user</span>=<span style="color: #483d8b;">'bob'</span>, \
                   passwd=<span style="color: #483d8b;">'1(@N(!10))'</span><span style="color: black;">&#41;</span> <span style="color: #808080; font-style: italic;"># MySQL database</span>
<span style="color: #66cc66;">&gt;&gt;&gt;</span> t = atpy.<span style="color: black;">Table</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">'postgres'</span>, database=<span style="color: #483d8b;">'python'</span>, \
                   <span style="color: #dc143c;">user</span>=<span style="color: #483d8b;">'alice'</span>, password=<span style="color: #483d8b;">'1(@N(!10))'</span><span style="color: black;">&#41;</span> <span style="color: #808080; font-style: italic;"># PostGreSQL database</span></pre></div></div>

<div>and so on. Writing tables is equally easy:</div>
<div>&nbsp;</div>

<div class="wp_syntax"><div class="code"><pre class="python" style="font-family:monospace;"><span style="color: #66cc66;">&gt;&gt;&gt;</span> t.<span style="color: black;">write</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">'mynewtable.fits'</span><span style="color: black;">&#41;</span> <span style="color: #808080; font-style: italic;"># FITS</span>
<span style="color: #66cc66;">&gt;&gt;&gt;</span> t.<span style="color: black;">write</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">'mynewtable.hdf5'</span><span style="color: black;">&#41;</span> <span style="color: #808080; font-style: italic;"># HDF5</span>
<span style="color: #66cc66;">&gt;&gt;&gt;</span> t.<span style="color: black;">write</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">'mynewtable.xml'</span><span style="color: black;">&#41;</span> <span style="color: #808080; font-style: italic;"># VO</span>
    etc.</pre></div></div>

<div>The file type is automatically recognized from the extension, but you can always override the default or define your own extensions. You can also create tables from scratch:</div>
<div>&nbsp;</div>

<div class="wp_syntax"><div class="code"><pre class="python" style="font-family:monospace;"><span style="color: #66cc66;">&gt;&gt;&gt;</span> <span style="color: #ff7700;font-weight:bold;">import</span> atpy
<span style="color: #66cc66;">&gt;&gt;&gt;</span> <span style="color: #ff7700;font-weight:bold;">import</span> numpy <span style="color: #ff7700;font-weight:bold;">as</span> np
<span style="color: #66cc66;">&gt;&gt;&gt;</span> ra = np.<span style="color: #dc143c;">random</span>.<span style="color: #dc143c;">random</span><span style="color: black;">&#40;</span><span style="color: #ff4500;">5</span><span style="color: black;">&#41;</span>
<span style="color: #66cc66;">&gt;&gt;&gt;</span> dec = np.<span style="color: #dc143c;">random</span>.<span style="color: #dc143c;">random</span><span style="color: black;">&#40;</span><span style="color: #ff4500;">5</span><span style="color: black;">&#41;</span>
<span style="color: #66cc66;">&gt;&gt;&gt;</span> flux = np.<span style="color: #dc143c;">random</span>.<span style="color: #dc143c;">random</span><span style="color: black;">&#40;</span><span style="color: #ff4500;">5</span><span style="color: black;">&#41;</span>
<span style="color: #66cc66;">&gt;&gt;&gt;</span> t = atpy.<span style="color: black;">Table</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
<span style="color: #66cc66;">&gt;&gt;&gt;</span> t.<span style="color: black;">add_column</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">'ra'</span>, ra, unit=<span style="color: #483d8b;">'deg'</span><span style="color: black;">&#41;</span>
<span style="color: #66cc66;">&gt;&gt;&gt;</span> t.<span style="color: black;">add_column</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">'dec'</span>, dec, unit=<span style="color: #483d8b;">'deg'</span><span style="color: black;">&#41;</span>
<span style="color: #66cc66;">&gt;&gt;&gt;</span> t.<span style="color: black;">add_column</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">'flux'</span>, flux, unit=<span style="color: #483d8b;">'mJy'</span><span style="color: black;">&#41;</span></pre></div></div>

<div>Either way, accessing and modifying the data is simple and straightforward:</div>
<div>&nbsp;</div>

<div class="wp_syntax"><div class="code"><pre class="python" style="font-family:monospace;"><span style="color: #66cc66;">&gt;&gt;&gt;</span> t<span style="color: black;">&#91;</span><span style="color: #483d8b;">'ra'</span><span style="color: black;">&#93;</span>
<span style="color: #dc143c;">array</span><span style="color: black;">&#40;</span><span style="color: black;">&#91;</span> <span style="color: #ff4500;">0.2331109</span> ,  <span style="color: #ff4500;">0.23474117</span>,  <span style="color: #ff4500;">0.62186612</span>,  <span style="color: #ff4500;">0.65640893</span>,  <span style="color: #ff4500;">0.71596516</span><span style="color: black;">&#93;</span><span style="color: black;">&#41;</span>
<span style="color: #66cc66;">&gt;&gt;&gt;</span> t<span style="color: black;">&#91;</span><span style="color: #483d8b;">'ra'</span><span style="color: black;">&#93;</span><span style="color: black;">&#91;</span><span style="color: #ff4500;">1</span><span style="color: black;">&#93;</span> = <span style="color: #ff4500;">0.1</span>
<span style="color: #66cc66;">&gt;&gt;&gt;</span> t<span style="color: black;">&#91;</span><span style="color: #483d8b;">'ra'</span><span style="color: black;">&#93;</span>
<span style="color: #dc143c;">array</span><span style="color: black;">&#40;</span><span style="color: black;">&#91;</span> <span style="color: #ff4500;">0.2331109</span> ,  <span style="color: #ff4500;">0.1</span>       ,  <span style="color: #ff4500;">0.62186612</span>,  <span style="color: #ff4500;">0.65640893</span>,  <span style="color: #ff4500;">0.71596516</span><span style="color: black;">&#93;</span><span style="color: black;">&#41;</span></pre></div></div>

<div>as is manipulating tables:</div>
<div>&nbsp;</div>

<div class="wp_syntax"><div class="code"><pre class="python" style="font-family:monospace;"><span style="color: #66cc66;">&gt;&gt;&gt;</span> t.<span style="color: black;">sort</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">'ra'</span><span style="color: black;">&#41;</span>
<span style="color: #66cc66;">&gt;&gt;&gt;</span> t.<span style="color: black;">rename_column</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">'ra'</span>,<span style="color: #483d8b;">'ra_j2000'</span><span style="color: black;">&#41;</span>
<span style="color: #66cc66;">&gt;&gt;&gt;</span> t.<span style="color: black;">remove_column</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">'flux'</span><span style="color: black;">&#41;</span></pre></div></div>

<div>Hopefully, if you are not already using ATpy, we&#8217;ve convinced you to at least give it a try! Installation instructions and a user-friendly manual are located <a href=" http://atpy.sourceforge.net/">here</a>. We&#8217;ve recently created a new <a href="http://groups.google.com/group/atpy-users">user discussion</a> group at Google groups, which we encourage you to make use of! To read ASCII tables, you will also need to install <a title="asciitable 0.2.4" href="http://cxc.harvard.edu/contrib/asciitable/" target="_blank">asciitable</a>. For more general information about using Python in Astronomy, we encourage you to visit the <a href="http://www.astropython.org/">astropython.org</a> web site!</div>
</div>]]></content:encoded>
			<wfw:commentRss>http://www.astrobetter.com/atpy-and-asciitable/feed/</wfw:commentRss>
		<slash:comments>10</slash:comments>
		</item>
		<item>
		<title>Parallel Processing in Python</title>
		<link>http://www.astrobetter.com/parallel-processing-in-python/</link>
		<comments>http://www.astrobetter.com/parallel-processing-in-python/#comments</comments>
		<pubDate>Mon, 09 Aug 2010 13:13:30 +0000</pubDate>
		<dc:creator>Guest</dc:creator>
				<category><![CDATA[python]]></category>
		<category><![CDATA[analysis]]></category>

		<guid isPermaLink="false">http://www.astrobetter.com/?p=2693</guid>
		<description><![CDATA[Today we have a guest post by Ian Crossfield (UCLA) on parallel computing with python. When analyzing astronomical data, one often finds oneself repeating the same tasks over and over again (e.g. fitting a model to a PSF, measuring the width of a spectral feature, etc.). Sometimes the results of one computation influence the next, [...]]]></description>
			<content:encoded><![CDATA[<p></p><p><em>Today we have a guest post by <a href="http://www.astro.ucla.edu/~ianc/">Ian Crossfield</a></em><em> (UCLA) on parallel computing with python.</em></p>
<p><a href="http://www.astrobetter.com/wp-content/uploads/2010/08/Virginia-Tech-Apple-Mac-Supercomputer-300x2251-e1281112665254.jpg"><img class="alignright size-full wp-image-2704" title="Virginia-Tech-Apple-Mac-Supercomputer-300x225" src="http://www.astrobetter.com/wp-content/uploads/2010/08/Virginia-Tech-Apple-Mac-Supercomputer-300x2251-e1281112665254.jpg" alt="" width="220" height="165" /></a>When analyzing astronomical data, one often finds oneself repeating the same tasks over and over again (e.g. fitting a model to a PSF, measuring the width of a spectral feature, etc.). Sometimes the results of one computation influence the next, in which case there&#8217;s no choice but to compute everything sequentially.  In many cases, though, the computations are independent and (with sufficient computing power) could all be performed simultaneously. You like your laptop or workstation (and don&#8217;t want to worry about the hassle of learning about your local computing cluster), but aren&#8217;t sure how to take advantage of the multiple CPU cores available on even mid-range modern computers.  What&#8217;s an astronomer to do?<br />
<div class="toggle"></p>
<p>Python has <a href="http://wiki.python.org/moin/ParallelProcessing">many and various options</a> to enable this sort of parallel processing; I&#8217;ve only tried two. One, &#8220;<a href="http://www.parallelpython.com/">Parallel Python</a>,&#8221; seemed to require a high degree of specificity about module dependencies and other issues I would rather not have to deal with; in constrast, getting the &#8220;<a href="http://www.boddie.org.uk/python/pprocess.html">PProcess</a>&#8221; module to run is simplicity itself, and I recommend it to anyone who has need of parallel-processing capabilities.</p>
<p>Just download the &#8220;<a href="http://www.boddie.org.uk/python/pprocess.html">pprocess</a>&#8221; package and put it in your Python path.  Then it&#8217;s simply a matter of using pprocess&#8217; slick &#8220;Map&#8221; wrapper functionality; essentially this creates a wrapper function that you call with <em>exactly</em> the same arguments you normally use for your function; see the example below.  You can spawn as many simultaneous computations as you&#8217;d like with the &#8220;limit&#8221; option, but you won&#8217;t see much benefit once you set this to the number of CPU cores you have at your disposal.</p>
<p>First, here is a schematic example</p>

<div class="wp_syntax"><div class="code"><pre class="python" style="font-family:monospace;"><span style="color: #808080; font-style: italic;"># Just replace this (calling one function twice, in series):</span>
desired_values = <span style="color: black;">&#91;</span>function<span style="color: black;">&#40;</span>args<span style="color: black;">&#41;</span> <span style="color: #ff7700;font-weight:bold;">for</span> args <span style="color: #ff7700;font-weight:bold;">in</span> <span style="color: black;">&#91;</span>args1,args2<span style="color: black;">&#93;</span><span style="color: black;">&#93;</span>
&nbsp;
<span style="color: #808080; font-style: italic;"># ... with this (calling one function twice, in parallel):</span>
<span style="color: #ff7700;font-weight:bold;">import</span> pprocess
nproc = <span style="color: #ff4500;">2</span>  	<span style="color: #808080; font-style: italic;"># maximum number of simultaneous processes desired</span>
results = pprocess.<span style="color: black;">Map</span><span style="color: black;">&#40;</span>limit=nproc, reuse=<span style="color: #ff4500;">1</span><span style="color: black;">&#41;</span>
parallel_function = results.<span style="color: black;">manage</span><span style="color: black;">&#40;</span>pprocess.<span style="color: black;">MakeReusable</span><span style="color: black;">&#40;</span>function<span style="color: black;">&#41;</span><span style="color: black;">&#41;</span>
parallel_function<span style="color: black;">&#40;</span>args1<span style="color: black;">&#41;</span>
parallel_function<span style="color: black;">&#40;</span>args2<span style="color: black;">&#41;</span>
desired_values = results<span style="color: black;">&#91;</span><span style="color: #ff4500;">0</span>:<span style="color: #ff4500;">2</span><span style="color: black;">&#93;</span></pre></div></div>

<p>Now for a concrete example:</p>

<div class="wp_syntax"><div class="code"><pre class="python" style="font-family:monospace;"><span style="color: #ff7700;font-weight:bold;">import</span> pprocess
<span style="color: #ff7700;font-weight:bold;">import</span> <span style="color: #dc143c;">time</span>
<span style="color: #ff7700;font-weight:bold;">import</span> numpy <span style="color: #ff7700;font-weight:bold;">as</span> np
&nbsp;
<span style="color: #808080; font-style: italic;"># Define a function to parallelize:</span>
<span style="color: #ff7700;font-weight:bold;">def</span> takeuptime<span style="color: black;">&#40;</span>ntrials<span style="color: black;">&#41;</span>:
    <span style="color: #483d8b;">&quot;&quot;&quot;A function to waste CPU cycles&quot;&quot;&quot;</span>
    <span style="color: #ff7700;font-weight:bold;">for</span> ii <span style="color: #ff7700;font-weight:bold;">in</span> <span style="color: #008000;">range</span><span style="color: black;">&#40;</span>ntrials<span style="color: black;">&#41;</span>:
        junk = np.<span style="color: black;">std</span><span style="color: black;">&#40;</span>np.<span style="color: #dc143c;">random</span>.<span style="color: black;">randn</span><span style="color: black;">&#40;</span>1e5<span style="color: black;">&#41;</span><span style="color: black;">&#41;</span>
    <span style="color: #ff7700;font-weight:bold;">return</span> junk
&nbsp;
list_of_args = <span style="color: black;">&#91;</span><span style="color: #ff4500;">500</span>, <span style="color: #ff4500;">500</span><span style="color: black;">&#93;</span>
&nbsp;
<span style="color: #808080; font-style: italic;"># Serial computation:</span>
tic=<span style="color: #dc143c;">time</span>.<span style="color: #dc143c;">time</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
serial_results = <span style="color: black;">&#91;</span>takeuptime<span style="color: black;">&#40;</span>args<span style="color: black;">&#41;</span> <span style="color: #ff7700;font-weight:bold;">for</span> args <span style="color: #ff7700;font-weight:bold;">in</span> list_of_args<span style="color: black;">&#93;</span>
<span style="color: #ff7700;font-weight:bold;">print</span> <span style="color: #483d8b;">&quot;%f s for traditional, serial computation.&quot;</span> <span style="color: #66cc66;">%</span> <span style="color: black;">&#40;</span><span style="color: #dc143c;">time</span>.<span style="color: #dc143c;">time</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>-tic<span style="color: black;">&#41;</span>
&nbsp;
<span style="color: #808080; font-style: italic;"># Parallel computation:</span>
nproc = <span style="color: #ff4500;">2</span>  	<span style="color: #808080; font-style: italic;"># maximum number of simultaneous processes desired</span>
results = pprocess.<span style="color: black;">Map</span><span style="color: black;">&#40;</span>limit=nproc, reuse=<span style="color: #ff4500;">1</span><span style="color: black;">&#41;</span>
parallel_function = results.<span style="color: black;">manage</span><span style="color: black;">&#40;</span>pprocess.<span style="color: black;">MakeReusable</span><span style="color: black;">&#40;</span>takeuptime<span style="color: black;">&#41;</span><span style="color: black;">&#41;</span>
tic=<span style="color: #dc143c;">time</span>.<span style="color: #dc143c;">time</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
<span style="color: black;">&#91;</span>parallel_function<span style="color: black;">&#40;</span>args<span style="color: black;">&#41;</span> <span style="color: #ff7700;font-weight:bold;">for</span> args <span style="color: #ff7700;font-weight:bold;">in</span> list_of_args<span style="color: black;">&#93;</span><span style="color: #66cc66;">;</span>  <span style="color: #808080; font-style: italic;"># Start computing things</span>
parallel_results = results<span style="color: black;">&#91;</span><span style="color: #ff4500;">0</span>:<span style="color: #ff4500;">3</span><span style="color: black;">&#93;</span>
<span style="color: #ff7700;font-weight:bold;">print</span> <span style="color: #483d8b;">&quot;%f s for parallel computation.&quot;</span> <span style="color: #66cc66;">%</span> <span style="color: black;">&#40;</span><span style="color: #dc143c;">time</span>.<span style="color: #dc143c;">time</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span> - tic<span style="color: black;">&#41;</span></pre></div></div>

<p>Do you parallelize your analysis?  Why or why not, and (if so) what do you use?</p>
<p>Information on other parallel computing packages can be found on the <a href="http://www.astrobetter.com/wiki/tiki-index.php?page=Parallel+computing+with+python">AstroBetter wiki</a>.</p>
</div>]]></content:encoded>
			<wfw:commentRss>http://www.astrobetter.com/parallel-processing-in-python/feed/</wfw:commentRss>
		<slash:comments>10</slash:comments>
		</item>
	</channel>
</rss>

