PiCloud: Advantages of running python in the cloud

by Eli on December 28, 2010

When we go to conferences, observing runs, collaborations and other travels the first thing that comes to mind for packing is our laptop (or iPad). The laptop is steadfast, resolute and is always there when we need to work or relax. But there’s one slight problem; when we need to do work on the run, which can computationally intensive, our laptop can be limiting.

Sometimes, even our desktop computers don’t have enough kick to get our computing problems done. If you’re a Python user, then you’re in luck. There is a new service called PiCloud that allows you to send algorithms and data on the fly for heavy computations. It works in a near seamless fashion with our native Python code and what could take hours of computation time will be done in a few minutes or less.

Here’s an example of a simple and computationally intensive algorithm.

def fib_sequence(nth):
	x = [0,1]
	if nth == 1:
		return 1
	for i in xrange(nth-1):
		x.append(x[-1] + x[-2])
		x = [ x[-2], x[-1]]
	return x[-1]

The algorithm could be more efficient, but I’m leaving it as is to illustrate how fast PiCloud can be. Now let’s setup the PiCloud module and compare the speed to running it on my laptop (3.06 GHz Intel Core 2 Duo with 4 GB of RAM).

#Laptop time
local = fib_sequence(10000000)
#PiCloud time
import cloud
# This has been anonymized, as the API keys are unique to
# each personal account
cloud.setkey(api_key=something1, api_secretkey='something2')
jid = cloud.call(fib_sequence, 10000000, _high_cpu=True)
remote = cloud.result(jid)

The time for my laptop to crunch and obtain the millionth element in the fibonacci sequence is 70 seconds, whereas PiCloud computed it in 48 seconds (including sending and receiving). The real power of PiCloud though is parallelism, which we cannot readily use with the fibonacci sequence. So let’s assume that we need to know the millionth fibonacci number 10 times.

num_parallel = 10
jids = cloud.map(fib_sequence, [10000000]*num_parallel, _high_cpu=True)
remote = cloud.result(jids)
return remote

The parallel processing task took 60 seconds in total, which is roughly 12 times faster than my laptop. These examples above are very simplistic and ideal ways of utilizing PiCloud to highlight its usefulness, particularly with parallel processing.

PiCloud is not just limited to Python native modules. You can install Numpy, Scipy or other modules with ease on your PiCloud account. In the next post, I’ll give some examples of installing the modules and how much quicker such modules are. Note, this post is a simplified version of one of several examples on PiCloud’s documentation.

PiCloud is a paid service and its pricing seems to be reasonable. If you’re keen on trying out PiCloud you will be happy to know that there’s free computing time available to test it. So why not give it a try and let us know what you think in the comments! The days of having the combination of a laptop and desktop may be irrelevant and all we need is a lightweight laptop to access services like PiCloud.

{ 2 comments… read them below or add one }

1 Lachlan July 12, 2017 at 11:52 am

Hi Eli … thanks for your blog post. I came across it looking for just the service you describe, ie. a really easy way to run parallel Python code in the cloud. But it looks like PiCloud is no more; is that correct? Do you know where I might be able to get the kind of services you describe in your blog?

Any hep would be much appreciated.



2 Ariel July 9, 2018 at 6:03 pm

Hi @Lachlan,

As I understand it, Picloud is indeed no more (I believe they were bought up by Dropbox). You might want to check our software, Cloudknot (https://richford.github.io/cloudknot/), which does something similar to this.


Leave a Comment

Previous post:

Next post: