PiCloud’s RESTful API allows you to utilize PiCloud directly over HTTP, within any programming environment. You will find this functionality useful if you wish to utilize PiCloud outside of a Python environment (e.g. a Java program) or cannot install the cloud library in your Python Environment (e.g. using App Engine).
The interface of PiCloud’s REST API is designed to be similar to cloud; you will find REST analogs to cloud.info(), cloud.kill(), etc. Unfortunately, it is not possible to replicate all functionality exactly in a language-agnostic means. For instance, the REST analog of cloud.call() is two-steps: after a function is first published to PiCloud via cloud.rest, you can POST to an associated URL from anywhere to create a job. While this process requires the function to be written in Python, it allows it to be invoked with arbitrary arguments from any environment that can make web requests.
Throughout this document, we will be showing example queries made using curl. You can make similar requests, substituting key with your api key and secret_key with your api secret key.
All queries to the REST API must be made over HTTPS (unencrypted HTTP is not supported) to the PiCloud API server, located at https://api.picloud.com.
Requests are authenticated with HTTP Basic Authentication. The username is your api key and the password is your api secret key.
Unless otherwise noted, responses will be a JSON encoded dictionary containing the protocol version and query-specific information.
curl example:
$ curl -k -u 'key:secret_key' https://api.picloud.com/packages/list/
{"version": "0.1", "servers": ["https://api.picloud.com/"]}
For a user with key 1337 and secret key of abcd, the request would be:
$ curl -k -u '1337:abcd' https://api.picloud.com/packages/list/
{"version": "0.1", "servers": ["https://api.picloud.com/"]}
Standard Python example:
import base64
import json
import urllib2
base64string = base64.encodestring('%s:%s' % (key, secret_key))[:-1] #[:-1] removes newline at end
http_headers = {'Authorization' : 'Basic %s' % base64string} #HTTP basic authentication
request = urllib2.Request('https://api.picloud.com/packages/list/', post_values={}, http_headers)
response = urllib2.urlopen(request)
data = json.load(response) #{"version": "0.1", "servers": ["https://api.picloud.com/"]}
Google App Engine urlfetch example:
import base64
import json
import urlfetch
base64string = base64.encodestring('%s:%s' % (key, secret_ley))[:-1]
http_headers = {'Authorization' : 'Basic %s' % base64string}
response = urlfetch.fetch(url='https://api.picloud.com/packages/list/',
payload={}, #POST DATA if method was urlfetch.POST
method=urlfetch.GET,
headers=http_headers)
data = json.load(response) #{"version": "0.1", "servers": ["https://api.picloud.com/"]}
After publishing a function to PiCloud using the cloud package, you will be able to invoke it from any environment.
A Python callable (function) is published by passing it into cloud.rest.publish(). A URL, dictated by the unique label you specify, will be returned. An example:
def square(x):
"""Returns square of a number"""
print 'Squaring %d' % x
return x*x
cloud.setkey(api-key, api-secretkey)
cloud.rest.publish(square, "square_func")
Result:
"https://api.picloud.com/r/unique_user_id/square_func"
Warning
Labels are unique. If a previously published function labeled square_func exists, it will be removed to make way for a new entry.
Make an HTTP GET request to the URL returned by cloud.rest.publish() to obtain information about the published function, including its label, encoding type of the return value, URL to access it, signature showing its parameters, and its description (docstring of the function).
To retrieve information about square_func:
$ curl -k -u 'key:secret_key' https://api.picloud.com/r/unique_id/square_func/
{
"label": "square_func",
"output_encoding": "json",
"version": "0.1",
"uri": "https://api.picloud.com/r/unique_id/square_func",
"signature": "square_func(x)"
"description": "Returns square of a number"
}
Note
This information is identical to that returned by cloud.rest.info().
Make an HTTP POST request to the returned URL to invoke the function on PiCloud. Keyword arguments to the function should be JSON-encoded and passed as GET or POST parameters:
$ curl -k -u 'key:secret_key' https://api.picloud.com/r/unique_id/square_func/ -X POST -d "x=5"
Result:
{
"jid": 12,
"version": "0.1"
}
This API call returns the jid of the started job which uniquely identifies the job on PiCloud, and can be used to retrieve information, kill, delete or get the job’s result, through either cloud or the REST API.
Warning
As arguments must be specified by keyword, there is no way to specify a variadic function’s unnamed arguments (*args). A variadic function can only accept keyword arguments (**kwargs).
Both the arguments and result, if any, of the function must be encodable by JSON. The data structures JSON supports is limited compared to the Python pickler; see the docs for more information. If you cannot represent your arguments/return values in JSON, you will need to use binary encoding.
Much of the job management functions in cloud have REST analogs. All of the below functions, unless otherwise noted, can be used regardless of whether a job was created through the REST API or cloud.call().
The REST version of cloud.info() is GET https://api.picloud.com/job/?jids=your_id:
$ curl -k -u 'key:secret_key' https://api.picloud.com/job/?jids=12
{
"info":
{
"12":
{
"status": "done",
"exception": null,
"runtime": 0.1,
"stderr": "",
"stdout": "Squaring 5"
}
},
"version": "0.1"
}
You can limit the data returned by specifying specific fields in the GET args:
$ curl -k -u 'key:secret_key' https://api.picloud.com/job/?jids=12&field=status&field=stdout
{
"info":
{
"12":
{
"status": "done",
"stdout": "Squaring 5"
}
},
"version": "0.1"
}
cloud.kill() can be invoked by a POST to https://api.picloud.com/job/kill/?jids=your_id:
$ curl -k -u 'key:secret_key' -X POST https://api.picloud.com/job/kill/?jids=12
If the jids field is not provided, all of your jobs will be killed.
cloud.delete() can be invoked by a POST to https://api.picloud.com/job/delete/?jids=your_id:
$ curl -k -u 'key:secret_key' -X POST https://api.picloud.com/job/delete/?jids=12
All of the above queries can deal with multiple jids.
Example:
$ curl -k -u 'key:secret_key' https://api.picloud.com/job/?jids=1-4,5-8
cloud.result() is emulated by GET https://api.picloud.com/job/result/?jid=your_id. (Note the singular jid). This function returns a JSON-encoded result:
$ curl -k -u 'key:secret_key' https://api.picloud.com/job/result/?jid=12
{
"version": "0.1",
"result": 25
}
If the job is not yet available, you’ll receive an error:
{"version": "0.1", "error_code": 455, "retry": false, "error_msg": "Requested job is not done."}
Note
The result can be retrieved only for one job per HTTP request; hence jid not jids.
Warning
Only jobs that were created with the PiCloud REST API can have their result accessed through REST.
If arguments or return value cannot be represented with JSON, you will need to make use of the the REST API’s binary mode. Binary data is represented in python as an str object. You are responsible for decoding binary arguments and encoding binary results. Alternatively, you can use Cloud.files to transfer binary objects.
Imagine we have a simple function to create a thumbnail from a JPEG image, similar to the example shown earlier:
def downscale_image(img, size = (32, 24)):
"""Reduce image to a fixed size"""
import Image
img.thumbnail(size, Image.ANTIALIAS) #thumbnail image
return img
Unfortunately, a PIL Image object cannot be encoded in JSON, so we will need to decode an Image from raw JPEG data (an in-memory binary string) and encode it into JPEG data (again, a binary string):
def downscale_image_wrapper(img_str, size = (32, 24)):
import Image
img = Image.open(StringIO(img_str))
downscaled_img = downscale_image(img, size = size)
out_file = StringIO()
downscaled_img.save(out_file, 'JPEG')
return out_file.getvalue()
Binary arguments are passed in as multipart/form-data as a file upload (meaning the MIME Content-Disposition sub-header has a filename parameter):
$ curl -k -u 'key:secret_key' https://api.staging.picloud.com/r/3/downscale_image_wrapper -F img_str=@something.jpg
{"jid": 15029, "version": "0.1"}
(where something.jpg is a JPEG image on disk)
In Python, you can make use of a third-party package, Poster, to upload multipart/form-data. You must specify a filename for an entry to be recognized by PiCloud as binary data:
params = []
# ....
params.append( MultipartParam(
"image",
filename='random_filename', #This field must be set!
filetype='application/octet-stream',
value=some_binary_data) )
On Google App Engine, you can use urlfetch with Poster as follows:
datagen, headers = poster.encode.multipart_encode(params)
data = str().join(datagen)
result = urlfetch.fetch(
url=your_url,
payload=data,
method=urlfetch.POST,
headers=http_headers)
Note
You can intermix binary and JSON arguments. JSON arguments appear in the multipart/form-data as text fields (no filename set in Content-Disposition / Poster MultipartParam).
If your result is binary, you will need to set out_encoding in cloud.rest.publish() to “raw”. In “raw” mode, the REST API result function (https://api.picloud.com/job/result/)’s will respond with the requested job’s result formatted as an octet stream (string). No JSON header will exist in the response. Of course, this mode requires that the published function’s result be a string.
If you are unsure if a result of an arbitrary job will be binary or not, you can check the Content-Type header in the response. JSON responses will have Content-Type: application/json set while raw responses will have Content-Type: application/octet-stream set.
Example:
cloud.files.publish(downscale_image_wrapper, label='downscale_image_wrapper', out_encoding = 'raw')
# Access return value of an invocation of downscale_image_wrapper
$ curl -k -u 'key:secret_key' https://api.staging.picloud.com/job/result/?jid=15029 > out_img.jpg
# You can now open out_img.jpg with a standard image viewer
You can use the REST API to access cloud files.
The REST equivalent of cloud.files.get() is a two step process. First:
$ curl -k -u 'key:secret_key' https://api.picloud.com/file/get/ -d name=your_file_name
This returns a JSON object:
{
"ticket": {
"Date": "Wed, 24 Aug 2011 00:11:18 +0000",
"Authorization": "AWS AKIAJCY7JV52WD4MJSNQ:sfs4g7Dy6GkNnwq0PFW19kQIz4E="
},
"version": "0.1",
"params": {
"action": "https://pi-user-files-test.s3.amazonaws.com/c79038dc410cd010e51bae8f7243ebcee8508971/your_file_name",
"bucket": "pi-user-files-test",
"key": "c79038dc410cd010e51bae8f7243ebcee8508971/abd_name"
}
}
To retrieve the file, issue a GET request to the URI in params[‘action’] field and copy all fields from the ticket dictionary to the HTTP request headers:
curl -H Authorization:"AWS AKIAJCY7JV52WD4MJSNQ:sfs4g7Dy6GkNnwq0PFW19kQIz4E=" -H Date:"Wed, 24 Aug 2011 00:11:18 +0000", https://pi-user-files-test.s3.amazonaws.com/c79038dc410cd010e51bae8f7243ebcee8508971/your_file
The result will be an octet stream whose contents are those of the file requested.
Warning
The ticket you receive from PiCloud can only be used once and must be used within 10 minutes.
The REST equivalent of cloud.files.put() is similar. First POST a request to make a new file to PiCloud:
$ curl -k -u 'key:secret_key' -X POST https://api.picloud.com/file/new/ -d name=your_file_name
This returns a JSON object similar to the Get example:
{
"ticket": {
"AWSAccessKeyId": "AKIAJCY7JV52WD4MJSNQ",
"success_action_redirect": "https://api.picloud.com/file/uploaded/3/13/63634b0d30d23061fa573b7f97677536182f6776/",
"acl": "private",
"key": "c79038dc470cd016e51bae8f7243ebcee8508971/your_file_name",
"signature": "5OnebYdDRtBbhiwg7rSoOLsipBQ=",
"policy": "eyJjb25kaXRpb25zIjogW3siYnVja2V0IjogInBpLXVzZXItZmlsZXMtdGVzdCJ9LCB7ImtleSI6ICJjNzkwMzhkYzQ3MGNkMDE2ZTUxYmFlOGY3MjQzZWJjZWU4NTA4OTcxL3RyZWVfbmV3In0sIHsiYWNsIjogInByaXZhdGUifSwgeyJzdWNjZXNzX2FjdGlvbl9yZWRpcmVjdCI6ICJodHRwOi8vc3RhZ2luZy5waWNsb3VkLmNvbS9weWFwaS9maWxlL3VwbG9hZGVkLzMvMTAzLzYzNjM0YjBkMzA1MjMwNjFmYTU3M2I3Zjk3Njc3NTM2MTgyZjY3NzYvIn0sIFsic3RhcnRzLXdpdGgiLCAiJENvbnRlbnQtVHlwZSIsICIiXV0sICJleHBpcmF0aW9uIjogIjIwMTEtMDgtMjRUMTM6NTY6MzZaIn0=",
"Content-Type": "application/octet-stream"
},
"version": "0.1",
"params": {
"action": "https://pi-user-files-test.s3.amazonaws.com/"
}
}
To put the object you must POST to URL in params[‘action’] with multipart/form-data. All items in ticket become form data. In addition, the actual file contents will be provided as a file upload with key ‘file’. Curl example:
$ curl -L "https:/.s3.amazonaws.com/" -F "AWSAccessKeyId=AKIAJCY7JV52WD4MJSNQ" -F "success_action_redirect=https://api.picloud.com/file/uploaded/3/106/9f19215d18200977284e4cc506a04bde70c030fb/" -F "acl=private" -F "key=c79038dc470cd016e51bae8f7243ebcee8508971/tree_new" -F "signature=Pw+QlFMNwsJZ5joo4NBDaYYxAQs=" -F "policy=eyJjb25kaXRpb25zIjogW3siYnVja2V0IjogInBpLXVzZXItZmlsZXMtdGVzdCJ9LCB7ImtleSI6ICJjNzkwMzhkYzQ3MGNkMDE2ZTUxYmFlOGY3MjQzZWJjZWU4NTA4OTcxL3RyZWVfbmV3In0sIHsiYWNsIjogInByaXZhdGUifSwgeyJzdWNjZXNzX2FjdGlvbl9yZWRpcmVjdCI6ICJodHRwOi8vc3RhZ2luZy5waWNsb3VkLmNvbS9weWFwaS9maWxlL3VwbG9hZGVkLzMvMTA2LzlmMTkyMTVkMTgyMDA5NzcyODRlNGNjNTA2YTA0YmRlNzBjMDMwZmIvIn0sIFsic3RhcnRzLXdpdGgiLCAiJENvbnRlbnQtVHlwZSIsICIiXV0sICJleHBpcmF0aW9uIjogIjIwMTEtMDgtMjRUMTQ6MTU6NDhaIn0=" -F content-type="application/octet-stream" -F file=your_file_to_upload
Warning
You must obey the HTTP 303 redirect (curl -L option) that this query will return or PiCloud will not know that your file is ready! (urllib2 and urlfetch do this automatically).
$ curl -k -u 'key:secret_key' https://api.picloud.com/file/list/
{"files": ["file_you_uploaded", "another_file"], "version": "0.1"}
$ curl -k -u 'key:secret_key' https://api.picloud.com/file/exists/ -d name=your_file_name
{"files": ["file_you_uploaded", "another_file"], "version": "0.1"}
$ curl -k -u 'key:secret_key' https://api.picloud.com/file/exists/ -X POST -d name=your_file_name
{"deleted": true, "version": "0.1"}