Interface for cloud - the clientside module for PiCloud
Sample usage:
import cloud
jid = cloud.call(lambda: 3*3)
>> Returns a job identifier
cloud.result(jid)
>> Returns 9
This will run the function lambda: 3*3 on PiCloud’s cluster, and return the result. Most functions, even user-defined ones, can be passed through cloud.call
For cloud to work, you must first do one the following:
Connect cloud to your PiCloud account, given your api_key. Your api_key is provided by PiCloud on the API Keys section of the PiCloud website.
The api_secretkey is stored on your machine. However, if you have not previously used this api_key or selected it in ‘picloud setup’, you will need to provide it.
server_url specifies the PiCloud server to connect to. Leave this blank to have cloud auto-resolve servers.
restart forces the cloud to reconnect
This command will disable the simulator if it is running.
Simulate cloud functionality locally. If force_restart or not already in simulation mode, restart cloud and enter simulation. In simulation mode, the cloud will be run locally, on multiple processors, via the multiprocessing library. Additional logging information by default will be enabled. For more information, see PiCloud documention
Invoke func (a callable) in the cloud. When invoked, func will be invoked on PiCloud’s cluster with the passed args that follow it. The invoked function is known as a ‘job’. The return value of the invoked function is known as the ‘internal result’.
Call will return an integer Job IDentifier (jid) which can be passed into the status and result methods to obtain the status of the job and the internal result respectively.
Example:
def mult3(x):
return 3*x
cloud.call(mult3, 2)
This will cause mult3 to be invoked on PiCloud’s cluster with x=2
See online documentation for additional information.
Reserved special kwargs (see docs for details):
A list of functions that should be run on the callee’s computer once this job finishes successfully.
A list of functions that should be run on the callee’s computer if this job errors.
An iterable of jids that represents all jobs that must complete successfully before the job created by this call function may be run.
A string specifying a custom environment you wish to run your job within. See environments overview at http://blog.picloud.com/2011/09/26/introducing-environments-run-anything-on-picloud/
This keyword can be used to speed up serialization, at the cost of some functionality. This affects the serialization of both the arguments and return values func will always be serialized by the enhanced serializer, with debugging features. Possible values keyword are:
Terminate the Python interpreter func runs in after func completes, preventing the interpreter from being used by subsequent jobs. See Technical Overview for more info.
A user-defined string label that is attached to the job. Labels can be used to filter when viewing jobs interactively (i.e. on the PiCloud website).
A positive integer denoting the job’s priority. PiCloud tries to run jobs with lower priority numbers before jobs with higher priority numbers.
Set this to True to enable profiling of your code. Profiling information is valuable for debugging, but may slow down your job.
In the very rare event of hardware failure, this flag indicates that the job can be restarted if the failure happened in the middle of the job. By default, this is true. This should be unset if the job has external state (e.g. it modifies a database entry)
Select the type of compute resources to use. PiCloud supports four types, specified as strings:
1 compute unit, 300 MB ram, low I/O (default)
2.5 compute units, 800 MB ram, medium I/O
3.25 compute units, 8 GB ram, high I/O
Up to 2 compute units (variable), 300 MB ram, low I/O, 1 IP per core
See http://www.picloud.com/pricing/ for pricing information
Block current thread of execution until the job specified by the integer jids completes. Completion is defined as the job finishing or erroring (including stalling). If the job errored, a CloudException detailing the exception that triggered the error is thrown. If multiple errors occur, it is undefined which job’s exception will be raised. If the job does not exist, a CloudException is thrown.
This method also accepts an iterable describing jids and blocks until all corresponding jobs finish. If an error is seen, join may terminate before all jobs finish.
If timeout is set to a number, join will raise a CloudTimeoutError if the job is still running after timeout seconds
If ignore_errors is True, no CloudException will be thrown if a job errored. Join will block until every job is complete, regardless of error status.
Returns the status of the job specified by the integer jids. If the job does not exist, a CloudException is thrown. This method also accepts an iterable describing jids, in which case a respective list of statuses is returned.
Blocks until the job specified by the integer jids has completed and then returns the internal result of the job. If the job errored, a CloudException detailing the exception that triggered the error is thrown. If the job does not exist, a CloudException is thrown.
This function also accepts an iterable describing jids, in which case a respective list of internal results is returned
If timeout is set to a number, result will raise a CloudTimeoutError if the job is still running after timeout seconds
If ignore_errors is True, a job that errored will not raise an exception. Instead, its return value is the CloudException describing the error.
Map func (a callable) over argument sequence(s). cloud.map is meant to mimic a regular map call such as:
map(lambda x,y: x*y, xlist, ylist)
args can be any number of iterables. If the iterables are of different size, ‘None’ is substituted for a missing entries.
Map will return an iterable describing integer Job IDentifiers (jids). Each jid corresponds to func being invoked on one item of the sequence(s) (a ‘job’). In practice, the jids can (and should) be treated as a single jid; the returned iterable may be passed directly into status, result, etc.
Using cloud.result on the returned jids will return a list of internal results (each being the result of applying the function to an item of the argument sequence(s)).
Example:
def mult(x,y):
return x*y
jids = cloud.map(mult, [1,3,5], [2,4,6])
This will cause mult3 to be invoked on PiCloud’s cluster with x=2
Results:
cloud.result(jids)
>> [2,12,30]
The result is [1*2,3*4,5*6]
See online documentation for additional information
Reserved special kwargs (see docs for details):
An iterable of jids that represents all jobs that must complete successfully before any jobs created by this map function may be run.
A string specifying a custom environment you wish to run your jobs within. See environments overview at http://blog.picloud.com/2011/09/26/introducing-environments-run-anything-on-picloud/
This keyword can be used to speed up serialization, at the cost of some functionality. This affects the serialization of both the map arguments and return values The map function will always be serialized by the enhanced serializer, with debugging features. Possible values keyword are:
Terminate the Python interpreter func runs in after func completes, preventing the interpreter from being used by subsequent jobs. See Technical Overview for more info.
A user-defined string label that is attached to the created jobs. Labels can be used to filter when viewing jobs interactively (i.e. on the PiCloud website).
A positive integer denoting the job’s priority. PiCloud tries to run jobs with lower priority numbers before jobs with higher priority numbers.
Set this to True to enable profiling of your code. Profiling information is valuable for debugging, but may slow down your job.
In the very rare event of hardware failure, this flag indicates that the job can be restarted if the failure happened in the middle of the job. By default, this is true. This should be unset if the job has external state (e.g. it modifies a database entry)
Select the type of compute resources to use. PiCloud supports four types, specified as strings:
1 compute unit, 300 MB ram, low I/O (default)
2.5 compute units, 800 MB ram, medium I/O
3.25 compute units, 8 GB ram, high I/O
variable compute units (2 cu max), 300 MB ram, low I/O, 1 IP per core
See http://www.picloud.com/pricing/ for pricing information
Similar to result, but returns an iterator that iterates, in order, through the internal results of jids as the respective job finishes, allowing quicker access to results and reducing memory requirements of result reading.
If a job being iterated over errors, an exception is raised. However, the iterator does not exhaust; if the exception is caught, one can continue iterating over the remaining jobs.
If timeout is set to a number, a call to the iterator’s next function will raise a CloudTimeoutError after timeout seconds if no result becomes available.
num_in_parallel controls how many results are read-ahead from the cloud Set this to 0 to use the allowed maximum.
If ignore_errors is True, a job that errored will return the CloudException describing its error, rather than raising an Exception
Request information about jobs specified by integer or iterable jids
As this function is designed for console debugging, the return value is a dictionary whose keys are the jids requested. The values are themselves dictionaries, whose keys in turn consist of valid values within the iterable info_requested. Each key maps to key-specific information about the job, e.g. stdout maps to standard output of job.
‘status’, ‘stdout’, ‘stderr’, ‘exception’, ‘runtime’- which return job status, standard output,, standard error, exception raised (if applicable) and real runtime of job respectively. If info_requested is None/empty, all info will be requested
The job must have completed for ‘exception’ to be meaningful. All other items can be accessed
while the job is running.
e.g. cloud.info(42,’stdout’) to get standard output of job 42
Kill any incomplete jobs specified by the integer or iterable jids. If no arguments are specified (or jids is None), kill every job that has been submitted.
Remove all data (result, stdout, etc.) related to jobs specified by the integer or iterable jids Jobs must have finished already to be deleted.
Note
In MP/Simulator mode, this does not delete any datalogs, only in-memory info
Returns a dictionary of information describing the current connection.
Returns true iff cloud processor is being run on the local machine.
Returns true iff current thread of execution is running on PiCloud servers.
Represents an exception that has occurred by a job on the cloud. CloudException will display the job id for failed cloud requests.
CloudException specifically for join timeouts
Return the directory where PiCloud configuration and logging files are stored. Configuration/logs are stored on a per-user basis
Terminate cloud connection (or simulation) and all threads utilized by cloud. Returns True if closed cloud; False if cloud was already closed
Note
cloud will automatically re-open if you invoke any cloud.* again
Warning
This function should never be called by a job running on PiCloud
Cloud Multiprocessing Interface
cloud.mp has effectively the same interface as cloud.
All jobs will be run locally, on multiple processors, via the multiprocessing library
Sample usage:
import cloud.mp
jid = cloud.mp.call(lambda: 3*3)
>> Returns a job identifier
cloud.mp.result(jid)
>> Returns 9
Invoke func (a callable) in the cloud. When invoked, func will be invoked on PiCloud’s cluster with the passed args that follow it. The invoked function is known as a ‘job’. The return value of the invoked function is known as the ‘internal result’.
Call will return an integer Job IDentifier (jid) which can be passed into the status and result methods to obtain the status of the job and the internal result respectively.
Example:
def mult3(x):
return 3*x
cloud.call(mult3, 2)
This will cause mult3 to be invoked on PiCloud’s cluster with x=2
See online documentation for additional information.
Reserved special kwargs (see docs for details):
A list of functions that should be run on the callee’s computer once this job finishes successfully.
A list of functions that should be run on the callee’s computer if this job errors.
An iterable of jids that represents all jobs that must complete successfully before the job created by this call function may be run.
A string specifying a custom environment you wish to run your job within. See environments overview at http://blog.picloud.com/2011/09/26/introducing-environments-run-anything-on-picloud/
This keyword can be used to speed up serialization, at the cost of some functionality. This affects the serialization of both the arguments and return values func will always be serialized by the enhanced serializer, with debugging features. Possible values keyword are:
Terminate the Python interpreter func runs in after func completes, preventing the interpreter from being used by subsequent jobs. See Technical Overview for more info.
A user-defined string label that is attached to the job. Labels can be used to filter when viewing jobs interactively (i.e. on the PiCloud website).
A positive integer denoting the job’s priority. PiCloud tries to run jobs with lower priority numbers before jobs with higher priority numbers.
Set this to True to enable profiling of your code. Profiling information is valuable for debugging, but may slow down your job.
In the very rare event of hardware failure, this flag indicates that the job can be restarted if the failure happened in the middle of the job. By default, this is true. This should be unset if the job has external state (e.g. it modifies a database entry)
Select the type of compute resources to use. PiCloud supports four types, specified as strings:
1 compute unit, 300 MB ram, low I/O (default)
2.5 compute units, 800 MB ram, medium I/O
3.25 compute units, 8 GB ram, high I/O
Up to 2 compute units (variable), 300 MB ram, low I/O, 1 IP per core
See http://www.picloud.com/pricing/ for pricing information
Block current thread of execution until the job specified by the integer jids completes. Completion is defined as the job finishing or erroring (including stalling). If the job errored, a CloudException detailing the exception that triggered the error is thrown. If multiple errors occur, it is undefined which job’s exception will be raised. If the job does not exist, a CloudException is thrown.
This method also accepts an iterable describing jids and blocks until all corresponding jobs finish. If an error is seen, join may terminate before all jobs finish.
If timeout is set to a number, join will raise a CloudTimeoutError if the job is still running after timeout seconds
If ignore_errors is True, no CloudException will be thrown if a job errored. Join will block until every job is complete, regardless of error status.
Returns the status of the job specified by the integer jids. If the job does not exist, a CloudException is thrown. This method also accepts an iterable describing jids, in which case a respective list of statuses is returned.
Blocks until the job specified by the integer jids has completed and then returns the internal result of the job. If the job errored, a CloudException detailing the exception that triggered the error is thrown. If the job does not exist, a CloudException is thrown.
This function also accepts an iterable describing jids, in which case a respective list of internal results is returned
If timeout is set to a number, result will raise a CloudTimeoutError if the job is still running after timeout seconds
If ignore_errors is True, a job that errored will not raise an exception. Instead, its return value is the CloudException describing the error.
Map func (a callable) over argument sequence(s). cloud.map is meant to mimic a regular map call such as:
map(lambda x,y: x*y, xlist, ylist)
args can be any number of iterables. If the iterables are of different size, ‘None’ is substituted for a missing entries.
Map will return an iterable describing integer Job IDentifiers (jids). Each jid corresponds to func being invoked on one item of the sequence(s) (a ‘job’). In practice, the jids can (and should) be treated as a single jid; the returned iterable may be passed directly into status, result, etc.
Using cloud.result on the returned jids will return a list of internal results (each being the result of applying the function to an item of the argument sequence(s)).
Example:
def mult(x,y):
return x*y
jids = cloud.map(mult, [1,3,5], [2,4,6])
This will cause mult3 to be invoked on PiCloud’s cluster with x=2
Results:
cloud.result(jids)
>> [2,12,30]
The result is [1*2,3*4,5*6]
See online documentation for additional information
Reserved special kwargs (see docs for details):
An iterable of jids that represents all jobs that must complete successfully before any jobs created by this map function may be run.
A string specifying a custom environment you wish to run your jobs within. See environments overview at http://blog.picloud.com/2011/09/26/introducing-environments-run-anything-on-picloud/
This keyword can be used to speed up serialization, at the cost of some functionality. This affects the serialization of both the map arguments and return values The map function will always be serialized by the enhanced serializer, with debugging features. Possible values keyword are:
Terminate the Python interpreter func runs in after func completes, preventing the interpreter from being used by subsequent jobs. See Technical Overview for more info.
A user-defined string label that is attached to the created jobs. Labels can be used to filter when viewing jobs interactively (i.e. on the PiCloud website).
A positive integer denoting the job’s priority. PiCloud tries to run jobs with lower priority numbers before jobs with higher priority numbers.
Set this to True to enable profiling of your code. Profiling information is valuable for debugging, but may slow down your job.
In the very rare event of hardware failure, this flag indicates that the job can be restarted if the failure happened in the middle of the job. By default, this is true. This should be unset if the job has external state (e.g. it modifies a database entry)
Select the type of compute resources to use. PiCloud supports four types, specified as strings:
1 compute unit, 300 MB ram, low I/O (default)
2.5 compute units, 800 MB ram, medium I/O
3.25 compute units, 8 GB ram, high I/O
variable compute units (2 cu max), 300 MB ram, low I/O, 1 IP per core
See http://www.picloud.com/pricing/ for pricing information
Similar to result, but returns an iterator that iterates, in order, through the internal results of jids as the respective job finishes, allowing quicker access to results and reducing memory requirements of result reading.
If a job being iterated over errors, an exception is raised. However, the iterator does not exhaust; if the exception is caught, one can continue iterating over the remaining jobs.
If timeout is set to a number, a call to the iterator’s next function will raise a CloudTimeoutError after timeout seconds if no result becomes available.
num_in_parallel controls how many results are read-ahead from the cloud Set this to 0 to use the allowed maximum.
If ignore_errors is True, a job that errored will return the CloudException describing its error, rather than raising an Exception
Request information about jobs specified by integer or iterable jids
As this function is designed for console debugging, the return value is a dictionary whose keys are the jids requested. The values are themselves dictionaries, whose keys in turn consist of valid values within the iterable info_requested. Each key maps to key-specific information about the job, e.g. stdout maps to standard output of job.
‘status’, ‘stdout’, ‘stderr’, ‘exception’, ‘runtime’- which return job status, standard output,, standard error, exception raised (if applicable) and real runtime of job respectively. If info_requested is None/empty, all info will be requested
The job must have completed for ‘exception’ to be meaningful. All other items can be accessed
while the job is running.
e.g. cloud.info(42,’stdout’) to get standard output of job 42
Kill any incomplete jobs specified by the integer or iterable jids. If no arguments are specified (or jids is None), kill every job that has been submitted.
Remove all data (result, stdout, etc.) related to jobs specified by the integer or iterable jids Jobs must have finished already to be deleted.
Note
In MP/Simulator mode, this does not delete any datalogs, only in-memory info
Returns a dictionary of information describing the current connection.
Terminate cloud connection (or simulation) and all threads utilized by cloud. Returns True if closed cloud; False if cloud was already closed
Note
cloud will automatically re-open if you invoke any cloud.* again
Warning
This function should never be called by a job running on PiCloud
For managing files on PiCloud’s S3 store.
Api keys must be configured before using any functions in this module. (via cloudconf.py, cloud.setkey, or being on PiCloud server)
Note
This module cannot be used to access files stored on a job’s mounted file system
Transfers the file specified by file_path to PiCloud. The file can be retrieved later using the get function.
If name is specified, the file will be stored as name on PiCloud. Otherwise it will be stored as the basename of file_path.
Example:
cloud.files.put('data/names.txt')
This will transfer the file from the local path ‘data/names.txt’ to PiCloud and store it as ‘names.txt’. It can be later retrieved via cloud.files.get(‘names.txt’)
Upload file if it has changed.
cloud.files.put(source,name) only if contents of local file (specified by source) differ from those on PiCloud (specified by name or basename(source)) (or if file does not exist on PiCloud)
Similar to put. putf, however, accepts a file object (file, StringIO, etc.) f instead of a file_path.
Note
f is not rewound. f.read() from current position will be placed on PiCloud
Warning
If the file object does not correspond to an actual file on disk, it will be read entirely into memory before being transferred to PiCloud.
List all files stored on PiCloud.
Check if a file named name is stored on PiCloud.
Deletes the file named name from PiCloud.
Retrieves the file named by file_name from PiCloud and stores it to save_path.
Example:
cloud.files.get('names.txt','data/names.txt')
This will retrieve the ‘names.txt’ file on PiCloud and save it locally to ‘data/names.txt’.
If save_path is None, save_path will be file_name
An optional byte_range can be specified using start_byte, end_byte, where only the data between start_byte and end_byte is returned. If end_byte exceeds the size of the file, the contents from start_byte to end of file returned.
An end_byte of None or exceeding file size is interpreted as end of file
Retrieves the file named by file_name from PiCloud. Return value is a CloudFile (file-like object) that can be read() to retrieve the file’s contents
A range can be specified through start_byte and end_byte, where only the data between those two offsets will be accessable in the CloudFile. If start_byte is set, the returned CloudFile.tell() will be start_byte
An end_byte of None or exceeding file size is interpretted as end of file
Download file if it has changed.
cloud.files.get(name,destination) only if contents of local file (specified by destination or basename(file_name)) differ from those on PiCloud (specified by *name) (or destination does not exist locally)
Return the md5 checksum of the file named name stored on PiCloud
A CloudFile represents a file stored on PiCloud as a readonly file-like stream. Seeking is not available.
Close the file, blocking further I/O operations
The md5 checksum of file contents
read([size]) -> read at most size bytes, returned as a string.
If the size argument is negative or omitted, read until EOF is reached.
readline([size]) -> next line from the file, as a string.
Retain newline. A non-negative size argument limits the maximum number of bytes to return (an incomplete line may be returned then). Return an empty string at EOF.
readlines([size]) -> list of strings, each a line from the file.
Call readline() repeatedly and return a list of the lines so read. The optional size argument, if given, is an approximate bound on the total number of bytes in the lines returned.
Returns current file position as an integer
For managing crons on PiCloud. Crons allow you to “register” a function to be invoked periodically on PiCloud according to a schedule you specify.
Api keys must be configured before using any functions in this module. (via cloudconf.py, cloud.setkey, or being on PiCloud server)
Register func (a callable) to be run periodically on PiCloud according to schedule
The cron can be managed in the future by the specified label
Schedule is a BSD-style cron timestamp, specified as either a string of 5 non- whitespace characters delimited by spaces or as a sequence (tuple, list, etc.) of 5 elements, each with non-whitespace characters. The ordering of the characters represent scheduling information for minutes, hours, days, months, and days of week. See full format specification at http://unixhelp.ed.ac.uk/CGI/man-cgi?crontab+5 Year is not supported. The N/S increment format is though.
PiCloud schedules your crons under the GMT (UTC+0) timezone.
Certain special kwargs associated with cloud.call can be attached to the periodic jobs:
A string specifying a custom environment jobs should use See environments overview at http://blog.picloud.com/2011/09/26/introducing-environments-run-anything-on-picloud/
This keyword can be used to speed up serialization, at the cost of some functionality. This affects the serialization of the spawned jobs’ return value. The stored function will always be serialized by the enhanced serializer, with debugging features. Possible values keyword are:
Set this to True to enable profiling of your code. Profiling information is valuable for debugging, but may slow down your jobs.
In the very rare event of hardware failure, this flag indicates that a spawned job can be restarted if the failure happened in the middle of it. By default, this is true. This should be unset if the function has external state (e.g. it modifies a database entry)
Select the type of compute resources to use. PiCloud supports four types, specified as strings:
1 compute unit, 300 MB ram, low I/O (default)
2.5 compute units, 800 MB ram, medium I/O
3.25 compute units, 8 GB ram, high I/O
Up to 2 compute units (variable), 300 MB ram, low I/O, 1 IP per core
See http://www.picloud.com/pricing/ for pricing information
Note
The callable must take no arguments. To schedule functions that take arguments, wrap it with a closure. e.g. func(x,y) –> lambda: func(x,y)
Deregister (delete) the cron specified by label
Manually run the cron specified by label Returns the ‘jid’ of the job created by this run command
List labels of active crons
Return a dictionary of relevant info about cron specified by label
Info includes:
Interface to publish functions to PiCloud, allowing them to be invoked via the REST API
Api keys must be configured before using any functions in this module. (via cloudconf.py, cloud.setkey, or being on PiCloud server)
Publish func (a callable) to PiCloud so it can be invoked through the PiCloud REST API
The published function will be managed in the future by a unique (URL encoded) label.
out_encoding specifies the format that the return value should be in when retrieving the result via the REST API. Valid values are “json” for a JSON-encoded object and RAW, where the return value must be an str (but can contain any characters).
The return value is the URL which can be HTTP POSTed to to invoke func. See http://docs.picloud.com/rest.html for information about PiCloud’s REST API
Certain special kwargs associated with cloud.call can be attached to the periodic jobs:
This keyword can be used to speed up serialization, at the cost of some functionality. This affects the serialization of the spawned jobs’ return value. The stored function will always be serialized by the enhanced serializer, with debugging features. Possible values keyword are:
A string specifying a custom environment you wish to run your generated jobs within. See environments overview at http://blog.picloud.com/2011/09/26/introducing-environments-run-anything-on-picloud/
A positive integer denoting the job’s priority. PiCloud tries to run jobs with lower priority numbers before jobs with higher priority numbers.
Set this to True to enable profiling of your code. Profiling information is valuable for debugging, but may slow down your jobs.
In the very rare event of hardware failure, this flag indicates that a spawned job can be restarted if the failure happened in the middle of it. By default, this is true. This should be unset if the function has external state (e.g. it modifies a database entry)
Select the type of compute resources to use. PiCloud supports four types, specified as strings:
1 compute unit, 300 MB ram, low I/O (default)
2.5 compute units, 800 MB ram, medium I/O
3.25 compute units, 8 GB ram, high I/O
Up to 2 compute units (variable), 300 MB ram, low I/O, 1 IP per core
See http://www.picloud.com/pricing/ for pricing information
Remove a published function from PiCloud
List labels of published functions
Retrieve information about a published function specified by label
This object provides the ability to programmatically edit the cloud configuration (found in cloudconf.py). commit() must be called to update the cloud module with new settings - and restart all active clouds
your application’s api key provided by PiCloud
Amount (in kb) of job results to hold in local memory; set to 0 to disable caching. This option only applies when connected to PiCloud.
Number of job statuses to hold in local memory; set to 0 to disable caching. This option only applies when connected to PiCloud.
Update cloud with new settings.
Warning
This will restart any active cloud instances, wiping mp/simulated jobs and setkey information
If set and cloud is in simulation mode, forces redirect_job_output (see multiprocessing section) to true
If this is is set and cloud is running in simulation mode, serialize_logging (see logging section) will be turned on
Should every HTTP connection be closed after receiving a response?
Filename where cloud log messages should be written. This path is relative to ~/.picloud/
logging level for cloud messages. This affects both messages saved to the cloud log file and sent through the python logging system. See http://docs.python.org/library/logging.html for more information
Maximum amount of data (in bytes) that can be sent to the PiCloud server per function or function argument. Value may be raised up to 16 MB
Maxmimum number of jobs that cloudmp should store. Set to 0 for no limit. This option only applies to cloud.mp and the cloud simulator. PiCloud recommends that any limit stays above 1024.
Number of subprocesses that cloud multiprocessing should utilize. Beware of using too low of a number of subprocesses, as deadlock can occur. If set to 0, this will be set to the number of cores available on this system.
logging level for printing cloud log messages to console. Must be equal or higher than log_level
Specify an http/https proxy server that should be used e.g. proxy.example.com:3128 for anonymous or username:password@proxy.example.com:3128 for authenticated
Number of days to wait until purging old serialization log directories
If set to true, job stdout and stderr will be redirected to files inside the serialize_logging_path. This option simulates PiCloud behavior and must be set to true for cloud.info to be able to request stdout or stderr information about jobs. If set to false, job stdout and stderr will be the same as the parent process.
Should log_filename (default of cloud.log) be written with cloud log messages? Note that cloud will always emit logging messages; this option controls if cloud should have its own log.
Should all object serialization meta-data be logged to the serialize_logging_path?
Path to save object serialization meta-data. This path is relative to ~/.picloud/
url to list of PiCloud servers
Request gziped HTTP responses
The Cloud Simulator simulates cloud.* functions locally, allowing for quicker debugging. If this is enabled, all cloud.* functions will be run locally, rather than on PiCloud. See web documentation.
PiCloud account management. This module allows the user to manage account information programmatically. Currently allows managing real time core requests.
Note that your api_key must be set before using any functions.
Returns a list of all api keys. If active_only is True, only active keys are returned. username and password should be your PiCloud login information.
Returns information including api_secretkey, active status, and note for the specified api_key. username and password should be your PiCloud login information.
Activates the specified api_key. username and password should be your PiCloud login information.
Deactivates the specified api_key. username and password should be your PiCloud login information.
Deactivates the specified api_key. username and password should be your PiCloud login information.
A module provided that emulates python multiprocessing’s Process Pool interface. This can be used to trivially replace multiprocessing Pool code.
Note that cloud.setkey must still be used before using this interface.
Not supported (mostly due to not making sense in the PiCloud context).
Warning
It is highly recommended that you do not use this module and instead use the cloud module directly. Much functionality that the cloud interface offers is missing here.
Equivalent to Multiprocessing apply. keyword arguments are not supported
Equivalent to Multiprocessing apply_async keyword arguments are not supported
Equivalent to Multiprocessing map chunksize is not used here
Equivalent to Multiprocessing map_async chunksize is not used here
Equivalent to Multiprocessing imap chunksize is used only to control the cloud.iresult stage
Same as imap
Result object that emulates multiprocessing.pool.AsyncResult
Return result when it arrives. If timeout is not None and none arrives, raise multiprocessing.TimeoutError in timeout seconds
Returns true if the job finished (done or errored)
Returns true if job finished successfully. Asserts that job has finished
Wait until result is available or timeout seconds pass