Last Updated: February 11, 2015
Distributed computing in Tethys Platform is made possible with HTCondor. Portal wide HTCondor computing resources are managed through the Tethys Compute Admin Pages. Accessing these resources in your app and configuring app specific resources is made possible through the Compute API.
HTCondor is a job and resources management middleware. It can be used to create High-Throughput Computing (HTC) systems from diverse computing units including desktop computers or cloud-computing resources. These HTC systems are known as HTCondor pools or clusters. In Tethys the Python library TethysCluster is used to automatically provision HTCondor clusters on Amazon Web Services (AWS) or Microsoft Azure. Portal-wide clusters can be configured by the Tethys Portal admin using the Tethys Compute Admin Pages, or app-specific clusters can be configured in apps using the ClusterManager. To run jobs to a clusters, it must have a
Scheduler configured. Portal-wide schedulers can also be configured by the Tethys Portal admin using the Tethys Compute Admin Pages, or app-specific schedulers can be set up through the Compute API.
To see how to configure a job with a
Scheduler see the Jobs API.
Working with the Cluster Manager¶
The cluster manager can be used to create new computing clusters. It is accessed through the
from tethys_sdk.compute import get_cluster_manager tethyscluster_config_file = '/path/to/TethysCluster/config/file' cluster_manager = get_cluster_manager(tethyscluster_config_file)
For more information on how to use the cluster manager see the TethysCompute documentation
Working with Schedulers¶
Portal-wide schedulers can be accessed through the
list_schedulers and the
from tethys_sdk.compute import list_schedulers, get_scheduler scheduler = list_schedulers() # this assumes the Tethys Portal administrator has created a scheduler named 'Default'. scheduler = get_scheduler('Default')
App-specific schedulers can be created with the
from tethys_sdk.compute import create_scheduler scheduler = create_scheduler(name='my_app_scheduler', host='example.com', username='root', private_key_path='/path/to/private/key')
Gets a list of all scheduler objects registered in the Tethys Portal
List of Schedulers
Gets the scheduler associated with the given name
name (str) -- The name of the scheduler to return
The scheduler with the given name or None if no scheduler has the name given.
create_scheduler(name, host, username=None, password=None, private_key_path=None, private_key_pass=None)¶
Creates a new scheduler
name (str) -- The name of the scheduler
host (str) -- The hostname or IP address of the scheduler
username (str, optional) -- The username to use when connecting to the scheduler
password (str, optional) -- The password for the username
private_key_path (str, optional) -- The path to the location of the SSH private key file
private_key_pass (str, optional) -- The passphrase for the private key
The newly created scheduler
The newly created scheduler object is not committed to the database.