Spatial Dataset Services API

Last Updated: December 2019

Important

This feature requires the tethys_dataset_services library to be installed. Starting with Tethys 5.0 or if you are using micro-tethys-platform, you will need to install tethys_dataset_services using conda or pip as follows:

# conda: conda-forge channel strongly recommended
conda install -c conda-forge tethys_dataset_services

# pip
pip install tethys_dataset_services

Spatial dataset services are web services that can be used to store and publish file-based spatial datasets (e.g.: Shapefile, GeoTiff, NetCDF). The spatial datasets published using spatial dataset services are made available in a variety of formats, many of which or more web friendly than the native format (e.g.: PNG, JPEG, GeoJSON, OGC Services).

One example of a spatial dataset service is GeoServer, which is capable of storing and serving vector and raster datasets in several popular formats including Shapefiles, GeoTiff, ArcGrid and others. GeoServer serves the data in a variety of formats via the Open Geospatial Consortium (OGC) standards including Web Feature Service (WFS), Web Map Service (WMS), and Web Coverage Service (WCS).

Another supported spatial dataset service is the THREDDS Data Server (TDS), which is a web server that specializes in serving gridded datasets using common protocols including OPeNDAP, OGC WMS, OGC WCS, and HTTP. Examples of data formats supported by THREDDS include NetCDF, HDF5, GRIB, and NEXRAD.

Tethys app developers can use this Spatial Dataset Services API to store and access :term:` spatial datasets` for use in their apps and publish any resulting datasets their apps may produce.

Spatial Dataset Engine References

The engines for some spatial dataset service engines in Tethys implement the SpatialDatasetEngine interface, which means they implement a common set of base methods for interacting with the service. The GeoServerSpatialDatasetEngine is an example of this pattern. Other engines are powered by excellent 3rd-party libraries, such as Siphon for THREDDS spatial dataset services. Refer to the following references for the APIs that are available for each spatial dataset service supported by Tethys.

Spatial Dataset Service Settings

Using dataset services in your app is accomplished by adding the spatial_dataset_service_settings() method to your app class, which is located in your app configuration file (app.py). This method should return a list or tuple of SpatialDatasetServiceSetting objects. For example:

from tethys_sdk.app_settings import SpatialDatasetServiceSetting

class App(TethysAppBase):
    """
    Tethys App Class for My First App.
    """
    ...
    def spatial_dataset_service_settings(self):
        """
        Example spatial_dataset_service_settings method.
        """
        sds_settings = (
            SpatialDatasetServiceSetting(
                name='primary_geoserver',
                description='GeoServer service for app to use.',
                engine=SpatialDatasetServiceSetting.GEOSERVER,
                required=True,
            ),
            SpatialDatasetServiceSetting(
                name='primary_thredds',
                description='THREDDS service for the app to use.',
                engine=SpatialDatasetServiceSetting.THREDDS,
                required=True
            ),
        )

        return sds_settings

Caution

The ellipsis in the code block above indicates code that is not shown for brevity. DO NOT COPY VERBATIM.

Assign Spatial Dataset Service

The SpatialDatasetServiceSetting can be thought of as a socket for a connection to a SpatialDatasetService. Before we can do anything with the SpatialDatasetServiceSetting we need to "plug in" or assign a SpatialDatasetService to the setting. The SpatialDatasetService contains the connection information and can be used by multiple apps. Assigning a SpatialDatasetService is done through the Admin Interface of Tethys Portal as follows:

Create SpatialDatasetService if one does not already exist
1. Access the Admin interface of Tethys Portal by clicking on the drop down menu next to your user name and selecting the "Site Admin" option.
2. Scroll down to the Tethys Services section of the Admin Interface and select the link titled Spatial Dataset Services.
3. Click on the Add Spatial Dataset Service button.
4. Fill in the connection information to the database server.
5. Press the Save button to save the new SpatialDatasetService.
Tip

You do not need to create a new SpatialDatasetService for each SpatialDatasetServiceSetting or each app. Apps and SpatialDatasetServiceSettings can share DatasetServices.
Navigate to App Settings Page
1. Return to the Home page of the Admin Interface using the Home link in the breadcrumbs or as you did in step 1a.
2. Scroll to the Tethys Apps section of the Admin Interface and select the Installed Apps link.
3. Select the link for your app from the list of installed apps.
Assign SpatialDatasetService to the appropriate SpatialDatasetServiceSetting
1. Scroll to the Spatial Dataset Services Settings section and locate the SpatialDatasetServiceSetting.
Note

If you don't see the SpatialDatasetServiceSetting in the list, uninstall the app and reinstall it again.
1. Assign the appropriate SpatialDatasetService to your SpatialDatasetServiceSetting using the drop down menu in the Spatial Dataset Service column.
2. Press the Save button at the bottom of the page to save your changes.

Note

During development you will assign the SpatialDatasetService setting yourself. However, when the app is installed in production, this steps is performed by the portal administrator upon installing your app, which may or may not be yourself.

Working with Spatial Dataset Services

After spatial dataset services have been properly configured, you can use the services to store, publish, and retrieve data for your apps. This process typically involves the following steps:

1. Get an Engine for the Spatial Dataset Service

Call the get_spatial_dataset_service() method of the app class to get the engine for the Spatial Dataset Service:

from .app import App

geoserver_engine = App.get_spatial_dataset_service('primary_geoserver', as_engine=True)

You can also create a SpatialDatasetEngine object directly. This can be useful if you want to vary the credentials for dataset access frequently (e.g.: using user specific credentials):

from tethys_dataset_services.engines import GeoServerSpatialDatasetEngine

spatial_dataset_engine = GeoServerSpatialDatasetEngine(endpoint='http://www.example.com/geoserver/rest', username='admin', password='geoserver')

Caution

Take care not to store API keys, usernames, or passwords in the source files of your app--especially if the source code is made public. This could compromise the security of your app and the spatial dataset service.

2. Use the Spatial Dataset Engine

After you have an engine object, simply call the desired methods on it. Consider the following example for uploading a shapefile to a GeoServer spatial dataset service:

from .app import App

# First get an engine
engine = App.get_spatial_dataset_service('primary_geoserver', as_engine=True)

# Create a workspace named after our app
engine.create_workspace(workspace_id='my_app', uri='http://www.example.com/apps/my-app')

# Path to shapefile base for foo.shp, side cars files (e.g.: .shx, .dbf) will be
# gathered in addition to the .shp file.
shapefile_base = '/path/to/foo'

# Notice the workspace in the store_id parameter
result = dataset_engine.create_shapefile_resource(store_id='my_app:foo', shapefile_base=shapefile_base)

# Check if it was successful
if not result['success']:
    raise

Note

The type of engine object returned and the methods available vary depending on the type of spatial dataset service.

HydroShare Dataset Engine Reference

SpatialDatasetEngine Interface