THREDDS Primer¶
Last Updated: December 2019
In this tutorial you will be introduced to THREDDS using the Docker container that is included with Tethys Platform. This primer is adapted from the THREDDS Data Server Documentation. Topics covered include:
Thematic Real-Time Environmental Distributed Data Services (THREDDS)
THREDDS Data Server (TDS)
Open-source Project for a Network Data Access Protocol (OpeNDAP)
NetCDF Subset Serivce (NCSS)
Web Mapping Service (WMS)
Web Coverage Service (WCS)
TDS Content Directory
Catalog
Datasets
Services
Tethys Docker Containers
THREDDS Docker Image
1. Install TDS¶
The THREDDS Data Server (TDS) can be installed either manually (see: Getting Started with the TDS) or using the Docker image developed by Unidata (see THREDDS Docker image). For this tutorial, we recommend using the Docker image to install THREDDS. We have made it easier to install the THREDDS Docker image by adding it as an option to the Tethys Docker command.
Warning
Docker needs to be installed to use the tethys docker
command (see Install Docker). You will also need to add your user to the docker group and logout and log back in (see Linux Post Install).
Initialize the THREDDS Docker container, making sure to bind the data directory when prompted:
tethys docker init -c thredds
Note
The command tethys docker init
only needs to be run the first time you are creating a container. If it already exists, you can skip to the next step.
Start the THREDDS Docker container:
tethys docker start -c thredds
Tip
To stop the docker container:
tethys docker stop -c thredds
For more information about the Docker interface in Tethys Platform see the docker command reference.
Obtain the endpoint for the THREDDS Docker container:
tethys docker ip
Tip
Alternatively, you may install TDS manually using the Getting Started with the TDS guide. You may also use an existing TDS server, provided you have access to it's content directory.
2. Explore the Content Directory¶
Navigate to the TDS Content Directory and explore the contents. This directory stores all of the TDS configuration information and data. Review the TDS Content Directory tutorial for an explanation of each of the files and folders in the content directory.
Tip
When using the THREDDS docker container, the location of this directory is the folder that you specified during the bind step. The default bind location is ~/.tethys/thredds
.
3. Customize TDS¶
The TDS Configuration File (threddsConfig.xml
) is used to control the behavior of the TDS including metadata about the server and which services are enabled.
Open
threddsConfig.xml
in a text editor.Customize the following parameters in the serverInformation section:
name: Name of the TDS Server
logoUrl: Logo for the TDS Server
logoAltText: Text description of logo for screen readers
abstract: Description of the data hosted on the TDS Server
keywords: Keywords for the type of data hosted on the TDS Server
- contact
name: Name of TDS Server Maintainer
organization: Organization of TDS Server Maintainer
email: Email of TDS Server Maintainer
- hostInstitution
name: Institution/organization hosting the TDS Server
webSite: Website of institution/organization hosting the TDS Server
logoUrl: Logo of institution/organization hosting the TDS Server
logoAltText: Text description of logo for screen readers
Save your changes and restart the TDS:
tethys docker restart -c thredds
Tip
You can view the logs TDS server in the Docker container using the Docker CLI:
docker logs tethys_thredds
This is useful for debugging issues when the container won't start, such as malformed XML in the threddsConfig.xml
.
Navigate to the following locations to see how this metadata is incorporated into the TDS:
Tip
See Basic TDS Configuration for more information about configuring TDS.
The TDS Configuration File is also used to control which data services are enabled. Services that are supported by TDS include Web Coverage Service (WCS), Web Map Service (WMS), ncISO, and NetCDF Subset Service (NCSS). Most of these services are disabled by default in the docker version of TDS and need to be enabled. Review the following documentation about how to enable the various data services in TDS:
You can further customize the look and feel of your TDS using CSS. Navigate to the
htmlSetup
section of the TDS Configuration File (threddsConfig.xml
) and note the names of the css files for different pages (e.g.tds.css
,tdsCat.css
,tdsDap.css
). If you create these files in thepublic
directory, you'll be able to customize the CSS of the associated pages. See the Customizing the TDS Look and Feel documentation for more details.
4. Client/Configuration Catalogs¶
Review the following documentation on catalog configuration (ignore exercises):
Tip
It is easy to confuse the TDS Root Configuration Catalog (catalog.xml
) and the TDS Configuration File (threddsConfig.xml
). Remember:
TDS Root Configuration Catalog (
catalog.xml
): Used to define and configure datasets hosted by the TDS server.TDS Configuration File (
threddsConfig.xml
): Used to customize TDS server information and behaviour.
5. NetCDF Data Exercise¶
Download this
National Water Model Short Range Forecast
data and extract it. The archive should contain the first 3 of 18 NetCDF files each representing a 1-hour increment in an 18 hour forecast produced by the National Water Model. Each file contains over 2.7 million forecast points where each point is associated a different stream reach on the National Hydrogrophy Dataset.Create a new
nwm
directory in thepublic
directory of the TDS Content Directory (i.e.:<TDS_CONTENT_DIRECTORY>/public/nwm
).Copy the NetCDF files from step 1 into
<TDS_CONTENT_DIRECTORY>/public/nwm
.Create a new catalog coniguration file at the root of the TDS Content Directory called
nwmCatalog.xml
with the following contents:
<?xml version="1.0" encoding="UTF-8"?>
<catalog xmlns="http://www.unidata.ucar.edu/namespaces/thredds/InvCatalog/v1.0" xmlns:xlink="http://www.w3.org/1999/xlink"
name="Unidata THREDDS-IDD NetCDF-OpenDAP Server" version="1.0.1">
<service name="latest" serviceType="Resolver" base="" />
<service name="all" serviceType="Compound" base="">
<service name="ncdods" serviceType="OPENDAP" base="/thredds/dodsC/" />
<service name="HTTPServer" serviceType="HTTPServer" base="/thredds/fileServer/" />
</service>
<dataset name="National Water Model Data" collectionType="TimeSeries">
<metadata inherited="true">
<serviceName>all</serviceName>
<authority>edu.ucar.unidata</authority>
<dataType>Points</dataType>
<dataFormat>NetCDF</dataFormat>
<documentation type="rights">Freely available</documentation>
<documentation xlink:href="https://water.noaa.gov/about/nwm" xlink:title="National Water Model documentation"></documentation>
<creator>
<name vocabulary="DIF">DOC/NOAA/NWS/OWP</name>
<contact url="https://water.noaa.gov/" email="nws.nwc.ops@noaa.gov" />
</creator>
<timeCoverage>
<start>2020-01-06T00:00:00</start>
<duration>18 hours</duration>
</timeCoverage>
</metadata>
<datasetScan name="NWM Short Range Data" ID="nwm_short_range" path="nwm" location="content/nwm/" harvest="true">
<metadata inherited="true">
<documentation type="summary">National Water Model (NWM) - a hydrologic modelling framework that simulates observed and forecast streamflow over the entire continental United States.</documentation>
<geospatialCoverage>
<northsouth>
<start>24.637987</start>
<size>24.795109</size>
<units>degrees_north</units>
</northsouth>
<eastwest>
<start>-125.946552</start>
<size>60.346914</size>
<units>degrees_east</units>
</eastwest>
<updown>
<start>0.0</start>
<size>0.0</size>
<units>km</units>
</updown>
</geospatialCoverage>
</metadata>
<sort>
<lexigraphicByName increasing="true"/>
</sort>
</datasetScan>
</dataset>
</catalog>
Add a new catalog reference to the
nwmCatalog.xml
at the bottom of thecatalog
section ofcatalog.xml
:
<catalogRef xlink:title="National Water Model Catalog" xlink:href="nwmCatalog.xml" name=""/>
Restart the THREDDS server:
tethys docker restart -c thredds
Navigate to http://localhost:8383/thredds/catalog/nwm/catalog.html to verify that the new data is available.