Add Additional Storage (Optional)

Last Updated: November 2021

The storage allocated to your VM by default will likely not be sufficient for data-intensive apps. This tutorial will guide you through the process of attaching additional storage to the VM and configuring Tethys Platform to use it.

1. Attach Additional Storage

Add additional storage to the VM as follows:

  1. Navigate to the Overview page for the VM Resource.

  2. From the navigation menu on the left, select Disks.

  3. Press the Create and attach a new disk button under the Data disks section.

  4. Fill out the options in the new row that appears as follows:

    • LUN: Logical Unit Number, a unique ID for the disk.

    • Disk name: Give the disk a name (e.g. my-first-tethys-storage-0).

    • Storage type: Choose the storage type. Standard SSD is ok for most Tethys Servers.

    • Size: Specify the size in GibiBytes (see: Gibibyte | Wikipedia).

    • Encryption: Choose the encryption method for the disk. See the info note in the column header for more details.

    • Host caching: Choose host caching option (not required, but may boost performance).

  5. Press the Save button at the top of the page when done.

2. Format Disk

After attaching the storage to the virtual machine, you will need to format it and add a file system to it:

  1. Login to the VM using SSH (see: Orientation to Azure VM | Connect with SSH).

  2. Run lsblk to list all storage devices:

    default
    lsblk

    This will result in a printout of the storage devices like this:

    default
    NAME    MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
loop0     7:0    0 61.8M  1 loop /snap/core20/1081
loop1     7:1    0 61.9M  1 loop /snap/core20/1169
loop2     7:2    0 32.4M  1 loop /snap/snapd/13270
loop3     7:3    0 67.2M  1 loop /snap/lxd/21803
loop4     7:4    0 67.2M  1 loop /snap/lxd/21835
loop5     7:5    0 32.5M  1 loop /snap/snapd/13640
sda       8:0    0   30G  0 disk
├─sda1    8:1    0 29.9G  0 part /
├─sda14   8:14   0    4M  0 part
└─sda15   8:15   0  106M  0 part /boot/efi
sdb       8:16   0   20G  0 disk
└─sdb1    8:17   0   20G  0 part /mnt
sdc       8:32   0  100G  0 disk

    The disk with the operating system installed on it is the one with the root directory (/) as the mountpoint (sda1 in this case). The name of the new disk that we attached to the VM is sdc, which was determined by the size and the fact that it has no mount point.

  3. Use fdisk to create a partition on the disk as follows:

    1. Run the fdisk with the path to the new disk.

      default
      sudo fdisk /dev/<new_disk_name>

      For example, the new disk in the example above is named sdc so the command would be as follows:

      default
      # Heads up! This is an example!
sudo fdisk /dev/sdc

    2. Enter g to create a new partition table:

      default
      Command (m for help): g
Created a new GPT disklabel (GUID: 83C2F8FE-4D08-A24B-B18D-B39F90DC9ED1).

    3. Enter n to create a new partition and accept the default values for the three prompts that follow (i.e. leave blank):

      default
      Command (m for help): n
Partition number (1-128, default 1):
First sector (2048-209715166, default 2048):
Last sector, +/-sectors or +/-size{K,M,G,T,P} (2048-209715166, default 209715166):

Created a new partition 1 of type 'Linux filesystem' and of size 100 GiB.

    4. Enter w to apply the changes you specified above. Alternatively, if you need to start over, enter q to exit without applying the changes.

      default
      Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

    5. Run lsblk again to confirm the partition was created and that it is the full size of the disk. In the case below, the partition sdc1 is now listed under sdc:

      default
      NAME    MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
loop0     7:0    0 61.8M  1 loop /snap/core20/1081
loop1     7:1    0 61.9M  1 loop /snap/core20/1169
loop2     7:2    0 32.4M  1 loop /snap/snapd/13270
loop3     7:3    0 67.2M  1 loop /snap/lxd/21803
loop4     7:4    0 67.2M  1 loop /snap/lxd/21835
loop5     7:5    0 32.5M  1 loop /snap/snapd/13640
sda       8:0    0   30G  0 disk
├─sda1    8:1    0 29.9G  0 part /
├─sda14   8:14   0    4M  0 part
└─sda15   8:15   0  106M  0 part /boot/efi
sdb       8:16   0   20G  0 disk
└─sdb1    8:17   0   20G  0 part /mnt
sdc       8:32   0  100G  0 disk
└─sdc1    8:33   0  100G  0 part

    6. Create a file system on the new partition using the mkfs command:

      default
      sudo mkfs -t ext4 -j -L <volume_label> /dev/<new_partition_name>

      For example, the new partition in the example above is named sdc1 so the command would be:

      default
      # Heads up! This is an example!
sudo mkfs -t ext4 -j -L mydisk /dev/sdc1

      The -L option lets you specify a label for the file system that will be used in the next step. The maximum length of the label is 16 ASCII characters. ext4 is the type of filesystem that is created on the disk and is the standard file system for Linux.

3. Mount Disk

Now that the disk has been formatted with a Linux filesystem, it can be mounted as follows:

  1. Create a directory where you intend the new disk to be mounted.

    default
    sudo mkdir <mount_directory>

    Continuing our example, this would be:

    default
    # Heads up! This is an example!
sudo mkdir /mydisk

  2. To manually mount the disk, use the mount command, specifying the new partition, and the mount directory (mountpoint).

    default
    sudo mount /dev/<partition_name> <mount_directory>

    where <partition_name> is the name of the new partition and <mount_directory> is the directory that you want to mount it to. In the example this would be:

    default
    # Heads up! This is an example!
sudo mount /dev/sdc1 /mydisk

  3. Run lsblk one more time to verify that the new partition has a mountpoint now. In the example above, sdc1 now lists /mydisk as the mountpoint:

    default
    NAME    MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
loop0     7:0    0 61.8M  1 loop /snap/core20/1081
loop1     7:1    0 61.9M  1 loop /snap/core20/1169
loop2     7:2    0 32.4M  1 loop /snap/snapd/13270
loop3     7:3    0 67.2M  1 loop /snap/lxd/21803
loop4     7:4    0 67.2M  1 loop /snap/lxd/21835
loop5     7:5    0 32.5M  1 loop /snap/snapd/13640
sda       8:0    0   30G  0 disk
├─sda1    8:1    0 29.9G  0 part /
├─sda14   8:14   0    4M  0 part
└─sda15   8:15   0  106M  0 part /boot/efi
sdb       8:16   0   20G  0 disk
└─sdb1    8:17   0   20G  0 part /mnt
sdc       8:32   0  100G  0 disk
└─sdc1    8:33   0  100G  0 part /mydisk

  4. You can now change into the mountpoint directory like any other directory. Any files written within that directory will be written to the new disk. For example:

    default
    cd /mydisk
ls -al
total 24
drwxr-xr-x 3 root root  4096 Nov 15 20:26 .
drwxr-xr-x 4 root root  4096 Nov 15 20:42 ..
drwx------ 2 root root 16384 Nov 15 20:26 lost+found

  5. Everything is owned by root at the moment. Change ownership of the <mount_directory> to be owned by your user account:

    default
    sudo chown -R ${USER} <mount_directory>

    For the example this would be:

    default
    # Heads up! This is example code
sudo chown -R ${USER} /mydisk

4. Configure Automount

This step will describe how to configure the VM to automatically mount the new disk if the server is restarted:

  1. Open /etc/fstab in your favorite text editor (e.g. vim, nano):

    default
    sudo vim /etc/fstab

  2. Add a new line to the file as follows:

    default
    LABEL=<volume_label> <mount_directory> ext4 defaults,rw 0 2

    where <volume_label> is the label you assigned when creating the file system and <mount_directory> is the directory that you mounted it to.

    For example:

    default
    LABEL=mydisk /mydisk ext4 defaults,rw 0 2

    Important

    Each item in the fstab file must be separated by exactly one space or tab character. The options (i.e.: defaults,rw) should be separated by commas and no spaces. For more details on the fstab file and the options see: Fstab.

  3. Exit SSH

    default
    exit

  4. Restart the VM using the control on the Overview page for the Virtual Machine resource, then login again after the machine starts.

  5. Use the df command to view disk space usage and confirm that the new disk was automatically mounted:

    default
    df -h

    which should yield something like this:

    default
    Filesystem      Size  Used Avail Use% Mounted on
/dev/root        29G   13G   17G  44% /
devtmpfs        2.0G     0  2.0G   0% /dev
tmpfs           2.0G   44K  2.0G   1% /dev/shm
tmpfs           394M  1.2M  393M   1% /run
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           2.0G     0  2.0G   0% /sys/fs/cgroup
/dev/loop0       62M   62M     0 100% /snap/core20/1169
/dev/loop2       68M   68M     0 100% /snap/lxd/21803
/dev/loop1       62M   62M     0 100% /snap/core20/1081
/dev/loop3       68M   68M     0 100% /snap/lxd/21835
/dev/loop5       33M   33M     0 100% /snap/snapd/13640
/dev/loop4       33M   33M     0 100% /snap/snapd/13270
/dev/sda15      105M  5.2M  100M   5% /boot/efi
/dev/sdc1        98G   61M   93G   1% /mydisk
/dev/sdb1        20G   45M   19G   1% /mnt
tmpfs           394M     0  394M   0% /run/user/1002

5. Configure Tethys

Now that the new disk has been formatted and will automatically mount when the system boots, Tethys needs to be configured to use it. This includes updating the Tethys config to use the new storage for app workspaces and configuring GeoServer, THREDDS, and any other data services to use a data directory on the drive.

Tethys Workspaces

THREDDS

GeoServer