Create Remote Archive Volume

This section explains how to create a remote archive volume for database backups.

Backups are stored on archive volumes in a compressed format. With on-premises installations of Exasol you can create archive volumes either locally in the cluster or on a remote location. For cloud deployments, only remote archive volumes are supported.

The following examples describe how to create a remote archive volume in an Amazon S3 bucket. Exasol recommends using Amazon S3 buckets to store backups for cloud deployments on AWS and when installing Exasol as an application on AWS instances.

This procedure can be carried out using either the Administration API or ConfD.

Prerequisites

General prerequisites

  • All nodes must be able to reach the remote target.
  • The user must have read-write access on the remote host

Amazon S3 specific prerequisites

  • An existing S3 bucket. If you do not have a bucket, see How Do I Create an S3 Bucket?.

    The bucket URL must be in the following format:

    http[s]://<bucketname>.s3[-<region>].amazonaws.com/[<optional-directory>/]

    An Amazon S3 bucket URL using the legacy global endpoint format (<bucket-name>.s3.amazonaws.com) may need up to 24 hours after bucket creation before it becomes available.

    A fully qualified Amazon S3 bucket URL that includes the AWS region (<bucket-name>.s3.<region-code>.amazonaws.com) will become available immediately.

  • Read-write access to the S3 bucket. If the nodes are on a private network, make sure that an S3 endpoint is configured for your VPC and that the route table for your subnet is updated accordingly to store backups in the bucket. For more information, see Endpoints for Amazon S3.
  • A secret access key for the S3 bucket. If you do not have a key, see Managing Access Keys for IAM Users.

Procedure - Administration API

The following examples use curl on a Linux terminal to send REST calls to endpoints in the Administration API. You can also use other interfaces and languages to interact with the API. For more information, see Administration API.

Placeholder values are styled as Bash variables, for example: $EXASOL_IP. Replace the placeholders with your own values.

To create a remote archive volume, send a PUT request to the /api/v1/databases/Exasol/volumes endpoint.

Specify the url to the bucket (url), the volume name (volumeName), and the volume type (volumeType) as a part of the request.

curl -X 'POST' \
  'https://$EXASOL_IP/api/v1/databases/Exasol/volumes' \
  -H 'accept: application/json' \
  -H 'Authorization: Basic $TOKEN' \
  -H 'Content-Type: application/json' \
  -d '{
  "url": "$S3_BUCKET",
  "volumeName": "$VOLUME_NAME",
  "volumeType": "s3"
}'

To verify that the new remote archive volume was created, send a GET request to the /api/v1/databases/Exasol/volumes endpoint.

curl -k -X 'GET' \
  'https://$EXASOL_IP/api/v1/databases/Exasol/volumes' \
  -H 'accept: application/json' \
  -H 'Authorization: Basic $TOKEN' 

The output should be similar to the following:

[
  {
    "id": "10003",
    "name": "$VOLUME_NAME",
    "type": "s3",
    "owner": [
      500,
      500
    ],
    "url": "$S3_BUCKET/$FOLDER"
  },
  {
    "id": "10002",
    "name": "default_backup_volume",
    "type": "s3",
    "owner": [
      500,
      500
    ],
    "url": "$S3_BUCKET/backup"
  },
  {
    "id": "10001",
    "name": "default_logrotation_volume",
    "type": "s3",
    "owner": [
      0,
      0
    ],
    "url": "$S3_BUCKET/logs"
  }
]

Procedure - ConfD

The following examples use the command-line tool confd_client in a Linux terminal connected to a database node. For more information about how to use this tool, see ConfD.

Placeholder values are indicated with UPPERCASE characters. Replace the placeholders with your own values.

  1. To find the name of the database, use the ConfD job db_list. For example:

    confd_client -c db_list
    - Exasol
  2. The remote archive volume must be created with the same owner as the database that will write backups to the volume. To find the owner, use the ConfD job db_info. For example:

    confd_client -c db_info -a 'db_name: Exasol' | grep owner -A 2
    owner:
    - 500
    - 500
  3. To create a remote archive volume, use the ConfD job remote_volume_add.

    confd_client -c remote_volume_add -a '{url: ARCHIVE_VOLUME_URL, vol_type: TYPE, username: VOLUME_USERNAME, password: VOLUME_PW, owner: OWNER_TUPLE}'

    For example, to create a remote archive volume on an S3 bucket:

    confd_client -c remote_volume_add -a '{url: https://my_bucket.s3.eu-west-1.amazonaws.com, vol_type: s3, username: backup_user, password: 123456789, owner: [500,500]}'
  4. To verify that the volume was created, use the ConfD job remote_volume_list. For example:

    confd_client -c remote_volume_list
    - r0001
  5. To check the properties of the volume, use the ConfD job remote_volume_info and insert the name of the volume. For example:

    confd_client -c remote_volume_info -a 'remote_volume_name: r0001'