Create Remote Archive Volume with Amazon S3

You need a remote archive volume to store your backups. Exasol recommends using S3 bucket to store your backups for Exasol on AWS.

This section explains how you can use an S3 bucket to create a remote archive volume and store your backup.

Prerequisites

You need the following to configure the remote backup :

  • All nodes must be able to reach the remote target.

  • An existing S3 bucket. The URL must be in the following format:

    • http://bucketname.s3.amazonaws.com/optional-directory/

    • https://bucketname.s3.amazonaws.com/optional-directory/

    If you do not have a bucket, see How Do I Create an S3 Bucket?.

  • Read-write access to the S3 bucket. In case the nodes are on a private network, make sure that an S3 endpoint is configured for your VPC and that the route table for your subnet is updated accordingly to store backups in the bucket. For more information, see Endpoints for Amazon S3.
  • A secret access key for the S3 bucket. If you do not have a key, see Managing Access Keys for IAM Users.

An AWS bucket URL using the legacy global endpoint format (<bucketname>.s3.amazonaws.com) may need up to 24 hours after bucket creation before it becomes available. A fully qualified Amazon S3 URL that includes the AWS region (<bucketname>.s3-<region>.amazonaws.com) will work immediately.

Procedure

This procedure can be carried out using the Administration API.

The examples in this procedure are written using Bash on a Linux terminal. You can also use other interfaces and languages to execute the curl commands.

Placeholder values are styled as Bash variables, starting with the dollar sign ($) and using UPPERCASE characters. Replace the placeholders with your own values before executing the curl command.

To create a remote archive volume, send a PUT request to the /api/v1/databases/Exasol/volumes endpoint.

Specify the url to the bucket (url), the volume name (volumeName), and the volume type (volumeType) as a part of the request. 

curl -X 'POST' \
  'https://$EXASOL_IP/api/v1/databases/Exasol/volumes' \
  -H 'accept: application/json' \
  -H 'Authorization: Basic $TOKEN' \
  -H 'Content-Type: application/json' \
  -d '{
  "url": "$S3_BUCKET",
  "volumeName": "$VOLUME_NAME",
  "volumeType": "s3"
}'

Verification

To verify that the new remote archive volume was created, send a GET request to the /api/v1/databases/Exasol/volumes endpoint. 

curl -k -X 'GET' \
  'https://$EXASOL_IP/api/v1/databases/Exasol/volumes' \
  -H 'accept: application/json' \
  -H 'Authorization: Basic $TOKEN' 

The output should be similar to the following:

[
  {
    "id": "10003",
    "name": "$VOLUME_NAME",
    "type": "s3",
    "owner": [
      500,
      500
    ],
    "url": "$S3_BUCKET/$FOLDER"
  },
  {
    "id": "10002",
    "name": "default_backup_volume",
    "type": "s3",
    "owner": [
      500,
      500
    ],
    "url": "$S3_BUCKET/backup"
  },
  {
    "id": "10001",
    "name": "default_logrotation_volume",
    "type": "s3",
    "owner": [
      0,
      0
    ],
    "url": "$S3_BUCKET/logs"
  }
]