Deployment Using CloudFormation Template

As an advanced user, you can deploy Exasol using the AWS CloudFormation template. CloudFormation templates are JSON or YAML formatted files that specify the AWS resources that make up your stack. This section provides you with information about the CloudFormation templates available within Exasol and how to deploy Exasol on AWS using these templates.

Prerequisites

Make sure you have an AWS account with the relevant permissions. If you do not have an AWS account, you can create one from the Amazon Console.
In your AWS account choose the AWS Region where you want to deploy Exasol and create a Key Pair in the selected region.

You must have subscribed to one of the following Exasol subscriptions on AWS Marketplace:
- Exasol Analytic Database (Single Node / Cluster, Bring-Your-Own-License)
- Exasol Analytic Database (Single Node / Cluster, Pay-As-You-Go)
Do not continue to configure or launch Exasol from the AWS Marketplace, as this may make your deployment invalid. Only use the Exasol Cloud Deployment Wizard or CloudFormation templates to deploy Exasol on AWS.

Sizing considerations

Considerations	Information
RAM	As a rule of thumb, the total amount of RAM of the cluster should be approximately 10% - 15% of your raw data. However, based on your use case, you may need more RAM.
Instance	Exasol supports all EC2 instances with main memory more than 15GiB. Exasol recommends instance types of category r4, r5, m4, m5, or c5 with more than 30GiB.
Storage	The data storage volumes should be properly sized to store the maximum recommended amount of data for this instance type, data replicas in case of a cluster configuration as well as index structures and space for temporary data. The template configures the correct storage configuration for every instance type.

Considerations

Information

RAM

As a rule of thumb, the total amount of RAM of the cluster should be approximately 10% - 15% of your raw data. However, based on your use case, you may need more RAM.

Instance

Exasol supports all EC2 instances with main memory more than 15GiB. Exasol recommends instance types of category r4, r5, m4, m5, or c5 with more than 30GiB.

Storage

The data storage volumes should be properly sized to store the maximum recommended amount of data for this instance type, data replicas in case of a cluster configuration as well as index structures and space for temporary data.

The template configures the correct storage configuration for every instance type.

Example

Let’s assume the size of raw data is 500 GiB. Based on the RAM rule-of-thumb, a good configuration for this data size would be:

r5.2xlarge, 8 vcores, 64 GiB main memory
m5.4xlarge, 16 vcores, 64 GiB main memory

The main difference between those instance type is the number of vcores. Because of its higher computing power, the m5.4xlarge type will be able to process more concurrent queries compared to the r5.2xlarge instance.

CloudFormation templates (CFT)

Exasol provides you with the following ready to use CloudFormation templates in an Amazon S3 bucket. You can choose a template with the option to either create a new VPC or use an existing VPC.

Click the URL to download the template for further modification or you can copy this URL and paste it in the CloudFormation console to deploy Exasol. For detailed steps on how to use these templates, see the Deploy Exasol on AWS section.

Template URLs:

Single Node, creating new VPC:

https://s3.eu-central-1.amazonaws.com/cloudtools.exasol.com/amazon/exasol_single_node_amazon_marketplace_generate_new_VNET.json

Single Node, using existing VPC:

https://s3.eu-central-1.amazonaws.com/cloudtools.exasol.com/amazon/exasol_single_node_amazon_marketplace.json

Cluster, using existing VPC:

https://s3.eu-central-1.amazonaws.com/cloudtools.exasol.com/amazon/exasol_cluster_amazon_marketplace.json

Cluster, creating new VPC:

https://s3.eu-central-1.amazonaws.com/cloudtools.exasol.com/amazon/exasol_cluster_amazon_marketplace_generate_new_VNET.json

Deploy Exasol on AWS

Follow these steps to deploy Exasol using the CloudFormation template:

Log in to AWS with your account and navigate to the CloudFormation page on the AWS Console.
On the CloudFormation page, click Create Stack.
On the Create Stack page, within the Specify template section, you have the following two options available - Amazon S3 URL or Upload a template file.
1. Amazon S3 URL – Specify the URL to a template in an S3 bucket.
  1. This option is selected by default. In the space provided for Amazon S3 URL, type or paste the URL you choose from the CloudFormation templates (CFT) section. Click Next to continue.
  2. On the Specify stack details page, enter a name for the stack. Review the parameters for the template, add values for the parameters that require inputs such as Key Pair, SYS User Password, or ADMIN User Password.
  3. In the VPC/Network/Security section, the Public IPs is set to false by default. Set this value to true if you want to connect to your instance over the internet, or the enable communication between your instances and other AWS services that have public endpoints.
  4. Select the AMI ID that matches your subscription model from the drop-down list. Review and/or change parameters in the Node Configuration section for a single node deployment, and Management Server Configurations and Data Node Configuration sections in a cluster deployment. Click Next to continue.
  5. On the Configure stack options page, you can optionally tag your stack using the Key and Value fields.
  6. (Optional) In the Permissions section, choose the IAM Role the CloudFormation uses to create, modify, or delete resources in the stack, and set the advanced options as needed.
  7. (Optional) License is applicable if your subscription model is Bring-your-own-license. Paste the entire content of the license file you have in the space provided.
  8. Click Next to continue.
  9. On the Review page, review the stack information, and in the Capabilities section, select the checkbox to acknowledge that the template will create IAM resources. Click Create stack to deploy the stack.
2. Upload a template file – Upload template from local storage.
  1. Select the Upload a template file option, and click Choose file. If you have downloaded the template from the CloudFormation templates (CFT) section, navigate to the location of the template file. Select the template and click Next to continue.
  2. On the Specify stack details page, enter a name for the stack. Review the parameters for the template, add values for the parameters that require inputs such as Key Pair, SYS User Password, or ADMIN User Password.
  3. In the VPC/Network/Security section,the Public IPs is set to false by default. Set this value to true if you want to connect to your instance over the internet, or the enable communication between your instances and other AWS services that have public endpoints.
  4. If you want to keep the Public IP address set to false, then you need to enable VPN or other methods to be able to access your instance. We advise you to contact your AWS administrator to obtain the recommended methods to connect to the database server or lauch the instance.
  5. Select the AMI ID that matches your subscription model from the drop-down list. Review and/or change parameters in the Node Configuration section for a single node deployment, and Management Server Configurations and Data Node Configuration sections in a cluster deployment. Click Next to continue.
  6. Refer to the Template parameters section for information on the list of parameters and their descriptions.
  7. On the Configure stack options page, you can optionally tag your stack using the Key and Value fields.
  8. (Optional) In the Permissions section, choose the IAM Role the CloudFormation uses to create, modify, or delete resources in the stack, and set the advanced options as needed.
  9. Click Next to continue.
  10. On the Review page, review the stack information, and in the Capabilities section, select the checkbox to acknowledge that the template will create IAM resources. Click Create stack to deploy the stack.

You can view the stack you created under AWS CloudFormation > Stacks, with the status CREATE_IN_PROGRESS. Once the stack is created successfully, the status is changed to CREATE_COMPLETE. Additionally, you can monitor the progress in the Events tab for the stack.

The deployment takes approximately 20 minutes to complete.

Determine the public IP address

You need the Public IP or DNS name displayed in the EC2 Console to connect to the database server, or launch the instance. To know the Public IP or DNS name:

If the Public IP parameter for your stack is set to false, then you will not see a Public IP address or DNS name displayed in the EC2 Console. In this case, you need to enable VPN or other methods to connect to the database server. We advise you to contact your AWS administrator to obtain the recommended method to connect to the database server or lauch the instance.

Open the EC2 Dashboard from the AWS Management Console.
Click on Running Instance. The Instances page is displayed with all the running instances.
Select the name of the instance you created.
In the Description section, the IP address displayed for Public DNS(IPv4) is the IP address of the database server.

Launch instance

You can connect to your instance using the Public IP address or the DNS name diplayed in the EC2 Console. To know the Public IP or DNS name for your instance, see Determine the public IP address.

Copy and paste this IP address prefixed with https in your browser. In the case of an Exasol cluster deployment, you need to the copy the IP address or DNS name of the management node.
You will receive a digital certificate to confirm. Once you have confirmed this, the installation is initialized, and the following screen is displayed.
Once installation is complete, you will be redirected to the EXAoperation screen.
It may take up to 45 minutes for the EXAoperation to be online after deployment.
Login with the admin user name and password provided while creating your stack.

Connect SQL client

Once you have your instance running, you can connect to it using any SQL client of your choice.

Exasol Single Node Deployment: In the case of a single node deployment, use the Public IP address of your instance and the admin user name and password to connect to the SQL client.

Exasol Cluster Deployment: In the case of a cluster deployment, use the Public IP address of the data node along with the admin user name and password to connect to the SQL client. You can also connect to all the data nodes by entering the pubic IP address of all the nodes seperated by a comma.

To know the Public IP or DNS name for your instance, see Determine the public IP address.

Template parameters

The following table lists all the parameters used in the templates (with a new VPC and with an existing VPC) and their descriptions.

Parameter	Existing VPC	New VPC	Description
Stack name	Yes	Yes	Name of the AWS CloudFormation Stack.
AWS Key Pair	Yes	Yes	Choose the key pair for SSH access of created instances. The user name for SSH access is `ec2-user`.
Default Tags	Yes	Yes	Details about the project for resource group arrangement and its owner. The parameters are applicable for an existing VPC only.
Database Name	Yes	Yes	Name of the generated database.
SYS user password	Yes	Yes	Password for the Exasol database administration user (`SYS`).
ADMIN user Password	Yes	Yes	Password for the EXAoperation system administration user (`ADMIN`).
VPC CIDR Block	No	Yes	If a new VPC is created, a valid CIDR Block has to be specified.
Database Subnet-Id	No	Yes	Choose a SubnetID from the drop-down menu.
Subnet CIDR Block	No	Yes	(Optional) Choose a valid CIDR Block if deployment into new VPC.
Database Placement Group	Yes	Yes	An existing placement group can be chosen here, else a new one is created.
Public IPs	Yes	Yes	If true, associate public IP addresses to all instances.
S3 Bucket	Yes	Yes	If true, creates the backup on an S3 bucket.
Create S3 endpoint	No	Yes	If you already have an endpoint for S3 or if the instances are public, select False. If set to True, it creates an endpoint for S3.
Create KMS endpoint	No	Yes	If you already have an endpoint KMS or if the instances are public, select False. If set to True, it creates an endpoint for KMS.
Create EC2 endpoint	No	Yes	If you already have an endpoint EC2 or if the instances are public, select False. If set to True, it creates an endpoint for EC2.
DNS Server	Yes	Yes	DNS Server (default AWS DNS Server 169.254.169.253)
NTP Server	Yes	Yes	IP of the default Amazon NTP server.
System Timezone	Yes	Yes	Required timezone for the DB (default Europe/Berlin).
Remote Access from Distinct IP	Yes	Yes	0.0.0.0/0 if allow access from anywhere.
Remote Access from defined Ports	Yes	Yes	Specifies the port with a comma separated values here. For example, if you want enable remote access from port 20,22,443,6583,8563, and 8835. Add `20,22,443,6583,8563,8835`.
License Server Instance Type	Yes	Yes	EC2 instance type of the license/ management server.
Management Server Instance Type	Yes	Yes	Specifies the instance type for the management server.
AMI ID	Yes	Yes	Select corresponding AMI from the AWS Marketplace.
Number of Data Nodes	Yes	Yes	Number of database nodes that store data and process queries (min:2, max:64).
Data Node Instance Type and Storage	Yes	Yes	Instance type of data nodes.
Number of Stand By Nodes	Yes	Yes	Specifies the number of standby node(s) that you want to add for failover. The replication factor is automatically set to two if there is more than one node in the system. You can implement any of the following two standby modes: Hot Standby: The standby node is always in active mode. It's a faster failover mechanism but it's more expensive. This is enabled by default when you add a standby node. Cold Standby: The standby node is in suspended mode. You need to shut down the stand by node to enable this option. It is a relatively slower failover mechanism but it's more cost-effective. In case of a failover, the node starts and it takes over for the failed node. To learn more about how these standby modes work, see Fail Safety (Cloud).
Encrypt EBS Volumes	Yes	Yes	Enable encryption of block storage.
Size in GB of Data Block Device Volumes	Yes	Yes	Size in GB per block device volume (for optimal performance >= 214GB).
Number of Data Block Devices	Yes	Yes	Number of storage volumes for each node.

Next steps

Connect to Exasol

After installing Exasol on AWS, you can do the following:

Load data

After you have connected your choice of tool to Exasol, you can load your data into Exasol and process further. To learn more about loading data into Exasol, see Loading Data.

Administration

For the maintenance tasks related to Exasol on AWS, see Administration (AWS).