Installation Using CloudFormation Template
As an advanced user, you can deploy Exasol using the AWS CloudFormation template. CloudFormation templates are JSON or YAML formatted files that specify the AWS resources that make up your stack. This section provides you with information about the CloudFormation templates available within Exasol and how to deploy Exasol on AWS using these templates.
Prerequisites
- Make sure you have an AWS account with the relevant permissions. If you do not have an AWS account, you can create one from the Amazon Console.
- In your AWS account choose the AWS Region where you want to deploy Exasol and create a Key Pair in the selected region.
- You must have subscribed to one of the following Exasol subscriptions on AWS Marketplace:
You must subscribe to Exasol from the AWS Marketplace. However, Exasol does not recommend you to continue with the configurations or launch Exasol from the AWS Marketplace. You can configure and deploy Exasol using the Exasol Cloud Deployment Wizard or using CloudFormation templates.
Sizing Considerations
Considerations | Information |
---|---|
RAM |
As a rule of thumb, the total amount of RAM of the cluster should be approximately 10% - 15 % of your raw data. However, based on your use case, you may need more RAM. |
Instance |
Exasol supports all EC2 instances with main memory more than 15GiB. Exasol recommends instance types of category r4, r5, m4, m5, or c5 with more than 30GiB. |
Storage |
The data storage volumes should be properly sized to store the maximum recommended amount of data for this instance type, data replicas in case of a cluster configuration as well as index structures and space for temporary data. The template configures the correct storage configuration for every instance type. |
Example
Let's assume the size of raw data 500GiB. Based on the RAM thumb rule, a good configuration for this data size would be:
- r5.2xlarge, 8 vcores, 64 GiB main memory
- m5.4xlarge, 16 vcores, 64 GiB main memory
The main difference between those instance type is the number of vcores. Because of its higher computing power, the m5.4xlarge type will be able to process more concurrent queries compared to the r5.2xlarge instance.
CloudFormation Templates (CFT)
Exasol provides you with the following ready to use CloudFormation templates in an Amazon S3 bucket. You can choose a template with the option to either create a new VPC or use an existing VPC.
Click the URL to download the template for further modification or you can copy this URL and paste it in the CloudFormation console to deploy Exasol. For detailed steps on how to use these templates, see the Deploy Exasol on AWS section.
Template URLs:
- Single Node, creating new VPC: https://s3.eu-central-1.amazonaws.com/cloudtools.exasol.com/amazon/exasol_single_node_amazon_marketplace_generate_new_VNET.json
- Single Node, using existing VPC: https://s3.eu-central-1.amazonaws.com/cloudtools.exasol.com/amazon/exasol_single_node_amazon_marketplace.json
- Cluster, using existing VPC: https://s3.eu-central-1.amazonaws.com/cloudtools.exasol.com/amazon/exasol_cluster_amazon_marketplace.json
- Cluster, creating new VPC: https://s3.eu-central-1.amazonaws.com/cloudtools.exasol.com/amazon/exasol_cluster_amazon_marketplace_generate_new_VNET.json
Deploy Exasol on AWS
Follow these steps to deploy Exasol using the CloudFormation template:
- Log in to AWS with your account and navigate to the CloudFormation page on the AWS Console.
- On the CloudFormation page, click Create Stack.
- On the Create Stack page, within the Specify template section, you have the following two options available
- Amazon S3 URL or Upload a template file.
- Amazon S3 URL – Specify the URL to a template in an S3 bucket.
- This option is selected by default. In the space provided for Amazon S3 URL, type or paste the URL you choose from the CloudFormation Templates (CFT) section. Click Next to continue.
- On the Specify stack details page, enter a name for the stack. Review the parameters for the template, add values for the parameters that require inputs such as Key Pair, SYS User Password, or ADMIN User Password.
- In the VPC/Network/Security section, the Public IPs is set to false by default. Set this value to true if you want to connect to your instance over the internet, or the enable communication between your instances and other AWS services that have public endpoints.
-
Select the AMI ID that matches your subscription model from the drop-down list. Review and/or change parameters in the Node Configuration section for a single node deployment, and Management Server Configurations and Data Node Configuration sections in a cluster deployment. Click Next to continue.
- On the Configure stack options page, you can optionally tag your stack using the Key and Value fields.
- (Optional) In the Permissions section, choose the IAM Role the CloudFormation uses to create, modify, or delete resources in the stack, and set the advanced options as needed.
- (Optional) License is applicable if your subscription model is Bring-your-own-license. Paste the entire content of the license file you have in the space provided.
- Click Next to continue.
- On the Review page, review the stack information, and in the Capabilities section, select the checkbox to acknowledge that the template will create IAM resources. Click Create stack to deploy the stack.
If you want to keep the Public IP address set to false, then you need to enable VPN or other methods to be able to access your instance. We advise you to contact your AWS administrator to obtain the recommended methods to connect to the database server or lauch the instance.
Refer to the Template Parameters section for information on the list of parameters and their descriptions.
- Upload a template file – Upload template from local storage.
- Select the Upload a template file option, and click Choose file. If you have downloaded the template from the CloudFormation Templates (CFT) section, navigate to the location of the template file. Select the template and click Next to continue.
- On the Specify stack details page, enter a name for the stack. Review the parameters for the template, add values for the parameters that require inputs such as Key Pair, SYS User Password, or ADMIN User Password.
- In the VPC/Network/Security section,the Public IPs is set to false by default. Set this value to true if you want to connect to your instance over the internet, or the enable communication between your instances and other AWS services that have public endpoints.
If you want to keep the Public IP address set to false, then you need to enable VPN or other methods to be able to access your instance. We advise you to contact your AWS administrator to obtain the recommended methods to connect to the database server or lauch the instance.
- Select the AMI ID that matches your subscription model from the drop-down list. Review and/or change parameters in the Node Configuration section for a single node deployment, and Management Server Configurations and Data Node Configuration sections in a cluster deployment. Click Next to continue.
Refer to the Template Parameters section for information on the list of parameters and their descriptions.
- On the Configure stack options page, you can optionally tag your stack using the Key and Value fields.
- (Optional) In the Permissions section, choose the IAM Role the CloudFormation uses to create, modify, or delete resources in the stack, and set the advanced options as needed.
- Click Next to continue.
- On the Review page, review the stack information, and in the Capabilities section, select the checkbox to acknowledge that the template will create IAM resources. Click Create stack to deploy the stack.
- Amazon S3 URL – Specify the URL to a template in an S3 bucket.
You can view the stack you created under AWS CloudFormation > Stacks, with the status CREATE_IN_PROGRESS. Once the stack is created successfully, the status is changed to CREATE_COMPLETE. Additionally, you can monitor the progress in the Events tab for the stack.
The deployment takes approximately 20 minutes to complete.
Determine the Public IP Address
You need the Public IP or DNS name displayed in the EC2 Console to connect to the database server, or launch the instance. To know the Public IP or DNS name:
If the Public IP parameter for your stack is set to false, then you will not see a Public IP address or DNS name displayed in the EC2 Console. In this case, you need to enable VPN or other methods to connect to the database server. We advise you to contact your AWS administrator to obtain the recommended method to connect to the database server or lauch the instance.
- Open the EC2 Dashboard from the AWS Management Console.
- Click on Running Instance. The Instances page is displayed with all the running instances.
- Select the name of the instance you created.
- In the Description section, the IP address displayed for Public DNS(IPv4) is the IP address of the database server.
Launch Instance
You can connect to your instance using the Public IP address or the DNS name diplayed in the EC2 Console. To know the Public IP or DNS name for your instance, see Determine the Public IP Address.
- Copy and paste this IP address prefixed with https in your browser. In the case of an Exasol cluster deployment, you need to the copy the IP address or DNS name of the management node.
- You will receive a digital certificate to confirm. Once you have confirmed this, the installation is initialized, and the following screen is displayed.
- Once installation is complete, you will be redirected to the EXAoperation screen.
It may take up to 45 minutes for the EXAoperation to be online after deployment.
- Login with the admin user name and password provided while creating your stack.
Connect to SQL Client
Once you have your instance running, you can connect to it using any SQL client of your choice.
Exasol Single Node Deployment: In the case of a single node deployment, use the Public IP address of your instance and the admin user name and password to connect to the SQL client.
Exasol Cluster Deployment: In the case of a cluster deployment, use the Public IP address of the data node along with the admin user name and password to connect to the SQL client. You can also connect to all the data nodes by entering the pubic IP address of all the nodes seperated by a comma.
To know the Public IP or DNS name for your instance, see Determine the Public IP Address.
Template Parameters
The following table lists all the parameters used in the templates (with a new VPC and with an existing VPC) and their descriptions.
Parameter | Description |
---|---|
Stack name |
Name of the AWS CloudFormation Stack. |
AWS Key Pair | Choose the key pair for SSH access of created instances. The user name for SSH access is ec2-user. |
Default Tags (only when deploying in an existing VPC) |
Details about the project for resource group arrangement and its owner. The parameters are applicable for an existing VPC only. |
Database Name | Name of the generated database. |
SYS user password | Password for the Exasol database administration user (SYS). |
ADMIN user Password | Password for the EXAoperation system administration user (ADMIN). |
VPC CIDR Block (only when deploying into a new VPC) | If a new VPC is created, a valid CIDR Block has to be specified. |
Database Subnet-Id (only when deploying into an existing VPC) | Choose a SubnetID from the drop-down menu. |
Subnet CIDR Block (only when deploying into a new VPC) | (Optional) Choose a valid CIDR Block if deployment into new VPC. |
Database Placement Group | An existing placement group can be chosen here, else a new one is created. |
Public IPs | If true, associate public IP addresses to all instances. |
S3 Bucket | If true, creates the backup on an S3 bucket. |
Create S3 endpoint (only when deploying into an existing VPC) | If you already have an end point for S3 or if the instances are public, select False. If set to True, it creates an endpoint for S3. |
Create KMS endpoint (only when deploying into an existing VPC) | If you already have an end point KMS or if the instances are public, select False. If set to True, it creates an endpoint for KMS. |
Create EC2 endpoint (only when deploying into an existing VPC) | If you already have an end point EC2 or if the instances are public, select False. If set to True, it creates an endpoint for EC2. |
DNS Server | DNS Server (default AWS DNS Server 169.254.169.253) |
NTP Server | IP of the default Amazon NTP server. |
System Timezone | Required timezone for the DB (default Europe/Berlin). |
Remote Access from Distinct IP | 0.0.0.0/0 if allow access from anywhere. |
Remote Access from defined Ports | Specifies the port with a comma separated values here. For example, if you want enable remote access from port 20,22,443,6583,8563, and 8835. Add 20,22,443,6583,8563,8835. |
License Server Instance Type (only when deploying into a new VPC) | EC2 instance type of the license/ management server. |
Management Server Instance Type (only when deploying into an existing VPC) | Specifies the instance type for the management server. |
AMI ID | Select corresponding AMI from the AWS Marketplace. |
Number of Data Nodes | Number of database nodes that store data and process queries (min:2, max:64). |
Data Node Instance Type and Storage | Instance type of data nodes. |
Number of Stand By Nodes |
Specifies the number of standby node(s) that you want to add for failover. The replication factor is automatically set to two if there is more than one node in the system. You can implement any of the following two standby modes:
To know more about how these standby modes work, see Fail Safety on Cloud Platform. |
Encrypt EBS Volumes | Enable encryption of block storage. |
Size in GB of Data Block Device Volumes (only when deploying into an existing VPC) | Size in GB per block device volume (for optimal performance >= 214GB). |
Number of Data Block Devices (only when deploying into an existing VPC) | Number of storage volumes for each node. |
License | In case of a Bring-your-own-license(BYOL), paste the entire content of the accquired license file in the space provided. . |
Next Steps
Connect to Exasol
After installing Exasol on AWS, you can do the following:
- Install drivers required to connect to other tools.
- Connect SQL clients to Exasol.
- Connect Business Intelligence tools (BI tools) to Exasol.
- Connect Data Integration - ETL tool to Exasol.
- Connect Data Warehouse Automation tools to Exasol.
Load Data
After you have connected your choice of tool to Exasol, you can load your data into Exasol and process further. To know more about loading data into Exasol, see Loading Data.
Administration
For the maintenance tasks related to Exasol on AWS, see AWS Administration.