Add and Activate Data Nodes
This article explains how to add data nodes to a cluster in an on-premises installation.
This procedure is carried out using Exasol Deployment Tool (c4) and ConfD.
Prerequisites
The parameter CCC_HOST_CLEANUP must be set to false in the configuration that was used for the deployment.
Setting the config path
When you make changes to an existing deployment you must run use the same configuration that was used when creating the deployment. Unless that configuration is in the current working directory (on the host where you run c4), you must prepend the c4 commands with the path to the configuration using CCC_CONFIG=<path-to-config>. For example:
In the following examples, the default configuration file config is used.
Procedure
Step 1: Prepare the hosts
Prepare the new host machines as needed depending on your existing deployment environment. For information about system requirements, see System Requirements.
All nodes in the cluster should be assigned consecutive and evenly spaced static IPv4 addresses in the same subnet.
Step 2: Reserve IP addresses
On a jump host that has SSH access to the cluster nodes, run the c4 command CCC_CONFIG=config c4 host reserve <PLAY_ID> with the following command-line parameters:
| Parameter | Description |
|---|---|
|
|
Private IP address of a host that you want to add to the cluster. You must repeat the parameter for each additional host, separated by a space: |
--ccc-host-reserved-external-addrs
|
Public IP address of a host that you want to add to the cluster. You must repeat the parameter for each additional host, separated by a a space: This parameter is only required if the existing data nodes were deployed with public IP addresses ( |
These parameters must be defined on the command line as part of the c4 host reserve command, they cannot be defined in the configuration file or as environment variables before the command.
For more details about parameter formats, see Parameters in c4.
If the CCC_HOST_RESERVED_ADDRS and CCC_HOST_RESERVED_EXTERNAL_ADDRS parameters are already defined in the configuration file, the values in the configuration file will be overwritten by the c4 host reserve command.
To find the PLAY_ID and other details of of the deployment, use c4 ps.
Example:
Determine the play ID of the deployment
CCC_CONFIG=config c4 ps
N PLAY_ID NODE MEDIUM INSTANCE EXTERNAL_IP INTERNAL_IP STAGE STATE UPTIME TTL
┌─ 1 3a4a7d8d 11 host c5d.2xlarge 203.0.113.11 10.0.0.11 d running 04:35:16 +∞
│ 1 3a4a7d8d 12 host c5d.2xlarge 203.0.113.12 10.0.0.11 d running 04:35:16 +∞
└─ 1 3a4a7d8d 13 host c5d.2xlarge 203.0.113.13 10.0.0.13 d running 04:35:15 +∞Reserve private and public IP addresses for the new nodes
CCC_CONFIG=config c4 host reserve --ccc-host-reserved-addrs 10.0.0.14 --ccc-host-reserved-addrs 10.0.0.15 --ccc-host-reserved-external-addrs 203.0.113.14 --ccc-host-reserved-external-addrs 203.0.113.15 3a4a7d8d
INFO[2024-06-18 08:53:46] Reserving new nodes for deployment: ****************************
INFO[2024-06-18 08:53:49] DoneStep 3: Connect to COS
Connect to EXAClusterOS (COS) on the cluster using c4 connect -t <DEPLOYMENT>[.<NODE>]/cos.
To find the deployment number and node ID, use c4 ps as described in the previous step. The deployment number is shown in column N. If you do not specify a node, c4 will connect to the first active node in the deployment.
Example:
For more information about how to use c4 connect, see How to use c4.
The following examples use the command-line tool confd_client, which is available on all database nodes. For more information, see ConfD.
Placeholder values are indicated with UPPERCASE characters. Replace the placeholders with your own values.
Step 4: Add the nodes to the deployment
To add the nodes to the deployment, use the ConfD job infra_instances_add with the following parameters:
| Parameter name | Data type | Description |
|---|---|---|
nid
|
integer | ID of a node with the configuration that should be used to create the new nodes |
num_nodes
|
integer | The number of nodes that you want to create |
Example:
The added nodes will start up and reach deployment stage c (COS service running, database not running). At this point the nodes are not yet part of the cluster.
Step 5: Add the nodes as reserve nodes
To add the new nodes to the cluster as reserve nodes, use the ConfD job db_add_reserve_nodes with the following parameters:
| Parameter Name | Data type | Description |
|---|---|---|
db_name
|
string | The name of the database |
node_list
|
list | List of node IDs (integers) to add as reserve nodes |
Example:
Step 6: Append the nodes to the data volume
The blocks on the new nodes must be appended to the current address space of the existing data volume. To append the nodes to the volume, use the ConfD job st_volume_append_node with the following parameters:
| Parameter name | Data type | Description |
|---|---|---|
vname
|
integer | name of the data volume |
node_num
|
integer | Number of nodes |
node_ids
|
list | List of node IDs (integers) |
To get the volume name, use the ConfD job st_volume_list. For example:
Example:
Step 7: Stop the database
If the database is running, stop it now using the ConfD job db_stop. For example:
Step 8: Activate the nodes as data nodes
To activate the nodes as data nodes, use the ConfD job db_enlarge with the following parameters:
| Parameter name | Data type | Description |
|---|---|---|
db_name
|
string | The name of the database |
num_new_nodes
|
string, integer | The number of nodes to be added as active nodes |
Example:
Step 9: Start the database
Start the database using the ConfD job db_start. For example:
Step 10: Reorganize the database
After restarting the database, you must run the REORGANIZE statement to reorganize the database internally. This statement redistributes the data across the nodes and reconstitutes the distribution and partitioning status.
You can run this command in the built-in SQL client in c4 using c4 connect -t <DEPLOYMENT>/db. For example:
Verification
To verify that the nodes have been added, use the ConfD job node_list.
