IMPORT/EXPORT FROM/INTO EXA: No More Special Configuration for IP and Port
Details
| Detail name | Value |
|---|---|
| Changelog Number | 21243 |
| Type | Improvement |
| Status | Resolved |
| Fix Versions | Exasol 8.32.0 |
| Resolution Date | 2024-11-08 |
Description
The new implementation enables the user to specify a list of hostnames to be used for import/export operations using Exa to Exa. Previously, when an ETL import/export job was started, the client would connect to the database and request parallel connections. The database would then send the host and port to establish a parallel connection. This method had a disadvantage when used on cloud platforms like GCP, AWS, Azure, etc., as well as on-prem clusters. Additionally, for establishing a parallel connections, port needs to be opened in the range [20000 - 21000]. And it also requires special IP configuration for the calculation of IP addresses. This further complicates the process and introduces challenges when configuring network settings. To avoid these problems, a new method has been introduced.
Advantages
The new implementation enables the user to specify a list of hostnames for import/export operations, which provides much more flexibility to the user.
It helps users to import/export data from cloud-based Exasol instances, which was previously difficult.
The connection string used for the Exa to Exa import/export operation also supports OpenID authentication and other parameters like SSLCertificate, AuthMethod, HostTimeOut, LoginTimeout, Encryption, LogMode, EXALogFile, and Fingerprint.
Example SQL import Query
SELECT * FROM (IMPORT FROM EXA AT "test.host.com,test2.host.com:8563;HostTimeOut=1000;Encryption=Y" USER 'XXXXX' IDENTIFIED BY 'YYYYYY' TABLE DemoSchema.DemoTB);
Parallel Connections are established using the database's existing port, so no additional ports need to be opened by the database.
Requirements
It is required to mention all the hostnames. If the number of hosts is less than the maximum parallel connections, the old implementation will be used.
New Method
In the new method, the database will try to establish a parallel connection to the other database using the new implementation, if it supports it. If not, it will use the old implementation. When the new database accepts the new implementation, the IP address and port required to start the parallel connection will be parsed from the connection string provided by the user. If the connection string contains a DNS entry, it will be resolved, and the IP address will be shuffled.
If the new and old implementation also fails, then the error from the new implementation will be thrown to the user. Additionally, every time the old interface is used, it will be recorded in the database, and warning message will be printed at 12:00:00. The counter will be reset to 0.
If the counter value reaches the maximum value before 12:00:00, it will be printed and reset to zero. The maximum value is set by the DB parameter (etlExaDeprecatedWarnings). The default value is 0, which means the counter value is not set. So, it will be printed only at 12:00:00. The main condition to print the warning message at 12:00:00 is that the database should be running for more than 24 Hours.
Warning Message: "The previous behavior for parallel connections has been deprecated. Please update connection string to include all Exasol hosts."
