Validate Parquet Reader Connection String and DB Parameters for Inappropriate Values

Details

Detail name Value
Changelog Number 26748
Type Improvement
Status Resolved
Fix Versions Exasol 2025.1.3, Exasol 2025.2.0
Resolution Date 2025-09-17

Description

The Parquet reader utilizes both connection string parameters and database (DB) parameters to configure it's behavior during data import. Some parameters represent settings like maximum parallel reads and batch sizes.

Currently, when invalid values (e.g., 0 or values less than 0) are provided, the Parquet reader silently defaults some parameters (e.g., to 1). This lack of transparency in handling inappropriate values can lead to user confusion, unintentional performance degradation, and inconsistent configuration behavior.

A clear and standardized approach is proposed to handle invalid input by raising errors for connection string parameters and internally defaulting DB parameters to their predefined safe values.

Improvement

  1. Connection String Parameters Validation
  • New Behavior
    If any connection string parameter in the FROM PARQUET statement is set to a value less than or equal to 0, an error will be thrown.
  • Example Behavior
  • The error will clearly specify the key, the invalid value, and guidance to provide a valid value.
IMPORT INTO STAGING_TABLE
  FROM PARQUET AT 'https://xxxxx.s3.amazonaws.com/;MaxConcurrentReads=3;MaxBatchFetchSize=-4'
  FILE 'nested/path/to/test.parquet' FILE 'nested/path/to/test_2.parquet'
  WITH
    SOURCE FILE HASH_SHA256 = "source_path"
    SOURCE ROW NUMBER = "row_number";

Generated Error
ETL-2224: Connection String parameter (MaxBatchFetchSize) contains an invalid value (-4). Please provide a value greater than 0.

  1. Database Parameter Validation
  • New Behavior
    If any DB parameter is set to a value less than or equal to 0, the system will automatically default the parameter to its internally predefined value. No error will be raised.
  • Example Behavior 
-etlParquetMaxRows=-3

System Response
The parameter -etlParquetMaxRows will automatically be set to the default value of 4096.