Validation of Parquet Reader Parameters During DB Startup
Details
| Detail name | Value |
|---|---|
| Changelog Number | 26758 |
| Type | Improvement |
| Status | Resolved |
| Fix Versions | Exasol 2025.1.5, Exasol 2025.2.0 |
| Resolution Date | 2025-12-05 |
Background
Currently, during database startup, if invalid values (e.g., negative or zero) are provided for Parquet Reader parameters, the system automatically replaces these values with default values and proceeds. While this prevents the database from failing, it can lead to unpredictable behavior and performance issues since users may mistakenly assume their configured values were applied successfully.
Improvement
The behavior has been enhanced to enforce stricter validation for Parquet Reader parameters during database startup. Instead of silently falling back to default values, the system will now throw an EXAoperation error (visible via ConfD job log_collect, see log_collect) for invalid parameter values and abort the database startup. This change ensures users are alerted about misconfigured parameters and encourages proper configuration prior to database execution.
The error message will be printed on node 0 only to simplify visibility.
| Parameter Name | Validation Rule | Valid Range | Action on Invalid Value |
|---|---|---|---|
| etlParquetMaxConnections | Throws exception if <= 0 | Greater than 0 | Throw error and abort DB startup. Example: Invalid value for parameter - etlParquetMaxConnections: <Provided Value>. The value must be greater than 0. Please provide a valid number. |
| etlParquetMaxConcurrentReads | Throws exception if <= 0 | Greater than 0 | Throw error and abort DB startup. Example: Invalid value for parameter - etlParquetMaxConcurrentReads: <Provided Value>. The value must be greater than 0. Please provide a valid number. |
| etlParquetMaxBatchFetchSize | Throws exception if <= 0 | Greater than 0 | Throw error and abort DB startup. Example: Invalid value for parameter - etlParquetMaxBatchFetchSize: <Provided Value>. The value must be greater than 0. Please provide a valid number. |
| etlParquetMaxRows | Throws exception if <= 0 | Greater than 0 | Throw error and abort DB startup. Example: Invalid value for parameter - etlParquetMaxRows: <Provided Value>. The value must be greater than 0. Please provide a valid number. |
| etlParquetMaxRetry | Throws exception if < 0 | Greater than or equal to 0 | Throw error and abort DB startup. Example: Invalid value for parameter - etlParquetMaxRetry: <Provided Value>. The value must be greater than or equal to 0. Please provide a valid number. |
| etlParquetDebugMode | Throws exception if < 0 | Greater than or equal to 0 | Throw error and abort DB startup. Example: Invalid value for parameter - etlParquetDebugMode: <Provided Value>. The value must be greater than or equal to 0. Please provide a valid number. |
Changed behavior
Previously, the database would silently replace invalid Parquet Reader parameter values with default values during startup, without notifying users, leading to potential issues or degraded performance. Now, the database strictly validates these parameters, throwing an error and halting startup if any value is invalid, requiring users to fix the configuration since fallback defaults are no longer applied. Clear error messages are provided, specifying the parameter, invalid value, expected range, and suggested fixes. To streamline debugging in distributed setups, validation and error reporting are limited to Node 0, reducing log clutter and simplifying issue identification.
