Optimized Memory Usage for ETL Parquet Imports with SOURCE COLUMN NAMES
Details
| Detail name | Value |
|---|---|
| Changelog Number | 29578 |
| Type | Improvement |
| Status | Resolved |
| Fix Versions | Exasol 2026.1.0, Exasol 2025.2.1, Exasol 2025.1.11 |
| Resolution Date | 2026-03-20 |
Background
When using the SOURCE COLUMN NAMES feature for ETL Parquet imports, situations involving Parquet files with an extremely large number of columns have been observed to consume higher memory compared to import queries without the SOURCE COLUMN NAMES option. Such higher memory usage could impact system performance and scalability in processing large data files.
Improvement
The ETL Parquet import process has been optimized to significantly reduce memory consumption associated with the use of the SOURCE COLUMN NAMES feature. Wherever possible, memory usage is now minimized based on the specific characteristics of Parquet files with an enormous number of columns. This enhancement ensures more efficient resource utilization and improved performance for Parquet file imports.
