JDBC Local File Import – Parquet Support

Details

Detail name Value
Changelog Number 29002
Type New Feature
Status Resolved
Fix Versions JDBC 26.2.8
Resolution Date 2026-06-30

Overview

JDBC-based local file import has been enhanced to support the Parquet file format, extending the existing functionality that previously supported only CSV and FBV formats.

This enhancement enables users to:

  • Import Parquet files directly from the local filesystem via JDBC
  • Leverage existing import capabilities such as error handling, column mapping, and source tracking
  • Configure performance and behavior using connection string parameters

The feature is designed to integrate seamlessly with the existing IMPORT syntax, ensuring consistency and ease of adoption.

Basic Syntax

A Parquet file can be imported using the following syntax:

IMPORT INTO <schema>.<table>
FROM LOCAL PARQUET FILE '<file_path>';

Example

IMPORT INTO TESTSCHEMA.TESTTB 
FROM LOCAL PARQUET FILE '/home/user/test.parquet';

This command reads the Parquet file from the local filesystem and loads its contents into the specified table.

Error Handling

The import process supports robust error handling mechanisms. Records that fail to load can be redirected either to a database table or to a local CSV file.

1. Errors into a Database Table

IMPORT INTO TESTSCHEMA.TESTTB 
FROM LOCAL PARQUET FILE '/home/user/test.parquet'
ERRORS INTO DEMO.PARQUET_ERROR 
REJECT LIMIT 1 ERRORS;
  • ERRORS INTO <schema>.<table> stores rejected rows in a database table.
  • REJECT LIMIT defines the maximum number of allowable errors before the import fails.

2. Errors into a Local CSV File

IMPORT INTO TESTSCHEMA.TESTTB 
FROM LOCAL PARQUET FILE '/home/user/test.parquet'
ERRORS INTO LOCAL CSV FILE '/home/user/error/error.csv' 
REJECT LIMIT 1 ERRORS;
  • Errors are written to a CSV file on the local system.
  • Useful for debugging and offline analysis.

Source Tracking

Source tracking allows capturing metadata about the origin of each row during import.

Supported Tracking Options

  • File Hash (HASH_SHA256)
  • Row Number (ROW_NUMBER)

Example

IMPORT INTO TESTSCHEMA.TESTTB 
FROM LOCAL PARQUET FILE '/home/user/test.parquet'
WITH 
  SOURCE FILE HASH_SHA256 = FILENAME
  SOURCE ROW NUMBER = ROW_NUMBER;

This enables:

  • Traceability of imported data
  • Debugging of problematic rows
  • Data lineage tracking

Source Tracking with Error Handling

IMPORT INTO TESTSCHEMA.TESTTB 
FROM LOCAL PARQUET FILE '/home/user/test.parquet'
WITH 
  SOURCE FILE HASH_SHA256 = FILENAME
  SOURCE ROW NUMBER = ROW_NUMBER
ERRORS INTO LOCAL CSV FILE '/home/user/error/error.csv' 
REJECT LIMIT 1 ERRORS;

Column Mapping

Column mapping allows explicit mapping between source columns in the Parquet file and target table columns.

Example

IMPORT INTO TESTSCHEMA.TESTTB 
FROM LOCAL PARQUET FILE '/home/user/test.parquet'
WITH 
  SOURCE FILE HASH_SHA256 = FILENAME
  SOURCE ROW NUMBER = ROW_NUMBER
  SOURCE COLUMN NAMES = ('c1');
  • SOURCE COLUMN NAMES defines the column names to be read from the Parquet file.
  • Useful when:
  • The file schema differs from the target table
  • Only a subset of columns is required

Column Mapping with Error Handling

IMPORT INTO TESTSCHEMA.TESTTB 
FROM LOCAL PARQUET FILE '/home/user/test.parquet'
WITH 
  SOURCE FILE HASH_SHA256 = FILENAME
  SOURCE ROW NUMBER = ROW_NUMBER
  SOURCE COLUMN NAMES = ('c1')
ERRORS INTO LOCAL CSV FILE '/home/user/error/error.csv' 
REJECT LIMIT 1 ERRORS;

Clause Ordering Rule (Important)

When using the ERRORS clause:

All WITH clause configurations (source tracking, column mapping, etc.) must be defined before the ERRORS clause.

Incorrect ordering may result in syntax errors or unexpected behavior.

Configuration Parameters

Parquet import behavior can be customized using connection string-style configuration parameters via the CONFIG clause.

Example

IMPORT INTO TESTSCHEMA.TESTTB 
FROM LOCAL PARQUET 
CONFIG 'MaxConnections=2;MaxBatchFetchSize=1'
FILE '/home/user/test.parquet'
WITH 
  SOURCE FILE HASH_SHA256 = FILENAME
  SOURCE ROW NUMBER = ROW_NUMBER
  SOURCE COLUMN NAMES = ('c1');

Parameters

  • For details regarding Parquet parameters, please refer to the documentation on supported Parquet import parameters.
  • Additionally, general CSV/FBV connection import parameters are also applicable. For more information, please refer to the documentation on importing local CSV/FBV files.

Important Limitation

For local Parquet import, MaxConcurrentReads must always be set to 1.

  • Increasing this value above 1 will cause the import to fail. In the JDBC driver, the default value is set to 1.
  • Typical failure: connection errors or read failures due to inability to establish multiple concurrent streams.

Importing Multiple Files

Multiple Parquet files can be imported in a single statement:

IMPORT INTO TESTSCHEMA.TESTTB 
FROM LOCAL PARQUET 
CONFIG 'MaxConnections=2;MaxBatchFetchSize=1'
FILE '/home/user/test.parquet'
FILE '/home/user/test2.parquet'
FILE '/home/user/test3.parquet'
WITH 
  SOURCE FILE HASH_SHA256 = FILENAME
  SOURCE ROW NUMBER = ROW_NUMBER
  SOURCE COLUMN NAMES = ('c1');

Benefits

  • Reduced overhead compared to multiple import statements
  • Unified processing and configuration
  • Efficient batch ingestion

Source Tracking Behavior for Files

When using source tracking:

  • Each file import is handled via a unique port assigned at runtime
  • The port is appended to the filename before hashing
  • As a result:
  • The same file imported multiple times produces different hash values
  • Each file in a multi-file import has a distinct hash

Implication

  • Hash values should be treated as execution-specific identifiers, not stable file fingerprints

Compatibility with Existing Features

All relevant parameters supported for local CSV and FBV imports are also supported for Parquet, including:

  • Secure communication settings
  • Certificate validation
  • ERRORS INTO and REJECT LIMIT clauses

This ensures consistency across file formats and minimizes the learning curve.

Supported DB

  • Exasol 25.1.11+
  • Exasol 25.2.1+

Summary

The addition of Parquet support to JDBC local file import provides:

  • Native support for a widely used columnar format
  • Seamless integration with existing import workflows
  • Full compatibility with:
  • Error handling
  • Source tracking
  • Column mapping
  • Configuration tuning

This enhancement significantly improves data ingestion flexibility and performance for modern data workloads.