Importing Parquet files

Learn how to load data from Parquet files stored locally, on remote file servers, or in the cloud.

You can use the IMPORT command to load data from Parquet files that are stored on cloud storage services, on remote file servers using FTP/FTPS, SFTP, and HTTP/HTTPS, and on local systems. Exasol supports the Apache Iceberg open table format in Parquet files.

Multiple Parquet files can be imported in parallel across all cluster nodes, enabling higher throughput for larger datasets. Single Parquet files can be read using multiple parallel operations for faster and more efficient data read operations.

Exasol automatically maps Parquet data types to compatible Exasol target columns. To learn more, see Data type mapping.

In Exasol 2026.1 and later you can use glob patterns to import multiple Parquet files dynamically and recursively from S3-compatible storage. To learn more, see Load files using pattern matching.

This article describes how to import Parquet files from cloud storage. To learn how to import Parquet files stored on local or remote systems, see Load data from local or remote files.

All Parquet import features are fully supported in Exasol 2026.1.0 and later, with partial support in earlier versions. For more details about which features are supported in your Exasol version, see the Release notes.

Connect to storage

This article describes the available features and considerations when importing Parquet files. To learn how to connect to your storage service or local storage, see the following articles:

Source data tracking

Additional SQL options enable row-origin tracking by appending SHA256 file hash and source row number as metadata during the import.

The target table must contain suitable columns to store the metadata.

SQL option Description Column type

SQL option	Description	Column type
`SOURCE FILE HASH_SHA256`	Allows secure mapping of rows to their respective file origins.	`CHAR(64)` `VARCHAR(n)` with `n`>=64 `HASHTYPE (256 BIT)`
`SOURCE ROW NUMBER`	Captures the position of the row in the source file (0-based index).	`DECIMAL(p,s)`

SOURCE FILE HASH_SHA256

Allows secure mapping of rows to their respective file origins.

CHAR(64)

VARCHAR(n) with n>=64

HASHTYPE (256 BIT)

SOURCE ROW NUMBER Captures the position of the row in the source file (0-based index). DECIMAL(p,s)

Column mapping

This feature is available in Exasol 2025.1.5 and later.

You can specify source column names by using WITH SOURCE COLUMN NAMES=() in the IMPORT statement. This allows you to explicitly map Parquet columns to columns in the target table.

Usage notes

Column names are case sensitive. If there is a case mismatch between the source/target column names, the target column defaults to NULL.
The columns in the source must be in the exact same order as the target columns.
Duplicate column names in the Parquet file are not allowed. The query will abort with error ETL-2231.
Columns in the source that are not mapped to a target column are ignored.
If a source column is missing, the corresponding target column defaults to NULL.

SkipCols cannot be used together with the SOURCE COLUMN NAMES option. If both are specified in the same query, the import operation fails with the exception ETL-2230.

Usage

Scenario 1: Column mapping for diverse schemas

When working with datasets where source file schemas vary:

Map target table columns to specific Parquet columns using the WITH SOURCE COLUMN NAMES=() option.
Unmapped Parquet columns are ignored.

Scenario 2: Strict import schema enforcement

Ensure that all target table columns have corresponding mapped source columns for the import operation to succeed.

Best practices

Use the WITH SOURCE COLUMN NAMES=() option when loading from schema-diverse Parquet datasets to enforce mapping consistency.
Make sure that there is no case mismatch between the column names in the Parquet file and the target database table to avoid unexpected NULL values.

For an example of how to use column mapping, see Example 3 - Use explicit column mapping.

Data type mapping

Automatic casting to Exasol data types is done by EXALoader for the following Parquet data types:

INT8, INT16, INT32, INT64, UINT8, UINT16, UINT32, UINT64, DECIMAL128, DOUBLE, FLOAT, BOOLEAN, STRING, LARGE_STRING, DATE32, DATE64, TIMESTAMP

Using Parquet data types that are not supported by will result in a runtime exception.

Target column data type in Exasol	Supported Parquet data types
Integer and decimal types	INT8, INT16, INT32, INT64, UINT8, UINT16, UINT32, UINT64, DECIMAL128, BOOLEAN
DOUBLE	INT8, INT16, INT32, INT64, UINT8, UINT16, UINT32, UINT64, FLOAT, DOUBLE
String types (VARCHAR, CHAR, HASHTYPE)	STRING, LARGE_STRING
DATE	DATE32, DATE64
BOOLEAN	BOOLEAN
TIMESTAMP	TIMESTAMP, DATE32, DATE64

Additional data type mappings

The following feature is supported in Exasol 2026.1 and later.

In Exasol 2026.1.0 and later, EXALoader can import values from Parquet columns into compatible Exasol target columns even when the source and target data types are different. Some of these mappings convert typed Parquet values directly, while others parse values stored as text.

For text-based mappings such as STRING → DECIMAL, the input must match the active format for the data type. If you have defined a session format using ALTER SESSION, that format is used. Otherwise, the default format is used.

For more details and examples, see Data type mapping details .

Parquet type	Target types in Exasol
`STRING`, `LARGE_STRING`	`DECIMAL`, `DOUBLE`, `BOOLEAN`, `DATE`, `TIMESTAMP`
`DECIMAL128`	`DOUBLE`, `VARCHAR`, `CHAR`
`DECIMAL256`	`DECIMAL`, `DOUBLE`, `VARCHAR`, `CHAR`
`DATE32`, `DATE64`	`VARCHAR`, `CHAR`
`TIMESTAMP`	`VARCHAR`, `CHAR`

DECIMAL256 is only supported in Exasol 2026.1 and later.

For DOUBLE target columns, small round-off differences are normal.

Data type mapping details

STRING/LARGE_STRING -> DECIMAL

Use this mapping when numeric values are stored as text in Parquet and should be imported into an Exasol DECIMAL column.

Example (default format):

Copy

CREATE TABLE T_STRING_DECIMAL (String_DT DECIMAL(18,4));

IMPORT INTO T_STRING_DECIMAL FROM PARQUET AT '***';

What to expect

Input like 123.4500 is imported as 123.4500
Input like 456.78 imported into DECIMAL(10,0) becomes 456

Supported formats

Numeric text such as: 123, 123.45, -10.50

Example (user-defined format):

No date or timestamp format applies here
Numeric parsing follows active numeric settings where relevant

Limitations

The precision and scale of the target column must allow the value
Fractional digits are not preserved if target scale is smaller
Invalid numeric text causes import failure

STRING / LARGE_STRING -> DOUBLE

Use this mapping when numeric values are stored as text in Parquet and should be imported into an Exasol DOUBLE column.

Example (default format)::

Copy

CREATE TABLE T_STRING_DOUBLE (String_DT DOUBLE);

IMPORT INTO T_STRING_DOUBLE FROM PARQUET AT '***';

What to expect

Input like 123.45 is imported as 123.45

Supported formats

Numeric text such as 123, 123.45, -0.5

Example (user-defined format):

No date or timestamp format applies here
Numeric parsing follows active numeric settings where relevant

Limitations

Small floating-point round-off differences may occur
Test validation uses ROUND(..., 4)
Invalid numeric text causes import failure

STRING / LARGE_STRING -> BOOLEAN

Use this mapping when boolean values are stored as text in Parquet and should be imported into an Exasol BOOLEAN column.

Example (default format):

Copy

CREATE TABLE T_STRING_BOOLEAN (String_DT BOOLEAN);

IMPORT INTO T_STRING_BOOLEAN FROM PARQUET AT '***';

What to expect

true becomes TRUE
0 becomes FALSE
yes becomes TRUE

Supported formats

true, false
TRUE, FALSE
1, 0
T, F
Y, N
yes, no

Example (user-defined format):

No date or timestamp format applies here

Limitations

Supported values are effectively case-insensitive for tested forms
Unsupported text such as maybe causes import failure

STRING / LARGE_STRING -> DATE

Use this mapping when dates are stored as text in Parquet and should be imported into an Exasol DATE column.

Example (default format):

Copy

CREATE TABLE T_STRING_DATE (String_DT DATE);

IMPORT INTO T_STRING_DATE FROM PARQUET AT '***';

Example (user-defined format):

Copy

ALTER SESSION SET NLS_DATE_FORMAT = 'DD-MM-YYYY';

CREATE TABLE T_STRING_DATE_FMT (String_DT DATE);

IMPORT INTO T_STRING_DATE_FMT FROM PARQUET AT '***';

What to expect

Input 26-08-2025 is imported as 2025-08-26 if NLS_DATE_FORMAT not specified by user.

Supported formats

Default date format YYYY-MM-DD
User specific format like DD-MM-YYYY, MM-DD-YYYY, YYYY-DD-MM, etc.

Limitations

Input must match the active date format exactly
If NLS_DATE_FORMAT is set, that format is used, otherwise the default date format is used
Mismatched format fails with an error

STRING / LARGE_STRING -> TIMESTAMP

Use this mapping when timestamps are stored as text in Parquet and should be imported into an Exasol TIMESTAMP column.

Example (default format):

Copy

CREATE TABLE T_STRING_TIMESTAMP (String_DT TIMESTAMP);

IMPORT INTO T_STRING_TIMESTAMP FROM PARQUET AT '***';

Example (user-defined format):

Copy

ALTER SESSION SET NLS_TIMESTAMP_FORMAT = 'DD-MM-YYYY HH24:MI:SS.FF3';

CREATE TABLE T_STRING_TIMESTAMP_FMT (String_DT TIMESTAMP);

IMPORT INTO T_STRING_TIMESTAMP_FMT FROM PARQUET AT '***';

What to expect

Input 26-08-2025 14:30:15.123 is imported as 2025-08-26 14:30:15.123

Supported formats

Default timestamp format YYYY-MM-DD HH24:MI:SS.FF3
User specific format like DD-MM-YYYY HH24:MI:SS.FF3, MM-DD-YYYY HH24:MI:SS.FF3, YYYY-DD-MM HH24:MI:SS.FF3, etc.
Supported fractional-second formats: FF1 to FF9

Limitations

Input must match the active timestamp format exactly
If NLS_TIMESTAMP_FORMAT is set, that format is used, otherwise the default timestamp format is used
Mismatched format fails with an error

DECIMAL128 -> DOUBLE

Use this mapping when a typed Parquet DECIMAL128 value should be imported into an Exasol DOUBLE column.

Example (default format):

Copy

CREATE TABLE T_DECIMAL128_DOUBLE (Decimal_128_DT DOUBLE);

IMPORT INTO T_DECIMAL128_DOUBLE FROM PARQUET AT '***';

What to expect

Input like 123.45 is imported as 123.45

Supported formats

Typed Parquet DECIMAL128 values

User-defined formats

No user-defined date or timestamp format applies

Limitations

Precision may be lost because the target is DOUBLE
Test validation uses ROUND(..., 9)

DECIMAL128 -> VARCHAR / CHAR

Use this mapping when a typed Parquet DECIMAL128 value should be stored as text in Exasol.

Example (default format):

Copy

CREATE TABLE T_DECIMAL128_VARCHAR (Decimal_128_DT VARCHAR(128));

IMPORT INTO T_DECIMAL128_VARCHAR FROM PARQUET AT '***';

What to expect

Input like 123.45 is imported as 123.45

Supported formats

Typed Parquet DECIMAL128 values

User-defined formats

No user-defined date or timestamp format applies

Limitations

Output is textual decimal representation

Tested by:

test_Decimal128_VarChar

DECIMAL256 -> DECIMAL

Use this mapping when a typed Parquet DECIMAL256 value should be imported into an Exasol DECIMAL column.

Example (default format):

Copy

CREATE TABLE T_DECIMAL256_DECIMAL (Decimal_256_DT DECIMAL(36,18));

IMPORT INTO T_DECIMAL256_DECIMAL FROM PARQUET AT '***';

What to expect

High-precision decimal values are imported if the target precision and scale allow them

Supported formats

Typed Parquet DECIMAL256 values

Example (user-defined format):

No user-defined date or timestamp format applies

Limitations

Target precision and scale must allow the value
Invalid target precision definitions fail before import

DECIMAL256 -> DOUBLE

Use this mapping when a typed Parquet DECIMAL256 value should be imported into an Exasol DOUBLE column.

Example (default format):

Copy

CREATE TABLE T_DECIMAL256_DOUBLE (Decimal_256_DT DOUBLE);

IMPORT INTO T_DECIMAL256_DOUBLE FROM PARQUET AT '***';

What to expect

Input like 123.45 is imported as 123.45

Supported formats

Typed Parquet DECIMAL256 values

Example (user-defined format):

No user-defined date or timestamp format applies

Limitations

Precision may be lost because the target is DOUBLE
Test validation uses ROUND(..., 6)

Tested by:

test_Decimal256_Double

DECIMAL256 -> VARCHAR / CHAR

Use this mapping when a typed Parquet DECIMAL256 value should be stored as text in Exasol.

Example (default format):

Copy

CREATE TABLE T_DECIMAL256_VARCHAR (Decimal_256_DT VARCHAR(128));

IMPORT INTO T_DECIMAL256_VARCHAR FROM PARQUET AT '***';

What to expect

High-precision decimal values are imported as decimal text

Supported formats

Typed Parquet DECIMAL256 values

Example (user-defined format):

No user-defined date or timestamp format applies

Limitations

Output is textual decimal representation

DATE32 / DATE64 -> VARCHAR / CHAR

Use this mapping when a typed Parquet date value should be stored as text in Exasol.

Example (default format):

Copy

CREATE TABLE T_DATE_VARCHAR (Date_32_DT VARCHAR(128));

IMPORT INTO T_DATE_VARCHAR FROM PARQUET AT '***';

What to expect

Typed date values are imported as formatted date text

Supported formats

Typed Parquet DATE32
Typed Parquet DATE64

Output uses user-defined date format if set, otherwise the default date format

Example (user-defined format):

Copy

ALTER SESSION SET NLS_DATE_FORMAT = 'DD-MM-YYYY';

Expected output shape:

A typed date may be formatted as 26-08-2025

Supported formats

Default date format YYYY-MM-DD
User specific format like DD-MM-YYYY, MM-DD-YYYY, YYYY-DD-MM, etc.

Limitations

This is a formatting path, not free-form text parsing
Output depends on the active date format

TIMESTAMP -> VARCHAR / CHAR

Use this mapping when a typed Parquet timestamp value should be stored as text instead of as an Exasol timestamp type.

Example (default format):

Copy

CREATE TABLE T_TIMESTAMP_VARCHAR (TimeStamp_MS_DT VARCHAR(128));

IMPORT INTO T_TIMESTAMP_VARCHAR FROM PARQUET AT '***';

What to expect

Typed timestamp values are imported as formatted timestamp text
Tested output shape includes values like 2024-01-01 12:00:00.000000000

Supported formats

Typed Parquet timestamp values with supported timestamp units
Supported fractional-second formats: FF1 to FF9

Output uses user-defined timestamp format if set, otherwise the default timestamp format

Example (user-defined format):

Copy

ALTER SESSION SET NLS_TIMESTAMP_FORMAT = 'DD-MM-YYYY HH24:MI:SS.FF3';

Expected output shape:

A typed timestamp may be formatted as 01-01-2024 12:00:00.000

Supported formats

Default timestamp format YYYY-MM-DD HH24:MI:SS.FF3
User specific format like DD-MM-YYYY HH24:MI:SS.FF3, MM-DD-YYYY HH24:MI:SS.FF3, YYYY-DD-MM HH24:MI:SS.FF3, etc.
Supported fractional-second formats: FF1 to FF9

Limitations

Output precision depends on timestamp formatting behavior
Output depends on the active timestamp format

Column skipping

The SkipCols configuration parameter enables selective exclusion of columns from the imported Parquet data based on their index positions (0-based). This provides precise control over which columns to include or exclude while importing data. Skipped columns are defined as part of the connection string using ranges or discrete column indices.

SkipCols cannot be used together with the SOURCE COLUMN NAMES option. If both are specified in the same query, the import operation fails with the exception ETL-2230.

For more information and examples of use, see Configuration parameters and Examples.

Load files using pattern matching

This feature is available in Exasol 2026.1 and later.

Pattern matching is only supported when importing Parquet files from S3 storage.

The FILE parameter in IMPORT FROM PARQUET supports glob patterns when accessing data from S3-compatible storage systems. This allows importing multiple files without specifying each file explicitly. Glob patterns are interpreted in the file path and enable flexible selection of files across directories and subdirectories.

Supported patterns

Pattern	Description
`*`	Matches zero or more characters
`?`	Matches exactly one character
`[abc]`	Matches one character from the specified set
`[!abc]` or `[^abc]`	Matches one character not in the specified set
`/**/`	Matches directories recursively

The characters *, ?, [, ] are interpreted as glob pattern operators if they are used in the file path. To match them literally, escape them using a backslash /

Examples

Import all Parquet files from the bucket root:

Copy

IMPORT INTO my_table
FROM PARQUET
AT 's3://bucket/'
USER '...'
IDENTIFIED BY '...'
FILE '*.parquet';

Import recursively from a directory and its subdirectories:

Copy

...
FILE 'data/**/*.parquet';

Import files from a directory (non-recursive):

Copy

...
FILE 'data/*';

Match single character:

Copy

...
FILE 'data/file?.parquet';

Match a character set:

Copy

...
FILE 'data/file[abc].parquet';

Exclude characters:

Copy

...
FILE 'data/file[!abc].parquet';

Usage notes

Glob patterns are supported only in the FILE clause.
The bucket name in the AT clause must be specified explicitly and does not support glob patterns.
Glob patterns are supported only for S3-compatible storage systems.
Recursive matching is only enabled when using the exact syntax /**/. For example:
- data/**/*.parquet → recursive
- data**/*.parquet → not recursive

Error handling

If no files match the specified pattern, the query fails.
If any matched file is not a valid Parquet file or has an incompatible schema, the entire import fails.
If multiple patterns are specified, each pattern must match at least one file.
Glob patterns are not allowed in the connection definition (AT clause).
Using glob patterns with unsupported storage systems results in an error.

Permissions

The user must have permission to use the s3:ListBucket action in AWS. Without this permission, the import fails. For more information, refer to the Amazon AWS documentation.

Configuration parameters

Configuration parameters can be set to be query-specific using the connection string, and as global defaults using database parameters. If a parameter is set in the connection string, it will override the default database parameter.

Usage notes

Connection string parameters are case-insensitive.
Connection string parameters override database parameter values when both are provided.
Default values are sourced from the database parameters if not expressly set.

Parameter	Description	Validation
`MaxConnections`	Number of Parquet files imported in parallel on a single node.	Must be > 0
`MaxConcurrentReads`	Number of parallel read operations per file. Set to `1` for sequential reads. `0` is not allowed. Set to `1` when importing local files.	Must be > 0
`MaxBatchFetchSize`	Maximum simultaneous buffers fetched during import.	Must be > 0
`MaxRows`	Maximum number of rows to read in a batch.	Must be > 0
`MaxRetry`	Maximum number of attempts to connect.	Must be ≥ 0
`SkipCols`	Skip specific column indexes during import. Example: `SkipCols=1,3..8,11` excludes columns 1, 3, 4, 5, 6, 7, 8, and 11. All other columns remain part of the imported dataset.	Only integers (`0`to `9`), commas (`,`) to separate values, and ranges with two dots (`..`) are allowed. A range must have exactly two integers, one before and one after the two dots (`..`). For example, `3..5` is valid, but `3..` or `..5` are invalid. The starting value must be less than or equal to the ending value. For example, `3..5` is valid, but `5..3` is invalid.
AWS S3 and S3-compatible storage only
`DebugMode`	Enable AWS structured logging.	Must be ≥ 0
`EndPoint`	URL when connecting to S3-compatible storage service (not AWS)	-
`UseProxy`	Boolean, specifies if the proxy specified in the `-etlProxy` database parameter should be used when connecting to an S3-compatible storage service (not AWS) `1` (default) = use proxy if one is specified `0` = do not use a proxy	-

Log levels (AWS)

The log level determines the granularity and amount of information logged by the system. Select a log level to tailor the logging output based on the use case, such as error debugging, performance monitoring, or high-level event tracking.

To enable structured logging for AWS, the database parameters -etlLogLevel and -traceLevel must both be set to at least 2.

Value	Log level	Description
0	OFF (default)	Disables logging completely. No log messages are written.
1	FATAL	Logs only critical errors that cause the application or service to terminate.
2	ERROR	Logs errors that occur during execution but do not stop the application.
3	WARN	Logs events that are potentially harmful to the process but are recoverable.
4	INFO	Logs informational messages about the execution flow, such as successful initializations and key events.
5	DEBUG	Logs detailed debug information, typically for development or troubleshooting purposes.
≥ 6	TRACE	Logs highly detailed information, useful for deep debugging and issue resolution.

Schema inference

This feature is supported in Exasol 2026.1 and later.

The schema inference feature allows you to import Parquet files without first creating a target table in the database. The loader will automatically create the table if it does not exist. The syntax is:

CREATE OR REPLACE TABLE <table_name> AS (IMPORT FROM PARQUET <standard parquet import statement options>)

If the target table already exists, an exception is thrown and the import job is aborted.

Column mapping is not supported with schema inference.

Data type mapping

The data type of the target table is derived from the Parquet data type using the following mapping:

Parquet data type	Exasol data type
`PARQUET_BOOL`	`BOOLEAN`
`PARQUET_INT8`	`DECIMAL(3,0)`
`PARQUET_INT16`	`DECIMAL(5,0)`
`PARQUET_INT32`	`DECIMAL(10,0)`
`PARQUET_INT64`	`DECIMAL(20,0)`
`PARQUET_UINT8`	`DECIMAL(3,0)`
`PARQUET_UINT16`	`DECIMAL(5,0)`
`PARQUET_UINT32`	`DECIMAL(10,0)`
`PARQUET_UINT64`	`DECIMAL(20,0)`
`PARQUET_DECIMAL128`	`DECIMAL(p,s)` Precision and scale `(p,s)` are set according to the Parquet file metadata. `p` and `s` must be ≤ 36
`PARQUET_FLOAT`	`DOUBLE PRECISION`
`PARQUET_DOUBLE`	`DOUBLE PRECISION`
`PARQUET_STRING`	`VARCHAR(n)` `n` (VARCHAR length) must be ≤ 2,000,000
`PARQUET_LARGE_STRING`	`VARCHAR(n)` `n` (VARCHAR length) must be ≤ 2,000,000
`PARQUET_DATE32`	`DATE`
`PARQUET_DATE64`	`DATE`
`PARQUET_TIMESTAMP`	`TIMESTAMP(p)`

Example:

Copy

CREATE OR REPLACE TABLE my_schema.my_table 
    AS (
    IMPORT FROM PARQUET AT 'my_s3_bucket.s3.eu-west-1.amazonaws.com;MaxConnections=2;MaxBatchFetchSize=4;SkipCols=1,3..8,11'' 
    USER 'my_user' 
    IDENTIFIED BY 'my_secret_key' FILE 'dup2.parquet'
    )
;

Examples

Exaple 1: Create a connection object

Copy

-- Amazon S3 connection
CREATE OR REPLACE CONNECTION my_conn
    TO 'https://my_s3_bucket.s3.eu-west-1.amazonaws.com'
    USER 'my_access_key' IDENTIFIED BY 'my_secret_key'
;

-- S3-compatible storage
CREATE OR REPLACE CONNECTION my_conn
    TO 's3://my_bucket;EndPoint=https://example.com:9000'
    USER 'my_user' IDENTIFIED BY 'my_secret'
;

-- Azure Blob Storage
CREATE CONNECTION my_conn
    TO 'DefaultEndpointsProtocol=https;EndpointSuffix=core.windows.net'
    USER 'my_account_name' IDENTIFIED BY 'my_account_key'
;

Example 2 - Create schema and table with file tracking

In this example we connect using a stored connection (see examples above), create a new schema and table, then load data from two Parquet files, using SQL options to store source metadata for row-origin tracking.

Copy

CREATE SCHEMA IF NOT EXISTS MY_SCHEMA;

CREATE TABLE MY_SCHEMA.MY_TABLE (sourcefile HASHTYPE (256 BIT), c1 INT, rownumber INT);

IMPORT INTO MY_SCHEMA.MY_TABLE
    FROM PARQUET AT my_conn
    FILE 'nested/path/to/test_1.parquet' FILE 'nested/path/to/test_2.parquet' 
    WITH 
      SOURCE FILE HASH_SHA256 = sourcefile
      SOURCE ROW NUMBER = rownumber 
;

Example 3 - Use explicit column mapping

Copy

CREATE SCHEMA IF NOT EXISTS MY_SCHEMA;

CREATE TABLE MY_SCHEMA.MY_TABLE (
  c1 INT,
  name VARCHAR(256),
  mapped_column VARCHAR(256),
  sourcefile HASHTYPE (256 BIT),
  rownumber INT
);

IMPORT INTO MY_SCHEMA.MY_TABLE
  FROM PARQUET AT my_s3_bucket
  FILE 'file_1.parquet'
  FILE 'file_2.parquet'
  WITH
    SOURCE COLUMN NAMES = ('col1', 'col2', 'renamed_col')
    SOURCE FILE HASH_SHA256 = sourcefile
    SOURCE ROW NUMBER = rownumber
;

Example 4 - Use configuration parameters in connection string

The following examples show a connection to AWS. The procedure is identical for all storage types.

Copy

-- Limit the number of files to import concurrently to 2
-- Increase the number of buffers to fetch during the import to 4
-- Exclude columns 1, 3, 4, 5, 6, 7, 8, and 11 from the imported dataset 
CREATE OR REPLACE CONNECTION my_conn
    TO 'https://my_s3_bucket.s3.eu-west-1.amazonaws.com;MaxConnections=2;MaxBatchFetchSize=4;SkipCols=1,3..8,11'
    USER 'my_access_key' IDENTIFIED BY 'my_secret_key'
;

-- Set a dynamic batch size
CREATE OR REPLACE CONNECTION my_conn 
    TO 'https://my_s3_bucket.s3.eu-west-1.amazonaws.com/;MaxRows=2048' 
    USER 'my_access_key' IDENTIFIED BY 'my_secret_key'
;

Importing Parquet files

Connect to storage

Source data tracking

Column mapping

Usage notes

Usage

Best practices

Data type mapping

Additional data type mappings

Data type mapping details

Column skipping

Load files using pattern matching

Supported patterns

Examples

Import all Parquet files from the bucket root:

Import recursively from a directory and its subdirectories:

Import files from a directory (non-recursive):

Match single character:

Match a character set:

Exclude characters:

Usage notes

Error handling

Permissions

Configuration parameters

Usage notes

Log levels (AWS)

Schema inference

Data type mapping

Example:

Examples

Exaple 1: Create a connection object

Example 2 - Create schema and table with file tracking

Example 3 - Use explicit column mapping

Example 4 - Use configuration parameters in connection string

PRODUCT

RESOURCES