Load Data from Azure Blob Storage

Exasol supports download/upload of blobs to Azure Blob Storage. You can use the IMPORT and EXPORT commands to load data from Azure Blob Storage.

By default, the IMPORT and EXPORT commands download or upload data blocks of 10 MB in parallel using two threads. The maximum file size that you can import is approximately 500GB. This is because Azure Blob Storage allows a maximum of 50,000 blocks in a single Blob. If you want to change the default settings, you can change the following parameters:

  • etlAzureBlockSize: Minimum size (in bytes) of Azure Blob upload blocks for IMPORT/EXPORTs. The default value is 10 MB.
  • etlAzureMaxThreads: Maximum number of threads to use for Azure Blob IMPORT/EXPORTs. The default value is 2.

You can modify the above parameters by specifying the parameters in Extra DB Parameters field for your database. To know more about it, see Edit a Database.

Contact Exasol Support before you modify these parameters.

Connection Information

Exasol recommends that you create a connection string using the values of the parameters from the Microsoft Azure Portal. The details are available in Storage Account > Access Keys > Settings. For more information, see View account access keys.

The following parameters should be used in the IMPORT statement:

  • AccountName: Your Azure Blob Storage account name is used in the USER field.
  • AccountKey: Your Azure Blob Storage account key is used in the IDENTIFIED BY field.

    For security reasons, make sure that you add your AccountKey only in the IDENTIFIED BY field.

  • All other parameters are used in AT field.
CREATE CONNECTION MY_BLOBSTORAGE
TO 'DefaultEndpointsProtocol=https;EndpointSuffix=core.windows.net'
USER '<AccountName>' 
IDENTIFIED BY '<AccountKey>';

Importing Data

Run the following statement to import data from Azure Blob Storage.

IMPORT INTO table1 FROM CSV AT CLOUD AZURE BLOBSTORAGE MY_BLOBSTORAGE FILE '<container>/<blob>'

Exporting Data

Run the following statement to export data to Azure Blob Storage.

EXPORT table1 INTO CSV AT CLOUD AZURE BLOBSTORAGE MY_BLOBSTORAGE FILE '<container>/<blob>'