Load data from Apache Hive

Learn how to connect and load data from Apache Hive using the Cloudera Hive JDBC driver.

Prerequisites

  • Hive must be reachable from the Exasol system.

  • The user credentials in the connection must be valid.

Download driver

Download the compatible Cloudera Hive JDBC driver.

If you want to use another driver, contact Support.

Add JDBC driver

  1. Create a configuration file settings.cfg with the following settings:

    Copy 
    DRIVERNAME=HiveCloudera
    PREFIX=jdbc:hive2:
    FETCHSIZE=100000
    INSERTSIZE=-1

    The file must end with an empty line (line break followed by zero characters).

    With some drivers you may receive an error message indicating file permission issues, or the query will hang without an error message. To resolve this issue, you may have to disable the security manager by adding the line NOSECURITY=YES in the configuration.

  2. Upload the settings.cfg file and the driver .jar file to BucketFS in the Exasol cluster.

    In Exasol 2025.1 and later you can upload files using Exasol Admin. You can also use any compatible file transfer tool, including curl on the command line. See also Manage files in BucketFS.

    In Exasol SaaS you must upload the driver through the web console.

    If the driver was downloaded as a .tar.gz or .zip archive, make sure that you extract and upload only the .jar file along with the settings.cfg file.

    Example (using curl in Linux):
    Copy 
    export WRITE_PW=<your_bucketfs_write_password>
    export DATABASE_NODE_IP=<ip_address_of_cluster_node>
    export PORT=<bucketfs_port> # default: 2581

    curl -v --insecure -X PUT -T settings.cfg https://w:$WRITE_PW@$DATABASE_NODE_IP:$PORT/default/drivers/jdbc/exasol/settings.cfg
    curl -v --insecure -X PUT -T exajdbc.jar https://w:$WRITE_PW@$DATABASE_NODE_IP:$PORT/default/drivers/jdbc/exasol/exajdbc.jar

    The option --insecure or -k tells curl to bypass the TLS certificate check. This option allows you to connect to a HTTPS server that does not have a valid certificate. Only use this option if certificate verification is not possible and you trust the server.

Create connection

To create the connection, use the following statements. Replace the connection string and credentials as needed.

Copy 
-- Connecting EXAloader (the native bulk loader in Exasol) via Cloudera Hive driver 
-- to Cloudera, MapR, and Hortonworks Hadoop distributions

-- cloudera-quickstart-vm-5.13.0-0-vmware
create or replace connection hive_conn to 'jdbc:hive2://192.168.42.133:10000' user 
'cloudera' identified by 'cloudera';

-- MapR-Sandbox-For-Hadoop-6.1.0-vmware
create or replace connection hive_conn to 'jdbc:hive2://192.168.42.134:10000' user 
'mapr' identified by 'mapr';

-- Azure Hortonworks Sandbox with HDP 2.6.4
create or replace connection hive_conn to 'jdbc:hive2://192.168.42.1:10000' user 
'raj_ops' identified by 'raj_ops';

To test the connection, use the following statements:

Copy 
-- EXPORT test for Cloudera driver
export exa_syscat
    into jdbc driver = 'HiveCloudera' at hive_conn table exa_syscat created by
    'create table exa_syscat (schema_name varchar(128), object_name varchar(128),
 object_type varchar(15), object_comment varchar(2000))'
    replace;

-- IMPORT test for Cloudera driver
import into(schema_name varchar(128), object_name varchar(128), object_type varchar(15), 
object_comment varchar(2000))
    from jdbc driver = 'HiveCloudera' at hive_conn table exa_syscat;

Load data

Use IMPORT to load data from a table or SQL statement using the connection that you created.

The IMPORT statement allows you to use Kerberos connections in your connection string. For more details, see the example in connection_def.