Adding New Packages to Existing Script Languages

You can download the current version of the pre-built container from the release section on GitHub.

All containers specify the minimum Exasol version they are compatible with. You can use them with later version of Exasol than specified. For example, the container standard-EXASOL-6.0.0-release-....tar.gz is compatible with Exasol 6.0.0 and later. For more information about the containers and their compatibility, refer to the Exasol Script Language Flavors documentation.

For instructions on how to upload the new container to BucketFS, see Manage Buckets and Files in BucketFS. The following example using curl defines an environment for the examples in the subsequent steps:

local_container_file = "path/to/container.tar.gz" 
bucketfs_host = "192.168.6.75" 
bucketfs_port = "1234" 
write_password = "write-password" 
bucketfs_name = "bfsdefault"
bucket_name = "default" 
path_in_bucket = "path/in/bucket/" 
container_file_in_bucket = "container.tar.gz"
container_name = "container"
protocol = "HTTP" # or HTTPS 

curl -vX PUT -T "$local_container_file" "$protocol://w:$write_password@$bucketfs_host:$bucketfs_port/$bucket_name/$path_in_bucket$container_file_in_bucket

The uploaded container must have the file extension .tar.gz, .tar.bz2, or .zip. Otherwise the container will not get extracted and its contents will not be available in BucketFS.

The value for path_in_bucket can be empty. If it is not empty it must end with a forward slash /.

When the container is uploaded to BucketFS it will be distributed and extracted to all nodes. After a few minutes, you can activate it and use it.

We recommend activating the container for your session first and testing it before activating it system-wide.

To activate a new container you must edit the session parameter SCRIPT_LANGUAGES. The SCRIPT_LANGUAGES parameter defines a list of aliases for script language/container combinations. You can change a script alias using ALTER SESSION (session-wide) and ALTER SYSTEM (system-wide) commands. Changing the script alias is especially useful if you have multiple language versions in parallel, or when migrating from one version to another.

We recommend adding new aliases for non-standard containers and only overwrite the built-in defined aliases if necessary.

The current session and system parameters are available in the EXA_PARAMETERS system table.

SELECT session_value FROM exa_parameters
WHERE parameter_name='SCRIPT_LANGUAGES';

The default values are internal macros to make the parameter compact and to dynamically use the last installed version. For example, the alias for bultin_java would look like the following: 

JAVA=localzmq+protobuf:///bfsdefault/default/EXAClusterOS/ScriptLanguages-release-standard-exasol-<version>/?lang=java#/buckets/bfsdefault/default/EXAClusterOS/ScriptLanguages-release-standard-exasol-<version>/exaudf/exaudfclient

In the next section, we explain how you can define your own script alias for another script language container.

Define Your Own Script Aliases

For containers derived from the script language containers on GitHub, you can use exaslct to build the ALTER SESSION statement by running it with the command "generate-language-activation" and specifying the bucket details. This method is less error-prone than writing it from scratch. For more information, see the documentation on GitHub.

The SCRIPT_LANGUAGES parameter is a string that lists all active script aliases separated with spaces. Each script alias consists of an ALIAS_NAME and a LANGUAGE_URL or the builtin_* macro.

The syntax for the SCRIPT_LANGUAGES parameter is defined as follows:

<ALIAS_NAME_1>=<LANGUAGE_URL_1> <ALIAS_NAME_2>=<LANGUAGE_URL2_> ...'

A script alias cannot contain spaces.

The LANGUAGE_URL contains the following information for the database:

• path_to_linux_container: BucketFS path to the Linux container which is used for the root file system of the UDF.
• path_to_client: BucketFS path to the script client. Usually, this path should be a sub-directory of the path to the Linux container.
• client_param_list: Parameter list for the script client.

The complete script alias is as shown below:

<alias>=localzmq+protobuf:///<path_to_linux_container>/[?<client_param_list>]#<path_to_client>

For containers derived from our script language containers on GitHub, you can use the following template to create your language URL, or use exaslct to generate the command for you:

<alias>=localzmq+protobuf:///<bucketfs_name>/<bucket_name>/<path_in_bucket><container_name>/?lang=<language>#buckets/<bucketfs_name>/<bucket_name>/<path_in_bucket><container_name>/exaudf/exaudfclient[_py3]

Exasol currently provides the following language parameter options: 

• java
• r
• python

To switch between Python 2 and Python 3 you need to change the name of the script client from exaudfclient to exaudfclient_py3.

The container_name corresponds to the file name of the container in the BucketFS without the file extension. The reason for this is that the BucketFS extracts files with the file extension .tar.gz, .tar.bz2 or .zip.

The value for path_in_bucket can be empty. However, if it is not empty it must end with a '/'.

Example

Based on the contents and examples from above, we get the following ALTER SESSION command: 

ALTER SESSION SET SCRIPT_LANGUAGES='PYTHON=builtin_python PYTHON3=builtin_python3 R=builtin_r JAVA=builtin_java MY_PYTHON=localzmq+protobuf:///bfsdefault/default/path/in/bucket/container/?lang=python#buckets
/bfsdefault/default/path/in/bucket/container/exaudf/exaudfclient';

Next, we add the new script alias MY_PYTHON with the script language container in container.tar.gz and python as the language to the SCRIPT_LANGUAGES parameter. You can now use this in a UDF by creating a new UDF with CREATE SCRIPT command and specify the new script as MY_PYTHON. For example, you can create a UDF as shown below: 

--/
CREATE MY_PYTHON SCALAR SCRIPT my_python_udf(input_parameter VARCHAR(2000))
RETURNS VARCHAR(2000) AS

def run(ctx):
  return ctx.input_parameter
/