Adding New Packages to Existing Script Languages

You can download the current version of the pre-built container from our release section on GitHub. All containers specify a minimal Exasol version they are compatible with, and you can use them with a higher version of Exasol than specified. For example, the container "standard-EXASOL-6.0.0-release-....tar.gz" is compatible with Exasol 6.0.0 and above versions. For more information about the containers and their compatibility, please refer to Exasol Script Language Flavors documentation.

To upload the new container into your BucketFS, you can follow the instructions provided in the Access Files in BucketFS section under BucketFS. The following example with curl defines an environment for the examples in the subsequent steps:

local_container_file = "path/to/container.tar.gz" 
bucketfs_host = "192.168.6.75" 
bucketfs_port = "1234" 
write_password = "write-password" 
bucketfs_name = "bfsdefault"
bucket_name = "default" 
path_in_bucket = "path/in/bucket/" 
container_file_in_bucket = "container.tar.gz"
container_name = "container"
protocol = "HTTP" # or HTTPS 

curl -vX PUT -T "$local_container_file" "$protocol://w:$write_password@$bucketfs_host:$bucketfs_port/$bucket_name/$path_in_bucket$container_file_in_bucket

It is important to upload the container with the file extension .tar.gz, .tar.bz2, or .zip, else the container will not get extracted, and its content will not be available in the BucketFS.

The value for path_in_bucket can be empty. However, if it is not empty it must end with a '/'.

On uploading the container to the BucketFS, it will be distributed and extracted to all nodes. After a few minutes, you can activate it and use it. We recommend activating the container for your session first and testing it thoroughly. Once you confirm it works, you can activate it system-wide.

To activate a new container, you need to adjust the session parameter SCRIPT_LANGUAGES. The SCRIPT_LANGUAGES parameter defines a list of aliases for script language and container combinations. You can change the script alias using ALTER SESSION (session-wide) and ALTER SYSTEM (system-wide) commands. Changing script alias is especially useful in the case of multiple language versions in parallel, or migrating from one version to another. We recommend adding new aliases for non-standard containers and only overwrite the built-in defined aliases if necessary.

The current session and system parameters are available in the EXA_PARAMETERS system table.

SELECT session_value FROM exa_parameters
WHERE parameter_name='SCRIPT_LANGUAGES';

The default values are internal macros to make the parameter compact and to dynamically use the last installed version. For example, the alias for bultin_java would look like the following: 

JAVA=localzmq+protobuf:///bfsdefault/default/EXAClusterOS/ScriptLanguages-release-standard-exasol-<version>/?lang=java#/buckets/bfsdefault/default/EXAClusterOS/ScriptLanguages-release-standard-exasol-<version>/exaudf/exaudfclient

In the next section, we explain how you can define your own script alias for another script language container.

Define Your Own Script Aliases

For containers derived from our script language containers on GitHub, you can use exaslct to build the ALTER SESSION statement by running it with the command "generate-language-activation", and specifying all of the bucket details. This method is less error-prone than writing it from scratch. For more information, please view our GitHub Documentation.

The SCRIPT_LANGUAGES parameter is a string that lists all active script aliases separated with spaces. Each script alias consists of an ALIAS_NAME and a LANGUAGE_URL or the builtin_* macro.

The syntax for the SCRIPT_LANGUAGES parameter is defined as follows:

<ALIAS_NAME_1>=<LANGUAGE_URL_1> <ALIAS_NAME_2>=<LANGUAGE_URL2_> ...'

A script alias cannot contain spaces.

The LANGUAGE_URL contains the following information for the database:

• path_to_linux_container: BucketFS path to the Linux container which is used for the root file system of the UDF.
• path_to_client: BucketFS path to the script client. Usually, this path should be a sub-directory of the path to the Linux container.
• client_param_list: Parameter list for the script client.

The complete script alias is as shown below:

<alias>=localzmq+protobuf:///<path_to_linux_container>/[?<client_param_list>]#<path_to_client>

For containers derived from our script language containers on GitHub, you can use the following template to create your language URL, or use exaslct to generate the command for you:

<alias>=localzmq+protobuf:///<bucketfs_name>/<bucket_name>/<path_in_bucket><container_name>/?lang=<language>#buckets/<bucketfs_name>/<bucket_name>/<path_in_bucket><container_name>/exaudf/exaudfclient[_py3]

Exasol currently provides the following language parameter options: 

• java
• r
• python

To switch between Python 2 and Python 3 you need to change the name of the script client from exaudfclient to exaudfclient_py3.

The container_name corresponds to the file name of the container in the BucketFS without the file extension. The reason for this is that the BucketFS extracts files with the file extension .tar.gz, .tar.bz2 or .zip.

The value for path_in_bucket can be empty. However, if it is not empty it must end with a '/'.

Example

Based on the contents and examples from above, we get the following ALTER SESSION command: 

ALTER SESSION SET SCRIPT_LANGUAGES='PYTHON=builtin_python PYTHON3=builtin_python3 R=builtin_r JAVA=builtin_java MY_PYTHON=localzmq+protobuf:///bfsdefault/default/path/in/bucket/container/?lang=python#buckets
/bfsdefault/default/path/in/bucket/container/exaudf/exaudfclient';

Next, we add the new script alias MY_PYTHON with the script language container in container.tar.gz and python as the language to the SCRIPT_LANGUAGES parameter. You can now use this in a UDF by creating a new UDF with CREATE SCRIPT command and specify the new script as MY_PYTHON. For example, you can create a UDF as shown below: 

--/
CREATE MY_PYTHON SCALAR SCRIPT my_python_udf(input_parameter VARCHAR(2000))
RETURNS VARCHAR(2000) AS

def run(ctx):
  return ctx.input_parameter
/