Monitoring

This section describes how you can monitor your Exasol 8 system through system log messages.

Logs in Exasol 8

Exasol 8 does not have an internal syslog service. Messages generated by the components in Exasol 8 are instead forwarded to the host syslog service. To monitor your Exasol 8 system, you can use a third-party syslog management/monitoring tool to subscribe to the relevant syslog messages.

For more information about how to set up logging, refer to the documentation for the respective syslog tool.

Syslog message structure

The syslog messages generated by the Exasol services have the following structure:

syslog message structure

The IDENT field in syslog contains the name of the Exasol service that is the source of the syslog message.

Source Description
Authentication Attempts to authenticate to the host.
ConfD Events in the ConfD administration interface.
DWAd Events related to a specified database, such as starting and stopping the database.
Health

Events related to the performance of the host system. This does not include storage devices.

Suggested values to monitor are: load (CPU load), mem (RAM utilization), and swap (swap utilization).

The message part always starts with the content of the variable EXA_NODE_ID, which is the physical ID of the node, followed by a colon. For example, 11: if the message comes from node n11.

Example:
May 30 08:03:18 ip-10-0-5-202 Authentication: 11: [confd] Successful authentication with username root
May 30 08:03:18 ip-10-0-5-202 Health: 11: Load 0.42431640625:0.83349609375:0.71142578125; freq: 2500.0, max 3202.069; cpus: 2; mem: 13.7159 GiB of 15.3379 GiB free; swap: 10 GiB of 10 GiB free; hp: 0 B of 0 B free, 0 B rsvd, 0 B anon; cache: 445.8359 MiB
...
May 30 08:05:01 ip-10-0-5-202 ConfD: 11: n0011 User root submitted db_list (11.111)
May 30 08:05:01 ip-10-0-5-202 Health: 11: Got the db list: ['Exasol']
...
May 30 08:07:27 ip-10-0-5-202 DWAd: 11: [Exasol] User 0 requests new system setup.
May 30 08:07:27 ip-10-0-5-202 DWAd: 11: [Exasol] User 0 requests startup of system.
May 30 08:07:28 ip-10-0-5-202 DWAd: 11: [Exasol] System started successfully in partition 75.

Logging

Facilities

All log messages generated by Exasol services are sent to the facility LOCAL0. You can use rsyslog to write all log messages containing this facility to a separate log file.

For example, to forward all Exasol syslog messages to the file exasol.log, add the following line to /etc/rsyslog.conf on the host:

local0.* /var/log/exasol.log

You can forward log messages over TCP to an external machine that runs a syslog service. For example:

local0.* @@syslog.example.com:514

Priorities

Syslog messages from Exasol services use all priorities that are defined in syslog:

LOG_EMERG     A panic condition.  This is normally broadcast to all users.
LOG_ALERT     A condition that should be corrected immediately, such as a corrupted system database.
LOG_CRIT      Critical conditions, e.g., hard device errors.
LOG_ERR       Errors.
LOG_WARNING   Warning messages.
LOG_NOTICE    Conditions that are not error conditions, but should possibly be handled specially.
LOG_INFO      Informational messages.
LOG_DEBUG     Messages that contain information normally of use only when debugging a program.

You can send Exasol syslog messages with specific priorities to different log files by combining facility and priority in rsyslog.conf. For example:

local0.err /var/log/exasol-errors.log

For more information about how to configure the rsyslog utility, refer to the rsyslog.conf manpage.

Thresholds

Since Exasol 8 does not have an internal syslog service, you have to set the desired warning thresholds for the monitored values in your syslog management tool. The actual threshold values to use depends on your system configuration.

For example, a good starting point for calculating a warning threshold for CPU load is number of threads per data node * 1.5.

In a scenario where each data node has 2 sockets with 6 cores and hyperthreading, the calculation would be:

2 x 6 x 2 = 24 threads -> 24 * 1.5 = 36