Improved system behavior in case of no space left on device (avoid emergency shutdown)

Details

Detail name Value
Changelog Number 34
Type Improvement
Status Resolved
Fix Versions Exasol 6.1.0
Resolution Date 2018-03-16

In previous versions the database has executed an emergency shut down in case of no space left on device. There were two possible scenarios:

1. Execute automatic database restart (recreation of temporary volume frees disk space)
2. Do not execute automatic database restart (recreation of temporary volume does not free any disk space)

The situation could be detected with the following monitoring messages:

Message type Message Remark
EXAOP_ERROR No space left on device.  
EXAOP_ALERT Emergency shutdown: Process 1 requested emergency shutdown. Reason: No space left on storage. Automatic restart seems not to be useful. Manual intervention is needed. automatic restart does not free disk space

In the current version the system behavior has been improved. Instead of executing an emergency shutdown, the database kills the session which has consumed the most database memory (temporary + persistent dbram). This situation can be detected with the following monitoring message:

Message type Message Remark
EXAOP_WARNING Full disk prevention: Session 1595105668223106290 was killed because system running out of disk space. Killed session had 15.6 MiB temporary and 1039.9 MiB persistent data allocated.  

It is still possible that the database has to execute an emergency shutdown (in case of no session kill creates free disk memory). However, this scenario should be very rare and indicates an incorrect database usage. This situation can be detected with the following monitoring message:

Message type Message Remark
EXAOP_ALERT Emergency shutdown: Process 1 requested emergency shutdown. Reason: Full disk handling cannot ensure normal operation of database. Automatic restart seems not to be useful. Manual intervention is needed. normal operation of database not possible