Database Management

A database comprises one or more clusters which handle all query operations. A cluster is a group of servers each having its own main memory (RAM) and CPUs. All queries are executed on a cluster, which has its own size that defines the compute power based on the number of VCPUs and main memory. By adding multiple clusters, it is possible to separate workloads between teams or tenants and allow for greater concurrency.

The data and metadata for the database is stored in a central data store, such as an S3 bucket, not in the underlying clusters. All clusters in a database access the same underlying data and metadata. Changes that are persisted from one cluster are persisted in the data store, and are also persistent for all other transactions in the database, regardless of which cluster they are connected to.

The first cluster of a database has a special role and is called the main cluster. All other clusters are called worker clusters. The main cluster communicates with all worker clusters and ensures they have a consistent view of transactions and metadata. The communication between clusters is metadata-only; every cluster has a direct connection with end users and the central data store. You can connect to the main cluster like any other cluster; however, the main cluster must be running when worker clusters are running. Similarly, stopping the main cluster will also stop all worker clusters in the database.