Sidecar Health Monitoring

Each sidecar instance exposes an HTTP route /health on port 9000 for health monitoring. It can be accessed calling the endpoint http://<instance-address>:9000/health and can be used by Load Balancers and container orchestrators to monitor sidecar instances' health. This route observes the health of the internal processes and returns a proper HTTP response code and a JSON blob containing extra information on the status of each of the sidecar services.

It is important to notice that the health monitoring refers to a single sidecar instance. A sidecar can be composed of multiple computing instances, thus all of them must be monitored in order to guarantee the health of the entire sidecar cluster.

The JSON blob returned by the /health route is the following:

{
  "status":"degraded",
  "components": {
    "alerter": {
      "status":"healthy"
    },
    "mysql-wire": {
      "status": "degraded",
      "error": "service is initializing",
    },
    ...
  },
  "timestamp":"0001-01-01T00:00:00Z"
}

The top-level status can be one of the following:

Sidecar Instance Status	HTTP Response Code	Description
healthy	200	All the monitored components in the instance are working properly and the it can receive requests normally.
degraded	200	Some services in the instance are not working, but it should be able to accept connections without issues if resiliency mode is active.
unhealthy	503	Not able to accept connections.
unknown	503	Instance components are yet to be tested.

Each components object status can be one the following:

Component Status	Description
healthy	Component is working as expected.
degraded	Component is not working as expected, but the error is not critical to the sidecar instance functionality.
unhealthy	Component is not working as expected and error is critical to the sidecar instance functionality.

As shown in the example JSON, when the component status is not healthy the status message will be accompanied by an error field containing more details.

Health API

The Cyral API in the control plane also allows you to query for the health of a specific sidecar cluster using the endpoint /v2/sidecars/${id}/health. The query will return a JSON object in the following format:

{
  "status": "HEALTHY"
}

The status in this result will be one of the following:

Status	Meaning
HEALTHY	All instances in that sidecar cluster are healthy
UNHEALTHY	All instances in that sidecar cluster are unhealthy, or the sidecar has no instances and has been deployed previously
UNKNOWN	There was never an instance in this sidecar cluster
DEGRADED	There is a mix of healthy, unhealthy and degraded instances in this sidecar cluster