Skip to main content
Version: v4.16

Sidecar Logging

Cyral sidecars output two types of logs: data activity logs, which can be used to monitor database activity, and diagnostic logs, which can be used to monitor the health and activity of the sidecar itself.

The sidecar emits both data activity logs and diagnostic logs to its stdout and stderr streams respectively. For multi-container sidecars, logs are aggregated by the cyral_log_shipper container and output to that container's standard streams.

Data Activity Logs​

Data activity logs describe queries and other operations performed on your data, and detail who is accessing which data and how they are doing so. Data activity logs can be used to monitor and visualize activity on your data repositories.

Data activity logs are structured JSON documents, which include information about database activity such as executed SQL statements, user identity information, and whether any policies were violated, etc. Please see specification below for a complete breakdown on the structure and content of the data activity logs.

Cyral admins can customize which types of database events generate data activity logs on a per-repository basis. Additionally, Cyral can be customized to redact sensitive values from data activity logs.

Data Activity Logs Specification​

Cyral data activity logs follow the structure shown below. The particular set of logged fields you'll see in your logs depends on what you have specified in your repository's log settings.

For help setting up logging, see log integrations.

The example below shows an SSO user, bob@hhiu.us, who logged in using the repo account dbuser001 and ran a SELECT query on the invoices-staging PostgreSQL repository. The query "triggered" two policies, one of which caused a mask to be applied to the card_number field, while the other imposed a row limit of 5 rows on the result of the query. Since this policy was not enforced, and the query returned 54 rows, the policy was violated.

{
    "time": "2023-07-06T03:52:42.000000Z",
    "activityId": "127.0.0.1:42118:1688615546256057574:7:1",
    "activityTime": "2023-07-06 03:52:42.858413348 +0000 UTC",
    "activityTimeNanos": 1688615562858413348,
    "activityTypes": [
        "query",
        "fullTableScan"
    ],
    "identity": {
        "endUser": "bob_hardy",
        "endUserEmail": "bob@hhiu.us",
        "group": "Finances",
        "userGroups": [
            "Contractors", 
            "Finances"
        ],
        "repoUser": "dbuser001",
        "dbRole": "readOnly"
    },
    "repo": {
        "id": "2SBMUz9xU0SIwXh3iXekMa6UecG",
        "name": "invoices-staging",
        "type": "postgresql",
        "host": "172.18.0.4",
        "port": 5432
    },
    "client": {
        "connectionId": "127.0.0.1:42118:1688615546256057574",
        "connectionTime": "2023-07-06 03:52:26.256057574 +0000 UTC",
        "connectionTimeNanos": 1688615546256057574,
        "host": "127.0.0.1",
        "port": 42118,
        "applicationName": "psql",
    },
    "connectionAuthorization": {
        "authorized": true,
        "reason": "Authorized by access rule for group Finances"
    },
    "sidecar": {
        "id": "2SBEUstTIgl2d1LDAVR3rVDE3Hq",
        "name": "sidecar-east-2",
        "autoScalingGroupInstance": "SINGLETON"
    },
    "request": {
        "statement": "SELECT * FROM playground.playground.transactions",
        "statementType": "SELECT",
        "isSensitive": true,
        "datasetsAccessed": [
            {
                "database": "playground",
                "dataset": "playground.transactions",
                "accessType": "read"
            }
        ],
        "fieldsAccessed": [
            {
                "database": "playground",
                "field": "playground.transactions.card_number",
                "label": "CCN",
                "tags": [
                  "PCI"
                ],
                "accessType": "read"
            },
            {
                "field": "playground.transactions.email",
                "label": "EMAIL",
                "accessType": "read"
            }
        ],
        "searchPath": [
            "public",
            "playground"
        ],
        "sessionParameters": {
            "database_name": "playground"
        }
    },
    "response": {
        "message": "Ok",
        "isError": false,
        "records": 54,
        "bytes": 11214,
        "executionTime": "1.340189ms",
        "executionTimeNanos": 1340189
    },
    "policyViolated": true,
    "triggeredPolicies": [
      {
        "id": "2f8ChKVHtHThQ7WxvZEM03gXPIt",
        "name": "mask credit card numbers",
        "type": "legacy",
        "enforce": true,
        "violated": false,
        "result": {
          "masks": {
            "playground.transactions.card_number": "constant_mask(\"****\")"
          }
        }
      },
      {
        "id": "2SE8mSmzCCE2zEYHO26ItCqe4QC",
        "name": "row limit reads of credit card numbers",
        "type": "rego",
        "enforce": false,
        "violated": true,
        "result": {
          "rowLimit": 5
        },
        "violations": [
          {
            "reason": "query returned 54 rows, exceeding the per-query row limit for one or more fields accessed by the query",
            severity: "high"
          }
        ]
      }
    ],
    "policyResult": {
      "masks": {
        "playground.transactions.card_number": "constant_mask(\"****\")"
      },
      "rowLimit": 5
    },
    "stream": "stdout",
    "svc": "pg-wire"
}

  • Each log entry begins with a header section that includes:

    • activityId: Unique identifier for this log entry.
    • activityTime: When this event began.
    • activityTimeNanos: Duration of the event.
    • activityTypes: What sort of event this entry describes. This may be query for a database query, authenticationFailure when a user fails to connect to a repository, authorizationFailure when a user is not authorized by an access rule, newConnection when a user connects to a repository, and closedConnection when they disconnect. Other possible events are: dml, privileged, fullTableScan, authenticationSuccess, networkAccessViolation, portScan, connectionError, multiplexedConnectionFailure.
    • svc: The sidecar service which generated this log message.

  • The identity block shows who performed the action:

    • endUser: The user who authenticated; usually an SSO user.
    • endUserEmail: Email address of the user.
    • repoUser: The native repo account used for the database session.
    • group: (Deprecated) This field will be filled in the following cases:
    • userGroups: End user's SSO groups. For users' group affiliations to be tracked using Looker and Tableau, please refer to the following pages: Looker and Tableau.
    • dbRole: (PostgreSQL, SQLServer, Oracle, Redshift, Denodo, Snowflake and DynamoDB only) The user database role. In some databases, the same user (repoUser) may assume different database roles, e.g. admin or readOnly, with different database-level permissions associated with each one of them.
    • authDB: (MongoDB only) The authentication database for the user, i.e., the database that the user exists in. For MongoDB, the user's name and the authentication database serve as a unique identifier for that user.

  • The cyralContext block has the fields user, userGroup, serviceName and attributes. It will be populated in case of Cyral Context-based service account resolution and it will contain the Cyral Context data passed in by the client along with the command or query executed.

  • The repo block shows:

    • id: Data repository ID in Cyral.
    • name: Data repository name in Cyral.
    • type: Type of repository, such as PostgreSQL or MongoDB.
    • host: Network host of the repository.
    • port: Network port of the repository.

  • The client block shows:

    • connectionId: A unique ID for the client connection.
    • connectionTime: Timestamp when client connection was initiated.
    • connectionTimeNanos: Duration of connection.
    • host: Client's network host.
    • port: Client's network port.
    • applicationName: Client application that sent the command.

  • The connectionAuthorization block shows whether the connection was authorized, why it was authorized or not (reason) and the accessConditionEvalResults, a summary of the evaluation of access conditions. Note that the reason may give some positive reason for authorization while overall authorization still fails because some access condition failed.

  • The connectionError field will contain the error message if a connection error occurred.

  • The networkAccessControl will be populated with the reason that access was granted or denied, unless the Network Shield feature is disabled.

  • The sidecar block shows which Cyral sidecar brokered the request:

    • id: Sidecar ID.
    • name: Sidecar name.
    • autoScalingGroupInstance: ID of the sidecar cluster node that brokered the request.

  • The request block shows details about the command or query that was run/attempted:

    • statement: The command or query itself. If you have turned on log redaction, then sensitive data will be omitted from the logged statement and replaced with a value of ${cyral-redact}.
    • preparedStatement: If the command was a PREPARE statement, statement will contain a PREPARE prefix and preparedStatement will have the prepared command.
    • cursorStatement: The actual statement associated with the cursor that will be executed through fetch statements.
    • statementType: Type of command.
    • preparedStatementType: If the command was a PREPARE statement, statementType will be PREPARE and preparedStatementType will contain the type of the statement being prepared (e.g. SELECT, INSERT, etc.).
    • cursorStatementType: The type of statement associated with the cursor that will be executed.
    • storedProcedureName: The name of the stored procedure that is being invoked by the request.
    • The statementData block contains extra information specific to the executed/attempted action.
    • isSensitive: Did this action affect a table or field considered to be sensitive in your Cyral configuration?
    • The datasetsAccessed block lists the tables and collections affected by this action. Inside this block, each accessed data location is listed as a database and a dataset with an accessType showing whether the action was a read, update, delete, or other action. The database field is populated only when the query explicitly has this information, otherwise is empty. Each accessed data location in datasetsAccessed can also contain the isMetadata field (MySQL, PostgreSQL, SQLServer and Redshift only), which will be set to true if the dataset contains database metadata, i.e., information about the database server such as the names of databases or tables, the data types of columns, or access privileges. Other terms that are sometimes used for this information are data dictionary and system catalog.
    • The fieldsAccessed block lists the labeled columns and fields, i.e., the fields present in your data map, within each dataset affected by this action. Each is a database and a field with an accessType and a label or tags in your data map. The database field is populated only when the query explicitly has this information, otherwise is empty. If masking has been applied to the field, the mask function used is also shown here.
    • rewrittenStatement: The rewritten command or query when dataset rewrites and/or masking have been applied. For Oracle, PostgreSQL, Redshift, Denodo, Snowflake and SQLServer this field is also populated in cases where row limiting and rate limiting are enforced, which are achieved by adding a suitable LIMIT clause for each dialect to the query.
    • The filters block lists the filters included in the statement (e.g. in the WHERE clause).
    • limit (MongoDB, MySQL, PostgreSQL, SQLServer, Oracle, Redshift and Denodo only) contains the values for the limit and offset elements of the command.
    • error: If an error occurred when processing this action, this field will describe the error. The type of the error can be Parse Error, Analysis Error, Empty Request Error or Request Too Big.
    • dremioQueryId: (Dremio only) The ID generated by the Dremio server for the query being executed.
    • defaultSchema: (Dremio only) The currently selected schema in your session.
    • database: (MongoDB only) The MongoDB database on which the current query is being executed.
    • baseStatement: (MongoDB only) The actual command associated with the cursor that will be executed through GET MORE requests.
    • baseStatementType: (MongoDB only) The type of command associated with the cursor that will be executed (e.g. FIND, AGGREGATE).
    • schema: (MySQL only) The currently selected database in the MySQL session.
    • informationSchemaAccessed (MySQL only) indicates whether the INFORMATION_SCHEMA database was accessed by the command.
    • The userConfigParameters block (PostgreSQL, SQLServer, Oracle, Redshift and Denodo only) captures custom configuration parameters that the database user has provided using SET or SELECT set_config statements in their queries. This facility captures parameters that are written as dot-separated strings. For example, userConfigParameters will show the contents of the JWT token if the API client set it before running the query.
    • searchPath: (PostgreSQL, SQLServer, Oracle, Redshift and Denodo only) List of schemas the database system looks in when unqualified tables are queried.
    • sessionParameters (PostgreSQL, SQLServer, Oracle, Redshift and Denodo only) stores session-related data. The key is the session parameter ID, for example, client_identifier or database_name, while the value shows the actual data associated with that session variable in the database.

  • The response block describes the database response to the command, including any message returned and whether this command resulted in an error (isError). This block indicates how many records and bytes were returned and how long the command took to run (executionTime and executionTimeNanos).

  • The policyViolated field is a boolean that indicates whether any policy was violated as a result of the command or query. such as exceeding a row limit when the policy is not enforced.

  • The blockedQuery is a string field present only when a query is blocked due to a policy violation. Example value: requestBlocked.

  • The triggeredPolicies block lists the policies that were triggered by this command. For each policy, information about the policy, the result of evaluating the policy, and whether the policy was violated is included. Here is a breakdown of the fields within a triggered policy:

    • id: The unique identifier of the policy.
    • name: The name of the policy.
    • type: The type of the policy (e.g. rego for policy template based policies).
    • enforce: Indicates whether the policy is enforced.
    • violated: Indicates whether the policy was violated.
    • violations: A list of violations with the following fields:
      • reason: A human friendly reason for the violation.
      • severity: The severity of the violation (e.g., low, medium, high).
    • result: The outcome of the policy evaluation. This is an object that may include:
      • block: Indicates whether the request has to be blocked.
      • rateLimits: Rate limits per label.
      • rowLimit: The row limit imposed by the policy.
      • masks: Details of any masking applied by the policy.
      • datasetRewrites: Dataset rewrites per dataset.
      • alerts: Alerts requested by the policy. Each alert includes:
        • severity: The severity of the alert (e.g., low, medium, high).
        • message: The alert message.

  • The policyResult block shows the combined result of all triggered policies. This block contains the same fields as the result field in the triggeredPolicies block, but encapsulates the composed result of all triggered policies. Please refer to the Policy Evaluation section for details on how the results of multiple policies are composed.

What Constitutes a Policy Violation?​

A policy violation occurs under the following conditions:

  1. Forbidden Field Access: If a user fires a query that accesses a field forbidden to them, it is considered a policy violation, regardless of whether the query was eventually blocked or not.
  2. Row/Rate Limit Exceeded: If a user fires a query that needs to be row or rate limited and the limit is not enforced due to configuration (resulting in more rows or a higher rate than allowed), it is considered a policy violation.

Diagnostic Logs​

Diagnostic logs are sidecar application-level logs, also sometimes referred to as "debug logs". These logs may be useful for diagnosing sidecar issues, analyzing sidecar performance, and general troubleshooting.

While diagnostic logs are still structured JSON documents, their structure is simpler than data activity logs. In general, a sidecar diagnostic log message may look like:

{
  "time": "2006-01-02T15:04:05.000000Z",
  "instanceId": "some-sidecar-instance-id",
  "level": "info",
  "msg": "Some log message",
  "stream": "stderr",
  "svc": "pg-wire"
}
  • time is the timestamp the log message was generated.
  • instanceId is the sidecar instance identifier which generated the log (useful in multi-instance sidecar clusters)
  • level is the log level. Valid log levels are trace, debug, info, warning, error, fatal, and panic.
  • msg is the log message.
  • stream is the standard stream on which the log was generated. This should always be stderr.
  • svc is the name of the sidecar microservice which generated the log message.

Log Management​

Cyral gives you the ability to send sidecar logs to various destinations such as Splunk, AWS CloudWatch, and many others. Please refer to the "Integrations -> Logs" section for complete details.

When using Cyral's logging integrations features, the log management destination for both the sidecar's data activity logs and diagnostic logs can be configured independently in the sidecar's Logging settings: on the Control Plane, navigate to Sidecars -> {sidecar name} -> Settings (tab) -> Logging.

Additionally, sidecar logs can be aggregated and managed using any log management tooling such as FluentD, Fluent Bit, etc. deployed to your infrastructure by consuming the standard output/error streams of the sidecars.

Learn more​