8. Logging and Auditing

xGT provides two types of logging output: security audit logging and general logging. Each is configured separately, and the logs will be written to different locations. General logging provides a variety of information about the state of the xGT server and operations being performed by it. The recorded events can be helpful in monitoring xGT’s usage and in diagnosing bugs or other issues.

The security audit consists of a record of authentication and access control events. It is used to track accesses of data in the xGT server and may be helpful in analyzing security incidents. Events logged in the security audit include:

  • Authentication events from the Python client.

  • Requests to run TQL queries or other transactions on the server.

  • Security access control violations, such as a user attempting to read or modify data for which the user does not have permission.

8.1. General Logging

General logging in xGT is configured in the /etc/xgtd/xgtd.conf configuration file as described in Configuring the Server. The keys logging.filepath and logging.prefix control the log output location. General logging is divided into components, which group the log messages by type (logging component). A logging.level.X key sets the logging level of a component X.

Available logging levels are debug, info, warning, and fatal in order from least to most severe. Only messages of the same level as the chosen logging level or a more severe level than the chosen level will be logged. For example, to log only messages of severity greater or equal to warning for the component compiler, add the following line to the configuration file:

{
  "logging.level.compiler" : "warning"
}

The logging level fatal causes the xGT server to terminate. This level is only used during startup. Consequently, setting the logging level to fatal means that nothing will be logged after the server finishes its boot process.

8.1.1. Logging Components

The available components are:

  • xgt: General log messages about the xGT server.

  • config: All lookups of and changes to xGT config settings.

  • compiler: Operations taken by the compiler while processing a request.

  • io: Ingest and egest frames.

  • exception: Each exception thrown by the xGT server. This is logged at the DEBUG level.

  • query_optimization: Operations performed during query optimization.

  • metrics: Any errors in the computation of metrics used for query optimization. These are at the WARNING level.

  • runtime: Information about the parallel runtime, including threads, tasks, and memory use.

  • security: Security related log messages, including failure to load groups or labels at server startup. Note that this is separate from security audit logging.

8.2. Security Audit Logging

Security audit logging in xGT uses Apache log4cxx and must be configured with a log4cxx configuration file. The path to the log4cxx configuration file must be supplied in the /etc/xgtd/xgtd.conf configuration file using the key audit.config_file:

{
  "audit.config_file" : "path/audit.properties"
}

For more information on configuring xGT, see Configuring the Server. The rest of this section focuses on the configuration of the audit system in the file indicated by audit.config_file.

xGT sends security audit log messages to a single logger named xgtdaudit. The log4cxx configuration file must set up this logger by defining its logging level and by specifying where the messages sent to the logger are written:

log4j.logger.xgtdaudit=LEVEL, outputLocationName, outputLocationName

The level defines the severity of messages that will be recorded for the logger. Possible values are TRACE, DEBUG, INFO, WARN, ERROR and FATAL in order from least to most severe. If a logger is assigned level X, it will only log messages of severity equal to or greater than X. For example, if the xgtdaudit level is set to WARN, only messages marked WARN, ERROR, or FATAL will be written.

8.2.1. Appenders

The audit messages sent to a logger can be output to multiple destinations, each defined by an appender. The log4cxx configuration file should define one or more appenders. The following example defines a rolling file appender that rolls after a file size of 10 megabytes and keeps 100 backup files.

log4j.appender.rollingFile=org.apache.log4j.RollingFileAppender
log4j.appender.rollingFile.File=debug.log
log4j.appender.rollingFile.MaxFileSize=10MB
log4j.appender.rollingFile.MaxBackupIndex=100
log4j.appender.rollingFile.layout=PatternLayout
log4j.appender.rollingFile.layout.ConversionPattern=%d{ISO8601} [%p] %c (%t) {%x} - %m%n

Using the PatternLayout allows the output format to be customized using conversion characters. In the above example, the message consists of and ISO8601 formatted date (%d), the message level (%p), the logger name (%c), the name of the thread that generated the logging event (%t), the nested diagnostic context associated with the thread (%x), and the log message (%m%n).

Instead of defining a custom layout, pre-formatted JSON messages may be logged by choosing JSONLayout:

log4j.appender.rollingFile.layout=JSONLayout

The following appender types are supported by log4cxx. More detail is available at https://logging.apache.org/log4cxx/.

Name

Description

Usage Notes

ConsoleAppender

Writes log messages to stdout or stderr.

The output target is specified by target=System.out or target=System.err.

FileAppender

Writes log messages to a file.

The file name is specified by File.

RollingFileAppender

Writes log messages to a file and rolls over when the file reaches a chosen size.

The roll-over file size is specified by MaxFileSize. The number of backup files kept is specified by MaxBackupIndex.

DailyRollingFileAppender

Writes log messages to a file and rolls over at a chosen frequency.

The roll-over frequency is specified by DatePattern. It may be yearly, monthly, daily, hourly, or every minute. To roll every minute, use DatePattern=’.’yyyy-MM-dd-HH-mm. To roll daily use ‘.’yyy-MM-dd. The number of backup files kept is specified by MaxBackupIndex.

SocketAppender

Sends messages to a remote log server.

NTEventLogAppender

Appends messages to NT EventLog.

SyslogAppender

Sends messages to a remote syslog daemon.

8.2.2. Sample log4cxx Configuration

The configuration file below outputs xgtdaudit logger messages with level greater than or equal to INFO to a file that rolls every hour and to the console. The file names are relative to the location from which the xGT server is running.

# Define an appender named rollingFile.
log4j.appender.rollingAudit=org.apache.log4j.DailyRollingFileAppender
log4j.appender.rollingAudit.File=audit.json.log
log4j.appender.rollingAudit.DatePattern='.'yyyy-MM-dd-HH
log4j.appender.rollingAudit.MaxBackupIndex=10
log4j.appender.rollingAudit.layout=PatternLayout
log4j.appender.rollingAudit.layout=JSONLayout

# Define an appender named stdoutAppender.
log4j.appender.stdoutAppender=ConsoleAppender
log4j.appender.stdoutAppender.target=System.out
log4j.appender.stdoutAppender.layout=PatternLayout
log4j.appender.stdoutAppender.layout.ConversionPattern=%d{ISO8601} [%p] - %m%n

# Set up the xgtdaudit logger.
log4j.logger.xgtdaudit=INFO, rollingAudit, stdoutAppender
log4j.additivity.xgtdaudit=false

Note that the xgtdaudit logger’s additivity setting is set to false. If set to true, which is the default, and if a root logger is defined with appenders, then all log messages will also be output to the root logger’s appenders. More information about log4xx can be found at https://logging.apache.org/log4cxx/.