Running the xgtd
server
Configuring xgtd
There are reasonable defaults provided for all of these configuration parameters.
It is recommended that you override these default values only after running xGT with the default values.
The only exceptions to this guidance is the security information: (i) the AWS credential information has no reasonable defaults for securely accessing data in S3 buckets, and (ii) to run with SSL encryption between the Python client and xgtd
server the configuration parameters for ssl
must be provided.
Configuring xgtd
is done using key-value pairs.
The pairs can be given either in configuration files written in JSON format or as command line arguments when launching xgtd
.
There are two standard locations for the configuration file: /home/xgtd/.xgtd.conf
and /etc/xgtd.conf
.
When launched, the xgtd
application will first look for a configuration file at /home/xgtd/.xgtd.conf
.
If the file doesn't exist, the xgtd
application will look at /etc/xgtd.conf
.
Only one of these configuration files will be read, and no message is given if neither of these files are found.
The xgtd
application will next process configuration files given as program arguments.
An arbitrary configuration file can be used by passing the following program argument to xgtd
: -c /path/to/xgtd.conf
.
Users may split configuration information into multiple files and repeat the -c
argument.
Users may also give a config file on the command line when a config file exists in one of the standard locations.
A config file in the standard location is processed first, and then files given on the command line are processed in the order they are given (left to right).
This results in later configuration for the same key overriding earlier configuration for that key.
xgtd
also supports providing a single key-value configuration item in the form of a program argument: -Dkey=value
or -Dkey:value
.
These -D
options can appear multiple times and repeated keys are set to the latest appearance in the command line (left to right).
Users can mix -c
and -D
options.
They are collectively processed in a left to right manner.
The following keys for configuring xGT are supported:
system.hostname
: string - Controls the hostname (or IP address) thexgtd
server will listen on for client connections. An IP address (string) or the string "localhost" are allowed. Defaults to "0.0.0.0".system.io_threads
: integer - Controls the number of I/O threads xGT uses per file. The number will never exceed the system cores count including hyperthreads. Defaults to 16.system.locale
: string - Locale for comparing strings. Defaults to C.system.max_memory
: integer - Controls the maximum amount of memory (in bytes) that xGT will use. The value given is capped at the available system RAM. Defaults to the available system RAM.system.pin_threads
: true/false - Controls whether the worker threads are pinned to a specific core. This means they won't be able to move around. Defaults to false.system.port
: integer - Controls the port thexgtd
server will listen on for client connections. Valid values are [0, 65535]. Defaults to4367
.system.ssl_root_dir
: string - The path to the root directory that holds the server's SSL certificates and private keys. Defaults to the home directory of the user running xgtd.system.usessl
: true/false - Controls whether to turn on ssl authentication between the client and server. Defaults to false.system.worker_threads
: integer - Controls the number of worker threads xGT uses. The number will never exceed the system cores count including hyperthreads. Defaults to the number of cores including hyperthreads on the system.stats.frame_stats
: true/false - Controls whether to count edge traversals globally for each frame. Defaults to false.stats.job_stats
: true/false - Controls whether to count edge traversals for jobs. Defaults to false.aws.access_key_id
: string - Used for S3 authentication if an.aws/credentials
file does not exist for the profile. Example:AKIAIOSFODNN7EXAMPLE
.aws.secret_access_key
: string - Used for S3 authentication if an.aws/credentials
file does not exist for the profile. Example:wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
.metrics.cache
: true/false - Controls whether the metrics cache is running and generating statistical information about each frame. These statistics are used for query order optimization. Defaults to false.
An example file containing xgtd
configuration file (/path/to/xgtd.conf
:
{
"system": {
"worker_threads": 16,
"io_threads": 10,
"port": 4369
},
"system.locale": "en_US.UTF-8"
}
Then, to launch xgtd
, you may do this command that uses this base configuration and supplements that with a definition for system.host:
$ xgtd -c /path/to/xgtd.conf -Dsystem.host=127.0.0.1
To launch xgtd
using the base configuration but overriding the worker_threads
configuration value:
$ xgtd -c /path/to/xgtd.conf -Dsystem.worker_threads=8
To launch xgtd
using multiple base configurations:
$ xgtd -c /path/to/xgtd_conf1.conf -Dsystem.port=4369 -c /path/to/xgtd_conf2.conf
Configuring xgtd
the old way (deprecated)
Some parameters of xGT can be configured with the JSON-formatted configuration file xgtd.conf
located on the computer where the server process is running.
The following variables for configuring xGT are supported:
worker_threads
: integer - Controls the number of worker threads xGT uses. The number will never exceed the system cores count including hyperthreads. Defaults to the number of cores including hyperthreads on the system.io_threads
: integer - Controls the number of I/O threads xGT uses per file. The number will never exceed the system cores count including hyperthreads. Defaults to 16.pin_threads
: true/false - Controls whether the worker threads are pinned to a specific core. This means they won't be able to move around. Defaults to false.max_memory
: integer - Controls the maximum amount of memory (in bytes) that xGT will use. The value given is capped at the available system RAM. Defaults to the available system RAM.locale
: string - Locale for comparing strings. Defaults to C.aws_access_key_id
: string - Used for S3 authentication if an.aws/credentials
file does not exist for the profile. Example:AKIAIOSFODNN7EXAMPLE
.aws_secret_access_key
: string - Used for S3 authentication if an.aws/credentials
file does not exist for the profile. Example:wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
.
Example xgtd.conf
file:
{
"worker_threads" : 4,
"pin_threads" : false
}
To configure xGT on AWS, login to the server and create the file /etc/xgtd.conf
or the file /home/xgtd/.xgtd.conf
.
The system will first look for the file /home/xgtd/.xgtd.conf
, and if unsuccessful, it will look for the file /etc/xgtd.conf
.
After creating or modifying the configuration file, the the xGT server must be restarted:
sudo systemctl restart xgtd
S3 credentials
When loading a file using the load
method and an s3:
protocol prefix, xGT will check the /home/xgtd/.aws/credentials
file for the two variables aws_access_key_id
and aws_secret_access_key
.
If the credentials file doesn't exist, xgt will check for the configuration variables aws.access_key_id
and aws.secret_access_key
.
The online AWS Access Keys document explains what these keys are and contains references to learn how to create and manage them.
(These values may also be specified at runtime in user code by passing them to a Connection
object.)
When reading the /home/xgtd/.aws/credentials
file, xGT also supports profile selection via the AWS_PROFILE
environment variable.
If no environment variable is found, xGT will use the default
profile.
Using an SSL secure channel
By default xGT uses an insecure channel, but an admin or user can enable a secure channel using SSL certifcates.
To run a secure server, pass the flags -s
(or --ssl
) and -d
(or --ssl_root_dir
) when starting the xgtd
executable.
The ssl_root_dir
argument should be the path to the root directory that holds the server's SSL certificates and private keys.
A user can also turn on the secure server by setting the configuration variable system.usessl
to True.
Setting the configuration variable system.ssl_root_dir
is an alternative way to specify the root directory that holds the server's SSL certificates and private keys.
xGT expects the following directory structure:
.
├── certs
│ ├── ca-chain.cert.pem
│ └── server.cert.pem
└── private
└── server.key.pem
To connect to an xGT server using SSL, the client needs to pass the following flags to the xgt.Connection()
method: ssl
, ssl_root_dir
, and ssl_server_cn
.
The ssl
flag needs to be set to true.
The ssl_root_dir
flag should be set to the root directory containing the SSL certificates and private keys.
The ssl_server_cn
flag should be set to the common name for the server listed on the server side SSL certificate.
The xGT client expects the following directory structure for SSL certificates and private keys:
.
├── certs
│ ├── ca-chain.cert.pem
│ └── client.cert.pem
└── private
└── client.key.pem