Scalable graph analytics using xGT

The Trovares xGT application is a property graph engine designed to support extremely large in-memory graphs. Our analytics tool provides large graph search capabilities for data scientists and business analysts. With rapid ingest rates and the ability to execute complex queries driven by Python scripts, xGT provides unprecedented performance and scalability, especially when processing very large data.

---

• 1. What Is xGT?
  • 1.1. Architecture
  • 1.2. Client
  • 1.3. Server
  • 1.4. Quick Start Guide
• 2. Managing an xGT Site
  • 2.1. Installing xGT
    • 2.1.1. Supported Environments
    • 2.1.2. Licensing
  • 2.2. Configuring the xGT Server
    • 2.2.1. Configuring Groups and Labels
    • 2.2.2. Configuring Administrator Privileges
    • 2.2.3. Configuring the locale
    • 2.2.4. Using an SSL Secure Channel
  • 2.3. Enterprise Data Movement
    • 2.3.1. Common Usage Patterns
    • 2.3.2. Configuring the Data Sandbox
    • 2.3.3. Granting Data Sandbox Access to Users
  • 2.4. Memory and Compute Resources
    • 2.4.1. Monitoring Usage
    • 2.4.2. Setup and Configuration On-premises
    • 2.4.3. xgtd Using the Whole Server Platform
    • 2.4.4. xgtd Sharing the Server Platform
    • 2.4.5. Multiple xgtd Instances Running on a Server Platform
  • 2.5. User Authentication
    • 2.5.1. Group Membership
    • 2.5.2. PAM Setup
    • 2.5.3. GSSAPI Setup
  • 2.6. Access Control
    • 2.6.1. Frame Access Control
    • 2.6.2. Row Access Control
    • 2.6.3. User Labels and Validation
  • 2.7. System Frames in xGT
  • 2.8. Logging and Auditing
    • 2.8.1. General Logging
    • 2.8.2. Security Audit Logging
  • 2.9. xGT License Management System
    • 2.9.1. xGT Licensing
    • 2.9.2. License Management System Components
    • 2.9.3. Preparing to Install
    • 2.9.4. Installing
    • 2.9.5. License File Format
    • 2.9.6. LM-X Error Messages
• 3. Using the xGT System
  • 3.1. Installing the xGT Client
    • 3.1.1. Installing from PyPi
    • 3.1.2. Installing from a Distribution Tarball
    • 3.1.3. Recommended Packages
    • 3.1.4. Verifying Installation
  • 3.2. Connecting to a Server
    • 3.2.1. Use SSH Tunneling
    • 3.2.2. Connect Via SSL-encrypted Network Connection
    • 3.2.3. Connecting to an Open Port
    • 3.2.4. User Authentication
  • 3.3. Using the Python Library
    • 3.3.1. Multiprocessing
  • 3.4. Driving from a Jupyter Notebook
    • 3.4.1. Useful Notebook Cells
  • 3.5. Capabilities of the xGT Client
    • 3.5.1. Mixed-language Programming
• 4. Data Analytics with xGT
  • 4.1. Graph Thinking
    • 4.1.1. Introduction
    • 4.1.2. Graph Model
    • 4.1.3. Graph Image
    • 4.1.4. Setting up the Graph Model in xGT
    • 4.1.5. Looking for Interesting Patterns
    • 4.1.6. Concluding Remarks
    • 4.1.7. Graph Thinking Jupyter Notebook
  • 4.2. Graph Data Model: Frames and Namespaces
    • 4.2.1. Types of Frames in xGT
    • 4.2.2. Namespaces
    • 4.2.3. Access Control
  • 4.3. Data Types and Operators in xGT
    • 4.3.1. Scalar Data Types
    • 4.3.2. Composite Data Types
    • 4.3.3. Supported Operators
  • 4.4. Frame Management
    • 4.4.1. Direct Frame Creation
    • 4.4.2. Implicit Frame Creation
    • 4.4.3. Retrieving Frames
    • 4.4.4. Dropping Frames
  • 4.5. Data Movement
    • 4.5.1. Getting Data into xGT
    • 4.5.2. Getting Data out of xGT
    • 4.5.3. Performance Considerations
    • 4.5.4. S3 Credentials
    • 4.5.5. Arrow Flight Server Endpoint
  • 4.6. Job Management and History
    • 4.6.1. Job Object
    • 4.6.2. Jobs with Immediate Query Results
    • 4.6.3. Querying the Job History
  • 4.7. The Trovares Query Language (TQL)
  • 4.8. TQL Structure
    • 4.8.1. Putting It All Together
    • 4.8.2. Non-linear Patterns
    • 4.8.3. Variable-length Edge Traversal
    • 4.8.4. Multiple Edge Frame Traversal
    • 4.8.5. Path Variable Capture
  • 4.9. TQL Constraints on Graph Elements
    • 4.9.1. Pattern Filters
    • 4.9.2. Functions
    • 4.9.3. Constant Expressions
    • 4.9.4. CASE Expressions
    • 4.9.5. Cypher Parameters
  • 4.10. TQL List Clauses/Expressions
    • 4.10.1. TQL UNWIND Clause
    • 4.10.2. TQL List Comprehension
  • 4.11. TQL CALL Clause
    • 4.11.1. Whole Graph Algorithms
  • 4.12. TQL Modifications, Additions and Deletions
    • 4.12.1. Property Modifications
    • 4.12.2. Topology Additions
    • 4.12.3. Topology Deletions
    • 4.12.4. Visibility of Changes to the Graph
  • 4.13. What Is a Result from a TQL Query?
    • 4.13.1. Describing the Results from a TQL Query
    • 4.13.2. Modifying How the Results Set Is Reported
    • 4.13.3. Specifying the Results Table
    • 4.13.4. Querying over a Table Frame
    • 4.13.5. Multiple Query Parts
    • 4.13.6. Union Subqueries
    • 4.13.7. Optional Matches
  • 4.14. TQL Row Access Control in Queries
  • 4.15. TQL for Cypher Users
    • 4.15.1. xGT TQL Queries
    • 4.15.2. Examples of TQL Queries
    • 4.15.3. Limitations on TQL’s Cypher Support
  • 4.16. Access Control
    • 4.16.1. Security Labels
    • 4.16.2. Frame Access Control
    • 4.16.3. Row Access Control
    • 4.16.4. Additional Rules and Corner Cases
  • 4.17. Transactions in xGT
    • 4.17.1. Transaction Basics
    • 4.17.2. Using Transactions in Python
    • 4.17.3. Data Loading and Saving
    • 4.17.4. Data Modification in TQL
    • 4.17.5. Background Statistics Collection
    • 4.17.6. Implicit Frame Creation
  • 4.18. Query Optimization
    • 4.18.1. Checking the Status of Statistics Collection
• 5. Useful Recipes
  • 5.1. Wrappers to the Python API
    • 5.1.1. Frame Drop
    • 5.1.2. Run Query
  • 5.2. Analytics Idioms
    • 5.2.1. Compute Column Statistics
    • 5.2.2. Compute a Column Histogram
    • 5.2.3. Generic Frame-type Idiom
  • 5.3. Data Management
    • 5.3.1. Interaction with Background Statistics Collection
• 6. Code Samples
• 7. Python API
  • 7.1. Main Features
    • 7.1.1. Data Loading
    • 7.1.2. Query Processing
    • 7.1.3. Example Script
  • 7.2. General
    • 7.2.1. xgt.Connection
    • 7.2.2. xgt.NoAuth
    • 7.2.3. xgt.BasicAuth
    • 7.2.4. xgt.KerberosAuth
    • 7.2.5. xgt.Job
  • 7.3. Data Access and Manipulation
    • 7.3.1. xgt.VertexFrame
    • 7.3.2. xgt.EdgeFrame
    • 7.3.3. xgt.TableFrame
  • 7.4. Exceptions
    • 7.4.1. xgt.XgtError
    • 7.4.2. xgt.XgtSecurityError
    • 7.4.3. xgt.XgtArithmeticError
    • 7.4.4. xgt.XgtConnectionError
    • 7.4.5. xgt.XgtFrameDependencyError
    • 7.4.6. xgt.XgtInternalError
    • 7.4.7. xgt.XgtIOError
    • 7.4.8. xgt.XgtNameError
    • 7.4.9. xgt.XgtNotImplemented
    • 7.4.10. xgt.XgtServerMemoryError
    • 7.4.11. xgt.XgtSyntaxError
    • 7.4.12. xgt.XgtTransactionError
    • 7.4.13. xgt.XgtTypeError
    • 7.4.14. xgt.XgtValueError
  • 7.5. Admin
• 8. xGT Release Notes
  • 8.1. 1.12.0 (9/2/2022)
    • 8.1.1. New Features
    • 8.1.2. Changed
    • 8.1.3. Fixed
  • 8.2. 1.11.0 (6/17/2022)
    • 8.2.1. New Features
    • 8.2.2. Changed
    • 8.2.3. Fixed
  • 8.3. 1.10.1 (3/7/2022)
    • 8.3.1. New Features
    • 8.3.2. Changed
    • 8.3.3. Fixed
  • 8.4. 1.10.0 (2/11/2022)
    • 8.4.1. New Features
    • 8.4.2. Changed
    • 8.4.3. Fixed
  • 8.5. 1.9.1 (1/17/2022)
    • 8.5.1. Fixed
  • 8.6. 1.9.0 (11/1/2021)
    • 8.6.1. New Features
    • 8.6.2. Changed
    • 8.6.3. Fixed
  • 8.7. 1.8.0 (8/30/2021)
    • 8.7.1. New Features
    • 8.7.2. Changed
    • 8.7.3. Fixed
  • 8.8. 1.7.1 (7/7/2021)
    • 8.8.1. Fixed
  • 8.9. 1.7.0 (6/21/2021)
    • 8.9.1. New Features
    • 8.9.2. Changed
    • 8.9.3. Fixed
    • 8.9.4. Unresolved
  • 8.10. 1.6.0 (4/2/2021)
    • 8.10.1. New Features
    • 8.10.2. Changed
    • 8.10.3. Fixed
  • 8.11. 1.5.1 (2/19/2021)
    • 8.11.1. Changed
    • 8.11.2. Fixed
  • 8.12. 1.5.0 (2/8/2021)
    • 8.12.1. New Features
    • 8.12.2. Changed
    • 8.12.3. Fixed
    • 8.12.4. Unresolved
  • 8.13. 1.4.2 (9/28/2020)
    • 8.13.1. Fixed
  • 8.14. 1.4.1 (9/18/2020)
    • 8.14.1. New Features
    • 8.14.2. Changed
    • 8.14.3. Fixed
  • 8.15. 1.4.0 (8/24/2020)
    • 8.15.1. New Features
    • 8.15.2. Changed
    • 8.15.3. Fixed
    • 8.15.4. Unresolved
  • 8.16. 1.3.0 (11/5/2019)
    • 8.16.1. New Features
    • 8.16.2. Changed
    • 8.16.3. Fixed
  • 8.17. 1.2.0 (6/4/2019)
    • 8.17.1. New Features
    • 8.17.2. Changed
    • 8.17.3. Fixed
  • 8.18. 1.1.0 (2/21/2019)
    • 8.18.1. New Features
    • 8.18.2. Changed
    • 8.18.3. Fixed
  • 8.19. 1.0.0 (12/1/2018)
    • 8.19.1. New Features
    • 8.19.2. Changed
    • 8.19.3. Fixed
  • 8.20. 0.18.0 (10/24/2018)
    • 8.20.1. Changed
    • 8.20.2. Fixed
  • 8.21. 0.17.0 (10/1/2018)
    • 8.21.1. Changed
    • 8.21.2. Fixed
  • 8.22. 0.16.0 (9/1/2018)
    • 8.22.1. Changed
    • 8.22.2. Fixed
• 9. Appendices
  • 9.1. Exhaustive List of LM-X Error Messages
• 10. Glossary