4. Data Analytics with xGT

The following sections are available in this chapter:

• 4.1. Graph Thinking
  • 4.1.1. Introduction
  • 4.1.2. Graph Model
  • 4.1.3. Graph Image
  • 4.1.4. Setting up the Graph Model in xGT
    • 4.1.4.1. Understanding the Vertices
    • 4.1.4.2. Understanding the Edges
    • 4.1.4.3. Data Loading
  • 4.1.5. Looking for Interesting Patterns
    • 4.1.5.1. Describing the First Relationship
    • 4.1.5.2. Describing the Second Relationship
    • 4.1.5.3. Understanding the Query Result
    • 4.1.5.4. Exploring the Query Result
  • 4.1.6. Concluding Remarks
  • 4.1.7. Graph Thinking Jupyter Notebook
• 4.2. Graph Data Model: Frames and Namespaces
  • 4.2.1. Types of Frames in xGT
  • 4.2.2. Namespaces
    • 4.2.2.1. Default Namespace
    • 4.2.2.2. Disabling the Default Namespace
  • 4.2.3. Access Control
    • 4.2.3.1. Frame Access Control
    • 4.2.3.2. Row Access Control
• 4.3. Data Types and Operators in xGT
  • 4.3.1. Scalar Data Types
    • 4.3.1.1. BOOLEAN
    • 4.3.1.2. INTEGER
    • 4.3.1.3. FLOAT
    • 4.3.1.4. DATE
    • 4.3.1.5. TIME
    • 4.3.1.6. DATETIME
    • 4.3.1.7. IPADDRESS
    • 4.3.1.8. TEXT
  • 4.3.2. Composite Data Types
    • 4.3.2.1. Lists
  • 4.3.3. Supported Operators
    • 4.3.3.1. Boolean Operators
    • 4.3.3.2. Comparison Operators
    • 4.3.3.3. Arithmetic Operators
    • 4.3.3.4. String Operators
    • 4.3.3.5. List Operators
• 4.4. Frame Management
  • 4.4.1. Direct Frame Creation
    • 4.4.1.1. Example
  • 4.4.2. Implicit Frame Creation
    • 4.4.2.1. Implicit Vertex Frame Creation
    • 4.4.2.2. Implicit Edge Frame Creation
    • 4.4.2.3. Implicit Table Frame Creation
    • 4.4.2.4. Schema Inference
    • 4.4.2.5. Data Sources Supported
  • 4.4.3. Retrieving Frames
  • 4.4.4. Dropping Frames
• 4.5. Data Movement
  • 4.5.1. Getting Data into xGT
    • 4.5.1.1. Reading from the Client Filesystem
    • 4.5.1.2. Reading from the Server Filesystem
    • 4.5.1.3. Reading from a URL
    • 4.5.1.4. Reading from an AWS S3 Bucket
    • 4.5.1.5. Inserting from Python
    • 4.5.1.6. Reading Into Edge Frames
    • 4.5.1.7. Reading Into Vertex Frames
    • 4.5.1.8. Reading from a CSV File
    • 4.5.1.9. Reading Parquet Files
    • 4.5.1.10. Compressed CSV Files
    • 4.5.1.11. Setting Row Labels
    • 4.5.1.12. Handling Ingest Errors
    • 4.5.1.13. Digging Deeper into Ingest Errors
  • 4.5.2. Getting Data out of xGT
    • 4.5.2.1. Saving to the Client Filesystem
    • 4.5.2.2. Saving to the Server Filesystem
    • 4.5.2.3. Saving to Python
    • 4.5.2.4. Writing Parquet Files
    • 4.5.2.5. Saving Row Labels
  • 4.5.3. Performance Considerations
  • 4.5.4. S3 Credentials
  • 4.5.5. Arrow Flight Server Endpoint
• 4.6. Job Management and History
  • 4.6.1. Job Object
  • 4.6.2. Jobs with Immediate Query Results
  • 4.6.3. Querying the Job History
• 4.7. The Trovares Query Language (TQL)
• 4.8. TQL Structure
  • 4.8.1. Putting It All Together
  • 4.8.2. Non-linear Patterns
  • 4.8.3. Variable-length Edge Traversal
  • 4.8.4. Multiple Edge Frame Traversal
  • 4.8.5. Path Variable Capture
• 4.9. TQL Constraints on Graph Elements
  • 4.9.1. Pattern Filters
  • 4.9.2. Functions
    • 4.9.2.1. Aggregation Functions
    • 4.9.2.2. Conversion Functions
    • 4.9.2.3. Mathematical Functions
    • 4.9.2.4. String Functions
    • 4.9.2.5. List Functions
    • 4.9.2.6. Path Functions
    • 4.9.2.7. Degree Functions
    • 4.9.2.8. Function to Enforce Distinct Vertices
  • 4.9.3. Constant Expressions
  • 4.9.4. CASE Expressions
    • 4.9.4.1. Simple CASE Expressions
    • 4.9.4.2. Generic CASE Expressions
    • 4.9.4.3. Examples
  • 4.9.5. Cypher Parameters
• 4.10. TQL List Clauses/Expressions
  • 4.10.1. TQL UNWIND Clause
  • 4.10.2. TQL List Comprehension
• 4.11. TQL CALL Clause
  • 4.11.1. Whole Graph Algorithms
    • 4.11.1.1. Breadth First Search
    • 4.11.1.2. PageRank
    • 4.11.1.3. Strongly Connected Components
    • 4.11.1.4. Weakly Connected Components
• 4.12. TQL Modifications, Additions and Deletions
  • 4.12.1. Property Modifications
  • 4.12.2. Topology Additions
  • 4.12.3. Topology Deletions
  • 4.12.4. Visibility of Changes to the Graph
• 4.13. What Is a Result from a TQL Query?
  • 4.13.1. Describing the Results from a TQL Query
  • 4.13.2. Modifying How the Results Set Is Reported
  • 4.13.3. Specifying the Results Table
  • 4.13.4. Querying over a Table Frame
  • 4.13.5. Multiple Query Parts
    • 4.13.5.1. Carrying Information Using WITH
    • 4.13.5.2. Aliasing in the WITH Clause
    • 4.13.5.3. Additional Filtering of WITH Partial Results
    • 4.13.5.4. Visibility of Changes
    • 4.13.5.5. Use Cases of Multiple Query Parts
    • 4.13.5.6. Additional Rules
    • 4.13.5.7. Performance Considerations
  • 4.13.6. Union Subqueries
    • 4.13.6.1. Visibility of Changes
  • 4.13.7. Optional Matches
    • 4.13.7.1. Valid Optional Structure
    • 4.13.7.2. Constraints
    • 4.13.7.3. Multiple Optional Matches
    • 4.13.7.4. Carrying Information Using WITH
• 4.14. TQL Row Access Control in Queries
• 4.15. TQL for Cypher Users
  • 4.15.1. xGT TQL Queries
    • 4.15.1.1. TQL is Typed
    • 4.15.1.2. Pattern Matching
    • 4.15.1.3. TQL Query Structure
    • 4.15.1.4. Type Inference in TQL Queries
    • 4.15.1.5. Pattern Variables
    • 4.15.1.6. Property and Topology Modifications as Part of Queries
    • 4.15.1.7. Aggregation Functions and Solution Modifiers
  • 4.15.2. Examples of TQL Queries
    • 4.15.2.1. Example 1
    • 4.15.2.2. Example 2
    • 4.15.2.3. Example 3
    • 4.15.2.4. Example 4
    • 4.15.2.5. Example 5
    • 4.15.2.6. Example 6
    • 4.15.2.7. Example 7
    • 4.15.2.8. Example 8
    • 4.15.2.9. Example 9
    • 4.15.2.10. Example 10
    • 4.15.2.11. Example 11
    • 4.15.2.12. Example 12
  • 4.15.3. Limitations on TQL’s Cypher Support
• 4.16. Access Control
  • 4.16.1. Security Labels
  • 4.16.2. Frame Access Control
    • 4.16.2.1. Creating Frames with Frame Security
    • 4.16.2.2. Namespace Labels
    • 4.16.2.3. Access Types for xGT Operations
  • 4.16.3. Row Access Control
    • 4.16.3.1. Attaching Row Labels to Data
    • 4.16.3.2. Security Labels Created by TQL Queries
    • 4.16.3.3. Performance
    • 4.16.3.4. Result Frames with Row Labels
    • 4.16.3.5. Viewing Security Labels
  • 4.16.4. Additional Rules and Corner Cases
    • 4.16.4.1. Edge Access
    • 4.16.4.2. Degree Computation
    • 4.16.4.3. Row Label Universe Requirements
• 4.17. Transactions in xGT
  • 4.17.1. Transaction Basics
  • 4.17.2. Using Transactions in Python
  • 4.17.3. Data Loading and Saving
  • 4.17.4. Data Modification in TQL
  • 4.17.5. Background Statistics Collection
  • 4.17.6. Implicit Frame Creation
• 4.18. Query Optimization
  • 4.18.1. Checking the Status of Statistics Collection