5.3. Data Management

Management of data, particularly managing data movement into and out of the xGT server, is important for effective data analysis.

5.3.1. Interaction with Background Statistics Collection

With some situations—depending upon the hardware platform, number of frames, number of incident edges to vertices, and data sizes—the xGT server may encounter transactional conflicts that result in an unexpected rollback. If this situation were to occur, one workaround to try is to turn off background statistics calculations before the data frame manipulation, then turn background statistics calculation back on afterward. The background statistics calculation is described in section Background Statistics Collection.

Adjusting the setting of whether to compute statistics while the xGT server is running does require the user to have xgtadmin privilege. It may be helpful to maintain two separate connections from a Python script to the xGT server, one with only a user privilege and one with the xgtadmin privilege.

The following code shows an example of having two separate connections, and turning off and on the background statistics calculation setting. Note that the configuration parameter for controlling whether the background statistics calculations are done is called metrics.cache.

def do_frame_manipulation(function_that_sometimes_has_rollbacks):
    """
    This function is only for showing certain xGT API calls.

    The parameter is assumed to be a callable function (or lambda) that
    contains logic for manipulating frames (ingesting, or dropping, or some
    combination) that may result in a transaction rollback.
    """

    admin_user = xgt.Connection(auth = xgt.BasicAuth('xgtadminuser',
                                                     getpass.getpass())
    regular_user = xgt.Connection(auth = xgt.BasicAuth(getpass.getuser(),
                                                       getpass.getpass())

    old_metrics_cache = admin_user.get_config(['metrics.cache'])
    admin_user.set_config({'metrics.cache' : False})
    function_that_sometimes_has_rollbacks(regular_user)
    if old_metrics_cache:
        admin_user.set_config({'metrics.cache' : old_metrics_cache})
    return None

This example uses the function described in set_config().