Distributed Deployment

GART is designed to be deployed in a distributed manner. This means that you can deploy different components of GART on different nodes of a cluster. To achieve this, you need to provide a configuration file, and GART provides a Python script to launch GART on a cluster easily.

User Configuration

Please ensure that the GART folder is accessible at the same path on all nodes of the cluster. Then you can create a configuration file in the following format:

 1{
 2    // Database port
 3    "db_host": "127.0.0.1",
 4    // Database port
 5    "db_port": 5432,
 6    // Database type: mysql or postgresql
 7    "db_type": "postgresql",
 8    // Database name
 9    "db_name": "ldbc",
10    // Database user
11    "db_user": "dbuser",
12    // Database password
13    "db_password": "123456",
14    "rgmapping_file": "/path/to/rgmapping.yaml",
15    "v6d_size": "750G",
16    "etcd_prefix": "gart_meta_",
17    // entrypoint of the kafka server
18    "kafka_server": "127.0.0.1:9092",
19    // total number of subgraphs
20    "total_subgraph_num": 2,
21    // path to the GART build folder
22    "gart_bin_path": "/path/to/GART/build/",
23    // whether to enable bulk load
24    "enable_bulkload": 1,
25    // where to launch the capturer
26    "capturer_host": "127.0.0.1",
27    // where to launch the converter
28    "converter_host": "127.0.0.1",
29    // path to kafka
30    "kafka_path": "/path/to/kafka",
31    "writer_hosts": [
32        {
33            // where to launch the writer to create subgraph 0
34            "subgraph_id": 0,
35            "host": "127.0.0.1"
36        },
37        {
38            // where to launch the writer to create subgraph 1
39            "subgraph_id": 1,
40            "host": "127.0.0.1"
41        }
42    ]
43}

Launch GART with configuration

GART provides a Python script to launch GART on a cluster easily. You can launch GART with the following command by providing the configuration file in the previous step:

./path/to/GART/scripts/distributed_deployment.py --config_file /path/to/user_config.json