imbajin commented on code in PR #510: URL: https://github.com/apache/hugegraph-doc/pull/510#discussion_r4221977811
########## content/en/docs/guides/toplingdb.md: ########## @@ -0,0 +1,125 @@ +--- +title: "ToplingDB development integration" +linkTitle: "ToplingDB (development)" +weight: 11 +description: "Build the optional ToplingDB runtime from development source, isolate its data, and understand startup and recovery boundaries." +--- + +> **Unreleased development interface.** This guide follows the [three-component runtime integration](https://github.com/apache/hugegraph/pull/3275). +> Use source containing that change. Existing releases and image tags do not establish support, and runtime acceptance remains a deployment prerequisite. + +## Choose the database owner + +HugeGraph defaults to standard RocksDB. Enable Topling separately for each process that owns local storage: + +| Component | Business provider setting | Start command | +| --- | --- | --- | +| Standalone RocksDB Server | `rocksdb.provider=topling` in each selected graph properties file | `bin/start-hugegraph.sh` | +| PD | `provider: topling` under `rocksdb` in `conf/application.yml` | `bin/start-hugegraph-pd.sh` | +| Store | `provider: topling` under `rocksdb` in `conf/application-pd.yml` | `bin/start-hugegraph-store.sh` | + +A standalone Server keeps `backend=rocksdb`; an HStore-backed Server uses `backend=hstore` and needs no local Topling runtime. +All local RocksDB graphs in one Server process must use the same provider. The business setting must match the runtime selected by its launcher. + +## Build and prepare a standard distribution + +Build the normal distributions from the source repository root using Java 17 and Maven 3.6.3 or later: + +```bash +mvn clean package -Dmaven.test.skip=true -Dmaven.javadoc.skip=true +``` + +Obtain a trusted Topling Easy Migrate JNI JAR and its SHA-256 through your artifact channel. Preparation requires a Java 17 JDK, Linux x86_64, +`sha256sum`, `unzip`, `od`, `ldd`, and GNU `mv`. The selected JAR's native dependencies, including its glibc and libaio requirements, must be installed. + +Stop the component, unpack a fresh standard distribution, and run from that component directory: + +```bash +export TOPLING_JNI_JAR=/absolute/path/to/rocksdbjni-topling.jar +export TOPLING_JNI_SHA256='<trusted-64-character-sha256>' +bash bin/prepare-topling.sh +``` + +The script verifies the copied JAR's hash, Topling Java marker, one Linux x86_64 JNI entry, ELF architecture and native dependencies. +Before installation, a disposable database verifies that the selected JNI actually applies EasyMigrate configuration to its persisted write-buffer size. +Unsupported builds are rejected even when their marker and hash match. The probe does not certify existing-data compatibility or every production setting. +A checksum receipt binds this verification to the installed JAR and native library; launchers reject missing or changed receipts. Use a fresh distribution +when upgrading from an older preparation script. Launch rejects inherited RocksDB JNI preloads in default/standard mode and competing JNI in Topling mode, while preserving unrelated preloads. +It installs `topling/rocksdbjni.jar`, its native library and optional web resources outside `lib`. Preparation refuses an existing `topling` directory; +replace a runtime by preparing another stopped, fresh distribution. Repeat preparation separately for Server, PD and Store as applicable. + +## Keep data outside the distribution + +Changing providers does not convert an existing database. Use separate, initially empty Topling directories and retain the standard-provider data. +Create persistent directories writable by the service account, outside source checkouts, build outputs and unpacked distribution directories: + +| Component | Configuration file and key | Example persistent path | +| --- | --- | --- | +| Server data | Graph properties: `rocksdb.data_path` | `/srv/hugegraph/topling/server/data` | +| Server WAL | Graph properties: `rocksdb.wal_path` | `/srv/hugegraph/topling/server/wal` | +| PD | `conf/application.yml`: `pd.data-path` | `/srv/hugegraph/topling/pd` | +| Store data | `conf/application.yml`: `app.data-path` | `/srv/hugegraph/topling/store/data` | +| Store Raft | `conf/application.yml`: `app.raft-path` | `/srv/hugegraph/topling/store/raft` | + +For a standalone Server, edit every selected graph configuration, for example `conf/graphs/hugegraph.properties`: Review Comment: ⚠️ Important: This tells readers to edit every selected graph config but gives the same data and WAL roots for all of them. In paired Server PR #3275, `RocksDBStore.wrapPath()` appends only the store name, while `RocksDBStoreProvider.onCloneConfig()` adds a graph suffix only when cloning; the local `conf/graphs/*.properties` files are loaded independently. Reusing this example across two graphs therefore opens the same RocksDB roots and the second graph cannot open them. Please state that each graph needs unique data and WAL paths, show graph-specific directories, and sync the Chinese guide. Evidence: [config scan](https://github.com/apache/hugegraph/blob/96f99de73820af6a4b41f09df1423da6302ab58a/hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/util/ConfigUtil.java#L90-L108), [local graph loading](https://github.com/apache/hugegraph/blob/96f99de73820af6a4b41f09df1423da6302ab58a/hugegraph-server/hugegraph-api/src/main/java/org/apache/hugegraph/core/GraphManager .java#L1263-L1273), [path wrapping](https://github.com/apache/hugegraph/blob/96f99de73820af6a4b41f09df1423da6302ab58a/hugegraph-server/hugegraph-rocksdb/src/main/java/org/apache/hugegraph/backend/store/rocksdb/RocksDBStore.java#L387-L395), [clone suffix](https://github.com/apache/hugegraph/blob/96f99de73820af6a4b41f09df1423da6302ab58a/hugegraph-server/hugegraph-rocksdb/src/main/java/org/apache/hugegraph/backend/store/rocksdb/RocksDBStoreProvider.java#L44-L53). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
