bitflicker64 commented on code in PR #510: URL: https://github.com/apache/hugegraph-doc/pull/510#discussion_r4255562251
########## content/en/docs/guides/toplingdb.md: ########## @@ -0,0 +1,125 @@ +--- +title: "ToplingDB development integration" +linkTitle: "ToplingDB (development)" +weight: 11 +description: "Build the optional ToplingDB runtime from development source, isolate its data, and understand startup and recovery boundaries." +--- + +> **Unreleased development interface.** This guide follows the [three-component runtime integration](https://github.com/apache/hugegraph/pull/3275). +> Use source containing that change. Existing releases and image tags do not establish support, and runtime acceptance remains a deployment prerequisite. + +## Choose the database owner + +HugeGraph defaults to standard RocksDB. Enable Topling separately for each process that owns local storage: + +| Component | Business provider setting | Start command | +| --- | --- | --- | +| Standalone RocksDB Server | `rocksdb.provider=topling` in each selected graph properties file | `bin/start-hugegraph.sh` | +| PD | `provider: topling` under `rocksdb` in `conf/application.yml` | `bin/start-hugegraph-pd.sh` | +| Store | `provider: topling` under `rocksdb` in `conf/application-pd.yml` | `bin/start-hugegraph-store.sh` | + +A standalone Server keeps `backend=rocksdb`; an HStore-backed Server uses `backend=hstore` and needs no local Topling runtime. +All local RocksDB graphs in one Server process must use the same provider. The business setting must match the runtime selected by its launcher. + +## Build and prepare a standard distribution + +Build the normal distributions from the source repository root using Java 17 and Maven 3.6.3 or later: + +```bash +mvn clean package -Dmaven.test.skip=true -Dmaven.javadoc.skip=true +``` + +Obtain a trusted Topling Easy Migrate JNI JAR and its SHA-256 through your artifact channel. Preparation requires a Java 17 JDK, Linux x86_64, +`sha256sum`, `unzip`, `od`, `ldd`, and GNU `mv`. The selected JAR's native dependencies, including its glibc and libaio requirements, must be installed. + +Stop the component, unpack a fresh standard distribution, and run from that component directory: + +```bash +export TOPLING_JNI_JAR=/absolute/path/to/rocksdbjni-topling.jar +export TOPLING_JNI_SHA256='<trusted-64-character-sha256>' +bash bin/prepare-topling.sh +``` + +The script verifies the copied JAR's hash, Topling Java marker, one Linux x86_64 JNI entry, ELF architecture and native dependencies. +Before installation, a disposable database verifies that the selected JNI actually applies EasyMigrate configuration to its persisted write-buffer size. +Unsupported builds are rejected even when their marker and hash match. The probe does not certify existing-data compatibility or every production setting. +A checksum receipt binds this verification to the installed JAR and native library; launchers reject missing or changed receipts. Use a fresh distribution +when upgrading from an older preparation script. Launch rejects inherited RocksDB JNI preloads in default/standard mode and competing JNI in Topling mode, while preserving unrelated preloads. +It installs `topling/rocksdbjni.jar`, its native library and optional web resources outside `lib`. Preparation refuses an existing `topling` directory; +replace a runtime by preparing another stopped, fresh distribution. Repeat preparation separately for Server, PD and Store as applicable. + +## Keep data outside the distribution + +Changing providers does not convert an existing database. Use separate, initially empty Topling directories and retain the standard-provider data. +Create persistent directories writable by the service account, outside source checkouts, build outputs and unpacked distribution directories: + +| Component | Configuration file and key | Example persistent path | +| --- | --- | --- | +| Server data | Graph properties: `rocksdb.data_path` | `/srv/hugegraph/topling/server/data` | +| Server WAL | Graph properties: `rocksdb.wal_path` | `/srv/hugegraph/topling/server/wal` | +| PD | `conf/application.yml`: `pd.data-path` | `/srv/hugegraph/topling/pd` | +| Store data | `conf/application.yml`: `app.data-path` | `/srv/hugegraph/topling/store/data` | +| Store Raft | `conf/application.yml`: `app.raft-path` | `/srv/hugegraph/topling/store/raft` | + +For a standalone Server, edit every selected graph configuration, for example `conf/graphs/hugegraph.properties`: Review Comment: Important: This paragraph tells readers to edit every selected graph configuration, but the block below gives one fixed `rocksdb.data_path` and `rocksdb.wal_path`. Copied into two graph files, both graphs point at the same RocksDB directories. Evidence, from apache/hugegraph#3275 at 96f99de: `RocksDBStore.wrapPath()` appends only the store name, and the store names are the fixed `m`, `g` and `s`, so both graphs resolve to `/srv/hugegraph/topling/server/data/g`, `/m` and `/s`. The `_<graph>` suffix is added only in `RocksDBStoreProvider.onCloneConfig()`, which does not run for files under `conf/graphs/`. The `dbs` map in `RocksDBStore` is per store instance, so a second open of the same directory has nothing to reuse and ends in `ConnectionException` (`Failed to open RocksDB '%s'`). From reading `InitStore`, the `bin/init-store.sh` step below hits this, because it initializes every non-HStore graph before closing any of them. Please say that each graph needs its own data and WAL directories and show graph-specific paths, such as `/srv/hugegraph/topling/server/hugegraph/data` and `.../hugegraph/wal`. The two Server rows in the path table above need the same change, and so do line 63 and the table in `content/cn/docs/guides/toplingdb.md`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
