bitflicker64 commented on code in PR #510: URL: https://github.com/apache/hugegraph-doc/pull/510#discussion_r4180479210
########## content/en/docs/guides/toplingdb.md: ########## @@ -0,0 +1,111 @@ +--- +title: "ToplingDB development integration" +linkTitle: "ToplingDB (development)" +weight: 11 +description: "Build the optional ToplingDB runtime from development source, isolate its data, and understand startup and recovery boundaries." +--- + +> **Unreleased development interface.** This guide follows the [three-component runtime integration](https://github.com/hugegraph/hugegraph/pull/266). +> Use source containing that change. Existing releases and image tags do not establish support, and runtime acceptance remains a deployment prerequisite. + +## Choose the database owner + +HugeGraph defaults to standard RocksDB. Enable Topling separately for each process that owns local storage: + +| Component | Business provider setting | Start command | +| --- | --- | --- | +| Standalone RocksDB Server | `rocksdb.provider=topling` in each selected graph properties file | `bin/start-hugegraph.sh` | +| PD | `provider: topling` under `rocksdb` in `conf/application.yml` | `bin/start-hugegraph-pd.sh` | +| Store | `provider: topling` under `rocksdb` in `conf/application-pd.yml` | `bin/start-hugegraph-store.sh` | + +A standalone Server keeps `backend=rocksdb`; an HStore-backed Server uses `backend=hstore` and needs no local Topling runtime. +All local RocksDB graphs in one Server process must use the same provider. The business setting must match the runtime selected by its launcher. + +## Build and prepare a standard distribution + +Build the normal distributions from the source repository root using its supported JDK and Maven: + +```bash +mvn clean package -Dmaven.test.skip=true -Dmaven.javadoc.skip=true +``` + +Obtain a trusted Topling Easy Migrate JNI JAR and its SHA-256 through your artifact channel. Preparation requires Linux x86_64, +`sha256sum`, `unzip`, `od`, `ldd`, and GNU `mv`. The selected JAR's native dependencies, including its glibc and libaio requirements, must be installed. + +Stop the component, unpack a fresh standard distribution, and run from that component directory: + +```bash +export TOPLING_JNI_JAR=/absolute/path/to/rocksdbjni-topling.jar +export TOPLING_JNI_SHA256='<trusted-64-character-sha256>' +bash bin/prepare-topling.sh +``` + +The script verifies the copied JAR's hash, Topling Java marker, one Linux x86_64 JNI entry, ELF architecture and native dependencies. +It installs `topling/rocksdbjni.jar`, its native library and optional web resources outside `lib`. Preparation refuses an existing `topling` directory; +replace a runtime by preparing another stopped, fresh distribution. Repeat preparation separately for Server, PD and Store as applicable. + +## Keep data outside the distribution + +Changing providers does not convert an existing database. Use separate, initially empty Topling directories and retain the standard-provider data. +Create persistent directories writable by the service account, outside source checkouts, build outputs and unpacked distribution directories: + +| Component | Configuration file and key | Example persistent path | +| --- | --- | --- | +| Server data | Graph properties: `rocksdb.data_path` | `/srv/hugegraph/topling/server/data` | +| Server WAL | Graph properties: `rocksdb.wal_path` | `/srv/hugegraph/topling/server/wal` | +| PD | `conf/application.yml`: `pd.data-path` | `/srv/hugegraph/topling/pd` | +| Store data | `conf/application.yml`: `app.data-path` | `/srv/hugegraph/topling/store/data` | +| Store Raft | `conf/application.yml`: `app.raft-path` | `/srv/hugegraph/topling/store/raft` | + +For a standalone Server, edit every selected graph configuration, for example `conf/graphs/hugegraph.properties`: + +```properties +backend=rocksdb +rocksdb.provider=topling +rocksdb.data_path=/srv/hugegraph/topling/server/data +rocksdb.wal_path=/srv/hugegraph/topling/server/wal +``` + +Set **both** Server data and WAL paths. Leaving WAL at its package-local default risks losing it when a build or package replacement removes that directory. +Keep the external data, WAL and Raft roots across runtime/package changes. + +For PD and Store, edit the existing `rocksdb` mapping in the provider configuration file listed above, retaining its other options: + +```yaml +rocksdb: + provider: topling +``` + +Set their persistent paths separately in `conf/application.yml`; keep the normal network addresses and cluster configuration. + +## Select, start and verify + +In the startup shell of each component that owns a Topling database, explicitly select the runtime: + +```bash +export TOPLINGDB_ROCKSDB_PROVIDER=topling +# Run this component's start command from the first table. +``` + +The launcher uses that component's `conf/toplingdb.yaml`. Set `TOPLINGDB_EASY_MIGRATE_CONF=/absolute/path/config.yaml` for an explicit override. +The shipped native profile disables its HTTP server and keeps `memtable_as_log_index=false`, which the Java write path requires. +Tune that profile for the host; it is separate from the business provider and data-path configuration. + +For a standalone Server, `bin/init-store.sh` and `bin/dump-store.sh` use the same explicit selection as `bin/start-hugegraph.sh`. +For HStore, start PD, then Store, then the HStore-backed Server with its default runtime environment; do not enable local Topling on that Server. Review Comment: Minor: "with its default runtime environment" does not tell readers to clear the variable they exported a few lines earlier. Readers who start PD, Store and Server from one shell will still have `TOPLINGDB_ROCKSDB_PROVIDER=topling` set when they start the Server. Evidence: at hugegraph/hugegraph#266 head 3ec2871, `bin/hugegraph-server.sh` and `bin/init-store.sh` source `bin/preload-topling.sh`. That script reads only `TOPLINGDB_ROCKSDB_PROVIDER`, not `backend`. On an unprepared HStore Server distribution it exits with `Error: run .../bin/prepare-topling.sh before enabling Topling`. That message tells the reader to prepare Topling on the Server, which this guide says not to do. Please say so explicitly, for example: "Start the HStore-backed Server from a shell where `TOPLINGDB_ROCKSDB_PROVIDER` is unset (or `rocksdb`)". Make the same change at line 95 of content/cn/docs/guides/toplingdb.md. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
