bitflicker64 commented on code in PR #510: URL: https://github.com/apache/hugegraph-doc/pull/510#discussion_r4175878829
########## content/en/docs/guides/toplingdb.md: ########## @@ -0,0 +1,154 @@ +--- +title: "ToplingDB development integration" Review Comment: Minor: The new guide is not listed in `content/en/docs/SUMMARY.md` or `content/cn/docs/SUMMARY.md`. Both files list every page under the Guides section (Security, Toolchain Local Testing, FAQ), and the SeaTunnel guide in 3287b608 (#464) added its SUMMARY entry with the page. Please add `- [ToplingDB (development)](guides/toplingdb)` to the English Guides list and `- [ToplingDB（开发版）](guides/toplingdb)` to the Chinese one, or say in the PR that the page is left out of SUMMARY on purpose while the feature is unreleased. ########## content/en/docs/guides/toplingdb.md: ########## @@ -0,0 +1,154 @@ +--- +title: "ToplingDB development integration" +linkTitle: "ToplingDB (development)" +weight: 11 +description: "Build the optional ToplingDB runtime from development source, isolate its data, and understand startup and recovery boundaries." +--- + +> **Unreleased development interface.** This guide describes the proposed replacement for the 2025 ToplingDB integration. +> It requires a complete source checkout containing the integration and its lifecycle and snapshot-recovery prerequisites. +> These changes are under review; existing HugeGraph releases and registry tags do not establish support for this interface. +> Follow the [integration proposal](https://github.com/hugegraph/hugegraph/pull/264) for its delivery status. + +## Choose the process that owns the database + +| Deployment | Process that loads Topling JNI | Provider setting | +|---|---|---| +| Standalone RocksDB Server | Server | `rocksdb.provider=topling` in `conf/graphs/hugegraph.properties` | +| HStore PD | Each PD | `provider: topling` under `rocksdb` in `conf/application.yml` | +| HStore Store | Each Store | `provider: topling` under `rocksdb` in `conf/application-pd.yml` | +| HStore-backed Server | None | No local Topling setting | + +PD and Store select their providers independently. The HStore-backed Server is a remote client and does not need a Topling JAR or native library. +Provider selection happens at process startup. All local RocksDB graphs within one Server process must agree on the provider. + +The historical `rocksdb.option_path` and `rocksdb.open_http` examples describe a different integration. +Do not copy that configuration into this development interface. The component's launch scripts select the Easy Migrate YAML file. + +## Build an optional distribution + +Build on Linux x86_64 with Java 11+, Maven 3.5+, `rsync`, `unzip`, and `tar`. +The Server mount preflight additionally requires util-linux 2.37+ `mountpoint` on `PATH`. +Native macOS execution is outside this integration's support matrix; use a Linux x86_64 container when working on macOS. + +Obtain a compatible Topling Easy Migrate JNI JAR from its producer, together with its source revision, dependency licenses, notices, and trusted SHA-256. +The HugeGraph source tree does not include this binary. A checksum identifies an artifact; it does not establish its origin, license, or runtime compatibility. +Verify redistribution rights and retain the required license and notice files before distributing a derived package or image. + +From the complete source checkout containing this feature: + +```bash +VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout) +mvn clean package \ + -pl hugegraph-server/hugegraph-dist,hugegraph-pd/hg-pd-dist,hugegraph-store/hg-store-dist \ + -am -Dmaven.test.skip=true -Dmaven.javadoc.skip=true -ntp + +export TOPLING_JNI_JAR=/absolute/path/to/rocksdbjni-topling.jar +export TOPLING_JNI_SHA256='<trusted 64-character SHA-256>' +install-dist/scripts/build-topling-distribution.sh server "$VERSION" +# Generate these only when running HStore: +install-dist/scripts/build-topling-distribution.sh pd "$VERSION" +install-dist/scripts/build-topling-distribution.sh store "$VERSION" +``` + +The Maven build creates standard packages. The generator creates separate `-topling` directories and archives beside them. +It copies the external JAR into private staging, verifies that copy, and prepares the native library. +Missing inputs, a checksum mismatch, or incompatible JAR structure stop generation; the generator does not search a Maven cache or download a substitute. +Build success is not a native runtime acceptance test. + +Each Topling package contains `lib/topling/rocksdbjni-topling.jar`, `lib/topling/runtime.properties`, +`library/librocksdbjni-linux64.so`, and the component's preparation and preload scripts. +Run `bin/prepare-topling.sh` again after replacing the installed Topling JAR. + +For container builds, use the matching checkout's `docker/README.md` and `docker/bake.hcl` with the same external JNI inputs. +Build locally before using the example `topling` tags and select `pull_policy: never` for that local deployment. +For registry images, verify the source revision, JNI identity, and digest first. A tag or a healthy container alone does not prove which JNI is running. + +## Configure isolated data + +Changing the provider does not migrate or convert data. Give Topling a separate data directory and keep the standard RocksDB directory unchanged. +Use these component-specific settings: + +| Component | Data setting | Example Topling root | +|---|---|---| +| Server | `rocksdb.data_path` | `/srv/hugegraph/topling/server` | +| PD | `pd.data-path` | `/srv/hugegraph/topling/pd` | +| Store | `app.data-path` | `/srv/hugegraph/topling/store` | + +For a standalone Server, edit the generated distribution's `conf/graphs/hugegraph.properties`: + +```properties +backend=rocksdb +rocksdb.provider=topling +rocksdb.data_path=/srv/hugegraph/topling/server Review Comment: Important: This example sets only `rocksdb.data_path`, so the Server WAL stays at the default `rocksdb.wal_path=rocksdb-data/wal`, which resolves inside the generated `-topling` distribution directory and not under `/srv/hugegraph/topling/server`. Evidence: `RocksDBOptions.WAL_PATH` defaults to `rocksdb-data/wal`. In hugegraph/hugegraph#264 at 9af9d14, `server_verify_graph_roots` in `bin/rocksdb-server-config.sh` treats both `rocksdb.data_path` and `rocksdb.wal_path` as provider-owned roots. `install-dist/scripts/build-topling-distribution.sh` moves an existing `-topling` directory to `$STAGING_ROOT/previous-directory` and its success cleanup runs `rm -rf "$STAGING_ROOT"`. Standalone (non-raft) Server writes keep the WAL enabled (`RocksDBStdSessions.StdSession` disables it only in raft mode) and HugeGraph does not flush memtables on close. A user who follows this page and later re-runs the generator command from the build section, or who keeps only the mounted data root as the mount paragraph says, loses every write not yet flushed to SST files, and the data directory still opens. Please add `rocksdb.wal_path` to the component table and to this properties block (for example `rocksdb.data_path=/srv/hugegraph/topling/server/data` and `rocksdb.wal_path=/srv/hugegraph/topling/server/wal`), state that both paths must sit under the mounted Topling root, and make the same change at line 83 of content/cn/docs/guides/toplingdb.md. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
