bitflicker64 opened a new pull request, #487:
URL: https://github.com/apache/hugegraph-doc/pull/487
The RocksDB backend had no page of its own: `config-option.md` lists a
subset of the `rocksdb.*` keys and
`config-guide.md` mentions two of them, so nothing documented the data
directory layout, the option
groups, how `optimize_mode` and `bulkload_mode` behave, the memory
accounting, SST ingestion, the state
of raft mode for this backend or the pinned RocksDB version. This adds the
page in English and Chinese,
built from the sources on hugegraph master (commit 36811483a).
| page | what was wrong | what changed | source (file:line on master) |
|------|----------------|--------------|------------------------------|
| config/config-backend-rocksdb.md (en, cn) | Page did not exist | New page,
frontmatter and layout follow `config-https.md`, weight 5 places it after
`config-option` | n/a |
| config/config-backend-rocksdb.md | Nothing documented what the backend is
or which backends 1.7.0 accepts | Overview: embedded LSM store inside the
server process, allowed backends are memory, rocksdb, hbase, hstore, no shared
storage so use hstore for distributed setups |
hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/backend/store/BackendProviderFactory.java:37-48;
hugegraph-server/hugegraph-rocksdb/src/main/java/org/apache/hugegraph/backend/store/rocksdb/RocksDBFeatures.java:25-27
|
| config/config-backend-rocksdb.md | RocksDB version was documented nowhere
| States rocksdbjni is pinned to 8.10.2 |
hugegraph-server/hugegraph-rocksdb/pom.xml:37-41 |
| config/config-backend-rocksdb.md | Backend selection was only implicit in
the guide | `backend=rocksdb`, `serializer=binary`, `store`, plus
`init-store.sh` before the first start |
hugegraph-server/hugegraph-dist/src/assembly/static/conf/graphs/hugegraph.properties:22-32;
hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/backend/serializer/SerializerFactory.java:34-43
|
| config/config-backend-rocksdb.md | Provider registration and the
`rocksdbsst` name were undocumented | Registration comes from the packaged
`backend.properties` (`hugegraph.backends`, default rocksdb/hbase/hstore,
`-Drocksdb-only` for a rocksdb-only build); `rocksdbsst` is registered but
rejected by the allowed backend list |
hugegraph-server/hugegraph-dist/src/main/java/org/apache/hugegraph/dist/RegisterUtil.java:57-117;
hugegraph-server/hugegraph-dist/src/main/resources/backend.properties:18;
hugegraph-server/hugegraph-dist/pom.xml:38,294-304;
hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/backend/store/BackendProviderFactory.java:48,74-77
|
| config/config-backend-rocksdb.md | Data directory layout was undocumented
| data_path and wal_path defaults, the `m`/`g`/`s` store subdirectories, one
column family per table named `<database>+<table>`, old column families always
reopened |
hugegraph-server/hugegraph-rocksdb/src/main/java/org/apache/hugegraph/backend/store/rocksdb/RocksDBStore.java:301-305,384-393;
hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/backend/store/BackendStoreProvider.java:28-30;
hugegraph-server/hugegraph-rocksdb/src/main/java/org/apache/hugegraph/backend/store/rocksdb/RocksDBTable.java:64;
hugegraph-server/hugegraph-rocksdb/src/main/java/org/apache/hugegraph/backend/store/rocksdb/RocksDBStdSessions.java:388-391
|
| config/config-backend-rocksdb.md | Multi-graph and snapshot paths were
undocumented | Graphs cannot share a data path, cloned configs get a
`_<newGraph>` suffix, deleting a graph deletes both dirs, snapshot path shape
and resume behavior, parallel open of up to 8 instances with 600s/30s timeouts
|
hugegraph-server/hugegraph-rocksdb/src/main/java/org/apache/hugegraph/backend/store/rocksdb/RocksDBStoreProvider.java:43-64;
hugegraph-server/hugegraph-rocksdb/src/main/java/org/apache/hugegraph/backend/store/rocksdb/RocksDBStore.java:97-104,209-253,686-757
|
| config/config-backend-rocksdb.md | Only 47 of the 70 `rocksdb.*` keys were
documented anywhere, and three defaults were wrong | All 70 keys with defaults,
ranges and allowed values, in eight tables that keep the declaration order of
the Options class |
hugegraph-server/hugegraph-rocksdb/src/main/java/org/apache/hugegraph/backend/store/rocksdb/RocksDBOptions.java:56-699
|
| config/config-backend-rocksdb.md | How the options reach RocksDB was
undocumented | optimize_mode presets applied before the explicit options,
bulkload_mode effects, block cache 0 means no cache, capped prefix extractor,
`uint64add` merge operator, create-if-missing and the two always-on flags |
hugegraph-server/hugegraph-rocksdb/src/main/java/org/apache/hugegraph/backend/store/rocksdb/RocksDBStdSessions.java:446-694
|
| config/config-backend-rocksdb.md | The bloom filter group looked
unconditional | Notes that the filter and index caching options are only read
when `bloom_filter_bits_per_key >= 0`, and that `optimize_filters_for_hits`
applies regardless |
hugegraph-server/hugegraph-rocksdb/src/main/java/org/apache/hugegraph/backend/store/rocksdb/RocksDBStdSessions.java:560,667-691
|
| config/config-backend-rocksdb.md | No memory guidance existed | What `GET
/metrics/backend` counts as memory, block cache is per column family, write
buffers are per column family with `db_write_buffer_size` as the per store cap,
row cache is per store |
hugegraph-server/hugegraph-rocksdb/src/main/java/org/apache/hugegraph/backend/store/rocksdb/RocksDBMetrics.java:159-167;
hugegraph-server/hugegraph-rocksdb/src/main/java/org/apache/hugegraph/backend/store/rocksdb/RocksDBStdSessions.java:494-502,577-578,659-665;
hugegraph-server/hugegraph-api/src/main/java/org/apache/hugegraph/api/metrics/MetricsAPI.java:85,115
|
| config/config-backend-rocksdb.md | `rocksdb.sst_path` had a one line
description and no behavior | Ingestion walks `<sst_path>/<column family>/`,
takes non-empty `*.sst` files and moves them into the column family |
hugegraph-server/hugegraph-rocksdb/src/main/java/org/apache/hugegraph/backend/store/rocksdb/RocksDBStdSessions.java:351-366;
hugegraph-server/hugegraph-rocksdb/src/main/java/org/apache/hugegraph/backend/store/rocksdb/RocksDBIngester.java:34-52
|
| config/config-backend-rocksdb.md | Raft mode for rocksdb was undocumented
| The raft wrapper accepts only backends without shared storage, sessions write
with WAL and sync off, init-store forces `raft.mode=false`, the shipped
template marks the raft options deprecated, and the peer endpoints now live
under `graphspaces/{graphspace}/graphs/{graph}/raft/` |
hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/backend/store/BackendProviderFactory.java:63-70;
hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/backend/store/raft/RaftBackendStoreProvider.java:83-87;
hugegraph-server/hugegraph-rocksdb/src/main/java/org/apache/hugegraph/backend/store/rocksdb/RocksDBStdSessions.java:715-723;
hugegraph-server/hugegraph-dist/src/main/java/org/apache/hugegraph/cmd/InitStore.java:283-284;
hugegraph-server/hugegraph-dist/src/assembly/static/conf/graphs/hugegraph.properties:59-60;
hugegraph-server/hugegraph-api/src/main/java/org/apache/hugegraph/api/raft/RaftAPI.j
ava:57 |
| config/config-backend-rocksdb.md | Backend feature limits were
undocumented | Supported scans and paging, no store side index, write batch
transactions, snapshots, no shared storage, olap tables, TTL filtered by the
server, unsupported conditions and property updates |
hugegraph-server/hugegraph-rocksdb/src/main/java/org/apache/hugegraph/backend/store/rocksdb/RocksDBFeatures.java:25-145
|
| config/config-backend-rocksdb.md | The riscv64 requirement was
undocumented | `libatomic.so.1` is preloaded by `bin/util.sh` for the server,
init-store and dump-store scripts, with the exact error message |
hugegraph-server/hugegraph-dist/src/assembly/static/bin/util.sh:27-63;
hugegraph-server/hugegraph-dist/src/assembly/static/bin/hugegraph-server.sh:48 |
Follow-up outside this PR: the page is not yet listed in
`data/docs_nav.json` or in the section index of
`config/_index.md`, both of which are shared files that several backend
pages need to touch at once.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
