bitflicker64 commented on code in PR #815: URL: https://github.com/apache/hugegraph-toolchain/pull/815#discussion_r4275948901
########## hugegraph-loader/README.md: ########## @@ -145,14 +147,86 @@ failure and logs an error naming the graph and warning that it may still be in ` unconfirmed mode unchanged. Inspect the graph mode and confirm there are no active imports before explicitly restoring `NONE` mode; an unconfirmed request cannot be rolled back automatically. +#### Concurrent imports and graph mode + +On a Server with loading-owner protocol 1, each ordinary import acquires its own +handle on the same live graph instance. After its workers stop, cleanup releases +only that handle; the graph stays in `LOADING` until the last import releases. +The protocol does not coordinate different Server processes or survive restart. + +Servers without the ownership endpoint keep the existing LOADING and cleanup +behavior, with no concurrent-import guarantee. Permission errors, network failures +and invalid ownership responses do not fall back to legacy mode. Restore and +Graph-source imports retain their existing RESTORING behavior. + +The paired Server change must be merged and released before the new guarantee is +available: [Server #3331](https://github.com/apache/hugegraph/pull/3331). + +On protocol-1 Servers, an ordinary import requires `NONE` or `LOADING` held by +active owners. Restore and Graph-source imports leave `RESTORING` after shutdown, +as before. A subsequent ordinary import is rejected until that mode is manually +recovered; an enum-set `LOADING` or `MERGING` mode has the same prerequisite. + +Before acquiring ownership, Loader logs `Preparing LOADING owner` with the +non-secret `instance/epoch` identity and recovery guidance. Owner tokens are not +logged. Owners have no automatic expiry, and cleanup failure still closes the +Loader context and clients; another `shutdown()` does not retry the release. +After an unclean exit or release failure, use the paired Server's administrator +recovery procedure below instead of looking for a token in Loader logs. + +Hubble's existing task lifecycle can finish, stop or reconnect without calling +`Loader.shutdown()`. This SDK change alone can therefore leave active owners and +block mode changes, including restore. Hubble integration requires the paired +[task cleanup fix #814](https://github.com/apache/hugegraph-toolchain/pull/814) and +[application shutdown fix #818](https://github.com/apache/hugegraph-toolchain/pull/818); +deploy those fixes together. If owners are already stranded, use the recovery +steps below. + +1. Stop all related imports and ordinary graph writes, and drain their in-flight + requests. The Server recovery endpoint cannot perform this external drain. +2. As an administrator with `ADMIN` or `ROOT` permission, GET + `/graphspaces/{graphspace}/graphs/{graph}/mode/owners` on the affected live + Server instance. Read its current `instance`, `epoch` and non-secret `revision` + immediately before recovery; do not reuse values from another instance or an + earlier attempt. +3. PUT the following body to that same endpoint, replacing the placeholders with + the values just read. This recovery action does not require an owner token. + + ```json + { + "action": "recover", + "instance": "<current instance UUID>", + "epoch": 0, + "revision": 0, + "confirm": "I'm sure all imports have stopped" + } + ``` + + Replace both example `0` values with the GET response's current `epoch` and + `revision`. Recovery requires both values to match; any owner or mode change + after GET makes this request stale. + Success returns `state: recovered`, `mode: NONE` and the incremented epoch, + clears existing owners, and prevents old-epoch acquire/release requests from + taking effect. A stale recovery request returns HTTP 409 so it cannot clear a + newer owner, including one acquired within the same epoch. Ordinary graph-status + write permission is insufficient (HTTP 403). +4. If the response is lost or the result is otherwise unknown, inspect the graph + mode and capability. Do not blindly obtain a new epoch and repeat recovery, + which could clear owners created after the first attempt. + +This procedure also clears a manually set `RESTORING` or `MERGING` mode once all +related work has stopped and drained. It depends on the paired Server change +linked above; older Servers retain their legacy enum-mode recovery behavior. +An enum PUT to `NONE` remains rejected while protocol-1 owners are active. Review Comment: Minor: this line says only an enum PUT to `NONE` is rejected while owners are active, but line 179 says owners block mode changes generally, and the paired Server agrees with line 179: at apache/hugegraph#3331 (33887ab) `LoadingMode.mode(GraphMode)` rejects every enum PUT with HTTP 409 while any owner is active, and its `docs/server-runtime.md` says so. That also affects clients still on the enum path. `HGVertexWriter`/`HGEdgeWriter` call `HGLoadContext.setLoadingMode()` in their constructors, which PUTs `LOADING`; the 409 message does not contain `Can not deserialize`, so every Spark writer task fails while a protocol-1 Loader import runs. A Loader from before this change fails at startup, and a concurrent `--restore` fails on its `RESTORING` PUT. Could this line say that any enum mode change is rejected, naming the Spark connector and older Loaders? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
