deepakpanda93 opened a new pull request, #19591:
URL: https://github.com/apache/hudi/pull/19591
### Describe the issue this Pull Request addresses
Closes #16814. (JIRA: HUDI-8964.)
The **Distributed Locking** section documents the storage-based, ZooKeeper,
HiveMetastore, DynamoDB and FileSystem lock
providers, but never mentions
`DynamoDBBasedImplicitPartitionKeyLockProvider`. That is exactly what @yihua
asked for on
the issue ("the docs of lock providers need to be updated") and what the
later backlog triage pinned down.
Confirmed the gap rather than assuming it: `ImplicitPartitionKey` appears
nowhere under `website/docs`,
`website/versioned_docs` or `website/learn`.
### Summary and Changelog
Adds a `DynamoDB-Based Lock Provider with Implicit Partition Key` subsection
after the existing DynamoDB one, covering
what the key is derived from, when to prefer it, what carries over from the
standard provider, and the one operational
consequence of hashed keys.
**What it does.** The class extends `DynamoDBBasedLockProviderBase` and
overrides a single method:
```java
String hudiTableBasePathNormalized = s3aToS3(lockConfiguration.getConfig()
.getString(HoodieCommonConfig.BASE_PATH.key()));
String partitionKey =
HashID.generateXXHashAsString(hudiTableBasePathNormalized, HashID.Size.BITS_64);
```
so the DynamoDB partition key is the 64-bit xxHash of the table base path,
with `s3a://` normalized to `s3://` so the
same table addressed either way takes one lock.
**Why it is worth documenting.** `DynamoDBBasedLockProvider` *requires*
`hoodie.write.lock.dynamodb.partition_key` —
it throws `"Config key is not found"` when absent — and that config has **no
default value**, only an infer function
falling back to `HoodieTableConfig.NAME`. So two tables that happen to share
a name, in different databases or under
different paths, resolve to the same lock and serialize writers that never
touch the same data. Deriving the key from
the base path removes that with no per-table configuration.
The section says "infers it from the table name" rather than "defaults to
the table name", because inference via
`withInferFunction` and a literal default are not the same thing and the
distinction shows up when the config is
inspected.
### Version scope — checked per release, not assumed
Applied to `next` and **every** 1.x versioned copy. I treated "applicable"
as more than "the class exists", because on
a recent docs PR (#19572) a config that existed at 1.0.0/1.0.1 turned out to
throw there:
| Release | Class present | Implementation |
|---|---|---|
| release-1.0.0 | yes | identical, md5 `a89ab18e` |
| release-1.0.1 | yes | identical, md5 `a89ab18e` |
| release-1.0.2 | yes | identical, md5 `d03bf9cb` |
| release-1.1.1 | yes | identical, md5 `d03bf9cb` |
| release-1.2.0 | yes | identical, md5 `d03bf9cb` |
The two variants differ by exactly one line — an import moving from
`S3Utils.s3aToS3` to `FSUtils.s3aToS3` — so the
xxHash derivation, the `s3a` normalization and the base-class inheritance
are the same in all five. Searching the
tracker for the class name returns only this documentation issue, with
nothing reporting it broken on any release.
0.14.x and 0.15.x are excluded.
### Placement
The page is laid out differently across versions, so the section adapts
rather than being pasted identically:
- `next`, 1.1.1 and 1.2.0 use `### DynamoDB-Based Lock Provider`, so the new
section is an `###`
- 1.0.0, 1.0.1 and 1.0.2 use `#### Amazon DynamoDB based`, so it is a `####`
Each is inserted immediately before that version's FileSystem provider
heading. The added text is byte-identical within
each of the two groups, and I verified in the rendered HTML that the new
heading sits at the same level as its siblings
on both layouts — `h4` next to `h4` on 1.0.2, `h3` next to `h3` on next.
### Site verification
`npm run build` passes with the warning set **byte-identical** to a freshly
built baseline at the same base commit
(`d11a5b0adee4`). The earlier baselines were stale after this branch was
rebased onto the new head, so the baseline was
rebuilt rather than reused; the stash/restore round-trip used to produce it
was checked byte-identical on the way back.
Rendering confirmed under `npm run serve` on `/docs`, `/docs/next`,
`/docs/1.1.1`, `/docs/1.0.2`, `/docs/1.0.1` and
`/docs/1.0.0`: the heading, its anchor, the TOC entry, the `properties` code
fence and the note all render on each.
`/docs/0.15.1` correctly shows none of it.
### Impact
Documentation only. No code, config, or behaviour change.
### Risk Level
none
### Documentation Update
This PR is the documentation update — the Concurrency Control page,
`/docs/concurrency_control#distributed-locking` and
`/docs/next/concurrency_control#distributed-locking`.
### Contributor's checklist
- [x] Read through [contributor's
guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Enough context is provided in the sections above
- [x] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
