rangareddy opened a new pull request, #19592:
URL: https://github.com/apache/hudi/pull/19592
### Describe the issue this Pull Request addresses
Addresses #16461 (HUDI-7659), which asked to *"update 0.14.0 release docs to
call out that row writer w/
clustering is enabled by default"*.
**The premise does not hold for 0.14.0, so this PR documents the behaviour
where it is actually true instead
of writing a false statement into a release-note page.** Details in
Verification below; the short version is
that the clustering fallback for that config read `false` at
`release-0.14.0`.
The real gap the ticket points at is genuine and unaddressed:
`clustering.md` never mentioned that on Spark
the execution strategies can rewrite file groups either through the **row
writer** or through the older RDD
path, which one runs by default, or the config that switches between them.
`MultipleSparkJobExecutionStrategy`
picks the path from `hoodie.datasource.write.row.writer.enable`; the docs
were silent.
### Summary and Changelog
Adds a **Row writer** subsection under `### Execution Strategy` in
`clustering.md`, covering the config, its
default, and how to force the RDD path.
The default is deliberately given two bullets rather than one sentence,
because "enabled by default" is two
distinct claims that merely agree on current releases:
* the config itself defaults to `true`, and `HoodieWriterUtils` calls
`setDefaultValue(ENABLE_ROW_WRITER)` on
every Spark datasource write — so clustering driven from a datasource
write takes the row path unless turned
off;
* clustering separately applies its **own fallback** when the key is absent
from the write config, which is
what a standalone or async clustering job sees.
### Verification
Docs change, so no test to add. Every statement was read off the tags rather
than recalled.
**The switch itself** — `MultipleSparkJobExecutionStrategy` chooses the
path, and the fallback has moved
around across releases:
| ref | `getBooleanOrDefault("hoodie.datasource.write.row.writer.enable",
…)` |
| --- | --- |
| `release-0.13.0`, `release-0.13.1` | `false` |
| **`release-0.14.0`** | **`false`** ← the release the ticket names |
| `release-0.14.1` | `true` |
| `release-0.15.0`, `release-0.15.1` | `false` (flipped back) |
| `release-1.0.0` … `release-1.2.0`, master | `true` |
That is why the 0.14.0 release notes are left untouched: for that release
the requested sentence would be
incorrect, and 0.15.0 reverted it again. Writing it there would mislead
anyone reading the 0.14.0 notes.
**The datasource half:**
`HoodieWriterUtils.setDefaultValue(ENABLE_ROW_WRITER)` is present at
`release-1.0.0`, `1.0.1`, `1.0.2`, `1.1.1` and `1.2.0` (and at `0.14.0`),
and `ENABLE_ROW_WRITER` itself
carries `defaultValue("true")` — so the two halves of the claim are
independently confirmed.
**Version coverage, and why the text is not identical across copies.**
Applied to the current docs plus the
five supported versioned copies. `1.0.0`, `1.0.1` and `1.0.2` gate the row
path on a second condition —
`HoodieDataTypeUtils.canUseRowWriter(schema, conf)`, which returns `false`
when
`parquet.avro.write-old-list-structure` is `false` **and** the schema
carries both a small-precision decimal
and a list or map field, logging `Cannot use row writer due to presence of
list or map with a small precision
decimal field`. That condition is **absent from 1.1.1 onwards**, so only the
three 1.0.x copies describe it:
```
website/docs/clustering.md +22 (config only)
website/versioned_docs/version-1.2.0/clustering.md +22 (config only)
website/versioned_docs/version-1.1.1/clustering.md +22 (config only)
website/versioned_docs/version-1.0.2/clustering.md +28 (config + schema
gate)
website/versioned_docs/version-1.0.1/clustering.md +28 (config + schema
gate)
website/versioned_docs/version-1.0.0/clustering.md +28 (config + schema
gate)
```
0.15.x and 0.14.x are left alone: their fallback was `false`, so this text
would not be true there.
`markdownlint` run over all six files before and after, compared by rule
*class* (these pages already carry
ten classes, so raw counts are meaningless): **no new class of finding**.
The added region is byte-identical
within each of the two variant groups, contains no `<` or `{` outside inline
code, and `#### Row writer` nests
correctly under `### Execution Strategy` and before `### Update Strategy` in
every copy.
**Not done:** the Docusaurus build (`website/node_modules` absent, full
install heavy). The section adds one
table and prose, both constructs already on this page.
### Impact
Documentation only — no code, config or format change. A behaviour that
previously required reading
`MultipleSparkJobExecutionStrategy` to discover is now stated, including the
part that trips people up: the
answer differs between a datasource write and a standalone clustering job.
Worth flagging for reviewers: **#17341 (HUDI-8749) "Deprecate clustering
with row writer" is open**, asking
whether this path is still needed given the file-group reader flow. If that
lands, this section becomes the
natural place to add the deprecation note — documenting it now does not
obstruct that, and arguably helps,
since users currently have no way to know which path they are on.
### Risk Level
none
### Documentation Update
This is the documentation update. Targets `asf-site`; applied to `docs/`
plus versioned copies 1.2.0, 1.1.1,
1.0.2, 1.0.1 and 1.0.0.
### Contributor's checklist
- [x] Read through [contributor's
guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Enough context is provided in the sections above
- [x] Adequate tests were added if applicable — n/a for docs; verification
tables above instead
- [ ] CI passes on my PR — `asf-site` PRs do not run the `master` gates;
`markdownlint` parity checked locally
as described
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
