rangareddy opened a new pull request, #19444:
URL: https://github.com/apache/hudi/pull/19444
### Describe the issue this Pull Request addresses
Closes #15478 (HUDI-5011).
The ticket asks for the Java write client to be exercised against the
embedded timeline server, after a
community report, with DIRECT markers given as the interim workaround. It
carries no reproduction and no
detail beyond that.
Nothing covered the combination. `HoodieJavaClientTestHarness`,
`TestJavaHoodieBackedMetadata`,
`TestHoodieMetadataBase` and `HoodieFileGroupReaderOnJavaTestBase` all call
`withEmbeddedTimelineServerEnabled(false)`, and the one test that
parameterises it,
`TestHoodieJavaWriteClientInsert#testWriteClientAndTableServiceClientWithTimelineServer`,
only checks that
the write client and table-service client share a passed-in
`EmbeddedTimelineService` — it performs no write
and never touches markers. `hudi-java-client` had no reference to
`WriteMarkersFactory` at all.
The marker angle matters because
`HoodieWriteConfig.Builder#getDefaultMarkersType` returns `DIRECT` for the
Java engine, so the workaround suggested on the ticket is already the
default; the timeline-server-based path
is only reached when `hoodie.write.markers.type` is set explicitly.
### Summary and Changelog
-
`TestHoodieJavaWriteClientInsert#testInsertWithEmbeddedTimelineServerAndMarkerType`:
parameterised over
both `MarkerType` values with `withEmbeddedTimelineServerEnabled(true)`.
It asserts the embedded timeline
server is present, performs a real insert and commit, and checks the
commit completed and a base file was
written.
Both cases pass. The `TIMELINE_SERVER_BASED` case genuinely instantiates
`TimelineServerBasedWriteMarkers` (confirmed from `WriteMarkersFactory`
debug output, with no "Falling back
to direct markers" warning), so the timeline-server marker path runs end
to end against the embedded
server. The reported problem does not reproduce on current master with
either marker type.
- `HoodieWriteConfig#getDefaultMarkersType`: corrected the comment. It read
"Timeline-server-based marker is
not supported for Flink and Java engines", which is not accurate —
`TestFlinkWriteClients#testMarkerType` already asserts Flink receives
`TimelineServerBasedWriteMarkers`
when the type is set explicitly, and the new test shows the same for Java.
`DIRECT` is the *default* for
those engines, not the only option. Comment only, no behaviour change.
Worth mentioning what I did not do: I first implemented an extra fallback in
`WriteMarkersFactory.get` so
that `JAVA`/`FLINK` asking for `TIMELINE_SERVER_BASED` would warn and fall
back to `DIRECT`, matching the two
fallbacks already there and the comment above.
`TestFlinkWriteClients#testMarkerType` asserts the opposite,
so that change would have broken it and removed a working capability rather
than fixing a bug. It was
reverted, and the misleading comment corrected instead.
### Impact
Test coverage plus a comment. No production behaviour change, no API, config
or format change.
### Risk Level
none
### Documentation Update
none — no new config and no default value change.
### Contributor's checklist
- [x] Read through [contributor's
guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Enough context is provided in the sections above
- [x] Adequate tests were added if applicable
- [x] CI passes on my PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
