nsivabalan opened a new pull request, #20413: URL: https://github.com/apache/hudi/pull/20413
### Describe the issue this Pull Request addresses Stacked on #20412. **Review only the last commit**; the first commit is #20412 and will drop out once it merges. `HoodieStreamingSink` always forced `hoodie.write.markers.type=DIRECT`, because the embedded timeline server used to be stopped after the first micro-batch. With #20412 the sink keeps one timeline server alive for the whole query, so that reason no longer holds. Related: #20400. ### Summary and Changelog - Removed the forced `DIRECT` markers override in `HoodieStreamingSink.addBatch`. Streaming writes now use the configured marker type, which defaults to `TIMELINE_SERVER_BASED` (and falls back to direct markers via `WriteMarkersFactory` when the timeline server is disabled). ### Impact Streaming writes stop creating one marker file per data file on storage and batch marker creation through the timeline server instead, matching batch writes and `HoodieStreamer`. Users can still set `hoodie.write.markers.type=DIRECT` or `hoodie.embed.timeline.server=false`. ### Risk Level low. Same marker path batch writes and `HoodieStreamer` already use. Verified with `TestStructuredStreaming` (all tests pass, COW and MOR, including async compaction and clustering). ### Documentation Update none ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
