kbuci opened a new pull request, #18214: URL: https://github.com/apache/hudi/pull/18214
### Describe the issue this Pull Request addresses When a writer configures SPILLABLE_DISK filesystem view when timeline server is disabled there can be correctness issues due to data being lost during serialization/de-serialization. See context in https://github.com/apache/hudi/issues/17957 ### Summary and Changelog - When HoodieWriteConfig is created, forcibly switch primary/secondary filesystem view to be IN_MEMORY if -- timeline server is disabled -- SPILLABLE_DISK is used ### Impact - A user not using timeline server but using SPILLABLE_DISK may see their jobs fail due to memory pressure. ### Risk Level low - it is arguably preferable for a user's job to fail more due to memory pressure rather than to encounter the subtle correctness failures in https://github.com/apache/hudi/issues/17957 ### Documentation Update - We can update docs to indicate that if SPILLABLE_DISK is used the timeline server should be enabled. ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [ ] Enough context is provided in the sections above - [ ] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
