rubenssoto commented on pull request #1880: URL: https://github.com/apache/hudi/pull/1880#issuecomment-763290674
Hello, Hudi will have nice features like clustering and clustering probably will rewrite a lot of data, so is it possible this rewrites without new data doesn't affect downstream consumer of spark structured streaming? It is something like delta lake has on compaction operation https://docs.delta.io/latest/best-practices.html On compaction has .option("dataChange", "false"), so the downstream consumer won't be affected. Thank you. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
