nsivabalan commented on pull request #4821: URL: https://github.com/apache/hudi/pull/4821#issuecomment-1060036673
@danny0405 : sorry, I don't think we can remove the data table relying on metadata table compaction. can you help me understand. Myself and @yihua did jam on this. here is our claim In metadata table, we filter out any additional commits which is complete is MDT, but not yet complete in Data table. so, if we loosen the dependency, there could be a partially failed commit in data table which gets archived. In which case when we inpsect the commit in MDT, we might assume that its already committed in data table as well which could lead to wrong results. If I am missing something, let me know. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
