rohit-m-99 commented on issue #5050: URL: https://github.com/apache/hudi/issues/5050#issuecomment-1072604667
Spoke with @nsivabalan on this, looks like the issue was not related to DeleteMarkers, rather unioning all the data. Clustering seems to still take a wild amount of resources given that our data < 10GB right now. However that is more discussed here: https://github.com/apache/hudi/issues/4891. Issue went away after ridding of a rollbakc and reducing our small file limit: https://hudi.apache.org/docs/configurations/#hoodieclusteringplanstrategysmallfilelimit Will reopen if issue reappears. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
