deepakpanda93 commented on issue #6811: URL: https://github.com/apache/hudi/issues/6811#issuecomment-4832656556
Hi @nochimow Gentle follow-up, this has been open a while. Did you get a chance to try the suggestions from @nsivabalan above (checking executors for OOMs / tuning Spark memory, relaxing hoodie.cleaner.commits.retained from 1, enabling clustering to compact small files, or moving to BUCKET_INDEX)? Did any of those help with the index-tagging stage latency, or are you still seeing the slow upserts? Also, a lot has improved since 0.12 — index lookups and the metadata table in particular have come a long way in the 1.x line. If this is still an issue for you, would you be able to retry on a recent Hudi 1.x release? That'll help us confirm whether it's still reproducible or can be closed out. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
