bwu2 commented on issue #1328: Hudi upsert hangs URL: https://github.com/apache/incubator-hudi/issues/1328#issuecomment-589446887 Thanks for your replies! @lamber-ken I will try again with that setting. Does increasing the memory available by setting `option("hoodie.memory.merge.max.size", "2004857600000")` work better than increasing executor memory or`hoodie.memory.merge.fraction`? Will it not result in an OOME? @vinothchandar Yes, we are re-running against real datasets in the next couple of days. I will report back. But do note that the original problem actually arose on a real dataset not a degenerate or artificial workload: it was not a particularly wide table (with 6 or 7 fields), with about 4m rows bulk inserted and then another 4m upserted (of which most were the same rows).
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
