lamber-ken edited a comment on issue #1491: [SUPPORT] OutOfMemoryError during upsert 53M records URL: https://github.com/apache/incubator-hudi/issues/1491#issuecomment-615473505 Hi @tverdokhlebd, glad to hear that your problem has been solved. > Upsert was hanging without this parameter When do upsert operation, hudi need to buffer input records in memory, controls the size of the buffer is controlled by the option `hoodie.memory.merge.max.size`. if the value of that option is small, hudi will spill contents to disk, in that case, the upsert performance will decline. > without "hoodie.memory.merge.max.size" parameter and got error (heap size). BTW, I can't understand why without that parameter, will get error(heap size), have you encountered a similar situation before? @vinothchandar
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
