bvaradar commented on issue #1491: [SUPPORT] OutOfMemoryError during upsert 30M records URL: https://github.com/apache/incubator-hudi/issues/1491#issuecomment-610079124 Ohh. I didn't realize that. Haven't tried ingesting 30M record ingestion in local mode with 2 threads. You might be stretching the limits here. The OOM error happens in executor. How much is executor memory ? Try giving more ? If you are running this with local docker, have you given docker desktop with enough memory to docker engine ?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
