kingkongpoon commented on issue #2557: URL: https://github.com/apache/hudi/issues/2557#issuecomment-783182791
> > When I first run the cow table (SaveMode.Overwrite), it's very fast.(about 700MB data in hdfs). but when I run an increment(SaveMode.Append), it's very slowly,and throw error > > Spark OOMs are hard to guess. Not sure if its directly related to Hudi. have you looked at the tuning guide? gc settings, memory overhead? https://cwiki.apache.org/confluence/display/HUDI/Tuning+Guide I have tried your advice , it seems to be better ,but the same code , configre and data, sometime also OOM, ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
