vinothchandar commented on issue #714: Performance Comparison of HoodieDeltaStreamer and DataSourceAPI URL: https://github.com/apache/incubator-hudi/issues/714#issuecomment-499981159 Great! Hudi writing is just an (advanced) spark job, so scaling tips that apply for spark also apply here generally. Defaults were tuned to 1500, to help handle upto 500GB input by default. I would increase parallelism by 1 for every 1GB of additional data (rough guess, you can easily experiment with values and see).. Please close this issue if things look good.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
