vinothchandar commented on issue #1694: URL: https://github.com/apache/hudi/issues/1694#issuecomment-637256106
Is there a reason why you are setting the shuffle parallelism to 5? When it seems like you have more executors? We can go step by step . Happy to work with you thru the tuning process. Can you please summarize your workload - records per partition, upsets vs insert ratio, ordered vs random keys. Below are some useful resources https://cwiki.apache.org/confluence/display/HUDI/Tuning+Guide https://cwiki.apache.org/confluence/display/HUDI/FAQ https://cwiki.apache.org/confluence/display/HUDI/FAQ#FAQ-HowdoImodelthedatastoredinHudi ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
