vinothchandar commented on issue #714: Performance Comparison of HoodieDeltaStreamer and DataSourceAPI URL: https://github.com/apache/incubator-hudi/issues/714#issuecomment-511508649 Hmmm.. what I notice is that every stage (3, 15, 16) that goes one pass over the input takes like 3 minutes, which is not expected.. is there a way I can locally run this workload? Or happy to jump on a video call sometime to debug together.. Its hard for me to see why these stages would take such time. for e.g 2,3 is a simple spark shuffle and countByKey. None of hudi's features have been execised at that point.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
