wangxianghu commented on pull request #1827: URL: https://github.com/apache/hudi/pull/1827#issuecomment-696467101
> My primary motive of suggesting parallelDo model, is to avoid splitting the classes and still reap benefits of parallel execution, provided by each engine. I don't think we are realizing them, as this stage yet. Please let me know your thoughts. @vinothchandar The `org.apache.hudi.table.SparkMarkerFiles` is used in many places and the refactor work is huge if rollback them in one class, so I refactored the function using bi function first(without packaging them in one class) just to show you the functional changes, thinking if you agree with this refactor then I can rollback them without splitting classes. It is just an example. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
