loukey-lj opened a new pull request #2433: URL: https://github.com/apache/hudi/pull/2433
InstantGenerateOperator support multiple parallelism. When InstantGenerateOperator subtask size greater than 1 we can set subtask 0 as a main subtask, only main task create new instant. The prerequisite of create new instant is exist subtask received data in current checkpoint. Every subtask will create a tmp file, flie name is make up by checkpointid,subtask index and received records size. The main subtask will check every subtask file and parse file to make sure is shuold to create new instant. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
