jerqi commented on PR #1514: URL: https://github.com/apache/incubator-uniffle/pull/1514#issuecomment-1937988904
One question: We can't judge the data is from the same task. If one map task A have two attempts taskAttempt 1, taskAttempt 2. TaskAttempt 1 can have two blocks, TaskAttempt 2 may have three blocks. Because Spark doesn't guarantee every taskAttempts have the same data. Because some transforms of Spark are indeterministic. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
