Github user jiangxb1987 commented on the issue:
    > A synthetic example:
    > => (computeKey(v._1, v._2), computeValue(v._1, 
    The above example may create some different output when retrying a subset 
of all the tasks. But I may not call it a data loss or data correctness issue. 
Let's image you run the query twice, each with different ordering of `rdd1` and 
`rdd2`, each run shall produce different outputs (even different # of output 
rows). The result produced by retrying a subset of all the tasks is still 
valid, it actually correspond to another input data representation,  though not 
the same as the initial input.
    Now I tend to believe there will not be data loss or data correctness 
issue, as long as you don't spread input data across partitions in a round 
robin way (or, in a way that is not related to the data itself), because on 
task retry you are guaranteed that all input data are covered (each row get 
recomputed exactly once, though maybe in different order). 


To unsubscribe, e-mail:
For additional commands, e-mail:

Reply via email to