[GitHub] [spark] squito commented on issue #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

2019-08-15 Thread GitBox
squito commented on issue #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24892#issuecomment-521687011 thanks for the looking into this @vanzin and @cloud-fan . I agree its unnecessarily

[GitHub] [spark] squito commented on issue #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

2019-08-12 Thread GitBox
squito commented on issue #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24892#issuecomment-520501415 Thanks for taking another look. Another thing to keep in mind w/ speculative

[GitHub] [spark] squito commented on issue #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

2019-08-12 Thread GitBox
squito commented on issue #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24892#issuecomment-520482491 Sorry I don't understand -- why don't you want to support speculative execution?

[GitHub] [spark] squito commented on issue #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

2019-08-07 Thread GitBox
squito commented on issue #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24892#issuecomment-519191919 > BTW, another way to fix this problem is: always include the task id (not task attempt

[GitHub] [spark] squito commented on issue #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

2019-08-06 Thread GitBox
squito commented on issue #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24892#issuecomment-518740629 > The task attempt id is not enough in this situation, because for the indeterminate