[GitHub] [spark] xuanyuanking edited a comment on issue #24110: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

xuanyuanking edited a comment on issue #24110: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24110#issuecomment-479711195 @cloud-fan As our offline discussion, thanks for your excellent idea about

[GitHub] [spark] xuanyuanking edited a comment on issue #24110: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

xuanyuanking edited a comment on issue #24110: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24110#issuecomment-479760400 `python/run-tests --testnames 'pyspark.sql.tests.test_dataframe

[GitHub] [spark] xuanyuanking edited a comment on issue #24110: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

xuanyuanking edited a comment on issue #24110: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24110#issuecomment-479711195 @cloud-fan Thanks for your excellent idea about keeping the "generation id"

[GitHub] [spark] xuanyuanking edited a comment on issue #24110: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

xuanyuanking edited a comment on issue #24110: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24110#issuecomment-474424797 ``` To help other people review this patch, can you add the following

[GitHub] [spark] xuanyuanking edited a comment on issue #24110: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

xuanyuanking edited a comment on issue #24110: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24110#issuecomment-474424797 ``` To help other people review this patch, can you add the following

[GitHub] [spark] xuanyuanking edited a comment on issue #24110: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

xuanyuanking edited a comment on issue #24110: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24110#issuecomment-474424797 ``` To help other people review this patch, can you add the following

[GitHub] [spark] xuanyuanking edited a comment on issue #24110: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

xuanyuanking edited a comment on issue #24110: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24110#issuecomment-473352597 Test this please.