[GitHub] [spark] mridulm edited a comment on pull request #35185: [SPARK-37831][CORE] add task partition id in TaskInfo and Task Metrics

2022-01-17 Thread GitBox
mridulm edited a comment on pull request #35185: URL: https://github.com/apache/spark/pull/35185#issuecomment-1014966382 > > Took an initial pass through the PR and added some comments - overall looks good. We would need to make sure that skew join and partition coalescing in SQL interact

[GitHub] [spark] mridulm edited a comment on pull request #35185: [SPARK-37831][CORE] add task partition id in TaskInfo and Task Metrics

2022-01-13 Thread GitBox
mridulm edited a comment on pull request #35185: URL: https://github.com/apache/spark/pull/35185#issuecomment-1012496679 @tgravescs Agree, for the first stage attempt this should be fine. But for any stage attempt which is computing a subset of tasks (retries, specific partition computat

[GitHub] [spark] mridulm edited a comment on pull request #35185: [SPARK-37831][CORE] add task partition id in TaskInfo and Task Metrics

2022-01-13 Thread GitBox
mridulm edited a comment on pull request #35185: URL: https://github.com/apache/spark/pull/35185#issuecomment-1012496679 @tgravescs Agree, for the first stage attempt this should be fine. But for any stage attempt which is computing a subset of tasks (retries, specific partition computat