[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r903857715 ## core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala: ## @@ -1218,6 +1262,53 @@ private[spark] class TaskSetManager( def executorAdded(): Unit = {

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r903856288 ## core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala: ## @@ -1069,25 +1084,70 @@ private[spark] class TaskSetManager( * Check if the task

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r903836187 ## core/src/main/scala/org/apache/spark/internal/config/package.scala: ## @@ -2073,6 +2073,41 @@ package object config { .timeConf(TimeUnit.MILLISECONDS)

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r903836187 ## core/src/main/scala/org/apache/spark/internal/config/package.scala: ## @@ -2073,6 +2073,41 @@ package object config { .timeConf(TimeUnit.MILLISECONDS)

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r903830839 ## core/src/main/scala/org/apache/spark/internal/config/package.scala: ## @@ -2073,6 +2073,41 @@ package object config { .timeConf(TimeUnit.MILLISECONDS)

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r903830839 ## core/src/main/scala/org/apache/spark/internal/config/package.scala: ## @@ -2073,6 +2073,41 @@ package object config { .timeConf(TimeUnit.MILLISECONDS)

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r903830839 ## core/src/main/scala/org/apache/spark/internal/config/package.scala: ## @@ -2073,6 +2073,41 @@ package object config { .timeConf(TimeUnit.MILLISECONDS)

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r902126766 ## core/src/main/scala/org/apache/spark/internal/config/package.scala: ## @@ -2073,6 +2073,37 @@ package object config { .timeConf(TimeUnit.MILLISECONDS)

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r902100661 ## core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala: ## @@ -1218,6 +1248,70 @@ private[spark] class TaskSetManager( def executorAdded(): Unit = {

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r899037493 ## core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala: ## @@ -863,6 +872,30 @@ private[spark] class TaskSchedulerImpl( executorUpdates) }

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r896408996 ## core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala: ## @@ -1069,25 +1084,56 @@ private[spark] class TaskSetManager( * Check if the task

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r896407476 ## core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala: ## @@ -1069,25 +1084,56 @@ private[spark] class TaskSetManager( * Check if the task

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r896402450 ## core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala: ## @@ -1069,25 +1084,56 @@ private[spark] class TaskSetManager( * Check if the task

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r896400772 ## core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala: ## @@ -1069,25 +1084,56 @@ private[spark] class TaskSetManager( * Check if the task

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r895722949 ## core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala: ## @@ -1218,6 +1249,71 @@ private[spark] class TaskSetManager( def executorAdded(): Unit = {

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r895718247 ## core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala: ## @@ -1218,6 +1249,71 @@ private[spark] class TaskSetManager( def executorAdded(): Unit = {

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r895715985 ## core/src/main/scala/org/apache/spark/internal/config/package.scala: ## @@ -2073,6 +2073,37 @@ package object config { .timeConf(TimeUnit.MILLISECONDS)

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r891311964 ## core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala: ## @@ -1217,6 +1260,71 @@ private[spark] class TaskSetManager( def executorAdded(): Unit = {

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r891337273 ## core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala: ## @@ -1108,45 +1164,48 @@ private[spark] class TaskSetManager( //

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r891335474 ## core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala: ## @@ -1217,6 +1260,71 @@ private[spark] class TaskSetManager( def executorAdded(): Unit = {

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r891334508 ## core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala: ## @@ -1217,6 +1260,71 @@ private[spark] class TaskSetManager( def executorAdded(): Unit = {

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r891324885 ## core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala: ## @@ -1217,6 +1260,71 @@ private[spark] class TaskSetManager( def executorAdded(): Unit = {

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r891322239 ## core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala: ## @@ -1217,6 +1260,71 @@ private[spark] class TaskSetManager( def executorAdded(): Unit = {

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r891318626 ## core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala: ## @@ -863,6 +872,29 @@ private[spark] class TaskSchedulerImpl( executorUpdates) }

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r891316334 ## core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala: ## @@ -800,6 +814,10 @@ private[spark] class TaskSetManager(

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r891314239 ## core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala: ## @@ -1217,6 +1260,71 @@ private[spark] class TaskSetManager( def executorAdded(): Unit = {

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r891312743 ## core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala: ## @@ -1217,6 +1260,71 @@ private[spark] class TaskSetManager( def executorAdded(): Unit = {

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r891311964 ## core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala: ## @@ -1217,6 +1260,71 @@ private[spark] class TaskSetManager( def executorAdded(): Unit = {

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r891307188 ## core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala: ## @@ -1217,6 +1260,71 @@ private[spark] class TaskSetManager( def executorAdded(): Unit = {

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r886322452 ## core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala: ## @@ -853,8 +857,11 @@ private[spark] class TaskSchedulerImpl( // (taskId, stageId,

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r886318441 ## core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala: ## @@ -853,8 +857,11 @@ private[spark] class TaskSchedulerImpl( // (taskId, stageId,

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r885645567 ## core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala: ## @@ -853,8 +857,11 @@ private[spark] class TaskSchedulerImpl( // (taskId, stageId,

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r885265232 ## core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala: ## @@ -109,6 +116,13 @@ private[spark] class TaskSetManager( private val

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r885264474 ## core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala: ## @@ -80,12 +82,17 @@ private[spark] class TaskSetManager( val copiesRunning = new

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r885262650 ## core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala: ## @@ -103,6 +104,9 @@ private[spark] class TaskSchedulerImpl( // of tasks that are very

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r885252781 ## core/src/main/scala/org/apache/spark/internal/config/package.scala: ## @@ -2073,6 +2073,41 @@ package object config { .timeConf(TimeUnit.MILLISECONDS)

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r885249138 ## core/src/main/scala/org/apache/spark/internal/config/package.scala: ## @@ -2073,6 +2073,41 @@ package object config { .timeConf(TimeUnit.MILLISECONDS)

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r885249138 ## core/src/main/scala/org/apache/spark/internal/config/package.scala: ## @@ -2073,6 +2073,41 @@ package object config { .timeConf(TimeUnit.MILLISECONDS)

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r885247661 ## core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala: ## @@ -1217,6 +1289,61 @@ private[spark] class TaskSetManager( def executorAdded(): Unit = {

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r878134911 ## core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala: ## @@ -863,6 +870,29 @@ private[spark] class TaskSchedulerImpl( executorUpdates) }

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r877700327 ## core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala: ## @@ -863,6 +870,29 @@ private[spark] class TaskSchedulerImpl( executorUpdates) }

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r877700761 ## core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala: ## @@ -769,6 +785,25 @@ private[spark] class TaskSetManager( } } + def

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r866424914 ## core/src/main/scala/org/apache/spark/SparkStatusTracker.scala: ## @@ -120,4 +120,8 @@ class SparkStatusTracker private[spark] (sc: SparkContext, store:

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r865888195 ## core/src/main/scala/org/apache/spark/internal/config/package.scala: ## @@ -2051,6 +2051,39 @@ package object config { .doubleConf

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r865580173 ## core/src/main/scala/org/apache/spark/internal/config/package.scala: ## @@ -2051,6 +2051,39 @@ package object config { .doubleConf

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r865537726 ## core/src/main/scala/org/apache/spark/SparkStatusTracker.scala: ## @@ -120,4 +120,8 @@ class SparkStatusTracker private[spark] (sc: SparkContext, store:

[GitHub] [spark] Ngone51 commented on a diff in pull request #36162: [SPARK-32170][CORE] Improve the speculation through the stage task metrics.

Ngone51 commented on code in PR #36162: URL: https://github.com/apache/spark/pull/36162#discussion_r855732467 ## core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala: ## @@ -1217,6 +1276,123 @@ private[spark] class TaskSetManager( def executorAdded(): Unit = {