GitHub user Ngone51 opened a pull request:
https://github.com/apache/spark/pull/20033
[SPARK-22847] [CORE] Remove duplicate code in AppStatusListener while
assigning schedulingPool for stage
## What changes were proposed in this pull request?
In AppStatusListener&#
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20039#discussion_r158424923
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/LiveListenerBus.scala ---
@@ -149,7 +158,11 @@ private[spark] class LiveListenerBus(conf
GitHub user Ngone51 opened a pull request:
https://github.com/apache/spark/pull/20053
Init lastReportTimestamp with system current time when start() calledâ¦
⦠in AsyncEventQueue
## What changes were proposed in this pull request?
```
if
GitHub user Ngone51 opened a pull request:
https://github.com/apache/spark/pull/20056
[SPARK-22878] [CORE] Count totalDroppedEvents for LiveListenerBus
## What changes were proposed in this pull request?
Count total dropped events from all queues' numDroppedEvent
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20053#discussion_r158572161
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/AsyncEventQueue.scala ---
@@ -112,6 +112,7 @@ private class AsyncEventQueue(val name: String, conf
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20039#discussion_r158572288
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/LiveListenerBus.scala ---
@@ -149,7 +158,11 @@ private[spark] class LiveListenerBus(conf
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20039#discussion_r158573360
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/LiveListenerBus.scala ---
@@ -149,7 +158,11 @@ private[spark] class LiveListenerBus(conf
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20039#discussion_r158573853
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/LiveListenerBus.scala ---
@@ -149,7 +158,11 @@ private[spark] class LiveListenerBus(conf
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20039#discussion_r158574199
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/LiveListenerBus.scala ---
@@ -125,13 +128,39 @@ private[spark] class LiveListenerBus(conf
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20056
cc: @maropu
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20056
@gatorsmile @HyukjinKwon @dongjoon-hyun Can any one of you have a look at
this pr, Thanks!
---
-
To unsubscribe, e-mail
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20056
please retest it. Thanks.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20056#discussion_r158751285
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/LiveListenerBus.scala ---
@@ -25,14 +25,13 @@ import scala.collection.JavaConverters._
import
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20056
It seems something wrong while testing 'ReplayListenerSuite':
```
[info] ReplayListenerSuite:
[info] - Simple replay (97 milliseconds)
[info] - Replay compressed inprogres
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20056
@maropu Actually, I didn't modify this unit test ever. And my unit test
locate in SparkListenerSuite haven'been started according to the "
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20039#discussion_r159076507
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/LiveListenerBus.scala ---
@@ -149,7 +158,11 @@ private[spark] class LiveListenerBus(conf
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20039#discussion_r159081427
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/LiveListenerBus.scala ---
@@ -149,7 +158,11 @@ private[spark] class LiveListenerBus(conf
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20056
@vanzin @squito Can you have a look at this pr ? Thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
GitHub user Ngone51 opened a pull request:
https://github.com/apache/spark/pull/20199
[Spark-22967][Hive]Fix VersionSuite's unit tests by change Windows path
into URI path
## What changes were proposed in this pull request?
Two unit test will fail due to Windows f
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20199#discussion_r160344054
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/client/VersionsSuite.scala ---
@@ -58,7 +58,7 @@ class VersionsSuite extends SparkFunSuite with
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20199
cc @HyukjinKwon
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20199#discussion_r160404821
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/client/VersionsSuite.scala ---
@@ -58,7 +58,7 @@ class VersionsSuite extends SparkFunSuite with
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20199
Ok, will try.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20199#discussion_r160709123
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/client/VersionsSuite.scala ---
@@ -842,6 +842,7 @@ class VersionsSuite extends SparkFunSuite
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20056
@squito by now, the variable `droppedEventsCounter ` is unused. So, I want
to reuse it.
---
-
To unsubscribe, e-mail: reviews
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20056
@squito yea, clean up will be fine. Maybe, I make things more complicated.
Thank for comment.
---
-
To unsubscribe, e-mail
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/21096
Thank you for your comments @markhamstra .
Yeah, I'm considering adding a UT to support this change. And thank for
reminding me of DAGScheduler's basic
GitHub user Ngone51 opened a pull request:
https://github.com/apache/spark/pull/21104
[SPARK-24021][CORE] fix bug in BlacklistTracker's
updateBlacklistForFetchFailure
## What changes were proposed in this pull request?
Thereâs a miswrite in BlacklistTrac
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/21104
ping @jerryshao
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/21096
ping @jiangxb1987 @squito
Would you please have a look at this PR? What's your opinions on the cache
str
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20930
Hi, @xuanyuanking , thank for your patient explanation, sincerely.
With regard to your latest explanation:
> stage 2's shuffleID is 1, but stage 3 failed by missing an ou
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20930
> because we can get the MapStatus, but get a 'null'. If I'm not mistaken,
this also because the ExecutorLost trigger removeOutputsOnExecutor
If there's a `null` Map
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20998#discussion_r183408481
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala ---
@@ -287,7 +287,7 @@ private[spark] class TaskSetManager(
None
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20998
Agree and thank you @squito .
And thanks for all of you. @felixcheung @mridulm @jiangxb1987 @srowen
---
-
To
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21131#discussion_r183619704
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala ---
@@ -689,6 +689,20 @@ private[spark] class TaskSchedulerImpl
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21131#discussion_r183619646
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala ---
@@ -689,6 +689,20 @@ private[spark] class TaskSchedulerImpl
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21131#discussion_r183690133
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala ---
@@ -917,4 +917,111 @@ class TaskSchedulerImplSuite extends
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21131#discussion_r183701269
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala ---
@@ -917,4 +917,111 @@ class TaskSchedulerImplSuite extends
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21131#discussion_r183789814
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala ---
@@ -917,4 +917,111 @@ class TaskSchedulerImplSuite extends
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21131#discussion_r183790463
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala ---
@@ -917,4 +917,111 @@ class TaskSchedulerImplSuite extends
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21131#discussion_r183797532
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala ---
@@ -917,4 +917,111 @@ class TaskSchedulerImplSuite extends
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20604
ping @squito
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20930
No wonder I can't understand the issue for a long time since I've thought
it happened on Spark2.3 . And now it makes sense. Thanks @j
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/21175
@manbuyun you need to add the unit test into `ChunkedByteBufferSuite.scala`
and push a new commit.
---
-
To unsubscribe, e-mail
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/21131
LGTM, and nice UT.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21175#discussion_r184590989
--- Diff:
core/src/test/scala/org/apache/spark/io/ChunkedByteBufferSuite.scala ---
@@ -56,6 +56,12 @@ class ChunkedByteBufferSuite extends SparkFunSuite
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21175#discussion_r184596199
--- Diff:
core/src/test/scala/org/apache/spark/io/ChunkedByteBufferSuite.scala ---
@@ -56,6 +56,12 @@ class ChunkedByteBufferSuite extends SparkFunSuite
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21175#discussion_r184597197
--- Diff:
core/src/test/scala/org/apache/spark/io/ChunkedByteBufferSuite.scala ---
@@ -56,6 +56,12 @@ class ChunkedByteBufferSuite extends SparkFunSuite
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21175#discussion_r184607965
--- Diff:
core/src/test/scala/org/apache/spark/io/ChunkedByteBufferSuite.scala ---
@@ -56,6 +56,12 @@ class ChunkedByteBufferSuite extends SparkFunSuite
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/21175
cc @kiszk @maropu @cloud-fan @jiangxb1987
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20604
ping @squito
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20604#discussion_r185159109
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -1643,7 +1646,10 @@ class SparkContext(config: SparkConf) extends
Logging
GitHub user Ngone51 opened a pull request:
https://github.com/apache/spark/pull/21209
[SPARK-24141][CORE] Fix bug in CoarseGrainedSchedulerBackend.killExecutors
## What changes were proposed in this pull request?
In method *CoarseGrainedSchedulerBackend.killExecutors
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/21209
ping @squito @vanzin
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21212#discussion_r186261650
--- Diff:
core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala
---
@@ -267,28 +269,30 @@ final class ShuffleBlockFetcherIterator
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/21209
ping @jiangxb1987
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/21096
Thanks for your opinions @squito @markhamstra .
Maybe, I should leave it for now.
---
-
To unsubscribe, e-mail
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20604#discussion_r186425765
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -1643,7 +1646,10 @@ class SparkContext(config: SparkConf) extends
Logging
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/21209
Thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21221#discussion_r187247534
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/PeakExecutorMetrics.scala ---
@@ -0,0 +1,127 @@
+/*
+ * Licensed to the Apache Software
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21221#discussion_r187244792
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -169,6 +179,27 @@ private[spark] class EventLoggingListener
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21221#discussion_r187238940
--- Diff: core/src/main/scala/org/apache/spark/Heartbeater.scala ---
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21221#discussion_r187248156
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -93,6 +94,10 @@ private[spark] class EventLoggingListener
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21221#discussion_r187236701
--- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
---
@@ -1753,9 +1766,21 @@ class DAGScheduler
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21221#discussion_r187239219
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -169,6 +179,27 @@ private[spark] class EventLoggingListener
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21221#discussion_r187823298
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -169,6 +179,27 @@ private[spark] class EventLoggingListener
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21221#discussion_r187824094
--- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
---
@@ -1753,9 +1766,21 @@ class DAGScheduler
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21221#discussion_r187825561
--- Diff: core/src/main/scala/org/apache/spark/Heartbeater.scala ---
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21369#discussion_r189438190
--- Diff:
core/src/main/scala/org/apache/spark/util/collection/ExternalAppendOnlyMap.scala
---
@@ -585,17 +592,15 @@ class ExternalAppendOnlyMap[K, V, C
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/21369
cc @JerryLead
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21369#discussion_r189892444
--- Diff:
core/src/main/scala/org/apache/spark/util/collection/ExternalAppendOnlyMap.scala
---
@@ -267,7 +273,7 @@ class ExternalAppendOnlyMap[K, V, C
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21369#discussion_r189892547
--- Diff:
core/src/main/scala/org/apache/spark/util/collection/ExternalAppendOnlyMap.scala
---
@@ -585,17 +591,25 @@ class ExternalAppendOnlyMap[K, V, C
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21369#discussion_r189894423
--- Diff:
core/src/test/scala/org/apache/spark/util/collection/ExternalAppendOnlyMapSuite.scala
---
@@ -414,6 +415,99 @@ class ExternalAppendOnlyMapSuite
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21369#discussion_r189939603
--- Diff:
core/src/main/scala/org/apache/spark/util/collection/ExternalAppendOnlyMap.scala
---
@@ -267,7 +273,7 @@ class ExternalAppendOnlyMap[K, V, C
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21440#discussion_r191175960
--- Diff:
core/src/test/scala/org/apache/spark/io/ChunkedByteBufferFileRegionSuite.scala
---
@@ -0,0 +1,154 @@
+/*
+ * Licensed to the Apache
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21440#discussion_r191104760
--- Diff:
core/src/main/scala/org/apache/spark/util/io/ChunkedByteBufferFileRegion.scala
---
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21440#discussion_r191175242
--- Diff:
core/src/main/scala/org/apache/spark/util/io/ChunkedByteBufferFileRegion.scala
---
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21440#discussion_r191175890
--- Diff:
core/src/test/scala/org/apache/spark/io/ChunkedByteBufferFileRegionSuite.scala
---
@@ -0,0 +1,154 @@
+/*
+ * Licensed to the Apache
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21440#discussion_r191117686
--- Diff:
core/src/main/scala/org/apache/spark/util/io/ChunkedByteBufferFileRegion.scala
---
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21440#discussion_r191176828
--- Diff:
core/src/test/scala/org/apache/spark/io/ChunkedByteBufferFileRegionSuite.scala
---
@@ -0,0 +1,154 @@
+/*
+ * Licensed to the Apache
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21440#discussion_r191178697
--- Diff:
core/src/test/scala/org/apache/spark/io/ChunkedByteBufferFileRegionSuite.scala
---
@@ -0,0 +1,154 @@
+/*
+ * Licensed to the Apache
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21440#discussion_r191182696
--- Diff:
core/src/test/scala/org/apache/spark/io/ChunkedByteBufferFileRegionSuite.scala
---
@@ -0,0 +1,154 @@
+/*
+ * Licensed to the Apache
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21440#discussion_r191623277
--- Diff:
core/src/test/scala/org/apache/spark/io/ChunkedByteBufferFileRegionSuite.scala
---
@@ -0,0 +1,154 @@
+/*
+ * Licensed to the Apache
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21486#discussion_r192591845
--- Diff: core/src/test/scala/org/apache/spark/HeartbeatReceiverSuite.scala
---
@@ -207,6 +210,55 @@ class HeartbeatReceiverSuite
assert
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21486#discussion_r192592142
--- Diff: core/src/test/scala/org/apache/spark/HeartbeatReceiverSuite.scala
---
@@ -207,6 +210,55 @@ class HeartbeatReceiverSuite
assert
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21486#discussion_r192592170
--- Diff: core/src/test/scala/org/apache/spark/HeartbeatReceiverSuite.scala
---
@@ -207,6 +210,55 @@ class HeartbeatReceiverSuite
assert
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/21494
Hi, @jiangxb1987 , can you explain more for what is `barrier scheduling` in
spark and elaborate an example which would only works with `barrier
scheduling`( but could not work under current spark
Github user Ngone51 closed the pull request at:
https://github.com/apache/spark/pull/20996
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user Ngone51 closed the pull request at:
https://github.com/apache/spark/pull/21096
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21486#discussion_r194606075
--- Diff: core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala ---
@@ -197,14 +197,14 @@ private[spark] class HeartbeatReceiver(sc:
SparkContext
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20977#discussion_r179338562
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/AsyncEventQueue.scala ---
@@ -166,7 +166,7 @@ private class AsyncEventQueue(val name: String, conf
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20977#discussion_r179490408
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/AsyncEventQueue.scala ---
@@ -166,7 +166,7 @@ private class AsyncEventQueue(val name: String, conf
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20987#discussion_r179647240
--- Diff: core/src/test/scala/org/apache/spark/executor/ExecutorSuite.scala
---
@@ -330,6 +362,15 @@ class FetchFailureHidingRDD(
case t
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20987#discussion_r179647613
--- Diff: core/src/test/scala/org/apache/spark/executor/ExecutorSuite.scala
---
@@ -173,8 +173,26 @@ class ExecutorSuite extends SparkFunSuite with
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20987
Things I'm concerned about is that does there exists a situation like 'a
task gets killed after it gets a FetchFailure, but re-run (not by resubmit)
later and gets a FetchFailure t
GitHub user Ngone51 opened a pull request:
https://github.com/apache/spark/pull/20996
[SPARK-23884][CORE] hasLaunchedTask should be true when launchedAnyTask be
true
## What changes were proposed in this pull request?
`hasLaunchedTask` should be `true` when
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20987
Yeahï¼agree.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
GitHub user Ngone51 opened a pull request:
https://github.com/apache/spark/pull/20998
[SPARK-23888][CORE] speculative task should not run on a given host where
another attempt is already running on
## What changes were proposed in this pull request?
There's a b
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20998
ping @pwendell @kayousterhout . pls help review, thanks :)
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20998
Hi, @felixcheung , thank for trigger a task and your comments.
> shouldn't this be up to the scheduler backend?
Actually, it is `TaskSchedulerImpl` who holds a thread to check
1 - 100 of 182 matches
Mail list logo
