GitHub user Ngone51 opened a pull request:
https://github.com/apache/spark/pull/20033
[SPARK-22847] [CORE] Remove duplicate code in AppStatusListener while
assigning schedulingPool for stage
## What changes were proposed in this pull request?
In AppStatusListener's
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21212#discussion_r186261650
--- Diff:
core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala
---
@@ -267,28 +269,30 @@ final class ShuffleBlockFetcherIterator
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/21096
Thanks for your opinions @squito @markhamstra .
Maybe, I should leave it for now.
---
-
To unsubscribe, e-mail
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20604#discussion_r186425765
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -1643,7 +1646,10 @@ class SparkContext(config: SparkConf) extends
Logging
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/21209
ping @jiangxb1987
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/21175
cc @kiszk @maropu @cloud-fan @jiangxb1987
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21175#discussion_r184607965
--- Diff:
core/src/test/scala/org/apache/spark/io/ChunkedByteBufferSuite.scala ---
@@ -56,6 +56,12 @@ class ChunkedByteBufferSuite extends SparkFunSuite
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20604
ping @squito
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/21209
Thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21221#discussion_r187244792
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -169,6 +179,27 @@ private[spark] class EventLoggingListener
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21221#discussion_r187238940
--- Diff: core/src/main/scala/org/apache/spark/Heartbeater.scala ---
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21221#discussion_r187248156
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -93,6 +94,10 @@ private[spark] class EventLoggingListener
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21221#discussion_r187236701
--- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
---
@@ -1753,9 +1766,21 @@ class DAGScheduler
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21221#discussion_r187239219
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -169,6 +179,27 @@ private[spark] class EventLoggingListener
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21221#discussion_r187247534
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/PeakExecutorMetrics.scala ---
@@ -0,0 +1,127 @@
+/*
+ * Licensed to the Apache Software
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21221#discussion_r187823298
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -169,6 +179,27 @@ private[spark] class EventLoggingListener
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21221#discussion_r187825561
--- Diff: core/src/main/scala/org/apache/spark/Heartbeater.scala ---
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21221#discussion_r187824094
--- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
---
@@ -1753,9 +1766,21 @@ class DAGScheduler
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/21369
cc @JerryLead
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21369#discussion_r189438190
--- Diff:
core/src/main/scala/org/apache/spark/util/collection/ExternalAppendOnlyMap.scala
---
@@ -585,17 +592,15 @@ class ExternalAppendOnlyMap[K, V, C
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21369#discussion_r189939603
--- Diff:
core/src/main/scala/org/apache/spark/util/collection/ExternalAppendOnlyMap.scala
---
@@ -267,7 +273,7 @@ class ExternalAppendOnlyMap[K, V, C
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21369#discussion_r189892444
--- Diff:
core/src/main/scala/org/apache/spark/util/collection/ExternalAppendOnlyMap.scala
---
@@ -267,7 +273,7 @@ class ExternalAppendOnlyMap[K, V, C
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21369#discussion_r189892547
--- Diff:
core/src/main/scala/org/apache/spark/util/collection/ExternalAppendOnlyMap.scala
---
@@ -585,17 +591,25 @@ class ExternalAppendOnlyMap[K, V, C
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21369#discussion_r189894423
--- Diff:
core/src/test/scala/org/apache/spark/util/collection/ExternalAppendOnlyMapSuite.scala
---
@@ -414,6 +415,99 @@ class ExternalAppendOnlyMapSuite
Github user Ngone51 closed the pull request at:
https://github.com/apache/spark/pull/20996
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user Ngone51 closed the pull request at:
https://github.com/apache/spark/pull/21096
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21486#discussion_r194606075
--- Diff: core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala ---
@@ -197,14 +197,14 @@ private[spark] class HeartbeatReceiver(sc:
SparkContext
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21565#discussion_r195927590
--- Diff:
core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala ---
@@ -488,9 +488,16 @@ private[spark] class ExecutorAllocationManager
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21440#discussion_r191623277
--- Diff:
core/src/test/scala/org/apache/spark/io/ChunkedByteBufferFileRegionSuite.scala
---
@@ -0,0 +1,154 @@
+/*
+ * Licensed to the Apache
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21440#discussion_r191176828
--- Diff:
core/src/test/scala/org/apache/spark/io/ChunkedByteBufferFileRegionSuite.scala
---
@@ -0,0 +1,154 @@
+/*
+ * Licensed to the Apache
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21440#discussion_r191175242
--- Diff:
core/src/main/scala/org/apache/spark/util/io/ChunkedByteBufferFileRegion.scala
---
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21440#discussion_r191175890
--- Diff:
core/src/test/scala/org/apache/spark/io/ChunkedByteBufferFileRegionSuite.scala
---
@@ -0,0 +1,154 @@
+/*
+ * Licensed to the Apache
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21440#discussion_r191175960
--- Diff:
core/src/test/scala/org/apache/spark/io/ChunkedByteBufferFileRegionSuite.scala
---
@@ -0,0 +1,154 @@
+/*
+ * Licensed to the Apache
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21440#discussion_r191104760
--- Diff:
core/src/main/scala/org/apache/spark/util/io/ChunkedByteBufferFileRegion.scala
---
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21440#discussion_r191117686
--- Diff:
core/src/main/scala/org/apache/spark/util/io/ChunkedByteBufferFileRegion.scala
---
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21440#discussion_r191178697
--- Diff:
core/src/test/scala/org/apache/spark/io/ChunkedByteBufferFileRegionSuite.scala
---
@@ -0,0 +1,154 @@
+/*
+ * Licensed to the Apache
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21440#discussion_r191182696
--- Diff:
core/src/test/scala/org/apache/spark/io/ChunkedByteBufferFileRegionSuite.scala
---
@@ -0,0 +1,154 @@
+/*
+ * Licensed to the Apache
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21486#discussion_r192592142
--- Diff: core/src/test/scala/org/apache/spark/HeartbeatReceiverSuite.scala
---
@@ -207,6 +210,55 @@ class HeartbeatReceiverSuite
assert
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21486#discussion_r192592170
--- Diff: core/src/test/scala/org/apache/spark/HeartbeatReceiverSuite.scala
---
@@ -207,6 +210,55 @@ class HeartbeatReceiverSuite
assert
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21486#discussion_r192591845
--- Diff: core/src/test/scala/org/apache/spark/HeartbeatReceiverSuite.scala
---
@@ -207,6 +210,55 @@ class HeartbeatReceiverSuite
assert
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/21494
Hi, @jiangxb1987 , can you explain more for what is `barrier scheduling` in
spark and elaborate an example which would only works with `barrier
scheduling`( but could not work under current spark
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/21209
ping @squito @vanzin
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
GitHub user Ngone51 opened a pull request:
https://github.com/apache/spark/pull/21209
[SPARK-24141][CORE] Fix bug in CoarseGrainedSchedulerBackend.killExecutors
## What changes were proposed in this pull request?
In method *CoarseGrainedSchedulerBackend.killExecutors
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21175#discussion_r184590989
--- Diff:
core/src/test/scala/org/apache/spark/io/ChunkedByteBufferSuite.scala ---
@@ -56,6 +56,12 @@ class ChunkedByteBufferSuite extends SparkFunSuite
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21175#discussion_r184596199
--- Diff:
core/src/test/scala/org/apache/spark/io/ChunkedByteBufferSuite.scala ---
@@ -56,6 +56,12 @@ class ChunkedByteBufferSuite extends SparkFunSuite
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21175#discussion_r184597197
--- Diff:
core/src/test/scala/org/apache/spark/io/ChunkedByteBufferSuite.scala ---
@@ -56,6 +56,12 @@ class ChunkedByteBufferSuite extends SparkFunSuite
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/21175
@manbuyun you need to add the unit test into `ChunkedByteBufferSuite.scala`
and push a new commit.
---
-
To unsubscribe, e-mail
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/21131
LGTM, and nice UT.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20604#discussion_r185159109
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -1643,7 +1646,10 @@ class SparkContext(config: SparkConf) extends
Logging
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21565#discussion_r200019612
--- Diff:
core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala ---
@@ -488,9 +488,16 @@ private[spark] class ExecutorAllocationManager
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20039#discussion_r159076507
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/LiveListenerBus.scala ---
@@ -149,7 +158,11 @@ private[spark] class LiveListenerBus(conf
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20039#discussion_r159081427
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/LiveListenerBus.scala ---
@@ -149,7 +158,11 @@ private[spark] class LiveListenerBus(conf
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20039#discussion_r158424923
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/LiveListenerBus.scala ---
@@ -149,7 +158,11 @@ private[spark] class LiveListenerBus(conf
GitHub user Ngone51 opened a pull request:
https://github.com/apache/spark/pull/20053
Init lastReportTimestamp with system current time when start() calledâ¦
⦠in AsyncEventQueue
## What changes were proposed in this pull request
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20039#discussion_r158574199
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/LiveListenerBus.scala ---
@@ -125,13 +128,39 @@ private[spark] class LiveListenerBus(conf
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20039#discussion_r158573853
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/LiveListenerBus.scala ---
@@ -149,7 +158,11 @@ private[spark] class LiveListenerBus(conf
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20039#discussion_r158573360
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/LiveListenerBus.scala ---
@@ -149,7 +158,11 @@ private[spark] class LiveListenerBus(conf
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20053#discussion_r158572161
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/AsyncEventQueue.scala ---
@@ -112,6 +112,7 @@ private class AsyncEventQueue(val name: String, conf
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20039#discussion_r158572288
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/LiveListenerBus.scala ---
@@ -149,7 +158,11 @@ private[spark] class LiveListenerBus(conf
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20056
cc: @maropu
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20056
@vanzin @squito Can you have a look at this pr ? Thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20199#discussion_r160709123
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/client/VersionsSuite.scala ---
@@ -842,6 +842,7 @@ class VersionsSuite extends SparkFunSuite
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20199
cc @HyukjinKwon
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20199#discussion_r160344054
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/client/VersionsSuite.scala ---
@@ -58,7 +58,7 @@ class VersionsSuite extends SparkFunSuite
GitHub user Ngone51 opened a pull request:
https://github.com/apache/spark/pull/20199
[Spark-22967][Hive]Fix VersionSuite's unit tests by change Windows path
into URI path
## What changes were proposed in this pull request?
Two unit test will fail due to Windows format
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20199
Ok, will try.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20199#discussion_r160404821
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/client/VersionsSuite.scala ---
@@ -58,7 +58,7 @@ class VersionsSuite extends SparkFunSuite
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20056
@squito yea, clean up will be fine. Maybe, I make things more complicated.
Thank for comment.
---
-
To unsubscribe, e-mail
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19285#discussion_r162549759
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -261,37 +263,93 @@ private[spark] class MemoryStore
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19285#discussion_r162548052
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -233,17 +235,13 @@ private[spark] class MemoryStore
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19285#discussion_r162548350
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -162,26 +162,33 @@ private[spark] class MemoryStore
Github user Ngone51 closed the pull request at:
https://github.com/apache/spark/pull/20053
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user Ngone51 closed the pull request at:
https://github.com/apache/spark/pull/20056
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19285#discussion_r162835021
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -162,26 +162,29 @@ private[spark] class MemoryStore
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19285#discussion_r162834864
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -162,26 +162,29 @@ private[spark] class MemoryStore
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19285#discussion_r162834903
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -162,26 +162,29 @@ private[spark] class MemoryStore
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19285#discussion_r162837169
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -346,85 +348,24 @@ private[spark] class MemoryStore(
} else
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19285#discussion_r162841437
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -162,26 +162,29 @@ private[spark] class MemoryStore
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19285#discussion_r162848405
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -162,26 +162,29 @@ private[spark] class MemoryStore
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19285#discussion_r163807859
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -702,6 +634,93 @@ private[spark] class MemoryStore
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20056
@squito by now, the variable `droppedEventsCounter ` is unused. So, I want
to reuse it.
---
-
To unsubscribe, e-mail: reviews
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20667#discussion_r170424196
--- Diff: core/src/main/scala/org/apache/spark/storage/BlockManagerId.scala
---
@@ -132,10 +133,15 @@ private[spark] object BlockManagerId
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20056
@gatorsmile @HyukjinKwon @dongjoon-hyun Can any one of you have a look at
this pr, Thanks!
---
-
To unsubscribe, e-mail
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20056
please retest it. Thanks.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
GitHub user Ngone51 opened a pull request:
https://github.com/apache/spark/pull/20056
[SPARK-22878] [CORE] Count totalDroppedEvents for LiveListenerBus
## What changes were proposed in this pull request?
Count total dropped events from all queues' numDroppedEvents
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20056#discussion_r158751285
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/LiveListenerBus.scala ---
@@ -25,14 +25,13 @@ import scala.collection.JavaConverters._
import
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20056
It seems something wrong while testing 'ReplayListenerSuite':
```
[info] ReplayListenerSuite:
[info] - Simple replay (97 milliseconds)
[info] - Replay compressed inprogress log file
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20056
@maropu Actually, I didn't modify this unit test ever. And my unit test
locate in SparkListenerSuite haven'been started according to the "Console
O
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22001#discussion_r209658945
--- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
---
@@ -929,6 +955,28 @@ class DAGScheduler(
// HadoopRDD whose
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22001#discussion_r209662081
--- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
---
@@ -402,6 +422,19 @@ class DAGScheduler
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/22202
Since `stage 1` is only a `ShuffleMapStage`, so, why there're no other
child stages to be submitted ?
---
-
To unsubscribe, e
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22163#discussion_r212163785
--- Diff:
core/src/main/java/org/apache/spark/shuffle/sort/ShuffleExternalSorter.java ---
@@ -206,14 +211,21 @@ private void writeSortedFile(boolean
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22163#discussion_r212167438
--- Diff:
core/src/main/java/org/apache/spark/shuffle/sort/ShuffleExternalSorter.java ---
@@ -206,14 +211,21 @@ private void writeSortedFile(boolean
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22163#discussion_r212160161
--- Diff:
core/src/main/java/org/apache/spark/shuffle/sort/ShuffleExternalSorter.java ---
@@ -206,14 +211,21 @@ private void writeSortedFile(boolean
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22163#discussion_r211954019
--- Diff:
core/src/main/java/org/apache/spark/shuffle/sort/ShuffleExternalSorter.java ---
@@ -206,14 +211,21 @@ private void writeSortedFile(boolean
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/18492#discussion_r216597619
--- Diff:
core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala ---
@@ -373,8 +373,14 @@ private[spark] class ExecutorAllocationManager
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19285#discussion_r162962203
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -233,17 +235,13 @@ private[spark] class MemoryStore
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19285#discussion_r162931279
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -233,17 +235,13 @@ private[spark] class MemoryStore
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22288#discussion_r216874530
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala ---
@@ -414,9 +425,48 @@ private[spark] class TaskSchedulerImpl
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/22288
As I mentioned at
https://github.com/apache/spark/pull/22288#discussion_r216874530, I'm quite
worry about this killing behaviour. I thik we should kill a executor iff it is
idle
1 - 100 of 182 matches
Mail list logo
