GitHub user squito opened a pull request:
https://github.com/apache/spark/pull/8314
XORShiftRandom.hashSeed is random in high bits
https://issues.apache.org/jira/browse/SPARK-10116
This is really trivial, just happened to notice it -- if
`XORShiftRandom.hashSeed` is really
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/5400#issuecomment-132645948
There was an old thread about how `LargeByteBufferHelperSuite.testMapFiles`
and using random number generator instead of repeating the same bytes over and
over again
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/7943#issuecomment-132702056
ok ready to give up on jenkins -- the yarn tests have passed regularly.
@tgravescs can you take another look?
---
If your project is set up for it, you can
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/7943#issuecomment-132333169
Jenkins, retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
GitHub user squito opened a pull request:
https://github.com/apache/spark/pull/8320
[SPARK-6236] [core] [wip] caching for blocks over 2GB
This allows caching of partitions that are greater than 2GB, and also
includes error messages for the other cases where we hit the 2GB limit
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/8320#issuecomment-132767770
@tgravescs You can try this out for handling caching for over 2GB. I've
tried it out some but haven't really stressed it, interested in your feedback.
---
If your
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/8314#issuecomment-132779542
hmm, some of the errors are just checks against the expected sequence from
the rng, which I can update (though some of these tests probably shouldn't
require a perfect
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/7699#issuecomment-133540222
@kayousterhout @markhamstra @pwendell @mateiz another one still waiting.
Mostly got buy-in from Mark on https://github.com/apache/spark/pull/4055, but
I'd still
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/8180#discussion_r37672673
--- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
---
@@ -47,15 +47,48 @@ import
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/8180#discussion_r37673715
--- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
---
@@ -812,7 +971,7 @@ class DAGScheduler(
case s: ResultStage
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/8180#discussion_r37678288
--- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
---
@@ -746,6 +848,63 @@ class DAGScheduler(
submitWaitingStages
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/8180#discussion_r37678367
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -1973,6 +1973,24 @@ class SparkContext(config: SparkConf) extends
Logging
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/8180#discussion_r37673081
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/ShuffleMapStage.scala ---
@@ -37,6 +45,9 @@ private[spark] class ShuffleMapStage
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/5636#issuecomment-133538881
@kayousterhout @markhamstra @pwendell @mateiz would really appreciate it if
you would take a look at this one. Its way over the 1-week maintainer waiting
period, and I
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/8180#discussion_r37677383
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -990,6 +999,110 @@ class DAGSchedulerSuite
assert
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/8180#discussion_r37678962
--- Diff: core/src/main/scala/org/apache/spark/MapOutputStatistics.scala ---
@@ -0,0 +1,23 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/8180#discussion_r37682553
--- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala ---
@@ -132,13 +133,46 @@ private[spark] abstract class MapOutputTracker(conf:
SparkConf
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/8180#issuecomment-133572807
I'm *super* nervous about adding this, given our current inability to deal
w/ the complexity of the scheduler.
---
If your project is set up for it, you can reply
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/8180#issuecomment-134294669
Iâve only recently looked at making changes to the scheduler, but it
seems to me there is widespread agreement among committers that it is very
error prone
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/5636#discussion_r37780181
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -473,6 +473,280 @@ class DAGSchedulerSuite
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/5636#discussion_r37780211
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -473,6 +473,280 @@ class DAGSchedulerSuite
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/8180#discussion_r37789233
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -990,6 +999,110 @@ class DAGSchedulerSuite
assert
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/8269#issuecomment-132240341
lgtm
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/7943#issuecomment-132253120
jenkins, retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/8178#discussion_r37154006
--- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
---
@@ -302,12 +302,12 @@ class DAGScheduler(
shuffleDep
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/8178#issuecomment-131649053
Jenkins, retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/8178#issuecomment-131649041
thanks for updating josh. still lgtm pending tests from me. (left one
minor comment, your discretion to update).
---
If your project is set up for it, you can reply
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/8090#issuecomment-131660810
@andrewor14 running in `local` mode is misleading for stage failures -- you
really need to have multiple block managers in place to understand the
behavior, eg
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/8218#discussion_r37155642
--- Diff:
network/shuffle/src/test/java/org/apache/spark/network/sasl/SaslIntegrationSuite.java
---
@@ -160,6 +164,111 @@ public void testNoSaslServer
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/8218#discussion_r37155626
--- Diff:
network/shuffle/src/test/java/org/apache/spark/network/sasl/SaslIntegrationSuite.java
---
@@ -160,6 +164,111 @@ public void testNoSaslServer
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/8218#discussion_r37155796
--- Diff:
network/common/src/main/java/org/apache/spark/network/client/TransportClient.java
---
@@ -85,6 +86,24 @@ public SocketAddress getSocketAddress
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/8218#issuecomment-131663356
I'm not an expert on this part of the code, but it looks sane. I just left
a few minor comments
---
If your project is set up for it, you can reply to this email
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/7943#discussion_r37237185
--- Diff:
yarn/src/test/scala/org/apache/spark/network/yarn/YarnShuffleServiceSuite.scala
---
@@ -0,0 +1,233 @@
+/*
+ * Licensed to the Apache
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/7943#issuecomment-131964770
jenkins, retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/8218#discussion_r37239534
--- Diff:
network/shuffle/src/test/java/org/apache/spark/network/sasl/SaslIntegrationSuite.java
---
@@ -39,44 +46,41 @@
import
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/8218#discussion_r37239500
--- Diff:
network/shuffle/src/test/java/org/apache/spark/network/sasl/SaslIntegrationSuite.java
---
@@ -19,17 +19,24 @@
import java.io.IOException
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6917#discussion_r33507904
--- Diff: dev/run-tests.py ---
@@ -639,32 +651,47 @@ def detect_binary_inop_with_mima():
run_cmd([os.path.join(SPARK_HOME, dev, mima
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/6983#issuecomment-116862765
Hi @ckadner. I'm not the best person to review this, but just randomly saw
your comment. If you merge with master, and then push to your same branch, the
diff here
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/5563#issuecomment-116861950
I have no opinion on what the right behavior should be, I'll leave that up
to folks that actually use mesos, but my thoughts are:
1) should we be adding support
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/6648#issuecomment-117228730
Jenkins, retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/6750#issuecomment-117220600
ping @kayousterhout @markhamstra @mateiz
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/7130#discussion_r33594302
--- Diff:
core/src/main/scala/org/apache/spark/util/collection/CollectionSpillable.scala
---
@@ -17,14 +17,15 @@
package
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/7130#discussion_r33594663
--- Diff:
core/src/test/scala/org/apache/spark/shuffle/ShuffleMemoryManagerSuite.scala ---
@@ -307,4 +321,20 @@ class ShuffleMemoryManagerSuite extends
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/7130#discussion_r33594631
--- Diff:
core/src/main/scala/org/apache/spark/util/collection/ExternalSorter.scala ---
@@ -603,8 +539,165 @@ private[spark] class ExternalSorter[K, V, C
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/7028#issuecomment-117343194
hey, sorry I haven't taken another look at this yet -- do you mind giving
me a bit more time to look before merging? I should be able to look this
evening, if not first
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/7004#discussion_r35784695
--- Diff:
core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala ---
@@ -23,6 +23,8 @@ import javax.annotation.Nullable
import
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/7750#issuecomment-126006883
any suggestions on a test that doesn't require 2 GB of memory?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
GitHub user squito opened a pull request:
https://github.com/apache/spark/pull/7750
[SPARK-9437][core] avoid overflow in SizeEstimator
https://issues.apache.org/jira/browse/SPARK-9437
You can merge this pull request into a Git repository by running:
$ git pull https
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/7004#issuecomment-126021604
Thanks @JDrit ! I will fix those final details as I merge.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/7004#discussion_r35784052
--- Diff:
core/src/main/scala/org/apache/spark/serializer/GenericAvroSerializer.scala ---
@@ -0,0 +1,140 @@
+/*
+ * Licensed to the Apache Software
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/7004#discussion_r35784682
--- Diff:
core/src/main/scala/org/apache/spark/serializer/GenericAvroSerializer.scala ---
@@ -0,0 +1,148 @@
+/*
+ * Licensed to the Apache Software
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/5636#discussion_r35719989
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -473,6 +473,322 @@ class DAGSchedulerSuite
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/5636#discussion_r35720078
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -473,6 +473,322 @@ class DAGSchedulerSuite
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/7739#issuecomment-125793292
Jenkins, ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/5636#discussion_r35720546
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -473,6 +473,322 @@ class DAGSchedulerSuite
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/5636#discussion_r35720582
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -473,6 +473,322 @@ class DAGSchedulerSuite
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/5636#discussion_r35719924
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -473,6 +473,322 @@ class DAGSchedulerSuite
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/5636#discussion_r35719826
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -473,6 +473,322 @@ class DAGSchedulerSuite
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/5636#discussion_r35720755
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -473,6 +473,322 @@ class DAGSchedulerSuite
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/5636#discussion_r35720839
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -473,6 +473,322 @@ class DAGSchedulerSuite
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/5636#discussion_r35720879
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -473,6 +473,322 @@ class DAGSchedulerSuite
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/7004#discussion_r35796975
--- Diff:
core/src/main/scala/org/apache/spark/serializer/GenericAvroSerializer.scala ---
@@ -0,0 +1,140 @@
+/*
+ * Licensed to the Apache Software
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/5636#discussion_r35800677
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -473,6 +473,326 @@ class DAGSchedulerSuite
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/5636#discussion_r35802042
--- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
---
@@ -1067,6 +1066,10 @@ class DAGScheduler
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/7750#issuecomment-126095693
`s1` `s2` are already longs, which means so is `size` -- so no change
needed there.
(not that I had checked this before you mentioned it ...)
---
If your project
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/5636#discussion_r35815039
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -473,6 +473,324 @@ class DAGSchedulerSuite
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/7750#issuecomment-126087430
@shivaram good idea. I updated one more spot where it seems there could be
overflow -- I will admit I don't completely understand what is going on there,
but the change
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/7004#issuecomment-126040715
sorry I am trying to understand this serialization thing a bit better ...
something doesn't make sense to me, but mostly outside of these changes. Once
I get a handle
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/5636#discussion_r35801109
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -473,6 +473,326 @@ class DAGSchedulerSuite
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/5636#discussion_r35800587
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -473,6 +473,326 @@ class DAGSchedulerSuite
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/5636#discussion_r35801802
--- Diff: core/src/main/scala/org/apache/spark/scheduler/Stage.scala ---
@@ -76,6 +76,34 @@ private[spark] abstract class Stage(
*/
private var
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/7004#issuecomment-126084533
@pwendell @JoshRosen ah sorry, I thought @vanzin @zsxwing gave it the
thumbs up earlier -- just following the comments I forgot to get the approval
of sean or you
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/5636#discussion_r35800358
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -473,6 +473,326 @@ class DAGSchedulerSuite
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/5636#discussion_r35800447
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -473,6 +473,326 @@ class DAGSchedulerSuite
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/5636#discussion_r35800885
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -473,6 +473,326 @@ class DAGSchedulerSuite
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/5636#discussion_r35800864
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -473,6 +473,326 @@ class DAGSchedulerSuite
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/5636#discussion_r35800844
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -473,6 +473,326 @@ class DAGSchedulerSuite
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/7004#issuecomment-126060583
merged to master, thanks @JDrit
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/5636#discussion_r35802160
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -473,6 +473,326 @@ class DAGSchedulerSuite
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/5636#issuecomment-126071592
@kayousterhout Do you want to take another look at this? I have a handful
of small comments, but I think this is basically ready
---
If your project is set up
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/7753#issuecomment-126283119
@jerryshao I'm not entirely sure I know what you mean by:
| A simple question, is it enough to only expose the maximum memory usage
of Netty layer?
can
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/7014#issuecomment-126283659
@aarondav are you OK with this now? I think tom addressed all your concerns
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/7753#issuecomment-126282507
Since we'll eventually want to add more metrics, can you put all the netty
metrics into another case class inside `ExecutorMetrics`?
Also, I'm wondering if we
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/7750#issuecomment-126140616
Jenkins, retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/7839#issuecomment-126948995
I think this is ready for review -- there are reasonable comments tests
in place.
Also I fixed the issue w/ a potential leak of applications which terminate
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/7808#issuecomment-126948708
I like the latest look! no strong opinion on whether the color difference
is still necessary or not.
---
If your project is set up for it, you can reply to this email
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/7770#discussion_r36135556
--- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
---
@@ -773,16 +773,26 @@ class DAGScheduler(
stage.pendingTasks.clear
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/7888#discussion_r36148992
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
---
@@ -413,25 +413,38 @@ class
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/7839#discussion_r36137193
--- Diff:
network/yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleService.java
---
@@ -100,11 +119,33 @@ private boolean isAuthenticationEnabled
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/7770#issuecomment-127410184
Hey Patrick,
that sounds good. I agree that there are probably a couple different
concerns here. My biggest concern was some overall direction we head
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/7770#issuecomment-127411771
Other concerns aside ... can somebody explain the logic for resetting the
accumulator values to me?
IIUC you reset the values:
a) on the initial
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/7770#discussion_r36189276
--- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
---
@@ -773,16 +773,26 @@ class DAGScheduler(
stage.pendingTasks.clear
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/7927#discussion_r36202856
--- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala
---
@@ -592,8 +592,14 @@ private[spark] class BlockManager(
val locations
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/7927#discussion_r36204820
--- Diff:
core/src/test/scala/org/apache/spark/storage/BlockManagerSuite.scala ---
@@ -443,6 +443,21 @@ class BlockManagerSuite extends SparkFunSuite
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/7014#issuecomment-126501269
@tomwhite there is a legit failure here, looks like you need to merge w/
master and fix a compile error
---
If your project is set up for it, you can reply
GitHub user squito opened a pull request:
https://github.com/apache/spark/pull/7839
[SPARK-9439] [yarn] [wip] External shuffle service should be robust to NM
restarts
https://issues.apache.org/jira/browse/SPARK-9439
In general, Yarn apps should be robust to NodeManager
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/7753#issuecomment-126786492
@jerryshao I don't think that's true. The executors metrics that get sent
back to the driver have both the max value, and the current value. So a custom
listener could
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/7808#issuecomment-126788335
This is really cool!
I have a couple of suggestions. I'm not a designer / UI expert at all, so
feel free to reject these (or to simply push them off
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/7770#discussion_r36019144
--- Diff: core/src/main/scala/org/apache/spark/ui/ToolTips.scala ---
@@ -62,6 +62,13 @@ private[spark] object ToolTips {
Time that the executor spent
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/7770#issuecomment-126818492
I admit I need more time to understand the code changes, but I'm excited
about this addition, it'll be a great feature to help users understand what is
going
701 - 800 of 3519 matches
Mail list logo
