Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/42#issuecomment-37763540
Jenkins, test this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/16#discussion_r10640426
--- Diff: core/src/main/scala/org/apache/spark/api/java/JavaRDDLike.scala
---
@@ -72,11 +72,12 @@ trait JavaRDDLike[T, This <: JavaRDDLike[T, This]]
exte
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/149#issuecomment-37762626
Thanks I've merged this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/157#issuecomment-37762544
Hey @baishuo I'd separately try to debug why SPARK_JAVA_OPTS isn't working.
In general we probably don't want to hard code debugging options like this i
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/149#issuecomment-37762064
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/159#issuecomment-37761904
Jenkins, test this please (?)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/148#issuecomment-37761875
@sryza when a user builds an application assembly jar, they are allowed to
bundle their own log4j.properties file in the jar. Is this not working for you
on YARN
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/149#discussion_r10637809
--- Diff: core/src/main/scala/org/apache/spark/Dependency.scala ---
@@ -43,12 +44,13 @@ abstract class NarrowDependency[T](rdd: RDD[T]) extends
Dependency
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/152#discussion_r10637523
--- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala ---
@@ -35,13 +35,21 @@ private[spark] case class
GetMapOutputStatuses(shuffleId: Int
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/152#discussion_r10637519
--- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala ---
@@ -35,13 +35,21 @@ private[spark] case class
GetMapOutputStatuses(shuffleId: Int
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/152#issuecomment-37744154
Jenkins, test this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/147#issuecomment-37744049
Hey @andrewor14 I submitted some small changes on top of this while you
were working on it over at #152.
---
If your project is set up for it, you can reply to this
GitHub user pwendell opened a pull request:
https://github.com/apache/spark/pull/152
Akka frame
This is a very small change on top of @andrewor14's patch in #147.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/pwendell/
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/148#issuecomment-37742890
@mridulm I think in YARN environments cluster operators can set a logging
file on all of the machines to be shared across applications (e.g. Spark,
MapReduce, etc). So
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/136#issuecomment-37742731
@mridulm I think the RDD definition is actually `private[spark]` and it's
just intended to be used internally for higher level algorithms.
---
If your project is s
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/145#issuecomment-37742370
https://github.com/sbt/sbt/blob/0.13/ivy/src/main/scala/sbt/Resolver.scala?source=c#L289
---
If your project is set up for it, you can reply to this email and have your
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/145#issuecomment-37742365
Thanks I've merged this. One small change I added is to use
`Resolver.mavenLocal` that sbt provides for you instead of hard coding it.
---
If your project is set u
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/147#issuecomment-37736922
LGMT pending a minor comment about unifying the code path with the Executor
thing that reads the frame size.
---
If your project is set up for it, you can reply to this
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/147#discussion_r10636474
--- Diff: core/src/main/scala/org/apache/spark/util/AkkaUtils.scala ---
@@ -121,4 +121,9 @@ private[spark] object AkkaUtils extends Logging {
def
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/147#discussion_r10636463
--- Diff: core/src/main/scala/org/apache/spark/util/AkkaUtils.scala ---
@@ -121,4 +121,9 @@ private[spark] object AkkaUtils extends Logging {
def
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/125#discussion_r10636342
--- Diff: dev/rat.bash ---
@@ -0,0 +1,49 @@
+#!/usr/bin/env bash
--- End diff --
could you remove the `.bash` extension here?
---
If your
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/125#issuecomment-37736072
@ScrapCodes this is a good start but right now it doesn't actually fail the
build if RAT doesn't succeed. Also, RAT reports a bunch of failures for python
fi
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/147#discussion_r10635964
--- Diff: core/src/test/scala/org/apache/spark/MapOutputTrackerSuite.scala
---
@@ -136,4 +142,30 @@ class MapOutputTrackerSuite extends FunSuite with
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/136#issuecomment-37733908
Even if it's private we can end up with cases where users have a e.g.
10,000 partition RDD with only a few items in each partition. Do we know a
priori when calling
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/136#issuecomment-37733845
Ah I see - so this isn't going to be externally a user-visible class (I
didn't notice it was `private[spark]`)? Would it make sense to throw an
assertion er
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/148#issuecomment-37733755
Seems reasonable to me. You still working on this or is it good to go?
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/136#issuecomment-37732233
I don't think we typically run jobs inside of getPartitions - so this
changes some semantics of calling that function. For instance a lot of the
other RDD constru
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/136#discussion_r10635447
--- Diff: core/src/main/scala/org/apache/spark/rdd/SlidedRDD.scala ---
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/136#discussion_r10635444
--- Diff: core/src/main/scala/org/apache/spark/rdd/SlidedRDD.scala ---
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/136#discussion_r10635413
--- Diff: core/src/main/scala/org/apache/spark/rdd/SlidedRDD.scala ---
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/149#discussion_r10635398
--- Diff: core/src/main/scala/org/apache/spark/Dependency.scala ---
@@ -43,12 +44,13 @@ abstract class NarrowDependency[T](rdd: RDD[T]) extends
Dependency
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/147#discussion_r10633644
--- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala ---
@@ -35,13 +35,21 @@ private[spark] case class
GetMapOutputStatuses(shuffleId: Int
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/147#discussion_r10633600
--- Diff: core/src/test/scala/org/apache/spark/MapOutputTrackerSuite.scala
---
@@ -136,4 +123,47 @@ class MapOutputTrackerSuite extends FunSuite with
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/141#issuecomment-37717834
@marmbrus mind closing this? Somehow github didn't detect the close id
correctly.
---
If your project is set up for it, you can reply to this email and have your
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/146#issuecomment-37716174
I think in the example in the PR description it should be `case class
Person(name: String, age: Int)` otherwise there is a casting error.
---
If your project is set up
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/140#discussion_r10616091
--- Diff: project/build.properties ---
@@ -14,4 +14,4 @@
# See the License for the specific language governing permissions and
# limitations under
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/42#discussion_r10598149
--- Diff: core/src/main/scala/org/apache/spark/storage/PutResult.scala ---
@@ -20,7 +20,13 @@ package org.apache.spark.storage
import java.nio.ByteBuffer
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/42#discussion_r10598046
--- Diff:
core/src/main/scala/org/apache/spark/storage/BlockManagerStatusListener.scala
---
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/42#discussion_r10597888
--- Diff: core/src/main/scala/org/apache/spark/scheduler/JobLogger.scala ---
@@ -80,187 +81,78 @@ class JobLogger(val user: String, val logDirName:
String
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/42#discussion_r10597813
--- Diff: core/src/main/scala/org/apache/spark/scheduler/JobLogger.scala ---
@@ -22,24 +22,25 @@ import java.text.SimpleDateFormat
import java.util.{Date
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/42#discussion_r10597778
--- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
---
@@ -196,13 +152,46 @@ class DAGScheduler
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/42#discussion_r10597551
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/SparkReplayerBus.scala ---
@@ -0,0 +1,98 @@
+/*
+ * Licensed to the Apache Software
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/42#discussion_r10597436
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/SparkReplayerBus.scala ---
@@ -0,0 +1,98 @@
+/*
+ * Licensed to the Apache Software
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/42#discussion_r10597261
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
---
@@ -634,6 +652,30 @@ private[spark] class Master(host: String, port: Int
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/42#discussion_r10597228
--- Diff: core/src/main/scala/org/apache/spark/ui/SparkUI.scala ---
@@ -68,19 +105,53 @@ private[spark] class SparkUI(sc: SparkContext) extends
Logging
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/42#discussion_r10597200
--- Diff: core/src/main/scala/org/apache/spark/ui/SparkUI.scala ---
@@ -68,19 +105,53 @@ private[spark] class SparkUI(sc: SparkContext) extends
Logging
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/42#discussion_r10597094
--- Diff: core/src/main/scala/org/apache/spark/ui/SparkUI.scala ---
@@ -68,19 +105,53 @@ private[spark] class SparkUI(sc: SparkContext) extends
Logging
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/42#discussion_r10597086
--- Diff: core/src/main/scala/org/apache/spark/ui/SparkUI.scala ---
@@ -68,19 +105,53 @@ private[spark] class SparkUI(sc: SparkContext) extends
Logging
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/42#discussion_r10597059
--- Diff: core/src/main/scala/org/apache/spark/ui/UIReloader.scala ---
@@ -0,0 +1,50 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/42#discussion_r10596945
--- Diff:
core/src/main/scala/org/apache/spark/ui/jobs/JobProgressListener.scala ---
@@ -106,121 +114,154 @@ private[spark] class JobProgressListener(val sc
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/42#discussion_r10596857
--- Diff: core/src/main/scala/org/apache/spark/util/FileLogger.scala ---
@@ -0,0 +1,167 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/42#discussion_r10596852
--- Diff: core/src/main/scala/org/apache/spark/util/FileLogger.scala ---
@@ -0,0 +1,167 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/42#discussion_r10596774
--- Diff: docs/configuration.md ---
@@ -490,7 +490,30 @@ Apart from these, the following properties are also
available, and may be useful
spark.logConf
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/42#discussion_r10596479
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/SparkReplayerBus.scala ---
@@ -0,0 +1,97 @@
+/*
+ * Licensed to the Apache Software
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/86#discussion_r10595816
--- Diff:
core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala ---
@@ -0,0 +1,176 @@
+/*
+ * Licensed to the Apache Software
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/138#issuecomment-37613709
I just went ahead and merged this
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/138#issuecomment-37613092
Jenkins, test this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/137#discussion_r10592099
--- Diff: sbin/stop-master.sh ---
@@ -25,3 +25,5 @@ sbin=`cd "$sbin"; pwd`
. "$sbin/spark-config.sh"
"$s
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/86#issuecomment-37601103
@sryza does this need to be bumped now with the workers/executors change?
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/126#discussion_r10582800
--- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala ---
@@ -181,15 +178,50 @@ private[spark] class MapOutputTracker(conf:
SparkConf
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/126#discussion_r10581648
--- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala ---
@@ -181,15 +178,50 @@ private[spark] class MapOutputTracker(conf:
SparkConf
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/126#discussion_r10581534
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/ShuffleMapTask.scala ---
@@ -17,28 +17,24 @@
package org.apache.spark.scheduler
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/113#issuecomment-37574904
Okay i'll merge this - thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project doe
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/120#issuecomment-37574405
Okay thanks @sryza and @tgravescs - I'll merge this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as wel
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/126#discussion_r10577891
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/ShuffleMapTask.scala ---
@@ -17,28 +17,24 @@
package org.apache.spark.scheduler
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/90#issuecomment-37563984
Do you mind closing this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/120#discussion_r10576455
--- Diff:
yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientArguments.scala
---
@@ -133,11 +148,11 @@ class ClientArguments(val args: Array
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/112#issuecomment-37503280
Merged into master and 0.9
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/113#issuecomment-37503197
@andrewor14 any comments or reservations on this one?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/120#issuecomment-37503046
Sandy - looks good to me. Are you still changing things? I noticed there
are a few comments that maybe should be updated:
```
alpha/src/main/scala/org
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/120#discussion_r10553741
--- Diff:
yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientArguments.scala
---
@@ -67,24 +67,39 @@ class ClientArguments(val args: Array[String
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/120#discussion_r10553724
--- Diff:
yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientArguments.scala
---
@@ -133,11 +148,11 @@ class ClientArguments(val args: Array
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/86#issuecomment-37502813
@mateiz maybe you could take a pass on this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/86#discussion_r10553709
--- Diff:
core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala ---
@@ -0,0 +1,176 @@
+/*
+ * Licensed to the Apache Software
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/120#discussion_r10553691
--- Diff:
yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientArguments.scala
---
@@ -133,11 +148,11 @@ class ClientArguments(val args: Array
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/126#discussion_r10553097
--- Diff: core/src/main/scala/org/apache/spark/ContextCleaner.scala ---
@@ -0,0 +1,135 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/126#discussion_r10553086
--- Diff: core/src/main/scala/org/apache/spark/ContextCleaner.scala ---
@@ -0,0 +1,135 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/126#discussion_r10553056
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/ShuffleMapTask.scala ---
@@ -17,28 +17,24 @@
package org.apache.spark.scheduler
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/126#discussion_r10553044
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/ShuffleMapTask.scala ---
@@ -17,28 +17,24 @@
package org.apache.spark.scheduler
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/126#discussion_r10553029
--- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala ---
@@ -181,15 +178,50 @@ private[spark] class MapOutputTracker(conf:
SparkConf
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/126#discussion_r10552946
--- Diff: core/src/main/scala/org/apache/spark/ContextCleaner.scala ---
@@ -0,0 +1,135 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/126#discussion_r10552754
--- Diff: core/src/main/scala/org/apache/spark/ContextCleaner.scala ---
@@ -0,0 +1,135 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/126#discussion_r10552717
--- Diff: core/src/main/scala/org/apache/spark/ContextCleaner.scala ---
@@ -0,0 +1,135 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/126#discussion_r10552693
--- Diff: core/src/main/scala/org/apache/spark/ContextCleaner.scala ---
@@ -0,0 +1,135 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/126#discussion_r10552645
--- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala ---
@@ -1025,6 +1025,14 @@ abstract class RDD[T: ClassTag](
checkpointData.flatMap
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/126#discussion_r10552599
--- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala ---
@@ -181,15 +178,50 @@ private[spark] class MapOutputTracker(conf:
SparkConf
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/42#discussion_r10551998
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -0,0 +1,93 @@
+/*
+ * Licensed to the Apache Software
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/42#discussion_r10551959
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/SparkReplayerBus.scala ---
@@ -0,0 +1,97 @@
+/*
+ * Licensed to the Apache Software
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/42#discussion_r10551852
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/SparkReplayerBus.scala ---
@@ -0,0 +1,97 @@
+/*
+ * Licensed to the Apache Software
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/42#discussion_r10551815
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -895,4 +894,42 @@ private[spark] object Utils extends Logging
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/133#issuecomment-37495174
LGTM @aarondav?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/132#issuecomment-37495012
Thanks, merged.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/132#issuecomment-37495076
I ported this fix into 0.9 as well.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/126#discussion_r10551122
--- Diff:
core/src/main/scala/org/apache/spark/util/TimeStampedWeakValueHashMap.scala ---
@@ -0,0 +1,112 @@
+/*
+ * Licensed to the Apache Software
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/126#discussion_r10551057
--- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala ---
@@ -181,15 +178,50 @@ private[spark] class MapOutputTracker(conf:
SparkConf
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/126#discussion_r10551026
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/ShuffleMapTask.scala ---
@@ -17,28 +17,24 @@
package org.apache.spark.scheduler
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/126#discussion_r10550372
--- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala ---
@@ -181,15 +186,49 @@ private[spark] class MapOutputTracker(conf:
SparkConf
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/126#discussion_r10550266
--- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala ---
@@ -181,15 +186,49 @@ private[spark] class MapOutputTracker(conf:
SparkConf
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/105#discussion_r10549050
--- Diff: core/src/main/scala/org/apache/spark/api/java/JavaRDDLike.scala
---
@@ -284,6 +284,7 @@ trait JavaRDDLike[T, This <: JavaRDDLike[T, This]]
exte
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/126#discussion_r10548591
--- Diff:
core/src/main/scala/org/apache/spark/storage/ShuffleBlockManager.scala ---
@@ -169,23 +169,32 @@ class ShuffleBlockManager(blockManager
1 - 100 of 377 matches
Mail list logo
