spark git commit: [SPARK-4307] Initialize FileDescriptor lazily in FileRegion.

2014-11-11 Thread adav
Repository: spark Updated Branches: refs/heads/master 65083e93d - ef29a9a9a [SPARK-4307] Initialize FileDescriptor lazily in FileRegion. Netty's DefaultFileRegion requires a FileDescriptor in its constructor, which means we need to have a opened file handle. In super large workloads, this

spark git commit: [SPARK-4307] Initialize FileDescriptor lazily in FileRegion.

2014-11-11 Thread adav
Repository: spark Updated Branches: refs/heads/branch-1.2 df8242c9b - e9d009dc3 [SPARK-4307] Initialize FileDescriptor lazily in FileRegion. Netty's DefaultFileRegion requires a FileDescriptor in its constructor, which means we need to have a opened file handle. In super large workloads,

spark git commit: [SPARK-4295][External]Fix exception in SparkSinkSuite

2014-11-11 Thread tdas
Repository: spark Updated Branches: refs/heads/master ef29a9a9a - f8811a569 [SPARK-4295][External]Fix exception in SparkSinkSuite Handle exception in SparkSinkSuite, please refer to [SPARK-4295] Author: maji2014 ma...@asiainfo.com Closes #3177 from maji2014/spark-4295 and squashes the

spark git commit: [SPARK-2492][Streaming] kafkaReceiver minor changes to align with Kafka 0.8

2014-11-11 Thread tdas
Repository: spark Updated Branches: refs/heads/branch-1.2 fe8a1cd29 - 7710b7156 [SPARK-2492][Streaming] kafkaReceiver minor changes to align with Kafka 0.8 Update the KafkaReceiver's behavior when auto.offset.reset is set. In Kafka 0.8, `auto.offset.reset` is a hint for out-range offset to

spark git commit: [Streaming][Minor]Replace some 'if-else' in Clock

2014-11-11 Thread tdas
Repository: spark Updated Branches: refs/heads/master c8850a3d6 - 6e03de304 [Streaming][Minor]Replace some 'if-else' in Clock Replace some 'if-else' statement by math.min and math.max in Clock.scala Author: huangzhaowei carlmartin...@gmail.com Closes #3088 from SaintBacchus/StreamingClock

spark git commit: SPARK-1830 Deploy failover, Make Persistence engine and LeaderAgent Pluggable

2014-11-11 Thread adav
Repository: spark Updated Branches: refs/heads/master 6e03de304 - deefd9d73 SPARK-1830 Deploy failover, Make Persistence engine and LeaderAgent Pluggable Author: Prashant Sharma prashan...@imaginea.com Closes #771 from ScrapCodes/deploy-failover-pluggable and squashes the following commits:

spark git commit: SPARK-4305 [BUILD] yarn-alpha profile won't build due to network/yarn module

2014-11-11 Thread tgraves
Repository: spark Updated Branches: refs/heads/master deefd9d73 - f820b563d SPARK-4305 [BUILD] yarn-alpha profile won't build due to network/yarn module SPARK-3797 introduced the `network/yarn` module, but its YARN code depends on YARN APIs not present in older versions covered by the

spark git commit: SPARK-4305 [BUILD] yarn-alpha profile won't build due to network/yarn module

2014-11-11 Thread tgraves
Repository: spark Updated Branches: refs/heads/branch-1.2 cc1f3a0d6 - 8f7e80f30 SPARK-4305 [BUILD] yarn-alpha profile won't build due to network/yarn module SPARK-3797 introduced the `network/yarn` module, but its YARN code depends on YARN APIs not present in older versions covered by the

spark git commit: [SPARK-4282][YARN] Stopping flag in YarnClientSchedulerBackend should be volatile

2014-11-11 Thread tgraves
Repository: spark Updated Branches: refs/heads/master f820b563d - 7f3718842 [SPARK-4282][YARN] Stopping flag in YarnClientSchedulerBackend should be volatile In YarnClientSchedulerBackend, a variable stopping is used as a flag and it's accessed by some threads so it should be volatile.

spark git commit: SPARK-2269 Refactor mesos scheduler resourceOffers and add unit test

2014-11-11 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 7f3718842 - a878660d2 SPARK-2269 Refactor mesos scheduler resourceOffers and add unit test Author: Timothy Chen tnac...@gmail.com Closes #1487 from tnachen/resource_offer_refactor and squashes the following commits: 4ea5dec [Timothy

spark git commit: [Release] Log build output for each distribution

2014-11-11 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.2 6a7ddf4ce - 307b69d73 [Release] Log build output for each distribution Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/307b69d7 Tree:

[2/8] spark git commit: Support cross building for Scala 2.11

2014-11-11 Thread pwendell
http://git-wip-us.apache.org/repos/asf/spark/blob/12f56334/repl/src/main/scala/org/apache/spark/repl/SparkIMain.scala -- diff --git a/repl/src/main/scala/org/apache/spark/repl/SparkIMain.scala

[8/8] spark git commit: Support cross building for Scala 2.11

2014-11-11 Thread pwendell
Support cross building for Scala 2.11 Let's give this another go using a version of Hive that shades its JLine dependency. Author: Prashant Sharma prashan...@imaginea.com Author: Patrick Wendell pwend...@gmail.com Closes #3159 from pwendell/scala-2.11-prashant and squashes the following

[3/8] spark git commit: Support cross building for Scala 2.11

2014-11-11 Thread pwendell
http://git-wip-us.apache.org/repos/asf/spark/blob/daaca14c/repl/scala-2.11/src/test/scala/org/apache/spark/repl/ReplSuite.scala -- diff --git a/repl/scala-2.11/src/test/scala/org/apache/spark/repl/ReplSuite.scala

[4/8] spark git commit: Support cross building for Scala 2.11

2014-11-11 Thread pwendell
http://git-wip-us.apache.org/repos/asf/spark/blob/12f56334/repl/scala-2.11/src/main/scala/org/apache/spark/repl/SparkIMain.scala -- diff --git a/repl/scala-2.11/src/main/scala/org/apache/spark/repl/SparkIMain.scala

[3/8] spark git commit: Support cross building for Scala 2.11

2014-11-11 Thread pwendell
http://git-wip-us.apache.org/repos/asf/spark/blob/12f56334/repl/scala-2.11/src/test/scala/org/apache/spark/repl/ReplSuite.scala -- diff --git a/repl/scala-2.11/src/test/scala/org/apache/spark/repl/ReplSuite.scala

[2/8] spark git commit: Support cross building for Scala 2.11

2014-11-11 Thread pwendell
http://git-wip-us.apache.org/repos/asf/spark/blob/daaca14c/repl/src/main/scala/org/apache/spark/repl/SparkIMain.scala -- diff --git a/repl/src/main/scala/org/apache/spark/repl/SparkIMain.scala

[4/8] spark git commit: Support cross building for Scala 2.11

2014-11-11 Thread pwendell
http://git-wip-us.apache.org/repos/asf/spark/blob/daaca14c/repl/scala-2.11/src/main/scala/org/apache/spark/repl/SparkIMain.scala -- diff --git a/repl/scala-2.11/src/main/scala/org/apache/spark/repl/SparkIMain.scala

[6/8] spark git commit: Support cross building for Scala 2.11

2014-11-11 Thread pwendell
http://git-wip-us.apache.org/repos/asf/spark/blob/daaca14c/repl/scala-2.10/src/main/scala/org/apache/spark/repl/SparkIMain.scala -- diff --git a/repl/scala-2.10/src/main/scala/org/apache/spark/repl/SparkIMain.scala

[7/8] spark git commit: Support cross building for Scala 2.11

2014-11-11 Thread pwendell
http://git-wip-us.apache.org/repos/asf/spark/blob/12f56334/pom.xml -- diff --git a/pom.xml b/pom.xml index 4e0cd6c..7bbde31 100644 --- a/pom.xml +++ b/pom.xml @@ -97,30 +97,26 @@ modulesql/catalyst/module

[1/8] spark git commit: Support cross building for Scala 2.11

2014-11-11 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 2ddb1415e - daaca14c1 http://git-wip-us.apache.org/repos/asf/spark/blob/daaca14c/repl/src/main/scala/org/apache/spark/repl/SparkMemberHandlers.scala -- diff --git

spark git commit: [MLLIB] SPARK-4347: Reducing GradientBoostingSuite run time.

2014-11-11 Thread meng
Repository: spark Updated Branches: refs/heads/master daaca14c1 - 2ef016b13 [MLLIB] SPARK-4347: Reducing GradientBoostingSuite run time. Before: [info] GradientBoostingSuite: [info] - Regression with continuous features: SquaredError (22 seconds, 115 milliseconds) [info] - Regression with

spark git commit: [MLLIB] SPARK-4347: Reducing GradientBoostingSuite run time.

2014-11-11 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.2 12f56334b - c9bb5e459 [MLLIB] SPARK-4347: Reducing GradientBoostingSuite run time. Before: [info] GradientBoostingSuite: [info] - Regression with continuous features: SquaredError (22 seconds, 115 milliseconds) [info] - Regression

spark git commit: [SPARK-3936] Add aggregateMessages, which supersedes mapReduceTriplets

2014-11-11 Thread rxin
Repository: spark Updated Branches: refs/heads/master 2ef016b13 - faeb41de2 [SPARK-3936] Add aggregateMessages, which supersedes mapReduceTriplets aggregateMessages enables neighborhood computation similarly to mapReduceTriplets, but it introduces two API improvements: 1. Messages are sent

spark git commit: [SPARK-3936] Add aggregateMessages, which supersedes mapReduceTriplets

2014-11-11 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.2 c9bb5e459 - 4a4cc7e91 [SPARK-3936] Add aggregateMessages, which supersedes mapReduceTriplets aggregateMessages enables neighborhood computation similarly to mapReduceTriplets, but it introduces two API improvements: 1. Messages are