spark git commit: SPARK-6878 [CORE] Fix for sum on empty RDD fails with exception

2015-04-14 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.3 8e5caa227 - 2954468b0 SPARK-6878 [CORE] Fix for sum on empty RDD fails with exception Author: Erik van Oosten evanoos...@ebay.com Closes #5489 from erikvanoosten/master and squashes the following commits: 1c91954 [Erik van Oosten]

spark git commit: SPARK-6878 [CORE] Fix for sum on empty RDD fails with exception

2015-04-14 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.2 daec1c635 - 899ffdcc0 SPARK-6878 [CORE] Fix for sum on empty RDD fails with exception Author: Erik van Oosten evanoos...@ebay.com Closes #5489 from erikvanoosten/master and squashes the following commits: 1c91954 [Erik van Oosten]

spark git commit: [SPARK-6731] Bump version of apache commons-math3

2015-04-14 Thread srowen
Repository: spark Updated Branches: refs/heads/master 77eeb10fd - 628a72f70 [SPARK-6731] Bump version of apache commons-math3 Version 3.1.1 is two years old and the newer version includes approximate percentile statistics (among other things). Author: Punyashloka Biswal

spark git commit: SPARK-6878 [CORE] Fix for sum on empty RDD fails with exception

2015-04-14 Thread srowen
Repository: spark Updated Branches: refs/heads/master 628a72f70 - 51b306b93 SPARK-6878 [CORE] Fix for sum on empty RDD fails with exception Author: Erik van Oosten evanoos...@ebay.com Closes #5489 from erikvanoosten/master and squashes the following commits: 1c91954 [Erik van Oosten]

spark git commit: [SPARK-6796][Streaming][WebUI] Add Active Batches and Completed Batches lists to StreamingPage

2015-04-14 Thread tdas
Repository: spark Updated Branches: refs/heads/master a76b921a9 - 6de282e2d [SPARK-6796][Streaming][WebUI] Add Active Batches and Completed Batches lists to StreamingPage This PR adds two lists, `Active Batches` and `Completed Batches`. Here is the screenshot:

spark git commit: [SPARK-5634] [core] Show correct message in HS when no incomplete apps f...

2015-04-14 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 971738936 - 30a6e0dcc [SPARK-5634] [core] Show correct message in HS when no incomplete apps f... ...ound. Author: Marcelo Vanzin van...@cloudera.com Closes #5515 from vanzin/SPARK-5634 and squashes the following commits: f74ecf1

spark git commit: [SPARK-6890] [core] Fix launcher lib work with SPARK_PREPEND_CLASSES.

2015-04-14 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 6de282e2d - 971738936 [SPARK-6890] [core] Fix launcher lib work with SPARK_PREPEND_CLASSES. The fix for SPARK-6406 broke the case where sub-processes are launched when SPARK_PREPEND_CLASSES is set, because the code now would only add the

spark git commit: [SPARK-5634] [core] Show correct message in HS when no incomplete apps f...

2015-04-14 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.2 3c13936aa - 5845a6236 [SPARK-5634] [core] Show correct message in HS when no incomplete apps f... ...ound. Author: Marcelo Vanzin van...@cloudera.com Closes #5515 from vanzin/SPARK-5634 and squashes the following commits: f74ecf1

spark git commit: [SPARK-6894]spark.executor.extraLibraryOptions = spark.executor.extraLibraryPath

2015-04-14 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 320bca450 - f63b44a5c [SPARK-6894]spark.executor.extraLibraryOptions = spark.executor.extraLibraryPath https://issues.apache.org/jira/browse/SPARK-6894 cc vanzin Author: WangTaoTheTonic wangtao...@huawei.com Closes #5506 from

spark git commit: [CORE] SPARK-6880: Fixed null check when all the dependent stages are cancelled due to previous stage failure

2015-04-14 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master f63b44a5c - dcf8a9f33 [CORE] SPARK-6880: Fixed null check when all the dependent stages are cancelled due to previous stage failure Fixed null check when all the dependent stages are cancelled due to previous stage failure. This happens

spark git commit: [SPARK-6081] Support fetching http/https uris in driver runner.

2015-04-14 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 51b306b93 - 320bca450 [SPARK-6081] Support fetching http/https uris in driver runner. Currently if passed uris such as http/https, it won't able to fetch them as it only calls HadoopFs get. This fix utilizes the existing util method to

spark git commit: [SPARK-2033] Automatically cleanup checkpoint

2015-04-14 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master dcf8a9f33 - 25998e4d7 [SPARK-2033] Automatically cleanup checkpoint Author: GuoQiang Li wi...@qq.com Closes #855 from witgo/cleanup_checkpoint_date and squashes the following commits: 1649850 [GuoQiang Li] review commit c0087e0

spark git commit: SPARK-1706: Allow multiple executors per worker in Standalone mode

2015-04-14 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 25998e4d7 - 8f8dc45f6 SPARK-1706: Allow multiple executors per worker in Standalone mode resubmit of https://github.com/apache/spark/pull/636 for a totally different algorithm https://issues.apache.org/jira/browse/SPARK-1706 In current

spark git commit: [SPARK-6905] Upgrade to snappy-java 1.1.1.7

2015-04-14 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master b075e4b72 - 6adb8bcbf [SPARK-6905] Upgrade to snappy-java 1.1.1.7 We should upgrade our snappy-java dependency to 1.1.1.7 in order to include a fix for a bug that results in worse compression in SnappyOutputStream (see

spark git commit: [SPARK-6905] Upgrade to snappy-java 1.1.1.7

2015-04-14 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-1.3 2954468b0 - db2154d7d [SPARK-6905] Upgrade to snappy-java 1.1.1.7 We should upgrade our snappy-java dependency to 1.1.1.7 in order to include a fix for a bug that results in worse compression in SnappyOutputStream (see

spark git commit: [SPARK-6905] Upgrade to snappy-java 1.1.1.7

2015-04-14 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-1.2 899ffdcc0 - 3c13936aa [SPARK-6905] Upgrade to snappy-java 1.1.1.7 We should upgrade our snappy-java dependency to 1.1.1.7 in order to include a fix for a bug that results in worse compression in SnappyOutputStream (see

spark git commit: [SPARK-6700] [yarn] Re-enable flaky test.

2015-04-14 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 8f8dc45f6 - b075e4b72 [SPARK-6700] [yarn] Re-enable flaky test. Test runs have been successful on jenkins. So let's re-enable the test and look out for any failures, and fix things appropriately. Author: Marcelo Vanzin

spark git commit: [SPARK-5808] [build] Package pyspark files in sbt assembly.

2015-04-14 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 6adb8bcbf - 65774370a [SPARK-5808] [build] Package pyspark files in sbt assembly. This turned out to be more complicated than I wanted because the layout of python/ doesn't really follow the usual maven conventions. So some extra code is

spark git commit: [SPARK-6766][Streaming] Fix issue about StreamingListenerBatchSubmitted and StreamingListenerBatchStarted (backport to branch 1.3)

2015-04-14 Thread tdas
Repository: spark Updated Branches: refs/heads/branch-1.3 db2154d7d - 1ab423f6e [SPARK-6766][Streaming] Fix issue about StreamingListenerBatchSubmitted and StreamingListenerBatchStarted (backport to branch 1.3) Backport SPARK-6766 #5414 to branch 1.3 Conflicts:

spark git commit: Revert [SPARK-6352] [SQL] Add DirectParquetOutputCommitter

2015-04-14 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master 4d4b24927 - a76b921a9 Revert [SPARK-6352] [SQL] Add DirectParquetOutputCommitter This reverts commit b29663eeea440b1d1a288d41b5ddf67e77c5bd54. I'm reverting this because it broke test compilation for the Hadoop 1.x profiles. Project:

spark git commit: [SPARK-6769][YARN][TEST] Usage of the ListenerBus in YarnClusterSuite is wrong

2015-04-14 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 65774370a - 4d4b24927 [SPARK-6769][YARN][TEST] Usage of the ListenerBus in YarnClusterSuite is wrong In YarnClusterSuite, a test case uses `SaveExecutorInfo` to handle ExecutorAddedEvent as follows. ``` private class SaveExecutorInfo