spark git commit: [SQL] remove unnecessary import

2014-12-04 Thread rxin
Repository: spark Updated Branches: refs/heads/master 3cdae038f - ed88db4cb [SQL] remove unnecessary import Author: Jacky Li jacky.li...@huawei.com Closes #3585 from jackylk/remove and squashes the following commits: 045423d [Jacky Li] remove unnecessary import Project:

spark git commit: [SPARK-4719][API] Consolidate various narrow dep RDD classes with MapPartitionsRDD

2014-12-04 Thread rxin
Repository: spark Updated Branches: refs/heads/master ed88db4cb - c3ad48603 [SPARK-4719][API] Consolidate various narrow dep RDD classes with MapPartitionsRDD MappedRDD, MappedValuesRDD, FlatMappedValuesRDD, FilteredRDD, GlommedRDD, FlatMappedRDD are not necessary. They can be implemented

spark git commit: [SPARK-4685] Include all spark.ml and spark.mllib packages in JavaDoc's MLlib group

2014-12-04 Thread meng
Repository: spark Updated Branches: refs/heads/master c3ad48603 - 20bfea4ab [SPARK-4685] Include all spark.ml and spark.mllib packages in JavaDoc's MLlib group This is #3554 from Lewuathe except that I put both `spark.ml` and `spark.mllib` in the group 'MLlib`. Closes #3554 jkbradley

spark git commit: [SPARK-4685] Include all spark.ml and spark.mllib packages in JavaDoc's MLlib group

2014-12-04 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.2 f9e1f89b2 - 2605acb04 [SPARK-4685] Include all spark.ml and spark.mllib packages in JavaDoc's MLlib group This is #3554 from Lewuathe except that I put both `spark.ml` and `spark.mllib` in the group 'MLlib`. Closes #3554 jkbradley

spark git commit: [SQL] Minor: Avoid calling Seq#size in a loop

2014-12-04 Thread rxin
Repository: spark Updated Branches: refs/heads/master 20bfea4ab - c6c7165e7 [SQL] Minor: Avoid calling Seq#size in a loop Just found this instance while doing some jstack-based profiling of a Spark SQL job. It is very unlikely that this is causing much of a perf issue anywhere, but it is

spark git commit: [SQL] Minor: Avoid calling Seq#size in a loop

2014-12-04 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.2 2605acb04 - dec838bcb [SQL] Minor: Avoid calling Seq#size in a loop Just found this instance while doing some jstack-based profiling of a Spark SQL job. It is very unlikely that this is causing much of a perf issue anywhere, but it

spark git commit: [docs] Fix outdated comment in tuning guide

2014-12-04 Thread rxin
Repository: spark Updated Branches: refs/heads/master c6c7165e7 - 529439bd5 [docs] Fix outdated comment in tuning guide When you use the SPARK_JAVA_OPTS env variable, Spark complains: ``` SPARK_JAVA_OPTS was detected (set to ' -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps '). This

spark git commit: [docs] Fix outdated comment in tuning guide

2014-12-04 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.2 dec838bcb - bf720ef98 [docs] Fix outdated comment in tuning guide When you use the SPARK_JAVA_OPTS env variable, Spark complains: ``` SPARK_JAVA_OPTS was detected (set to ' -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps ').

spark git commit: [SPARK-4575] [mllib] [docs] spark.ml pipelines doc + bug fixes

2014-12-04 Thread meng
Repository: spark Updated Branches: refs/heads/master 529439bd5 - 469a6e5f3 [SPARK-4575] [mllib] [docs] spark.ml pipelines doc + bug fixes Documentation: * Added ml-guide.md, linked from mllib-guide.md * Updated mllib-guide.md with small section pointing to ml-guide.md Examples: *

spark git commit: [SPARK-4575] [mllib] [docs] spark.ml pipelines doc + bug fixes

2014-12-04 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.2 bf720ef98 - 266a81492 [SPARK-4575] [mllib] [docs] spark.ml pipelines doc + bug fixes Documentation: * Added ml-guide.md, linked from mllib-guide.md * Updated mllib-guide.md with small section pointing to ml-guide.md Examples: *

spark git commit: [FIX][DOC] Fix broken links in ml-guide.md

2014-12-04 Thread meng
Repository: spark Updated Branches: refs/heads/master 469a6e5f3 - 7e758d709 [FIX][DOC] Fix broken links in ml-guide.md and some minor changes in ScalaDoc. Author: Xiangrui Meng m...@databricks.com Closes #3601 from mengxr/SPARK-4575-fix and squashes the following commits: c559768 [Xiangrui

spark git commit: [SPARK-4683][SQL] Add a beeline.cmd to run on Windows

2014-12-04 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 7e758d709 - 28c7acace [SPARK-4683][SQL] Add a beeline.cmd to run on Windows Tested locally with a Win7 VM. Connected to a Spark SQL Thrift server instance running on Mac OS X with the following command line: ``` bin\beeline.cmd -u

spark git commit: [SPARK-4683][SQL] Add a beeline.cmd to run on Windows

2014-12-04 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.2 34fdca0a5 - 2fbe488a0 [SPARK-4683][SQL] Add a beeline.cmd to run on Windows Tested locally with a Win7 VM. Connected to a Spark SQL Thrift server instance running on Mac OS X with the following command line: ``` bin\beeline.cmd -u

[3/3] spark git commit: Revert Preparing Spark release v1.2.0-rc1

2014-12-04 Thread pwendell
Revert Preparing Spark release v1.2.0-rc1 This reverts commit 1056e9ec13203d0c51564265e94d77a054498fdb. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/078894c7 Tree:

[1/3] spark git commit: Revert HOTFIX: Rolling back incorrect version change

2014-12-04 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.2 2fbe488a0 - 078894c7f Revert HOTFIX: Rolling back incorrect version change This reverts commit 3a4609eada2ee0bfbcce0f4127b6a5363ae528e5. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

spark git commit: [SPARK-4253] Ignore spark.driver.host in yarn-cluster and standalone-cluster modes

2014-12-04 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master 28c7acace - 8106b1e36 [SPARK-4253] Ignore spark.driver.host in yarn-cluster and standalone-cluster modes In yarn-cluster and standalone-cluster modes, we don't know where driver will run until it is launched. If the `spark.driver.host`

spark git commit: [SPARK-4253] Ignore spark.driver.host in yarn-cluster and standalone-cluster modes

2014-12-04 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-1.2 078894c7f - d9aee07fe [SPARK-4253] Ignore spark.driver.host in yarn-cluster and standalone-cluster modes In yarn-cluster and standalone-cluster modes, we don't know where driver will run until it is launched. If the

spark git commit: [SPARK-4253] Ignore spark.driver.host in yarn-cluster and standalone-cluster modes

2014-12-04 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-1.1 6c532254a - 5ac55c841 [SPARK-4253] Ignore spark.driver.host in yarn-cluster and standalone-cluster modes In yarn-cluster and standalone-cluster modes, we don't know where driver will run until it is launched. If the

spark git commit: [HOTFIX] Fixing two issues with the release script.

2014-12-04 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 8106b1e36 - 8dae26f83 [HOTFIX] Fixing two issues with the release script. 1. The version replacement was still producing some false changes. 2. Uploads to the staging repo specifically. Author: Patrick Wendell pwend...@gmail.com Closes

spark git commit: [HOTFIX] Fixing two issues with the release script.

2014-12-04 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.2 d9aee07fe - ead01b6d5 [HOTFIX] Fixing two issues with the release script. 1. The version replacement was still producing some false changes. 2. Uploads to the staging repo specifically. Author: Patrick Wendell pwend...@gmail.com

[1/2] spark git commit: Preparing development version 1.2.1-SNAPSHOT

2014-12-04 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.2 ead01b6d5 - bc05df8a2 Preparing development version 1.2.1-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/bc05df8a Tree:

[2/2] spark git commit: Preparing Spark release v1.2.0-rc2

2014-12-04 Thread pwendell
Preparing Spark release v1.2.0-rc2 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/2b72c569 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/2b72c569 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/2b72c569

Git Push Summary

2014-12-04 Thread pwendell
Repository: spark Updated Tags: refs/tags/v1.2.0-rc2 [created] 2b72c569a - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

spark git commit: [SPARK-4745] Fix get_existing_cluster() function with multiple security groups

2014-12-04 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-1.2 bc05df8a2 - a00d0aa6e [SPARK-4745] Fix get_existing_cluster() function with multiple security groups The current get_existing_cluster() function would only find an instance belonged to a cluster if the instance's security groups ==

spark git commit: [SPARK-4745] Fix get_existing_cluster() function with multiple security groups

2014-12-04 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master 8dae26f83 - 794f3aec2 [SPARK-4745] Fix get_existing_cluster() function with multiple security groups The current get_existing_cluster() function would only find an instance belonged to a cluster if the instance's security groups ==

spark git commit: [SPARK-4745] Fix get_existing_cluster() function with multiple security groups

2014-12-04 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-1.1 5ac55c841 - d01fdd3bb [SPARK-4745] Fix get_existing_cluster() function with multiple security groups The current get_existing_cluster() function would only find an instance belonged to a cluster if the instance's security groups ==

spark git commit: [SPARK-4459] Change groupBy type parameter from K to U

2014-12-04 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-1.2 a00d0aa6e - 0d159de39 [SPARK-4459] Change groupBy type parameter from K to U Please see https://issues.apache.org/jira/browse/SPARK-4459 Author: Saldanha salda...@phusca-l24858.wlan.na.novartis.net Closes #3327 from alokito/master

spark git commit: [SPARK-4459] Change groupBy type parameter from K to U

2014-12-04 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-1.1 d01fdd3bb - e98aa5476 [SPARK-4459] Change groupBy type parameter from K to U Please see https://issues.apache.org/jira/browse/SPARK-4459 Author: Saldanha salda...@phusca-l24858.wlan.na.novartis.net Closes #3327 from alokito/master

spark git commit: [SPARK-4459] Change groupBy type parameter from K to U

2014-12-04 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master 794f3aec2 - 743a889d2 [SPARK-4459] Change groupBy type parameter from K to U Please see https://issues.apache.org/jira/browse/SPARK-4459 Author: Saldanha salda...@phusca-l24858.wlan.na.novartis.net Closes #3327 from alokito/master and

spark git commit: [SPARK-4652][DOCS] Add docs about spark-git-repo option

2014-12-04 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-1.2 0d159de39 - f5c5647b8 [SPARK-4652][DOCS] Add docs about spark-git-repo option There might be some cases when WIPS spark version need to be run on EC2 cluster. In order to setup this type of cluster more easily, add --spark-git-repo

spark git commit: [SPARK-4652][DOCS] Add docs about spark-git-repo option

2014-12-04 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master 743a889d2 - ab8177da2 [SPARK-4652][DOCS] Add docs about spark-git-repo option There might be some cases when WIPS spark version need to be run on EC2 cluster. In order to setup this type of cluster more easily, add --spark-git-repo option

spark git commit: [SPARK-4397] Move object RDD to the front of RDD.scala.

2014-12-04 Thread rxin
Repository: spark Updated Branches: refs/heads/master ab8177da2 - ed92b47e8 [SPARK-4397] Move object RDD to the front of RDD.scala. I ran into multiple cases that SBT/Scala compiler was confused by the implicits in continuous compilation mode. Adding explicit return types fixes the problem.

spark git commit: [SPARK-4421] Wrong link in spark-standalone.html

2014-12-04 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-1.1 bf637e041 - b09382a46 [SPARK-4421] Wrong link in spark-standalone.html Modified the link of building Spark. (backport version of #3279.) Author: Masayoshi TSUZUKI tsudu...@oss.nttdata.co.jp Closes #3280 from

spark git commit: [SPARK-4421] Wrong link in spark-standalone.html

2014-12-04 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master ed92b47e8 - ddfc09c36 [SPARK-4421] Wrong link in spark-standalone.html Modified the link of building Spark. Author: Masayoshi TSUZUKI tsudu...@oss.nttdata.co.jp Closes #3279 from tsudukim/feature/SPARK-4421 and squashes the following

spark git commit: [SPARK-4421] Wrong link in spark-standalone.html

2014-12-04 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-1.2 f5c5647b8 - b905e114e [SPARK-4421] Wrong link in spark-standalone.html Modified the link of building Spark. Author: Masayoshi TSUZUKI tsudu...@oss.nttdata.co.jp Closes #3279 from tsudukim/feature/SPARK-4421 and squashes the following

spark git commit: Fix typo in Spark SQL docs.

2014-12-04 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master ddfc09c36 - 15cf3b012 Fix typo in Spark SQL docs. Author: Andy Konwinski andykonwin...@gmail.com Closes #3611 from andyk/patch-3 and squashes the following commits: 7bab333 [Andy Konwinski] Fix typo in Spark SQL docs. Project:

spark git commit: Fix typo in Spark SQL docs.

2014-12-04 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-1.2 b905e114e - 63b1bc14a Fix typo in Spark SQL docs. Author: Andy Konwinski andykonwin...@gmail.com Closes #3611 from andyk/patch-3 and squashes the following commits: 7bab333 [Andy Konwinski] Fix typo in Spark SQL docs. (cherry picked

spark git commit: Fix typo in Spark SQL docs.

2014-12-04 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-1.1 b09382a46 - 8ee2d1873 Fix typo in Spark SQL docs. Author: Andy Konwinski andykonwin...@gmail.com Closes #3611 from andyk/patch-3 and squashes the following commits: 7bab333 [Andy Konwinski] Fix typo in Spark SQL docs. (cherry picked

spark git commit: [SPARK-4464] Description about configuration options need to be modified in docs.

2014-12-04 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master 15cf3b012 - ca379039f [SPARK-4464] Description about configuration options need to be modified in docs. Added description about -h and -host. Modified description about -i and -ip which are now deprecated. Added description about

spark git commit: [SPARK-4464] Description about configuration options need to be modified in docs.

2014-12-04 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-1.2 63b1bc14a - 6c4363178 [SPARK-4464] Description about configuration options need to be modified in docs. Added description about -h and -host. Modified description about -i and -ip which are now deprecated. Added description about

[1/2] spark git commit: Revert [HOT FIX] [YARN] Check whether `/lib` exists before listing its files

2014-12-04 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master ca379039f - fd8525334 Revert [HOT FIX] [YARN] Check whether `/lib` exists before listing its files This reverts commit 90ec643e9af4c8bbb9000edca08c07afb17939c7. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

[2/2] spark git commit: Revert SPARK-2624 add datanucleus jars to the container in yarn-cluster

2014-12-04 Thread andrewor14
Revert SPARK-2624 add datanucleus jars to the container in yarn-cluster This reverts commit a975dc32799bb8a14f9e1c76defaaa7cfbaf8b53. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/fd852533 Tree:

[1/2] spark git commit: Revert [HOT FIX] [YARN] Check whether `/lib` exists before listing its files

2014-12-04 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.2 6c4363178 - a8d8077dc Revert [HOT FIX] [YARN] Check whether `/lib` exists before listing its files This reverts commit 38cb2c3a36a5c9ead4494cbc3dde008c2f0698ce. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

spark git commit: [SPARK-4753][SQL] Use catalyst for partition pruning in newParquet.

2014-12-04 Thread pwendell
Repository: spark Updated Branches: refs/heads/master fd8525334 - f5801e813 [SPARK-4753][SQL] Use catalyst for partition pruning in newParquet. Author: Michael Armbrust mich...@databricks.com Closes #3613 from marmbrus/parquetPartitionPruning and squashes the following commits: 4f138f8

spark git commit: [SPARK-4753][SQL] Use catalyst for partition pruning in newParquet.

2014-12-04 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.2 a8d8077dc - d12ea49f5 [SPARK-4753][SQL] Use catalyst for partition pruning in newParquet. Author: Michael Armbrust mich...@databricks.com Closes #3613 from marmbrus/parquetPartitionPruning and squashes the following commits: 4f138f8

Git Push Summary

2014-12-04 Thread pwendell
Repository: spark Updated Tags: refs/tags/v1.2.0-rc2 [deleted] 2b72c569a - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org