git commit: SPARK-2636: Expose job ID in JobWaiter API

2014-09-02 Thread rxin
Repository: spark Updated Branches: refs/heads/master 44d3a6a75 - fbf2678c1 SPARK-2636: Expose job ID in JobWaiter API This PR adds the async actions to the Java API. User can call these async actions to get the FutureAction and use JobWaiter (for SimpleFutureAction) to retrieve job Id.

git commit: [MLlib] Squash bug in IndexedRowMatrix

2014-09-02 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.1 e13631291 - e6972ea7f [MLlib] Squash bug in IndexedRowMatrix Kill this bug fast before it does damage. Author: Reza Zadeh riz...@gmail.com Closes #2224 from rezazadeh/indexrmbug and squashes the following commits: 53386d6 [Reza

git commit: SPARK-3331 [BUILD] PEP8 tests fail because they check unzipped py4j code

2014-09-02 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master 0f16b23cd - 32ec0a8cd SPARK-3331 [BUILD] PEP8 tests fail because they check unzipped py4j code PEP8 tests run on files under ./python, but unzipped py4j code is found at ./python/build/py4j. Py4J code fails style checks and can fail

git commit: [SPARK-3061] Fix Maven build under Windows

2014-09-02 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 32ec0a8cd - 378b2315b [SPARK-3061] Fix Maven build under Windows The Maven build was failing on Windows because it tried to call the unix `unzip` utility to extract the Py4J files into core's build directory. I've fixed this issue by

git commit: [SPARK-1919] Fix Windows spark-shell --jars

2014-09-02 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 378b2315b - 8f1f9aaf4 [SPARK-1919] Fix Windows spark-shell --jars We were trying to add `file:/C:/path/to/my.jar` to the class path. We should add `C:/path/to/my.jar` instead. Tested on Windows 8.1. Author: Andrew Or andrewo...@gmail.com

git commit: [SPARK-3332] Revert spark-ec2 patch that identifies clusters using tags

2014-09-02 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-1.1 e6972ea7f - ccf35202e [SPARK-3332] Revert spark-ec2 patch that identifies clusters using tags This reverts #1899 and #2163, two patches that modified `spark-ec2` so that clusters are identified using tags instead of security groups.

git commit: [SPARK-3347] [yarn] Fix yarn-alpha compilation.

2014-09-02 Thread tgraves
Repository: spark Updated Branches: refs/heads/master 8f1f9aaf4 - 066f31a6b [SPARK-3347] [yarn] Fix yarn-alpha compilation. Missing import. Oops. Author: Marcelo Vanzin van...@cloudera.com Closes #2236 from vanzin/SPARK-3347 and squashes the following commits: 594fc39 [Marcelo Vanzin]

git commit: SPARK-3052. Misleading and spurious FileSystem closed errors whenever a ...

2014-09-02 Thread matei
Repository: spark Updated Branches: refs/heads/master 066f31a6b - 81b9d5b62 SPARK-3052. Misleading and spurious FileSystem closed errors whenever a ... ...job fails while reading from Hadoop Author: Sandy Ryza sa...@cloudera.com Closes #1956 from sryza/sandy-spark-3052 and squashes the

git commit: SPARK-3328 fixed make-distribution script --with-tachyon option.

2014-09-02 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.1 dff42a77b - 7267e402c SPARK-3328 fixed make-distribution script --with-tachyon option. Directory path for dependencies jar and resources in Tachyon 0.5.0 has been changed. Author: Prudhvi Krishna prudhvi...@gmail.com Closes #2228

git commit: [SPARK-1986][GraphX]move lib.Analytics to org.apache.spark.examples

2014-09-02 Thread ankurdave
Repository: spark Updated Branches: refs/heads/master 644e31524 - 7c92b49d6 [SPARK-1986][GraphX]move lib.Analytics to org.apache.spark.examples to support ~/spark/bin/run-example GraphXAnalytics triangles /soc-LiveJournal1.txt --numEPart=256 Author: Larry Xiao xia...@sjtu.edu.cn Closes

git commit: [SPARK-3123][GraphX]: override the setName function to set EdgeRDD's name manually just as VertexRDD does.

2014-09-02 Thread ankurdave
Repository: spark Updated Branches: refs/heads/master 7c92b49d6 - 7c9bbf172 [SPARK-3123][GraphX]: override the setName function to set EdgeRDD's name manually just as VertexRDD does. Author: uncleGen husty...@gmail.com Closes #2033 from uncleGen/master_origin and squashes the following

git commit: [SPARK-2981][GraphX] EdgePartition1D Int overflow

2014-09-02 Thread ankurdave
Repository: spark Updated Branches: refs/heads/master 7c9bbf172 - aa7de128c [SPARK-2981][GraphX] EdgePartition1D Int overflow minor fix detail is here: https://issues.apache.org/jira/browse/SPARK-2981 Author: Larry Xiao xia...@sjtu.edu.cn Closes #1902 from larryxiao/2981 and squashes the

git commit: [SPARK-2981][GraphX] EdgePartition1D Int overflow

2014-09-02 Thread ankurdave
Repository: spark Updated Branches: refs/heads/branch-1.1 7267e402c - 9b0cff2d4 [SPARK-2981][GraphX] EdgePartition1D Int overflow minor fix detail is here: https://issues.apache.org/jira/browse/SPARK-2981 Author: Larry Xiao xia...@sjtu.edu.cn Closes #1902 from larryxiao/2981 and squashes

git commit: [SPARK-2981][GraphX] EdgePartition1D Int overflow

2014-09-02 Thread ankurdave
Repository: spark Updated Branches: refs/heads/branch-1.0 5481196ab - d60f60ccc [SPARK-2981][GraphX] EdgePartition1D Int overflow minor fix detail is here: https://issues.apache.org/jira/browse/SPARK-2981 Author: Larry Xiao xia...@sjtu.edu.cn Closes #1902 from larryxiao/2981 and squashes

git commit: [SPARK-1981][Streaming][Hotfix] Fixed docs related to kinesis

2014-09-02 Thread tdas
Repository: spark Updated Branches: refs/heads/master aa7de128c - e9bb12bea [SPARK-1981][Streaming][Hotfix] Fixed docs related to kinesis - Include kinesis in the unidocs - Hide non-public classes from docs Author: Tathagata Das tathagata.das1...@gmail.com Closes #2239 from

git commit: [SPARK-1981][Streaming][Hotfix] Fixed docs related to kinesis

2014-09-02 Thread tdas
Repository: spark Updated Branches: refs/heads/branch-1.1 9b0cff2d4 - 0c8183cb3 [SPARK-1981][Streaming][Hotfix] Fixed docs related to kinesis - Include kinesis in the unidocs - Hide non-public classes from docs Author: Tathagata Das tathagata.das1...@gmail.com Closes #2239 from

git commit: [SPARK-2823][GraphX]fix GraphX EdgeRDD zipPartitions

2014-09-02 Thread ankurdave
Repository: spark Updated Branches: refs/heads/branch-1.1 0c8183cb3 - ffdb2fcf8 [SPARK-2823][GraphX]fix GraphX EdgeRDD zipPartitions If the users set “spark.default.parallelism” and the value is different with the EdgeRDD partition number, GraphX jobs will throw:

git commit: [SPARK-2823][GraphX]fix GraphX EdgeRDD zipPartitions

2014-09-02 Thread ankurdave
Repository: spark Updated Branches: refs/heads/master e9bb12bea - 9b225ac30 [SPARK-2823][GraphX]fix GraphX EdgeRDD zipPartitions If the users set “spark.default.parallelism” and the value is different with the EdgeRDD partition number, GraphX jobs will throw:

git commit: [SPARK-3341][SQL] The dataType of Sqrt expression should be DoubleType.

2014-09-02 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master 9b225ac30 - 0cd91f666 [SPARK-3341][SQL] The dataType of Sqrt expression should be DoubleType. Author: Takuya UESHIN ues...@happy-camper.st Closes #2233 from ueshin/issues/SPARK-3341 and squashes the following commits: e497320 [Takuya

git commit: [SQL] Renamed ColumnStat to ColumnMetrics to avoid confusion between ColumnStats

2014-09-02 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master 0cd91f666 - 19d3e1e8e [SQL] Renamed ColumnStat to ColumnMetrics to avoid confusion between ColumnStats Class names of these two are just too similar. Author: Cheng Lian lian.cs@gmail.com Closes #2189 from liancheng/column-metrics and

git commit: [SPARK-3300][SQL] No need to call clear() and shorten build()

2014-09-02 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master 19d3e1e8e - 24ab38401 [SPARK-3300][SQL] No need to call clear() and shorten build() The function `ensureFreeSpace` in object `ColumnBuilder` clears old buffer before copying its content to new buffer. This PR fixes it. Author: Liang-Chi

git commit: SPARK-3358: [EC2] Switch back to HVM instances for m3.X.

2014-09-02 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 24ab38401 - c64cc435e SPARK-3358: [EC2] Switch back to HVM instances for m3.X. During regression tests of Spark 1.1 we discovered perf issues with PVM instances when running PySpark. This reverts a change added in #1156 which changed the

git commit: SPARK-3358: [EC2] Switch back to HVM instances for m3.X.

2014-09-02 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.1 ffdb2fcf8 - bc4a2057e SPARK-3358: [EC2] Switch back to HVM instances for m3.X. During regression tests of Spark 1.1 we discovered perf issues with PVM instances when running PySpark. This reverts a change added in #1156 which changed

[2/2] git commit: Revert [maven-release-plugin] prepare release v1.1.0-rc3

2014-09-02 Thread pwendell
Revert [maven-release-plugin] prepare release v1.1.0-rc3 This reverts commit b2d0493b223c5f98a593bb6d7372706cc02bebad. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/a52aabd5 Tree:

[1/2] git commit: Revert [maven-release-plugin] prepare for next development iteration

2014-09-02 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.1 bc4a2057e - a52aabd58 Revert [maven-release-plugin] prepare for next development iteration This reverts commit 865e6f63f63f5e881a02d1a4e3b4c5d0e86fcd8e. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

git commit: [maven-release-plugin] prepare for next development iteration

2014-09-02 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.1 2f9b2bd78 - 37b10086b [maven-release-plugin] prepare for next development iteration Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/37b10086 Tree:

git commit: [maven-release-plugin] prepare release v1.1.0-rc4

2014-09-02 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.1 a52aabd58 - 2f9b2bd78 [maven-release-plugin] prepare release v1.1.0-rc4 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/2f9b2bd7 Tree:

Git Push Summary

2014-09-02 Thread pwendell
Repository: spark Updated Tags: refs/tags/v1.1.0-rc4 [created] 5918ea4c9 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org