spark git commit: [SPARKR][MINOR] Update R DESCRIPTION file

2016-08-22 Thread meng
Repository: spark Updated Branches: refs/heads/branch-2.0 eaea1c86b -> d16f9a0b7 [SPARKR][MINOR] Update R DESCRIPTION file ## What changes were proposed in this pull request? Update DESCRIPTION ## How was this patch tested? Run install and CRAN tests Author: Felix Cheung

spark git commit: [SPARK-17182][SQL] Mark Collect as non-deterministic

2016-08-22 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.0 225898961 -> eaea1c86b [SPARK-17182][SQL] Mark Collect as non-deterministic ## What changes were proposed in this pull request? This PR marks the abstract class `Collect` as non-deterministic since the results of `CollectList` and

spark git commit: [SPARK-17182][SQL] Mark Collect as non-deterministic

2016-08-22 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 920806ab2 -> 2cdd92a7c [SPARK-17182][SQL] Mark Collect as non-deterministic ## What changes were proposed in this pull request? This PR marks the abstract class `Collect` as non-deterministic since the results of `CollectList` and

spark git commit: [SPARK-16577][SPARKR] Add CRAN documentation checks to run-tests.sh

2016-08-22 Thread shivaram
Repository: spark Updated Branches: refs/heads/branch-2.0 ff2f87380 -> 225898961 [SPARK-16577][SPARKR] Add CRAN documentation checks to run-tests.sh ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was this patch tested? This change

spark git commit: [SPARK-16577][SPARKR] Add CRAN documentation checks to run-tests.sh

2016-08-22 Thread shivaram
Repository: spark Updated Branches: refs/heads/master 37f0ab70d -> 920806ab2 [SPARK-16577][SPARKR] Add CRAN documentation checks to run-tests.sh ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was this patch tested? This change adds

spark git commit: [SPARK-17090][FOLLOW-UP][ML] Add expert param support to SharedParamsCodeGen

2016-08-22 Thread yliang
Repository: spark Updated Branches: refs/heads/master 6d93f9e02 -> 37f0ab70d [SPARK-17090][FOLLOW-UP][ML] Add expert param support to SharedParamsCodeGen ## What changes were proposed in this pull request? Add expert param support to SharedParamsCodeGen where aggregationDepth a expert param

spark git commit: [SPARK-17144][SQL] Removal of useless CreateHiveTableAsSelectLogicalPlan

2016-08-22 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 8e223ea67 -> 6d93f9e02 [SPARK-17144][SQL] Removal of useless CreateHiveTableAsSelectLogicalPlan ## What changes were proposed in this pull request? `CreateHiveTableAsSelectLogicalPlan` is a dead code after refactoring. ## How was this

spark git commit: [SPARK-16550][SPARK-17042][CORE] Certain classes fail to deserialize in block manager replication

2016-08-22 Thread rxin
Repository: spark Updated Branches: refs/heads/master 71afeeea4 -> 8e223ea67 [SPARK-16550][SPARK-17042][CORE] Certain classes fail to deserialize in block manager replication ## What changes were proposed in this pull request? This is a straightforward clone of JoshRosen 's original patch.

spark git commit: [SPARK-16508][SPARKR] doc updates and more CRAN check fixes

2016-08-22 Thread shivaram
Repository: spark Updated Branches: refs/heads/branch-2.0 01a4d69f3 -> b65b041af [SPARK-16508][SPARKR] doc updates and more CRAN check fixes replace ``` ` ``` in code doc with `\code{thing}` remove added `...` for drop(DataFrame) fix remaining CRAN check warnings create doc with knitr

spark git commit: [SPARK-16508][SPARKR] doc updates and more CRAN check fixes

2016-08-22 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master 84770b59f -> 71afeeea4 [SPARK-16508][SPARKR] doc updates and more CRAN check fixes ## What changes were proposed in this pull request? replace ``` ` ``` in code doc with `\code{thing}` remove added `...` for drop(DataFrame) fix remaining

spark git commit: [SPARK-17162] Range does not support SQL generation

2016-08-22 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.0 6dcc1a3f0 -> 01a4d69f3 [SPARK-17162] Range does not support SQL generation ## What changes were proposed in this pull request? The range operator previously didn't support SQL generation, which made it not possible to use in views.

spark git commit: [SPARK-17162] Range does not support SQL generation

2016-08-22 Thread rxin
Repository: spark Updated Branches: refs/heads/master 929cb8bee -> 84770b59f [SPARK-17162] Range does not support SQL generation ## What changes were proposed in this pull request? The range operator previously didn't support SQL generation, which made it not possible to use in views. ##

spark git commit: [MINOR][SQL] Fix some typos in comments and test hints

2016-08-22 Thread yhuai
Repository: spark Updated Branches: refs/heads/master 6f3cd36f9 -> 929cb8bee [MINOR][SQL] Fix some typos in comments and test hints ## What changes were proposed in this pull request? Fix some typos in comments and test hints ## How was this patch tested? N/A. Author: Sean Zhong

spark git commit: [SPARKR][MINOR] Add Xiangrui and Felix to maintainers

2016-08-22 Thread shivaram
Repository: spark Updated Branches: refs/heads/branch-2.0 94eff0875 -> 6dcc1a3f0 [SPARKR][MINOR] Add Xiangrui and Felix to maintainers ## What changes were proposed in this pull request? This change adds Xiangrui Meng and Felix Cheung to the maintainers field in the package description. ##

spark git commit: [SPARKR][MINOR] Add Xiangrui and Felix to maintainers

2016-08-22 Thread shivaram
Repository: spark Updated Branches: refs/heads/master 0583ecda1 -> 6f3cd36f9 [SPARKR][MINOR] Add Xiangrui and Felix to maintainers ## What changes were proposed in this pull request? This change adds Xiangrui Meng and Felix Cheung to the maintainers field in the package description. ## How

spark git commit: [SPARK-17173][SPARKR] R MLlib refactor, cleanup, reformat, fix deprecation in test

2016-08-22 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master 342278c09 -> 0583ecda1 [SPARK-17173][SPARKR] R MLlib refactor, cleanup, reformat, fix deprecation in test ## What changes were proposed in this pull request? refactor, cleanup, reformat, fix deprecation in test ## How was this patch

spark git commit: [SPARK-16320][DOC] Document G1 heap region's effect on spark 2.0 vs 1.6

2016-08-22 Thread yhuai
Repository: spark Updated Branches: refs/heads/branch-2.0 79195982a -> 94eff0875 [SPARK-16320][DOC] Document G1 heap region's effect on spark 2.0 vs 1.6 ## What changes were proposed in this pull request? Collect GC discussion in one section, and documenting findings about G1 GC heap region

spark git commit: [SPARK-16320][DOC] Document G1 heap region's effect on spark 2.0 vs 1.6

2016-08-22 Thread yhuai
Repository: spark Updated Branches: refs/heads/master 209e1b3c0 -> 342278c09 [SPARK-16320][DOC] Document G1 heap region's effect on spark 2.0 vs 1.6 ## What changes were proposed in this pull request? Collect GC discussion in one section, and documenting findings about G1 GC heap region

spark git commit: [SPARKR][MINOR] Fix Cache Folder Path in Windows

2016-08-22 Thread shivaram
Repository: spark Updated Branches: refs/heads/branch-2.0 2add45fab -> 79195982a [SPARKR][MINOR] Fix Cache Folder Path in Windows ## What changes were proposed in this pull request? This PR tries to fix the scheme of local cache folder in Windows. The name of the environment variable should

spark git commit: [SPARKR][MINOR] Fix Cache Folder Path in Windows

2016-08-22 Thread shivaram
Repository: spark Updated Branches: refs/heads/master b264cbb16 -> 209e1b3c0 [SPARKR][MINOR] Fix Cache Folder Path in Windows ## What changes were proposed in this pull request? This PR tries to fix the scheme of local cache folder in Windows. The name of the environment variable should be

spark git commit: [SPARK-15113][PYSPARK][ML] Add missing num features num classes

2016-08-22 Thread mlnick
Repository: spark Updated Branches: refs/heads/master bd9655063 -> b264cbb16 [SPARK-15113][PYSPARK][ML] Add missing num features num classes ## What changes were proposed in this pull request? Add missing `numFeatures` and `numClasses` to the wrapped Java models in PySpark ML pipelines.

spark git commit: [SPARK-17085][STREAMING][DOCUMENTATION AND ACTUAL CODE DIFFERS - UNSUPPORTED OPERATIONS]

2016-08-22 Thread srowen
Repository: spark Updated Branches: refs/heads/master 8d35a6f68 -> bd9655063 [SPARK-17085][STREAMING][DOCUMENTATION AND ACTUAL CODE DIFFERS - UNSUPPORTED OPERATIONS] Changes in Spark Stuctured Streaming doc in this link

spark git commit: [SPARK-17085][STREAMING][DOCUMENTATION AND ACTUAL CODE DIFFERS - UNSUPPORTED OPERATIONS]

2016-08-22 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 49cc44de3 -> 2add45fab [SPARK-17085][STREAMING][DOCUMENTATION AND ACTUAL CODE DIFFERS - UNSUPPORTED OPERATIONS] Changes in Spark Stuctured Streaming doc in this link

spark git commit: [SPARK-17115][SQL] decrease the threshold when split expressions

2016-08-22 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.0 e62b29f29 -> 49cc44de3 [SPARK-17115][SQL] decrease the threshold when split expressions ## What changes were proposed in this pull request? In 2.0, we change the threshold of splitting expressions from 16K to 64K, which cause very

spark git commit: [SPARK-17115][SQL] decrease the threshold when split expressions

2016-08-22 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 4b6c2cbcb -> 8d35a6f68 [SPARK-17115][SQL] decrease the threshold when split expressions ## What changes were proposed in this pull request? In 2.0, we change the threshold of splitting expressions from 16K to 64K, which cause very bad

spark git commit: [SPARK-16968] Document additional options in jdbc Writer

2016-08-22 Thread srowen
Repository: spark Updated Branches: refs/heads/master 083de00cb -> 4b6c2cbcb [SPARK-16968] Document additional options in jdbc Writer ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) This is the document for previous JDBC Writer options. ##

spark git commit: [SPARK-17127] Make unaligned access in unsafe available for AArch64

2016-08-22 Thread srowen
Repository: spark Updated Branches: refs/heads/master b2074b664 -> 083de00cb [SPARK-17127] Make unaligned access in unsafe available for AArch64 ## # What changes were proposed in this pull request? >From the spark of version 2.0.0 , when MemoryMode.OFF_HEAP is set , whether >the