spark git commit: [SPARK-11608][MLLIB][DOC] Added migration guide for MLlib 1.6

Repository: spark Updated Branches: refs/heads/master 6a880afa8 -> 8148cc7a5 [SPARK-11608][MLLIB][DOC] Added migration guide for MLlib 1.6 No known breaking changes, but some deprecations and changes of behavior. CC: mengxr Author: Joseph K. Bradley Closes #10235

spark git commit: [SPARK-12310][SPARKR] Add write.json and write.parquet for SparkR

Repository: spark Updated Branches: refs/heads/branch-1.6 a2d584ed9 -> ac0e2ea7c [SPARK-12310][SPARKR] Add write.json and write.parquet for SparkR Add ```write.json``` and ```write.parquet``` for SparkR, and deprecated ```saveAsParquetFile```. Author: Yanbo Liang

spark git commit: [SPARK-12310][SPARKR] Add write.json and write.parquet for SparkR

Repository: spark Updated Branches: refs/heads/master 2eb5af5f0 -> 22f6cd86f [SPARK-12310][SPARKR] Add write.json and write.parquet for SparkR Add ```write.json``` and ```write.parquet``` for SparkR, and deprecated ```saveAsParquetFile```. Author: Yanbo Liang Closes

[spark] Git Push Summary

Repository: spark Updated Tags: refs/tags/v1.6.0-rc3 [created] 168c89e07 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

[1/2] spark git commit: Preparing Spark release v1.6.0-rc3

Repository: spark Updated Branches: refs/heads/branch-1.6 e1adf6d7d -> aee88eb55 Preparing Spark release v1.6.0-rc3 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/168c89e0 Tree:

spark git commit: [SPARK-12364][ML][SPARKR] Add ML example for SparkR

Repository: spark Updated Branches: refs/heads/branch-1.6 dffa6100d -> 04e868b63 [SPARK-12364][ML][SPARKR] Add ML example for SparkR We have DataFrame example for SparkR, we also need to add ML example under ```examples/src/main/r```. cc mengxr jkbradley shivaram Author: Yanbo Liang

spark git commit: [SPARK-11608][MLLIB][DOC] Added migration guide for MLlib 1.6

Repository: spark Updated Branches: refs/heads/branch-1.6 aee88eb55 -> dffa6100d [SPARK-11608][MLLIB][DOC] Added migration guide for MLlib 1.6 No known breaking changes, but some deprecations and changes of behavior. CC: mengxr Author: Joseph K. Bradley Closes

[spark] Git Push Summary

Repository: spark Updated Tags: refs/tags/v1.6.0-rc3 [deleted] 00a39d9c0 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

spark git commit: [SPARK-12361][PYSPARK][TESTS] Should set PYSPARK_DRIVER_PYTHON before Python tests

Repository: spark Updated Branches: refs/heads/master d252b2d54 -> 6a880afa8 [SPARK-12361][PYSPARK][TESTS] Should set PYSPARK_DRIVER_PYTHON before Python tests Although this patch still doesn't solve the issue why the return code is 0 (see JIRA description), it resolves the issue of python

spark git commit: [SPARK-12320][SQL] throw exception if the number of fields does not line up for Tuple encoder

Repository: spark Updated Branches: refs/heads/master 1a8b2a17d -> a783a8ed4 [SPARK-12320][SQL] throw exception if the number of fields does not line up for Tuple encoder Author: Wenchen Fan Closes #10293 from cloud-fan/err-msg. Project:

spark git commit: [SPARK-12215][ML][DOC] User guide section for KMeans in spark.ml

Repository: spark Updated Branches: refs/heads/master 22f6cd86f -> 26d70bd2b [SPARK-12215][ML][DOC] User guide section for KMeans in spark.ml cc jkbradley Author: Yu ISHIKAWA Closes #10244 from yu-iskw/SPARK-12215. Project:

spark git commit: [SPARK-12215][ML][DOC] User guide section for KMeans in spark.ml

Repository: spark Updated Branches: refs/heads/branch-1.6 ac0e2ea7c -> 16edd933d [SPARK-12215][ML][DOC] User guide section for KMeans in spark.ml cc jkbradley Author: Yu ISHIKAWA Closes #10244 from yu-iskw/SPARK-12215. (cherry picked from commit

spark git commit: [SPARK-9694][ML] Add random seed Param to Scala CrossValidator

Repository: spark Updated Branches: refs/heads/master 7b6dc29d0 -> 860dc7f2f [SPARK-9694][ML] Add random seed Param to Scala CrossValidator Add random seed Param to Scala CrossValidator Author: Yanbo Liang Closes #9108 from yanboliang/spark-9694. Project:

spark git commit: [SPARK-12324][MLLIB][DOC] Fixes the sidebar in the ML documentation

Repository: spark Updated Branches: refs/heads/branch-1.6 fb08f7b78 -> a2d584ed9 [SPARK-12324][MLLIB][DOC] Fixes the sidebar in the ML documentation This fixes the sidebar, using a pure CSS mechanism to hide it when the browser's viewport is too narrow. Credit goes to the original author

spark git commit: [SPARK-12324][MLLIB][DOC] Fixes the sidebar in the ML documentation

Repository: spark Updated Branches: refs/heads/master 1a3d0cd9f -> a6325fc40 [SPARK-12324][MLLIB][DOC] Fixes the sidebar in the ML documentation This fixes the sidebar, using a pure CSS mechanism to hide it when the browser's viewport is too narrow. Credit goes to the original author Titan-C

spark git commit: [SPARK-12318][SPARKR] Save mode in SparkR should be error by default

Repository: spark Updated Branches: refs/heads/master 54c512ba9 -> 2eb5af5f0 [SPARK-12318][SPARKR] Save mode in SparkR should be error by default shivaram Please help review. Author: Jeff Zhang Closes #10290 from zjffdu/SPARK-12318. Project:

spark git commit: [SPARK-12318][SPARKR] Save mode in SparkR should be error by default

Repository: spark Updated Branches: refs/heads/branch-1.6 16edd933d -> f81512729 [SPARK-12318][SPARKR] Save mode in SparkR should be error by default shivaram Please help review. Author: Jeff Zhang Closes #10290 from zjffdu/SPARK-12318. (cherry picked from commit

spark git commit: [SPARK-12345][MESOS] Filter SPARK_HOME when submitting Spark jobs with Mesos cluster mode.

Repository: spark Updated Branches: refs/heads/master 26d70bd2b -> ad8c1f0b8 [SPARK-12345][MESOS] Filter SPARK_HOME when submitting Spark jobs with Mesos cluster mode. SPARK_HOME is now causing problem with Mesos cluster mode since spark-submit script has been changed recently to take

spark git commit: [SPARK-6518][MLLIB][EXAMPLE][DOC] Add example code and user guide for bisecting k-means

Repository: spark Updated Branches: refs/heads/branch-1.6 e5b85713d -> e1adf6d7d [SPARK-6518][MLLIB][EXAMPLE][DOC] Add example code and user guide for bisecting k-means This PR includes only an example code in order to finish it quickly. I'll send another PR for the docs soon. Author: Yu

spark git commit: [SPARK-6518][MLLIB][EXAMPLE][DOC] Add example code and user guide for bisecting k-means

Repository: spark Updated Branches: refs/heads/master ad8c1f0b8 -> 7b6dc29d0 [SPARK-6518][MLLIB][EXAMPLE][DOC] Add example code and user guide for bisecting k-means This PR includes only an example code in order to finish it quickly. I'll send another PR for the docs soon. Author: Yu

spark git commit: [SPARK-12345][MESOS] Filter SPARK_HOME when submitting Spark jobs with Mesos cluster mode.

Repository: spark Updated Branches: refs/heads/branch-1.6 f81512729 -> e5b85713d [SPARK-12345][MESOS] Filter SPARK_HOME when submitting Spark jobs with Mesos cluster mode. SPARK_HOME is now causing problem with Mesos cluster mode since spark-submit script has been changed recently to take

spark git commit: [SPARK-12309][ML] Use sqlContext from MLlibTestSparkContext for spark.ml test suites

Repository: spark Updated Branches: refs/heads/master 860dc7f2f -> d252b2d54 [SPARK-12309][ML] Use sqlContext from MLlibTestSparkContext for spark.ml test suites Use ```sqlContext``` from ```MLlibTestSparkContext``` rather than creating new one for spark.ml test suites. I have checked

spark git commit: [SPARK-8745] [SQL] remove GenerateProjection

Repository: spark Updated Branches: refs/heads/master a6325fc40 -> 54c512ba9 [SPARK-8745] [SQL] remove GenerateProjection cc rxin Author: Davies Liu Closes #10316 from davies/remove_generate_projection. Project: http://git-wip-us.apache.org/repos/asf/spark/repo

spark git commit: [SPARK-12380] [PYSPARK] use SQLContext.getOrCreate in mllib

Repository: spark Updated Branches: refs/heads/branch-1.6 04e868b63 -> 552b38f87 [SPARK-12380] [PYSPARK] use SQLContext.getOrCreate in mllib MLlib should use SQLContext.getOrCreate() instead of creating new SQLContext. Author: Davies Liu Closes #10338 from

spark git commit: [SPARK-12380] [PYSPARK] use SQLContext.getOrCreate in mllib

Repository: spark Updated Branches: refs/heads/master 3a44aebd0 -> 27b98e99d [SPARK-12380] [PYSPARK] use SQLContext.getOrCreate in mllib MLlib should use SQLContext.getOrCreate() instead of creating new SQLContext. Author: Davies Liu Closes #10338 from

spark git commit: [SPARK-9690][ML][PYTHON] pyspark CrossValidator random seed

Repository: spark Updated Branches: refs/heads/master 9657ee878 -> 3a44aebd0 [SPARK-9690][ML][PYTHON] pyspark CrossValidator random seed Extend CrossValidator with HasSeed in PySpark. This PR replaces [https://github.com/apache/spark/pull/7997] CC: yanboliang thunterdb mmenestret Would one

spark git commit: [SPARK-11677][SQL] ORC filter tests all pass if filters are actually not pushed down.

Repository: spark Updated Branches: refs/heads/master edf65cd96 -> 9657ee878 [SPARK-11677][SQL] ORC filter tests all pass if filters are actually not pushed down. Currently ORC filters are not tested properly. All the tests pass even if the filters are not pushed down or disabled. In this

spark git commit: [SPARK-12164][SQL] Decode the encoded values and then display

Repository: spark Updated Branches: refs/heads/master a783a8ed4 -> edf65cd96 [SPARK-12164][SQL] Decode the encoded values and then display Based on the suggestions from marmbrus cloud-fan in https://github.com/apache/spark/pull/10165 , this PR is to print the decoded values(user objects) in

spark git commit: [MINOR] Add missing interpolation in NettyRPCEnv

Repository: spark Updated Branches: refs/heads/branch-1.6 552b38f87 -> 638b89bc3 [MINOR] Add missing interpolation in NettyRPCEnv ``` Exception in thread "main" org.apache.spark.rpc.RpcTimeoutException: Cannot receive any reply in ${timeout.duration}. This timeout is controlled by

spark git commit: [MINOR] Add missing interpolation in NettyRPCEnv

Repository: spark Updated Branches: refs/heads/master 27b98e99d -> 861549acd [MINOR] Add missing interpolation in NettyRPCEnv ``` Exception in thread "main" org.apache.spark.rpc.RpcTimeoutException: Cannot receive any reply in ${timeout.duration}. This timeout is controlled by

spark git commit: [SPARK-12365][CORE] Use ShutdownHookManager where Runtime.getRuntime.addShutdownHook() is called

Repository: spark Updated Branches: refs/heads/master 38d9795a4 -> f590178d7 [SPARK-12365][CORE] Use ShutdownHookManager where Runtime.getRuntime.addShutdownHook() is called SPARK-9886 fixed ExternalBlockStore.scala This PR fixes the remaining references to

spark git commit: [SPARK-12365][CORE] Use ShutdownHookManager where Runtime.getRuntime.addShutdownHook() is called

Repository: spark Updated Branches: refs/heads/branch-1.6 fb02e4e3b -> 4af64385b [SPARK-12365][CORE] Use ShutdownHookManager where Runtime.getRuntime.addShutdownHook() is called SPARK-9886 fixed ExternalBlockStore.scala This PR fixes the remaining references to

spark git commit: [SPARK-12186][WEB UI] Send the complete request URI including the query string when redirecting.

Repository: spark Updated Branches: refs/heads/branch-1.6 4af64385b -> 154567dca [SPARK-12186][WEB UI] Send the complete request URI including the query string when redirecting. Author: Rohit Agarwal Closes #10180 from mindprince/SPARK-12186. (cherry picked from commit

spark git commit: [SPARK-12186][WEB UI] Send the complete request URI including the query string when redirecting.

Repository: spark Updated Branches: refs/heads/master f590178d7 -> fdb382275 [SPARK-12186][WEB UI] Send the complete request URI including the query string when redirecting. Author: Rohit Agarwal Closes #10180 from mindprince/SPARK-12186. Project:

spark git commit: [SPARK-12390] Clean up unused serializer parameter in BlockManager

Repository: spark Updated Branches: refs/heads/master d1508dd9b -> 97678edea [SPARK-12390] Clean up unused serializer parameter in BlockManager No change in functionality is intended. This only changes internal API. Author: Andrew Or Closes #10343 from

spark git commit: [SPARK-12057][SQL] Prevent failure on corrupt JSON records

Repository: spark Updated Branches: refs/heads/branch-1.6 4ad08035d -> d509194b8 [SPARK-12057][SQL] Prevent failure on corrupt JSON records This PR makes JSON parser and schema inference handle more cases where we have unparsed records. It is based on #10043. The last commit fixes the failed

spark git commit: [SPARK-12057][SQL] Prevent failure on corrupt JSON records

Repository: spark Updated Branches: refs/heads/master 437583f69 -> 9d66c4216 [SPARK-12057][SQL] Prevent failure on corrupt JSON records This PR makes JSON parser and schema inference handle more cases where we have unparsed records. It is based on #10043. The last commit fixes the failed

spark git commit: [SPARK-11904][PYSPARK] reduceByKeyAndWindow does not require checkpointing when invFunc is None

Repository: spark Updated Branches: refs/heads/master 97678edea -> 437583f69 [SPARK-11904][PYSPARK] reduceByKeyAndWindow does not require checkpointing when invFunc is None when invFunc is None, `reduceByKeyAndWindow(func, None, winsize, slidesize)` is equivalent to

spark git commit: [SPARK-10248][CORE] track exceptions in dagscheduler event loop in tests

Repository: spark Updated Branches: refs/heads/branch-1.6 638b89bc3 -> fb02e4e3b [SPARK-10248][CORE] track exceptions in dagscheduler event loop in tests `DAGSchedulerEventLoop` normally only logs errors (so it can continue to process more events, from other jobs). However, this is not

spark git commit: [SPARK-10248][CORE] track exceptions in dagscheduler event loop in tests

Repository: spark Updated Branches: refs/heads/master ce5fd4008 -> 38d9795a4 [SPARK-10248][CORE] track exceptions in dagscheduler event loop in tests `DAGSchedulerEventLoop` normally only logs errors (so it can continue to process more events, from other jobs). However, this is not

spark git commit: [SPARK-12386][CORE] Fix NPE when spark.executor.port is set.

Repository: spark Updated Branches: refs/heads/branch-1.6 154567dca -> 4ad08035d [SPARK-12386][CORE] Fix NPE when spark.executor.port is set. Author: Marcelo Vanzin Closes #10339 from vanzin/SPARK-12386. (cherry picked from commit

spark git commit: [SPARK-12386][CORE] Fix NPE when spark.executor.port is set.

Repository: spark Updated Branches: refs/heads/master fdb382275 -> d1508dd9b [SPARK-12386][CORE] Fix NPE when spark.executor.port is set. Author: Marcelo Vanzin Closes #10339 from vanzin/SPARK-12386. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

spark git commit: MAINTENANCE: Automated closing of pull requests.

Repository: spark Updated Branches: refs/heads/master 861549acd -> ce5fd4008 MAINTENANCE: Automated closing of pull requests. This commit exists to close the following pull requests on Github: Closes #1217 (requested by ankurdave, srowen) Closes #4650 (requested by andrewor14) Closes #5307

[2/2] spark git commit: Revert "[SPARK-12105] [SQL] add convenient show functions"

Revert "[SPARK-12105] [SQL] add convenient show functions" This reverts commit 31b391019ff6eb5a483f4b3e62fd082de7ff8416. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/1a3d0cd9 Tree:

[1/2] spark git commit: Revert "[HOTFIX] Compile error from commit 31b3910"

Repository: spark Updated Branches: refs/heads/master 554d840a9 -> 1a3d0cd9f Revert "[HOTFIX] Compile error from commit 31b3910" This reverts commit 840bd2e008da5b22bfa73c587ea2c57666fffc60. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: