spark git commit: [SPARK-15784] Add Power Iteration Clustering to spark.ml

2018-06-04 Thread meng
Repository: spark Updated Branches: refs/heads/master b3417b731 -> e8c1a0c2f [SPARK-15784] Add Power Iteration Clustering to spark.ml ## What changes were proposed in this pull request? According to the discussion on JIRA. I rewrite the Power Iteration Clustering API in `spark.ml`. ## How w

svn commit: r27269 - in /dev/spark/2.4.0-SNAPSHOT-2018_06_04_20_01-b3417b7-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-06-04 Thread pwendell
Author: pwendell Date: Tue Jun 5 03:16:08 2018 New Revision: 27269 Log: Apache Spark 2.4.0-SNAPSHOT-2018_06_04_20_01-b3417b7 docs [This commit notification would consist of 1466 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

spark git commit: [SPARK-16451][REPL] Fail shell if SparkSession fails to start.

2018-06-04 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master dbb4d8382 -> b3417b731 [SPARK-16451][REPL] Fail shell if SparkSession fails to start. Currently, in spark-shell, if the session fails to start, the user sees a bunch of unrelated errors which are caused by code in the shell initialization t

spark git commit: [SPARK-24215][PYSPARK] Implement _repr_html_ for dataframes in PySpark

2018-06-04 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master ff0501b0c -> dbb4d8382 [SPARK-24215][PYSPARK] Implement _repr_html_ for dataframes in PySpark ## What changes were proposed in this pull request? Implement `_repr_html_` for PySpark while in notebook and add config named "spark.sql.repl.e

svn commit: r27263 - in /dev/spark/2.4.0-SNAPSHOT-2018_06_04_16_01-b24d3db-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-06-04 Thread pwendell
Author: pwendell Date: Mon Jun 4 23:15:58 2018 New Revision: 27263 Log: Apache Spark 2.4.0-SNAPSHOT-2018_06_04_16_01-b24d3db docs [This commit notification would consist of 1466 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

spark git commit: [SPARK-24300][ML] change the way to set seed in ml.cluster.LDASuite.generateLDAData

2018-06-04 Thread meng
Repository: spark Updated Branches: refs/heads/master b24d3dba6 -> ff0501b0c [SPARK-24300][ML] change the way to set seed in ml.cluster.LDASuite.generateLDAData ## What changes were proposed in this pull request? Using different RNG in all different partitions. ## How was this patch tested?

spark git commit: [SPARK-24290][ML] add support for Array input for instrumentation.logNamedValue

2018-06-04 Thread meng
Repository: spark Updated Branches: refs/heads/master 7297ae04d -> b24d3dba6 [SPARK-24290][ML] add support for Array input for instrumentation.logNamedValue ## What changes were proposed in this pull request? Extend instrumentation.logNamedValue to support Array input change the logging for "

spark git commit: [SPARK-21896][SQL] Fix StackOverflow caused by window functions inside aggregate functions

2018-06-04 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 0be5aa274 -> 7297ae04d [SPARK-21896][SQL] Fix StackOverflow caused by window functions inside aggregate functions ## What changes were proposed in this pull request? This PR explicitly prohibits window functions inside aggregates. Current

svn commit: r27257 - in /dev/spark/2.4.0-SNAPSHOT-2018_06_04_12_01-0be5aa2-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-06-04 Thread pwendell
Author: pwendell Date: Mon Jun 4 19:16:54 2018 New Revision: 27257 Log: Apache Spark 2.4.0-SNAPSHOT-2018_06_04_12_01-0be5aa2 docs [This commit notification would consist of 1466 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

spark git commit: [SPARK-23903][SQL] Add support for date extract

2018-06-04 Thread ueshin
Repository: spark Updated Branches: refs/heads/master 1d9338bb1 -> 0be5aa274 [SPARK-23903][SQL] Add support for date extract ## What changes were proposed in this pull request? Add support for date `extract` function: ```sql spark-sql> SELECT EXTRACT(YEAR FROM TIMESTAMP '2000-12-16 12:21:13')

svn commit: r27247 - in /dev/spark/2.4.0-SNAPSHOT-2018_06_04_00_01-1d9338b-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-06-04 Thread pwendell
Author: pwendell Date: Mon Jun 4 07:18:38 2018 New Revision: 27247 Log: Apache Spark 2.4.0-SNAPSHOT-2018_06_04_00_01-1d9338b docs [This commit notification would consist of 1466 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---