[30/61] [partial] spark-website git commit: Add docs for Spark 2.4.0 and update the latest link

2018-11-08 Thread lixiao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/52917ac4/site/docs/2.4.0/api/java/index.html -- diff --git a/site/docs/2.4.0/api/java/index.html b/site/docs/2.4.0/api/java/index.html new file mode 100644 index

[20/61] [partial] spark-website git commit: Add docs for Spark 2.4.0 and update the latest link

2018-11-08 Thread lixiao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/52917ac4/site/docs/2.4.0/api/java/org/apache/spark/OneToOneDependency.html -- diff --git a/site/docs/2.4.0/api/java/org/apache/spark/OneToOneDependency.html

[25/61] [partial] spark-website git commit: Add docs for Spark 2.4.0 and update the latest link

2018-11-08 Thread lixiao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/52917ac4/site/docs/2.4.0/api/java/org/apache/spark/CleanBroadcast.html -- diff --git a/site/docs/2.4.0/api/java/org/apache/spark/CleanBroadcast.html

[11/61] [partial] spark-website git commit: Add docs for Spark 2.4.0 and update the latest link

2018-11-08 Thread lixiao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/52917ac4/site/docs/2.4.0/api/java/org/apache/spark/UnknownReason.html -- diff --git a/site/docs/2.4.0/api/java/org/apache/spark/UnknownReason.html

[13/61] [partial] spark-website git commit: Add docs for Spark 2.4.0 and update the latest link

2018-11-08 Thread lixiao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/52917ac4/site/docs/2.4.0/api/java/org/apache/spark/TaskCommitDenied.html -- diff --git a/site/docs/2.4.0/api/java/org/apache/spark/TaskCommitDenied.html

[33/61] [partial] spark-website git commit: Add docs for Spark 2.4.0 and update the latest link

2018-11-08 Thread lixiao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/52917ac4/site/docs/2.4.0/api/java/allclasses-noframe.html -- diff --git a/site/docs/2.4.0/api/java/allclasses-noframe.html

[55/61] spark-website git commit: update website

2018-11-08 Thread lixiao
update website Project: http://git-wip-us.apache.org/repos/asf/spark-website/repo Commit: http://git-wip-us.apache.org/repos/asf/spark-website/commit/62847a7d Tree: http://git-wip-us.apache.org/repos/asf/spark-website/tree/62847a7d Diff:

[46/61] [partial] spark-website git commit: Add docs for Spark 2.4.0 and update the latest link

2018-11-08 Thread lixiao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/52917ac4/site/docs/2.4.0/api/R/column_collection_functions.html -- diff --git a/site/docs/2.4.0/api/R/column_collection_functions.html

[43/61] [partial] spark-website git commit: Add docs for Spark 2.4.0 and update the latest link

2018-11-08 Thread lixiao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/52917ac4/site/docs/2.4.0/api/R/gapplyCollect.html -- diff --git a/site/docs/2.4.0/api/R/gapplyCollect.html b/site/docs/2.4.0/api/R/gapplyCollect.html new file mode 100644

[45/61] [partial] spark-website git commit: Add docs for Spark 2.4.0 and update the latest link

2018-11-08 Thread lixiao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/52917ac4/site/docs/2.4.0/api/R/columnfunctions.html -- diff --git a/site/docs/2.4.0/api/R/columnfunctions.html b/site/docs/2.4.0/api/R/columnfunctions.html new file mode

[08/61] [partial] spark-website git commit: Add docs for Spark 2.4.0 and update the latest link

2018-11-08 Thread lixiao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/52917ac4/site/docs/2.4.0/api/java/org/apache/spark/api/java/JavaNewHadoopRDD.html -- diff --git a/site/docs/2.4.0/api/java/org/apache/spark/api/java/JavaNewHadoopRDD.html

[02/61] [partial] spark-website git commit: Add docs for Spark 2.4.0 and update the latest link

2018-11-08 Thread lixiao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/52917ac4/site/docs/2.4.0/api/java/org/apache/spark/api/java/function/CoGroupFunction.html -- diff --git

[01/61] [partial] spark-website git commit: Add docs for Spark 2.4.0 and update the latest link

2018-11-08 Thread lixiao
Repository: spark-website Updated Branches: refs/heads/asf-site 44d255281 -> 02491b5e1 http://git-wip-us.apache.org/repos/asf/spark-website/blob/52917ac4/site/docs/2.4.0/api/java/org/apache/spark/api/java/function/ForeachFunction.html

[07/61] [partial] spark-website git commit: Add docs for Spark 2.4.0 and update the latest link

2018-11-08 Thread lixiao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/52917ac4/site/docs/2.4.0/api/java/org/apache/spark/api/java/JavaPairRDD.html -- diff --git a/site/docs/2.4.0/api/java/org/apache/spark/api/java/JavaPairRDD.html

[04/61] [partial] spark-website git commit: Add docs for Spark 2.4.0 and update the latest link

2018-11-08 Thread lixiao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/52917ac4/site/docs/2.4.0/api/java/org/apache/spark/api/java/JavaSparkContext.html -- diff --git a/site/docs/2.4.0/api/java/org/apache/spark/api/java/JavaSparkContext.html

[10/61] [partial] spark-website git commit: Add docs for Spark 2.4.0 and update the latest link

2018-11-08 Thread lixiao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/52917ac4/site/docs/2.4.0/api/java/org/apache/spark/api/java/JavaDoubleRDD.html -- diff --git a/site/docs/2.4.0/api/java/org/apache/spark/api/java/JavaDoubleRDD.html

[15/61] [partial] spark-website git commit: Add docs for Spark 2.4.0 and update the latest link

2018-11-08 Thread lixiao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/52917ac4/site/docs/2.4.0/api/java/org/apache/spark/SparkFirehoseListener.html -- diff --git a/site/docs/2.4.0/api/java/org/apache/spark/SparkFirehoseListener.html

[28/61] [partial] spark-website git commit: Add docs for Spark 2.4.0 and update the latest link

2018-11-08 Thread lixiao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/52917ac4/site/docs/2.4.0/api/java/org/apache/spark/Accumulable.html -- diff --git a/site/docs/2.4.0/api/java/org/apache/spark/Accumulable.html

[56/61] spark-website git commit: address comments

2018-11-08 Thread lixiao
address comments Project: http://git-wip-us.apache.org/repos/asf/spark-website/repo Commit: http://git-wip-us.apache.org/repos/asf/spark-website/commit/02a3641f Tree: http://git-wip-us.apache.org/repos/asf/spark-website/tree/02a3641f Diff:

[35/61] [partial] spark-website git commit: Add docs for Spark 2.4.0 and update the latest link

2018-11-08 Thread lixiao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/52917ac4/site/docs/2.4.0/api/R/write.text.html -- diff --git a/site/docs/2.4.0/api/R/write.text.html b/site/docs/2.4.0/api/R/write.text.html new file mode 100644 index

[37/61] [partial] spark-website git commit: Add docs for Spark 2.4.0 and update the latest link

2018-11-08 Thread lixiao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/52917ac4/site/docs/2.4.0/api/R/startsWith.html -- diff --git a/site/docs/2.4.0/api/R/startsWith.html b/site/docs/2.4.0/api/R/startsWith.html new file mode 100644 index

[60/61] spark-website git commit: update date

2018-11-08 Thread lixiao
update date Project: http://git-wip-us.apache.org/repos/asf/spark-website/repo Commit: http://git-wip-us.apache.org/repos/asf/spark-website/commit/3726c326 Tree: http://git-wip-us.apache.org/repos/asf/spark-website/tree/3726c326 Diff:

[09/61] [partial] spark-website git commit: Add docs for Spark 2.4.0 and update the latest link

2018-11-08 Thread lixiao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/52917ac4/site/docs/2.4.0/api/java/org/apache/spark/api/java/JavaFutureAction.html -- diff --git a/site/docs/2.4.0/api/java/org/apache/spark/api/java/JavaFutureAction.html

[32/61] [partial] spark-website git commit: Add docs for Spark 2.4.0 and update the latest link

2018-11-08 Thread lixiao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/52917ac4/site/docs/2.4.0/api/java/constant-values.html -- diff --git a/site/docs/2.4.0/api/java/constant-values.html b/site/docs/2.4.0/api/java/constant-values.html new

[49/61] [partial] spark-website git commit: Add docs for Spark 2.4.0 and update the latest link

2018-11-08 Thread lixiao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/52917ac4/site/docs/2.4.0/api/R/00Index.html -- diff --git a/site/docs/2.4.0/api/R/00Index.html b/site/docs/2.4.0/api/R/00Index.html new file mode 100644 index

[06/61] [partial] spark-website git commit: Add docs for Spark 2.4.0 and update the latest link

2018-11-08 Thread lixiao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/52917ac4/site/docs/2.4.0/api/java/org/apache/spark/api/java/JavaRDD.html -- diff --git a/site/docs/2.4.0/api/java/org/apache/spark/api/java/JavaRDD.html

[52/61] spark-website git commit: update website

2018-11-08 Thread lixiao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/62847a7d/site/releases/spark-release-1-6-2.html -- diff --git a/site/releases/spark-release-1-6-2.html b/site/releases/spark-release-1-6-2.html index 5145941..b64f7c3

[38/61] [partial] spark-website git commit: Add docs for Spark 2.4.0 and update the latest link

2018-11-08 Thread lixiao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/52917ac4/site/docs/2.4.0/api/R/spark.lda.html -- diff --git a/site/docs/2.4.0/api/R/spark.lda.html b/site/docs/2.4.0/api/R/spark.lda.html new file mode 100644 index

spark git commit: [MINOR] update known_translations

2018-11-06 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.4 d6c713fa5 -> 4c91b224a [MINOR] update known_translations ## What changes were proposed in this pull request? update known_translations after running `translate-contributors.py` during 2.4.0 release ## How was this patch tested? N/A

spark git commit: [MINOR] update known_translations

2018-11-06 Thread lixiao
Repository: spark Updated Branches: refs/heads/master cee230160 -> a241a150d [MINOR] update known_translations ## What changes were proposed in this pull request? update known_translations after running `translate-contributors.py` during 2.4.0 release ## How was this patch tested? N/A

spark git commit: [SPARK-25913][SQL] Extend UnaryExecNode by unary SparkPlan nodes

2018-11-04 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 6c9e5ac9d -> 950e7374a [SPARK-25913][SQL] Extend UnaryExecNode by unary SparkPlan nodes ## What changes were proposed in this pull request? In the PR, I propose to extend `UnaryExecNode` instead of `SparkPlan` by unary nodes. Closes

spark git commit: [SPARK-25899][TESTS] Fix flaky CoarseGrainedSchedulerBackendSuite

2018-10-31 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.4 5cc2987db -> 3d2fce5a3 [SPARK-25899][TESTS] Fix flaky CoarseGrainedSchedulerBackendSuite ## What changes were proposed in this pull request? I saw CoarseGrainedSchedulerBackendSuite failed in my PR and finally reproduced the

spark git commit: [SPARK-25899][TESTS] Fix flaky CoarseGrainedSchedulerBackendSuite

2018-10-31 Thread lixiao
Repository: spark Updated Branches: refs/heads/master bc9f9b4d6 -> 6be3cce75 [SPARK-25899][TESTS] Fix flaky CoarseGrainedSchedulerBackendSuite ## What changes were proposed in this pull request? I saw CoarseGrainedSchedulerBackendSuite failed in my PR and finally reproduced the following

spark git commit: [SPARK-25883][SQL][MINOR] Override method `prettyName` in `from_avro`/`to_avro`

2018-10-31 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 49bea5a7e -> 0ad93b093 [SPARK-25883][SQL][MINOR] Override method `prettyName` in `from_avro`/`to_avro` ## What changes were proposed in this pull request? Previously in from_avro/to_avro, we override the method `simpleString` and `sql`

spark git commit: [SPARK-25862][SQL] Remove rangeBetween APIs introduced in SPARK-21608

2018-10-30 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 243ce319a -> 9cf9a83af [SPARK-25862][SQL] Remove rangeBetween APIs introduced in SPARK-21608 ## What changes were proposed in this pull request? This patch removes the rangeBetween functions introduced in SPARK-21608. As explained in

spark git commit: [SPARK-25179][PYTHON][DOCS] Document BinaryType support in Arrow conversion

2018-10-29 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.4 b6ba0dd47 -> 7f4fce426 [SPARK-25179][PYTHON][DOCS] Document BinaryType support in Arrow conversion ## What changes were proposed in this pull request? This PR targets to document binary type in "Apache Arrow in Spark". ## How was

spark git commit: [SPARK-25179][PYTHON][DOCS] Document BinaryType support in Arrow conversion

2018-10-29 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 4e990d9dd -> fbaf15050 [SPARK-25179][PYTHON][DOCS] Document BinaryType support in Arrow conversion ## What changes were proposed in this pull request? This PR targets to document binary type in "Apache Arrow in Spark". ## How was this

spark git commit: [SPARK-25816][SQL] Fix attribute resolution in nested extractors

2018-10-28 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.3 3afb3a20e -> 53aeb3d65 [SPARK-25816][SQL] Fix attribute resolution in nested extractors Extractors are made of 2 expressions, one of them defines the the value to be extract from (called `child`) and the other defines the way of

spark git commit: [SPARK-25816][SQL] Fix attribute resolution in nested extractors

2018-10-28 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 4427a96bc -> ca2fca143 [SPARK-25816][SQL] Fix attribute resolution in nested extractors ## What changes were proposed in this pull request? Extractors are made of 2 expressions, one of them defines the the value to be extract from

spark git commit: [SPARK-25816][SQL] Fix attribute resolution in nested extractors

2018-10-28 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.4 0f74bac64 -> 00771dced [SPARK-25816][SQL] Fix attribute resolution in nested extractors ## What changes were proposed in this pull request? Extractors are made of 2 expressions, one of them defines the the value to be extract from

spark git commit: [SPARK-24499][SQL][DOC][FOLLOWUP] Fix some broken links

2018-10-20 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 3370865b0 -> ed9d0aac9 [SPARK-24499][SQL][DOC][FOLLOWUP] Fix some broken links ## What changes were proposed in this pull request? Fix some broken links in the new document. I have clicked through all the links. Hopefully i haven't missed

spark git commit: [SPARK-24499][SQL][DOC][FOLLOWUP] Fix some broken links

2018-10-20 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.4 e3a60b0d6 -> d6a02c568 [SPARK-24499][SQL][DOC][FOLLOWUP] Fix some broken links ## What changes were proposed in this pull request? Fix some broken links in the new document. I have clicked through all the links. Hopefully i haven't

spark git commit: Revert "[SPARK-25758][ML] Deprecate computeCost on BisectingKMeans"

2018-10-19 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.4 1001d2314 -> 432697c7b Revert "[SPARK-25758][ML] Deprecate computeCost on BisectingKMeans" This reverts commit c2962546d9a5900a5628a31b83d2c4b22c3a7936. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

[2/4] spark git commit: [SPARK-24499][SQL][DOC] Split the page of sql-programming-guide.html to multiple separate pages

2018-10-18 Thread lixiao
http://git-wip-us.apache.org/repos/asf/spark/blob/71535516/docs/sql-programming-guide.md -- diff --git a/docs/sql-programming-guide.md b/docs/sql-programming-guide.md index e45e50d..42b00c9 100644 ---

[1/4] spark git commit: [SPARK-24499][SQL][DOC] Split the page of sql-programming-guide.html to multiple separate pages

2018-10-18 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.4 71a6a9ce8 -> 715355164 http://git-wip-us.apache.org/repos/asf/spark/blob/71535516/docs/sql-pyspark-pandas-with-arrow.md -- diff --git

[3/4] spark git commit: [SPARK-24499][SQL][DOC] Split the page of sql-programming-guide.html to multiple separate pages

2018-10-18 Thread lixiao
http://git-wip-us.apache.org/repos/asf/spark/blob/71535516/docs/sql-data-sources-troubleshooting.md -- diff --git a/docs/sql-data-sources-troubleshooting.md b/docs/sql-data-sources-troubleshooting.md new file mode 100644 index

[4/4] spark git commit: [SPARK-24499][SQL][DOC] Split the page of sql-programming-guide.html to multiple separate pages

2018-10-18 Thread lixiao
[SPARK-24499][SQL][DOC] Split the page of sql-programming-guide.html to multiple separate pages 1. Split the main page of sql-programming-guide into 7 parts: - Getting Started - Data Sources - Performance Turing - Distributed SQL Engine - PySpark Usage Guide for Pandas with Apache Arrow -

[3/4] spark git commit: [SPARK-24499][SQL][DOC] Split the page of sql-programming-guide.html to multiple separate pages

2018-10-18 Thread lixiao
http://git-wip-us.apache.org/repos/asf/spark/blob/987f3865/docs/sql-data-sources-troubleshooting.md -- diff --git a/docs/sql-data-sources-troubleshooting.md b/docs/sql-data-sources-troubleshooting.md new file mode 100644 index

[4/4] spark git commit: [SPARK-24499][SQL][DOC] Split the page of sql-programming-guide.html to multiple separate pages

2018-10-18 Thread lixiao
[SPARK-24499][SQL][DOC] Split the page of sql-programming-guide.html to multiple separate pages ## What changes were proposed in this pull request? 1. Split the main page of sql-programming-guide into 7 parts: - Getting Started - Data Sources - Performance Turing - Distributed SQL Engine -

[1/4] spark git commit: [SPARK-24499][SQL][DOC] Split the page of sql-programming-guide.html to multiple separate pages

2018-10-18 Thread lixiao
Repository: spark Updated Branches: refs/heads/master c2962546d -> 987f38658 http://git-wip-us.apache.org/repos/asf/spark/blob/987f3865/docs/sql-pyspark-pandas-with-arrow.md -- diff --git

[2/4] spark git commit: [SPARK-24499][SQL][DOC] Split the page of sql-programming-guide.html to multiple separate pages

2018-10-18 Thread lixiao
http://git-wip-us.apache.org/repos/asf/spark/blob/987f3865/docs/sql-programming-guide.md -- diff --git a/docs/sql-programming-guide.md b/docs/sql-programming-guide.md index fb03ed2..42b00c9 100644 ---

spark git commit: [SPARK-25716][SQL][MINOR] remove unnecessary collection operation in valid constraints generation

2018-10-15 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 56247c1d1 -> 0820484ba [SPARK-25716][SQL][MINOR] remove unnecessary collection operation in valid constraints generation ## What changes were proposed in this pull request? Project logical operator generates valid constraints using two

spark git commit: [SPARK-25727][FOLLOWUP] Move outputOrdering to case class field for InMemoryRelation

2018-10-15 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 9426fd0c2 -> 56247c1d1 [SPARK-25727][FOLLOWUP] Move outputOrdering to case class field for InMemoryRelation ## What changes were proposed in this pull request? The PR addresses [the

spark git commit: [SPARK-25714][SQL][FOLLOWUP] improve the comment inside BooleanSimplification rule

2018-10-13 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.4 663481933 -> c4efcf1dd [SPARK-25714][SQL][FOLLOWUP] improve the comment inside BooleanSimplification rule ## What changes were proposed in this pull request? improve the code comment added in

spark git commit: [SPARK-25714][SQL][FOLLOWUP] improve the comment inside BooleanSimplification rule

2018-10-13 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 26c1b959c -> b73f76beb [SPARK-25714][SQL][FOLLOWUP] improve the comment inside BooleanSimplification rule ## What changes were proposed in this pull request? improve the code comment added in

spark git commit: [MINOR] Fix code comment in BooleanSimplification.

2018-10-13 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.4 5554a33f2 -> 765cbcace [MINOR] Fix code comment in BooleanSimplification. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/765cbcac Tree:

spark git commit: [MINOR] Fix code comment in BooleanSimplification.

2018-10-13 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 34f229bc2 -> 8812746d4 [MINOR] Fix code comment in BooleanSimplification. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/8812746d Tree:

spark git commit: Revert "[SPARK-25714] Fix Null Handling in the Optimizer rule BooleanSimplification"

2018-10-12 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.3 182bc85f2 -> b3d1b1bcb Revert "[SPARK-25714] Fix Null Handling in the Optimizer rule BooleanSimplification" This reverts commit 182bc85f2db0b3268b9b93ff91210811b00e1636. Project: http://git-wip-us.apache.org/repos/asf/spark/repo

spark git commit: [SPARK-25714] Fix Null Handling in the Optimizer rule BooleanSimplification

2018-10-12 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.3 5324a85a2 -> 182bc85f2 [SPARK-25714] Fix Null Handling in the Optimizer rule BooleanSimplification ## What changes were proposed in this pull request? ```Scala val df1 = Seq(("abc", 1), (null, 3)).toDF("col1", "col2")

spark git commit: [SPARK-25714] Fix Null Handling in the Optimizer rule BooleanSimplification

2018-10-12 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.4 0f58b989d -> 5554a33f2 [SPARK-25714] Fix Null Handling in the Optimizer rule BooleanSimplification ## What changes were proposed in this pull request? ```Scala val df1 = Seq(("abc", 1), (null, 3)).toDF("col1", "col2")

spark git commit: [SPARK-25714] Fix Null Handling in the Optimizer rule BooleanSimplification

2018-10-12 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 3946de773 -> c9ba59d38 [SPARK-25714] Fix Null Handling in the Optimizer rule BooleanSimplification ## What changes were proposed in this pull request? ```Scala val df1 = Seq(("abc", 1), (null, 3)).toDF("col1", "col2")

spark git commit: [SPARK-25660][SQL] Fix for the backward slash as CSV fields delimiter

2018-10-12 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.4 bb211cf27 -> 1a335444e [SPARK-25660][SQL] Fix for the backward slash as CSV fields delimiter ## What changes were proposed in this pull request? The PR addresses the exception raised on accessing chars out of delimiter string. In

spark git commit: [SPARK-25660][SQL] Fix for the backward slash as CSV fields delimiter

2018-10-12 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 8e039a755 -> c7eadb5e6 [SPARK-25660][SQL] Fix for the backward slash as CSV fields delimiter ## What changes were proposed in this pull request? The PR addresses the exception raised on accessing chars out of delimiter string. In

spark git commit: [SPARK-25708][SQL] HAVING without GROUP BY means global aggregate

2018-10-12 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.4 1961f8e62 -> 3dba5d41f [SPARK-25708][SQL] HAVING without GROUP BY means global aggregate According to the SQL standard, when a query contains `HAVING`, it indicates an aggregate operator. For more details please refer to

spark git commit: [SPARK-25708][SQL] HAVING without GROUP BY means global aggregate

2018-10-12 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 368513048 -> 78e133141 [SPARK-25708][SQL] HAVING without GROUP BY means global aggregate ## What changes were proposed in this pull request? According to the SQL standard, when a query contains `HAVING`, it indicates an aggregate

spark git commit: [SPARK-25690][SQL] Analyzer rule HandleNullInputsForUDF does not stabilize and can be applied infinitely

2018-10-11 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.4 e80ab130e -> 1961f8e62 [SPARK-25690][SQL] Analyzer rule HandleNullInputsForUDF does not stabilize and can be applied infinitely ## What changes were proposed in this pull request? The HandleNullInputsForUDF rule can generate new If

spark git commit: [SPARK-25690][SQL] Analyzer rule HandleNullInputsForUDF does not stabilize and can be applied infinitely

2018-10-11 Thread lixiao
Repository: spark Updated Branches: refs/heads/master c9d7d83ed -> 368513048 [SPARK-25690][SQL] Analyzer rule HandleNullInputsForUDF does not stabilize and can be applied infinitely ## What changes were proposed in this pull request? The HandleNullInputsForUDF rule can generate new If node

spark git commit: [SPARK-25630][TEST] Reduce test time of HadoopFsRelationTest

2018-10-08 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 6353425af -> 6a60fb0aa [SPARK-25630][TEST] Reduce test time of HadoopFsRelationTest ## What changes were proposed in this pull request? There was 5 suites extends `HadoopFsRelationTest`, for testing "orc"/"parquet"/"text"/"json" data

spark git commit: [SPARK-25671] Build external/spark-ganglia-lgpl in Jenkins Test

2018-10-06 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.4 48e2e6fcc -> c8b94099a [SPARK-25671] Build external/spark-ganglia-lgpl in Jenkins Test ## What changes were proposed in this pull request? Currently, we do not build external/spark-ganglia-lgpl in Jenkins tests when the code is

spark git commit: [SPARK-25671] Build external/spark-ganglia-lgpl in Jenkins Test

2018-10-06 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 756a3ab18 -> 8bb242902 [SPARK-25671] Build external/spark-ganglia-lgpl in Jenkins Test ## What changes were proposed in this pull request? Currently, we do not build external/spark-ganglia-lgpl in Jenkins tests when the code is changed.

spark git commit: [MINOR] Clean up the joinCriteria in SQL parser

2018-10-06 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 7ef65c053 -> 5a617ec4e [MINOR] Clean up the joinCriteria in SQL parser ## What changes were proposed in this pull request? Clean up the joinCriteria parsing in the parser by directly using identifierList ## How was this patch tested? N/A

spark git commit: [HOT-FIX] Fix compilation errors.

2018-10-06 Thread lixiao
Repository: spark Updated Branches: refs/heads/master edf428661 -> 7ef65c053 [HOT-FIX] Fix compilation errors. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/7ef65c05 Tree:

spark git commit: [SPARK-25610][SQL][TEST] Improve execution time of DatasetCacheSuite: cache UDF result correctly

2018-10-05 Thread lixiao
Repository: spark Updated Branches: refs/heads/master bbd038d24 -> 2c6f4d61b [SPARK-25610][SQL][TEST] Improve execution time of DatasetCacheSuite: cache UDF result correctly ## What changes were proposed in this pull request? In this test case, we are verifying that the result of an UDF is

spark git commit: [SPARK-25653][TEST] Add tag ExtendedHiveTest for HiveSparkSubmitSuite

2018-10-05 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 1c9486c1a -> bbd038d24 [SPARK-25653][TEST] Add tag ExtendedHiveTest for HiveSparkSubmitSuite ## What changes were proposed in this pull request? The total run time of `HiveSparkSubmitSuite` is about 10 minutes. While the related code is

spark git commit: [SPARK-25635][SQL][BUILD] Support selective direct encoding in native ORC write

2018-10-05 Thread lixiao
Repository: spark Updated Branches: refs/heads/master a433fbcee -> 1c9486c1a [SPARK-25635][SQL][BUILD] Support selective direct encoding in native ORC write ## What changes were proposed in this pull request? Before ORC 1.5.3, `orc.dictionary.key.threshold` and

spark git commit: [SPARK-25626][SQL][TEST] Improve the test execution time of HiveClientSuites

2018-10-05 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 7dcc90fbb -> a433fbcee [SPARK-25626][SQL][TEST] Improve the test execution time of HiveClientSuites ## What changes were proposed in this pull request? Improve the runtime by reducing the number of partitions created in the test. The

spark git commit: [SPARK-25605][TESTS] Run cast string to timestamp tests for a subset of timezones

2018-10-04 Thread lixiao
Repository: spark Updated Branches: refs/heads/master f27d96b9f -> 8113b9c96 [SPARK-25605][TESTS] Run cast string to timestamp tests for a subset of timezones ## What changes were proposed in this pull request? The test `cast string to timestamp` used to run for all time zones. So it run

spark git commit: [SPARK-25606][TEST] Reduce DateExpressionsSuite test time costs in Jenkins

2018-10-04 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 85a93595d -> f27d96b9f [SPARK-25606][TEST] Reduce DateExpressionsSuite test time costs in Jenkins ## What changes were proposed in this pull request? Reduce `DateExpressionsSuite.Hour` test time costs in Jenkins by reduce iteration

spark git commit: [SPARK-25609][TESTS] Reduce time of test for SPARK-22226

2018-10-04 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 3ae4f07de -> 85a93595d [SPARK-25609][TESTS] Reduce time of test for SPARK-6 ## What changes were proposed in this pull request? The PR changes the test introduced for SPARK-6, so that we don't run analysis and optimization on the

spark-website git commit: Add Xingbo Jiang to the list of committers

2018-10-02 Thread lixiao
Repository: spark-website Updated Branches: refs/heads/asf-site 4ea5a5d8f -> 9ba1fcee8 Add Xingbo Jiang to the list of committers Project: http://git-wip-us.apache.org/repos/asf/spark-website/repo Commit: http://git-wip-us.apache.org/repos/asf/spark-website/commit/9ba1fcee Tree:

spark git commit: [SPARK-25592] Setting version to 3.0.0-SNAPSHOT

2018-10-02 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 71876633f -> 9bf397c0e [SPARK-25592] Setting version to 3.0.0-SNAPSHOT ## What changes were proposed in this pull request? This patch is to bump the master branch version to 3.0.0-SNAPSHOT. ## How was this patch tested? N/A Closes

spark git commit: [SPARK-25453][SQL][TEST][.FFFFFFFFF] OracleIntegrationSuite IllegalArgumentException: Timestamp format must be yyyy-mm-dd hh:mm:ss

2018-09-30 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.4 7b1094b54 -> 82990e5ef [SPARK-25453][SQL][TEST][.F] OracleIntegrationSuite IllegalArgumentException: Timestamp format must be -mm-dd hh:mm:ss ## What changes were proposed in this pull request? This PR aims to fix the

spark git commit: [SPARK-25453][SQL][TEST][.FFFFFFFFF] OracleIntegrationSuite IllegalArgumentException: Timestamp format must be yyyy-mm-dd hh:mm:ss

2018-09-30 Thread lixiao
Repository: spark Updated Branches: refs/heads/master fb8f4c056 -> 21f0b73db [SPARK-25453][SQL][TEST][.F] OracleIntegrationSuite IllegalArgumentException: Timestamp format must be -mm-dd hh:mm:ss ## What changes were proposed in this pull request? This PR aims to fix the failed

spark git commit: [SPARK-25505][SQL][FOLLOWUP] Fix for attributes cosmetically different in Pivot clause

2018-09-30 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 4da541a5d -> fb8f4c056 [SPARK-25505][SQL][FOLLOWUP] Fix for attributes cosmetically different in Pivot clause ## What changes were proposed in this pull request? #22519 introduced a bug when the attributes in the pivot clause are

spark git commit: [SPARK-25505][SQL][FOLLOWUP] Fix for attributes cosmetically different in Pivot clause

2018-09-30 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.4 c886f050b -> 7b1094b54 [SPARK-25505][SQL][FOLLOWUP] Fix for attributes cosmetically different in Pivot clause ## What changes were proposed in this pull request? #22519 introduced a bug when the attributes in the pivot clause are

spark git commit: [SPARK-25568][CORE] Continue to update the remaining accumulators when failing to update one accumulator

2018-09-29 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.2 ef3616825 -> 5dd14f5d9 [SPARK-25568][CORE] Continue to update the remaining accumulators when failing to update one accumulator ## What changes were proposed in this pull request? Since we don't fail a job when `AccumulatorV2.merge`

spark git commit: [SPARK-25568][CORE] Continue to update the remaining accumulators when failing to update one accumulator

2018-09-29 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.3 eb78380c0 -> 73408f085 [SPARK-25568][CORE] Continue to update the remaining accumulators when failing to update one accumulator ## What changes were proposed in this pull request? Since we don't fail a job when `AccumulatorV2.merge`

spark git commit: [SPARK-25568][CORE] Continue to update the remaining accumulators when failing to update one accumulator

2018-09-29 Thread lixiao
Repository: spark Updated Branches: refs/heads/master f4b138082 -> b6b8a6632 [SPARK-25568][CORE] Continue to update the remaining accumulators when failing to update one accumulator ## What changes were proposed in this pull request? Since we don't fail a job when `AccumulatorV2.merge`

spark git commit: [SPARK-25568][CORE] Continue to update the remaining accumulators when failing to update one accumulator

2018-09-29 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.4 fef30271b -> 6f510c69b [SPARK-25568][CORE] Continue to update the remaining accumulators when failing to update one accumulator ## What changes were proposed in this pull request? Since we don't fail a job when `AccumulatorV2.merge`

spark git commit: [SPARK-25429][SQL] Use Set instead of Array to improve lookup performance

2018-09-28 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 7deef7a49 -> a28146568 [SPARK-25429][SQL] Use Set instead of Array to improve lookup performance ## What changes were proposed in this pull request? Use `Set` instead of `Array` to improve `accumulatorIds.contains(acc.id)` performance.

spark git commit: [SPARK-25458][SQL] Support FOR ALL COLUMNS in ANALYZE TABLE

2018-09-28 Thread lixiao
Repository: spark Updated Branches: refs/heads/master b7d80349b -> 7deef7a49 [SPARK-25458][SQL] Support FOR ALL COLUMNS in ANALYZE TABLE ## What changes were proposed in this pull request? **Description from the JIRA :** Currently, to collect the statistics of all the columns, users need to

spark git commit: [SPARK-25505][SQL] The output order of grouping columns in Pivot is different from the input order

2018-09-28 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.4 a43a082e0 -> b2a1e2f8d [SPARK-25505][SQL] The output order of grouping columns in Pivot is different from the input order ## What changes were proposed in this pull request? The grouping columns from a Pivot query are inferred as

spark git commit: [SPARK-25505][SQL] The output order of grouping columns in Pivot is different from the input order

2018-09-28 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 3b7395fe0 -> e120a38c0 [SPARK-25505][SQL] The output order of grouping columns in Pivot is different from the input order ## What changes were proposed in this pull request? The grouping columns from a Pivot query are inferred as "input

spark git commit: [SPARK-25454][SQL] add a new config for picking minimum precision for integral literals

2018-09-26 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.3 2381d60a2 -> 26d893a4f [SPARK-25454][SQL] add a new config for picking minimum precision for integral literals ## What changes were proposed in this pull request? https://github.com/apache/spark/pull/20023 proposed to allow precision

spark git commit: [SPARK-25454][SQL] add a new config for picking minimum precision for integral literals

2018-09-26 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.4 8d1720079 -> 2ff91f213 [SPARK-25454][SQL] add a new config for picking minimum precision for integral literals ## What changes were proposed in this pull request? https://github.com/apache/spark/pull/20023 proposed to allow precision

spark git commit: [SPARK-25454][SQL] add a new config for picking minimum precision for integral literals

2018-09-26 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 51540c2fa -> d0990e3df [SPARK-25454][SQL] add a new config for picking minimum precision for integral literals ## What changes were proposed in this pull request? https://github.com/apache/spark/pull/20023 proposed to allow precision

spark git commit: [SPARK-25465][TEST] Refactor Parquet test suites in project Hive

2018-09-22 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 40edab209 -> 6ca87eb2e [SPARK-25465][TEST] Refactor Parquet test suites in project Hive ## What changes were proposed in this pull request? Current the file

spark git commit: [SPARK-19724][SQL] allowCreatingManagedTableUsingNonemptyLocation should have legacy prefix

2018-09-21 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.4 604828eda -> ce6636112 [SPARK-19724][SQL] allowCreatingManagedTableUsingNonemptyLocation should have legacy prefix One more legacy config to go ... Closes #22515 from rxin/allowCreatingManagedTableUsingNonemptyLocation. Authored-by:

spark git commit: [SPARK-19724][SQL] allowCreatingManagedTableUsingNonemptyLocation should have legacy prefix

2018-09-21 Thread lixiao
Repository: spark Updated Branches: refs/heads/master d25f425c9 -> 4a1120953 [SPARK-19724][SQL] allowCreatingManagedTableUsingNonemptyLocation should have legacy prefix One more legacy config to go ... Closes #22515 from rxin/allowCreatingManagedTableUsingNonemptyLocation. Authored-by:

spark git commit: [SPARK-25494][SQL] Upgrade Spark's use of Janino to 3.0.10

2018-09-20 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 5d25e1544 -> 596af211a [SPARK-25494][SQL] Upgrade Spark's use of Janino to 3.0.10 ## What changes were proposed in this pull request? This PR upgrades Spark's use of Janino from 3.0.9 to 3.0.10. Note that 3.0.10 is a out-of-band release

<    1   2   3   4   5   6   7   8   9   10   >