spark git commit: [SPARK-23352][PYTHON] Explicitly specify supported types in Pandas UDFs

2018-02-12 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master 6efd5d117 -> c338c8cf8 [SPARK-23352][PYTHON] Explicitly specify supported types in Pandas UDFs ## What changes were proposed in this pull request? This PR targets to explicitly specify supported types in Pandas UDFs. The main change here

svn commit: r24943 - in /dev/spark/2.3.1-SNAPSHOT-2018_02_12_02_01-79e8650-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-02-12 Thread pwendell
Author: pwendell Date: Mon Feb 12 10:15:46 2018 New Revision: 24943 Log: Apache Spark 2.3.1-SNAPSHOT-2018_02_12_02_01-79e8650 docs [This commit notification would consist of 1443 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

svn commit: r24948 - in /dev/spark/2.4.0-SNAPSHOT-2018_02_12_04_35-c338c8c-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-02-12 Thread pwendell
Author: pwendell Date: Mon Feb 12 12:49:35 2018 New Revision: 24948 Log: Apache Spark 2.4.0-SNAPSHOT-2018_02_12_04_35-c338c8c docs [This commit notification would consist of 1444 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-23391][CORE] It may lead to overflow for some integer multiplication

2018-02-12 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.2 169483455 -> 14b5dbfa9 [SPARK-23391][CORE] It may lead to overflow for some integer multiplication In the `getBlockData`,`blockId.reduceId` is the `Int` type, when it is greater than 2^28, `blockId.reduceId*8` will overflow In the

spark git commit: [MINOR][TEST] spark.testing` No effect on the SparkFunSuite unit test

2018-02-12 Thread wenchen
Repository: spark Updated Branches: refs/heads/master c338c8cf8 -> caeb108e2 [MINOR][TEST] spark.testing` No effect on the SparkFunSuite unit test ## What changes were proposed in this pull request? Currently, we use SBT and MAVN to spark unit test, are affected by the parameters of

spark git commit: [SPARK-23391][CORE] It may lead to overflow for some integer multiplication

2018-02-12 Thread srowen
Repository: spark Updated Branches: refs/heads/master 0e2c266de -> 4a4dd4f36 [SPARK-23391][CORE] It may lead to overflow for some integer multiplication ## What changes were proposed in this pull request? In the `getBlockData`,`blockId.reduceId` is the `Int` type, when it is greater than

spark git commit: [SPARK-23391][CORE] It may lead to overflow for some integer multiplication

2018-02-12 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.3 1e3118c2e -> d31c4ae7b [SPARK-23391][CORE] It may lead to overflow for some integer multiplication ## What changes were proposed in this pull request? In the `getBlockData`,`blockId.reduceId` is the `Int` type, when it is greater than

spark git commit: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-12 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.3 79e8650cc -> 1e3118c2e [SPARK-22977][SQL] fix web UI SQL tab for CTAS ## What changes were proposed in this pull request? This is a regression in Spark 2.3. In Spark 2.2, we have a fragile UI support for SQL data writing commands. We

spark git commit: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-12 Thread wenchen
Repository: spark Updated Branches: refs/heads/master caeb108e2 -> 0e2c266de [SPARK-22977][SQL] fix web UI SQL tab for CTAS ## What changes were proposed in this pull request? This is a regression in Spark 2.3. In Spark 2.2, we have a fragile UI support for SQL data writing commands. We

[1/2] spark git commit: Preparing Spark release v2.3.0-rc3

2018-02-12 Thread sameerag
Repository: spark Updated Branches: refs/heads/branch-2.3 d31c4ae7b -> 70be6038d Preparing Spark release v2.3.0-rc3 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/89f6fcba Tree:

[spark] Git Push Summary

2018-02-12 Thread sameerag
Repository: spark Updated Tags: refs/tags/v2.3.0-rc3 [created] 89f6fcbaf - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

spark git commit: [SPARK-23388][SQL] Support for Parquet Binary DecimalType in VectorizedColumnReader

2018-02-12 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 4a4dd4f36 -> 5bb11411a [SPARK-23388][SQL] Support for Parquet Binary DecimalType in VectorizedColumnReader ## What changes were proposed in this pull request? Re-add support for parquet binary DecimalType in VectorizedColumnReader ##

spark git commit: [SPARK-23388][SQL] Support for Parquet Binary DecimalType in VectorizedColumnReader

2018-02-12 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.3 70be6038d -> 4e138207e [SPARK-23388][SQL] Support for Parquet Binary DecimalType in VectorizedColumnReader ## What changes were proposed in this pull request? Re-add support for parquet binary DecimalType in VectorizedColumnReader

[2/2] spark git commit: Preparing development version 2.3.1-SNAPSHOT

2018-02-12 Thread sameerag
Preparing development version 2.3.1-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/70be6038 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/70be6038 Diff:

svn commit: r24954 - in /dev/spark/2.4.0-SNAPSHOT-2018_02_12_08_02-4a4dd4f-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-02-12 Thread pwendell
Author: pwendell Date: Mon Feb 12 16:21:15 2018 New Revision: 24954 Log: Apache Spark 2.4.0-SNAPSHOT-2018_02_12_08_02-4a4dd4f docs [This commit notification would consist of 1444 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

svn commit: r24961 - in /dev/spark/2.3.1-SNAPSHOT-2018_02_12_10_01-d31c4ae-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-02-12 Thread pwendell
Author: pwendell Date: Mon Feb 12 18:15:26 2018 New Revision: 24961 Log: Apache Spark 2.3.1-SNAPSHOT-2018_02_12_10_01-d31c4ae docs [This commit notification would consist of 1443 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-23378][SQL] move setCurrentDatabase from HiveExternalCatalog to HiveClientImpl

2018-02-12 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 0c66fe4f2 -> fba01b9a6 [SPARK-23378][SQL] move setCurrentDatabase from HiveExternalCatalog to HiveClientImpl ## What changes were proposed in this pull request? This removes the special case that `alterPartitions` call from

svn commit: r24978 - in /dev/spark/2.3.1-SNAPSHOT-2018_02_12_14_01-9632c46-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-02-12 Thread pwendell
Author: pwendell Date: Mon Feb 12 22:15:13 2018 New Revision: 24978 Log: Apache Spark 2.3.1-SNAPSHOT-2018_02_12_14_01-9632c46 docs [This commit notification would consist of 1443 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-12 Thread lixiao
Repository: spark Updated Branches: refs/heads/master fba01b9a6 -> 6cb59708c [SPARK-23313][DOC] Add a migration guide for ORC ## What changes were proposed in this pull request? This PR adds a migration guide documentation for ORC.

spark git commit: [SPARK-22002][SQL][FOLLOWUP][TEST] Add a test to check if the original schema doesn't have metadata.

2018-02-12 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.3 4e138207e -> 9632c461e [SPARK-22002][SQL][FOLLOWUP][TEST] Add a test to check if the original schema doesn't have metadata. ## What changes were proposed in this pull request? This is a follow-up pr of #19231 which modified the

svn commit: r24968 - in /dev/spark/2.4.0-SNAPSHOT-2018_02_12_12_01-5bb1141-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-02-12 Thread pwendell
Author: pwendell Date: Mon Feb 12 20:15:07 2018 New Revision: 24968 Log: Apache Spark 2.4.0-SNAPSHOT-2018_02_12_12_01-5bb1141 docs [This commit notification would consist of 1444 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-22002][SQL][FOLLOWUP][TEST] Add a test to check if the original schema doesn't have metadata.

2018-02-12 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 5bb11411a -> 0c66fe4f2 [SPARK-22002][SQL][FOLLOWUP][TEST] Add a test to check if the original schema doesn't have metadata. ## What changes were proposed in this pull request? This is a follow-up pr of #19231 which modified the behavior

spark git commit: [SPARK-23323][SQL] Support commit coordinator for DataSourceV2 writes

2018-02-12 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 4104b68e9 -> c1bcef876 [SPARK-23323][SQL] Support commit coordinator for DataSourceV2 writes ## What changes were proposed in this pull request? DataSourceV2 batch writes should use the output commit coordinator if it is required by the

svn commit: r24992 - in /dev/spark/2.4.0-SNAPSHOT-2018_02_12_20_01-c1bcef8-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-02-12 Thread pwendell
Author: pwendell Date: Tue Feb 13 04:15:30 2018 New Revision: 24992 Log: Apache Spark 2.4.0-SNAPSHOT-2018_02_12_20_01-c1bcef8 docs [This commit notification would consist of 1444 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-12 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.3 9632c461e -> 2b80571e2 [SPARK-23313][DOC] Add a migration guide for ORC ## What changes were proposed in this pull request? This PR adds a migration guide documentation for ORC.

svn commit: r24988 - in /dev/spark/2.3.1-SNAPSHOT-2018_02_12_18_01-43f5e40-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-02-12 Thread pwendell
Author: pwendell Date: Tue Feb 13 02:15:16 2018 New Revision: 24988 Log: Apache Spark 2.3.1-SNAPSHOT-2018_02_12_18_01-43f5e40 docs [This commit notification would consist of 1443 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-23352][PYTHON][BRANCH-2.3] Explicitly specify supported types in Pandas UDFs

2018-02-12 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.3 befb22de8 -> 43f5e4067 [SPARK-23352][PYTHON][BRANCH-2.3] Explicitly specify supported types in Pandas UDFs ## What changes were proposed in this pull request? This PR backports https://github.com/apache/spark/pull/20531: It

svn commit: r24987 - /dev/spark/v2.3.0-rc3-bin/

2018-02-12 Thread sameerag
Author: sameerag Date: Tue Feb 13 00:54:47 2018 New Revision: 24987 Log: Apache Spark v2.3.0-rc3 Added: dev/spark/v2.3.0-rc3-bin/ dev/spark/v2.3.0-rc3-bin/SparkR_2.3.0.tar.gz (with props) dev/spark/v2.3.0-rc3-bin/SparkR_2.3.0.tar.gz.asc

spark git commit: [SPARK-23230][SQL] When hive.default.fileformat is other kinds of file types, create textfile table cause a serde error

2018-02-12 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 6cb59708c -> 4104b68e9 [SPARK-23230][SQL] When hive.default.fileformat is other kinds of file types, create textfile table cause a serde error When hive.default.fileformat is other kinds of file types, create textfile table cause a serde

spark git commit: [SPARK-23230][SQL] When hive.default.fileformat is other kinds of file types, create textfile table cause a serde error

2018-02-12 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.3 2b80571e2 -> befb22de8 [SPARK-23230][SQL] When hive.default.fileformat is other kinds of file types, create textfile table cause a serde error When hive.default.fileformat is other kinds of file types, create textfile table cause a

svn commit: r24942 - in /dev/spark/2.4.0-SNAPSHOT-2018_02_12_00_01-6efd5d1-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-02-12 Thread pwendell
Author: pwendell Date: Mon Feb 12 08:16:24 2018 New Revision: 24942 Log: Apache Spark 2.4.0-SNAPSHOT-2018_02_12_00_01-6efd5d1 docs [This commit notification would consist of 1444 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-23379][SQL] skip when setting the same current database in HiveClientImpl

2018-02-12 Thread lixiao
Repository: spark Updated Branches: refs/heads/master c1bcef876 -> ed4e78bd6 [SPARK-23379][SQL] skip when setting the same current database in HiveClientImpl ## What changes were proposed in this pull request? If the target database name is as same as the current database, we should be able

spark git commit: [SPARK-23303][SQL] improve the explain result for data source v2 relations

2018-02-12 Thread lixiao
Repository: spark Updated Branches: refs/heads/master ed4e78bd6 -> f17b936f0 [SPARK-23303][SQL] improve the explain result for data source v2 relations ## What changes were proposed in this pull request? The current explain result for data source v2 relation is unreadable: ``` == Parsed

svn commit: r24996 - in /dev/spark/v2.3.0-rc3-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _site/api/java/org/apache/spark

2018-02-12 Thread sameerag
Author: sameerag Date: Tue Feb 13 05:31:05 2018 New Revision: 24996 Log: Apache Spark v2.3.0-rc3 docs [This commit notification would consist of 1446 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-20090][FOLLOW-UP] Revert the deprecation of `names` in PySpark

2018-02-12 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master f17b936f0 -> 407f67249 [SPARK-20090][FOLLOW-UP] Revert the deprecation of `names` in PySpark ## What changes were proposed in this pull request? Deprecating the field `name` in PySpark is not expected. This PR is to revert the change. ##

spark git commit: [SPARK-20090][FOLLOW-UP] Revert the deprecation of `names` in PySpark

2018-02-12 Thread gurwls223
Repository: spark Updated Branches: refs/heads/branch-2.3 43f5e4067 -> 3737c3d32 [SPARK-20090][FOLLOW-UP] Revert the deprecation of `names` in PySpark ## What changes were proposed in this pull request? Deprecating the field `name` in PySpark is not expected. This PR is to revert the change.