spark git commit: [SPARK-23384][WEB-UI] When it has no incomplete(completed) applications found, the last updated time is not formatted and client local time zone is not show in history server web ui.

2018-02-13 Thread srowen
Repository: spark Updated Branches: refs/heads/master 9dae71516 -> 300c40f50 [SPARK-23384][WEB-UI] When it has no incomplete(completed) applications found, the last updated time is not formatted and client local time zone is not show in history server web ui. ## What changes were proposed

spark git commit: [SPARK-23318][ML] FP-growth: WARN FPGrowth: Input data is not cached

2018-02-13 Thread srowen
Repository: spark Updated Branches: refs/heads/master 407f67249 -> 9dae71516 [SPARK-23318][ML] FP-growth: WARN FPGrowth: Input data is not cached ## What changes were proposed in this pull request? Cache the RDD of items in ml.FPGrowth before passing it to mllib.FPGrowth. Cache only when

spark git commit: [SPARK-23384][WEB-UI] When it has no incomplete(completed) applications found, the last updated time is not formatted and client local time zone is not show in history server web ui.

2018-02-13 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.3 3737c3d32 -> 1c81c0c62 [SPARK-23384][WEB-UI] When it has no incomplete(completed) applications found, the last updated time is not formatted and client local time zone is not show in history server web ui. ## What changes were

svn commit: r25026 - in /dev/spark/2.3.1-SNAPSHOT-2018_02_13_02_01-3737c3d-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-02-13 Thread pwendell
Author: pwendell Date: Tue Feb 13 10:16:19 2018 New Revision: 25026 Log: Apache Spark 2.3.1-SNAPSHOT-2018_02_13_02_01-3737c3d docs [This commit notification would consist of 1443 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

svn commit: r24999 - in /dev/spark/2.4.0-SNAPSHOT-2018_02_13_00_01-407f672-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-02-13 Thread pwendell
Author: pwendell Date: Tue Feb 13 08:16:17 2018 New Revision: 24999 Log: Apache Spark 2.4.0-SNAPSHOT-2018_02_13_00_01-407f672 docs [This commit notification would consist of 1444 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-23053][CORE] taskBinarySerialization and task partitions calculate in DagScheduler.submitMissingTasks should keep the same RDD checkpoint status

2018-02-13 Thread irashid
Repository: spark Updated Branches: refs/heads/branch-2.2 14b5dbfa9 -> 73263b215 [SPARK-23053][CORE] taskBinarySerialization and task partitions calculate in DagScheduler.submitMissingTasks should keep the same RDD checkpoint status ## What changes were proposed in this pull request? When

spark git commit: [SPARK-23053][CORE] taskBinarySerialization and task partitions calculate in DagScheduler.submitMissingTasks should keep the same RDD checkpoint status

2018-02-13 Thread irashid
Repository: spark Updated Branches: refs/heads/branch-2.3 1c81c0c62 -> dbb1b399b [SPARK-23053][CORE] taskBinarySerialization and task partitions calculate in DagScheduler.submitMissingTasks should keep the same RDD checkpoint status ## What changes were proposed in this pull request? When

spark git commit: [SPARK-20659][CORE] Removing sc.getExecutorStorageStatus and making StorageStatus private

2018-02-13 Thread vanzin
Repository: spark Updated Branches: refs/heads/master 300c40f50 -> 116c581d2 [SPARK-20659][CORE] Removing sc.getExecutorStorageStatus and making StorageStatus private ## What changes were proposed in this pull request? In this PR StorageStatus is made to private and simplified a bit

spark git commit: [SPARK-23189][CORE][WEB UI] Reflect stage level blacklisting on executor tab

2018-02-13 Thread irashid
Repository: spark Updated Branches: refs/heads/master 116c581d2 -> d6e1958a2 [SPARK-23189][CORE][WEB UI] Reflect stage level blacklisting on executor tab ## What changes were proposed in this pull request? The purpose of this PR to reflect the stage level blacklisting on the executor tab

svn commit: r25033 - in /dev/spark/2.4.0-SNAPSHOT-2018_02_13_08_02-091a000-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-02-13 Thread pwendell
Author: pwendell Date: Tue Feb 13 16:21:03 2018 New Revision: 25033 Log: Apache Spark 2.4.0-SNAPSHOT-2018_02_13_08_02-091a000 docs [This commit notification would consist of 1443 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

svn commit: r25031 - in /dev/spark/2.3.1-SNAPSHOT-2018_02_13_06_01-1c81c0c-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-02-13 Thread pwendell
Author: pwendell Date: Tue Feb 13 14:15:07 2018 New Revision: 25031 Log: Apache Spark 2.3.1-SNAPSHOT-2018_02_13_06_01-1c81c0c docs [This commit notification would consist of 1443 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-23053][CORE] taskBinarySerialization and task partitions calculate in DagScheduler.submitMissingTasks should keep the same RDD checkpoint status

2018-02-13 Thread irashid
Repository: spark Updated Branches: refs/heads/master d6e1958a2 -> 091a000d2 [SPARK-23053][CORE] taskBinarySerialization and task partitions calculate in DagScheduler.submitMissingTasks should keep the same RDD checkpoint status ## What changes were proposed in this pull request? When we

spark git commit: [SPARK-23382][WEB-UI] Spark Streaming ui about the contents of the for need to have hidden and show features, when the table records very much.

2018-02-13 Thread srowen
Repository: spark Updated Branches: refs/heads/master 091a000d2 -> bd2473172 [SPARK-23382][WEB-UI] Spark Streaming ui about the contents of the for need to have hidden and show features, when the table records very much. ## What changes were proposed in this pull request? Spark Streaming ui

spark git commit: [SPARK-23392][TEST] Add some test cases for images feature

2018-02-13 Thread srowen
Repository: spark Updated Branches: refs/heads/master bd2473172 -> 263531466 [SPARK-23392][TEST] Add some test cases for images feature ## What changes were proposed in this pull request? Add some test cases for images feature ## How was this patch tested? Add some test cases in

spark git commit: [SPARK-23217][ML] Add cosine distance measure to ClusteringEvaluator

2018-02-13 Thread srowen
Repository: spark Updated Branches: refs/heads/master 05d051293 -> 4e0fb010c [SPARK-23217][ML] Add cosine distance measure to ClusteringEvaluator ## What changes were proposed in this pull request? The PR provided an implementation of ClusteringEvaluator using the cosine distance measure.

spark git commit: [SPARK-23316][SQL] AnalysisException after max iteration reached for IN query

2018-02-13 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 263531466 -> 05d051293 [SPARK-23316][SQL] AnalysisException after max iteration reached for IN query ## What changes were proposed in this pull request? Added flag ignoreNullability to DataType.equalsStructurally. The previous semantic is

spark git commit: [SPARK-23316][SQL] AnalysisException after max iteration reached for IN query

2018-02-13 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.3 dbb1b399b -> ab01ba718 [SPARK-23316][SQL] AnalysisException after max iteration reached for IN query ## What changes were proposed in this pull request? Added flag ignoreNullability to DataType.equalsStructurally. The previous semantic

spark git commit: [SPARK-23400][SQL] Add a constructors for ScalaUDF

2018-02-13 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master d58fe2883 -> 2ee76c22b [SPARK-23400][SQL] Add a constructors for ScalaUDF ## What changes were proposed in this pull request? In this upcoming 2.3 release, we changed the interface of `ScalaUDF`. Unfortunately, some Spark packages (e.g.,

spark git commit: [SPARK-23400][SQL] Add a constructors for ScalaUDF

2018-02-13 Thread zsxwing
Repository: spark Updated Branches: refs/heads/branch-2.3 320ffb130 -> 4f6a457d4 [SPARK-23400][SQL] Add a constructors for ScalaUDF ## What changes were proposed in this pull request? In this upcoming 2.3 release, we changed the interface of `ScalaUDF`. Unfortunately, some Spark packages

spark git commit: [SPARK-23154][ML][DOC] Document backwards compatibility guarantees for ML persistence

2018-02-13 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master 4e0fb010c -> d58fe2883 [SPARK-23154][ML][DOC] Document backwards compatibility guarantees for ML persistence ## What changes were proposed in this pull request? Added documentation about what MLlib guarantees in terms of loading ML

spark git commit: [SPARK-23154][ML][DOC] Document backwards compatibility guarantees for ML persistence

2018-02-13 Thread jkbradley
Repository: spark Updated Branches: refs/heads/branch-2.3 ab01ba718 -> 320ffb130 [SPARK-23154][ML][DOC] Document backwards compatibility guarantees for ML persistence ## What changes were proposed in this pull request? Added documentation about what MLlib guarantees in terms of loading ML

svn commit: r25046 - in /dev/spark/2.4.0-SNAPSHOT-2018_02_13_12_02-2ee76c2-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-02-13 Thread pwendell
Author: pwendell Date: Tue Feb 13 20:23:20 2018 New Revision: 25046 Log: Apache Spark 2.4.0-SNAPSHOT-2018_02_13_12_02-2ee76c2 docs [This commit notification would consist of 1443 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

svn commit: r25047 - in /dev/spark/2.3.1-SNAPSHOT-2018_02_13_14_01-4f6a457-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-02-13 Thread pwendell
Author: pwendell Date: Tue Feb 13 22:15:18 2018 New Revision: 25047 Log: Apache Spark 2.3.1-SNAPSHOT-2018_02_13_14_01-4f6a457 docs [This commit notification would consist of 1443 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-23235][CORE] Add executor Threaddump to api

2018-02-13 Thread irashid
Repository: spark Updated Branches: refs/heads/master 2ee76c22b -> a5a4b8350 [SPARK-23235][CORE] Add executor Threaddump to api ## What changes were proposed in this pull request? Extending api with the executor thread dump data. For this new REST URL is introduced: - GET

spark git commit: [SPARK-23399][SQL] Register a task completion listener first for OrcColumnarBatchReader

2018-02-13 Thread wenchen
Repository: spark Updated Branches: refs/heads/master d6f5e172b -> 357babde5 [SPARK-23399][SQL] Register a task completion listener first for OrcColumnarBatchReader ## What changes were proposed in this pull request? This PR aims to resolve an open file leakage issue reported at

spark git commit: [SPARK-23399][SQL] Register a task completion listener first for OrcColumnarBatchReader

2018-02-13 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.3 4f6a457d4 -> bb26bdb55 [SPARK-23399][SQL] Register a task completion listener first for OrcColumnarBatchReader This PR aims to resolve an open file leakage issue reported at

svn commit: r25051 - in /dev/spark/2.4.0-SNAPSHOT-2018_02_13_20_01-357babd-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-02-13 Thread pwendell
Author: pwendell Date: Wed Feb 14 04:15:27 2018 New Revision: 25051 Log: Apache Spark 2.4.0-SNAPSHOT-2018_02_13_20_01-357babd docs [This commit notification would consist of 1444 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: Revert "[SPARK-23303][SQL] improve the explain result for data source v2 relations"

2018-02-13 Thread lixiao
Repository: spark Updated Branches: refs/heads/master a5a4b8350 -> d6f5e172b Revert "[SPARK-23303][SQL] improve the explain result for data source v2 relations" This reverts commit f17b936f0ddb7d46d1349bd42f9a64c84c06e48d. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

spark git commit: [SPARK-23230][SQL][BRANCH-2.2] When hive.default.fileformat is other kinds of file types, create textfile table cause a serde error

2018-02-13 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.2 73263b215 -> a95c3e29d [SPARK-23230][SQL][BRANCH-2.2] When hive.default.fileformat is other kinds of file types, create textfile table cause a serde error When hive.default.fileformat is other kinds of file types, create textfile

svn commit: r25052 - in /dev/spark/2.3.1-SNAPSHOT-2018_02_13_22_01-bb26bdb-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-02-13 Thread pwendell
Author: pwendell Date: Wed Feb 14 06:15:41 2018 New Revision: 25052 Log: Apache Spark 2.3.1-SNAPSHOT-2018_02_13_22_01-bb26bdb docs [This commit notification would consist of 1443 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]