spark git commit: [SPARK-15045] [CORE] Remove dead code in TaskMemoryManager.cleanUpAllAllocatedMemory for pageTable

Repository: spark Updated Branches: refs/heads/branch-2.0 0914296cb -> e28d21d3f [SPARK-15045] [CORE] Remove dead code in TaskMemoryManager.cleanUpAllAllocatedMemory for pageTable ## What changes were proposed in this pull request? Removed the DeadCode as suggested. Author: Abhinav Gupta

spark git commit: [SPARK-15045] [CORE] Remove dead code in TaskMemoryManager.cleanUpAllAllocatedMemory for pageTable

Repository: spark Updated Branches: refs/heads/master 1a9b34158 -> 1a5c6fcef [SPARK-15045] [CORE] Remove dead code in TaskMemoryManager.cleanUpAllAllocatedMemory for pageTable ## What changes were proposed in this pull request? Removed the DeadCode as suggested. Author: Abhinav Gupta

spark git commit: [SPARK-15132][MINOR][SQL] Debug log for generated code should be printed with proper indentation

Repository: spark Updated Branches: refs/heads/branch-2.0 2023faf6c -> 0914296cb [SPARK-15132][MINOR][SQL] Debug log for generated code should be printed with proper indentation ## What changes were proposed in this pull request? Similar to #11990, GenerateOrdering and

spark git commit: [SPARK-15132][MINOR][SQL] Debug log for generated code should be printed with proper indentation

Repository: spark Updated Branches: refs/heads/master 428374195 -> 1a9b34158 [SPARK-15132][MINOR][SQL] Debug log for generated code should be printed with proper indentation ## What changes were proposed in this pull request? Similar to #11990, GenerateOrdering and GenerateColumnAccessor

spark git commit: [MINOR] remove dead code

Repository: spark Updated Branches: refs/heads/branch-2.0 e12ec46c6 -> 2023faf6c [MINOR] remove dead code Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/2023faf6 Tree:

spark git commit: [SPARK-15131][SQL] Shutdown StateStore management thread when SparkContext has been shutdown

Repository: spark Updated Branches: refs/heads/master ef55e46c9 -> bde27b89a [SPARK-15131][SQL] Shutdown StateStore management thread when SparkContext has been shutdown ## What changes were proposed in this pull request? Make sure that whenever the StateStoreCoordinator cannot be

spark git commit: [SPARK-14993][SQL] Fix Partition Discovery Inconsistency when Input is a Path to Parquet File

Repository: spark Updated Branches: refs/heads/master 8fb1463d6 -> ef55e46c9 [SPARK-14993][SQL] Fix Partition Discovery Inconsistency when Input is a Path to Parquet File What changes were proposed in this pull request? When we load a dataset, if we set the path to ```/path/a=1```, we

spark git commit: [SPARK-14993][SQL] Fix Partition Discovery Inconsistency when Input is a Path to Parquet File

Repository: spark Updated Branches: refs/heads/branch-2.0 d90359d63 -> 689b0fc81 [SPARK-14993][SQL] Fix Partition Discovery Inconsistency when Input is a Path to Parquet File What changes were proposed in this pull request? When we load a dataset, if we set the path to ```/path/a=1```,

spark git commit: [SPARK-6339][SQL] Supports CREATE TEMPORARY VIEW tableIdentifier AS query

Repository: spark Updated Branches: refs/heads/branch-2.0 fa3c5507f -> d90359d63 [SPARK-6339][SQL] Supports CREATE TEMPORARY VIEW tableIdentifier AS query ## What changes were proposed in this pull request? This PR support new SQL syntax CREATE TEMPORARY VIEW. Like: ``` CREATE TEMPORARY VIEW

spark git commit: [SPARK-6339][SQL] Supports CREATE TEMPORARY VIEW tableIdentifier AS query

Repository: spark Updated Branches: refs/heads/master fa79d346e -> 8fb1463d6 [SPARK-6339][SQL] Supports CREATE TEMPORARY VIEW tableIdentifier AS query ## What changes were proposed in this pull request? This PR support new SQL syntax CREATE TEMPORARY VIEW. Like: ``` CREATE TEMPORARY VIEW

spark git commit: [SPARK-14896][SQL] Deprecate HiveContext in python

Repository: spark Updated Branches: refs/heads/branch-2.0 aca46ecf8 -> fa3c5507f [SPARK-14896][SQL] Deprecate HiveContext in python ## What changes were proposed in this pull request? See title. ## How was this patch tested? PySpark tests. Author: Andrew Or Closes

spark git commit: [SPARK-14896][SQL] Deprecate HiveContext in python

Repository: spark Updated Branches: refs/heads/master b28137764 -> fa79d346e [SPARK-14896][SQL] Deprecate HiveContext in python ## What changes were proposed in this pull request? See title. ## How was this patch tested? PySpark tests. Author: Andrew Or Closes

spark git commit: [MINOR][SQL] Fix typo in DataFrameReader csv documentation

Repository: spark Updated Branches: refs/heads/branch-2.0 701c66729 -> aca46ecf8 [MINOR][SQL] Fix typo in DataFrameReader csv documentation ## What changes were proposed in this pull request? Typo fix ## How was this patch tested? No tests My apologies for the tiny PR, but I stumbled across

spark git commit: [MINOR][SQL] Fix typo in DataFrameReader csv documentation

Repository: spark Updated Branches: refs/heads/master a432a2b86 -> b28137764 [MINOR][SQL] Fix typo in DataFrameReader csv documentation ## What changes were proposed in this pull request? Typo fix ## How was this patch tested? No tests My apologies for the tiny PR, but I stumbled across

spark git commit: [SPARK-15116] In REPL we should create SparkSession first and get SparkContext from it

Repository: spark Updated Branches: refs/heads/master eb019af9a -> a432a2b86 [SPARK-15116] In REPL we should create SparkSession first and get SparkContext from it ## What changes were proposed in this pull request? see https://github.com/apache/spark/pull/12873#discussion_r61993910. The

spark git commit: [SPARK-13001][CORE][MESOS] Prevent getting offers when reached max cores

Repository: spark Updated Branches: refs/heads/branch-2.0 23789e358 -> 1e7d9bfb5 [SPARK-13001][CORE][MESOS] Prevent getting offers when reached max cores Similar to https://github.com/apache/spark/pull/8639 This change rejects offers for 120s when reached `spark.cores.max` in coarse-grained

spark git commit: [SPARK-13001][CORE][MESOS] Prevent getting offers when reached max cores

Repository: spark Updated Branches: refs/heads/master cdce4e62a -> eb019af9a [SPARK-13001][CORE][MESOS] Prevent getting offers when reached max cores Similar to https://github.com/apache/spark/pull/8639 This change rejects offers for 120s when reached `spark.cores.max` in coarse-grained

[3/4] spark git commit: [SPARK-15031][EXAMPLE] Use SparkSession in Scala/Python/Java example.

http://git-wip-us.apache.org/repos/asf/spark/blob/23789e35/examples/src/main/java/org/apache/spark/examples/ml/JavaPolynomialExpansionExample.java -- diff --git

[2/4] spark git commit: [SPARK-15031][EXAMPLE] Use SparkSession in Scala/Python/Java example.

http://git-wip-us.apache.org/repos/asf/spark/blob/23789e35/examples/src/main/python/ml/naive_bayes_example.py -- diff --git a/examples/src/main/python/ml/naive_bayes_example.py b/examples/src/main/python/ml/naive_bayes_example.py

[4/4] spark git commit: [SPARK-15031][EXAMPLE] Use SparkSession in Scala/Python/Java example.

[SPARK-15031][EXAMPLE] Use SparkSession in Scala/Python/Java example. ## What changes were proposed in this pull request? This PR aims to update Scala/Python/Java examples by replacing `SQLContext` with newly added `SparkSession`. - Use **SparkSession Builder Pattern** in 154(Scala 55, Java

[1/4] spark git commit: [SPARK-15031][EXAMPLE] Use SparkSession in Scala/Python/Java example.

Repository: spark Updated Branches: refs/heads/branch-2.0 c0715f33b -> 23789e358 http://git-wip-us.apache.org/repos/asf/spark/blob/23789e35/examples/src/main/scala/org/apache/spark/examples/ml/NGramExample.scala -- diff --git

[4/4] spark git commit: [SPARK-15031][EXAMPLE] Use SparkSession in Scala/Python/Java example.

[SPARK-15031][EXAMPLE] Use SparkSession in Scala/Python/Java example. ## What changes were proposed in this pull request? This PR aims to update Scala/Python/Java examples by replacing `SQLContext` with newly added `SparkSession`. - Use **SparkSession Builder Pattern** in 154(Scala 55, Java

[2/4] spark git commit: [SPARK-15031][EXAMPLE] Use SparkSession in Scala/Python/Java example.

http://git-wip-us.apache.org/repos/asf/spark/blob/cdce4e62/examples/src/main/python/ml/naive_bayes_example.py -- diff --git a/examples/src/main/python/ml/naive_bayes_example.py b/examples/src/main/python/ml/naive_bayes_example.py

[3/4] spark git commit: [SPARK-15031][EXAMPLE] Use SparkSession in Scala/Python/Java example.

http://git-wip-us.apache.org/repos/asf/spark/blob/cdce4e62/examples/src/main/java/org/apache/spark/examples/ml/JavaPolynomialExpansionExample.java -- diff --git

[1/4] spark git commit: [SPARK-15031][EXAMPLE] Use SparkSession in Scala/Python/Java example.

Repository: spark Updated Branches: refs/heads/master cf2e9da61 -> cdce4e62a http://git-wip-us.apache.org/repos/asf/spark/blob/cdce4e62/examples/src/main/scala/org/apache/spark/examples/ml/NGramExample.scala -- diff --git

spark git commit: [SPARK-12299][CORE] Remove history serving functionality from Master

Repository: spark Updated Branches: refs/heads/master 0c00391f7 -> cf2e9da61 [SPARK-12299][CORE] Remove history serving functionality from Master Remove history server functionality from standalone Master. Previously, the Master process rebuilt a SparkUI once the application was completed

spark git commit: [SPARK-12299][CORE] Remove history serving functionality from Master

Repository: spark Updated Branches: refs/heads/branch-2.0 eeb18f6d7 -> c0715f33b [SPARK-12299][CORE] Remove history serving functionality from Master Remove history server functionality from standalone Master. Previously, the Master process rebuilt a SparkUI once the application was

spark git commit: [SPARK-15121] Improve logging of external shuffle handler

Repository: spark Updated Branches: refs/heads/branch-2.0 45862f6c9 -> eeb18f6d7 [SPARK-15121] Improve logging of external shuffle handler ## What changes were proposed in this pull request? Add more informative logging in the external shuffle service to aid in debugging who is connecting

spark git commit: [SPARK-15121] Improve logging of external shuffle handler

Repository: spark Updated Branches: refs/heads/master 6ae9fc00e -> 0c00391f7 [SPARK-15121] Improve logging of external shuffle handler ## What changes were proposed in this pull request? Add more informative logging in the external shuffle service to aid in debugging who is connecting to

spark git commit: [SPARK-15126][SQL] RuntimeConfig.set should return Unit

Repository: spark Updated Branches: refs/heads/branch-2.0 e868a15a7 -> 45862f6c9 [SPARK-15126][SQL] RuntimeConfig.set should return Unit ## What changes were proposed in this pull request? Currently we return RuntimeConfig itself to facilitate chaining. However, it makes the output in

spark git commit: [SPARK-15126][SQL] RuntimeConfig.set should return Unit

Repository: spark Updated Branches: refs/heads/master 0fd3a4748 -> 6ae9fc00e [SPARK-15126][SQL] RuntimeConfig.set should return Unit ## What changes were proposed in this pull request? Currently we return RuntimeConfig itself to facilitate chaining. However, it makes the output in

svn commit: r1742332 - /spark/news/_posts/2016-04-17-submit-summit-agenda-posted.md

Author: srowen Date: Wed May 4 20:30:20 2016 New Revision: 1742332 URL: http://svn.apache.org/viewvc?rev=1742332=rev Log: Remove apparently duplicate Spark news entry Removed: spark/news/_posts/2016-04-17-submit-summit-agenda-posted.md

svn commit: r1742331 - in /spark: downloads.md site/downloads.html

Author: srowen Date: Wed May 4 20:27:40 2016 New Revision: 1742331 URL: http://svn.apache.org/viewvc?rev=1742331=rev Log: SPARK-15128: fix link to Spark artifacts on Maven Central Modified: spark/downloads.md spark/site/downloads.html Modified: spark/downloads.md URL:

spark git commit: [SPARK-15103][SQL] Refactored FileCatalog class to allow StreamFileCatalog to infer partitioning

Repository: spark Updated Branches: refs/heads/branch-2.0 c59615432 -> e868a15a7 [SPARK-15103][SQL] Refactored FileCatalog class to allow StreamFileCatalog to infer partitioning ## What changes were proposed in this pull request? File Stream Sink writes the list of written files in a

spark git commit: [SPARK-15103][SQL] Refactored FileCatalog class to allow StreamFileCatalog to infer partitioning

Repository: spark Updated Branches: refs/heads/master 6274a520f -> 0fd3a4748 [SPARK-15103][SQL] Refactored FileCatalog class to allow StreamFileCatalog to infer partitioning ## What changes were proposed in this pull request? File Stream Sink writes the list of written files in a metadata

[1/2] spark git commit: [SPARK-15115][SQL] Reorganize whole stage codegen benchmark suites

Repository: spark Updated Branches: refs/heads/branch-2.0 54d90bd3e -> c59615432 http://git-wip-us.apache.org/repos/asf/spark/blob/c5961543/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/SortBenchmark.scala

[2/2] spark git commit: [SPARK-15115][SQL] Reorganize whole stage codegen benchmark suites

[SPARK-15115][SQL] Reorganize whole stage codegen benchmark suites ## What changes were proposed in this pull request? We currently have a single suite that is very large, making it difficult to maintain and play with specific primitives. This patch reorganizes the file by creating multiple

spark git commit: [MINOR] Add python3 compatibility in python examples

Repository: spark Updated Branches: refs/heads/branch-2.0 21e368de4 -> 54d90bd3e [MINOR] Add python3 compatibility in python examples ## What changes were proposed in this pull request? Add python3 compatibility in python examples ## How was this patch tested? manual tests Author: Zheng

[2/2] spark git commit: [SPARK-15115][SQL] Reorganize whole stage codegen benchmark suites

[SPARK-15115][SQL] Reorganize whole stage codegen benchmark suites ## What changes were proposed in this pull request? We currently have a single suite that is very large, making it difficult to maintain and play with specific primitives. This patch reorganizes the file by creating multiple

spark git commit: [MINOR] Add python3 compatibility in python examples

Repository: spark Updated Branches: refs/heads/master b85d21fb9 -> 4530250f5 [MINOR] Add python3 compatibility in python examples ## What changes were proposed in this pull request? Add python3 compatibility in python examples ## How was this patch tested? manual tests Author: Zheng RuiFeng

spark git commit: [SPARK-14951] [SQL] Support subexpression elimination in TungstenAggregate

Repository: spark Updated Branches: refs/heads/branch-2.0 b3c491217 -> 21e368de4 [SPARK-14951] [SQL] Support subexpression elimination in TungstenAggregate ## What changes were proposed in this pull request? We can support subexpression elimination in TungstenAggregate by using current

spark git commit: [SPARK-14951] [SQL] Support subexpression elimination in TungstenAggregate

Repository: spark Updated Branches: refs/heads/master d864c55cf -> b85d21fb9 [SPARK-14951] [SQL] Support subexpression elimination in TungstenAggregate ## What changes were proposed in this pull request? We can support subexpression elimination in TungstenAggregate by using current

spark git commit: [SPARK-15109][SQL] Accept Dataset[_] in joins

Repository: spark Updated Branches: refs/heads/branch-2.0 343c28504 -> b3c491217 [SPARK-15109][SQL] Accept Dataset[_] in joins ## What changes were proposed in this pull request? This patch changes the join API in Dataset so they can accept any Dataset, rather than just DataFrames. ## How

spark git commit: [SPARK-15109][SQL] Accept Dataset[_] in joins

Repository: spark Updated Branches: refs/heads/master e597ec6f1 -> d864c55cf [SPARK-15109][SQL] Accept Dataset[_] in joins ## What changes were proposed in this pull request? This patch changes the join API in Dataset so they can accept any Dataset, rather than just DataFrames. ## How was

spark git commit: [SPARK-15022][SPARK-15023][SQL][STREAMING] Add support for testing against the `ProcessingTime(intervalMS > 0)` trigger and `ManualClock`

Repository: spark Updated Branches: refs/heads/branch-2.0 ae79032dc -> 343c28504 [SPARK-15022][SPARK-15023][SQL][STREAMING] Add support for testing against the `ProcessingTime(intervalMS > 0)` trigger and `ManualClock` ## What changes were proposed in this pull request? Currently in

spark git commit: [SPARK-15022][SPARK-15023][SQL][STREAMING] Add support for testing against the `ProcessingTime(intervalMS > 0)` trigger and `ManualClock`

Repository: spark Updated Branches: refs/heads/master a45647746 -> e597ec6f1 [SPARK-15022][SPARK-15023][SQL][STREAMING] Add support for testing against the `ProcessingTime(intervalMS > 0)` trigger and `ManualClock` ## What changes were proposed in this pull request? Currently in

spark git commit: [SPARK-4224][CORE][YARN] Support group acls

Repository: spark Updated Branches: refs/heads/master abecbcd5e -> a45647746 [SPARK-4224][CORE][YARN] Support group acls ## What changes were proposed in this pull request? Currently only a list of users can be specified for view and modify acls. This change enables a group of

spark git commit: [SPARK-14844][ML] Add setFeaturesCol and setPredictionCol to KMeansM…

Repository: spark Updated Branches: refs/heads/master f152fae30 -> abecbcd5e [SPARK-14844][ML] Add setFeaturesCol and setPredictionCol to KMeansM… ## What changes were proposed in this pull request? Introduction of setFeaturesCol and setPredictionCol methods to KMeansModel in ML library.

spark git commit: [SPARK-14844][ML] Add setFeaturesCol and setPredictionCol to KMeansM…

Repository: spark Updated Branches: refs/heads/branch-2.0 1fba7595f -> dcbe85ff2 [SPARK-14844][ML] Add setFeaturesCol and setPredictionCol to KMeansM… ## What changes were proposed in this pull request? Introduction of setFeaturesCol and setPredictionCol methods to KMeansModel in ML

spark git commit: [SPARK-14127][SQL] Native "DESC [EXTENDED | FORMATTED] " DDL command

Repository: spark Updated Branches: refs/heads/branch-2.0 b99f715e8 -> 1fba7595f [SPARK-14127][SQL] Native "DESC [EXTENDED | FORMATTED] " DDL command ## What changes were proposed in this pull request? This PR implements native `DESC [EXTENDED | FORMATTED] ` DDL command. Sample output: ```

spark git commit: [SPARK-14127][SQL] Native "DESC [EXTENDED | FORMATTED] " DDL command

Repository: spark Updated Branches: refs/heads/master 6c12e801e -> f152fae30 [SPARK-14127][SQL] Native "DESC [EXTENDED | FORMATTED] " DDL command ## What changes were proposed in this pull request? This PR implements native `DESC [EXTENDED | FORMATTED] ` DDL command. Sample output: ```

spark git commit: [SPARK-15029] improve error message for Generate

Repository: spark Updated Branches: refs/heads/branch-2.0 64ad9ba27 -> b99f715e8 [SPARK-15029] improve error message for Generate ## What changes were proposed in this pull request? This PR improve the error message for `Generate` in 3 cases: 1. generator is nested in expressions, e.g.

spark git commit: [SPARK-15029] improve error message for Generate

Repository: spark Updated Branches: refs/heads/master bc3760d40 -> 6c12e801e [SPARK-15029] improve error message for Generate ## What changes were proposed in this pull request? This PR improve the error message for `Generate` in 3 cases: 1. generator is nested in expressions, e.g. `SELECT

spark git commit: [SPARK-14237][SQL] De-duplicate partition value appending logic in various buildReader() implementations

Repository: spark Updated Branches: refs/heads/branch-2.0 2ff667606 -> 64ad9ba27 [SPARK-14237][SQL] De-duplicate partition value appending logic in various buildReader() implementations ## What changes were proposed in this pull request? Currently, various `FileFormat` data sources share

spark git commit: [SPARK-14237][SQL] De-duplicate partition value appending logic in various buildReader() implementations

Repository: spark Updated Branches: refs/heads/master 695f0e919 -> bc3760d40 [SPARK-14237][SQL] De-duplicate partition value appending logic in various buildReader() implementations ## What changes were proposed in this pull request? Currently, various `FileFormat` data sources share