spark git commit: [SPARK-23129][CORE] Make deserializeStream of DiskMapIterator init lazily

2018-01-24 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.3 a857ad566 -> 012695256 [SPARK-23129][CORE] Make deserializeStream of DiskMapIterator init lazily ## What changes were proposed in this pull request? Currently,the deserializeStream in ExternalAppendOnlyMap#DiskMapIterator init when Di

spark git commit: [SPARK-23129][CORE] Make deserializeStream of DiskMapIterator init lazily

2018-01-24 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 6f0ba8472 -> 45b4bbfdd [SPARK-23129][CORE] Make deserializeStream of DiskMapIterator init lazily ## What changes were proposed in this pull request? Currently,the deserializeStream in ExternalAppendOnlyMap#DiskMapIterator init when DiskMa

svn commit: r24425 - in /dev/spark/2.4.0-SNAPSHOT-2018_01_24_16_01-6f0ba84-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-01-24 Thread pwendell
Author: pwendell Date: Thu Jan 25 00:15:03 2018 New Revision: 24425 Log: Apache Spark 2.4.0-SNAPSHOT-2018_01_24_16_01-6f0ba84 docs [This commit notification would consist of 1443 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

svn commit: r24423 - in /dev/spark/2.3.1-SNAPSHOT-2018_01_24_14_01-a857ad5-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-01-24 Thread pwendell
Author: pwendell Date: Wed Jan 24 22:15:33 2018 New Revision: 24423 Log: Apache Spark 2.3.1-SNAPSHOT-2018_01_24_14_01-a857ad5 docs [This commit notification would consist of 1442 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

spark git commit: [MINOR][SQL] add new unit test to LimitPushdown

2018-01-24 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.3 500c94434 -> a857ad566 [MINOR][SQL] add new unit test to LimitPushdown ## What changes were proposed in this pull request? This PR is repaired as follows 1、update y -> x in "left outer join" test case ,maybe is mistake. 2、add a new

spark git commit: [MINOR][SQL] add new unit test to LimitPushdown

2018-01-24 Thread lixiao
Repository: spark Updated Branches: refs/heads/master bc9641d90 -> 6f0ba8472 [MINOR][SQL] add new unit test to LimitPushdown ## What changes were proposed in this pull request? This PR is repaired as follows 1、update y -> x in "left outer join" test case ,maybe is mistake. 2、add a new tes

spark git commit: [SPARK-23198][SS][TEST] Fix KafkaContinuousSourceStressForDontFailOnDataLossSuite to test ContinuousExecution

2018-01-24 Thread zsxwing
Repository: spark Updated Branches: refs/heads/branch-2.3 30272c668 -> 500c94434 [SPARK-23198][SS][TEST] Fix KafkaContinuousSourceStressForDontFailOnDataLossSuite to test ContinuousExecution ## What changes were proposed in this pull request? Currently, `KafkaContinuousSourceStressForDontFa

spark git commit: [SPARK-23198][SS][TEST] Fix KafkaContinuousSourceStressForDontFailOnDataLossSuite to test ContinuousExecution

2018-01-24 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master 0e178e152 -> bc9641d90 [SPARK-23198][SS][TEST] Fix KafkaContinuousSourceStressForDontFailOnDataLossSuite to test ContinuousExecution ## What changes were proposed in this pull request? Currently, `KafkaContinuousSourceStressForDontFailOn

svn commit: r24419 - in /dev/spark/2.4.0-SNAPSHOT-2018_01_24_12_01-0e178e1-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-01-24 Thread pwendell
Author: pwendell Date: Wed Jan 24 20:15:14 2018 New Revision: 24419 Log: Apache Spark 2.4.0-SNAPSHOT-2018_01_24_12_01-0e178e1 docs [This commit notification would consist of 1443 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

spark git commit: [SPARK-22297][CORE TESTS] Flaky test: BlockManagerSuite "Shuffle registration timeout and maxAttempts conf"

2018-01-24 Thread vanzin
Repository: spark Updated Branches: refs/heads/master 840dea64a -> 0e178e152 [SPARK-22297][CORE TESTS] Flaky test: BlockManagerSuite "Shuffle registration timeout and maxAttempts conf" ## What changes were proposed in this pull request? [Ticket](https://issues.apache.org/jira/browse/SPARK-22

svn commit: r24416 - in /dev/spark/2.3.1-SNAPSHOT-2018_01_24_10_01-2221a30-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-01-24 Thread pwendell
Author: pwendell Date: Wed Jan 24 18:15:36 2018 New Revision: 24416 Log: Apache Spark 2.3.1-SNAPSHOT-2018_01_24_10_01-2221a30 docs [This commit notification would consist of 1442 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

spark git commit: [SPARK-23152][ML] - Correctly guard against empty datasets

2018-01-24 Thread srowen
Repository: spark Updated Branches: refs/heads/master bbb87b350 -> 840dea64a [SPARK-23152][ML] - Correctly guard against empty datasets ## What changes were proposed in this pull request? Correctly guard against empty datasets in `org.apache.spark.ml.classification.Classifier` ## How was th

spark git commit: [SPARK-22837][SQL] Session timeout checker does not work in SessionManager.

2018-01-24 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.3 2221a3035 -> 30272c668 [SPARK-22837][SQL] Session timeout checker does not work in SessionManager. ## What changes were proposed in this pull request? Currently we do not call the `super.init(hiveConf)` in `SparkSQLSessionManager.init

spark git commit: [SPARK-22837][SQL] Session timeout checker does not work in SessionManager.

2018-01-24 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 8c273b416 -> bbb87b350 [SPARK-22837][SQL] Session timeout checker does not work in SessionManager. ## What changes were proposed in this pull request? Currently we do not call the `super.init(hiveConf)` in `SparkSQLSessionManager.init`. S

spark git commit: [SPARK-23020][CORE][FOLLOWUP] Fix Java style check issues.

2018-01-24 Thread vanzin
Repository: spark Updated Branches: refs/heads/branch-2.3 4336e67f4 -> 2221a3035 [SPARK-23020][CORE][FOLLOWUP] Fix Java style check issues. ## What changes were proposed in this pull request? This is a follow-up of #20297 which broke lint-java checks. This pr fixes the lint-java issues. ```

spark git commit: [SPARK-23020][CORE][FOLLOWUP] Fix Java style check issues.

2018-01-24 Thread vanzin
Repository: spark Updated Branches: refs/heads/master e18d6f532 -> 8c273b416 [SPARK-23020][CORE][FOLLOWUP] Fix Java style check issues. ## What changes were proposed in this pull request? This is a follow-up of #20297 which broke lint-java checks. This pr fixes the lint-java issues. ``` [ERR

spark git commit: [SPARK-20906][SPARKR] Add API doc example for Constrained Logistic Regression

2018-01-24 Thread felixcheung
Repository: spark Updated Branches: refs/heads/branch-2.3 17317c8fb -> 4336e67f4 [SPARK-20906][SPARKR] Add API doc example for Constrained Logistic Regression ## What changes were proposed in this pull request? doc only changes ## How was this patch tested? manual Author: Felix Cheung Cl

spark git commit: [SPARK-20906][SPARKR] Add API doc example for Constrained Logistic Regression

2018-01-24 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master 0ec95bb7d -> e18d6f532 [SPARK-20906][SPARKR] Add API doc example for Constrained Logistic Regression ## What changes were proposed in this pull request? doc only changes ## How was this patch tested? manual Author: Felix Cheung Closes

[2/5] spark git commit: [SPARK-22577][CORE] executor page blacklist status should update with TaskSet level blacklisting

2018-01-24 Thread irashid
http://git-wip-us.apache.org/repos/asf/spark/blob/0ec95bb7/core/src/test/resources/spark-events/application_1516285256255_0012 -- diff --git a/core/src/test/resources/spark-events/application_1516285256255_0012 b/core/src/test/re

[4/5] spark git commit: [SPARK-22577][CORE] executor page blacklist status should update with TaskSet level blacklisting

2018-01-24 Thread irashid
http://git-wip-us.apache.org/repos/asf/spark/blob/0ec95bb7/core/src/test/resources/HistoryServerExpectations/one_stage_attempt_json_expectation.json -- diff --git a/core/src/test/resources/HistoryServerExpectations/one_stage_attem

[1/5] spark git commit: [SPARK-22577][CORE] executor page blacklist status should update with TaskSet level blacklisting

2018-01-24 Thread irashid
Repository: spark Updated Branches: refs/heads/master de36f65d3 -> 0ec95bb7d http://git-wip-us.apache.org/repos/asf/spark/blob/0ec95bb7/core/src/test/scala/org/apache/spark/deploy/history/HistoryServerSuite.scala -- diff --git

[5/5] spark git commit: [SPARK-22577][CORE] executor page blacklist status should update with TaskSet level blacklisting

2018-01-24 Thread irashid
[SPARK-22577][CORE] executor page blacklist status should update with TaskSet level blacklisting ## What changes were proposed in this pull request? In this PR stage blacklisting is propagated to UI by introducing a new Spark listener event (SparkListenerExecutorBlacklistedForStage) which indic

[3/5] spark git commit: [SPARK-22577][CORE] executor page blacklist status should update with TaskSet level blacklisting

2018-01-24 Thread irashid
http://git-wip-us.apache.org/repos/asf/spark/blob/0ec95bb7/core/src/test/resources/spark-events/app-20180109111548- -- diff --git a/core/src/test/resources/spark-events/app-20180109111548- b/core/src/test/resources/spark-e

svn commit: r24412 - in /dev/spark/2.4.0-SNAPSHOT-2018_01_24_08_01-de36f65-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-01-24 Thread pwendell
Author: pwendell Date: Wed Jan 24 16:17:53 2018 New Revision: 24412 Log: Apache Spark 2.4.0-SNAPSHOT-2018_01_24_08_01-de36f65 docs [This commit notification would consist of 1442 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

svn commit: r24411 - in /dev/spark/2.3.1-SNAPSHOT-2018_01_24_06_01-17317c8-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-01-24 Thread pwendell
Author: pwendell Date: Wed Jan 24 14:16:14 2018 New Revision: 24411 Log: Apache Spark 2.3.1-SNAPSHOT-2018_01_24_06_01-17317c8 docs [This commit notification would consist of 1442 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

spark git commit: [SPARK-23148][SQL] Allow pathnames with special characters for CSV / JSON / text

2018-01-24 Thread gurwls223
Repository: spark Updated Branches: refs/heads/branch-2.3 84a189a34 -> 17317c8fb [SPARK-23148][SQL] Allow pathnames with special characters for CSV / JSON / text …JSON / text ## What changes were proposed in this pull request? Fix for JSON and CSV data sources when file names include chara

spark git commit: [SPARK-23148][SQL] Allow pathnames with special characters for CSV / JSON / text

2018-01-24 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master 7af1a325d -> de36f65d3 [SPARK-23148][SQL] Allow pathnames with special characters for CSV / JSON / text …JSON / text ## What changes were proposed in this pull request? Fix for JSON and CSV data sources when file names include character

spark git commit: [SPARK-23174][BUILD][PYTHON] python code style checker update

2018-01-24 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master 4e7b49041 -> 7af1a325d [SPARK-23174][BUILD][PYTHON] python code style checker update ## What changes were proposed in this pull request? Referencing latest python code style checking from PyPi/pycodestyle Removed pending TODO For now, in to

spark git commit: [SPARK-23177][SQL][PYSPARK][BACKPORT-2.3] Extract zero-parameter UDFs from aggregate

2018-01-24 Thread gurwls223
Repository: spark Updated Branches: refs/heads/branch-2.3 d656be74b -> 84a189a34 [SPARK-23177][SQL][PYSPARK][BACKPORT-2.3] Extract zero-parameter UDFs from aggregate ## What changes were proposed in this pull request? We extract Python UDFs in logical aggregate which depends on aggregate ex

svn commit: r24409 - in /dev/spark/2.3.1-SNAPSHOT-2018_01_24_02_01-d656be7-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-01-24 Thread pwendell
Author: pwendell Date: Wed Jan 24 10:16:19 2018 New Revision: 24409 Log: Apache Spark 2.3.1-SNAPSHOT-2018_01_24_02_01-d656be7 docs [This commit notification would consist of 1442 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

svn commit: r24406 - in /dev/spark/2.4.0-SNAPSHOT-2018_01_24_00_01-4e7b490-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-01-24 Thread pwendell
Author: pwendell Date: Wed Jan 24 08:17:11 2018 New Revision: 24406 Log: Apache Spark 2.4.0-SNAPSHOT-2018_01_24_00_01-4e7b490 docs [This commit notification would consist of 1442 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---