[2/2] spark git commit: [SPARK-24589][CORE] Correctly identify tasks in output commit coordinator [branch-2.1].

[SPARK-24589][CORE] Correctly identify tasks in output commit coordinator [branch-2.1]. When an output stage is retried, it's possible that tasks from the previous attempt are still running. In that case, there would be a new task for the same partition in the new attempt, and the coordinator

[1/2] spark git commit: [SPARK-22897][CORE] Expose stageAttemptId in TaskContext

Repository: spark Updated Branches: refs/heads/branch-2.1 77d11df4f -> 46e6b6c0e [SPARK-22897][CORE] Expose stageAttemptId in TaskContext stageAttemptId added in TaskContext and corresponding construction modification Added a new test in TaskContextSuite, two cases are tested: 1. Normal case

svn commit: r27669 - in /dev/spark/2.4.0-SNAPSHOT-2018_06_22_12_01-4e7d867-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

Author: pwendell Date: Fri Jun 22 19:15:42 2018 New Revision: 27669 Log: Apache Spark 2.4.0-SNAPSHOT-2018_06_22_12_01-4e7d867 docs [This commit notification would consist of 1468 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-24372][BUILD] Add scripts to help with preparing releases.

Repository: spark Updated Branches: refs/heads/master 33e77fa89 -> 4e7d8678a [SPARK-24372][BUILD] Add scripts to help with preparing releases. The "do-release.sh" script asks questions about the RC being prepared, trying to find out as much as possible automatically, and then executes the

spark git commit: [SPARK-24518][CORE] Using Hadoop credential provider API to store password

Repository: spark Updated Branches: refs/heads/master 39dfaf2fd -> 33e77fa89 [SPARK-24518][CORE] Using Hadoop credential provider API to store password ## What changes were proposed in this pull request? In our distribution, because we don't do such fine-grained access control of config

svn commit: r27662 - in /dev/spark/2.4.0-SNAPSHOT-2018_06_22_08_02-39dfaf2-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

Author: pwendell Date: Fri Jun 22 15:19:47 2018 New Revision: 27662 Log: Apache Spark 2.4.0-SNAPSHOT-2018_06_22_08_02-39dfaf2 docs [This commit notification would consist of 1468 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-24519] Make the threshold for highly compressed map status configurable

Repository: spark Updated Branches: refs/heads/master 92c2f00bd -> 39dfaf2fd [SPARK-24519] Make the threshold for highly compressed map status configurable **Problem** MapStatus uses hardcoded value of 2000 partitions to determine if it should use highly compressed map status. We should make

spark git commit: [SPARK-22897][CORE] Expose stageAttemptId in TaskContext

Repository: spark Updated Branches: refs/heads/branch-2.2 751b00820 -> a6459 [SPARK-22897][CORE] Expose stageAttemptId in TaskContext stageAttemptId added in TaskContext and corresponding construction modification Added a new test in TaskContextSuite, two cases are tested: 1. Normal case

svn commit: r27653 - in /dev/spark/2.4.0-SNAPSHOT-2018_06_22_04_02-92c2f00-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

Author: pwendell Date: Fri Jun 22 11:18:31 2018 New Revision: 27653 Log: Apache Spark 2.4.0-SNAPSHOT-2018_06_22_04_02-92c2f00 docs [This commit notification would consist of 1468 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-23934][SQL] Adding map_from_entries function

Repository: spark Updated Branches: refs/heads/master dc8a6befa -> 92c2f00bd [SPARK-23934][SQL] Adding map_from_entries function ## What changes were proposed in this pull request? The PR adds the `map_from_entries` function that returns a map created from the given array of entries. ## How