svn commit: r29029 - in /dev/spark/2.4.0-SNAPSHOT-2018_08_29_20_02-3a66a7f-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

Author: pwendell Date: Thu Aug 30 03:16:10 2018 New Revision: 29029 Log: Apache Spark 2.4.0-SNAPSHOT-2018_08_29_20_02-3a66a7f docs [This commit notification would consist of 1478 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-25253][PYSPARK][FOLLOWUP] Undefined name: from pyspark.util import _exception_message

Repository: spark Updated Branches: refs/heads/master ec3e99863 -> 3a66a7fca [SPARK-25253][PYSPARK][FOLLOWUP] Undefined name: from pyspark.util import _exception_message HyukjinKwon ## What changes were proposed in this pull request? add __from pyspark.util import \_exception_message__ to

spark git commit: [SPARK-24909][CORE] Always unregister pending partition on task completion.

Repository: spark Updated Branches: refs/heads/master 6b1b10ca8 -> ec3e99863 [SPARK-24909][CORE] Always unregister pending partition on task completion. Spark scheduler can hang when fetch failures, executor lost, task running on lost executor, and multiple stage attempts. To fix this we

svn commit: r29023 - in /dev/spark/2.4.0-SNAPSHOT-2018_08_29_16_02-6b1b10c-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

Author: pwendell Date: Wed Aug 29 23:16:28 2018 New Revision: 29023 Log: Apache Spark 2.4.0-SNAPSHOT-2018_08_29_16_02-6b1b10c docs [This commit notification would consist of 1478 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [DOC] Fix comment on SparkPlanGraphEdge

Repository: spark Updated Branches: refs/heads/master 20b7c684c -> 6b1b10ca8 [DOC] Fix comment on SparkPlanGraphEdge ## What changes were proposed in this pull request? `fromId` is the child, and `toId` is the parent, see line 127 in `buildSparkPlanGraphNode` above. The edges in Spark UI

svn commit: r29018 - in /dev/spark/2.4.0-SNAPSHOT-2018_08_29_08_02-20b7c68-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

Author: pwendell Date: Wed Aug 29 15:16:36 2018 New Revision: 29018 Log: Apache Spark 2.4.0-SNAPSHOT-2018_08_29_08_02-20b7c68 docs [This commit notification would consist of 1478 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-25248][.1][PYSPARK] update barrier Python API

Repository: spark Updated Branches: refs/heads/master 3864480e1 -> 20b7c684c [SPARK-25248][.1][PYSPARK] update barrier Python API ## What changes were proposed in this pull request? I made one pass over the Python APIs for barrier mode and updated them to match the Scala doc in #22240 .

spark git commit: [SPARK-25266][CORE] Fix memory leak in Barrier Execution Mode

Repository: spark Updated Branches: refs/heads/master 1fd59c129 -> 3864480e1 [SPARK-25266][CORE] Fix memory leak in Barrier Execution Mode ## What changes were proposed in this pull request? BarrierCoordinator uses Timer and TimerTask. `TimerTask#cancel()` is invoked in

svn commit: r29010 - in /dev/spark/2.4.0-SNAPSHOT-2018_08_29_04_02-1fd59c1-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

Author: pwendell Date: Wed Aug 29 11:20:54 2018 New Revision: 29010 Log: Apache Spark 2.4.0-SNAPSHOT-2018_08_29_04_02-1fd59c1 docs [This commit notification would consist of 1478 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [WIP][SPARK-25044][SQL] (take 2) Address translation of LMF closure primitive args to Object in Scala 2.12

Repository: spark Updated Branches: refs/heads/master 82c18c240 -> 1fd59c129 [WIP][SPARK-25044][SQL] (take 2) Address translation of LMF closure primitive args to Object in Scala 2.12 ## What changes were proposed in this pull request? Alternative take on

svn commit: r29007 - in /dev/spark/2.4.0-SNAPSHOT-2018_08_29_00_02-82c18c2-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

Author: pwendell Date: Wed Aug 29 07:16:59 2018 New Revision: 29007 Log: Apache Spark 2.4.0-SNAPSHOT-2018_08_29_00_02-82c18c2 docs [This commit notification would consist of 1478 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-23030][SQL][PYTHON] Use Arrow stream format for creating from and collecting Pandas DataFrames

Repository: spark Updated Branches: refs/heads/master ff8dcc1d4 -> 82c18c240 [SPARK-23030][SQL][PYTHON] Use Arrow stream format for creating from and collecting Pandas DataFrames ## What changes were proposed in this pull request? This changes the calls of `toPandas()` and