[GitHub] spark pull request #22457: [SPARK-24626] Add statistics prefix to parallelFi...

2018-09-18 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/22457 [SPARK-24626] Add statistics prefix to parallelFileListingInStatsComputation ## What changes were proposed in this pull request? To be more consistent with other statistics based configs.

[GitHub] spark issue #22457: [SPARK-24626] Add statistics prefix to parallelFileListi...

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22457 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22457: [SPARK-24626] Add statistics prefix to parallelFileListi...

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22457 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96207/ Test FAILed. ---

[GitHub] spark issue #22457: [SPARK-24626] Add statistics prefix to parallelFileListi...

2018-09-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22457 **[Test build #96207 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96207/testReport)** for PR 22457 at commit [`8d46d72`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #22173: [SPARK-24335] Spark external shuffle server improvement ...

2018-09-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22173 **[Test build #96208 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96208/testReport)** for PR 22173 at commit [`0a43f22`](https://github.com/apache/spark/commit/0a

[GitHub] spark issue #22457: [SPARK-24626] Add statistics prefix to parallelFileListi...

2018-09-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22457 **[Test build #96207 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96207/testReport)** for PR 22457 at commit [`8d46d72`](https://github.com/apache/spark/commit/8d

[GitHub] spark issue #22457: [SPARK-24626] Add statistics prefix to parallelFileListi...

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22457 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3206/

[GitHub] spark issue #22457: [SPARK-24626] Add statistics prefix to parallelFileListi...

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22457 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #22173: [SPARK-24335] Spark external shuffle server impro...

2018-09-18 Thread redsanket
Github user redsanket commented on a diff in the pull request: https://github.com/apache/spark/pull/22173#discussion_r218621553 --- Diff: common/network-common/src/main/java/org/apache/spark/network/server/ChunkFetchRequestHandler.java --- @@ -0,0 +1,136 @@ +/* + * License

[GitHub] spark issue #22456: [SPARK-19355][SQL] Fix variable names numberOfOutput -> ...

2018-09-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22456 **[Test build #96206 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96206/testReport)** for PR 22456 at commit [`793fc19`](https://github.com/apache/spark/commit/79

[GitHub] spark issue #22456: [SPARK-19355][SQL] Fix variable names numberOfOutput -> ...

2018-09-18 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/22456 cc @hvanhovell @cloud-fan also @viirya please don't use such cryptic variable names ... we also need to fix the documentation for the config flag - it's arcane. ---

[GitHub] spark issue #22456: [SPARK-19355][SQL] Fix variable names numberOfOutput -> ...

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22456 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22456: [SPARK-19355][SQL] Fix variable names numberOfOutput -> ...

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22456 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3205/

[GitHub] spark pull request #22456: [SPARK-19355][SQL] Fix variable names numberOfOut...

2018-09-18 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/22456 [SPARK-19355][SQL] Fix variable names numberOfOutput ## What changes were proposed in this pull request? SPARK-19355 introduced a variable / method called numberOfOutput, which is a really bad nam

[GitHub] spark issue #22429: [SPARK-25440][SQL] Dumping query execution info to a fil...

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22429 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22429: [SPARK-25440][SQL] Dumping query execution info to a fil...

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22429 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96192/ Test FAILed. ---

[GitHub] spark issue #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_json

2018-09-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22237 **[Test build #96205 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96205/testReport)** for PR 22237 at commit [`5884173`](https://github.com/apache/spark/commit/58

[GitHub] spark issue #22429: [SPARK-25440][SQL] Dumping query execution info to a fil...

2018-09-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22429 **[Test build #96192 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96192/testReport)** for PR 22429 at commit [`24dbbba`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_json

2018-09-18 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22237 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark issue #22376: [SPARK-25021][K8S][BACKPORT] Add spark.executor.pyspark....

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22376 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96193/ Test FAILed. ---

[GitHub] spark issue #22376: [SPARK-25021][K8S][BACKPORT] Add spark.executor.pyspark....

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22376 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22376: [SPARK-25021][K8S][BACKPORT] Add spark.executor.pyspark....

2018-09-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22376 **[Test build #96193 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96193/testReport)** for PR 22376 at commit [`4a0cffb`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_json

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22237 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_json

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22237 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96200/ Test FAILed. ---

[GitHub] spark pull request #22392: [SPARK-23200] Reset Kubernetes-specific config on...

2018-09-18 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22392#discussion_r218616946 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/Checkpoint.scala --- @@ -54,6 +54,10 @@ class Checkpoint(ssc: StreamingContext, val checkp

[GitHub] spark issue #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_json

2018-09-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22237 **[Test build #96200 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96200/testReport)** for PR 22237 at commit [`5884173`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #22448: [SPARK-25417][SQL] Improve findTightestCommonType to coe...

2018-09-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22448 **[Test build #96204 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96204/testReport)** for PR 22448 at commit [`0de3328`](https://github.com/apache/spark/commit/0d

[GitHub] spark issue #16677: [SPARK-19355][SQL] Use map output statistics to improve ...

2018-09-18 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16677 1. `numOutputs` is the number or records 2. 8 bytes per `MapStatus`. --- - To unsubscribe, e-mail: reviews-unsubscr...@spa

[GitHub] spark issue #22448: [SPARK-25417][SQL] Improve findTightestCommonType to coe...

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22448 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #22392: [SPARK-23200] Reset Kubernetes-specific config on...

2018-09-18 Thread ssaavedra
Github user ssaavedra commented on a diff in the pull request: https://github.com/apache/spark/pull/22392#discussion_r218616371 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/Checkpoint.scala --- @@ -54,6 +54,10 @@ class Checkpoint(ssc: StreamingContext, val checkpo

[GitHub] spark issue #22448: [SPARK-25417][SQL] Improve findTightestCommonType to coe...

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22448 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3204/

[GitHub] spark issue #22392: [SPARK-23200] Reset Kubernetes-specific config on Checkp...

2018-09-18 Thread ssaavedra
Github user ssaavedra commented on the issue: https://github.com/apache/spark/pull/22392 @skonto this seems to be related to how the Checkpointing code works in general. Variables that are not explicitly whitelisted as per the list I'm changing in this proposal seems to be reloaded fr

[GitHub] spark issue #16677: [SPARK-19355][SQL] Use map output statistics to improve ...

2018-09-18 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16677 two questions about this (i just saw this from a different place): 1. is numOutput about number of records? 2. how much memory usage will be increased by, for the driver, at scale? -

[GitHub] spark pull request #16677: [SPARK-19355][SQL] Use map output statistics to i...

2018-09-18 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/16677#discussion_r218614872 --- Diff: core/src/main/scala/org/apache/spark/scheduler/MapStatus.scala --- @@ -44,18 +45,23 @@ private[spark] sealed trait MapStatus { * necessary for

[GitHub] spark issue #22448: [SPARK-25417][SQL] Improve findTightestCommonType to coe...

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22448 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96195/ Test FAILed. ---

[GitHub] spark issue #22448: [SPARK-25417][SQL] Improve findTightestCommonType to coe...

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22448 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22448: [SPARK-25417][SQL] Improve findTightestCommonType to coe...

2018-09-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22448 **[Test build #96195 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96195/testReport)** for PR 22448 at commit [`8104b1f`](https://github.com/apache/spark/commit/8

[GitHub] spark pull request #22414: [SPARK-25424][SQL] Window duration and slide dura...

2018-09-18 Thread raghavgautam
Github user raghavgautam commented on a diff in the pull request: https://github.com/apache/spark/pull/22414#discussion_r218613023 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/TimeWindow.scala --- @@ -35,6 +35,10 @@ case class TimeWindow(

[GitHub] spark issue #22455: [SPARK-24572][SparkR] "eager execution" for R shell, IDE

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22455 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22455: [SPARK-24572][SparkR] "eager execution" for R shell, IDE

2018-09-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22455 **[Test build #96203 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96203/testReport)** for PR 22455 at commit [`cd8a704`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #22455: [SPARK-24572][SparkR] "eager execution" for R shell, IDE

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22455 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96203/ Test PASSed. ---

[GitHub] spark pull request #21950: [SPARK-24914][SQL][WIP] Add configuration to avoi...

2018-09-18 Thread bersprockets
Github user bersprockets commented on a diff in the pull request: https://github.com/apache/spark/pull/21950#discussion_r218608537 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala --- @@ -1051,11 +1052,27 @@ private[hive] object HiveClientImp

[GitHub] spark issue #17654: [SPARK-20351] [ML] Add trait hasTrainingSummary to repla...

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17654 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96194/ Test FAILed. ---

[GitHub] spark issue #17654: [SPARK-20351] [ML] Add trait hasTrainingSummary to repla...

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17654 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-

[GitHub] spark issue #17654: [SPARK-20351] [ML] Add trait hasTrainingSummary to repla...

2018-09-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17654 **[Test build #96194 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96194/testReport)** for PR 17654 at commit [`3bca3b1`](https://github.com/apache/spark/commit/3

[GitHub] spark pull request #22452: [SPARK-25456][SQL][TEST] Fix PythonForeachWriterS...

2018-09-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22452 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22452: [SPARK-25456][SQL][TEST] Fix PythonForeachWriterSuite

2018-09-18 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/22452 merged to master & 2.4 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #21524: [SPARK-24212][ML][doc] Add the example and user guide fo...

2018-09-18 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/21524 @tengpeng would you like to update this? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional command

[GitHub] spark issue #22455: [SPARK-24572][SparkR] "eager execution" for R shell, IDE

2018-09-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22455 **[Test build #96203 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96203/testReport)** for PR 22455 at commit [`cd8a704`](https://github.com/apache/spark/commit/cd

[GitHub] spark issue #22455: [SPARK-24572][SparkR] "eager execution" for R shell, IDE

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22455 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22455: [SPARK-24572][SparkR] "eager execution" for R shell, IDE

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22455 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3203/

[GitHub] spark pull request #22455: [SPARK-24572][SparkR] "eager execution" for R she...

2018-09-18 Thread adrian555
GitHub user adrian555 opened a pull request: https://github.com/apache/spark/pull/22455 [SPARK-24572][SparkR] "eager execution" for R shell, IDE ## What changes were proposed in this pull request? Check the `spark.sql.repl.eagerEval.enabled` configuration property in SparkD

[GitHub] spark issue #22173: [SPARK-24335] Spark external shuffle server improvement ...

2018-09-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22173 **[Test build #96202 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96202/testReport)** for PR 22173 at commit [`9bf5fe0`](https://github.com/apache/spark/commit/9b

[GitHub] spark issue #21950: [SPARK-24914][SQL][WIP] Add configuration to avoid OOM d...

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21950 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21950: [SPARK-24914][SQL][WIP] Add configuration to avoid OOM d...

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21950 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96190/ Test FAILed. ---

[GitHub] spark issue #21950: [SPARK-24914][SQL][WIP] Add configuration to avoid OOM d...

2018-09-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21950 **[Test build #96190 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96190/testReport)** for PR 21950 at commit [`e776ef2`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #22451: [SPARK-24777][SQL] Add write benchmark for AVRO

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22451 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22451: [SPARK-24777][SQL] Add write benchmark for AVRO

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22451 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96191/ Test FAILed. ---

[GitHub] spark issue #22451: [SPARK-24777][SQL] Add write benchmark for AVRO

2018-09-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22451 **[Test build #96191 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96191/testReport)** for PR 22451 at commit [`34af59d`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark

2018-09-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #96199 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96199/testReport)** for PR 21990 at commit [`def4f3e`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96199/ Test PASSed. ---

[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22305: [WIP][SPARK-24561][SQL][Python] User-defined window aggr...

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22305 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22305: [WIP][SPARK-24561][SQL][Python] User-defined window aggr...

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22305 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3202/

[GitHub] spark pull request #22173: [SPARK-24335] Spark external shuffle server impro...

2018-09-18 Thread redsanket
Github user redsanket commented on a diff in the pull request: https://github.com/apache/spark/pull/22173#discussion_r218590278 --- Diff: common/network-common/src/main/java/org/apache/spark/network/server/ChunkFetchRequestHandler.java --- @@ -0,0 +1,136 @@ +/* + * License

[GitHub] spark issue #22305: [WIP][SPARK-24561][SQL][Python] User-defined window aggr...

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22305 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22305: [WIP][SPARK-24561][SQL][Python] User-defined window aggr...

2018-09-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22305 **[Test build #96201 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96201/testReport)** for PR 22305 at commit [`bd62334`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #22305: [WIP][SPARK-24561][SQL][Python] User-defined window aggr...

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22305 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96201/ Test FAILed. ---

[GitHub] spark issue #22305: [WIP][SPARK-24561][SQL][Python] User-defined window aggr...

2018-09-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22305 **[Test build #96201 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96201/testReport)** for PR 22305 at commit [`bd62334`](https://github.com/apache/spark/commit/bd

[GitHub] spark issue #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_json

2018-09-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22237 **[Test build #96200 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96200/testReport)** for PR 22237 at commit [`5884173`](https://github.com/apache/spark/commit/58

[GitHub] spark pull request #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_...

2018-09-18 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/22237#discussion_r218585584 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/JsonExpressionsSuite.scala --- @@ -450,7 +450,7 @@ class JsonExpressionsSui

[GitHub] spark pull request #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_...

2018-09-18 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/22237#discussion_r218584877 --- Diff: docs/sql-programming-guide.md --- @@ -1877,6 +1877,10 @@ working with timestamps in `pandas_udf`s to get the best performance, see # Mi

[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark

2018-09-18 Thread RussellSpitzer
Github user RussellSpitzer commented on the issue: https://github.com/apache/spark/pull/21990 @HyukjinKwon so you want me to rewrite the code in python? I will note SparkR is doing this exact same thing. --- - To un

[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark

2018-09-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #96199 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96199/testReport)** for PR 21990 at commit [`def4f3e`](https://github.com/apache/spark/commit/de

[GitHub] spark issue #22376: [SPARK-25021][K8S][BACKPORT] Add spark.executor.pyspark....

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22376 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22376: [SPARK-25021][K8S][BACKPORT] Add spark.executor.pyspark....

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22376 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3201/

[GitHub] spark issue #22376: [SPARK-25021][K8S][BACKPORT] Add spark.executor.pyspark....

2018-09-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22376 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/3201/ --- --

[GitHub] spark issue #22376: [SPARK-25021][K8S][BACKPORT] Add spark.executor.pyspark....

2018-09-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22376 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/3201/ ---

[GitHub] spark pull request #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_...

2018-09-18 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/22237#discussion_r218577348 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -595,10 +607,7 @@ case class JsonToStructs(

[GitHub] spark issue #22452: [SPARK-25456][SQL][TEST] Fix PythonForeachWriterSuite

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22452 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22452: [SPARK-25456][SQL][TEST] Fix PythonForeachWriterSuite

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22452 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96189/ Test PASSed. ---

[GitHub] spark issue #22452: [SPARK-25456][SQL][TEST] Fix PythonForeachWriterSuite

2018-09-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22452 **[Test build #96189 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96189/testReport)** for PR 22452 at commit [`33725ce`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #22376: [SPARK-25021][K8S][BACKPORT] Add spark.executor.pyspark....

2018-09-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22376 **[Test build #96198 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96198/testReport)** for PR 22376 at commit [`7c20ed9`](https://github.com/apache/spark/commit/7c

[GitHub] spark pull request #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_...

2018-09-18 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/22237#discussion_r218574370 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/JsonExpressionsSuite.scala --- @@ -402,13 +402,13 @@ class JsonExpressionsS

[GitHub] spark issue #22454: [SPARK-25445][BUILD][FOLLOWUP] Resolve issues in release...

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22454 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96188/ Test FAILed. ---

[GitHub] spark issue #22454: [SPARK-25445][BUILD][FOLLOWUP] Resolve issues in release...

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22454 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22454: [SPARK-25445][BUILD][FOLLOWUP] Resolve issues in release...

2018-09-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22454 **[Test build #96188 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96188/testReport)** for PR 22454 at commit [`09309f2`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #22173: [SPARK-24335] Spark external shuffle server improvement ...

2018-09-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22173 **[Test build #96197 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96197/testReport)** for PR 22173 at commit [`b488f0c`](https://github.com/apache/spark/commit/b4

[GitHub] spark issue #22454: [SPARK-25445][BUILD][FOLLOWUP] Resolve issues in release...

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22454 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96186/ Test FAILed. ---

[GitHub] spark issue #22454: [SPARK-25445][BUILD][FOLLOWUP] Resolve issues in release...

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22454 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22454: [SPARK-25445][BUILD][FOLLOWUP] Resolve issues in release...

2018-09-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22454 **[Test build #96186 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96186/testReport)** for PR 22454 at commit [`01a7c82`](https://github.com/apache/spark/commit/0

[GitHub] spark pull request #21809: [SPARK-24851][UI] Map a Stage ID to it's Associat...

2018-09-18 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/21809#discussion_r218570004 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala --- @@ -105,15 +105,30 @@ private[ui] class StagePage(parent: StagesTab, store: App

[GitHub] spark issue #22376: [SPARK-25021][K8S][BACKPORT] Add spark.executor.pyspark....

2018-09-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22376 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/3200/ --- --

[GitHub] spark issue #22376: [SPARK-25021][K8S][BACKPORT] Add spark.executor.pyspark....

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22376 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3200/

[GitHub] spark issue #22376: [SPARK-25021][K8S][BACKPORT] Add spark.executor.pyspark....

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22376 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #21809: [SPARK-24851][UI] Map a Stage ID to it's Associat...

2018-09-18 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/21809#discussion_r218567654 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala --- @@ -105,15 +105,30 @@ private[ui] class StagePage(parent: StagesTab, store: App

[GitHub] spark pull request #22454: [SPARK-25445][BUILD][FOLLOWUP] Resolve issues in ...

2018-09-18 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22454#discussion_r218567663 --- Diff: dev/create-release/release-build.sh --- @@ -446,6 +432,8 @@ if [[ "$1" == "publish-release" ]]; then # Clean-up Zinc nailgun process $

[GitHub] spark issue #22376: [SPARK-25021][K8S][BACKPORT] Add spark.executor.pyspark....

2018-09-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22376 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/3200/ ---

[GitHub] spark issue #22454: [SPARK-25445][BUILD][FOLLOWUP] Resolve issues in release...

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22454 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96185/ Test FAILed. ---

[GitHub] spark issue #22454: [SPARK-25445][BUILD][FOLLOWUP] Resolve issues in release...

2018-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22454 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

<    1   2   3   4   5   6   >