[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22219 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22323: [SPARK-25262][K8S] Allow SPARK_LOCAL_DIRS to be t...

2018-09-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22323 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binaryFiles i...

2018-09-06 Thread bomeng
Github user bomeng commented on the issue: https://github.com/apache/spark/pull/22356 Thanks for taking my codes. Looks good. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional command

[GitHub] spark pull request #22215: [SPARK-25222][K8S] Improve container status loggi...

2018-09-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22215 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22323: [SPARK-25262][K8S] Allow SPARK_LOCAL_DIRS to be tmpfs ba...

2018-09-06 Thread mccheah
Github user mccheah commented on the issue: https://github.com/apache/spark/pull/22323 I'm going to merge this, for Spark 3.0 we should look at https://github.com/apache/spark/pull/22323#discussion_r215799370 though. --- --

[GitHub] spark pull request #22323: [SPARK-25262][K8S] Allow SPARK_LOCAL_DIRS to be t...

2018-09-06 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/22323#discussion_r215806032 --- Diff: docs/running-on-kubernetes.md --- @@ -215,6 +215,19 @@ spark.kubernetes.driver.volumes.persistentVolumeClaim.checkpointpvc.options.clai

[GitHub] spark issue #22215: [SPARK-25222][K8S] Improve container status logging

2018-09-06 Thread mccheah
Github user mccheah commented on the issue: https://github.com/apache/spark/pull/22215 Yup this can merge now, thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mai

[GitHub] spark issue #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binaryFiles i...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22356 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95771/ Test PASSed. ---

[GitHub] spark issue #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binaryFiles i...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22356 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binaryFiles i...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22356 **[Test build #95771 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95771/testReport)** for PR 22356 at commit [`6e1d8fd`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #22298: [SPARK-25021][K8S] Add spark.executor.pyspark.memory lim...

2018-09-06 Thread mccheah
Github user mccheah commented on the issue: https://github.com/apache/spark/pull/22298 Yeah this looks ok to me, would like a +1 from @felixcheung and @holdenk. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark

[GitHub] spark issue #21308: [SPARK-24253][SQL] Add DeleteSupport mix-in for DataSour...

2018-09-06 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/21308 @tigerquoll, there is currently no support to expose partitions through the v2 API. That would be a different operation. If you wanted to implement partition operations through this API, then you wou

[GitHub] spark issue #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binaryFiles i...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22356 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95769/ Test PASSed. ---

[GitHub] spark issue #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binaryFiles i...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22356 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binaryFiles i...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22356 **[Test build #95769 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95769/testReport)** for PR 22356 at commit [`84dd4a7`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #21308: [SPARK-24253][SQL] Add DeleteSupport mix-in for DataSour...

2018-09-06 Thread tigerquoll
Github user tigerquoll commented on the issue: https://github.com/apache/spark/pull/21308 @rdblue what about those data sources that support record deletion and partition dropping as two semantically different operations - Kudu and Hbase being two examples. All systems t

[GitHub] spark issue #21899: [SPARK-24912][SQL] Don't obscure source of OOM during br...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21899 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95773/ Test PASSed. ---

[GitHub] spark issue #21899: [SPARK-24912][SQL] Don't obscure source of OOM during br...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21899 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21899: [SPARK-24912][SQL] Don't obscure source of OOM during br...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21899 **[Test build #95773 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95773/testReport)** for PR 21899 at commit [`e927cd3`](https://github.com/apache/spark/commit/e

[GitHub] spark pull request #22323: [SPARK-25262][K8S] Allow SPARK_LOCAL_DIRS to be t...

2018-09-06 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/22323#discussion_r215799370 --- Diff: docs/running-on-kubernetes.md --- @@ -215,6 +215,19 @@ spark.kubernetes.driver.volumes.persistentVolumeClaim.checkpointpvc.options.clai

[GitHub] spark issue #22354: [SPARK-23243][CORE][2.3] Fix RDD.repartition() data corr...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22354 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95770/ Test PASSed. ---

[GitHub] spark issue #22354: [SPARK-23243][CORE][2.3] Fix RDD.repartition() data corr...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22354 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22354: [SPARK-23243][CORE][2.3] Fix RDD.repartition() data corr...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22354 **[Test build #95770 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95770/testReport)** for PR 22354 at commit [`1e01b00`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #21221: [SPARK-23429][CORE] Add executor memory metrics to heart...

2018-09-06 Thread mccheah
Github user mccheah commented on the issue: https://github.com/apache/spark/pull/21221 @edwinalu - this can merge now that Spark 2.4's release branch has been cut, but there's conflicting files now. Can we clear the conflicts and then we can merge this? ---

[GitHub] spark issue #22146: [SPARK-24434][K8S] pod template files

2018-09-06 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/22146 @mccheah @erikerlandson given that banch-2.4 has already been cut, we have more flexibility in terms of timing of getting this in, as well as more space for thoughts to get this right. I don't st

[GitHub] spark issue #22010: [SPARK-21436][CORE] Take advantage of known partitioner ...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22010 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22010: [SPARK-21436][CORE] Take advantage of known partitioner ...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22010 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95768/ Test PASSed. ---

[GitHub] spark issue #22010: [SPARK-21436][CORE] Take advantage of known partitioner ...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22010 **[Test build #95768 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95768/testReport)** for PR 22010 at commit [`4c89653`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #22010: [SPARK-21436][CORE] Take advantage of known partitioner ...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22010 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95767/ Test FAILed. ---

[GitHub] spark issue #22010: [SPARK-21436][CORE] Take advantage of known partitioner ...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22010 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-

[GitHub] spark issue #22010: [SPARK-21436][CORE] Take advantage of known partitioner ...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22010 **[Test build #95767 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95767/testReport)** for PR 22010 at commit [`7ed7589`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #22352: [SPARK-25208][SQL][FOLLOW-UP] Reduce code size.

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22352 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95766/ Test PASSed. ---

[GitHub] spark issue #22352: [SPARK-25208][SQL][FOLLOW-UP] Reduce code size.

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22352 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22352: [SPARK-25208][SQL][FOLLOW-UP] Reduce code size.

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22352 **[Test build #95766 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95766/testReport)** for PR 22352 at commit [`ee47126`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #22162: [spark-24442][SQL] Added parameters to control the defau...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22162 **[Test build #95775 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95775/testReport)** for PR 22162 at commit [`f93e58e`](https://github.com/apache/spark/commit/f9

[GitHub] spark pull request #22227: [SPARK-25202] [SQL] Implements split with limit s...

2018-09-06 Thread phegstrom
Github user phegstrom commented on a diff in the pull request: https://github.com/apache/spark/pull/7#discussion_r215764191 --- Diff: python/pyspark/sql/functions.py --- @@ -1669,20 +1669,33 @@ def repeat(col, n): return Column(sc._jvm.functions.repeat(_to_java_column(c

[GitHub] spark pull request #22192: [SPARK-24918][Core] Executor Plugin API

2018-09-06 Thread NiharS
Github user NiharS commented on a diff in the pull request: https://github.com/apache/spark/pull/22192#discussion_r215760787 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -218,6 +244,8 @@ private[spark] class Executor( env.metricsSystem.repor

[GitHub] spark issue #22227: [SPARK-25202] [SQL] Implements split with limit sql func...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/7 **[Test build #95774 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95774/testReport)** for PR 7 at commit [`64b0afc`](https://github.com/apache/spark/commit/64

[GitHub] spark pull request #22192: [SPARK-24918][Core] Executor Plugin API

2018-09-06 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/22192#discussion_r215750952 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -218,6 +244,8 @@ private[spark] class Executor( env.metricsSystem.repo

[GitHub] spark pull request #22192: [SPARK-24918][Core] Executor Plugin API

2018-09-06 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/22192#discussion_r215749405 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -136,6 +136,32 @@ private[spark] class Executor( // for fetching remote

[GitHub] spark pull request #22192: [SPARK-24918][Core] Executor Plugin API

2018-09-06 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/22192#discussion_r215749556 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -136,6 +136,32 @@ private[spark] class Executor( // for fetching remote

[GitHub] spark pull request #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binary...

2018-09-06 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/22356#discussion_r215749840 --- Diff: core/src/test/scala/org/apache/spark/FileSuite.scala --- @@ -299,6 +301,25 @@ class FileSuite extends SparkFunSuite with LocalSparkContext {

[GitHub] spark issue #22192: [SPARK-24918][Core] Executor Plugin API

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22192 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22192: [SPARK-24918][Core] Executor Plugin API

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22192 **[Test build #95772 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95772/testReport)** for PR 22192 at commit [`b0be9e5`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #22192: [SPARK-24918][Core] Executor Plugin API

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22192 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95772/ Test FAILed. ---

[GitHub] spark issue #22192: [SPARK-24918][Core] Executor Plugin API

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22192 **[Test build #95772 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95772/testReport)** for PR 22192 at commit [`b0be9e5`](https://github.com/apache/spark/commit/b0

[GitHub] spark issue #21899: [SPARK-24912][SQL] Don't obscure source of OOM during br...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21899 **[Test build #95773 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95773/testReport)** for PR 21899 at commit [`e927cd3`](https://github.com/apache/spark/commit/e9

[GitHub] spark issue #22165: [SPARK-25017][Core] Add test suite for BarrierCoordinato...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22165 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95761/ Test PASSed. ---

[GitHub] spark issue #22165: [SPARK-25017][Core] Add test suite for BarrierCoordinato...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22165 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22165: [SPARK-25017][Core] Add test suite for BarrierCoordinato...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22165 **[Test build #95761 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95761/testReport)** for PR 22165 at commit [`ecf12bd`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binaryFiles i...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22356 **[Test build #95771 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95771/testReport)** for PR 22356 at commit [`6e1d8fd`](https://github.com/apache/spark/commit/6e

[GitHub] spark issue #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binaryFiles i...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22356 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2914/

[GitHub] spark issue #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binaryFiles i...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22356 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22144: [SPARK-24935][SQL] : Problem with Executing Hive UDF's f...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22144 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95763/ Test PASSed. ---

[GitHub] spark issue #22144: [SPARK-24935][SQL] : Problem with Executing Hive UDF's f...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22144 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22144: [SPARK-24935][SQL] : Problem with Executing Hive UDF's f...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22144 **[Test build #95763 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95763/testReport)** for PR 22144 at commit [`173035d`](https://github.com/apache/spark/commit/1

[GitHub] spark pull request #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binary...

2018-09-06 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22356#discussion_r215737907 --- Diff: core/src/test/scala/org/apache/spark/FileSuite.scala --- @@ -299,6 +301,25 @@ class FileSuite extends SparkFunSuite with LocalSparkContext {

[GitHub] spark pull request #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binary...

2018-09-06 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22356#discussion_r215737727 --- Diff: core/src/test/scala/org/apache/spark/FileSuite.scala --- @@ -299,6 +301,25 @@ class FileSuite extends SparkFunSuite with LocalSparkContext {

[GitHub] spark issue #22355: [SPARK-25358][SQL] MutableProjection supports fallback t...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22355 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22355: [SPARK-25358][SQL] MutableProjection supports fallback t...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22355 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95765/ Test PASSed. ---

[GitHub] spark issue #22355: [SPARK-25358][SQL] MutableProjection supports fallback t...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22355 **[Test build #95765 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95765/testReport)** for PR 22355 at commit [`c25c426`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #22354: [SPARK-23243][CORE][2.3] Fix RDD.repartition() data corr...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22354 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22354: [SPARK-23243][CORE][2.3] Fix RDD.repartition() data corr...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22354 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2913/

[GitHub] spark issue #22354: [SPARK-23243][CORE][2.3] Fix RDD.repartition() data corr...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22354 **[Test build #95770 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95770/testReport)** for PR 22354 at commit [`1e01b00`](https://github.com/apache/spark/commit/1e

[GitHub] spark issue #22354: [SPARK-23243][CORE][2.3] Fix RDD.repartition() data corr...

2018-09-06 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/22354 thats weird, swear it had already passed, test failure does not look related --- - To unsubscribe, e-mail: reviews-unsubscr...@

[GitHub] spark issue #22354: [SPARK-23243][CORE][2.3] Fix RDD.repartition() data corr...

2018-09-06 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/22354 test this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h.

[GitHub] spark pull request #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binary...

2018-09-06 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/22356#discussion_r215730938 --- Diff: core/src/test/scala/org/apache/spark/FileSuite.scala --- @@ -299,6 +301,25 @@ class FileSuite extends SparkFunSuite with LocalSparkContext {

[GitHub] spark pull request #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binary...

2018-09-06 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/22356#discussion_r215730668 --- Diff: core/src/test/scala/org/apache/spark/FileSuite.scala --- @@ -299,6 +301,25 @@ class FileSuite extends SparkFunSuite with LocalSparkContext {

[GitHub] spark issue #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binaryFiles i...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22356 **[Test build #95769 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95769/testReport)** for PR 22356 at commit [`84dd4a7`](https://github.com/apache/spark/commit/84

[GitHub] spark issue #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binaryFiles i...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22356 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binaryFiles i...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22356 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2912/

[GitHub] spark issue #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binaryFiles i...

2018-09-06 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22356 CC @bomeng @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark pull request #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binary...

2018-09-06 Thread srowen
GitHub user srowen opened a pull request: https://github.com/apache/spark/pull/22356 [SPARK-22357][CORE][FOLLOWUP] SparkContext.binaryFiles ignore minPartitions parameter ## What changes were proposed in this pull request? This adds a test following https://github.com/apach

[GitHub] spark issue #22192: [SPARK-24918][Core] Executor Plugin API

2018-09-06 Thread NiharS
Github user NiharS commented on the issue: https://github.com/apache/spark/pull/22192 @mridulm I'm inclined to agree. I was doing some extra testing on separate thread and it seems to be performing well. I'll push this shortly. I'm going to also test setting the class loader and then

[GitHub] spark issue #22354: [SPARK-23243][CORE][2.3] Fix RDD.repartition() data corr...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22354 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95764/ Test FAILed. ---

[GitHub] spark issue #22354: [SPARK-23243][CORE][2.3] Fix RDD.repartition() data corr...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22354 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22354: [SPARK-23243][CORE][2.3] Fix RDD.repartition() data corr...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22354 **[Test build #95764 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95764/testReport)** for PR 22354 at commit [`1e01b00`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #22048: [SPARK-25108][SQL] Fix the show method to display the wi...

2018-09-06 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22048 OK, it isn't just me. We'll see if it's a broader problem. I manually cherry-picked to branch-2.4 --- - To unsubscribe, e

[GitHub] spark pull request #21669: [SPARK-23257][K8S] Kerberos Support for Spark on ...

2018-09-06 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21669#discussion_r215715959 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/hadoopsteps/HadoopKerberosKeytabResolverStep.scala --- @@ -0

[GitHub] spark issue #22048: [SPARK-25108][SQL] Fix the show method to display the wi...

2018-09-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22048 Ur, it seems that I'm facing the same situation. In this case, should we just cherry-pick to `branch-2.4`? --- - To unsubs

[GitHub] spark issue #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_json

2018-09-06 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/22237 @HyukjinKwon I re-targeted the changes for Spark 3.0. Please, take a look at it one more time. --- - To unsubscribe, e-mail: rev

[GitHub] spark issue #22010: [SPARK-21436][CORE] Take advantage of known partitioner ...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22010 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2911/

[GitHub] spark issue #22010: [SPARK-21436][CORE] Take advantage of known partitioner ...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22010 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2910/

[GitHub] spark issue #22010: [SPARK-21436][CORE] Take advantage of known partitioner ...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22010 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22010: [SPARK-21436][CORE] Take advantage of known partitioner ...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22010 Build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-

[GitHub] spark issue #22323: [SPARK-25262][K8S] Allow SPARK_LOCAL_DIRS to be tmpfs ba...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22323 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22323: [SPARK-25262][K8S] Allow SPARK_LOCAL_DIRS to be tmpfs ba...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22323 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2909/

[GitHub] spark issue #22323: [SPARK-25262][K8S] Allow SPARK_LOCAL_DIRS to be tmpfs ba...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22323 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/2909/ --- --

[GitHub] spark issue #22048: [SPARK-25108][SQL] Fix the show method to display the wi...

2018-09-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22048 @srowen . Sure. I'll try to backport this. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additiona

[GitHub] spark issue #22323: [SPARK-25262][K8S] Allow SPARK_LOCAL_DIRS to be tmpfs ba...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22323 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/2909/ ---

[GitHub] spark pull request #22140: [SPARK-25072][PySpark] Forbid extra value for cus...

2018-09-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22140 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22048: [SPARK-25108][SQL] Fix the show method to display the wi...

2018-09-06 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22048 @dongjoon-hyun could I ask a favor; can you try merging this? the merge script should say it's merged and offer to back-port to 2.4. However I keep getting errors about it not being mergeable, althou

[GitHub] spark issue #22345: [SPARK-12321][SQL][FOLLOW-UP] Add tests for fromString

2018-09-06 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22345 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.

[GitHub] spark issue #22140: [SPARK-25072][PySpark] Forbid extra value for custom Row

2018-09-06 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/22140 merged to master, branch 2.4 and 2.3. Thanks @xuanyuanking ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.o

[GitHub] spark issue #22010: [SPARK-21436][CORE] Take advantage of known partitioner ...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22010 **[Test build #95768 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95768/testReport)** for PR 22010 at commit [`4c89653`](https://github.com/apache/spark/commit/4c

[GitHub] spark issue #22140: [SPARK-25072][PySpark] Forbid extra value for custom Row

2018-09-06 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/22140 > yea, to me it looks less sense actually but seems at least working for now: good point, I guess it only fails when you supply a schema. --- -

[GitHub] spark issue #22354: [SPARK-23243][CORE][2.3] Fix RDD.repartition() data corr...

2018-09-06 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/22354 +1 thanks @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: revie

[GitHub] spark issue #22323: [SPARK-25262][K8S] Allow SPARK_LOCAL_DIRS to be tmpfs ba...

2018-09-06 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/22323 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@sp

[GitHub] spark issue #22271: [SPARK-25268][GraphX]run Parallel Personalized PageRank ...

2018-09-06 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/22271 Thank you @jkbradley for merging. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_json

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22237 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95760/ Test PASSed. ---

<    1   2   3   4   5   >