[GitHub] spark issue #22354: [SPARK-23243][CORE][2.3] Fix RDD.repartition() data corr...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22354 **[Test build #95770 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95770/testReport)** for PR 22354 at commit [`1e01b00`](https://github.com/apache/spark/commit/1e

[GitHub] spark issue #22354: [SPARK-23243][CORE][2.3] Fix RDD.repartition() data corr...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22354 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22354: [SPARK-23243][CORE][2.3] Fix RDD.repartition() data corr...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22354 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2913/

[GitHub] spark issue #22355: [SPARK-25358][SQL] MutableProjection supports fallback t...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22355 **[Test build #95765 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95765/testReport)** for PR 22355 at commit [`c25c426`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #22355: [SPARK-25358][SQL] MutableProjection supports fallback t...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22355 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95765/ Test PASSed. ---

[GitHub] spark issue #22355: [SPARK-25358][SQL] MutableProjection supports fallback t...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22355 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binary...

2018-09-06 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22356#discussion_r215737727 --- Diff: core/src/test/scala/org/apache/spark/FileSuite.scala --- @@ -299,6 +301,25 @@ class FileSuite extends SparkFunSuite with LocalSparkContext {

[GitHub] spark pull request #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binary...

2018-09-06 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22356#discussion_r215737907 --- Diff: core/src/test/scala/org/apache/spark/FileSuite.scala --- @@ -299,6 +301,25 @@ class FileSuite extends SparkFunSuite with LocalSparkContext {

[GitHub] spark issue #22144: [SPARK-24935][SQL] : Problem with Executing Hive UDF's f...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22144 **[Test build #95763 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95763/testReport)** for PR 22144 at commit [`173035d`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #22144: [SPARK-24935][SQL] : Problem with Executing Hive UDF's f...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22144 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22144: [SPARK-24935][SQL] : Problem with Executing Hive UDF's f...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22144 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95763/ Test PASSed. ---

[GitHub] spark issue #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binaryFiles i...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22356 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binaryFiles i...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22356 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2914/

[GitHub] spark issue #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binaryFiles i...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22356 **[Test build #95771 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95771/testReport)** for PR 22356 at commit [`6e1d8fd`](https://github.com/apache/spark/commit/6e

[GitHub] spark issue #22165: [SPARK-25017][Core] Add test suite for BarrierCoordinato...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22165 **[Test build #95761 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95761/testReport)** for PR 22165 at commit [`ecf12bd`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #22165: [SPARK-25017][Core] Add test suite for BarrierCoordinato...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22165 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95761/ Test PASSed. ---

[GitHub] spark issue #22165: [SPARK-25017][Core] Add test suite for BarrierCoordinato...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22165 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21899: [SPARK-24912][SQL] Don't obscure source of OOM during br...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21899 **[Test build #95773 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95773/testReport)** for PR 21899 at commit [`e927cd3`](https://github.com/apache/spark/commit/e9

[GitHub] spark issue #22192: [SPARK-24918][Core] Executor Plugin API

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22192 **[Test build #95772 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95772/testReport)** for PR 22192 at commit [`b0be9e5`](https://github.com/apache/spark/commit/b0

[GitHub] spark issue #22192: [SPARK-24918][Core] Executor Plugin API

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22192 **[Test build #95772 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95772/testReport)** for PR 22192 at commit [`b0be9e5`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #22192: [SPARK-24918][Core] Executor Plugin API

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22192 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95772/ Test FAILed. ---

[GitHub] spark issue #22192: [SPARK-24918][Core] Executor Plugin API

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22192 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binary...

2018-09-06 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/22356#discussion_r215749840 --- Diff: core/src/test/scala/org/apache/spark/FileSuite.scala --- @@ -299,6 +301,25 @@ class FileSuite extends SparkFunSuite with LocalSparkContext {

[GitHub] spark pull request #22192: [SPARK-24918][Core] Executor Plugin API

2018-09-06 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/22192#discussion_r215749405 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -136,6 +136,32 @@ private[spark] class Executor( // for fetching remote

[GitHub] spark pull request #22192: [SPARK-24918][Core] Executor Plugin API

2018-09-06 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/22192#discussion_r215749556 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -136,6 +136,32 @@ private[spark] class Executor( // for fetching remote

[GitHub] spark pull request #22192: [SPARK-24918][Core] Executor Plugin API

2018-09-06 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/22192#discussion_r215750952 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -218,6 +244,8 @@ private[spark] class Executor( env.metricsSystem.repo

[GitHub] spark issue #22227: [SPARK-25202] [SQL] Implements split with limit sql func...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/7 **[Test build #95774 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95774/testReport)** for PR 7 at commit [`64b0afc`](https://github.com/apache/spark/commit/64

[GitHub] spark pull request #22192: [SPARK-24918][Core] Executor Plugin API

2018-09-06 Thread NiharS
Github user NiharS commented on a diff in the pull request: https://github.com/apache/spark/pull/22192#discussion_r215760787 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -218,6 +244,8 @@ private[spark] class Executor( env.metricsSystem.repor

[GitHub] spark pull request #22227: [SPARK-25202] [SQL] Implements split with limit s...

2018-09-06 Thread phegstrom
Github user phegstrom commented on a diff in the pull request: https://github.com/apache/spark/pull/7#discussion_r215764191 --- Diff: python/pyspark/sql/functions.py --- @@ -1669,20 +1669,33 @@ def repeat(col, n): return Column(sc._jvm.functions.repeat(_to_java_column(c

[GitHub] spark issue #22162: [spark-24442][SQL] Added parameters to control the defau...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22162 **[Test build #95775 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95775/testReport)** for PR 22162 at commit [`f93e58e`](https://github.com/apache/spark/commit/f9

[GitHub] spark issue #22352: [SPARK-25208][SQL][FOLLOW-UP] Reduce code size.

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22352 **[Test build #95766 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95766/testReport)** for PR 22352 at commit [`ee47126`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #22352: [SPARK-25208][SQL][FOLLOW-UP] Reduce code size.

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22352 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95766/ Test PASSed. ---

[GitHub] spark issue #22352: [SPARK-25208][SQL][FOLLOW-UP] Reduce code size.

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22352 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22010: [SPARK-21436][CORE] Take advantage of known partitioner ...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22010 **[Test build #95767 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95767/testReport)** for PR 22010 at commit [`7ed7589`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #22010: [SPARK-21436][CORE] Take advantage of known partitioner ...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22010 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-

[GitHub] spark issue #22010: [SPARK-21436][CORE] Take advantage of known partitioner ...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22010 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95767/ Test FAILed. ---

[GitHub] spark issue #22010: [SPARK-21436][CORE] Take advantage of known partitioner ...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22010 **[Test build #95768 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95768/testReport)** for PR 22010 at commit [`4c89653`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #22010: [SPARK-21436][CORE] Take advantage of known partitioner ...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22010 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95768/ Test PASSed. ---

[GitHub] spark issue #22010: [SPARK-21436][CORE] Take advantage of known partitioner ...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22010 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22146: [SPARK-24434][K8S] pod template files

2018-09-06 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/22146 @mccheah @erikerlandson given that banch-2.4 has already been cut, we have more flexibility in terms of timing of getting this in, as well as more space for thoughts to get this right. I don't st

[GitHub] spark issue #21221: [SPARK-23429][CORE] Add executor memory metrics to heart...

2018-09-06 Thread mccheah
Github user mccheah commented on the issue: https://github.com/apache/spark/pull/21221 @edwinalu - this can merge now that Spark 2.4's release branch has been cut, but there's conflicting files now. Can we clear the conflicts and then we can merge this? ---

[GitHub] spark issue #22354: [SPARK-23243][CORE][2.3] Fix RDD.repartition() data corr...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22354 **[Test build #95770 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95770/testReport)** for PR 22354 at commit [`1e01b00`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #22354: [SPARK-23243][CORE][2.3] Fix RDD.repartition() data corr...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22354 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22354: [SPARK-23243][CORE][2.3] Fix RDD.repartition() data corr...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22354 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95770/ Test PASSed. ---

[GitHub] spark pull request #22323: [SPARK-25262][K8S] Allow SPARK_LOCAL_DIRS to be t...

2018-09-06 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/22323#discussion_r215799370 --- Diff: docs/running-on-kubernetes.md --- @@ -215,6 +215,19 @@ spark.kubernetes.driver.volumes.persistentVolumeClaim.checkpointpvc.options.clai

[GitHub] spark issue #21899: [SPARK-24912][SQL] Don't obscure source of OOM during br...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21899 **[Test build #95773 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95773/testReport)** for PR 21899 at commit [`e927cd3`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #21899: [SPARK-24912][SQL] Don't obscure source of OOM during br...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21899 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95773/ Test PASSed. ---

[GitHub] spark issue #21899: [SPARK-24912][SQL] Don't obscure source of OOM during br...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21899 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21308: [SPARK-24253][SQL] Add DeleteSupport mix-in for DataSour...

2018-09-06 Thread tigerquoll
Github user tigerquoll commented on the issue: https://github.com/apache/spark/pull/21308 @rdblue what about those data sources that support record deletion and partition dropping as two semantically different operations - Kudu and Hbase being two examples. All systems t

[GitHub] spark issue #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binaryFiles i...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22356 **[Test build #95769 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95769/testReport)** for PR 22356 at commit [`84dd4a7`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binaryFiles i...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22356 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binaryFiles i...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22356 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95769/ Test PASSed. ---

[GitHub] spark issue #21308: [SPARK-24253][SQL] Add DeleteSupport mix-in for DataSour...

2018-09-06 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/21308 @tigerquoll, there is currently no support to expose partitions through the v2 API. That would be a different operation. If you wanted to implement partition operations through this API, then you wou

[GitHub] spark issue #22298: [SPARK-25021][K8S] Add spark.executor.pyspark.memory lim...

2018-09-06 Thread mccheah
Github user mccheah commented on the issue: https://github.com/apache/spark/pull/22298 Yeah this looks ok to me, would like a +1 from @felixcheung and @holdenk. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark

[GitHub] spark issue #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binaryFiles i...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22356 **[Test build #95771 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95771/testReport)** for PR 22356 at commit [`6e1d8fd`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binaryFiles i...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22356 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22215: [SPARK-25222][K8S] Improve container status logging

2018-09-06 Thread mccheah
Github user mccheah commented on the issue: https://github.com/apache/spark/pull/22215 Yup this can merge now, thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mai

[GitHub] spark issue #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binaryFiles i...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22356 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95771/ Test PASSed. ---

[GitHub] spark pull request #22323: [SPARK-25262][K8S] Allow SPARK_LOCAL_DIRS to be t...

2018-09-06 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/22323#discussion_r215806032 --- Diff: docs/running-on-kubernetes.md --- @@ -215,6 +215,19 @@ spark.kubernetes.driver.volumes.persistentVolumeClaim.checkpointpvc.options.clai

[GitHub] spark issue #22323: [SPARK-25262][K8S] Allow SPARK_LOCAL_DIRS to be tmpfs ba...

2018-09-06 Thread mccheah
Github user mccheah commented on the issue: https://github.com/apache/spark/pull/22323 I'm going to merge this, for Spark 3.0 we should look at https://github.com/apache/spark/pull/22323#discussion_r215799370 though. --- --

[GitHub] spark pull request #22215: [SPARK-25222][K8S] Improve container status loggi...

2018-09-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22215 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binaryFiles i...

2018-09-06 Thread bomeng
Github user bomeng commented on the issue: https://github.com/apache/spark/pull/22356 Thanks for taking my codes. Looks good. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional command

[GitHub] spark pull request #22323: [SPARK-25262][K8S] Allow SPARK_LOCAL_DIRS to be t...

2018-09-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22323 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22219 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21308: [SPARK-24253][SQL] Add DeleteSupport mix-in for DataSour...

2018-09-06 Thread tigerquoll
Github user tigerquoll commented on the issue: https://github.com/apache/spark/pull/21308 @rdblue Actually: https://issues.apache.org/jira/browse/SPARK-22389. --- - To unsubscribe, e-mail: reviews-unsubscr...@

[GitHub] spark issue #22192: [SPARK-24918][Core] Executor Plugin API

2018-09-06 Thread NiharS
Github user NiharS commented on the issue: https://github.com/apache/spark/pull/22192 I tested @vanzin's suggestion of storing the classloader, changing it to replClassLoader, and changing back after init (and then changing again for shutdown). It's working well (and this time I made

[GitHub] spark issue #21308: [SPARK-24253][SQL] Add DeleteSupport mix-in for DataSour...

2018-09-06 Thread tigerquoll
Github user tigerquoll commented on the issue: https://github.com/apache/spark/pull/21308 @rdblue I think our debate is whether we should expose an API to represent direct operations on partitions in the new datasource api. --- -

[GitHub] spark issue #22318: [SPARK-25150][SQL] Rewrite condition when deduplicate Jo...

2018-09-06 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22318 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache

[GitHub] spark pull request #22350: [SPARK-25356][SQL]Add Parquet block size option t...

2018-09-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/22350#discussion_r215812113 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala --- @@ -123,6 +123,9 @@ class ParquetFileF

[GitHub] spark issue #21221: [SPARK-23429][CORE] Add executor memory metrics to heart...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21221 **[Test build #95776 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95776/testReport)** for PR 21221 at commit [`ee4aa1d`](https://github.com/apache/spark/commit/ee

[GitHub] spark issue #22348: [SPARK-25354][SQL] Reduce unneeded operation in nextKeyV...

2018-09-06 Thread SongYadong
Github user SongYadong commented on the issue: https://github.com/apache/spark/pull/22348 In my test, total executing time of 4885 spark applications: origin: 447213 seconds after : 444584 seconds time saved : about 0.6% --- --

[GitHub] spark pull request #22138: [SPARK-25151][SS] Apply Apache Commons Pool to Ka...

2018-09-06 Thread HeartSaVioR
Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/spark/pull/22138#discussion_r215818860 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/FetchedPoolSuite.scala --- @@ -0,0 +1,299 @@ +/* + * Licensed to

[GitHub] spark issue #22227: [SPARK-25202] [SQL] Implements split with limit sql func...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/7 **[Test build #95774 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95774/testReport)** for PR 7 at commit [`64b0afc`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #22227: [SPARK-25202] [SQL] Implements split with limit sql func...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/7 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22227: [SPARK-25202] [SQL] Implements split with limit sql func...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/7 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95774/ Test FAILed. ---

[GitHub] spark issue #22138: [SPARK-25151][SS] Apply Apache Commons Pool to KafkaData...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22138 **[Test build #95777 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95777/testReport)** for PR 22138 at commit [`5b56bd0`](https://github.com/apache/spark/commit/5b

[GitHub] spark pull request #22350: [SPARK-25356][SQL]Add Parquet block size option t...

2018-09-06 Thread 10110346
Github user 10110346 commented on a diff in the pull request: https://github.com/apache/spark/pull/22350#discussion_r215819785 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala --- @@ -123,6 +123,9 @@ class ParquetFileForm

[GitHub] spark pull request #22350: [SPARK-25356][SQL]Add Parquet block size option t...

2018-09-06 Thread 10110346
Github user 10110346 closed the pull request at: https://github.com/apache/spark/pull/22350 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21320: [SPARK-4502][SQL] Parquet nested column pruning - founda...

2018-09-06 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21320 Thanks @mallman for schema pruning work which will be a big win in our pattern of accessing our data. I'm testing this new feature, and find `where clause` on the selected nested column can

[GitHub] spark issue #22162: [spark-24442][SQL] Added parameters to control the defau...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22162 **[Test build #95775 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95775/testReport)** for PR 22162 at commit [`f93e58e`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #22162: [spark-24442][SQL] Added parameters to control the defau...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22162 Build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-

[GitHub] spark issue #22162: [spark-24442][SQL] Added parameters to control the defau...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22162 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95775/ Test PASSed. ---

[GitHub] spark issue #22140: [SPARK-25072][PySpark] Forbid extra value for custom Row

2018-09-06 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/22140 Thanks @BryanCutler @HyukjinKwon ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional command

[GitHub] spark issue #22326: [SPARK-25314][SQL] Fix Python UDF accessing attibutes fr...

2018-09-06 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/22326 Gental ping @mgaido91 @HyukjinKwon @dilipbiswal, great thanks for advice, please have a look when you have time. --- - To u

[GitHub] spark issue #22352: [SPARK-25208][SQL][FOLLOW-UP] Reduce code size.

2018-09-06 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22352 thanks, merging to master/2.4 (since it's a followup) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For ad

[GitHub] spark pull request #22352: [SPARK-25208][SQL][FOLLOW-UP] Reduce code size.

2018-09-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22352 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18756: [SPARK-21548][SQL] "Support insert into serial columns o...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18756 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22329: [SPARK-25328][PYTHON] Add an example for having two colu...

2018-09-06 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22329 Thanks guys :-) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22311: [WIP][SPARK-25305][SQL] Respect attribute name in Collap...

2018-09-06 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/22311 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: rev

[GitHub] spark issue #22311: [WIP][SPARK-25305][SQL] Respect attribute name in Collap...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22311 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2915/

[GitHub] spark issue #22311: [WIP][SPARK-25305][SQL] Respect attribute name in Collap...

2018-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22311 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22311: [WIP][SPARK-25305][SQL] Respect attribute name in Collap...

2018-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22311 **[Test build #95778 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95778/testReport)** for PR 22311 at commit [`96b4cca`](https://github.com/apache/spark/commit/96

[GitHub] spark issue #22354: [SPARK-23243][CORE][2.3] Fix RDD.repartition() data corr...

2018-09-06 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22354 thanks, merging to 2.3! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: rev

[GitHub] spark pull request #22354: [SPARK-23243][CORE][2.3] Fix RDD.repartition() da...

2018-09-06 Thread cloud-fan
Github user cloud-fan closed the pull request at: https://github.com/apache/spark/pull/22354 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20999: [SPARK-14922][SPARK-17732][SPARK-23866][SQL] Support par...

2018-09-06 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20999 you're not wrong, but I just meant you don't need to describe "Credit for that should be given to him." in the PR description. --- --

[GitHub] spark issue #22327: [SPARK-25330][BUILD][BRANCH-2.3] Revert Hadoop 2.7 to 2....

2018-09-06 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22327 From further discussion on the JIRA, looks like there is not an obvious immediate resolution, and we are about to release 2.3.2 and 2.4, and have a nontrivial problem vs a theoretical set of bug fixe

[GitHub] spark issue #22354: [SPARK-23243][CORE][2.3] Fix RDD.repartition() data corr...

2018-09-06 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22354 cc @jerryshao --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h.

[GitHub] spark issue #22227: [SPARK-25202] [SQL] Implements split with limit sql func...

2018-09-06 Thread phegstrom
Github user phegstrom commented on the issue: https://github.com/apache/spark/pull/7 hey @HyukjinKwon , the SparkR test now failing is ``` Duplicated \argument entries in documentation object 'column_string_functions': 'pattern' Functions with \usage entries

[GitHub] spark pull request #22349: [SPARK-25345][ML] Deprecate public APIs from Imag...

2018-09-06 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/22349#discussion_r215840879 --- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala --- @@ -35,6 +35,8 @@ import org.apache.spark.sql.types._ */ @Experimen

[GitHub] spark pull request #20876: [SPARK-23653][SQL] Capture sql statements user in...

2018-09-06 Thread LantaoJin
Github user LantaoJin closed the pull request at: https://github.com/apache/spark/pull/20876 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

<    1   2   3   4   5   >