[GitHub] spark pull request #22025: SPARK-25043: print master and appId from spark-sq...

2018-08-07 Thread abellina
GitHub user abellina opened a pull request: https://github.com/apache/spark/pull/22025 SPARK-25043: print master and appId from spark-sql on startup ## What changes were proposed in this pull request? A small change to print the master and appId from spark-sql as with

[GitHub] spark pull request #22026: [SPARK-25045][CORE] Make `RDDBarrier.mapParititio...

2018-08-07 Thread jiangxb1987
GitHub user jiangxb1987 opened a pull request: https://github.com/apache/spark/pull/22026 [SPARK-25045][CORE] Make `RDDBarrier.mapParititions` similar to `RDD.mapPartitions` ## What changes were proposed in this pull request? Signature of the function passed to

[GitHub] spark pull request #21954: [SPARK-23908][SQL] Add transform function.

2018-08-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21954#discussion_r208268129 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -0,0 +1,212 @@ +/* + *

[GitHub] spark pull request #21954: [SPARK-23908][SQL] Add transform function.

2018-08-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21954#discussion_r208269300 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -0,0 +1,212 @@ +/* + *

[GitHub] spark issue #22026: [SPARK-25045][CORE] Make `RDDBarrier.mapParititions` sim...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22026 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22026: [SPARK-25045][CORE] Make `RDDBarrier.mapParititions` sim...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22026 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22022: [SPARK-24948][SHS][BACKPORT-2.2] Delegate check access p...

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22022 **[Test build #94369 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94369/testReport)** for PR 22022 at commit

[GitHub] spark issue #22022: [SPARK-24948][SHS][BACKPORT-2.2] Delegate check access p...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22022 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21305 **[Test build #94384 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94384/testReport)** for PR 21305 at commit

[GitHub] spark issue #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21305 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21305 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20611: [SPARK-23425][SQL]Support wildcard in HDFS path for load...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20611 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94359/ Test PASSed. ---

[GitHub] spark issue #20611: [SPARK-23425][SQL]Support wildcard in HDFS path for load...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20611 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22009 **[Test build #94377 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94377/testReport)** for PR 22009 at commit

[GitHub] spark issue #22023: [SPARK-23928][TESTS][FOLLOWUP] Set seed to avoid flakine...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22023 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94366/ Test PASSed. ---

[GitHub] spark issue #22023: [SPARK-23928][TESTS][FOLLOWUP] Set seed to avoid flakine...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22023 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22023: [SPARK-23928][TESTS][FOLLOWUP] Set seed to avoid ...

2018-08-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22023 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spark on K8...

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21669 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/1917/ ---

[GitHub] spark issue #22022: [SPARK-24948][SHS][BACKPORT-2.2] Delegate check access p...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22022 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21608: [SPARK-24626] [SQL] Improve location size calcula...

2018-08-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21608#discussion_r208269963 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceUtils.scala --- @@ -49,4 +51,11 @@ object DataSourceUtils {

[GitHub] spark issue #22022: [SPARK-24948][SHS][BACKPORT-2.2] Delegate check access p...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22022 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94369/ Test FAILed. ---

[GitHub] spark issue #21986: [SPARK-23937][SQL] Add map_filter SQL function

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21986 **[Test build #94363 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94363/testReport)** for PR 21986 at commit

[GitHub] spark issue #22025: SPARK-25043: print master and appId from spark-sql on st...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22025 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22025: SPARK-25043: print master and appId from spark-sql on st...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22025 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22023: [SPARK-23928][TESTS][FOLLOWUP] Set seed to avoid flakine...

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22023 **[Test build #94366 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94366/testReport)** for PR 22023 at commit

[GitHub] spark issue #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spark on K8...

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21669 **[Test build #94379 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94379/testReport)** for PR 21669 at commit

[GitHub] spark pull request #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spar...

2018-08-07 Thread ifilonenko
Github user ifilonenko commented on a diff in the pull request: https://github.com/apache/spark/pull/21669#discussion_r208257021 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -336,7 +336,7 @@ private[spark] class SparkSubmit extends Logging {

[GitHub] spark pull request #22017: [SPARK-23938][SQL] Add map_zip_with function

2018-08-07 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/22017#discussion_r208260664 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -365,3 +364,101 @@ case class

[GitHub] spark pull request #22017: [SPARK-23938][SQL] Add map_zip_with function

2018-08-07 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/22017#discussion_r208257687 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeExtractors.scala --- @@ -267,22 +267,23 @@ case class

[GitHub] spark issue #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spark on K8...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21669 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spark on K8...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21669 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spark on K8...

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21669 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/1917/ ---

[GitHub] spark issue #22026: [SPARK-25045][CORE] Make `RDDBarrier.mapParititions` sim...

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22026 **[Test build #94380 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94380/testReport)** for PR 22026 at commit

[GitHub] spark pull request #21954: [SPARK-23908][SQL] Add transform function.

2018-08-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21954#discussion_r208267632 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -0,0 +1,212 @@ +/* + *

[GitHub] spark issue #20611: [SPARK-23425][SQL]Support wildcard in HDFS path for load...

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20611 **[Test build #94359 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94359/testReport)** for PR 20611 at commit

[GitHub] spark issue #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22009 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22009 Build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-07 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22009 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22008: [SPARK-24928][SQL] Optimize cross join according to stat...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22008 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22008: [SPARK-24928][SQL] Optimize cross join according to stat...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22008 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94365/ Test FAILed. ---

[GitHub] spark issue #22008: [SPARK-24928][SQL] Optimize cross join according to stat...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22008 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22008: [SPARK-24928][SQL] Optimize cross join according to stat...

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22008 **[Test build #94378 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94378/testReport)** for PR 22008 at commit

[GitHub] spark issue #22025: SPARK-25043: print master and appId from spark-sql on st...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22025 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22023: [SPARK-23928][TESTS][FOLLOWUP] Set seed to avoid flakine...

2018-08-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22023 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21954: [SPARK-23908][SQL] Add transform function.

2018-08-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21954#discussion_r208266333 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -0,0 +1,212 @@ +/* + *

[GitHub] spark issue #18323: [SPARK-21117][SQL] Built-in SQL Function Support - WIDTH...

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18323 **[Test build #94376 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94376/testReport)** for PR 18323 at commit

[GitHub] spark issue #22021: [SPARK-24948][SHS][BACKPORT-2.3] Delegate check access p...

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22021 **[Test build #94370 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94370/testReport)** for PR 22021 at commit

[GitHub] spark issue #22006: [SPARK-25031][SQL] Fix MapType schema print

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22006 **[Test build #94368 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94368/testReport)** for PR 22006 at commit

[GitHub] spark issue #21978: SPARK-25006: Add CatalogTableIdentifier.

2018-08-07 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/21978 @cloud-fan, that's fine with me since #17185 is already merged. Would this conflict with #17185? We can just add a case that detects whether the first identifier in the seq is a catalog when

[GitHub] spark issue #22027: [SPARK-25010][SQL][FOLLOWUP] Shuffle should also produce...

2018-08-07 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/22027 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21986: [SPARK-23937][SQL] Add map_filter SQL function

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21986 **[Test build #94371 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94371/testReport)** for PR 21986 at commit

[GitHub] spark issue #21986: [SPARK-23937][SQL] Add map_filter SQL function

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21986 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22024: [SPARK-25034][CORE] Remove allocations in onBlockFetchSu...

2018-08-07 Thread vincent-grosbois
Github user vincent-grosbois commented on the issue: https://github.com/apache/spark/pull/22024 Actually I spoke too soon! We spotted some regression in TorrentBroadcast:

[GitHub] spark issue #21986: [SPARK-23937][SQL] Add map_filter SQL function

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21986 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94371/ Test PASSed. ---

[GitHub] spark pull request #21977: SPARK-25004: Add spark.executor.pyspark.memory li...

2018-08-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/21977#discussion_r208302473 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala --- @@ -60,14 +61,26 @@ private[spark] object PythonEvalType { */

[GitHub] spark issue #21991: [SPARK-25018] [Infra] Use `Co-authored-by` and `Signed-o...

2018-08-07 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21991 Thanks all! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22030: [SPARK-25048][SQL] Pivoting by multiple columns in Scala...

2018-08-07 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/22030 @maryannxue Please, have a look at the PR. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22030: [SPARK-25048][SQL] Pivoting by multiple columns in Scala...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22030 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21845: [SPARK-24886][INFRA] Fix the testing script to increase ...

2018-08-07 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/21845 i'm also more than happy to bump the timeout in the PRB build, but i think that's just putting duct tape on a band-aid and spray painting it to hide the layers of tape. the builds and

[GitHub] spark issue #22020: [SPARK-25041][build] upgrade genJavaDoc-plugin from 0.10...

2018-08-07 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22020 Merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21305 **[Test build #94373 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94373/testReport)** for PR 21305 at commit

[GitHub] spark issue #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21305 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21986: [SPARK-23937][SQL] Add map_filter SQL function

2018-08-07 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21986 Thanks! merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21978: SPARK-25006: Add CatalogTableIdentifier.

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21978 **[Test build #94375 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94375/testReport)** for PR 21978 at commit

[GitHub] spark pull request #21986: [SPARK-23937][SQL] Add map_filter SQL function

2018-08-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21986 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.

2018-08-07 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/21977 Okay, workers are tracked in `PythonWorkerFactory`. Its `create` method returns an idle worker if one is available. When a task finishes, it calls `SparkEnv.releasePythonWorker` that calls

[GitHub] spark issue #21978: SPARK-25006: Add CatalogTableIdentifier.

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21978 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94375/ Test PASSed. ---

[GitHub] spark pull request #21608: [SPARK-24626] [SQL] Improve location size calcula...

2018-08-07 Thread Achuth17
Github user Achuth17 commented on a diff in the pull request: https://github.com/apache/spark/pull/21608#discussion_r208295767 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceUtils.scala --- @@ -49,4 +51,11 @@ object DataSourceUtils {

[GitHub] spark issue #18323: [SPARK-21117][SQL] Built-in SQL Function Support - WIDTH...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18323 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94376/ Test FAILed. ---

[GitHub] spark issue #18323: [SPARK-21117][SQL] Built-in SQL Function Support - WIDTH...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18323 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #22021: [SPARK-24948][SHS][BACKPORT-2.3] Delegate check access p...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22021 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22021: [SPARK-24948][SHS][BACKPORT-2.3] Delegate check access p...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22021 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94370/ Test PASSed. ---

[GitHub] spark issue #22006: [SPARK-25031][SQL] Fix MapType schema print

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22006 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94368/ Test PASSed. ---

[GitHub] spark issue #22006: [SPARK-25031][SQL] Fix MapType schema print

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22006 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22030: [SPARK-25048][SQL] Pivoting by multiple columns in Scala...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22030 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22030: [SPARK-25048][SQL] Pivoting by multiple columns in Scala...

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22030 **[Test build #94385 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94385/testReport)** for PR 22030 at commit

[GitHub] spark pull request #21948: [SPARK-24991][SQL] use InternalRow in DataSourceW...

2018-08-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/21948#discussion_r208305810 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/MemorySinkV2Suite.scala --- @@ -44,16 +46,16 @@ class MemorySinkV2Suite extends

[GitHub] spark issue #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21305 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94373/ Test PASSed. ---

[GitHub] spark pull request #22020: [SPARK-25041][build] upgrade genJavaDoc-plugin fr...

2018-08-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22020 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21977: SPARK-25004: Add spark.executor.pyspark.memory li...

2018-08-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/21977#discussion_r208310790 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala --- @@ -60,14 +61,26 @@ private[spark] object PythonEvalType { */

[GitHub] spark issue #20184: [SPARK-22987][Core] UnsafeExternalSorter cases OOM when ...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20184 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.

2018-08-07 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/21977 Posting this as a comment instead of in a thread so it doesn't get lost. In response to @holdenk's question about memory allocation to workers: That `useDaemon` flag controls whether

[GitHub] spark issue #21978: SPARK-25006: Add CatalogTableIdentifier.

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21978 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

<    2   3   4   5   6   7