[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21990 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21990 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97510/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #97510 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97510/testReport)** for PR 21990 at commit [`3629c78`](https://github.com/apache/spark/commit/3629c78118eb77589ae1126e877f0fd2b57441ce). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21990 LGTM. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #97510 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97510/testReport)** for PR 21990 at commit [`3629c78`](https://github.com/apache/spark/commit/3629c78118eb77589ae1126e877f0fd2b57441ce). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21990 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97497/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #97497 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97497/testReport)** for PR 21990 at commit [`3629c78`](https://github.com/apache/spark/commit/3629c78118eb77589ae1126e877f0fd2b57441ce). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #97497 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97497/testReport)** for PR 21990 at commit [`3629c78`](https://github.com/apache/spark/commit/3629c78118eb77589ae1126e877f0fd2b57441ce). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97472/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #97472 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97472/testReport)** for PR 21990 at commit [`d9b2a55`](https://github.com/apache/spark/commit/d9b2a55275b74c406d9f9c435bf1b53a6ef4b35a). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #97472 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97472/testReport)** for PR 21990 at commit [`d9b2a55`](https://github.com/apache/spark/commit/d9b2a55275b74c406d9f9c435bf1b53a6ef4b35a). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user RussellSpitzer commented on the issue: https://github.com/apache/spark/pull/21990 Addressed Comments from @HyukjinKwon , I'm interested in @ueshin 's suggestions, but I can't figure out how to do that unless we bake it into the Extensions constructor. If we place it in the Sessions constructor then invocations of "newSession" will reapply existing extensions. I added a note in the code. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #97471 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97471/testReport)** for PR 21990 at commit [`8ab76d4`](https://github.com/apache/spark/commit/8ab76d4385d1849c43ffd940d539c143e0ba8c81). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97471/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #97471 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97471/testReport)** for PR 21990 at commit [`8ab76d4`](https://github.com/apache/spark/commit/8ab76d4385d1849c43ffd940d539c143e0ba8c81). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96827/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #96827 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96827/testReport)** for PR 21990 at commit [`cf7cf75`](https://github.com/apache/spark/commit/cf7cf75efc616151d75e1da10630731fa98e117e). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96828/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #96828 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96828/testReport)** for PR 21990 at commit [`4ddaff8`](https://github.com/apache/spark/commit/4ddaff8fcd1bfedc221eda7d6f37e5e9f28bfb4e). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #96828 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96828/testReport)** for PR 21990 at commit [`4ddaff8`](https://github.com/apache/spark/commit/4ddaff8fcd1bfedc221eda7d6f37e5e9f28bfb4e). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #96827 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96827/testReport)** for PR 21990 at commit [`cf7cf75`](https://github.com/apache/spark/commit/cf7cf75efc616151d75e1da10630731fa98e117e). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96825/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #96825 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96825/testReport)** for PR 21990 at commit [`27c9d28`](https://github.com/apache/spark/commit/27c9d28430d5c0cc5404947d9ec690d53a626c2d). * This patch **fails to build**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #96825 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96825/testReport)** for PR 21990 at commit [`27c9d28`](https://github.com/apache/spark/commit/27c9d28430d5c0cc5404947d9ec690d53a626c2d). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #96824 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96824/testReport)** for PR 21990 at commit [`8a26728`](https://github.com/apache/spark/commit/8a2672806c6513692435bef113935f2e6f67a910). * This patch **fails to build**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96824/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #96824 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96824/testReport)** for PR 21990 at commit [`8a26728`](https://github.com/apache/spark/commit/8a2672806c6513692435bef113935f2e6f67a910). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21990 Looks close to go. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user RussellSpitzer commented on the issue: https://github.com/apache/spark/pull/21990 I'm fine with anything really, I still think the ideal solution is probably not to tie the creation of the py4j gateway to the SparkContext, but that's probably a much bigger refactor. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/21990 I'm +1 on switching to the builder and not using the private interface. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96257/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #96257 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96257/testReport)** for PR 21990 at commit [`67d9772`](https://github.com/apache/spark/commit/67d9772e1f470bba83e311cd10cfe586a7f11b92). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #96257 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96257/testReport)** for PR 21990 at commit [`67d9772`](https://github.com/apache/spark/commit/67d9772e1f470bba83e311cd10cfe586a7f11b92). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #96253 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96253/testReport)** for PR 21990 at commit [`7775c08`](https://github.com/apache/spark/commit/7775c0886a7ae2be35e3caa974763f7b23954c6f). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96253/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #96253 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96253/testReport)** for PR 21990 at commit [`7775c08`](https://github.com/apache/spark/commit/7775c0886a7ae2be35e3caa974763f7b23954c6f). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user RussellSpitzer commented on the issue: https://github.com/apache/spark/pull/21990 Added new method of injecting extensions, this way the "getOrCreate" code from the scala method is not needed. @HyukjinKwon --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #96199 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96199/testReport)** for PR 21990 at commit [`def4f3e`](https://github.com/apache/spark/commit/def4f3e91b38e7d63c03fc7206489fd01220e878). * This patch passes all tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class SparkExtensionsTest(unittest.TestCase, SQLTestUtils):` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96199/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user RussellSpitzer commented on the issue: https://github.com/apache/spark/pull/21990 @HyukjinKwon so you want me to rewrite the code in python? I will note SparkR is doing this exact same thing. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #96199 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96199/testReport)** for PR 21990 at commit [`def4f3e`](https://github.com/apache/spark/commit/def4f3e91b38e7d63c03fc7206489fd01220e878). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21990 > We could just translate the rest of GetOrCreate into Python but then every time there is a patch of the code in scala it will need a Python mod as well. @RussellSpitzer, I actually think we already duplicate some codes in Python side and Scala side at this code path now (see `getOrCreate` at Scala side and `__init__` at Python side). They are now duplicatedly executed with this change. I believe this isn't a orthogonal stuff to handle separately. If that requires a bit of duplicated codes in Python side to avoid duplicated code path executions, then I think that can be a workaround to get through for now. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user RussellSpitzer commented on the issue: https://github.com/apache/spark/pull/21990 What I wanted was to just call the Scala Methods, instead of having half the code and half in python, but we create the JVM in the SparkContext creation code so this ends up not being a good method I think. We could just translate the rest of GetOrCreate into Python but then every time there is a patch of the code in scala it will need a Python mod as well. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user RussellSpitzer commented on the issue: https://github.com/apache/spark/pull/21990 @HyukjinKwon So i've been staring at this for a while today, and I guess the big issue is that we always need to make a Python SparkContext to get a handle on the JavaGateway, so everything that happens before the context is made cannot just be a wrapper of SQL methods and must reimplement. Unless we decided to refactor the code so that the JVM is more generally available (probably not possible) we will be stuck with redoing code in python ... --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94975/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #94975 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94975/testReport)** for PR 21990 at commit [`21ff627`](https://github.com/apache/spark/commit/21ff6273a7fcf54c2f40abde951d4aab89f96a02). * This patch passes all tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class SparkExtensionsTest(unittest.TestCase, SQLTestUtils):` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #94975 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94975/testReport)** for PR 21990 at commit [`21ff627`](https://github.com/apache/spark/commit/21ff6273a7fcf54c2f40abde951d4aab89f96a02). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94873/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #94873 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94873/testReport)** for PR 21990 at commit [`0eea205`](https://github.com/apache/spark/commit/0eea205ca0591c68975412873b34393f6bf19437). * This patch passes all tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class SparkExtensionsTest(unittest.TestCase, SQLTestUtils):` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #94873 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94873/testReport)** for PR 21990 at commit [`0eea205`](https://github.com/apache/spark/commit/0eea205ca0591c68975412873b34393f6bf19437). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #94870 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94870/testReport)** for PR 21990 at commit [`d5c37b7`](https://github.com/apache/spark/commit/d5c37b732f1948d8240bd8de33a080ac5db03571). * This patch **fails Python style tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class SparkExtensionsTest(unittest.TestCase, SQLTestUtils):` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94870/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #94870 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94870/testReport)** for PR 21990 at commit [`d5c37b7`](https://github.com/apache/spark/commit/d5c37b732f1948d8240bd8de33a080ac5db03571). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94866/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #94866 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94866/testReport)** for PR 21990 at commit [`7ba70b5`](https://github.com/apache/spark/commit/7ba70b524f9779529142f6c70b04610b5b068a05). * This patch **fails Python style tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class SparkExtensionsTest(unittest.TestCase, SQLTestUtils):` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #94866 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94866/testReport)** for PR 21990 at commit [`7ba70b5`](https://github.com/apache/spark/commit/7ba70b524f9779529142f6c70b04610b5b068a05). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21990 @RussellSpitzer, also please ping me here if you face any difficulties. am wiling to help or push some changes to your branch. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94151/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #94151 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94151/testReport)** for PR 21990 at commit [`84c2513`](https://github.com/apache/spark/commit/84c2513a2cb7a50ba01cdc0d49e2ad7f9583799b). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #94151 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94151/testReport)** for PR 21990 at commit [`84c2513`](https://github.com/apache/spark/commit/84c2513a2cb7a50ba01cdc0d49e2ad7f9583799b). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94149/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #94149 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94149/testReport)** for PR 21990 at commit [`f790ae0`](https://github.com/apache/spark/commit/f790ae000ae1c3f4030162028186448d345e2984). * This patch **fails Python style tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #94149 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94149/testReport)** for PR 21990 at commit [`f790ae0`](https://github.com/apache/spark/commit/f790ae000ae1c3f4030162028186448d345e2984). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
