[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20404 Thanks all for your review, greatly appreciated. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20404 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/20404 I completely agree with @HyukjinKwon. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20404 How about you @ueshin? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20404 I am fine with not fixing the case here for now if we go for it only with master branch. Maybe, we can try to fix the pre existing issue too seperately by syncrhoziing everything if anyone feels strongly against within 2.4.0 timeline. To be honest, the case sounds rare and the fix could be a little bit overkill .. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20404 Thanks Felix. I would incline to not fix the case mentioned by Felix. What's your opinion @HyukjinKwon @ueshin ? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/20404 Itâs your call - there are some risks but seems like there are pre existing code that has the same problem. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20404 @felixcheung what is your opinion on this, do we really need to handle this case? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20404 I think the same issue also existed in Scala `SparkSession` code, because `setDefaultSession` doesn't hold a lock which holds by `getOrCreate` (SparkSession). For example: ```scala val globalSession = SparkSession.build.getOrCreate() val session1 = globalSession.newSession() globalSession.sparkContext.stop() // Thread1 val localSession = SparkSession.build.getOrCreate() // Thread2 set default session during Thread1's getOrCreate SparkSession.setDefaultSession(session1) // After Thread1's getOrCreate is finished, it will still overwrite the default session to "localSession", not the one (session1) user explicitly set ``` I think this is the similar scenario as mentioned for PySpark. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20404 Thanks all for your comments. I think @felixcheung 's case really makes thing complex, I'm not sure if user will use it in such way. I will try to address it. Appreciate your comments! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20404 Yup, sorry for rushing the suggestion. Wanted to show an idea. BTW, if we should replace it if it's already stopped, I assume we need the synchronization with `defaultSession` anyway if I didn't misunderstand. Did I maybe miss something? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/20404 @HyukjinKwon I'm afraid that's not enough because we need to synchronize at all the places we touch the `defaultSession`, otherwise another thread can set the value between `defaultSession.get() eq null` and `defaultSession.set(session)`. Fortunately `defaultSession` is an `AtomicReference`, so maybe we can use `defaultSession.compareAndSet(null, session)` without the synchronization in the method `setDefaultSessionIfUnset()` if we only need to set the default session is `null`. I'm not sure whether we want to replace the default session which is already stopped, but in that case, we may also have to send the original default session to compare. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20404
How about something like this?
```diff
--- a/python/pyspark/sql/session.py
+++ b/python/pyspark/sql/session.py
@@ -213,10 +213,10 @@ class SparkSession(object):
self._jsc = self._sc._jsc
self._jvm = self._sc._jvm
if jsparkSession is None:
-if self._jvm.SparkSession.getDefaultSession().isDefined() \
-and not
self._jvm.SparkSession.getDefaultSession().get() \
-.sparkContext().isStopped():
-jsparkSession =
self._jvm.SparkSession.getDefaultSession().get()
+default_session = self._jvm.SparkSession.getDefaultSession()
+if default_session.isDefined() \
+and not
default_session.get().sparkContext().isStopped():
+jsparkSession = default_session
else:
jsparkSession = self._jvm.SparkSession(self._jsc.sc())
self._jsparkSession = jsparkSession
@@ -230,7 +230,7 @@ class SparkSession(object):
if SparkSession._instantiatedSession is None \
or SparkSession._instantiatedSession._sc._jsc is None:
SparkSession._instantiatedSession = self
-self._jvm.SparkSession.setDefaultSession(self._jsparkSession)
+
self._jvm.SparkSession.setDefaultSessionIfUnset(self._jsparkSession)
def _repr_html_(self):
return """
diff --git
a/sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala
b/sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala
index 734573ba31f..99747ef88bc 100644
--- a/sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala
+++ b/sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala
@@ -1031,6 +1031,18 @@ object SparkSession {
// Private methods from now on
+ /**
+ * This if for Python blabla
+ */
+ private[sql] def setDefaultSessionIfUnset(session: SparkSession): Unit =
{
+// Global synchronization blabla
+SparkSession.synchronized {
+ if (defaultSession.get() eq null) {
+defaultSession.set(session)
+ }
+}
+ }
+
/** The active SparkSession for the current thread. */
private val activeThreadSession = new
InheritableThreadLocal[SparkSession]
```
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20404 For perfectness, I think we should hold a lock with JVM instance but I wonder if it's easily possible. I roughly knew this but I think underestimated this because I believe that will quite unlikely happens. I think reverting https://github.com/apache/spark/commit/cc4b8510c1445fb742c0d750958d352adfa84902 doesn't fully resolve the issue because I think the same thing can also happen between `if` and the next line. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/20404 @felixcheung I see, in that case, we should revert the last commit (cc4b8510c1445fb742c0d750958d352adfa84902) to check the default session is updated or not? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/20404 LGTM. I'd like to leave this to @felixcheung to confirm setting the default session is okay or not (https://github.com/apache/spark/pull/20404#discussion_r164362178). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20404 Hi all, can you please review again, thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86766/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20404 **[Test build #86766 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86766/testReport)** for PR 20404 at commit [`cc4b851`](https://github.com/apache/spark/commit/cc4b8510c1445fb742c0d750958d352adfa84902). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20404 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/337/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20404 **[Test build #86766 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86766/testReport)** for PR 20404 at commit [`cc4b851`](https://github.com/apache/spark/commit/cc4b8510c1445fb742c0d750958d352adfa84902). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86765/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20404 **[Test build #86765 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86765/testReport)** for PR 20404 at commit [`e5f4b58`](https://github.com/apache/spark/commit/e5f4b580d3ecf69f30638ca1bc36073e132c489e). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20404 **[Test build #86765 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86765/testReport)** for PR 20404 at commit [`e5f4b58`](https://github.com/apache/spark/commit/e5f4b580d3ecf69f30638ca1bc36073e132c489e). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/332/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86762/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20404 **[Test build #86762 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86762/testReport)** for PR 20404 at commit [`1ed62ef`](https://github.com/apache/spark/commit/1ed62ef1dbc2c46f2d21eb6147418a86bd59c55c). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86758/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20404 **[Test build #86758 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86758/testReport)** for PR 20404 at commit [`dd1c991`](https://github.com/apache/spark/commit/dd1c9912101389983011b47292da08eff0f43cdc). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20404 **[Test build #86762 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86762/testReport)** for PR 20404 at commit [`1ed62ef`](https://github.com/apache/spark/commit/1ed62ef1dbc2c46f2d21eb6147418a86bd59c55c). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/330/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/329/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86760/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20404 **[Test build #86760 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86760/testReport)** for PR 20404 at commit [`d9f77ea`](https://github.com/apache/spark/commit/d9f77eaadecedc115dd2dc64843761a3bb08d46e). * This patch **fails Python style tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20404 **[Test build #86760 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86760/testReport)** for PR 20404 at commit [`d9f77ea`](https://github.com/apache/spark/commit/d9f77eaadecedc115dd2dc64843761a3bb08d46e). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20404 **[Test build #86759 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86759/testReport)** for PR 20404 at commit [`ec94c05`](https://github.com/apache/spark/commit/ec94c054a8e0abf2d13434556d0434b4690590d3). * This patch **fails Python style tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86759/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20404 **[Test build #86759 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86759/testReport)** for PR 20404 at commit [`ec94c05`](https://github.com/apache/spark/commit/ec94c054a8e0abf2d13434556d0434b4690590d3). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/328/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20404 **[Test build #86758 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86758/testReport)** for PR 20404 at commit [`dd1c991`](https://github.com/apache/spark/commit/dd1c9912101389983011b47292da08eff0f43cdc). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/327/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20404 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20404 **[Test build #86756 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86756/testReport)** for PR 20404 at commit [`dd1c991`](https://github.com/apache/spark/commit/dd1c9912101389983011b47292da08eff0f43cdc). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86756/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20404 Thanks @HyukjinKwon for your help. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20404 **[Test build #86756 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86756/testReport)** for PR 20404 at commit [`dd1c991`](https://github.com/apache/spark/commit/dd1c9912101389983011b47292da08eff0f43cdc). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/325/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20404 I think I made a duplicated effort .. thanks for taking this in. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86752/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20404 **[Test build #86752 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86752/testReport)** for PR 20404 at commit [`eec4386`](https://github.com/apache/spark/commit/eec438619607008ef50e85399a90f6b0ce1e70f1). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20404 **[Test build #86752 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86752/testReport)** for PR 20404 at commit [`eec4386`](https://github.com/apache/spark/commit/eec438619607008ef50e85399a90f6b0ce1e70f1). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/321/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20404 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86700/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20404 **[Test build #86700 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86700/testReport)** for PR 20404 at commit [`d9189ad`](https://github.com/apache/spark/commit/d9189ad1763d4ef867027f4ade2a332d589fe698). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/280/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20404 @zjffdu @HyukjinKwon please help to review. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20404 **[Test build #86700 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86700/testReport)** for PR 20404 at commit [`d9189ad`](https://github.com/apache/spark/commit/d9189ad1763d4ef867027f4ade2a332d589fe698). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
