Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/20404
Thanks all for your review, greatly appreciated.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20404
Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user ueshin commented on the issue:
https://github.com/apache/spark/pull/20404
I completely agree with @HyukjinKwon.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20404
How about you @ueshin?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20404
I am fine with not fixing the case here for now if we go for it only with
master branch. Maybe, we can try to fix the pre existing issue too seperately
by syncrhoziing everything if anyone
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/20404
Thanks Felix. I would incline to not fix the case mentioned by Felix.
What's your opinion @HyukjinKwon @ueshin ?
---
-
To
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/20404
Itâs your call - there are some risks but seems like there are pre
existing code that has the same problem.
---
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/20404
@felixcheung what is your opinion on this, do we really need to handle this
case?
---
-
To unsubscribe, e-mail:
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/20404
I think the same issue also existed in Scala `SparkSession` code, because
`setDefaultSession` doesn't hold a lock which holds by `getOrCreate`
(SparkSession).
For example:
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/20404
Thanks all for your comments. I think @felixcheung 's case really makes
thing complex, I'm not sure if user will use it in such way. I will try to
address it. Appreciate your comments!
---
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20404
Yup, sorry for rushing the suggestion. Wanted to show an idea. BTW, if we
should replace it if it's already stopped, I assume we need the synchronization
with `defaultSession` anyway if I
Github user ueshin commented on the issue:
https://github.com/apache/spark/pull/20404
@HyukjinKwon I'm afraid that's not enough because we need to synchronize at
all the places we touch the `defaultSession`, otherwise another thread can set
the value between `defaultSession.get() eq
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20404
How about something like this?
```diff
--- a/python/pyspark/sql/session.py
+++ b/python/pyspark/sql/session.py
@@ -213,10 +213,10 @@ class SparkSession(object):
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20404
For perfectness, I think we should hold a lock with JVM instance but I
wonder if it's easily possible. I roughly knew this but I think underestimated
this because I believe that will quite
Github user ueshin commented on the issue:
https://github.com/apache/spark/pull/20404
@felixcheung I see, in that case, we should revert the last commit
(cc4b8510c1445fb742c0d750958d352adfa84902) to check the default session is
updated or not?
---
Github user ueshin commented on the issue:
https://github.com/apache/spark/pull/20404
LGTM.
I'd like to leave this to @felixcheung to confirm setting the default
session is okay or not
(https://github.com/apache/spark/pull/20404#discussion_r164362178).
---
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/20404
Hi all, can you please review again, thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86766/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20404
**[Test build #86766 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86766/testReport)**
for PR 20404 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20404
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/337/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20404
**[Test build #86766 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86766/testReport)**
for PR 20404 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86765/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20404
**[Test build #86765 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86765/testReport)**
for PR 20404 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20404
**[Test build #86765 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86765/testReport)**
for PR 20404 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/332/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86762/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20404
**[Test build #86762 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86762/testReport)**
for PR 20404 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86758/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20404
**[Test build #86758 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86758/testReport)**
for PR 20404 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20404
**[Test build #86762 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86762/testReport)**
for PR 20404 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/330/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/329/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86760/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20404
**[Test build #86760 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86760/testReport)**
for PR 20404 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20404
**[Test build #86760 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86760/testReport)**
for PR 20404 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20404
**[Test build #86759 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86759/testReport)**
for PR 20404 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86759/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20404
**[Test build #86759 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86759/testReport)**
for PR 20404 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/328/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20404
**[Test build #86758 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86758/testReport)**
for PR 20404 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/327/
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/20404
Jenkins, retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20404
**[Test build #86756 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86756/testReport)**
for PR 20404 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86756/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/20404
Thanks @HyukjinKwon for your help.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20404
**[Test build #86756 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86756/testReport)**
for PR 20404 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/325/
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20404
I think I made a duplicated effort .. thanks for taking this in.
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86752/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20404
**[Test build #86752 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86752/testReport)**
for PR 20404 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20404
**[Test build #86752 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86752/testReport)**
for PR 20404 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/321/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20404
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86700/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20404
**[Test build #86700 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86700/testReport)**
for PR 20404 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/280/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/20404
@zjffdu @HyukjinKwon please help to review. Thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20404
**[Test build #86700 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86700/testReport)**
for PR 20404 at commit
77 matches
Mail list logo
