Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/21356
Merging to master / 2.3.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21356
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90914/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21356
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21356
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21356
**[Test build #90914 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90914/testReport)**
for PR 21356 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21356
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90913/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21356
**[Test build #90913 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90913/testReport)**
for PR 21356 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21356
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3426/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21356
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21356
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3425/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21356
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21356
**[Test build #90914 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90914/testReport)**
for PR 21356 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21356
**[Test build #90913 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90913/testReport)**
for PR 21356 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21356
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90873/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21356
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21356
**[Test build #90873 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90873/testReport)**
for PR 21356 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21356
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21356
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3400/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21356
**[Test build #90873 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90873/testReport)**
for PR 21356 at commit
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/21356
I think the behavior in your change (trying to keep things running) is
fine. Since the queues are isolated now, there's no risk of bad listeners
affecting important ones (like the ones used for
Github user squito commented on the issue:
https://github.com/apache/spark/pull/21356
the problem was not actually an interrupted exception from the listener, it
was that the Thread's state was getting set to interrupted, and then there
would be a failure later in `queue.take()`:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21356
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21356
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90795/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21356
**[Test build #90795 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90795/testReport)**
for PR 21356 at commit
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21356
The code here is just propagating the interrupt state and is not doing
anything wrong. @squito do you know what would cause this in the first place?
---
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/21356
@zsxwing Imran knows more since he debugged it, but I see a lot of code
like this in Hadoop libraries:
```
try { ... } catch (InterruptedException ie) {
// log or something
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21356
@vanzin I just want to understand the real issue here to make sure the
current solution is correct.
---
-
To unsubscribe,
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/21356
So, to clarify a little more what the previous code did:
`ListenerBus.postToAll()` catches non-fatal exceptions from each individual
listener and ignores them; but `InterruptedException` is not one
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21356
> hdfs can interrupt the thread.
@squito Could you clarify this? This sounds like a bug in HDFS. I'm curious
when this will happen. Although we don't interrupt the AsyncEventQueue thread
Github user skonto commented on the issue:
https://github.com/apache/spark/pull/21356
> I'm pretty sure the existing use of PoisonPill is intentionally put at
the end of the queue, so that the > everything is processed before the queue is
shut down.
Yes I guessed that, I was just
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21356
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3337/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21356
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user squito commented on the issue:
https://github.com/apache/spark/pull/21356
> As an alternative design option, PoisonPill could be handled differently,
since some msgs should have higher priority and can be considered them as part
of your "control plane".
I'm
Github user squito commented on the issue:
https://github.com/apache/spark/pull/21356
I pushed an update which only removes the listener which was active at the
interrupt. Note that is not the same thing as the listener which *caused* the
interrupt, necessarily -- we have no idea
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21356
**[Test build #90795 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90795/testReport)**
for PR 21356 at commit
Github user skonto commented on the issue:
https://github.com/apache/spark/pull/21356
Also PoisonPill could be handled differently some msgs should have higher
priority, consider them as part of your control plane.
---
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21356
Since `InterruptedException` is fatal, maybe it's not safe to keep calling
this listener, but stopping the entire queue seems an overkill. Shall we just
remove this listener from the queue?
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21356
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90769/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21356
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21356
**[Test build #90769 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90769/testReport)**
for PR 21356 at commit
Github user squito commented on the issue:
https://github.com/apache/spark/pull/21356
> does this mean a problematic listener can kill the queue and crash other
listeners in the same queue? Shall we do some isolation?
yeah I think marcelo was asking about this above
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21356
also cc @JoshRosen @zsxwing
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21356
does this mean a problematic listener can kill the queue and crash other
listeners in the same queue? Shall we do some isolation?
Not introduced by this PR, but I'm wondering why we only
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21356
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21356
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3317/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21356
**[Test build #90769 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90769/testReport)**
for PR 21356 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21356
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21356
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90753/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21356
**[Test build #90753 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90753/testReport)**
for PR 21356 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21356
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3310/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21356
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21356
**[Test build #90753 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90753/testReport)**
for PR 21356 at commit
52 matches
Mail list logo
