Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21402
@redsanket you should close it by yourself.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21402
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user redsanket commented on the issue:
https://github.com/apache/spark/pull/21402
@cloud-fan yes we can close this
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21402
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21402
shall we close it since #22173 is merged?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user redsanket commented on the issue:
https://github.com/apache/spark/pull/21402
@Victsm @vanzin i want to get this going, is it better if I have a PR up
with the requested changes and concerns?
---
-
To
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/21402
Another question before I forget about it again: with the current code, if
`chunkFetchHandlerThreads()` is equal or greater the size of the main thread
pool won't that allow the current bad behavior
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21402
**[Test build #91593 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91593/testReport)**
for PR 21402 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21402
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91593/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21402
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21402
**[Test build #91593 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91593/testReport)**
for PR 21402 at commit
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21402
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21402
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/21402
> StreamRequest will not block the server netty handler thread.
Hmm, I'm not so sure that's accurate. I think the main difference is that I
don't think there is currently any code path that
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21402
**[Test build #91444 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91444/testReport)**
for PR 21402 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21402
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91444/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21402
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21402
**[Test build #91444 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91444/testReport)**
for PR 21402 at commit
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/21402
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21402
I'm fine of adding metrics in another PR, please add a TODO in the code
comment.
---
-
To unsubscribe, e-mail:
Github user Victsm commented on the issue:
https://github.com/apache/spark/pull/21402
@felixcheung
Adding these metrics are indeed things we have been working on recently.
I'd prefer to propose it in a separate ticket.
---
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21402
while we are here, could we also add or at least propose some metrics
around this, such as number of open block failure, or even number of block
threads?
we have suffer a lot from the
Github user Victsm commented on the issue:
https://github.com/apache/spark/pull/21402
Using a percentage to configure the number of threads to handle chunk fetch
requests does make sense. Will update the PR for this change.
---
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21402
cc @zsxwing @jiangxb1987
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21402
I think we should still respect `spark.shuffle.io.serverThreads`, can we
set a percentage of the server threads as the upper bound to handle chunk fetch
requests?
---
Github user Victsm commented on the issue:
https://github.com/apache/spark/pull/21402
`StreamRequest` will not block the server netty handler thread. Only
`ChunkFetchRequest` does, as this request is the one that actually triggers
seeking the disk to return the shuffle block to the
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21402
what's the strategy here? if the number of block fetch requests has reached
the limitation, shall we fail following block fetch requests immediately?
---
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21402
to confirm, `StreamRequest` will not block the server netty handler thread,
right?
---
-
To unsubscribe, e-mail:
28 matches
Mail list logo
