cloud-fan commented on issue #22173: [SPARK-24355] Spark external shuffle
server improvement to better handle block fetch requests.
URL: https://github.com/apache/spark/pull/22173#issuecomment-587273834
SPARK-30602 looks too big for this particular issue.
What I am looking for is to
cloud-fan commented on issue #22173: [SPARK-24355] Spark external shuffle
server improvement to better handle block fetch requests.
URL: https://github.com/apache/spark/pull/22173#issuecomment-573387824
Sounds good to me. We can explore different solutions after 3.0.
cloud-fan commented on issue #22173: [SPARK-24355] Spark external shuffle
server improvement to better handle block fetch requests.
URL: https://github.com/apache/spark/pull/22173#issuecomment-572459561
Unfortunately, I'm not able to minimize our internal workload, so I switch
to TPCDS to
cloud-fan commented on issue #22173: [SPARK-24355] Spark external shuffle
server improvement to better handle block fetch requests.
URL: https://github.com/apache/spark/pull/22173#issuecomment-571202152
When I say sync mode, I mean a thread that handles fetch request has to
finish reading
cloud-fan commented on issue #22173: [SPARK-24355] Spark external shuffle
server improvement to better handle block fetch requests.
URL: https://github.com/apache/spark/pull/22173#issuecomment-571181860
It's good to know that the underlying channel write thread pool has the same
cloud-fan commented on issue #22173: [SPARK-24355] Spark external shuffle
server improvement to better handle block fetch requests.
URL: https://github.com/apache/spark/pull/22173#issuecomment-570509238
We hit significant performance regression in our internal workload caused by
this
