Thomas Graves created SPARK-21530:
-------------------------------------
Summary: Update description of
spark.shuffle.maxChunksBeingTransferred
Key: SPARK-21530
URL: https://issues.apache.org/jira/browse/SPARK-21530
Project: Spark
Issue Type: Improvement
Components: Shuffle
Affects Versions: 2.3.0
Reporter: Thomas Graves
We should expand the description of spark.shuffle.maxChunksBeingTransferred to
include what happens when the max is hit. In this case it actually closes
incoming connections so if the retry logic on the reducer doesn't happen could
end up in failure.
The way its currently worded doesn't tell users bad things can happen when the
max is hit.
introduced with https://github.com/apache/spark/pull/18388/files
SPARK-21175
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
