jinhai created SPARK-37101:
------------------------------
Summary: In class ShuffleBlockPusher, use config instead of key
Key: SPARK-37101
URL: https://issues.apache.org/jira/browse/SPARK-37101
Project: Spark
Issue Type: Improvement
Components: Shuffle
Affects Versions: 3.2.0
Reporter: jinhai
In class ShuffleBlockPusher
{code:java}
// code placeholder
private[this] val maxBytesInFlight =
conf.getSizeAsMb("spark.reducer.maxSizeInFlight", "48m") * 1024 * 1024
private[this] val maxReqsInFlight =
conf.getInt("spark.reducer.maxReqsInFlight", Int.MaxValue)
{code}
We can use config.REDUCER_MAX_SIZE_IN_FLIGHT and
config.REDUCER_MAX_REQS_IN_FLIGHT instead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
