Ngone51 commented on code in PR #52919:
URL: https://github.com/apache/spark/pull/52919#discussion_r2517443431
##########
core/src/main/scala/org/apache/spark/storage/BlockManagerMaster.scala:
##########
@@ -39,6 +41,11 @@ class BlockManagerMaster(
val timeout = RpcUtils.askRpcTimeout(conf)
+ val waitBlockRemovalTimeout =
+ RpcTimeout(
+ conf,
+ Seq(CLEANER_REFERENCE_TRACKING_BLOCKING_TIMEOUT.key,
NETWORK_TIMEOUT.key), "120s")
Review Comment:
One small problem of this constructor is that the `timeoutProp` could be
`NETWORK_TIMEOUT,key` when `CLEANER_REFERENCE_TRACKING_BLOCKING_TIMEOUT` is not
set. And it is not good for debugging when timeout exception thrown with a
non-specific `timeoutProp`.
My original proposal with fallback conf is to do like this:
```scala
private[spark] val CLEANER_REFERENCE_TRACKING_BLOCKING_TIMEOUT =
ConfigBuilder("spark.cleaner.referenceTracking.blocking.timeout")
.version("4.1.0")
.fallbackConf(Network.NETWORK_TIMEOUT)
```
##########
core/src/main/scala/org/apache/spark/storage/BlockManagerMaster.scala:
##########
@@ -39,6 +41,11 @@ class BlockManagerMaster(
val timeout = RpcUtils.askRpcTimeout(conf)
+ val waitBlockRemovalTimeout =
+ RpcTimeout(
+ conf,
+ Seq(CLEANER_REFERENCE_TRACKING_BLOCKING_TIMEOUT.key,
NETWORK_TIMEOUT.key), "120s")
Review Comment:
One small problem of this constructor is that the `timeoutProp` could be
`NETWORK_TIMEOUT.key` when `CLEANER_REFERENCE_TRACKING_BLOCKING_TIMEOUT` is not
set. And it is not good for debugging when timeout exception thrown with a
non-specific `timeoutProp`.
My original proposal with fallback conf is to do like this:
```scala
private[spark] val CLEANER_REFERENCE_TRACKING_BLOCKING_TIMEOUT =
ConfigBuilder("spark.cleaner.referenceTracking.blocking.timeout")
.version("4.1.0")
.fallbackConf(Network.NETWORK_TIMEOUT)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
