FMX commented on code in PR #3366:
URL: https://github.com/apache/celeborn/pull/3366#discussion_r2209136246
##########
common/src/main/scala/org/apache/celeborn/common/CelebornConf.scala:
##########
@@ -4376,6 +4380,40 @@ object CelebornConf extends Logging {
.longConf
.createOptional
+ val WORKER_OPEN_HDFS_OUTPUT_STREAM_MAX: ConfigEntry[Int] =
+ buildConf("celeborn.worker.openHdfsOutputStream.max")
+ .categories("worker")
+ .doc("If the number of opened hdfs output streams on a worker exceeds
this configuration value, " +
+ "the worker will be marked as high-load in the heartbeat report, " +
+ "and the master will not include that node in the response of
RequestSlots.")
+ .version("0.7.0")
+ .intConf
+ .createWithDefault(10000)
Review Comment:
Maybe the default value can be set to 1000.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
