waitinfuture commented on code in PR #1800:
URL:
https://github.com/apache/incubator-celeborn/pull/1800#discussion_r1319617783
##########
worker/src/main/java/org/apache/celeborn/service/deploy/worker/memory/MemoryManager.java:
##########
@@ -246,6 +252,13 @@ protected void switchServingState() {
servingState = currentServingState();
if (lastState == servingState) {
if (servingState != ServingState.NONE_PAUSED) {
+ // force to append pause spent time even we are in pause state
+ if (trimCounter.incrementAndGet() >=
forceAppendPauseSpentTimeThreshold) {
+ logger.debug(
+ "Trigger action: TRIM for {} times, force to append pause spent
time.",
+ trimCounter.incrementAndGet());
Review Comment:
get
##########
common/src/main/scala/org/apache/celeborn/common/CelebornConf.scala:
##########
@@ -676,6 +676,8 @@ class CelebornConf(loadDefaults: Boolean) extends Cloneable
with Logging with Se
def metricsAppTopDiskUsageCount: Int = get(METRICS_APP_TOP_DISK_USAGE_COUNT)
def metricsAppTopDiskUsageWindowSize: Int =
get(METRICS_APP_TOP_DISK_USAGE_WINDOW_SIZE)
def metricsAppTopDiskUsageInterval: Long =
get(METRICS_APP_TOP_DISK_USAGE_INTERVAL)
+ def metricsWorkerForceAppendPauseSpentTimeThreshold: Int =
+ get(METRICS_WORKER_PAUSE_SPENT_TINE_FORCE_APPEND_THRESHOLD)
Review Comment:
typo METRICS_WORKER_PAUSE_SPENT_TIME_FORCE_APPEND_THRESHOLD
##########
worker/src/main/java/org/apache/celeborn/service/deploy/worker/memory/MemoryManager.java:
##########
@@ -65,8 +67,11 @@ public class MemoryManager {
private final AtomicLong diskBufferCounter = new AtomicLong(0);
private final LongAdder pausePushDataCounter = new LongAdder();
private final LongAdder pausePushDataAndReplicateCounter = new LongAdder();
+ private final LongAdder pausePushDataTime = new LongAdder();
Review Comment:
Since `switchServingState` will be called sequentially in a single thread, I
think it's safe to use long/int for `pausePushDataTime` and `trimCounter`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
