SteNicholas commented on code in PR #3824:
URL: https://github.com/apache/celeborn/pull/3824#discussion_r3845598414
##########
worker/src/main/java/org/apache/celeborn/service/deploy/worker/memory/ChannelsLimiter.java:
##########
@@ -156,5 +174,75 @@ public void onTrim() {
}
}
+ /**
+ * Resumes {@code ratio} fraction of paused channels that have a
likely-large stuck half-frame
+ * ({@link TransportFrameDecoder#hasLikelyLargeIncompleteFrame()}), each in
frame-drain mode.
+ */
+ @Override
+ public int drainIncompleteFrame(double ratio) {
+ List<Channel> candidates = new ArrayList<>();
+ for (Channel ch : channels) {
+ if (!ch.isActive() || ch.config().isAutoRead()) {
+ continue;
+ }
+ TransportFrameDecoder decoder = frameDecoderOf(ch);
+ if (decoder != null && decoder.hasLikelyLargeIncompleteFrame()) {
+ candidates.add(ch);
+ }
+ }
+
+ if (candidates.isEmpty()) {
+ // Only worth logging when backpressure is actually active; if isPaused
is false there are
+ // no paused channels to scan, so a zero result is expected and
uninteresting.
+ if (isPaused.get()) {
+ logger.info(
Review Comment:
[P2] With the default 10 ms memory check and 5 ms drain interval, sustained
backpressure with no qualifying candidate emits this INFO message about 100
times per second per paused limiter—potentially 200/s when push and replication
are both paused. This can exacerbate an incident and consume substantial log
storage. Please rate-limit it or lower it to DEBUG.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
