[bookkeeper] branch branch-4.12 updated: Limit log output to prevent in case of the large group of the log segments

[eolivelli]

This is an automated email from the ASF dual-hosted git repository.

eolivelli pushed a commit to branch branch-4.12
in repository https://gitbox.apache.org/repos/asf/bookkeeper.git


The following commit(s) were added to refs/heads/branch-4.12 by this push:
     new b98d949  Limit log output to prevent in case of the large group of the 
log segments
b98d949 is described below

commit b98d94911e2ebbcf614c00a5398ab430dcc13b3b
Author: Andrey Yegorov <8622884+dl...@users.noreply.github.com>
AuthorDate: Mon Feb 1 01:54:43 2021 -0800

    Limit log output to prevent in case of the large group of the log segments
    
    Descriptions of the changes in this PR:
    
    Limit logged data
    
    ### Motivation
    
    Overly long log line at ReadAheadEntryReader
    Found a log with single log line of over 5 mil characters long
    "org.apache.distributedlog.ReadAheadEntryReader - Starting the readahead 
entry reader for ..." + the details of ~16000 segments.
    
    ### Changes
    
    Output details of up to 10 segments plus count of segments.
    
    Master Issue: #2561
    
    Reviewers: Enrico Olivelli <eolive...@gmail.com>, Nicolo Boschi 
<boschi1...@gmail.com>
    
    This closes #2562 from dlg99/master-log
    
    (cherry picked from commit a7f2a2eceaaeb4e101c6fd51a6868b48c8d1214f)
    Signed-off-by: Enrico Olivelli <eolive...@apache.org>
---
 .../java/org/apache/distributedlog/ReadAheadEntryReader.java   | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git 
a/stream/distributedlog/core/src/main/java/org/apache/distributedlog/ReadAheadEntryReader.java
 
b/stream/distributedlog/core/src/main/java/org/apache/distributedlog/ReadAheadEntryReader.java
index 11a7e7e..915d81a 100644
--- 
a/stream/distributedlog/core/src/main/java/org/apache/distributedlog/ReadAheadEntryReader.java
+++ 
b/stream/distributedlog/core/src/main/java/org/apache/distributedlog/ReadAheadEntryReader.java
@@ -33,6 +33,8 @@ import java.util.concurrent.TimeUnit;
 import java.util.concurrent.atomic.AtomicBoolean;
 import java.util.concurrent.atomic.AtomicReference;
 import java.util.function.Function;
+import java.util.stream.Collectors;
+
 import org.apache.bookkeeper.common.concurrent.FutureEventListener;
 import org.apache.bookkeeper.common.concurrent.FutureUtils;
 import org.apache.bookkeeper.common.util.OrderedScheduler;
@@ -411,8 +413,12 @@ class ReadAheadEntryReader implements
     }
 
     public void start(final List<LogSegmentMetadata> segmentList) {
-        logger.info("Starting the readahead entry reader for {} : segments = 
{}",
-                readHandler.getFullyQualifiedName(), segmentList);
+        // Managed to get 5mil character long log line from here.
+        // Will limit the output.
+        logger.info("Starting the readahead entry reader for {} : number of 
segments: {}, top 10 segments = {}",
+                readHandler.getFullyQualifiedName(), segmentList.size(),
+                segmentList.size() > 10
+                        ? 
segmentList.stream().limit(10).collect(Collectors.toList()) : segmentList);
         started.set(true);
         processLogSegments(segmentList);
     }