[ https://issues.apache.org/jira/browse/HBASE-18517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16115671#comment-16115671 ]
Hadoop QA commented on HBASE-18517: ----------------------------------- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s{color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 33m 20s{color} | {color:green} Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 13s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 34m 0s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:bdc94b1 | | JIRA Issue | HBASE-18517 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12880548/HBASE-18517.master.001.patch | | Optional Tests | asflicense | | uname | Linux 056d807a4a39 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh | | git revision | master / 637f7ab | | modules | C: . U: . | | Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/7948/console | | Powered by | Apache Yetus 0.4.0 http://yetus.apache.org | This message was automatically generated. > limit max log message width in log4j > ------------------------------------ > > Key: HBASE-18517 > URL: https://issues.apache.org/jira/browse/HBASE-18517 > Project: HBase > Issue Type: Bug > Affects Versions: 3.0.0, 1.5.0 > Reporter: Vikas Vishwakarma > Assignee: Vikas Vishwakarma > Fix For: 3.0.0, 1.5.0 > > Attachments: HBASE-18517.branch-1.001.patch, > HBASE-18517.master.001.patch > > > We had two cases now in our prod / pilot setups which is leading to humongous > log lines in RegionServer logs. > In first case, one of the phoenix user had constructed a query with a really > large list of Id filters (61 MB) that translated into HBase scan that was > running slow which lead to responseTooSlow messages in the logs with the > entire filter list being printed in the logs, example > ipc.RpcServer - (responseTooSlow): > {"call":"Scan(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ScanRequest)","starttimems":1501457864417,"responsesize":11,"method":"Scan","param":"region > { type: REGION_NAME value: ..... > org.apache.hadoop.hbase.filter.FirstKeyOnlyFilter\\022\351\\200\\036\\n(org.apache.phoenix.filter.SkipScanFilter > ... <a really long filter list leading to single log lines of size 61MB> ... > There was another case where a use case had created a table with really large > key/region names. This was causing humongous log lines for flush and > compaction on these regions filling up the RS logs > These large logs usually cause issues with disk I/O load, loading the splunk > servers, even machine perf degradations. With 61 MB log lines basic log > processing commands like vim, scrolling the logs, wc -l , etc were getting > stuck. High GC activity was also noted on this cluster although not 100% sure > if it was related to above issue. > We should consider limiting the message size in logs which can be easily done > by adding a maximum width format modifier on the message conversion character > in log4j.properties > log4j.appender.console.layout.ConversionPattern=...: %m%n > to > log4j.appender.console.layout.ConversionPattern=...: %.10000m%n -- This message was sent by Atlassian JIRA (v6.4.14#64029)