[
https://issues.apache.org/jira/browse/MAPREDUCE-7453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17770474#comment-17770474
]
ASF GitHub Bot commented on MAPREDUCE-7453:
-------------------------------------------
zhengchenyu commented on PR #6102:
URL: https://github.com/apache/hadoop/pull/6102#issuecomment-1741028712
@virajjasani Thanks for your reply.
For CLA before HADOOP-18649:
CLA is org.apache.hadoop.yarn.ContainerLogAppender which is extend from
FileAppender.
In default, yarn.app.container.log.filesize is set to 0, then
log4j.appender.CLA.totalLogFileSize is set to 0. So maxEvents is 0, then
eventBuffer is null. Then we will append directly. There are no limit.
For CLA after HADOOP-18649:
CLA is org.apache.log4j.RollingFileAppender.
In default, yarn.app.container.log.filesize is set to 0,
RollingFileAppender::maxFileSize is set to 0. This configure means that we will
roll the log every time. Then will reproduce the problem which is reported by
me.
For CRLA before HADOOP-18649:
CRLA is ContainerRollingLogAppender which is extend from
org.apache.log4j.RollingFileAppender.
In default, RollingFileAppender::maxFileSize is not set, so use default
value 10M. The default value is comming from:
https://github.com/apache/logging-log4j1/blob/b7e9154128cd4ae1244c877a6fda8f834a0f2247/src/main/java/org/apache/log4j/RollingFileAppender.java#L50
For CRLA after HADOOP-18649:
CRLA is org.apache.log4j.RollingFileAppender.
In default, yarn.app.container.log.filesize is set to 0,
RollingFileAppender::maxFileSize is set to 0. This configure means that we will
roll the log every time. Then will reproduce the problem which is reported by
me.
> Container logs are missing when yarn.app.container.log.filesize is set to
> default value 0.
> ------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-7453
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7453
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Affects Versions: 3.3.6
> Reporter: zhengchenyu
> Assignee: zhengchenyu
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
>
> Since HADOOP-18649, in container-log4j.properties,
> log4j.appender.\{APPENDER}.MaxFileSize is set to
> ${yarn.app.container.log.filesize}, but yarn.app.container.log.filesize is 0
> in default. So log is missing. This log is always rolling and only show the
> latest log.
> This is the running log like below:
> {code:java}
> Log Type: syslog
> Log Upload Time: Fri Sep 08 11:36:09 +0800 2023
> Log Length: 0
> Log Type: syslog.1
> Log Upload Time: Fri Sep 08 11:36:09 +0800 2023
> Log Length: 179
> 2023-09-08 11:31:34,494 INFO [AsyncDispatcher event handler]
> org.apache.hadoop.yarn.util.RackResolver: Got an error when resolve
> hostNames. Falling back to /default-rack for all. {code}
> Note: log4j.appender.\{APPENDER}.MaxFileSize is not set before, then use
> default value 10M, so no problem before HADOOP-18649
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
