[
https://issues.apache.org/jira/browse/HADOOP-16206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17498910#comment-17498910
]
Duo Zhang commented on HADOOP-16206:
------------------------------------
After some investigating, ContainerLogAppender and ContainerRollingLogAppender
are used for container logging. The goal is to limit the maximum size of logs.
ContainerLogAppender will use a in memory queue to hold the tail of the log
messages after exceeding the size limit, while ContainerRollingLogAppender is
just a rolling file appender, we control the total size by setting the size
limit for one file and the number of files to keep.
So for these two appenders, I think maybe we could just leave the rolling file
based one, as they have almost the same ability. And if we choose the rolling
file based one, we do not need to implement a special appender any more, just
use the RollingFileAppender for log4j2 is enough. We can read system properties
in log4j2.properties so it is easy to control where the log.
For TaskLogAppender, it is almost the same with ContainerLogAppender but I
haven't seen where we plan to reference it. Searching the repo, I can only find
references in 0.21.0's release note.
https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/site/markdown/release/0.21.0/RELEASENOTES.0.21.0.md
{noformat}
MAPREDUCE-1086 | Major | hadoop commands in streaming tasks are trying to write
to tasktracker's log
This patch makes TT to set HADOOP_ROOT_LOGGER to INFO,TLA by default in the
environment of taskjvm and its children.
MAPREDUCE-1607 | Major | Task controller may not set permissions for a task
cleanup attempt's log directory
Fixed initialization of a task-cleanup attempt's log directory by setting
correct permissions via task-controller. Added new log4j properties
hadoop.tasklog.iscleanup and log4j.appender.TLA.isCleanup to
conf/log4j.properties. Changed the userlogs for a task-cleanup attempt to go
into its own directory instead of the original attempt directory. This is an
incompatible change as old userlogs of cleanup attempt-dirs before this release
will no longer be visible.
{noformat}
I think it provides the same ability with ContainerLogAppender before we have
yarn. So I prefer we just remove it.
For HttpRequestLogAppender, it is just a dummy appender, see the code here
https://github.com/apache/hadoop/blob/5eab9719cbf6b9bddbdb4454a5f8e1ae12495492/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/http/HttpRequestLog.java#L86
We will replace it with the actual log writer at runtime. There is a related
issue https://issues.apache.org/jira/browse/HADOOP-17526, where we want to
replace it with Slf4jRequestLog, so we could just a normal log4j2 appender to
configure it. So after HADOOP-17526, we could also remove it.
For Log4jWarningErrorMetricsAppender, I think we need to port it to log4j2.
Thanks.
> Migrate from Log4j1 to Log4j2
> -----------------------------
>
> Key: HADOOP-16206
> URL: https://issues.apache.org/jira/browse/HADOOP-16206
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: 3.3.0
> Reporter: Akira Ajisaka
> Assignee: Duo Zhang
> Priority: Major
> Attachments: HADOOP-16206-wip.001.patch
>
>
> This sub-task is to remove log4j1 dependency and add log4j2 dependency.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
