[
https://issues.apache.org/jira/browse/HDFS-17382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17822554#comment-17822554
]
ASF GitHub Bot commented on HDFS-17382:
---------------------------------------
steveloughran commented on code in PR #6584:
URL: https://github.com/apache/hadoop/pull/6584#discussion_r1508958305
##########
hadoop-hdfs-project/hadoop-hdfs/pom.xml:
##########
@@ -35,6 +35,17 @@ https://maven.apache.org/xsd/maven-4.0.0.xsd";>
</properties>
<dependencies>
+ <dependency>
+ <groupId>log4j</groupId>
+ <artifactId>apache-log4j-extras</artifactId>
+ <version>${log4j-extras.version}</version>
+ <exclusions>
Review Comment:
what else comes in as a dependency
##########
hadoop-hdfs-project/hadoop-hdfs/pom.xml:
##########
@@ -35,6 +35,17 @@ https://maven.apache.org/xsd/maven-4.0.0.xsd";>
</properties>
<dependencies>
+ <dependency>
+ <groupId>log4j</groupId>
+ <artifactId>apache-log4j-extras</artifactId>
+ <version>${log4j-extras.version}</version>
Review Comment:
declare this in hadoop-project pom with version and exclusions, then declare
in this file without those. this is to ensure consistent importing across
different modules
> Add Apache Log4j Extras Library to Hadoop 3.3 for Enhanced Log Rolling
> Capabilities
> -----------------------------------------------------------------------------------
>
> Key: HDFS-17382
> URL: https://issues.apache.org/jira/browse/HDFS-17382
> Project: Hadoop HDFS
> Issue Type: Improvement
> Affects Versions: 3.4.0
> Reporter: woosuk.ro
> Priority: Minor
> Labels: pull-request-available
> Attachments: image-2024-02-25-00-46-08-841.png
>
>
> In the current Hadoop 3.4 version, the system relies on Log4j 1.x for logging
> purposes. This dependency limits the logging functionality, especially when
> it comes to rolling log files such as 'hdfs-audit.log'. Rolling log files is
> crucial for managing log size and ensuring logs are rotated out over time to
> prevent excessive disk space usage. However, the Log4j 1.x version integrated
> within Hadoop lacks the necessary capabilities to efficiently handle log
> rolling.
> This library extends logging capabilities, including more flexible and
> configurable log rolling features. By deploying this library, we can enable
> advanced rolling strategies such as time-based rolling, size-based rolling,
> and compression of rolled logs, which are not supported by the default Log4j
> 1.x setup in Hadoop.
> The integration of Apache Log4j Extras into Hadoop will significantly improve
> log management by allowing for more sophisticated and configurable log
> rotation policies. This enhancement is particularly important for maintaining
> system performance and reliability, as well as for compliance with log
> retention policies.
> Although there are plans to upgrade to Log4j 2 in the forthcoming Hadoop 3.5
> version, which will inherently solve these issues by providing enhanced
> logging features, there is an immediate need to enable advanced log rolling
> capabilities in the current and previous versions of Hadoop.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
