[
https://issues.apache.org/jira/browse/HIVE-29665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18097809#comment-18097809
]
László Bodor edited comment on HIVE-29665 at 7/21/26 10:07 AM:
---------------------------------------------------------------
"The goal of this task is to create a holistic overview of the loggers used by
Hive and document their purpose, ownership, and expected usage."
!Screenshot 2026-07-21 at 11.52.09.png|width=348,height=195!
above statement makes sense to me
due to the nature of log4j configs, I don't think there is such a thing as
templating, and using a common source of truth in terms of logging properties
files, where ideally the purpose of each logger is defined, so this should be
in the form of a central README.md I believe
having multiple log4j2.properties files is okay, though, as we expect different
logging behavior in different components
so what we might want to be clear and strict about in the hive repository is:
1. every log4j2 properties file has a brief description: why they are there,
where are they used
2. every logger entry that ever appears in a log4j properties file has a
corresponding description in a central readme: must-have: what is the logger
typically used for
nice-to-have: recommended level in production
current antipattern:
{code}
logger.HadoopConf.name = org.apache.hadoop.conf.Configuration
logger.HadoopConf.level = ERROR
{code}
why ERROR? is it too verbose otherwise?
so I feel we could use an extra testing/workflow step that scans log4j files,
which fails is there is a logger entry which has no corresponding README entry,
hence remains undocumented
was (Author: abstractdog):
"The goal of this task is to create a holistic overview of the loggers used by
Hive and document their purpose, ownership, and expected usage."
!Screenshot 2026-07-21 at 11.52.09.png|width=348,height=195!
above statement makes sense to me
due to the nature of log4j configs, I don't think there is such a thing as
templating, and using a common source of truth in terms of logging properties
files, where ideally the purpose of each logger is defined, so this should be
in the form of a central README.md I believe
having multiple log4j2.properties files is okay, though, as we expect different
logging behavior in different components
so what we might want to be clear and strict about in the hive repository is:
1. every log4j2 properties file has a brief description: why they are there,
where are they used
2. every logger entry that ever appears in a log4j properties file has a
corresponding description in a central readme: must-have: what is the logger
typically used for
nice-to-have: recommended level in production
current antipattern:
{code}
logger.HadoopConf.name = org.apache.hadoop.conf.Configuration
logger.HadoopConf.level = ERROR
{code}
why ERROR? is it too verbose otherwise?
> Review and consolidate Hive logging configuration and logger usage
> ------------------------------------------------------------------
>
> Key: HIVE-29665
> URL: https://issues.apache.org/jira/browse/HIVE-29665
> Project: Hive
> Issue Type: Task
> Reporter: Attila Turoczy
> Priority: Major
> Labels: cloud
> Attachments: Screenshot 2026-07-21 at 11.52.09.png
>
>
> Hive uses several different loggers across the project and its dependencies.
> Some of these loggers are Hive-specific, while others come from third-party
> libraries such as AWS SDK, DataNucleus, ZooKeeper, JPOX, and Apache HTTP
> components.
> Examples include:
>
> * {{org.apache.hadoop.hive.ql.log.PerfLogger}}
> * {{com.amazonaws}}
> * {{DataNucleus}}
> * {{Datastore}}
> * {{org.apache.zookeeper.ClientCnxnSocketNIO}}
> * {{org.apache.zookeeper.server.NIOServerCnxn}}
> * {{JPOX}}
> * {{org.apache.http}}
> Currently, it is not fully clear which loggers are still required, which ones
> are actively used, and whether some of them could be consolidated, renamed,
> removed, or configured in a more consistent way.
> The goal of this task is to create a holistic overview of the loggers used by
> Hive and document their purpose, ownership, and expected usage.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
