Thank you for the reply. And yes, it is not only for the root logger, we have tons of different loggers in hadoop, as hadoop is constructed with several different projects actually, and they are developed by different groups of people...
And on splitting the config in shell, actually this is exactly what I have done in HBase See https://github.com/apache/hbase/blob/3ec3df5887e9271f7e75779eafe2439012cfb2c3/bin/hbase#L829 And also https://github.com/apache/hbase/blob/3ec3df5887e9271f7e75779eafe2439012cfb2c3/bin/hbase.cmd#L336 For HBase maybe it is acceptable but hadoop is another story. Hadoop is widely used almost in every bigdata platform. Among the companies I've worked with, they all have their own hadoop deployment systems which have modified version of start up scripts... I can imagine that if we do the same trick in HBase, then people will ask again and again on the mailing list why passing -Dhadoop.root.logger does not work, as well as other options like -Dyarn.root.logger, -Dhadoop.security.logger, etc... So it will be good if we can support configure level and appenders in one property. Or maybe another way is to enhance the ability in the Properties section, so we can split one property into two properties, something like <Properties> <Property name="hadoop.root.logger" type="array">${sys:hadoop.root.logger:-INFO,Console}</Property> <Property name="hadoop.root.logger.level">${hadoop.root.logger[0]}</Property> <Property name="hadoop.root.logger.appender"> ${hadoop.root.logger[1]} </Property> <Properties> Notice that since we could add multiple appenders here, maybe we need to support something like ${hadoop.root.logger[1:] to combine all the appenders... This could also solve the problem as people could still pass -Dhadoop.root.logger. But I'm not sure if adding the ability to split a string in configuation could introduce new possible security concerns... Thanks. Ralph Goers <[email protected]> 于2022年1月10日周一 07:14写道： > We already support this with two variables. But if they only want to pass > one then we have two options: > 1. Modify PropertiesConfiguration to support a new element that allows a > Level and AppenderRef which then gets split internally. > 2. Create a Lookup that extracts the relevant portion of the data. > > Ralph > > > On Jan 9, 2022, at 3:08 PM, Gary Gregory <[email protected]> wrote: > > > > I think it is reasonable to say we can support this through 2 instead of > 1 > > variable. > > > > Duo? > > > > Gary > > > > On Sun, Jan 9, 2022, 16:24 Ralph Goers <[email protected]> > wrote: > > > >> I’m looking at this and have a couple of concerns. The script has > >> > >> > >> HADOOP_ROOT_LOGGER=${HADOOP_ROOT_LOGGER:-${HADOOP_LOGLEVEL},console} > >> > >> > HADOOP_DAEMON_ROOT_LOGGER=${HADOOP_DAEMON_ROOT_LOGGER:-${HADOOP_LOGLEVEL},RFA} > >> HADOOP_SECURITY_LOGGER=${HADOOP_SECURITY_LOGGER:-INFO,NullAppender} > >> > >> So it seems you need this for more than just the root logger. > >> > >> Second, you are asking us to accept “level, appender” as the value and > >> under the covers split the > >> value and assign the level to the level attribute and the appender name > to > >> the AppenderRef. > >> > >> While this certainly can be done it seems like it would be just as easy > to > >> do the split in the script > >> and create two different variables for the logging configuration to pick > >> up. Is there a reason that > >> you can’t do this - for example perhaps users have hacked your scripts > and > >> now you can’t > >> change them without breaking them? If so, was this “supported”? > >> > >> Ralph > >> > >> > >> > >>> On Jan 9, 2022, at 8:01 AM, 张铎(Duo Zhang) <[email protected]> > wrote: > >>> > >>> I brought this up in the incubator mailing list, and was suggested to > >>> report directly to the log4j community. > >>> > >>> https://issues.apache.org/jira/browse/HADOOP-16206 > >>> > >>> In hadoop we started to try migrating to log4j2 long ago, but it is not > >>> easy. For now, one of the most blocker issues is the lack of support of > >>> 'log4j.rootLogger=INFO,Console' grammar. Notice that we do not want to > >> stay > >>> on the bridge api, we want to fully migrate to log4j2 finally, so > >>> supporting the above grammar in log4j1 bridge is not enough. > >>> > >>> This is the main log4j configuration file for hadoop > >>> > >>> > >> > https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties > >>> > >>> We have this in the file > >>> > >>> # Define the root logger to the system property "hadoop.root.logger". > >>>> log4j.rootLogger=${hadoop.root.logger} > >>> > >>> > >>> So our users could simply pass -Dhadoop.root.logger to control the > level > >>> and appender. > >>> > >>> > >> > https://github.com/apache/hadoop/blob/39efbc6b6fe53abed15f5639edcbaaa2a9dda6d2/hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh#L904 > >>> > >>> Here we use an environment variable in our start up scripts. And I > >> believe > >>> there are lots of other hadoop deployment systems which will have their > >> own > >>> start scripts. > >>> > >>> So in general, it is just impossible for us to drop the > >>> -Dhadoop.root.logger way of configuring our logging system. > >>> > >>> So here I want to ask if it is possible for log4j2 to still support the > >>> 'log4j.rootLogger=INFO,Console' grammar. I'm not saying you must add it > >>> back with no change, we just need a way to configure the level and > >> appeners > >>> at once, so something like > >>> 'log4j2.rootLogger.levelAndAppender=INFO,Console' is also OK. > >>> > >>> Thanks. > >> > >> > >
