[jira] [Comment Edited] (HBASE-10092) Move up on to log4j2
[ https://issues.apache.org/jira/browse/HBASE-10092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16295995#comment-16295995 ] Appy edited comment on HBASE-10092 at 12/19/17 7:58 AM: Finished a pass. Review: # Let's do some basic verifications (reverting this patch would be hell, wouldn't want to do it because we missed basic checks) ** An example of new log message for exception (can get it from a test throwing exception) ** Make sure tests have debug messages in them # Please add markers for fatal. It's very important piece of info in logs (searching for 'fatal' takes you nearest to the cause for process crash) . We shouldn't just change them to errors. # What about properties file? Are they being used by slf4j? Do they just work or need change? # What about logging guards? An analysis of work involved would be good. A count of is*Enabled strings would be enough to decide if it should be followup or get removed overtime. But definitely let's not wait this patch on guards. # Let's move backup and http too if possible (or reason why it's not possible) # Update the change summary and add to commit message. Mention: ** programmatic was removed because not supported in slf4j. ** Log guards not removed. ** Any remaining uses of old logger ** Anything else of relevance. was (Author: appy): Finished a pass. Review: - Let's do some basic verifications (reverting this patch would be hell, wouldn't want to do it because we missed basic checks) ** An example of new log message for exception (can get it from a test throwing exception) ** Make sure tests have debug messages in them - Please add markers for fatal. It's very important piece of info in logs (searching for 'fatal' takes you nearest to the cause for process crash) . We shouldn't just change them to errors. - What about properties file? Are they being used by slf4j? Do they just work or need change? - What about logging guards? An analysis of work involved would be good. A count of is*Enabled strings would be enough to decide if it should be followup or get removed overtime. But definitely let's not wait this patch on guards. - Let's move backup and http too if possible (or reason why it's not possible) - Update the change summary and add to commit message. Mention: ** programmatic was removed because not supported in slf4j. ** Log guards not removed. ** Any remaining uses of old logger ** Anything else of relevance. > Move up on to log4j2 > > > Key: HBASE-10092 > URL: https://issues.apache.org/jira/browse/HBASE-10092 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: Balazs Meszaros >Priority: Critical > Fix For: 2.0.0-beta-1 > > Attachments: 10092.txt, 10092v2.txt, HBASE-10092-preview-v0.patch, > HBASE-10092.master.001.patch, HBASE-10092.master.002.patch, HBASE-10092.patch > > > Allows logging with less friction. See http://logging.apache.org/log4j/2.x/ > This rather radical transition can be done w/ minor change given they have an > adapter for apache's logging, the one we use. They also have and adapter for > slf4j so we likely can remove at least some of the 4 versions of this module > our dependencies make use of. > I made a start in attached patch but am currently stuck in maven dependency > resolve hell courtesy of our slf4j. Fixing will take some concentration and > a good net connection, an item I currently lack. Other TODOs are that will > need to fix our little log level setting jsp page -- will likely have to undo > our use of hadoop's tool here -- and the config system changes a little. > I will return to this project soon. Will bring numbers. > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-10092) Move up on to log4j2
[
https://issues.apache.org/jira/browse/HBASE-10092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16296013#comment-16296013
]
Appy edited comment on HBASE-10092 at 12/19/17 1:50 AM:
Here's ref to parameterized logging for those like me who are unfamiliar with
it : https://www.slf4j.org/faq.html#logging_performance
Good that slf4j supports it.
We have is{Debug, Trace}Enabled in total of 1000 places.
If it's not just removing the guards but also formatting the message, then it's
a *lot of work*. My take would be doing it overtime.
bq. As for a fatal marker, you can even rewrite those logs via a log4j2 config
to use the fatal log level.
Example would help [~jvz], thanks. If it's simple enough, would prefer this.
bq. Otherwise, I'd suggest a less confusing marker name so it's not confused
with a level. Something like "crash" could work?
Legacy reason. Every operator/field person/dev would be used to it.
was (Author: appy):
Here's ref to parameterized logging for those like me who are unfamiliar with
it : https://www.slf4j.org/faq.html#logging_performance
Good that slf4j supports it.
We have is{Debug, Trace}Enabled in total of 1000 places.
If it's not just removing the guards but also formatting the message, then it's
a *lot of work*. My take would be doing it overtime.
bq. As for a fatal marker, you can even rewrite those logs via a log4j2 config
to use the fatal log level.
Example would help. If it's simple enough, would prefer this.
bq. Otherwise, I'd suggest a less confusing marker name so it's not confused
with a level. Something like "crash" could work?
Legacy reason. Every operator/field person/dev would be used to it.
> Move up on to log4j2
>
>
> Key: HBASE-10092
> URL: https://issues.apache.org/jira/browse/HBASE-10092
> Project: HBase
> Issue Type: Sub-task
>Reporter: stack
>Assignee: Balazs Meszaros
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: 10092.txt, 10092v2.txt, HBASE-10092-preview-v0.patch,
> HBASE-10092.master.001.patch, HBASE-10092.master.002.patch, HBASE-10092.patch
>
>
> Allows logging with less friction. See http://logging.apache.org/log4j/2.x/
> This rather radical transition can be done w/ minor change given they have an
> adapter for apache's logging, the one we use. They also have and adapter for
> slf4j so we likely can remove at least some of the 4 versions of this module
> our dependencies make use of.
> I made a start in attached patch but am currently stuck in maven dependency
> resolve hell courtesy of our slf4j. Fixing will take some concentration and
> a good net connection, an item I currently lack. Other TODOs are that will
> need to fix our little log level setting jsp page -- will likely have to undo
> our use of hadoop's tool here -- and the config system changes a little.
> I will return to this project soon. Will bring numbers.
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (HBASE-10092) Move up on to log4j2
[ https://issues.apache.org/jira/browse/HBASE-10092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14017871#comment-14017871 ] Andrew Purtell edited comment on HBASE-10092 at 6/4/14 5:10 PM: bq. Need to make tgzs and see that stuff lands in right place. Add some excludes for transitive includes of old log4js. I did this and checked mvn dependency:tree. It looks good. I then had to add back log4j 1 and slf4j in test scope so we could actually instantiate Hadoop classes for running unit tests (Java logging is a disaster), not sure what that is going to do in an assembly. bq. Need to try in standalone, pseudo, and cluster and bring up shell to make sure logs are ending up in right place Yes, the new log4j2.xml files are untested and I'm sure need work. was (Author: apurtell): bq. Need to make tgzs and see that stuff lands in right place. Add some excludes for transitive includes of old log4js. I did this and checked mvn dependency:tree. It looks good. I then had to add back log4j 1 in test scope so we could actually instantiate Hadoop classes for running unit tests, not sure what that is going to do in an assembly. bq. Need to try in standalone, pseudo, and cluster and bring up shell to make sure logs are ending up in right place Yes, the new log4j2.xml files are untested and I'm sure need work. Move up on to log4j2 Key: HBASE-10092 URL: https://issues.apache.org/jira/browse/HBASE-10092 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Attachments: 10092.txt, 10092v2.txt, HBASE-10092.patch Allows logging with less friction. See http://logging.apache.org/log4j/2.x/ This rather radical transition can be done w/ minor change given they have an adapter for apache's logging, the one we use. They also have and adapter for slf4j so we likely can remove at least some of the 4 versions of this module our dependencies make use of. I made a start in attached patch but am currently stuck in maven dependency resolve hell courtesy of our slf4j. Fixing will take some concentration and a good net connection, an item I currently lack. Other TODOs are that will need to fix our little log level setting jsp page -- will likely have to undo our use of hadoop's tool here -- and the config system changes a little. I will return to this project soon. Will bring numbers. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (HBASE-10092) Move up on to log4j2
[ https://issues.apache.org/jira/browse/HBASE-10092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14015927#comment-14015927 ] Andrew Purtell edited comment on HBASE-10092 at 6/2/14 9:36 PM: The patch from HBASE-11284 Change summary:\\ - 53 files changed, 497 insertions, 531 deletions - POM changes: Import log4j2 artifacts only into the compiler classpath. Include the artifacts that wire log4j2 up to commons-logging and slf4j. Exclude log4j1 except in test scope (required by Hadoop). Bring slf4j into test scope also (required by Hadoop, ZooKeeper, and Thrift). - org.apache.log4j.Level is an internal implementation detail of 1.x no longer available. Remove all uses. - org.apache.log4j.Appender is also an internal implementation detail of 1.x. One unit test and a log management utility requires reimplementation or removal. For now offending code is commented out and marked with TODO - Replace log4j.properties files with log4j2.xml and log4j2-test.xml files. Not sure if these are functional yet. May require tweaks if significantly deviating from old behavior. conf/hbase-env.sh and bin/hbase required minor related changes. Haven't figured out how to do custom log levels yet like we had in the old log4j.properties files. was (Author: apurtell): The patch from HBASE-11284 Move up on to log4j2 Key: HBASE-10092 URL: https://issues.apache.org/jira/browse/HBASE-10092 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Attachments: 10092.txt, 10092v2.txt, HBASE-10092.patch Allows logging with less friction. See http://logging.apache.org/log4j/2.x/ This rather radical transition can be done w/ minor change given they have an adapter for apache's logging, the one we use. They also have and adapter for slf4j so we likely can remove at least some of the 4 versions of this module our dependencies make use of. I made a start in attached patch but am currently stuck in maven dependency resolve hell courtesy of our slf4j. Fixing will take some concentration and a good net connection, an item I currently lack. Other TODOs are that will need to fix our little log level setting jsp page -- will likely have to undo our use of hadoop's tool here -- and the config system changes a little. I will return to this project soon. Will bring numbers. -- This message was sent by Atlassian JIRA (v6.2#6252)
