Good news, another approach for implementing LOG4J2-3341 has been landed, and this time it is OK for us to pass hbase.root.logger in. This means the most blocker issue for backporting the log4j2 support to branch-2 is gone. We could try it once 2.17.2 is out.
Andrew Purtell <apurt...@apache.org> 于2022年2月8日周二 09:39写道： > When I was looking at the PR I also noticed a potentially annoying and > compatibility breaking problem with the properties file format. Is it > really true that with log4j1 the properties begin with 'log4j.' and with > log4j2 the 'log4j.' prefix is removed? If so user configured files won't be > compatible unless we can intercept the parse of the file and map the one to > the other, somehow. Pardon my ignorance if that would not be necessary and > there is actually a path to compatibility. Anyway I mention it because if > it is an issue, for both of these issues, perhaps we can plug something in > rather than rely on the log4j community? It is just a thought... > I do not get the point here. You mean we could do a configuration file translation? In the new release version we will provide a new log4j2 format properties file, and I guess for most users they do not need to modify this file, just modify hbase-env.sh to control the level and appenders is enough. Even if users have provided their own version of log4j.properties, we have log4j-1.2-api, it will read the log4j.properties and initialize the logging system based on log4j1 properties file, unless you have implemented your log4j1 appenders. > > On Mon, Feb 7, 2022 at 4:26 PM 张铎(Duo Zhang) <palomino...@gmail.com> > wrote: > > > Yes, the current solution does not work, they do not want to do variable > > lookup before entering the format free configuration processing step. > > > > Will try to push the log4j community to give another solution. > > > > Thanks. > > > > Andrew Purtell <apurt...@apache.org> 于2022年2月8日周二 03:54写道： > > > > > So it looks like LOG4J2-3341 is going to be reverted? Backport seems > > > premature until this is resolved with the new solution. > > > > > > On Sat, Feb 5, 2022 at 12:04 PM Andrew Purtell <apurt...@apache.org> > > > wrote: > > > > > > > Thank you for providing a PR, will review on Monday. > > > > > > > > On Sat, Feb 5, 2022 at 5:16 AM 张铎(Duo Zhang) <palomino...@gmail.com> > > > > wrote: > > > > > > > >> https://github.com/apache/hbase/pull/4096 > > > >> > > > >> The PR based on log4j 2.17.2-SNAPSHOT is ready. > > > >> > > > >> PTAL. > > > >> > > > >> Next I will build the tarball and test whether the logging works as > > > >> expected. > > > >> > > > >> 张铎(Duo Zhang) <palomino...@gmail.com> 于2022年2月5日周六 16:05写道： > > > >> > > > >> > On the config file format, on master branch I switched to xml > > because > > > >> most > > > >> > log4j2 related resources(for example, answers on stackoverflow...) > > are > > > >> xml > > > >> > based. But LOG4J2-3341 can only work with properties based config > > file > > > >> > format so the plan is to change back to properties file. > > > >> > Of course you still need to tweak the config files, as the config > > > names > > > >> > are not exactly the same, but if we have LOG4J2-3341 then I do not > > > think > > > >> > our users need to tweak the scripts, this is good news at least. > > > >> > > > > >> > We have already tried our best to not introduce log4j dependencies > > to > > > >> our > > > >> > downstream users so I think the current branch-2.5 is already > > > >> > 'incompatible' enough for our users as in the old time we were > > likely > > > to > > > >> > pull in log4j and slf4j-log4j if they depend on > hbase-testing-util. > > > But > > > >> for > > > >> > me, I do not think introducing log4j as a dependency is the > correct > > > way > > > >> so > > > >> > I would like to include this and add a section in the release > > > >> announcement > > > >> > to say this. > > > >> > > > > >> > In short, I'm +1 on switching to log4j2 on branch-2.5 and I will > > offer > > > >> my > > > >> > help to land the changes. > > > >> > > > > >> > Thanks. > > > >> > > > > >> > Andrew Purtell <apurt...@apache.org> 于2022年2月1日周二 02:49写道： > > > >> > > > > >> >> That is good news. > > > >> >> WDYT about integrating this work into branch-2.5 / 2.5.0, and > > > >> branch-2, of > > > >> >> course. Is it compatible enough? Needing to update a properties > > file > > > >> and > > > >> >> tweaks to launch scripts, if any, would be fine and can be > handled > > > in a > > > >> >> release note. Switching away from a properties based format to an > > XML > > > >> >> format would not (just as example). > > > >> >> > > > >> >> On Sun, Jan 30, 2022 at 4:57 AM 张铎(Duo Zhang) < > > palomino...@gmail.com > > > > > > > >> >> wrote: > > > >> >> > > > >> >> > Good news, LOG4J2-3341 has been landed. It will be released in > > > log4j > > > >> >> > 2.17.2. > > > >> >> > > > > >> >> > Will start to test it soon. > > > >> >> > > > > >> >> > Thanks. > > > >> >> > > > > >> >> > 张铎(Duo Zhang) <palomino...@gmail.com> 于2022年1月25日周二 17:15写道： > > > >> >> > > > > >> >> > > I would prefer we have LOG4J2-3341 first before releasing any > > 2.x > > > >> >> > releases > > > >> >> > > with log4j2. I pinged the log4j community once but no > response > > > yet. > > > >> >> Will > > > >> >> > > try to ping again soon. > > > >> >> > > > > > >> >> > > Andrew Purtell <andrew.purt...@gmail.com> 于2022年1月25日周二 > > 10:09写道： > > > >> >> > > > > > >> >> > >> Reload4J is a great option when we really should maintain > > fairly > > > >> >> strict > > > >> >> > >> compatibility (patch releases) and less so when not (minors, > > > >> majors). > > > >> >> > >> > > > >> >> > >> We haven’t released 2.5.0 yet. My fault, mainly... It’s been > > > >> lagging. > > > >> >> > >> Perhaps we could backport Duo’s work from 3.0.0-alpha into > > > >> >> branch-2.5 / > > > >> >> > >> 2.5.0? This would be a minor release, so, while some > > operational > > > >> >> > >> compatibility breaks would be somewhat surprising, it could > be > > > >> more > > > >> >> > >> understandable. > > > >> >> > >> > > > >> >> > >> > On Jan 24, 2022, at 5:00 PM, Zach York <zy...@apache.org> > > > >> wrote: > > > >> >> > >> > > > > >> >> > >> > +1 on the original discussion > > > >> >> > >> > > > > >> >> > >> > I think that all makes sense, we can't know whether one > > > >> dependency > > > >> >> is > > > >> >> > >> more > > > >> >> > >> > vulnerable/going to be more work to maintain or not. > > However, > > > I > > > >> >> think > > > >> >> > >> > perhaps the more interesting question is whether we should > > be > > > >> okay > > > >> >> > with > > > >> >> > >> > using EOL dependencies in any active release line. CVEs > > > >> happen... I > > > >> >> > >> think > > > >> >> > >> > it's important to be able to react quickly. By depending > on > > > any > > > >> EOL > > > >> >> > >> code in > > > >> >> > >> > our active release lines, we severely limit our ability to > > > >> quickly > > > >> >> > deal > > > >> >> > >> > with CVEs. I understand it's not always easy (the > backwards > > > >> >> > >> incompatibility > > > >> >> > >> > of log4j2 and the properties files are good examples), but > > > it's > > > >> >> also > > > >> >> > >> been > > > >> >> > >> > 6+ years since log4j 1.x has become EOL. > > > >> >> > >> > > > > >> >> > >> >> On Fri, Jan 21, 2022 at 11:47 AM Andrew Purtell < > > > >> >> apurt...@apache.org > > > >> >> > > > > > >> >> > >> wrote: > > > >> >> > >> >> > > > >> >> > >> >> I will offer a guess as answer to the question originally > > > >> >> proposed, > > > >> >> > >> which > > > >> >> > >> >> is, no the Logging PMC is not going to behave in a > > > cooperative > > > >> >> manner > > > >> >> > >> to > > > >> >> > >> >> Reload4J. The best we can hope for, and I personally > doubt > > > it, > > > >> is > > > >> >> > they > > > >> >> > >> will > > > >> >> > >> >> not be actively hostile. Decisions made by Apache PMC can > > > >> >> > >> unfortunately be > > > >> >> > >> >> made on the basis of years-long running arguments and > > > >> personality > > > >> >> > >> >> conflicts, and Logging is unfortunately one of them. Just > > my > > > >> >> opinion. > > > >> >> > >> You > > > >> >> > >> >> won't change it. Fortunately that is neither here nor > > there. > > > We > > > >> >> > aren't > > > >> >> > >> here > > > >> >> > >> >> to judge up front if Reload4J can respond adequately to > > > >> >> hypothetical > > > >> >> > >> future > > > >> >> > >> >> security issues. That is not knowable and remains to be > > seen. > > > >> It > > > >> >> also > > > >> >> > >> >> remains to be seen if Log4J 2 is going to respond > > adequately > > > to > > > >> >> > future > > > >> >> > >> >> security issues. Or Netty. Or Jersey. Or Jackson. Or any > of > > > our > > > >> >> other > > > >> >> > >> risky > > > >> >> > >> >> third party dependencies (as measured by having > > "interesting" > > > >> >> CVEs). > > > >> >> > >> What > > > >> >> > >> >> we can do is look at the current track record, which has > > been > > > >> >> > adequate > > > >> >> > >> so > > > >> >> > >> >> far. > > > >> >> > >> >> > > > >> >> > >> >> > > > >> >> > >> >> On Fri, Jan 21, 2022 at 11:35 AM Wei-Chiu Chuang > > > >> >> > >> >> <weic...@cloudera.com.invalid> wrote: > > > >> >> > >> >> > > > >> >> > >> >>> The reload4j is maintained by one of the Apache Logging > > PMC. > > > >> And > > > >> >> a > > > >> >> > >> >> release > > > >> >> > >> >>> was made to address the latest log4j 1.x CVEs announced > > > only a > > > >> >> day > > > >> >> > >> after. > > > >> >> > >> >>> > > > >> >> > >> >>> There is a thread in the private@logging that mentioned > > the > > > >> >> “new” > > > >> >> > >> >> log4j1.x > > > >> >> > >> >>> CVEs were actually not new. They were announced years > > before > > > >> for > > > >> >> 2.x > > > >> >> > >> >> which > > > >> >> > >> >>> happens to also impact 1.x. But because 1.x was > considered > > > >> EOL, > > > >> >> they > > > >> >> > >> >> didn’t > > > >> >> > >> >>> bother to mention 1.x were affected. It is only now that > > > 1.x’s > > > >> >> > getting > > > >> >> > >> >>> interest that they did this. > > > >> >> > >> >>> > > > >> >> > >> >>> Sean Busbey <bus...@apache.org>於 2022年1月22日 > 週六，上午2:47寫道： > > > >> >> > >> >>> > > > >> >> > >> >>>> It's relevant to what kind of mitigation this is. The > > > >> >> effectiveness > > > >> >> > >> of > > > >> >> > >> >>>> reload4j to deal with "the critical CVEs of log4j 1" is > > > >> limited > > > >> >> by > > > >> >> > >> how > > > >> >> > >> >>>> likely it is that they know about them. > > > >> >> > >> >>>> > > > >> >> > >> >>>> Otherwise at the next CVE we're back in the same place > > > where > > > >> >> > >> downstream > > > >> >> > >> >>>> users aren't meaningfully more protected. And in that > > case > > > >> >> perhaps > > > >> >> > we > > > >> >> > >> >>> would > > > >> >> > >> >>>> do better for our users by e.g. putting more emphasis > > > >> upgrading > > > >> >> > >> across > > > >> >> > >> >>> our > > > >> >> > >> >>>> releases or providing breaking changes to get the pain > > over > > > >> >> with. > > > >> >> > >> >>>> > > > >> >> > >> >>>> On Fri, Jan 21, 2022 at 11:03 AM Andrew Purtell < > > > >> >> > >> >>> andrew.purt...@gmail.com> > > > >> >> > >> >>>> wrote: > > > >> >> > >> >>>> > > > >> >> > >> >>>>> That does not seem germane to this discussion. We > don’t > > > >> >> > investigate > > > >> >> > >> >> and > > > >> >> > >> >>>>> attempt to manage the security reporting arrangement > of > > > any > > > >> of > > > >> >> our > > > >> >> > >> >>> other > > > >> >> > >> >>>>> third party dependencies. > > > >> >> > >> >>>>> > > > >> >> > >> >>>>>> On Jan 21, 2022, at 7:59 AM, Sean Busbey < > > > >> bus...@apache.org> > > > >> >> > >> >> wrote: > > > >> >> > >> >>>>>> > > > >> >> > >> >>>>>> Has anyone asked the ASF Logging PMC if they'll > > forward > > > >> >> security > > > >> >> > >> >>>> reports > > > >> >> > >> >>>>>> against log4j 1 to the reload4j project? > > > >> >> > >> >>>>>> > > > >> >> > >> >>>>>>> On Fri, Jan 21, 2022 at 3:33 AM Pankaj Kumar < > > > >> >> > >> >>> pankajku...@apache.org> > > > >> >> > >> >>>>> wrote: > > > >> >> > >> >>>>>>> > > > >> >> > >> >>>>>>> +1 for reload4j. > > > >> >> > >> >>>>>>> > > > >> >> > >> >>>>>>> Regards, > > > >> >> > >> >>>>>>> Pankaj > > > >> >> > >> >>>>>>> > > > >> >> > >> >>>>>>>> On Fri, Jan 21, 2022, 2:39 PM 张铎(Duo Zhang) < > > > >> >> > >> >> palomino...@gmail.com > > > >> >> > >> >>>> > > > >> >> > >> >>>>> wrote: > > > >> >> > >> >>>>>>>> > > > >> >> > >> >>>>>>>> Already filed HBASE-26691. > > > >> >> > >> >>>>>>>> > > > >> >> > >> >>>>>>>> Wei-Chiu Chuang <weic...@apache.org> 于2022年1月21日周五 > > > >> 16:53写道： > > > >> >> > >> >>>>>>>> > > > >> >> > >> >>>>>>>>> +1 I am doing the same in Hadoop. > > > >> >> > >> >>>>>>>>> > > > >> >> > >> >>>>>>>>> On Fri, Jan 21, 2022 at 4:51 PM Viraj Jasani < > > > >> >> > >> >> vjas...@apache.org> > > > >> >> > >> >>>>>>> wrote: > > > >> >> > >> >>>>>>>>> > > > >> >> > >> >>>>>>>>>> +1 for Reload4J migration in active release > > branches. > > > >> >> > >> >>>>>>>>>> > > > >> >> > >> >>>>>>>>>> > > > >> >> > >> >>>>>>>>>> On Fri, 21 Jan 2022 at 12:52 PM, Andrew Purtell < > > > >> >> > >> >>>>>>>>> andrew.purt...@gmail.com> > > > >> >> > >> >>>>>>>>>> wrote: > > > >> >> > >> >>>>>>>>>> > > > >> >> > >> >>>>>>>>>>> +1 for migrating to Reload4J. It is binary and > > > >> >> configuration > > > >> >> > >> >>>>>>>> compatible > > > >> >> > >> >>>>>>>>>>> with log4j 1 so meets our compatibility > > guidelines. > > > >> >> > >> >>>>>>>>>>> > > > >> >> > >> >>>>>>>>>>> If this is an agreeable plan I can make the > > changes > > > >> in a > > > >> >> PR > > > >> >> > >> >> and > > > >> >> > >> >>> we > > > >> >> > >> >>>>>>>> can > > > >> >> > >> >>>>>>>>> do > > > >> >> > >> >>>>>>>>>>> a round of new releases. > > > >> >> > >> >>>>>>>>>>> > > > >> >> > >> >>>>>>>>>>>> On Jan 20, 2022, at 10:16 PM, Duo Zhang < > > > >> >> > zhang...@apache.org > > > >> >> > >> >>> > > > >> >> > >> >>>>>>>> wrote: > > > >> >> > >> >>>>>>>>>>>> > > > >> >> > >> >>>>>>>>>>>> On master we have already migrated to log4j2, > > but > > > >> for > > > >> >> all > > > >> >> > >> >>> other > > > >> >> > >> >>>>>>>>>> release > > > >> >> > >> >>>>>>>>>>>> lines we are still on log4j1. > > > >> >> > >> >>>>>>>>>>>> > > > >> >> > >> >>>>>>>>>>>> Recently there are several new CVEs for log4j1, > > so > > > I > > > >> >> think > > > >> >> > we > > > >> >> > >> >>>>>>>> should > > > >> >> > >> >>>>>>>>>> also > > > >> >> > >> >>>>>>>>>>>> address them for release lines other than > master. > > > >> >> > >> >>>>>>>>>>>> > > > >> >> > >> >>>>>>>>>>>> One possible solution is to also migrate log4j2 > > but > > > >> use > > > >> >> > >> >> log4j12 > > > >> >> > >> >>>>>>>>> bridge > > > >> >> > >> >>>>>>>>>> to > > > >> >> > >> >>>>>>>>>>>> maintain the compatibility, but we have already > > > known > > > >> >> that > > > >> >> > >> >>>>>>> log4j12 > > > >> >> > >> >>>>>>>>>> bridge > > > >> >> > >> >>>>>>>>>>>> can not work perfectly with hadoop, as hadoop > has > > > >> some > > > >> >> > >> >>> customized > > > >> >> > >> >>>>>>>>>> log4j1 > > > >> >> > >> >>>>>>>>>>>> appender implementations, which inherit some > > log4j1 > > > >> >> > appenders > > > >> >> > >> >>>>>>> which > > > >> >> > >> >>>>>>>>> are > > > >> >> > >> >>>>>>>>>>> not > > > >> >> > >> >>>>>>>>>>>> part of the log4j12 bridge. > > > >> >> > >> >>>>>>>>>>>> > > > >> >> > >> >>>>>>>>>>>> Reload4j is a fork of the log4j1 and has fixed > > the > > > >> >> critical > > > >> >> > >> >>> CVEs, > > > >> >> > >> >>>>>>>> so > > > >> >> > >> >>>>>>>>> it > > > >> >> > >> >>>>>>>>>>> is > > > >> >> > >> >>>>>>>>>>>> less hurt to replace log4j with reload4j. > > > >> >> > >> >>>>>>>>>>>> > > > >> >> > >> >>>>>>>>>>>> Suggestions are welcomed. > > > >> >> > >> >>>>>>>>>>>> > > > >> >> > >> >>>>>>>>>>>> Thanks. Regards > > > >> >> > >> >>>>>>>>>>> > > > >> >> > >> >>>>>>>>>> > > > >> >> > >> >>>>>>>>> > > > >> >> > >> >>>>>>>> > > > >> >> > >> >>>>>>> > > > >> >> > >> >>>>> > > > >> >> > >> >>>> > > > >> >> > >> >>> > > > >> >> > >> >> > > > >> >> > >> >> > > > >> >> > >> >> -- > > > >> >> > >> >> Best regards, > > > >> >> > >> >> Andrew > > > >> >> > >> >> > > > >> >> > >> >> Unrest, ignorance distilled, nihilistic imbeciles - > > > >> >> > >> >> It's what we’ve earned > > > >> >> > >> >> Welcome, apocalypse, what’s taken you so long? > > > >> >> > >> >> Bring us the fitting end that we’ve been counting on > > > >> >> > >> >> - A23, Welcome, Apocalypse > > > >> >> > >> >> > > > >> >> > >> > > > >> >> > > > > > >> >> > > > > >> >> > > > >> >> > > > >> >> -- > > > >> >> Best regards, > > > >> >> Andrew > > > >> >> > > > >> >> Unrest, ignorance distilled, nihilistic imbeciles - > > > >> >> It's what we’ve earned > > > >> >> Welcome, apocalypse, what’s taken you so long? > > > >> >> Bring us the fitting end that we’ve been counting on > > > >> >> - A23, Welcome, Apocalypse > > > >> >> > > > >> > > > > >> > > > > > > > > > > > > -- > > > > Best regards, > > > > Andrew > > > > > > > > Unrest, ignorance distilled, nihilistic imbeciles - > > > > It's what we’ve earned > > > > Welcome, apocalypse, what’s taken you so long? > > > > Bring us the fitting end that we’ve been counting on > > > > - A23, Welcome, Apocalypse > > > > > > > > > > > > > -- > > > Best regards, > > > Andrew > > > > > > Unrest, ignorance distilled, nihilistic imbeciles - > > > It's what we’ve earned > > > Welcome, apocalypse, what’s taken you so long? > > > Bring us the fitting end that we’ve been counting on > > > - A23, Welcome, Apocalypse > > > > > > > > -- > Best regards, > Andrew > > Unrest, ignorance distilled, nihilistic imbeciles - > It's what we’ve earned > Welcome, apocalypse, what’s taken you so long? > Bring us the fitting end that we’ve been counting on > - A23, Welcome, Apocalypse >
