Thank you for the notification, it will be included. The RC is still in progress as some minor issue came up during the release process.
On Tue, Dec 4, 2018 at 3:41 PM Ralph Goers <ralph.go...@dslextreme.com> wrote: > I am noticing that the release notes on the 1.9 branch have not been > updated to say changes to the logging configuration will be required in the > next release as it will be upgraded to Log4j 2. > > Ralph > > > On Nov 30, 2018, at 8:33 PM, Ralph Goers <ralph.go...@dslextreme.com> > wrote: > > > > Ok, I’m fine with that. > > > > Ralph > > > >> On Nov 30, 2018, at 3:31 PM, Mike Percy <mpe...@apache.org> wrote: > >> > >> Hi Ralph, > >> Let’s update the release notes to indicate that it is coming and do it > in the next release so people are forewarned of the change. After speaking > with others it sounds like Flume users are willing to accept the change > (some have already had to do it because of Solr) so I have changed my view > regarding whether we should wait for Flume 2.0 to do it. I’m now on board > doing the log4j2 switch in a 1.10 release if that is what we want to > release next time. > >> > >> Mike > >> > >>> On Nov 30, 2018, at 2:20 PM, Ralph Goers <ralph.go...@dslextreme.com> > wrote: > >>> > >>> Mike, > >>> > >>> This response confuses me. Are you saying that because the customers > are good with the config change Flume 1.9 can go out with Log4j 2 or are > you saying you are good with updating the release notes to say it will > happen in the next release? > >>> > >>> Ralph > >>> > >>>> On Nov 29, 2018, at 10:53 PM, Mike Percy <mpe...@apache.org> wrote: > >>>> > >>>> After speaking with a couple of large users of Flume, because log4j2 > is ABI compatible with log4j1 It sounds like the config change will be > acceptable to them at upgrade time and so I am +1 on this proposal. > >>>> > >>>> Regards, > >>>> Mike > >>>> > >>>>> On Nov 28, 2018, at 5:37 AM, Ralph Goers <ralph.go...@dslextreme.com> > wrote: > >>>>> > >>>>> Again, please update the release notes to indicate the logging > configuration change will be coming with the next release. > >>>>> > >>>>> Ralph > >>>>> > >>>>>> On Nov 28, 2018, at 1:17 AM, Denes Arvay <de...@cloudera.com.INVALID> > wrote: > >>>>>> > >>>>>> Hi All, > >>>>>> > >>>>>> I agree that the current content doesn't justify the 2.0 release, > so I'd > >>>>>> vote for moving forward with the 1.9 as Ferenc recommended, i.e. > without > >>>>>> the 2 previously mentioned breaking changes. > >>>>>> I also agree with the proposed new features, especially with the > plugin > >>>>>> system/modularization, which was discussed on the other thread: > >>>>>> https://s.apache.org/2nm9 > >>>>>> > >>>>>> So, Ferenc, I think you are good to go with the proposed plan, > unless > >>>>>> anybody vetoes. > >>>>>> > >>>>>> Regards, > >>>>>> Denes > >>>>>> > >>>>>> On Tue, Nov 27, 2018 at 4:37 PM Ralph Goers < > ralph.go...@dslextreme.com> > >>>>>> wrote: > >>>>>> > >>>>>>> While Log4j 2 supports using properties files for configuration it > is not > >>>>>>> syntactically compatible with Log4j 1. However, migrating a Log4j 1 > >>>>>>> configuration to Log4j 2 isn’t a difficult task. > >>>>>>> > >>>>>>> Ralph > >>>>>>> > >>>>>>>> On Nov 27, 2018, at 4:38 AM, Tristan Stevens > >>>>>>> <tris...@cloudera.com.INVALID> wrote: > >>>>>>>> > >>>>>>>> I did start thinking a while back about a REST API that could be > used for > >>>>>>>> configuring Flume, after which you could potentially bolt on a > UI. I > >>>>>>>> stopped work on it because it got tricky around configuring > sources (and > >>>>>>>> the way in which they relate to channels etc). If someone had > more time > >>>>>>> I’d > >>>>>>>> be happy to push to a repo the work that I did. This could be a > really > >>>>>>>> useful 2.0 feature. > >>>>>>>> > >>>>>>>> Regarding move from Log4j1.x to 2, it’d be interesting to see > what other > >>>>>>>> Apache projects, such as Hadoop, Hive etc did around this (in > fact, are > >>>>>>>> they even at 2.x yet?). For me, if you need to change config, > it’s not a > >>>>>>>> minor release, it become major, unless there’s a way you can > >>>>>>> automatically > >>>>>>>> migrate config from one to the other. I’ve not checked, but are > we sure > >>>>>>>> that .properties file won’t work with log4j2? > >>>>>>>> > >>>>>>>> Tristan > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> On 26 November 2018 at 19:54:01, Mike Percy (mpe...@apache.org) > wrote: > >>>>>>>> > >>>>>>>> Yeah, I agree with this, especially classloader isolation would > be great > >>>>>>> to > >>>>>>>> have as well on the plugin side. > >>>>>>>> > >>>>>>>> Mike > >>>>>>>> > >>>>>>>> On Mon, Nov 26, 2018 at 11:50 AM Ralph Goers < > ralph.go...@dslextreme.com > >>>>>>>> > >>>>>>>> wrote: > >>>>>>>> > >>>>>>>>> I think people want to do more than just upgrade Log4j for a > Flume 2.0 > >>>>>>>>> release. I would like to make configuration much more pluggable > for > >>>>>>>>> example. There was also talk about splitting all the sources, > sinks, > >>>>>>>>> interceptors, etc out of the core module and making them some > sort of > >>>>>>>>> plugin. At this point those are just at the idea stage. > >>>>>>>>> > >>>>>>>>>> On Nov 26, 2018, at 12:19 PM, Bessenyei Balázs Donát < > >>>>>>> bes...@apache.org> > >>>>>>>> > >>>>>>>>> wrote: > >>>>>>>>>> > >>>>>>>>>> I wonder, is there anything blocking us from releasing 2.0 next > instead > >>>>>>>>> of 1.x? > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> Donat > >>>>>>>>>> > >>>>>>>>>>> On Mon, Nov 26, 2018 at 6:27 PM Mike Percy <mpe...@apache.org> > wrote: > >>>>>>>>>>> > >>>>>>>>>>> If only the PMC would be affected by this decision, we could > have had > >>>>>>>>> this > >>>>>>>>>>> discussion on the PMC private list. But this decision impacts > >>>>>>>> everybody > >>>>>>>>>>> that uses Flume. So let's hear from anybody who cares about > this, > >>>>>>>>> including > >>>>>>>>>>> committers, contributors, and users on whether they are okay > with > >>>>>>>>> switching > >>>>>>>>>>> to log4j2 in a minor release version, knowing that they will > need to > >>>>>>>>> change > >>>>>>>>>>> their config files when they upgrade Flume. > >>>>>>>>>>> > >>>>>>>>>>> Ferenc, it seems like we will have to ship one or the other > with the > >>>>>>>>> binary > >>>>>>>>>>> artifacts at release time. It seems to me that we still have > to make a > >>>>>>>>>>> choice about the built and shipped default, even if both could > work at > >>>>>>>>>>> runtime, right? > >>>>>>>>>>> > >>>>>>>>>>> Thanks, > >>>>>>>>>>> Mike > >>>>>>>>>>> > >>>>>>>>>>> On Mon, Nov 26, 2018 at 5:58 AM Ferenc Szabo > >>>>>>>>> <fsz...@cloudera.com.invalid> > >>>>>>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>>> I have directed it to the community, the PMC members. > >>>>>>>>>>>> I am looking for a decision from the PMC members (including > You) on > >>>>>>>>> whether > >>>>>>>>>>>> we should continue as planned or not removing log4j2 form the > minor > >>>>>>>>> release > >>>>>>>>>>>> branch. > >>>>>>>>>>>> > >>>>>>>>>>>> On Mon, Nov 26, 2018 at 2:43 PM Ralph Goers < > >>>>>>>>> ralph.go...@dslextreme.com> > >>>>>>>>>>>> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>>> Was this directed at me? I am not sure what decision you are > >>>>>>>> referring > >>>>>>>>>>>> to. > >>>>>>>>>>>>> I stated my position in the email you replied to. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Ralph > >>>>>>>>>>>>> > >>>>>>>>>>>>>> On Nov 26, 2018, at 5:29 AM, Ferenc Szabo > >>>>>>>>> <fsz...@cloudera.com.INVALID > >>>>>>>>>>>>> > >>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> With the new release, you can just replace the jars on the > >>>>>>>> classpath > >>>>>>>>> as > >>>>>>>>>>>>> we > >>>>>>>>>>>>>> removed every code dependency and using slf4j. So you do > not need > >>>>>>>> to > >>>>>>>>>>>> fork > >>>>>>>>>>>>>> and change anything. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Let me know your decision and we will continue with that. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Regards, > >>>>>>>>>>>>>> Ferenc > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> On Mon, Nov 26, 2018 at 12:10 PM Apache < > >>>>>>> ralph.go...@dslextreme.com> > >>>>>>>> > >>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> As I said, I am fine with adding a notice to the release > notes > >>>>>>>>> stating > >>>>>>>>>>>>> the > >>>>>>>>>>>>>>> logging will be changed in the next release regardless of > its > >>>>>>>>> version > >>>>>>>>>>>>>>> number. This way the revert only needs to happen on the > release > >>>>>>>>> branch > >>>>>>>>>>>>> for > >>>>>>>>>>>>>>> this release. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Ralph > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> On Nov 25, 2018, at 9:59 PM, Mike Percy < > mpe...@apache.org> > >>>>>>>> wrote: > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> While the committer veto rules are well documented here > >>>>>>>>>>>>>>>> <https://www.apache.org/foundation/voting.html#Veto>, > and there > >>>>>>>> is > >>>>>>>>>>>> no > >>>>>>>>>>>>>>>> mention of a time limit, I propose we shelve that > discussion and > >>>>>>>>> work > >>>>>>>>>>>>> on > >>>>>>>>>>>>>>>> getting to consensus on what we as a community want to > include in > >>>>>>>>> the > >>>>>>>>>>>>>>> Flume > >>>>>>>>>>>>>>>> 1.9 release. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> As far as I can tell, the decision we have to make is > whether to > >>>>>>>>>>>>> include > >>>>>>>>>>>>>>>> the FLUME-2050 logging changes in the Flume 1.9 release. > We can > >>>>>>>>>>>> either: > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> 1) Require users upgrading from Flume 1.8 to Flume 1.9 to > have to > >>>>>>>>>>>>> modify > >>>>>>>>>>>>>>>> their log4j configuration files to use the different > log4j2 XML > >>>>>>>>>>>> format, > >>>>>>>>>>>>>>>> with the procedure for doing so documented in the release > notes. > >>>>>>>>>>>>>>>> -or- > >>>>>>>>>>>>>>>> 2) Defer that change to a future release where > incompatible > >>>>>>>> changes > >>>>>>>>>>>> are > >>>>>>>>>>>>>>>> expected, such as Flume 2.0. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Also, maybe there are other options I haven't thought > of...? > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> I would like to get some input from more people on this > matter. > >>>>>>>> How > >>>>>>>>>>>> do > >>>>>>>>>>>>>>>> others feel about this? > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>> Mike > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> On Sat, Nov 24, 2018 at 9:36 PM Ralph Goers < > >>>>>>>>>>>>> ralph.go...@dslextreme.com> > >>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> I should also point out that the time to raise this was > a year > >>>>>>>> ago > >>>>>>>>>>>>> when > >>>>>>>>>>>>>>>>> the PR for FLUME-2050 was reviewed and committed. As it > stands > >>>>>>>>> now I > >>>>>>>>>>>>>>> could > >>>>>>>>>>>>>>>>> be a jerk and vote -1 on the patch for FLUME-3296 with > valid > >>>>>>>>>>>> technical > >>>>>>>>>>>>>>>>> grounds. If this was causing a true binary > incompatibility I > >>>>>>>> would > >>>>>>>>>>>>>>> approve > >>>>>>>>>>>>>>>>> reverting it in a heartbeat, but I just don’t see how > having > >>>>>>>> users > >>>>>>>>>>>>> have > >>>>>>>>>>>>>>> to > >>>>>>>>>>>>>>>>> change logging configuration is “intolerable”, > especially with > >>>>>>>> the > >>>>>>>>>>>>> known > >>>>>>>>>>>>>>>>> security issues in Log4j 1, however unlikely user’s > might be to > >>>>>>>>>>>>>>> encounter > >>>>>>>>>>>>>>>>> them. > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> That said, I wouldn’t veto the revert on the release > branch, but > >>>>>>>> I > >>>>>>>>>>>>> would > >>>>>>>>>>>>>>>>> suggest that the release notes provide fair warning that > the > >>>>>>>> next > >>>>>>>>>>>>>>> release > >>>>>>>>>>>>>>>>> will upgrade the logging dependency. It would also be > nice if > >>>>>>>>>>>>> releases > >>>>>>>>>>>>>>>>> could be more frequent than once a year. > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> I would also like to say that I’m not doing this for the > fun of > >>>>>>>>> it. > >>>>>>>>>>>> My > >>>>>>>>>>>>>>>>> company uses Flume for some of its most critical > processing. We > >>>>>>>>> run > >>>>>>>>>>>>>>>>> Veracode scans on all of our software and I expect Flume > would > >>>>>>>> be > >>>>>>>>>>>>>>> flagged > >>>>>>>>>>>>>>>>> if I hadn’t repacked it with Log4j 2. It also may not > show up > >>>>>>>>> since > >>>>>>>>>>>>> the > >>>>>>>>>>>>>>> CVE > >>>>>>>>>>>>>>>>> is marked against Log4j 2 since Log4j 1 is EOL, but the > security > >>>>>>>>>>>>>>> scanning > >>>>>>>>>>>>>>>>> tools should be flagging that as well. > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> FWIW I’ve had to hack or enhance a few Flume components > to make > >>>>>>>> it > >>>>>>>>>>>>> work > >>>>>>>>>>>>>>>>> for my needs but overall it works really, really well. > I’d like > >>>>>>>> to > >>>>>>>>>>>>>>> commit > >>>>>>>>>>>>>>>>> back some of the changes but the main one - Flume > configuration > >>>>>>>> - > >>>>>>>>> I > >>>>>>>>>>>>>>> really > >>>>>>>>>>>>>>>>> don’t like and need to redo the whole thing. > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Ralph > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> On Nov 24, 2018, at 5:11 PM, Mike Percy < > mpe...@apache.org> > >>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> I wasn't aware of this security issue. Do you have a > link to > >>>>>>>> the > >>>>>>>>>>>>>>> details? > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> There's no ASF requirement for package names; it's > really just > >>>>>>>> a > >>>>>>>>>>>> Java > >>>>>>>>>>>>>>>>>> language convention, and especially not enforced for > >>>>>>>>> compatibility > >>>>>>>>>>>>>>> shims. > >>>>>>>>>>>>>>>>>> Even Flume has some of those in the legacy sources. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> I understand that it's very difficult to provide a > >>>>>>>> compatibility > >>>>>>>>>>>>> layer. > >>>>>>>>>>>>>>>>>> Maybe also a boring task. I'm just saying that without > log4j1 > >>>>>>>>>>>>> backwards > >>>>>>>>>>>>>>>>>> compatibility provided by log4j2, there will have to be > a > >>>>>>>> really > >>>>>>>>>>>>>>> critical > >>>>>>>>>>>>>>>>>> reason to inflict this migration pain on Flume users -- > >>>>>>>> something > >>>>>>>>>>>>> that > >>>>>>>>>>>>>>>>>> simply can't be tolerated, even in a minor release, > like a new > >>>>>>>>> and > >>>>>>>>>>>>>>> highly > >>>>>>>>>>>>>>>>>> dangerous security bug. Without such a motivation, I > don't see > >>>>>>>>> how > >>>>>>>>>>>>> this > >>>>>>>>>>>>>>>>>> incompatible dependency change can be justified. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Mike > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> On Sat, Nov 24, 2018 at 3:20 PM Ralph Goers < > >>>>>>>>>>>>>>> ralph.go...@dslextreme.com> > >>>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> On Nov 24, 2018, at 2:35 PM, Mike Percy < > mpe...@apache.org> > >>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Flume has long had a policy of backwards > compatibility with > >>>>>>>> its > >>>>>>>>>>>> own > >>>>>>>>>>>>>>>>>>>> configuration files and people expect things to "just > work" > >>>>>>>>> when > >>>>>>>>>>>>>>>>>>> upgrading > >>>>>>>>>>>>>>>>>>>> Flume. If log4j2 can't parse the log4j1 config file > format > >>>>>>>> then > >>>>>>>>>>>>> it's > >>>>>>>>>>>>>>> an > >>>>>>>>>>>>>>>>>>>> incompatible upgrade and should not be done in a > minor Flume > >>>>>>>>>>>>> release. > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> If log4j2 wants to be a drop-in replacement for > log4j1 then > >>>>>>>> by > >>>>>>>>>>>>>>> default > >>>>>>>>>>>>>>>>> it > >>>>>>>>>>>>>>>>>>>> should find and parse the traditional > log4j.properties config > >>>>>>>>>>>>> files, > >>>>>>>>>>>>>>> at > >>>>>>>>>>>>>>>>>>>> least as a fallback, rather than force users to > convert to > >>>>>>>> the > >>>>>>>>>>>> new > >>>>>>>>>>>>>>> XML > >>>>>>>>>>>>>>>>>>>> format before upgrading. > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> That is simply not possible: > >>>>>>>>>>>>>>>>>>> 1. Log4j 1 requires the use of fully qualified class > names. > >>>>>>>> The > >>>>>>>>>>>>>>> package > >>>>>>>>>>>>>>>>>>> names log4j 1 used didn’t conform to ASF naming > guidelines and > >>>>>>>>>>>> don’t > >>>>>>>>>>>>>>>>> line > >>>>>>>>>>>>>>>>>>> up with the package names used in Log4j2. > >>>>>>>>>>>>>>>>>>> 2. Log4j 1 did not delineate between what was public > and what > >>>>>>>>> was > >>>>>>>>>>>>>>>>> private > >>>>>>>>>>>>>>>>>>> so there is code all over the place mucking with the > internals > >>>>>>>>> of > >>>>>>>>>>>>>>> Log4j. > >>>>>>>>>>>>>>>>>>> Log4j 2 implemented a compatibility layer by > implementing the > >>>>>>>>>>>>> classes > >>>>>>>>>>>>>>>>> that > >>>>>>>>>>>>>>>>>>> are being used but by and large they don’t actually do > >>>>>>>> anything. > >>>>>>>>>>>>>>>>>>> 3. The Appenders and Filters in Log4j 2 implement > different > >>>>>>>>>>>>> interfaces > >>>>>>>>>>>>>>>>> and > >>>>>>>>>>>>>>>>>>> cannot use components from Log4j 1. > >>>>>>>>>>>>>>>>>>> 4. The Appenders in Log4j 2 are not identical with > Log4j 1. In > >>>>>>>>>>>> fact, > >>>>>>>>>>>>>>>>> many > >>>>>>>>>>>>>>>>>>> people didn’t use the Rolling File Appender that > shipped with > >>>>>>>>>>>> Log4j > >>>>>>>>>>>>>>> but > >>>>>>>>>>>>>>>>>>> used the one from Log4j extras. So it is hard to.know > what > >>>>>>>>> Log4j 2 > >>>>>>>>>>>>>>> would > >>>>>>>>>>>>>>>>>>> have needed to be compatible with. > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> Many organizations (including mine) have security > requirements > >>>>>>>>>>>> that > >>>>>>>>>>>>>>> say > >>>>>>>>>>>>>>>>>>> they must use software that is supported with security > fixes. > >>>>>>>>>>>> Log4j > >>>>>>>>>>>>> 1 > >>>>>>>>>>>>>>>>> has a > >>>>>>>>>>>>>>>>>>> known security bug that will never be fixed as it > reached > >>>>>>>>>>>>> end-of-life > >>>>>>>>>>>>>>> in > >>>>>>>>>>>>>>>>>>> August of 2015. This means the use of Log4j 1 is not > >>>>>>>> acceptable > >>>>>>>>>>>> for > >>>>>>>>>>>>>>> any > >>>>>>>>>>>>>>>>>>> security conscious organization. > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> While folks have brought up this issue from time to > time most > >>>>>>>>> seem > >>>>>>>>>>>>> to > >>>>>>>>>>>>>>>>>>> adapt quite quickly to the change. Most of the issues > are > >>>>>>>>>>>> developers > >>>>>>>>>>>>>>>>>>> wanting to know how to make something they were doing > in Log4j > >>>>>>>> 1 > >>>>>>>>>>>>> work > >>>>>>>>>>>>>>> in > >>>>>>>>>>>>>>>>>>> Log4j 2. > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> As you can imagine, since Log4j 2 has been GA now for > 4 1/2 > >>>>>>>>> years > >>>>>>>>>>>>> and > >>>>>>>>>>>>>>>>>>> Log4j 1 has been EOL for over 3, making the > configuration > >>>>>>>>>>>> compatible > >>>>>>>>>>>>>>>>> with > >>>>>>>>>>>>>>>>>>> Log4j 1 is not a high priority. FWIW, there was an > effort to > >>>>>>>> do > >>>>>>>>>>>>> that a > >>>>>>>>>>>>>>>>> few > >>>>>>>>>>>>>>>>>>> years ago, and while we were able to get some basic > stuff to > >>>>>>>>> work > >>>>>>>>>>>>>>>>> making it > >>>>>>>>>>>>>>>>>>> work in a general way wasn’t possible. > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> Ralph > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Mike > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> On Fri, Nov 23, 2018 at 9:39 AM Ralph Goers < > >>>>>>>>>>>>>>>>> ralph.go...@dslextreme.com> > >>>>>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> No, you are correct. However, requiring a change to > logging > >>>>>>>>>>>>>>>>>>> configuration > >>>>>>>>>>>>>>>>>>>>> has never been considered a binary compatibility > break in > >>>>>>>> any > >>>>>>>>>>>>>>> project > >>>>>>>>>>>>>>>>> I > >>>>>>>>>>>>>>>>>>>>> have ever worked on. > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> Ralph > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> On Nov 23, 2018, at 10:05 AM, Ferenc Szabo > >>>>>>>>>>>>>>>>> <fsz...@cloudera.com.INVALID > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> As you mentioned, you have the freedom to use Log4j > 2 and > >>>>>>>> the > >>>>>>>>>>>>> same > >>>>>>>>>>>>>>>>> time > >>>>>>>>>>>>>>>>>>>>> we > >>>>>>>>>>>>>>>>>>>>>> have to keep the out of the box experience the same > in a > >>>>>>>>> minor > >>>>>>>>>>>>>>>>> version. > >>>>>>>>>>>>>>>>>>>>>> Users should be able to upgrade flume without > changing any > >>>>>>>> of > >>>>>>>>>>>>> their > >>>>>>>>>>>>>>>>>>>>>> configurations. > >>>>>>>>>>>>>>>>>>>>>> If they have a log4j.properties (Log4j 1) then they > would > >>>>>>>> not > >>>>>>>>>>>> be > >>>>>>>>>>>>>>> able > >>>>>>>>>>>>>>>>>>> to > >>>>>>>>>>>>>>>>>>>>>> use it after the upgrade without changing it. > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> Or am I missing a feature that would solve this > case? > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> On Fri, Nov 23, 2018 at 5:49 PM Ralph Goers < > >>>>>>>>>>>>>>>>>>> ralph.go...@dslextreme.com> > >>>>>>>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> Also, please put details in the Jira issues. It is > much > >>>>>>>>> easier > >>>>>>>>>>>>> to > >>>>>>>>>>>>>>>>> find > >>>>>>>>>>>>>>>>>>>>> out > >>>>>>>>>>>>>>>>>>>>>>> why something was done by searching Jira later on > then > >>>>>>>>>>>> searching > >>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>>>> mailing list. > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> Ralph > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> On Nov 23, 2018, at 9:47 AM, Ralph Goers < > >>>>>>>>>>>>>>>>> ralph.go...@dslextreme.com > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> I do not understand this at all. Log4j 2 provides > runtime > >>>>>>>>>>>>>>>>>>> compatibility > >>>>>>>>>>>>>>>>>>>>>>> with Log4j 1. What is the problem that requires a > revert? > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> I have been running Flume with Log4j 2 since 1.6 > so I > >>>>>>>> don’t > >>>>>>>>>>>>>>>>>>> understand > >>>>>>>>>>>>>>>>>>>>>>> what the problem could possibly be. > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> Ralph > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>> On Nov 23, 2018, at 8:50 AM, Ferenc Szabo < > >>>>>>>>>>>>>>> szabofe...@apache.org> > >>>>>>>>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>> Hi everyone > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>> I am about to branch the 1.9 release from trunk. > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>> On the 1.9 branch we will revert the following > breaking > >>>>>>>>>>>>> changes: > >>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-2957. Remove Guava from our public API: > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>> > https://github.com/apache/flume/commit/7f85df9e473ee675d461d5b76650694c5a6c0088 > >>>>>>>>>>>>>>>>>>>>>>>>> - part of FLUME-2050. Upgrade to Log4j 2.10.0: > >>>>>>>>>>>>>>>>>>>>>>>>> as the new release should work with the previous > >>>>>>>>>>>>> configurations > >>>>>>>>>>>>>>> we > >>>>>>>>>>>>>>>>>>>>> have > >>>>>>>>>>>>>>>>>>>>>>>>> to release it with log4j 1.x > >>>>>>>>>>>>>>>>>>>>>>>>> For the log4j2 upgrade, we will provide a guide, > how to > >>>>>>>>>>>>> replace > >>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>> jars > >>>>>>>>>>>>>>>>>>>>>>>>> if users would like to start using it in the 1.9 > release > >>>>>>>>> on > >>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>> wiki > >>>>>>>>>>>>>>>>>>>>>>> page. > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>> Because of these changes the first release > candidate > >>>>>>>> might > >>>>>>>>>>>> be > >>>>>>>>>>>>>>>>>>>>> postponed > >>>>>>>>>>>>>>>>>>>>>>> to > >>>>>>>>>>>>>>>>>>>>>>>>> Monday. > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>> Regards, > >>>>>>>>>>>>>>>>>>>>>>>>> Ferenc Szabo > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Nov 7, 2018 at 5:04 PM > ema...@cloudera.com < > >>>>>>>>>>>>>>>>>>>>> ema...@cloudera.com > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>> Hi Ferenc, > >>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>> +1 > >>>>>>>>>>>>>>>>>>>>>>>>>> I am working on FLUME-3281 Update to Kafka 2.0 > client, > >>>>>>>>>>>> should > >>>>>>>>>>>>>>> be > >>>>>>>>>>>>>>>>>>> able > >>>>>>>>>>>>>>>>>>>>>>> to > >>>>>>>>>>>>>>>>>>>>>>>>>> finish it > >>>>>>>>>>>>>>>>>>>>>>>>>> till the suggested deadline. > >>>>>>>>>>>>>>>>>>>>>>>>>> I am also happy to do some reviews. > >>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>> Regards > >>>>>>>>>>>>>>>>>>>>>>>>>> Endre > >>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>> On 2018/11/06 21:23:17, Ferenc Szabo < > >>>>>>>>>>>> szabofe...@apache.org> > >>>>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>>>>>>>>> Hello Flume Community, > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> 1.8 was released about a year ago and since > that quite > >>>>>>>> a > >>>>>>>>>>>> few > >>>>>>>>>>>>>>> bug > >>>>>>>>>>>>>>>>>>>>>>> fixes, > >>>>>>>>>>>>>>>>>>>>>>>>>>> improvements, features and documentation were > >>>>>>>>> introduced. > >>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to propose to publish the next > minor > >>>>>>>>> release > >>>>>>>>>>>> of > >>>>>>>>>>>>>>>>> Flume > >>>>>>>>>>>>>>>>>>>>>>>>>>> to make these changes available to the users. > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> I would be more than happy to be the Release > Manager > >>>>>>>>> with > >>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>> help > >>>>>>>>>>>>>>>>>>>>> of > >>>>>>>>>>>>>>>>>>>>>>>>>>> Denes Arvay for anything that requires PMC > access - if > >>>>>>>>>>>> both > >>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>>>> community > >>>>>>>>>>>>>>>>>>>>>>>>>>> and he are > >>>>>>>>>>>>>>>>>>>>>>>>>>> OK with it. > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> Among others the following changes will be > included in > >>>>>>>>> the > >>>>>>>>>>>>>>> next > >>>>>>>>>>>>>>>>>>>>>>> release: > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> Fixed bugs: > >>>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3117 Application can be dead loop when > call > >>>>>>>>>>>>>>>>> System.exit() > >>>>>>>>>>>>>>>>>>> in > >>>>>>>>>>>>>>>>>>>>>>>>>>> methodconfigure > >>>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3237 Handling RuntimeExceptions coming > from > >>>>>>>> the > >>>>>>>>>>>> JMS > >>>>>>>>>>>>>>>>>>> provider > >>>>>>>>>>>>>>>>>>>>>>> in > >>>>>>>>>>>>>>>>>>>>>>>>>>> JMSSource > >>>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3201 Fix SyslogUtil to handle RFC3164 > format > >>>>>>>> in > >>>>>>>>>>>>>>> december > >>>>>>>>>>>>>>>>>>>>>>>>>> correctly > >>>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3056 TestApplication hangs indefinitely > >>>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-2976 Exception when JMS source tries to > >>>>>>>> connect > >>>>>>>>>>>> to a > >>>>>>>>>>>>>>>>>>>>> Weblogic > >>>>>>>>>>>>>>>>>>>>>>>>>>> server without authentication > >>>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3270 Close JMS resources in > JMSMessageConsumer > >>>>>>>>>>>>>>>>> constructor > >>>>>>>>>>>>>>>>>>>>> in > >>>>>>>>>>>>>>>>>>>>>>>>>> case > >>>>>>>>>>>>>>>>>>>>>>>>>>> of failure > >>>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3222 java.nio.file.NoSuchFileException > thrown > >>>>>>>>> when > >>>>>>>>>>>>>>> files > >>>>>>>>>>>>>>>>>>> are > >>>>>>>>>>>>>>>>>>>>>>>>>> being > >>>>>>>>>>>>>>>>>>>>>>>>>>> deleted from the TAILDIR source > >>>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-2894 Flume components should stop in > the > >>>>>>>> correct > >>>>>>>>>>>>> order > >>>>>>>>>>>>>>>>>>>>>>> (graceful > >>>>>>>>>>>>>>>>>>>>>>>>>>> shutdown) > >>>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-2973 Deadlock in hdfs sink > >>>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3278 Handling -D keystore parameters > in Kafka > >>>>>>>>>>>>>>> components > >>>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3265 Cannot set batch-size for > >>>>>>>>>>>>> LoadBalancingRpcClient > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> Improvements: > >>>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3186 Make asyncHbaseClient > configuration > >>>>>>>>>>>> parameters > >>>>>>>>>>>>>>>>>>>>> available > >>>>>>>>>>>>>>>>>>>>>>>>>> from > >>>>>>>>>>>>>>>>>>>>>>>>>>> flume config > >>>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3239 Do not rename files in > >>>>>>>> SpoolDirectorySource > >>>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3227 Add Rate Limiter to StressSource > >>>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-2050 Upgrade to log4j2 (when GA) > >>>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3246 Validate flume configuration to > prevent > >>>>>>>>>>>> larger > >>>>>>>>>>>>>>>>> source > >>>>>>>>>>>>>>>>>>>>>>> batch > >>>>>>>>>>>>>>>>>>>>>>>>>>> size than the channel transaction capacity > >>>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3050 add counters for error conditions > and > >>>>>>>>> expose > >>>>>>>>>>>> to > >>>>>>>>>>>>>>>>>>> monitor > >>>>>>>>>>>>>>>>>>>>>>> URL > >>>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3269 Support JSSE keystore/trustore -D > system > >>>>>>>>>>>>>>> properties > >>>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3223 Flume HDFS Sink should retry > close prior > >>>>>>>> to > >>>>>>>>>>>>>>>>>>> performing > >>>>>>>>>>>>>>>>>>>>> a > >>>>>>>>>>>>>>>>>>>>>>>>>>> recoverLease > >>>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3276 Components supporting SSL/TLS > should be > >>>>>>>>> able > >>>>>>>>>>>> to > >>>>>>>>>>>>>>>>>>> specify > >>>>>>>>>>>>>>>>>>>>>>>>>> cipher > >>>>>>>>>>>>>>>>>>>>>>>>>>> suite list > >>>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3275 Components supporting SSL/TLS > should be > >>>>>>>>> able > >>>>>>>>>>>> to > >>>>>>>>>>>>>>>>>>> specify > >>>>>>>>>>>>>>>>>>>>>>>>>>> protocol list > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> New Features: > >>>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3142 Adding HBase2 sink > >>>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-2442 Need an alternative to providing > clear > >>>>>>>> text > >>>>>>>>>>>>>>>>> passwords > >>>>>>>>>>>>>>>>>>>>> in > >>>>>>>>>>>>>>>>>>>>>>>>>> flume > >>>>>>>>>>>>>>>>>>>>>>>>>>> config > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> There are 26 open tickets targeted for 1.9 in > patch > >>>>>>>>>>>>> available > >>>>>>>>>>>>>>>>>>> state: > >>>>>>>>>>>>>>>>>>>>>>>>>>> https://s.apache.org/flume-1.9-target-tickets > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> We also have a number of open pull requests on > Github: > >>>>>>>>>>>>>>>>>>>>>>>>>>> https://github.com/apache/flume/pulls > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> There are a few tickets in progress that would > be good > >>>>>>>>> to > >>>>>>>>>>>>>>>>> include > >>>>>>>>>>>>>>>>>>> in > >>>>>>>>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>>>>>>>> release so > >>>>>>>>>>>>>>>>>>>>>>>>>>> I would say that we focus on reviewing and > testing > >>>>>>>> them > >>>>>>>>> in > >>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>> next > >>>>>>>>>>>>>>>>>>>>> 2 > >>>>>>>>>>>>>>>>>>>>>>>>>>> weeks. > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to propose to target the 23rd of > November > >>>>>>>>> for > >>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>> first > >>>>>>>>>>>>>>>>>>>>>>>>>>> release candidate. > >>>>>>>>>>>>>>>>>>>>>>>>>>> That would mean that the branch date would be > a few > >>>>>>>> days > >>>>>>>>>>>>>>> before > >>>>>>>>>>>>>>>>>>>>> that. > >>>>>>>>>>>>>>>>>>>>>>>>>>> Any significant code change should get in by > that > >>>>>>>> date. > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody has any concerns then I am going to > create > >>>>>>>> an > >>>>>>>>>>>>>>> umbrella > >>>>>>>>>>>>>>>>>>>>>>> ticket > >>>>>>>>>>>>>>>>>>>>>>>>>> to > >>>>>>>>>>>>>>>>>>>>>>>>>>> track the release process. > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> Some movement can be expected in the related > JIRA > >>>>>>>>> tickets. > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> Kind regards, > >>>>>>>>>>>>>>>>>>>>>>>>>>> Ferenc > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>> > >>>>> > >>>> > >>>> > >>> > >>> > >> > >> > > > > > > > > >
