[GitHub] flume pull request #250: FLUME-3146 Use public API HdfsDataOutputStream#getC...
GitHub user majorendre opened a pull request: https://github.com/apache/flume/pull/250 FLUME-3146 Use public API HdfsDataOutputStream#getCurrentBlockReplica⦠â¦tion where applicable Took over this issue from Wei-Chiu Chuang. Added a few lines to the tests. All tests pass, the new feature works. You can merge this pull request into a Git repository by running: $ git pull https://github.com/majorendre/flume FLUME-3146 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flume/pull/250.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #250 commit c41c97dd7d8b4d92ba99e27d404dc2ddc1b3e7ee Author: Endre Major Date: 2018-11-27T10:14:06Z FLUME-3146 Use public API HdfsDataOutputStream#getCurrentBlockReplication where applicable ---
Re: [DISCUSS] Flume 1.9 release proposal
Again, please update the release notes to indicate the logging configuration change will be coming with the next release. Ralph > On Nov 28, 2018, at 1:17 AM, Denes Arvay wrote: > > Hi All, > > I agree that the current content doesn't justify the 2.0 release, so I'd > vote for moving forward with the 1.9 as Ferenc recommended, i.e. without > the 2 previously mentioned breaking changes. > I also agree with the proposed new features, especially with the plugin > system/modularization, which was discussed on the other thread: > https://s.apache.org/2nm9 > > So, Ferenc, I think you are good to go with the proposed plan, unless > anybody vetoes. > > Regards, > Denes > > On Tue, Nov 27, 2018 at 4:37 PM Ralph Goers > wrote: > >> While Log4j 2 supports using properties files for configuration it is not >> syntactically compatible with Log4j 1. However, migrating a Log4j 1 >> configuration to Log4j 2 isn’t a difficult task. >> >> Ralph >> >>> On Nov 27, 2018, at 4:38 AM, Tristan Stevens >> wrote: >>> >>> I did start thinking a while back about a REST API that could be used for >>> configuring Flume, after which you could potentially bolt on a UI. I >>> stopped work on it because it got tricky around configuring sources (and >>> the way in which they relate to channels etc). If someone had more time >> I’d >>> be happy to push to a repo the work that I did. This could be a really >>> useful 2.0 feature. >>> >>> Regarding move from Log4j1.x to 2, it’d be interesting to see what other >>> Apache projects, such as Hadoop, Hive etc did around this (in fact, are >>> they even at 2.x yet?). For me, if you need to change config, it’s not a >>> minor release, it become major, unless there’s a way you can >> automatically >>> migrate config from one to the other. I’ve not checked, but are we sure >>> that .properties file won’t work with log4j2? >>> >>> Tristan >>> >>> >>> >>> On 26 November 2018 at 19:54:01, Mike Percy (mpe...@apache.org) wrote: >>> >>> Yeah, I agree with this, especially classloader isolation would be great >> to >>> have as well on the plugin side. >>> >>> Mike >>> >>> On Mon, Nov 26, 2018 at 11:50 AM Ralph Goers >> >>> wrote: >>> I think people want to do more than just upgrade Log4j for a Flume 2.0 release. I would like to make configuration much more pluggable for example. There was also talk about splitting all the sources, sinks, interceptors, etc out of the core module and making them some sort of plugin. At this point those are just at the idea stage. > On Nov 26, 2018, at 12:19 PM, Bessenyei Balázs Donát < >> bes...@apache.org> >>> wrote: > > I wonder, is there anything blocking us from releasing 2.0 next instead of 1.x? > > > Donat > > On Mon, Nov 26, 2018 at 6:27 PM Mike Percy wrote: >> >> If only the PMC would be affected by this decision, we could have had this >> discussion on the PMC private list. But this decision impacts >>> everybody >> that uses Flume. So let's hear from anybody who cares about this, including >> committers, contributors, and users on whether they are okay with switching >> to log4j2 in a minor release version, knowing that they will need to change >> their config files when they upgrade Flume. >> >> Ferenc, it seems like we will have to ship one or the other with the binary >> artifacts at release time. It seems to me that we still have to make a >> choice about the built and shipped default, even if both could work at >> runtime, right? >> >> Thanks, >> Mike >> >> On Mon, Nov 26, 2018 at 5:58 AM Ferenc Szabo >> wrote: >> >>> I have directed it to the community, the PMC members. >>> I am looking for a decision from the PMC members (including You) on whether >>> we should continue as planned or not removing log4j2 form the minor release >>> branch. >>> >>> On Mon, Nov 26, 2018 at 2:43 PM Ralph Goers < ralph.go...@dslextreme.com> >>> wrote: >>> Was this directed at me? I am not sure what decision you are >>> referring >>> to. I stated my position in the email you replied to. Ralph > On Nov 26, 2018, at 5:29 AM, Ferenc Szabo >>> wrote: > > With the new release, you can just replace the jars on the >>> classpath as we > removed every code dependency and using slf4j. So you do not need >>> to >>> fork > and change anything. > > Let me know your decision and we will continue with that. > > Regards, > Ferenc > > On Mon, Nov 26, 2018 at 12:10 PM Apache < >> ralph.go...@dslextreme.com> >>> wrote: > >> As I said, I am fine with adding a notice to the release notes stating the >> logging
[GitHub] flume pull request #249: FLUME-3299 Fix log4j scopes in pom files
Github user asfgit closed the pull request at: https://github.com/apache/flume/pull/249 ---
Re: [DISCUSS] Flume 1.9 release proposal
Hi All, I agree that the current content doesn't justify the 2.0 release, so I'd vote for moving forward with the 1.9 as Ferenc recommended, i.e. without the 2 previously mentioned breaking changes. I also agree with the proposed new features, especially with the plugin system/modularization, which was discussed on the other thread: https://s.apache.org/2nm9 So, Ferenc, I think you are good to go with the proposed plan, unless anybody vetoes. Regards, Denes On Tue, Nov 27, 2018 at 4:37 PM Ralph Goers wrote: > While Log4j 2 supports using properties files for configuration it is not > syntactically compatible with Log4j 1. However, migrating a Log4j 1 > configuration to Log4j 2 isn’t a difficult task. > > Ralph > > > On Nov 27, 2018, at 4:38 AM, Tristan Stevens > wrote: > > > > I did start thinking a while back about a REST API that could be used for > > configuring Flume, after which you could potentially bolt on a UI. I > > stopped work on it because it got tricky around configuring sources (and > > the way in which they relate to channels etc). If someone had more time > I’d > > be happy to push to a repo the work that I did. This could be a really > > useful 2.0 feature. > > > > Regarding move from Log4j1.x to 2, it’d be interesting to see what other > > Apache projects, such as Hadoop, Hive etc did around this (in fact, are > > they even at 2.x yet?). For me, if you need to change config, it’s not a > > minor release, it become major, unless there’s a way you can > automatically > > migrate config from one to the other. I’ve not checked, but are we sure > > that .properties file won’t work with log4j2? > > > > Tristan > > > > > > > > On 26 November 2018 at 19:54:01, Mike Percy (mpe...@apache.org) wrote: > > > > Yeah, I agree with this, especially classloader isolation would be great > to > > have as well on the plugin side. > > > > Mike > > > > On Mon, Nov 26, 2018 at 11:50 AM Ralph Goers > > > wrote: > > > >> I think people want to do more than just upgrade Log4j for a Flume 2.0 > >> release. I would like to make configuration much more pluggable for > >> example. There was also talk about splitting all the sources, sinks, > >> interceptors, etc out of the core module and making them some sort of > >> plugin. At this point those are just at the idea stage. > >> > >>> On Nov 26, 2018, at 12:19 PM, Bessenyei Balázs Donát < > bes...@apache.org> > > > >> wrote: > >>> > >>> I wonder, is there anything blocking us from releasing 2.0 next instead > >> of 1.x? > >>> > >>> > >>> Donat > >>> > >>> On Mon, Nov 26, 2018 at 6:27 PM Mike Percy wrote: > > If only the PMC would be affected by this decision, we could have had > >> this > discussion on the PMC private list. But this decision impacts > > everybody > that uses Flume. So let's hear from anybody who cares about this, > >> including > committers, contributors, and users on whether they are okay with > >> switching > to log4j2 in a minor release version, knowing that they will need to > >> change > their config files when they upgrade Flume. > > Ferenc, it seems like we will have to ship one or the other with the > >> binary > artifacts at release time. It seems to me that we still have to make a > choice about the built and shipped default, even if both could work at > runtime, right? > > Thanks, > Mike > > On Mon, Nov 26, 2018 at 5:58 AM Ferenc Szabo > >> > wrote: > > > I have directed it to the community, the PMC members. > > I am looking for a decision from the PMC members (including You) on > >> whether > > we should continue as planned or not removing log4j2 form the minor > >> release > > branch. > > > > On Mon, Nov 26, 2018 at 2:43 PM Ralph Goers < > >> ralph.go...@dslextreme.com> > > wrote: > > > >> Was this directed at me? I am not sure what decision you are > > referring > > to. > >> I stated my position in the email you replied to. > >> > >> Ralph > >> > >>> On Nov 26, 2018, at 5:29 AM, Ferenc Szabo > >> >> > >> wrote: > >>> > >>> With the new release, you can just replace the jars on the > > classpath > >> as > >> we > >>> removed every code dependency and using slf4j. So you do not need > > to > > fork > >>> and change anything. > >>> > >>> Let me know your decision and we will continue with that. > >>> > >>> Regards, > >>> Ferenc > >>> > >>> On Mon, Nov 26, 2018 at 12:10 PM Apache < > ralph.go...@dslextreme.com> > > > >> wrote: > >>> > As I said, I am fine with adding a notice to the release notes > >> stating > >> the > logging will be changed in the next release regardless of its > >> version > number. This way the revert only needs to happen on the release > >> branch > >> for > this release. > > Ralph > > > On Nov 25, 2018, at 9:59
