@Israel - IMHO JIRA is not a good use-case for these discussion. The discussion can easily be tracked on flume.markmail.com and the link is provided
On Tue, Dec 17, 2013 at 7:19 AM, Mike Percy <[email protected]> wrote: > > I have created a JIRA Brainstorming task to track this. > > Hmm, I think there is a risk of losing this discussion in the flood of JIRA > traffic due to email filters. So I'm going to respond here on this thread. > If you want to reference this thread in the future, you can use this URL: > http://markmail.org/message/7x7tewbxqw4ubjyp > > My 2 cents below. > > > Do contrib components get released with Flume? Can they break > compatibility with older versions (what does this mean if they are not > getting released?) etc. How supported are these? > > I think you hit the nail on the head here. > > Recently I commented on the Storm sink indicating I would not help maintain > it therefore I would not help review it. Implicitly, I meant that someone > else should probably take responsibility for maintaining it to some extent. > So I think having at least one person who hopefully has the bandwidth to > understand and maintain a module should be a prerequisite for inclusion > into the main line of Flume (i.e. the bits included in a typical Flume > release). > Mike - You have echoed my thoughts. We have the same situation in MINA (and in Netty as well). People write a lot of codecs, but we can't get all of them into the release. A Codec would be brought in only if one of the Committers volunteers to maintain it. This was my thought for Flume as well, if one of the committer volunteers to maintain it, bring it in, else we have to find a way to keep things easy for Users. How, not sure at the moment. > > Generally, my concern with creating a contrib module is that, in my view, > it is where code goes to die, since why have a contrib module unless we > have no intention of maintaining that code? As an example, see the various > states of stability of code in Pig's piggybank. Is there good stuff in > there? Absolutely. Is everything in there relatively stable / usable, even > on a release? Nope. Those things may or may not work at any time. > Personally I'm not sure adding a contrib is a very good idea. > So let's assume, we have contrib, and if it gets release with Flume core, we don't have a choice but to atleast ensure it's compiling and tests are running. Sooner or later, one of the core jar upgrade will force us to fix the code in contrib. People can contribute patches, but this would still consume Dev cycles. If contrib is source only release or not compatible with current release, it better not be part of released. This will add a lot of confusion for users. Zookeeper does have contrib, not sure how they feel about it Still, I am worried about impact it might have on future contributions > > As another dimension to this discussion, I think there is a limit to the > number of dependencies Flume can reasonably pull in and keep straight > without shading or classloading tricks, which themselves add another layer > of pain/difficulty to debugging. > > > https://cwiki.apache.org/confluence/display/FLUME/Flume+NG+Plugins > > I think this page has a lot of value but maybe the problem with it is that > it is not accessible from the home page of http://flume.apache.org/ > > TL;DR: > 1. In my view we should only accept new, big components in the main project > if they have a good chance of continuing to be maintained. > +1, if one of the Committers volunteers to maintain it > 2. The more dependencies Flume has, the harder it is to upgrade any one > component and keep all the dependencies from conflicting (i.e. JAR hell). > We are almost there :) > 3. Is there an alternative to contrib that would solve the problem of > giving new plugins a home if they cannot be included in the main project? > Is there a way we could make plugins that live on e.g. GitHub more visible? > 4. If we add a contrib module, I agree that we need to be clear on why > contrib exists and what it means to have something live in there. > +1 > > Mike > Summarizing my suggestions 1. For a contribution to make it into Flume, one of committers had to step in 2. Not very much convinced with contrib, but don't have solution otherwise as well > > > On Mon, Dec 16, 2013 at 12:17 PM, Israel Ekpo <[email protected]> > wrote: > > > I have created a JIRA Brainstorming task to track this. > > > > I think it will be very helpful for us to collect our thoughts on the > JIRA > > ticket for future references. > > > > I think this is a very important topic. > > > > https://issues.apache.org/jira/browse/FLUME-2274 > > > > > > > > > > On Mon, Dec 16, 2013 at 12:51 PM, Ashish <[email protected]> > wrote: > > > > > Quick question before I put down my thoughts. What does "core Flume" > > > represents? Is it the current Flume trunk structure or just few > modules? > > > > > > > > > On Sat, Dec 14, 2013 at 5:51 AM, Hari Shreedharan < > > > [email protected] > > > > wrote: > > > > > > > Hi all > > > > > > > > Over the past few weeks, there were some discussions about including > > > > additional features in core flume itself. This led to a discussion > > about > > > > adding a contrib module. I thought I’d start an official discussion > > > > regarding this. The argument for contrib module was that there are > too > > > many > > > > components without generic use which are getting submitted/committed. > > The > > > > use-cases for such components are limited, and hence they should not > > be > > > > part of core flume itself. First, we should answer the question if we > > > want > > > > to separate components into a contrib module and why? What components > > go > > > > into contrib and what into core flume? What does it mean to be make a > > > > component part of the contrib module. Do contrib components get > > released > > > > with Flume? Can they break compatibility with older versions (what > does > > > > this mean if they are not getting released?) etc. How supported are > > > these? > > > > > > > > Please respond and let us know your view! > > > > > > > > > > > > Thanks, > > > > Hari > > > > > > > > > > > > > > > > > -- > > > thanks > > > ashish > > > > > > Blog: http://www.ashishpaliwal.com/blog > > > My Photo Galleries: http://www.pbase.com/ashishpaliwal > > > > > > > > > > > -- > > °O° > > "Good Enough" is not good enough. > > To give anything less than your best is to sacrifice the gift. > > Quality First. Measure Twice. Cut Once. > > http://www.israelekpo.com/ > > > -- thanks ashish Blog: http://www.ashishpaliwal.com/blog My Photo Galleries: http://www.pbase.com/ashishpaliwal
