On 12/18/2013 08:46 AM, Ashish wrote:
I am with you for most discussion, but on two points, we differ a bit
- Easy In - Easy Out, would be a bit difficult to achieve. Easy in is easy,
but on what basis we move a component out. Someone not maintaining it or
someone not using it. Would be difficult for us to know. Probably there may
be easy way of doing this, BigTop deals with these cases more often than me.
There are multiple ways of doing it.
One way would be to kick it out if it is broken. Broken as in, build is
broken and/or tests not passing. Or if someone is introducing a sizable
change and that component would need a big update but no one willing to
do it.
I would not consider a component as being a target for being kicked out
if it is just about some bugs not being actively fixed. Maybe the
current version is good enough and not everyone hit such issues.
And if a component is broken, then someone would call it for removal on
the mailinglist and/or jira and notify the last few people who touched
that part. All of this with ample notice (1-2 weeks) so interested
parties can jump in, in case they are on vacation or not following the
mailing-lists every single day for instance.
Overall, I would rely more on best judgment rather than process.
In any case I assume the process would have to be adjusted.
- Committers are facilitators agree, but we do need to review the
contributions, else the whole process would break.
Reviewing patches is still somewhat blurry. Some people will vaguely
look at the code, while some others will do a complete and thorough
testing themselves.
So depending on the community wishes and the patch being reviewed, the
following could happen:
* Commiter could raise the quality bar. Since no one is familiar with
this area, it is fair to ask for a higher quality.
For instance insisting on extensive unit tests or even going beyond unit
tests by providing integration tests running against the real thing. As
an example, new projects being added in Apache Bigtop need to provide
deployment recipes and integration tests.
With regards to Apache Flume, Apache Bigtop could probably help with
integration tests since it already has everything needed to deploy
clusters and run feature tests. So integration tests could probably be
contributed to Apache Bigtop which would run them for the Apache community.
* Commiter looks at the code before comiting it if everything looks
fine. Remaining issues can be resolved later on.
From the discussion it seems that nobody seems to be inclined towards
contrib. How about giving components place under sources/sinks/interceptors
modules which are already present. Most of the times, only these would be
the things that would come in. These components maintain compatibility with
Flume's current version, get released alongside. Extra burden for core
Dev's for sure, but would save a lot of trouble for Users. Final call will
still be with Flume Dev's
@Bruno - Buddy Please do not stop work on Redis component. This is the last
thing we want. I am not a Committer here, but definitely willing to pitch
in for review and testing it a bit (would learn redis as well :) )
cheers
ashish
I haven't stopped working on the Redis component :)
We are happily using Flume along with the Redis component to push a few
billions events every day (and still ramping up) in elasticsearch,
Since I have to get stuff done, I cannot wait for the community to
decide if source/sinks are ok or not.
So I already converted and using my redis component into a Flume plugin,
and am using that.
While not rocket science, re-preparing a patch for Apache Flume based on
my latest updates will still take some time. So I am waiting for the
conclusion of this discussion before spending time on it.
Either way the code should be open sourced, whether it is part of Apache
Flume or on Github.
In the mean time I am working on pushing what I have in Github and will
do so as soon as I can.
Thanks,
Bruno
