On 11/29/2013 08:17 AM, Otis Gospodnetic wrote:
If I understood this correctly, then I think you can also think of this
from the opposite direction. Instead of moving sources/sinks OUT of Flume,
let them develop outside Flume, and if they prove to be good, in high
demand, often used with Flume, or in need of "adoption" by core Flume
developers, then BRING IN such sources/sinks and, optionally(?), their
authors with them. Maybe this way you can have the best of both worlds.
If I develop source/sink for the new, fancy, FooBarBaz data store, can I
get it in Flume? No. and that's good, because if FooBarBaz doesn't take
off, who'll want to maintain it?
If I develop source/sink for Kafka, which is known to be very popular and
often used with Flume, and there are either core Flume developers
interested in maintaining this source/sink, or if there are external
developers who have been working on this outside Apache (i.e. the chances
are they'll follow the project to Apache and continue contributing to it,
eventually earning the committer rights), then this source/sink for Kafka
should be considered for adoption by Flume.
This way you let the external ecosystem live on, develop independently,
compete for user and developer adoption, and once good sources/sinks emerge
from there, they can be moved under Flume if everyone involved agrees
that's the right thing to do.
Otis
--
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/
If a component gets developed outside of Apache, what is the cost to
integrate it? How would that work? I assume one would have to go through
a code grant. If so, would a developer have to go track all past
contributors to agree on the code grant terms? Or should a CLA be
prepared in case of such event?
Why is it bad to get a new source/sink for the new, fancy, FooBrBaz data
store? What if it does take off? I would contend that the success of
that data store does not matter as long as there is a community
interested in it.
And why is it so costly to include it when it is so cheap to remove it
when no one cares about it anymore?
Including people from the outer layers also increases the chances of
getting them interested in the core of Apache Flume. They will most
likely notice something to improve somewhere else and start helping in
core parts. Even more so if they are going through all the steps to get
their code in Apache Flume.
Imho, the cost on passing on possible contributors is quite high
comparing to deleting unmaintained parts. Not just for the core Apache
Flume developers, but for the users in general as well.
Thanks,
Bruno
