One further piece of food for thought:
The Spark project has the following directory layout [1] in this regard:
examples/
external/
|
+-- flume
+-- kafka
+-- mqtt
+-- twitter
+-- zeromq
Note how 'kafka" is a connector to another OSS tool -- like
storm-kafka's spout -- where as 'twitter' is their implementation of
pulling data from Twitter's (streaming) API. Of course, the 'kafka'
code similarly connects to an API, but there's is still a small
difference between 'twitter' (hosted API run by Twitter) and 'kafka'
(your own Kafka infrastructure). Both sub-projects fit nicely under
'external' though IMHO.
As I said -- just further brainstorming.
Michael
[1] https://github.com/apache/incubator-spark
On 03/14/2014 08:06 PM, Nathan Marz wrote:
> How about we make a folder under root called "other" in which everything
> non-core can go. We can do further subfolders if we want called "examples"
> and "connectors" - I don't care either way. I think this will first of all
> make it clear these things are not part of the core project, and it will
> also prevent the root of the source from getting cluttered with too much
> stuff.
>
>
> On Thu, Mar 13, 2014 at 4:16 PM, Ted Dunning <[email protected]> wrote:
>
>> Taylor,
>>
>> You guys have been doing a generally excellent job. I was just chiming in
>> on the chance that there was doubt.
>>
>>
>> On Thu, Mar 13, 2014 at 4:09 PM, P. Taylor Goetz <[email protected]>
>> wrote:
>>
>>> Thanks Ted,
>>>
>>> We're being very careful when pulling in additional code by taking steps
>>> to preserve commit history (chain of evidence), and when necessary,
>>> initiate the IP clearance process (haven't had to yet).
>>>
>>
>> Cool.
>>
>>
>>> The latter is kind of a gray area as far as I can tell from questions
>> I've
>>> asked on general@. It seems to be a judgment call based on the size of
>>> the contribution.
>>>
>>
>> It is exactly that.
>>
>>
>>>
>>> If there's anything else we can do to make sure we get these things
>> right,
>>> or do a better job, please let us know.
>>>
>>
>> So far, things are going swimmingly, due in no small part to your efforts.
>>
>>
>>
>>>
>>> -Taylor
>>>
>>>> On Mar 13, 2014, at 4:03 PM, Ted Dunning <[email protected]>
>> wrote:
>>>>
>>>> Having a committer sign off on each addition has a very large role at
>>>> Apache. One of the key aspects of Apache software releases is that all
>>> of
>>>> the code is traceable back to the original contributor and there is a
>>>> logical chain that allows Apache to stand behind the licensing of the
>>> code.
>>>>
>>>> This licensing and chain of evidence is a big part of what makes open
>>>> source palatable to risk averse businesses. It is really important to
>>>> maintain.
>>>>
>>>> Storm has a very good record of doing this before being part of Apache
>>>> which makes integration into Apache processes easier, but it is
>> important
>>>> to hang on to that careful approach.
>>>>
>>>>
>>>>
>>>>
>>>>> On Thu, Mar 13, 2014 at 12:58 PM, P. Taylor Goetz <[email protected]>
>>> wrote:
>>>>>
>>>>> Exactly.
>>>>>
>>>>> That's why I proposed that anything that's brought in require at least
>>> on
>>>>> committer to "sponsor" it:
>>>
>>>
>>
>
>
>
