If anybody wants to experiment a little with a feature idea -- absolutely, individual forked repositories are certainly an awesome place for such attempts.
However, for something that is a significant undertaking, like a new runner or new SDK, I think feature branches in the main repository make total sense. We'd avoid important disadvantages of lower visibility, harder for others to jump in, comment, learn, etc., harder testing because Apache Jenkins wouldn't be able to do it automatically, etc. In summary, I think there's a spectrum of feature complexities and longevity considerations. As such, I'd support being flexible as appropriate, but have a default answer of starting with a feature branch in the main repository for new major components. On Thu, May 19, 2016 at 3:09 AM, Ismaël Mejía <[email protected]> wrote: > I agree with Aljoscha, about not putting the feature branches in the main > repo, however how can we make people aware of the new developments ? > > -Ismaël > > On Thu, May 19, 2016 at 11:56 AM, Aljoscha Krettek <[email protected]> > wrote: > > > +1 > > > > When we say feature branch, are we talking about a branch in the main > repo? > > I would propose that feature branches live in the repos of the committers > > who are working on a feature. > > > > On Thu, 19 May 2016 at 11:54 Jean-Baptiste Onofré <[email protected]> > wrote: > > > > > +1 > > > > > > it looks good to me. > > > > > > Regards > > > JB > > > > > > On 05/19/2016 07:01 AM, Frances Perry wrote: > > > > Hi Beamers -- > > > > > > > > I’m thrilled by the recent energy and activity on writing new Beam > > > runners! > > > > But that also means it’s probably time for us to figure out how, as a > > > > community, we want to support this process. ;-) > > > > > > > > Back near the beginning, we had a thread [1] discussing that feature > > > > branches are the preferred way of doing development of features or > > > > components that may take a while to reach maturity. I think new > > > components > > > > like runners and SDKs meet the bar to be started from a feature > branch. > > > > (Other features, like an IO connector or library of PTransforms, > might > > > also > > > > qualify depending on complexity.) > > > > > > > > We should also lay out what it takes to be considered mature enough > to > > be > > > > merged into master, since once that happens the component gets > released > > > to > > > > users and failing tests become blocking issues. Here are some initial > > > > thoughts to kick off the discussion... > > > > > > > > In order to be merged into master, new components / major features > > > should: > > > > > > > > - > > > > > > > > have at least 2 contributors interested in maintaining it, and 1 > > > > committer interested in supporting it > > > > - > > > > > > > > provide both end-user and developer-facing documentation > > > > - > > > > > > > > have at least a basic level of unit test coverage > > > > - > > > > > > > > run all existing applicable integration tests with other Beam > > > components > > > > and create additional tests as appropriate > > > > > > > > > > > > In addition... > > > > > > > > A runner should: > > > > > > > > - > > > > > > > > be able to handle a subset of the model that address a > significant > > > set > > > > of use cases (aka. ‘traditional batch’ or ‘processing time > > > streaming’) > > > > - > > > > > > > > update the capability matrix with the current status > > > > > > > > > > > > An SDK* should: > > > > > > > > - > > > > > > > > provide the ability to construct graphs with all the basic > building > > > > blocks of the model (ParDo, GroupByKey, Window, Trigger, etc) > > > > - > > > > > > > > begin fleshing out the common composite transforms (Count, Join, > > etc) > > > > and IO connectors (Text, Kafka, etc) > > > > - > > > > > > > > have at least one runner that can execute the complete model (may > > be > > > a > > > > direct runner) > > > > - > > > > > > > > provide integration tests for executing against current and > future > > > > runners > > > > > > > > > > > > * A note on DSLs: I think it’s important to separate out an SDK > from a > > > > DSL, because in my mind the former is by definition equivalent to the > > > Beam > > > > model, while the latter may select portions of the model or change > the > > > > user-visible abstractions in order to provide a domain-specific > > > experience. > > > > We may want to encourage some DSLs to live separately from Beam > because > > > > they may look completely non-Beam-like to their end users. But we can > > > > probably punt this decision until we have concrete examples to > discuss. > > > > > > > > Another fun part of this growth is that we’ll likely grow new > > committers. > > > > And given the breadth of Beam, I think it would be useful to annotate > > our > > > > committers [2] page with which components folks are the most > > > knowledgeable > > > > about. > > > > > > > > Looking forward to your thoughts. > > > > > > > > [1] > > > > > > > > > > http://mail-archives.apache.org/mod_mbox/incubator-beam-dev/201602.mbox/%3CCAAzyFAymVNpjQgZdz2BoMknnE3H9rYRbdnUemamt9Pavw8ugsw%40mail.gmail.com%3E > > > > > > > > [2] http://beam.incubator.apache.org/team/ > > > > > > > > > > -- > > > Jean-Baptiste Onofré > > > [email protected] > > > http://blog.nanthrax.net > > > Talend - http://www.talend.com > > > > > >
