Fully agree with Davor for feature idea impl.
Regards
JB
On 05/19/2016 08:59 PM, Davor Bonaci wrote:
If anybody wants to experiment a little with a feature idea -- absolutely,
individual forked repositories are certainly an awesome place for such
attempts.
However, for something that is a significant undertaking, like a new runner
or new SDK, I think feature branches in the main repository make total
sense. We'd avoid important disadvantages of lower visibility, harder for
others to jump in, comment, learn, etc., harder testing because Apache
Jenkins wouldn't be able to do it automatically, etc.
In summary, I think there's a spectrum of feature complexities and
longevity considerations. As such, I'd support being flexible as
appropriate, but have a default answer of starting with a feature branch in
the main repository for new major components.
On Thu, May 19, 2016 at 3:09 AM, Ismaël Mejía <[email protected]> wrote:
I agree with Aljoscha, about not putting the feature branches in the main
repo, however how can we make people aware of the new developments ?
-Ismaël
On Thu, May 19, 2016 at 11:56 AM, Aljoscha Krettek <[email protected]>
wrote:
+1
When we say feature branch, are we talking about a branch in the main
repo?
I would propose that feature branches live in the repos of the committers
who are working on a feature.
On Thu, 19 May 2016 at 11:54 Jean-Baptiste Onofré <[email protected]>
wrote:
+1
it looks good to me.
Regards
JB
On 05/19/2016 07:01 AM, Frances Perry wrote:
Hi Beamers --
I’m thrilled by the recent energy and activity on writing new Beam
runners!
But that also means it’s probably time for us to figure out how, as a
community, we want to support this process. ;-)
Back near the beginning, we had a thread [1] discussing that feature
branches are the preferred way of doing development of features or
components that may take a while to reach maturity. I think new
components
like runners and SDKs meet the bar to be started from a feature
branch.
(Other features, like an IO connector or library of PTransforms,
might
also
qualify depending on complexity.)
We should also lay out what it takes to be considered mature enough
to
be
merged into master, since once that happens the component gets
released
to
users and failing tests become blocking issues. Here are some initial
thoughts to kick off the discussion...
In order to be merged into master, new components / major features
should:
-
have at least 2 contributors interested in maintaining it, and 1
committer interested in supporting it
-
provide both end-user and developer-facing documentation
-
have at least a basic level of unit test coverage
-
run all existing applicable integration tests with other Beam
components
and create additional tests as appropriate
In addition...
A runner should:
-
be able to handle a subset of the model that address a
significant
set
of use cases (aka. ‘traditional batch’ or ‘processing time
streaming’)
-
update the capability matrix with the current status
An SDK* should:
-
provide the ability to construct graphs with all the basic
building
blocks of the model (ParDo, GroupByKey, Window, Trigger, etc)
-
begin fleshing out the common composite transforms (Count, Join,
etc)
and IO connectors (Text, Kafka, etc)
-
have at least one runner that can execute the complete model (may
be
a
direct runner)
-
provide integration tests for executing against current and
future
runners
* A note on DSLs: I think it’s important to separate out an SDK
from a
DSL, because in my mind the former is by definition equivalent to the
Beam
model, while the latter may select portions of the model or change
the
user-visible abstractions in order to provide a domain-specific
experience.
We may want to encourage some DSLs to live separately from Beam
because
they may look completely non-Beam-like to their end users. But we can
probably punt this decision until we have concrete examples to
discuss.
Another fun part of this growth is that we’ll likely grow new
committers.
And given the breadth of Beam, I think it would be useful to annotate
our
committers [2] page with which components folks are the most
knowledgeable
about.
Looking forward to your thoughts.
[1]
http://mail-archives.apache.org/mod_mbox/incubator-beam-dev/201602.mbox/%3CCAAzyFAymVNpjQgZdz2BoMknnE3H9rYRbdnUemamt9Pavw8ugsw%40mail.gmail.com%3E
[2] http://beam.incubator.apache.org/team/
--
Jean-Baptiste Onofré
[email protected]
http://blog.nanthrax.net
Talend - http://www.talend.com
--
Jean-Baptiste Onofré
[email protected]
http://blog.nanthrax.net
Talend - http://www.talend.com
