Thanks Devor for detailed explanation of the thought process. That certainly helps.
I do favor feature branches as they are must of major work. The proposed development/feature branch for the time being is perfectly fine. Just a caution that keeping the master updated could be taxing. We can certainly revisit this later once the project is stable in Incubation. re: timing of major releases Beam being new and rapidly evolving, makes sense to favor growth right now. Regards Sandeep On Fri, Feb 19, 2016 at 1:42 AM, Davor Bonaci <[email protected]> wrote: > Thanks Sandeep for your insightful comments! I'll add a little more details > to follow up on Frances' email to clarify what we were proposing. > > We were indeed proposing the concept of the development / feature branches > (for the time being). I think the goal of having the master branch in a > "working state" is a clear, non-contentions goal that equally benefits all > users and all developers. "Working state" doesn't necessarily mean > production-ready, but means that "most" things work out of the box. Not > having this goal would be hugely taxing for everyone. > > Now, this goal can be achieved either via: > > - direct commits to master with a proper presubmit coverage, > - direct commits to a separate branch(es) with regular integration to > the master, or > - a combination of these approaches on a per-pull request basis. > > I think everyone universally prefers the first choice. Google committers > certainly do. > > However, we start this project from a point of zero presubmit coverage, and > expectation of a ton of refactoring and breaking changes. We felt letting > all these changes into master right away without proper coverage would make > everyone's life a little sad. Here's a very simple example -- the vision we > proposed involves completely changing the runner-facing APIs. Every commit > of this work will break parts of our project -- such works needs to be > separated, otherwise *everyone* gets blocked. > > [ Additionally, Beam is a somewhat unique project -- it makes sense with > services, such as Dataflow Service, Flink or Spark, that are maintained as > separate projects -- so, we need a decent cross-project integration > coverage. ] > > With this in mind, our proposal is keep the development / feature branches > *for the time being* -- until we replicate a portion of the presubmit > coverage we have internally at Google. This is the "default" place for > executing against the vision, cleaning runner API, etc. As Frances said, > there are classes of commits, such as a new IO connector or a new > domain-specific PTransform, where we can simplify this and go directly to > master. In any case, master should never be behind the development branch > by more than a week or so. > > -- > > re: timing of major releases > This is often a contentions topic. The other side of the argument is > two-fold: > > - Releasing rarely creates a bigger cliff. Hence, it becomes a higher > burden and less likely to happen. > - Releasing rarely optimizes experience for existing users, not new > users. If we are growing quickly, that may not be the best metric to > optimize for. > > In reality, we are likely to release more often as we mature. Later on, we > are likely to release less often. > > I think decisions in this space should be delayed until the time we need to > make a choice and consider the payload we want to release. That said, > having some timeline in mind is often beneficial to make right design > tradeoffs. > > -- > > re: @evolving idea > We are already doing this and planning to continue -- this works really > well. See @Experimental here > < > https://github.com/GoogleCloudPlatform/DataflowJavaSDK/blob/a8347d186700e83edb91a3c673200abec1b1d4e7/sdk/src/main/java/com/google/cloud/dataflow/sdk/annotations/Experimental.java > > > . > > On Thu, Feb 18, 2016 at 10:17 AM, Robert Bradshaw < > [email protected]> wrote: > > > Yes, (b) would be a so-called feature branch. (The proposal here is to > > discard the idea of having a separate long-lived "develop" branch.) > > > > On Thu, Feb 18, 2016 at 10:08 AM, Frances Perry <[email protected]> > > wrote: > > > > > Our developers are going to be a varied group -- so "main development" > > will > > > look quite different to different developers. In particular, look at: > > > (a) a developer writing a java sdk extension for a new IO connector > > > (b) a developer changing the beam model > > > > > > I think it's fine for work like (a) to occur on master, but I think > > things > > > like (b) should happen on a development branch so that we can keep the > > > master branch in a working state. There are going to be a number of > > large, > > > backwards incompatible, churn-y changes to Runner APIs in the near > > future. > > > I'd like us to be able to do those in a way that doesn't affect folks > who > > > are attempting more surface level contributions. > > > > > > Frances > > > > > > On Thu, Feb 18, 2016 at 8:07 AM, Robert Bradshaw < > > > [email protected]> wrote: > > > > > > > +1 to using master for main development (and most non-ASF projects > use > > > > master like this too). Not having master (the default when one > clones, > > > > etc.) be at HEAD is often surprising. Tags are easy enough to use > when > > > one > > > > wants a stable version. > > > > > > > > - Robert > > > > > > > > > > > > On Wed, Feb 17, 2016 at 11:38 PM, Jean-Baptiste Onofré < > > [email protected]> > > > > wrote: > > > > > > > > > Thanks Henry, I remember now, and Frances posted the link. > > > > > > > > > > I agree: we should use the master branch as dev branch as all other > > ASF > > > > > projects do. > > > > > > > > > > Regards > > > > > JB > > > > > > > > > > > > > > > On 02/18/2016 08:04 AM, Henry Saputra wrote: > > > > > > > > > >> Actually no, it is a bit different. > > > > >> The concept of develop branch is following the "successful git > > > branching > > > > >> model" blog post [1] that introduce using develop branch as active > > > > branch > > > > >> for development and use master as stable branch. > > > > >> > > > > >> I would recommend using master branch instead as default branch to > > do > > > > >> active development to match other ASF projects. > > > > >> > > > > >> Some projects using develop from origin company, like Twill [2], > had > > > > also > > > > >> moved to using master as default active branch. > > > > >> > > > > >> Just my 2 cents. > > > > >> > > > > >> Thx. > > > > >> > > > > >> Henry > > > > >> > > > > >> > > > > >> [1] http://nvie.com/posts/a-successful-git-branching-model/ > > > > >> [2] http://twill.incubator.apache.org/HowToContribute.html > > > > >> > > > > >> On Wed, Feb 17, 2016 at 10:52 PM, Jean-Baptiste Onofré < > > > [email protected] > > > > > > > > > >> wrote: > > > > >> > > > > >> Hi, > > > > >>> > > > > >>> Correct me if I'm wrong, but I'm assuming that develop == master > > > (from > > > > a > > > > >>> git perspective). > > > > >>> > > > > >>> I configured Jenkins this way as it's the "regular" naming ;) > > > > >>> > > > > >>> I think Frances said "develop" from a dev perspective. All > projects > > > use > > > > >>> master (it's what I'm doing in Falcon, Lens, Karaf, Camel, etc, > > etc). > > > > >>> > > > > >>> Maybe I'm wrong ;) > > > > >>> > > > > >>> Regards > > > > >>> JB > > > > >>> > > > > >>> > > > > >>> On 02/18/2016 06:46 AM, Sandeep Deshmukh wrote: > > > > >>> > > > > >>> Hi All, > > > > >>>> > > > > >>>> I have some comments on the repository structure and most of > them > > > are > > > > >>>> wrt > > > > >>>> my experience in another Apache incubating project. > > > > >>>> > > > > >>>> > > > > >>>> 1. Most active projects use *master* as default development > > > > branch > > > > >>>> than > > > > >>>> *develop*. For example, Flink, Spark, Storm, Samza, Pig, > > Hive, > > > > and > > > > >>>> Hadoop use master branch. > > > > >>>> 2. Released artifacts are always hosted on downloads > > page.Maser > > > > >>>> need > > > > >>>> not > > > > >>>> be the one with production ready state. > > > > >>>> 3. It is quite intuitive to use *master* otherwise new > > > > contributors > > > > >>>> needs to go through documentation to understand process of > > each > > > > >>>> project. > > > > >>>> 4. Overall, the process becomes simple if *master* is the > > > default > > > > >>>> branch. > > > > >>>> > > > > >>>> > > > > >>>> Another suggestion is related to release with major version > > change. > > > > >>>> Major > > > > >>>> release twice a year is a lot of burden on the end user if they > > want > > > > to > > > > >>>> upgrade to a newer version. To address this issue, newly added > > APIs > > > > can > > > > >>>> be > > > > >>>> marked as @evolving so that users are aware of possible change > in > > > the > > > > >>>> upcoming release but the stable one should be carefully changed. > > > > >>>> > > > > >>>> Regards, > > > > >>>> Sandeep > > > > >>>> > > > > >>>> On Sat, Feb 13, 2016 at 2:34 AM, Frances Perry > > > <[email protected] > > > > > > > > > >>>> wrote: > > > > >>>> > > > > >>>> Thanks for all the feedback! Please keep it coming as needed. > > > > >>>> > > > > >>>>> > > > > >>>>> We've gone ahead and created components matching this > structure: > > > > >>>>> > > > > >>>>> > > > > >>>>> > > > > >>>>> > > > > > > > > > > https://issues.apache.org/jira/browse/BEAM/?selectedTab=com.atlassian.jira.jira-projects-plugin:components-panel > > > > >>>>> > > > > >>>>> We'll work on transition existing state from Google-internal > > tools > > > > into > > > > >>>>> this over the next few weeks. > > > > >>>>> > > > > >>>>> > > > > >>>>> On Fri, Feb 12, 2016 at 7:47 AM, Kenneth Knowles > > > > >>>>> <[email protected] > > > > >>>>> > > > > >>>>>> > > > > >>>>>> wrote: > > > > >>>>> > > > > >>>>> On Thu, Feb 11, 2016 at 8:53 AM, Maximilian Michels < > > > [email protected]> > > > > >>>>> > > > > >>>>>> wrote: > > > > >>>>>> > > > > >>>>>> As for the /develop branch, I would suggest to > > > > >>>>>> > > > > >>>>>>> make it mandatory to have it in a usable state at all times. > > > > >>>>>>> > > > > >>>>>>> > > > > >>>>>>> +1 > > > > >>>>>> > > > > >>>>>> If breakage is accidentally committed (as will happen) then a > > CTR > > > > >>>>>> > > > > >>>>>> rollback > > > > >>>>> > > > > >>>>> is a encouraged. > > > > >>>>>> > > > > >>>>>> Kenn > > > > >>>>>> > > > > >>>>>> > > > > >>>>>> > > > > >>>>> > > > > >>>> -- > > > > >>> Jean-Baptiste Onofré > > > > >>> [email protected] > > > > >>> http://blog.nanthrax.net > > > > >>> Talend - http://www.talend.com > > > > >>> > > > > >>> > > > > >> > > > > > -- > > > > > Jean-Baptiste Onofré > > > > > [email protected] > > > > > http://blog.nanthrax.net > > > > > Talend - http://www.talend.com > > > > > > > > > > > > > > >
