Hi,
I meant in the hypothetical case that we did this. There is going to be an
interim period where we wouldn't have this. The automation comes at the
expense of something else.
Ariel
On Thu, May 7, 2015 at 11:40 AM, Josh McKenzie <josh.mcken...@datastax.com>
wrote:
> >
> > So who and when is going to implement the automation?
>
>
> I don't believe we have sufficient consensus that this is necessary to
> start doling out action-items for implementation.
>
> On Thu, May 7, 2015 at 10:16 AM, Ariel Weisberg <
> ariel.weisb...@datastax.com
> > wrote:
>
> > Hi,
> >
> > If it were automated I would have no problem with it. That would be less
> > work for me because the problems detected would occur anyways and have to
> > be dealt with by me. I just don't want to deal with extra steps and
> latency
> > manually.
> >
> > So who and when is going to implement the automation?
> >
> > Ariel
> >
> > On Thu, May 7, 2015 at 11:11 AM, Benedict Elliott Smith <
> > belliottsm...@datastax.com> wrote:
> >
> > > It's odd, because I honestly think this release process will be easier,
> > > since the stricter we make it the smoother it can become. It requires
> > well
> > > formed commits from everyone, and lets the committers asynchronously
> > > confirm their work, and for it to never be in question *who* needs to
> fix
> > > something, nor what the effect of their fixing it will be. It means we
> > can,
> > > as Ariel said, perform a bisect and honestly know its result is
> accurate.
> > > Small commits don't need to worry about fast-forwarding; in fact,
> nobody
> > > does. It can either be automated, or we can fast forward at a time that
> > > suits us. In which case the process is *the same* as it is currently.
> > >
> > > I have no interest in making the commit process harder.
> > >
> > >
> > > On Thu, May 7, 2015 at 3:59 PM, Jake Luciani <jak...@gmail.com> wrote:
> > >
> > > > Ok let's focus then on the idea that trunk is releasable. Releasable
> > > > to me doesn't mean it can't contain a bad merge.
> > > >
> > > > It means it doesn't contain some untested and unstable feature. We
> > > > can always "release from trunk" and we still have a release process.
> > > >
> > > > The idea that trunk must contain. a first time it hits the branch,
> > > > releasable code is way overboard
> > > >
> > > > On Thu, May 7, 2015 at 10:50 AM, Benedict Elliott Smith
> > > > <belliottsm...@datastax.com> wrote:
> > > > >>
> > > > >> This breaks your model of applying every commit ref by ref.
> > > > >
> > > > >
> > > > > How? The rebase only affects commits after the "real" branch, so it
> > > still
> > > > > cleanly fast forwards?
> > > > >
> > > > > Merging is *hard*. Especially 2.1 -> 3.0, with many breaking API
> > > changes
> > > > > (this is before 8099, which is going to make a *world* of hurt, and
> > > will
> > > > > stick around for a year). It is *very* easy to break things, with
> > even
> > > > the
> > > > > utmost care.
> > > > >
> > > > > On Thu, May 7, 2015 at 3:46 PM, Jake Luciani <jak...@gmail.com>
> > wrote:
> > > > >
> > > > >> You then fetch and repair
> > > > >> your local version and try again.
> > > > >>
> > > > >> This breaks your model of applying every commit ref by ref.
> > > > >>
> > > > >> I'm all for trying to avoid extra work/stability but we already
> have
> > > > >> added a layer of testing every change before commit. I'm not
> going
> > to
> > > > >> accept we need to also add a layer of testing before every merge.
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >> On Thu, May 7, 2015 at 10:36 AM, Benedict Elliott Smith
> > > > >> <belliottsm...@datastax.com> wrote:
> > > > >> >>
> > > > >> >> wouldn't you need to force push?
> > > > >> >
> > > > >> >
> > > > >> > git push --force-with-lease
> > > > >> >
> > > > >> > This works essentially like CAS; if the remote repositories are
> > not
> > > > the
> > > > >> > same as the one you have modified, it will fail. You then fetch
> > and
> > > > >> repair
> > > > >> > your local version and try again.
> > > > >> >
> > > > >> > So what does this buy us?
> > > > >> >
> > > > >> >
> > > > >> > This buys us a clean development process. We bought into "always
> > > > >> > releasable". It's already a tall order; if we start weakening
> the
> > > > >> > constraints before we even get started, I am unconvinced we will
> > > > >> > successfully deliver. A monthly release cycle requires *strict*
> > > > >> processes,
> > > > >> > not *almost* strict, or strict*ish*.
> > > > >> >
> > > > >> > Something that could also help make a more streamlined process:
> if
> > > > actual
> > > > >> > commits were constructed on development branches ready for
> commit,
> > > > with a
> > > > >> > proper commit message and CHANGES.txt updated. Even more
> ideally:
> > > with
> > > > >> git
> > > > >> > rerere data for merging up to each of the branches. If we had
> > that,
> > > > and
> > > > >> > each of the branches had been tested in CI, we would be much
> > closer
> > > > than
> > > > >> we
> > > > >> > are currently, as the risk-at-commit is minimized.
> > > > >> >
> > > > >> > On Thu, May 7, 2015 at 2:48 PM, Jake Luciani <jak...@gmail.com>
> > > > wrote:
> > > > >> >
> > > > >> >> git rebase -i trunk_staging <ref~1>
> > > > >> >> fix the problem
> > > > >> >> git rebase --continue
> > > > >> >>
> > > > >> >> In this situation, if there was an untested follow on commit
> > > wouldn't
> > > > >> >> you need to force push?
> > > > >> >>
> > > > >> >> On Thu, May 7, 2015 at 9:28 AM, Benedict Elliott Smith
> > > > >> >> <belliottsm...@datastax.com> wrote:
> > > > >> >> >>
> > > > >> >> >> If we do it, we'll end up in weird situations which will be
> > > > annoying
> > > > >> for
> > > > >> >> >> everyone
> > > > >> >> >
> > > > >> >> >
> > > > >> >> > Such as? I'm not disputing, but if we're to assess the
> relative
> > > > >> >> > strengths/weaknesses, we need to have specifics to discuss.
> > > > >> >> >
> > > > >> >> > If we do go with this suggestion, we will most likely want to
> > > > enable a
> > > > >> >> > shared git rerere cache, so that rebasing is not painful when
> > > there
> > > > >> are
> > > > >> >> > future commits.
> > > > >> >> >
> > > > >> >> > If instead we go with "repairing" commits, we cannot have a
> > > > "queue" of
> > > > >> >> > things to merge up to. Say you have a string of commits
> waiting
> > > for
> > > > >> >> > approval C1 to C4; you made C1, and it broke something. You
> > > > introduce
> > > > >> C5
> > > > >> >> to
> > > > >> >> > fix it, but the tests are still broken. Did you not really
> fix
> > > it?
> > > > Or
> > > > >> >> > perhaps one of C2 to C4 are to blame, but which? And have you
> > > > >> >> accidentally
> > > > >> >> > broken *them* with your commit? Who knows. Either way, we
> > > > definitely
> > > > >> >> cannot
> > > > >> >> > fast forward. At the very best we can hope that the new merge
> > did
> > > > not
> > > > >> >> > conflict or mess up the other people's C2 to C4 commits, and
> > they
> > > > >> have to
> > > > >> >> > now merge on top. But what if another merge comes in, C6, in
> > the
> > > > >> >> meantime;
> > > > >> >> > and C2 really did also break the tests in some way; how do we
> > > > >> determine
> > > > >> >> C2
> > > > >> >> > was to blame, and not C6, or C3 or C4? What do the committers
> > for
> > > > >> each of
> > > > >> >> > these do? We end up in a lengthy tussle, and aren't able to
> > > commit
> > > > >> any of
> > > > >> >> > these to the mainline until all of them are resolved. Really
> we
> > > > have
> > > > >> to
> > > > >> >> > prevent any merges to the staging repository until the
> mistakes
> > > are
> > > > >> >> fixed.
> > > > >> >> > Since our races in these scenario are the length of time
> taken
> > > for
> > > > >> cassci
> > > > >> >> > to vet them, these problems are much more likely than current
> > > race
> > > > to
> > > > >> >> > commit.
> > > > >> >> >
> > > > >> >> > In the scheme I propose, in this scenario, the person who
> broke
> > > the
> > > > >> build
> > > > >> >> > rebases everyone's branches to his now fixed commit, and the
> > next
> > > > >> broken
> > > > >> >> > commit gets blamed, and all other commits being merged in on
> > top
> > > > can
> > > > >> go
> > > > >> >> in
> > > > >> >> > smoothly. The only pain point I can think of is the
> > multi-branch
> > > > >> rebase,
> > > > >> >> > but this is solved by git rerere.
> > > > >> >> >
> > > > >> >> > I agree running tests is painful, but at least for the build,
> > > this
> > > > >> should
> > > > >> >> >> be the responsibility of the committer to build before
> merging
> > > > >> >> >
> > > > >> >> >
> > > > >> >> > Why make the distinction if we're going to have staging
> > commits?
> > > > It's
> > > > >> a
> > > > >> >> bit
> > > > >> >> > of a waste of time to run three ant real-clean && ant tasks,
> > and
> > > > >> >> increases
> > > > >> >> > the race window for merging (which is painful whether or not
> > > > involves
> > > > >> a
> > > > >> >> > rebase), and it is not a *typical* occurrence ("alarming" is
> > > > >> subjective)
> > > > >> >> >
> > > > >> >> > On Thu, May 7, 2015 at 2:12 PM, Sylvain Lebresne <
> > > > >> sylv...@datastax.com>
> > > > >> >> > wrote:
> > > > >> >> >
> > > > >> >> >> > If one of them breaks, we go and edit the _staging branch
> in
> > > > place
> > > > >> to
> > > > >> >> >> correct
> > > > >> >> >> > the problem, and let CI run again.
> > > > >> >> >>
> > > > >> >> >> I would strongly advise against *in place* edits. If we do
> it,
> > > > we'll
> > > > >> >> end up
> > > > >> >> >> in
> > > > >> >> >> weird situations which will be annoying for everyone.
> Editing
> > > > commits
> > > > >> >> that
> > > > >> >> >> have
> > > > >> >> >> been shared is almost always a bad idea and that's
> especially
> > > true
> > > > >> for
> > > > >> >> >> branch
> > > > >> >> >> that will have some amount of concurrency like those staging
> > > > >> branches.
> > > > >> >> >>
> > > > >> >> >> Even if such problems are rare, better to avoid them in the
> > > first
> > > > >> place
> > > > >> >> by
> > > > >> >> >> simply
> > > > >> >> >> commit new "fixup" commits as we currently do. Granted this
> > give
> > > > you
> > > > >> a
> > > > >> >> >> slightly
> > > > >> >> >> less clean history but to the best of my knowledge, this
> > hasn't
> > > > been
> > > > >> a
> > > > >> >> pain
> > > > >> >> >> point so far.
> > > > >> >> >>
> > > > >> >> >> > wait for CI; all clear
> > > > >> >> >> >
> > > > >> >> >> > git checkout cassandra-2.0; git merge
> cassandra-2.0_staging
> > > > >> >> >> > git checkout cassandra-2.1; git merge
> cassandra-2.1_staging
> > > > >> >> >> > git checkout trunk; git merge trunk_staging
> > > > >> >> >> >
> > > > >> >> >> > This does introduce some extra steps to the merge process
> > > > >> >> >>
> > > > >> >> >> If we do this, we should really automate that last part
> (have
> > > the
> > > > CI
> > > > >> >> >> environment merge the staging branch to the non-staging ones
> > on
> > > > >> >> success).
> > > > >> >> >>
> > > > >> >> >> > It seems if we want an "always releasable" set of
> branches,
> > we
> > > > need
> > > > >> >> >> something
> > > > >> >> >> > along these lines.
> > > > >> >> >>
> > > > >> >> >> Agreed as far as having staging branches vetoed by CI goes.
> > Less
> > > > sure
> > > > >> >> about
> > > > >> >> >> the edit-commit-in-place part as said above.
> > > > >> >> >>
> > > > >> >> >> > I certainly break tests by mistake, or the build itself,
> > with
> > > > >> alarming
> > > > >> >> >> regularity.
> > > > >> >> >>
> > > > >> >> >> I agree running tests is painful, but at least for the
> build,
> > > this
> > > > >> >> should
> > > > >> >> >> be
> > > > >> >> >> the responsibility of the committer to build before merging.
> > We
> > > > all
> > > > >> >> forget
> > > > >> >> >> it from
> > > > >> >> >> times to times and that's ok, but it's not ok if it's
> > > "alarmingly
> > > > >> >> regular".
> > > > >> >> >>
> > > > >> >> >> --
> > > > >> >> >> Sylvain
> > > > >> >> >>
> > > > >> >>
> > > > >> >>
> > > > >> >>
> > > > >> >> --
> > > > >> >> http://twitter.com/tjake
> > > > >> >>
> > > > >>
> > > > >>
> > > > >>
> > > > >> --
> > > > >> http://twitter.com/tjake
> > > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > http://twitter.com/tjake
> > > >
> > >
> >
>
>
>
> --
> Joshua McKenzie
> DataStax -- The Apache Cassandra Company
>
