On Jul 29, 2014, at 8:40 PM, Sheng Yang <sh...@yasker.org> wrote: > And talking about to achieve good code quality, I think compulsive review > and testing through some kind of automation system is much more helpful. > > --Sheng >
This new git workflow is not directly about good code quality, as a wise man told me the other day "crap gets in, crap gets out, not matter what you do with git". IMHO, code quality will indeed be fixed with more reviews and a CI pipeline, but it will also mean accepting only patches that have tests with it. So no unittests, no integration tests, no commit. But I think we are way far from this. There is currently a BVT automatic testing infra proposal on the table from citrix, but this needs to be on ASF infra, and it seems no-one has wanted to push this proposal much further. Starting with a better git workflow should bring us up to standard methods of developing with git. x.y-forward has been a mess, calling for [MERGE] request for new features has been totally neglected by developers, folks fix a bug in master but don't care to check if it applies on other branches and don't fix it there. Or maybe they fix on a release branch and don't fix on master. The end result of this bad practice is this: -I have personally 0% confidence that a bug fixed somewhere is fixed everywhere (it gets even worse when you bring in bad JIRA practices with bugs being closed when they are not closed). -We have quite a bit of regressions -Our new releases are pretty much different/diverging products -We don't know what's in a future release (exemplified by difficulty for RM to get list of new features) -And more importantly we don't have a stable branch. This proposed git workflow would give us: -A stable branch (master). You checkout master get a shippable product. Very few commits to master are made, development is moved to 'develop' branch. Only hot fixes of current release get merged in master (which means it's still a shippable product). I am in favor of being draconian with this, and I am sure we would see a lot of reverts in the first weeks of adopting this workflow. But we need to stabilize master. -Everyone works on branches and knows where to merge. Developers will know where they branched from and where they need to merge into. This will also mean that history, tracking, reverting should also be easier. (personally I am not entirely against cherry-picks and I could live with some minimal amount done by the RM, but they problem with cherry picking right now is with the way we apply patches to various branches…untraceable). -This will also mean that our release branches are clearly defined and a RM can actually pick features to merge into a release, but these release branches should/will be merged into master instead of being abandoned like we do now. -Now about minor releases. Say we release 4.5, the tag is in master (not on a release branch), we have a few hot fixes that John Smith says are very important. Those hot fixes get merged into master (and into develop), we then call a vote on the latest master (e.g 4.5.1-tentative) if it passes tag it (4.5.1) and we have a new minor release without new features. Master is stable. The big advantage I see with this is that our release process should be much easier and we will have one continuous shippable product, instead of real forks. -sebastien > > On Tue, Jul 29, 2014 at 4:14 PM, Sheng Yang <sh...@yasker.org> wrote: > >> Hi Leo, >> >> I am afraid the answer is too generic. We'd better look into details to >> see what's the exactly problem. >> >> So I am trying to sort it out. >> >> The first question we want to ask is: >> >> A. What's current develop process? >> >> I don't know where is it documented, but as I know, it works like >> this(please correct me if I am wrong) >> >> 1. Development happen on MASTER first. >> >> - Can merge feature branch. >> >> >> 2. Feature freeze, cut staging release branch. No major feature can be >> checked in. >> >> - Say we would release 4.4, then RM create branch 4.4. >> - All the major feature would be worked on MASTER. >> - All the fix for 4.4 need to be applied on 4.4, and MASTER. >> >> 3. Code freeze. Branch out staging tree, try to enforce quality. >> >> - RM branch 4.4-forward from 4.4 branch. >> - 4.4 branch freezed, only allow to be checked in by RM. >> - Any 4.4 bug fix need to be applied on 4.4-forward, and after all the >> test and check, developer would ask RM to cherry-pick it to 4.4 branch. RM >> would be the gate keeper for 4.4 branch. >> - In the meanwhile, MASTER would still serve as unstable tree for next >> release. >> >> 4. Release. Release branch would be release. >> >> - Release based on branch 4.4, then dropped branch 4.4 >> - 4.4-forward branch would be rename to 4.4, and continue to serving >> as release base for 4.4.1 release. >> >> 5. For release of 4.4.1 and after. Repeat step 3 and 4(probably not >> strictly in fact). For release of 4.5. Repeat steps 1-4. >> >> The issue with current development process? >> 1. Cherry-pick is needed for RM to choose stable fix commit after code >> freeze to enforce quality. >> 2. Bug fix to current release branch need to be checked in both release >> branch and MASTER. >> 3. I believe there would be more. Please comment. >> >> B. What's the purposed process. >> >> The idea of gitflow is from >> http://nvie.com/posts/a-successful-git-branching-model/ >> >> To my understanding, compare to our current process, it suggested: >> 1. Develop branched out from MASTER branch, which would be what's majority >> of work happens on. >> >> - Develop branch can merge feature branch. >> - MASTER would always be stable. >> >> 2. Feature freeze: Create Release branch: When ready to release, branch >> out Release branch from Develop branch. >> >> - Release bug fixes only happen on Release branch. >> - Develop branch need to constantly merge Release branch for latest >> bug fixes. >> - In the meanwhile, Develop branch can still merge feature branch. >> >> 3. Code freeze: No such concept in the flow. Need to know how to guide the >> check in. >> >> 4. Release. Release branch merge into MASTER. Tagged and release. >> >> 5. The original post didn't mention minor release. I suppose it may work >> like this: >> >> 1. Branch out MASTER to 4.4 release >> 2. Any bug fixes happened on 4.4 need to be merge to MASTER. >> 3. But, how to fix a 4.4 bug in 4.5's minor release? Seems what's >> needed is MASTER merged 4.5 which merged 4.4 which merged 4.3. And how can >> we fix it in develop branch(which meant for test)? I can't see how you can >> avoid cherry-pick or extra commit if you want to have a maintenance >> release. >> >> Before realizing the maintenance release issue, I was quite like this >> approach, which means no more duplicate work need to be done. But sadly the >> most usage for cherry-pick is due to we need to fix multiple versions. >> Current I can't see how utilize gitflow or git merge would solve the issue. >> Is there a better way to handle this? >> >> And, I think gitflow model doesn't have "code freeze" period which we can >> have a gatekeeper to keep the quality. So, if we want to compare to gitflow >> model, we would simply skip "code freeze" stage in our current process. I >> don't think it make much sense. >> >> I think all gitflow has is eliminate one time check in for fixing current >> release branch(we need to check in both release branch and master branch) >> compare to our current process, but: >> 1. How would it handle maintenance release? >> 2. How would code freeze work? >> 3. How about the workload for merge between release and develop branch? It >> would be quite frequently. >> >> As a developer, I hate multiple branches check in as much as anyone else, >> probably even more. But I am not sure gitflow can help us here. >> >> Btw, that no-cherry-picking( >> http://www.draconianoverlord.com/2013/09/07/no-cherry-picking.html) article >> has a mistake in it. The result it's showing in fact shows that git cherry >> -v DID recongize the cherry-pickup result by showing "-" in front of the >> commit. I guess git is smart enough to know that. >> >> Here is the manual of git-cherry. >> >> SYNOPSIS >> git cherry [-v] [<upstream> [<head> [<limit>]]] >> >> DESCRIPTION >> Determine whether there are commits in <head>..<upstream> that are >> equivalent to those in the range <limit>..<head>. >> >> The equivalence test is based on the diff, after removing >> whitespace and line numbers. git-cherry therefore detects when commits have >> been "copied" by means of git-cherry-pick(1), git-am(1) or git-rebase(1). >> >> Outputs the SHA1 of every commit in <limit>..<head>, prefixed with >> - for commits that have an equivalent in <upstream>, and + for commits that >> do not. >> >> --Sheng >> >> On Tue, Jul 29, 2014 at 12:52 AM, Leo Simons <lsim...@schubergphilis.com> >> wrote: >> >>> On Jul 29, 2014, at 5:45 AM, Sheng Yang <sh...@yasker.org> wrote: >>>> I am trying to catch up, by reading the thread and checking what's >>> gitflow >>>> etc, but could someone already familiar with the topic give an overview >>> of >>>> the issue? >>>> >>>> For example, we can start by asking these questions: >>>> 1. What's the issue with current development process? >>> >>> Right now it is quite hard to get to a stable release, or to produce high >>> quality contributions, and this happens in part because of the git workflow >>> in use. >>> >>> Cherry-picking is an approach where git can provide 0 assistance with >>> branch and merge management. Read >>> http://www.draconianoverlord.com/2013/09/07/no-cherry-picking.html >>> for an introduction to that subject, for example. >>> >>>> 2. What's the purposed new approach to the process? >>> >>> To use a branch/merge workflow rather than a cherry-pick workflow, >>> preserving commit ids across branches. >>> >>> To adopt a specific well-documented workflow called git-flow that has >>> tool support. Read >>> http://nvie.com/posts/a-successful-git-branching-model/ >>> for an introduction to git-flow. >>> >>> To then tune this workflow to cloudstack. In particular, so far, I’ve >>> noted >>> * not deleting release branches once releases are finished >>> * consider using support/ branches for point releases rather than hotfix/ >>> branches >>> >>> Note that this new workflow implies a variety of things, including: >>> * sharing responsibility for merges among many committers (as opposed to >>> a release manager responsible for cherry picking) >>> * distributing the ‘merge pain’ throughout the development process as >>> features finish up (rather than ‘big bang’ during release time) >>> >>>> 3. What's the pro/con of the new process? And what's the pro/con of the >>> old >>>> one? >>> >>> This is very difficult to summarize fairly. >>> >>> IMHO the only advantage of the old process is that it is what everyone >>> knows already. It’s main downsides are that it is not using git’s excellent >>> branch management features, does not allow comparing or merging between >>> branches, requires contributions to be re-written for multiple branches, >>> and encourages developers to ignore merge conflicts until release time. >>> >>> IMHO any workflow that does not rely on cherry-picking has only >>> advantages compared to the current process. >>> >>> Git-flow has many people that like it and many people that don’t. But, >>> the people that don’t like it usually use another branch/merge model, not >>> cherry-picking. It’s main advantages among available branch/merge workflwos >>> are being well-defined, oft-used and tool-supported. >>> >>>> That would make the question much more clear. >>> >>> Hope this helps, >>> >>> >>> cheers, >>> >>> >>> Leo >>> >>> >>
