Fwd: Rebase safely (Re: cherry picking and merge)
[ sorry for the dup ] Begin forwarded message: On Aug 6, 2014, at 12:44 PM, Nico Williams n...@cryptonector.com wrote: It's not a good idea to rebase a branch in a repo that others pull from. Well, so rebase is then out, as I don’t want to rebase _my_ tree, I want to rebase _the_ tree. Recall, I don’t want to cherry pick for my tree, I want to cherry pick for the tree. [ reads rest of email ] Oh, wait, maybe I have misunderstood the prohibition. I have: upstream —— fsf | \ | v Me — master — coworker. Me is a git clone of master, coworker is a git clone of master. Master is a bare repo on a shared server where we put all of our work. upstream is a bare git repo of the fsf git tree for gcc. fsf is a box owned by other that hosts the gcc git repository. I do a git pull fsf in upstream from time to time, and a corresponding git merge fsf in Me from time to time. When I like my work I do a git push (to master exclusively). To go to upstream, we submit patches by hand, git is not really involved. I never pull into master from upstream (don’t even think that’s possible since they are both bare). I read the prohibition as don’t rebase my branch called master on Me and push it to master on master as others then pull master from master. Did I misunderstand? Instead, the prohibition is you can use push/pull freely and you can have as many coworkers as you want, just don’t use push -f and don’t let anyone push/pull from your own private clone. I had envisions that the rebase of master on Me once pushed to master and then pull from master to coworker is the exact case that would screw us. The only use-case I've seen where a rebase-based workflow doesn't work Well, and now mine, which I claim is a the canonical open source use case. Can't use source, unless you import the source, can’t be real unless you can change the source, once you do that, you then need to merge in newer sources, and if the company has or will have more than a single individual and these folks are ever to work together, then they need to share the source between them. I’m trying to envision how anyone could ever use rebase. If you can’t share your work, it isn’t work. is where you have multiple upstreams that you're following. I only have a single (for this repo) upstream. Now, I found the stack-overflow commentary first, and all the horrors of it, and all the nuances. I carefully read what people were doing, how what I wanted to related to what they were doing, and it sure felt like I was in the, don’t go there camp. A lot of people rant about rebase. They're wrong. They've distorted your perspective. What I saw were the people that screwed their world and were trying to recover. It was a question, how do I recover, and what did I do wrong. There was no rant. Or, at least, I’m impervious to rants and don’t actually see them. I deal in the cold hard facts and transform the rant into what happened, what they did wrong, and how to avoid doing it myself. No, you’ve set my perspective, let me quote you: It's not a good idea to rebase a branch in a repo that others pull from. this matches the doc, matches the experience of users on stack overflow and matches what what I think is true. You are free to correct that if I am wrong. I don’t know why you think my perspective is distorted. Either, I can rebase all my patches, all my coworkers patches, and push those up to master and have all my coworkers pull from master and develop (meaning branches off master as well as patches to master) as normal, or I can’t. There's just two simple rules to follow and you'll be safe: 1) NEVER git push -f (--force) to a published repo/branch. I can never use push -f. That seems trivial. git config --system receive.denyNonFastForwards true seems to be exactly what I would do to my master to enforce this rule. This then seems to permanently be a non-issue. 2) NEVER work directly in a published repo. Instead work in a private clone. I only ever work in a private to me clone, so I’m safe. The only published repo is a bare repo, which can’t be worked in by design, so, again, I think I’m perfectly safe. So, if that is true, why do others write such things as (from http://ctoinsights.wordpress.com/2012/06/29/git-flow-with-rebase/): The way to get the best of both worlds is to be explicit about when to use one versus the other. For us the simple rules to follow are: • Rebase feature branches. • Never rebase develop or master branch. (Always merge into develop and master.) • Never rebase public branches. master is public, I want to rebase master. This violates rule 3 above, but not any of your rules? I develop on master, thus violating rule 2. This does’t violate any of your rules? I want to rebase master, thus, violating rule 1. This doesn’t violate any of your rules? I violate
Re: Rebase safely (Re: cherry picking and merge)
On Aug 6, 2014, at 10:11 PM, Nico Williams n...@cryptonector.com wrote: Nah. Sun managed this for decades without a hitch, and for products much larger than GCC. See above. Ok. Ah, ok, perfect. I see how that method of working would cure the cherry-pick and merge don’t work problem mentioned at the top of the thread. Do some experiments based on the above hardcopy. If that doesn't convince you that it works, oh well, I'll have given it a good try. Thank you for taking the time to diagram that as it appears to violate everyones how to use git guide. I see the workflow does an onto, which was the ‘fix’ people talked about on stack overflow, and I see just how things would work. If the old master branches are deleted and gc is run, then all the old references go away, and then the refs from email and bugzilla then don’t work. Did you guys ever remove them and then prune (or gc)? Now, the biggest issue, if that is recognized as `fixing’ the cherry-pick problem, then certainly the problem is understood to be a problem. If one recognized it as a problem, then one can envision cherry-pick and merge working together so that the problem doesn’t need fixing in the first place. And, if it doesn’t need fixing, then the cost of the solution isn’t needed either. The biggest problem with git, is that two features don’t work nicely together when they could; in this case, cherry-pick and merge). Because they don’t, it makes it hard for people to predict what will happen when they use it. This makes it more expensive to use and less suitable than a system that is more predictable. You improve git, by fixing the problem and making the features work nicely together and making it predicable. I still favor fixing the underlying problem with cherry-pick and merge not working. :-) That said, I see how to work around the bug with rebase, if I need to. I wish the top google hit were your guide and I wish I never saw all the other pages… I see now your position, and I see why all the guides are wrong, if you know just how to use rebase. I wish the git documentation were improved to say as the first sentence under cherry-pick, this feature sucks and doesn’t really work well, it can cause excess merge conflicts. rebase can be used to work around the bugs in cherry-pick for now. And under rebase, instead of saying what it said now, that how one can can trivially and effortlessly use git, instead of saying, Do not rebase commits that you have pushed to a public repository which I now see is wrong.-- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cherry picking and merge
On Aug 6, 2014, at 8:43 AM, Jakub Narębski jna...@gmail.com wrote: I gave a solution for git using branches and it works just fine. It retains the simple 3-point merge as well. It works for this simple case, but I think it has unfortunate potential to go silently wrong. That just means that you want to have two commands. One for the people that when the remove a patch, they want it gone. The other for people that when they remove a patch, they want it to magically reappear. I’m of the former class of individuals. Now, I would argue that that is the wrong solution of course. See below for uncherry-pick. Now, if I needed a solution to the one problem that was mentioned, I would then request an uncherry-pick command to undo the cherry-pick. The semantics of it are, the patch is removed from the tree, and when merged, that patch isn’t removed from the source. See, we then retain the useful property that everything that should work does, and the system is predictable because it then does exactly what the user said to do. Conceptually of course, it doesn’t have anything to do with cherry, if you merge a branch accidentally, and then remove it, and merge it, I think you still wind up with the work being removed. Conceptually, it is just an undo a change, cherry, merge, file rename, whatever. Now, why is this preferable? Because the advanced user gets to explain what they want to git, and then git does what they want. It also works for beginning users, it does what they ask it to do. If you are afraid you know better what command that they really wanted to use instead of the command they are using, you can prompt them and ask, did you mean this or that? After 20 times being asked, it would get old and then even a new user would just issue the commands they want. I’m not in favor of that, I’d prefer that the system just do what they tell it to do. Also, it prevents fully removing (commits, not only refs) the branch you cherry-picked from. The commit you cherry picked may no longer be (or may no longer should be) in the repository. I’m picking from trunk, when it goes, I go. :-) Also, this could be avoided by using feature branches and merging instead of committing to one branch and cherry-picking to other branches. If the problem remains unfixed, at least the documentation should be changed to say cherry will mess up merge. If you never merge, never a problem. For me, I would read that, and say, well, trivially, cherry isn’t for me (til they fix the bug that causes it to mess up merges). I can’t see anything on http://git-scm.com/docs/git-cherry-pick which says it will mess up merges.-- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cherry picking and merge
On Aug 1, 2014, at 4:40 PM, Nico Williams n...@cryptonector.com wrote: As for rebase, I still don't understand why it doesn't work for you. http://git-scm.com/docs/git-rebase says: Rebasing (or any other form of rewriting) a branch that others have based work on is a bad idea If you read stack-overflow, you will discover a ton of people that like doing this, and they get hammered because of it. My use case fits exactly (as near as I can tell) the hammer case. If you make a different command that isn’t guaranteed to screw me and my co-workers over, and tell us to use that, then I’d be happy to use it. Bet the farm that it won’t bite you, just to be bitten isn’t what I want to try and recover from. You didn't really explain. If you say it will never bit you and then fix all the documentation to not say it will bite you… I’d be happy to contemplate it again. Now, I found the stack-overflow commentary first, and all the horrors of it, and all the nuances. I carefully read what people were doing, how what I wanted to related to what they were doing, and it sure felt like I was in the, don’t go there camp. Suppose we're right and it's the right solution for you, then you might be ecstatic, but you gotta try it first. So, I like to know if I’m driving off a cliff, before I do. I’m the type of person that would rather know were the road goes, and merely avoid driving off the cliff. When stack-overflow is littered with the bodies of people that thought it would be fun, I tend to just say, that’s not for me. The only case where I can imagine not using a rebase-heavy workflow is where I have to track multiple forked upstreams and so I want to merge each into my branch. So, sounds like I fit that use case and rebase could be my friend. How do I square what you said and: Rebasing (or any other form of rewriting) a branch that others have based work on is a bad idea ? I want all old refs in old emails to work. I want all refs in bugzilla to work. I want to see the original dates of all the work. I want git blame to report those artifacts in email and bugzilla. I have coworkers that I push to, pull from (through a single sharing point, we call the master tree). We work on gcc, we pull git gcc down to a local copy, then merge it into our tree. I want to cherry pick changes from upstream. I do work and push to our master, I pull work of coworkers from the master, my coworkers do the same. Isn’t this the canonical open source use case? (I find that many users are allergic to rebasing. Many people have told me that rebase is lying, that history must be immutable, and so on, all ignoring that: git users don't rebase published branches, So, when I push, and someone else pulls, is that published? I thought it was.-- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cherry picking and merge
On Jul 31, 2014, at 7:43 PM, brian m. carlson sand...@crustytoothpaste.net wrote: You're not the first person to be surprised by the way merge works. I’m not the first, because the merge command is broken. Once fixed, I would be happy to be the last. Until then, the bug remains unfixed. I’m sending the mail to petition for this bug to be fixed. From the git-merge manage: [This behavior] occurs because only the heads and the merge base are considered when performing a merge, not the individual commits. From: google(”git merge”): git-merge - Join two or more development histories together Either, the command should do as documented, or be fixed. In that reference, there is no mention that merge will not merge. There is no mention that merge isn’t the command I want to merge, but that I should use rebase. Further, google(“git rebase”) says: There is no difference in the end product of the integration, Clearly, this is a lie. There is a difference. Now, about rebase: Do not rebase commits that you have pushed to a public repository. If you follow that guideline, you’ll be fine. If you don’t, people will hate you, and you’ll be scorned by friends and family. Since everything I do goes up and down into repositories and I don’t want my friends and family to scorn me, rebase isn’t the command I want to use. I want to use the simple, it works, named for the operation I want to perform, merge. I’m a simple user, and the simple command I want to work. You can name the old merge command, merge-mostly or merge-fast and the new one can be called merge. (That was added after 1.7.9.5.) I don’t want bugs documented, I want them fixed. I’m not reporting a doc bug, the doc is correct. If you want the behavior of applying multiple patches in a row, you want to use git rebase, not git merge. Since rebase re-applies the patches of each of your commits on top of another branch, the identical change won't cause conflicts. But, I don’t want the series of patches, I just want a simple, merged feature X on trunk single commit that merge does. Given branch B, master M, and cherry picked C, what I want merged is B-(M+C), not B-M. The problem with B-M, is that when you do B += C (aka cherry pick from master onto your branch), then M += B-M (merge your branch into master), that C is then replicated. This replication is wrong, always wrong, never right, incorrect, broken. This is the bug I want fixed.-- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cherry picking and merge
On Aug 1, 2014, at 9:27 AM, Jakub Narębski jna...@gmail.com wrote: Note that you should try to avoid cherry-picking, as they do not leave trace in the graph of revisions. Fine, then I want a new command to merge in a change into my branch from another branch and I want merge to account for the motion and not duplicate it when I merge that branch back into master. Funny thing is, cherry and merge seem to be documented mostly to do exactly what I want. For example if you are creating a bugfix, instead of putting it directly on maint, and then cherry-picking to master, it is better to create a separate feature branch for this fix You’re assuming that I’m the author of master, I’m not, I’m merely a contributor. This tail doesn’t wag that dog. What that means is that I cannot change the world to work around a simple bug in git. There is also git-imerge, third party tool that is intended to help merging changes (and make it possible to do it in incremental way). Then remove git merge and replace it with git-imerge. :-) Anyway, I read that, and I can see some beauty of that that might be nice in complex merges. The problem is, I want git merge to work. I was curious if svn handles this better the same or worse, and it did it just fine. I know that a while ago, svn could not handle this, it would do what git does currently. Apparently they figured out it was a bug and fixed it. Have you guys figured out it is a bug yet? The first step in solving a problem, is admitting you have a problem.-- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cherry picking and merge
On Aug 1, 2014, at 12:22 PM, Nico Williams n...@cryptonector.com wrote: If you always rebase I can’t use rebase unless you make rebase work with multiple users and pushing pulling.-- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cherry picking and merge
On Aug 1, 2014, at 11:57 AM, Philip Oakley philipoak...@iee.org wrote: But that goes both ways, and is a philosophical issue about what is to be expected in various cases. The problem is, users expect merge to merge. There isn’t a user that expects it to scramble the source code, because the command is called merge, not scramble. That word has semantics that were not invented by your project. You cannot change the semantic of the word. Merge has a nice mathematical definition. Merge branch into master means, place into into master the work from that branch. git already does this 99% correct, it is missing one corner case. This is not a philosophical issue. It is a definitional one. For some central control use styles, the ideas behind _distributed_ version control are anathema and (Git) just grinds away at the policies that are expected. This is irrelevant to the issue at hand. That said, Git doesn't claim to be perfect Again, irrelevant. (and can't because Do you mean, and can’t be? If so, you are wrong in the case at hand. svn is an existence proof that you are wrong. of the 'relativity' that comes with being distributed - truth has to give way to a web of trust). Also the artefacts that Git validates are at a different level of abstraction i.e. the whole project as a commit, rather than just a few/one file at a time. Ah, so that gives me an idea. [ pause ] If we try the cherry-pick as retroactively creating a feature branch, cherrying into that, then merge unconditionally so that no change happens that into trunk (thus killing those conflicts), and then git merge that feature branch into branch then it all works perfectly. See, another existence proof that you are wrong, this time with git itself. It was 13 lines of code, so, apparently, it is possible and easy to do, in git. Now, we just want the cherry-pick to create a temporary cherry branch, cherry the pick into it, merge and drop into trunk and merge into branch… I tested with the below and it worked just fine. Things to clean up, we want the meta data on the cherry on the merge commit, but, you get the idea. branch=b master=master base=$(git merge-base $branch $master) cherry=$1 git checkout -b cherry-$branch $base git cherry-pick $cherry git checkout $master git merge -s ours cherry-$branch git checkout $branch git merge cherry-$branch git branch -d cherry-$branch git cherry-pick --strategy=ours --allow-empty $cherry git commit --allow-empty I tested that with two cherries with further changes on master to ensure that it works for more than a single one, no problem. Wow, even tried a merge of master back into b, and it worked just fine, no conflicts, yet, all the code was jammed up together nicely. So, if you wish to continue your position, please explain why it can’t get this better, given the existence proof above of it working better in git. In your example (when generalized) I’m not interested in other bugs that I didn’t state, in this email. I don’t care about those. Please don’t detract from fixing this issue, because you can identify other things that might not be perfect. We attain perfection one step at a time. the problem is deciding when, in the change sequence, the cherry pick is to be backed out, especially if there are conflicts in the change sequence that would need fixing anyway, and in a long change sequence that would be a lot of conflict fix-ups, hence the current choice of getting the merge conflicts all resolved in the one go. I have two possible conflict fixups in the above. In my case (I have a specific patch in gcc-land i wanted to cherry), those fixups were trivial (no conflicts). When they are trivial, I don’t care much that there were two of them. When non-trivial, well, I’m resigned to the idea that I have to explain what is going on. Selecting a compatible workflow is a problem of usage, Not when the workflow is mandated on you to work around trivial little bugs that can be fixed but for which the author’s don't even comprehend the bug. rather than a problem in Git. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cherry picking and merge
On Aug 1, 2014, at 12:01 PM, Jakub Narębski jna...@gmail.com wrote: It can work in Subversion because Subversion stores information about what was merged in (and this includes cherry-picks, or whatever it is named in svn) in svn:mergeinfo property. Git does not track what was merged in, instead it represent the history as the graph of revisions, and tracks merges (by storing that it came from two or more commits) and not merged-in information. So, as a dumb user that just wants it to work, I am unsympathetic to the `but software is hard’ excuse. I am aware that some bugs are harder to fix than others. svn took a long time to fix this bug, but they did. I can wait, the only question is, will it be a week, a month, a year, or a decade. When merging Git uses only what is being merged and its common ancestor (3-point merge). It is simple, and simple works!!! I gave a solution for git using branches and it works just fine. It retains the simple 3-point merge as well. Unfortunately, it does not see cherry-picked commits - it is invisible to merge as being on the chain from one of merged commits to the common ancestor. Im the solution that I sketched in my previous email, that information is then exposed so that the right merge happens. The rebase command handles I can’t use rebase as it is unfriendly to coworkers. cherry-picked commits by detecting that the change was already applied. I think that git-imerge does the same (but I have not used it myself). Have you tried git-imerge? No, not yet. I’m not as interested in using it, as I would like git itself to just work.-- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cherry picking and merge
On Aug 1, 2014, at 1:02 PM, Jonathan Nieder jrnie...@gmail.com wrote: Do you mean that git merge should be aware of what changes you have already cherry-picked? Yes, it amounts to that. It isn't, and that's deliberate Deliberate bugs are still bugs. In time, users will either wear you down until you fix it, or they will move on to other systems that work better. (git merge is designed to be simple as possible, though no more simple than that). I sketched a solution that retains a simple git merge… This way, if on a side branch someone makes a change that would conflict with master and then backs it out, then the branch can still merge cleanly. Yeah, my solution doesn’t impinge upon that working nicely. In it, I make cherry use scratch branches to record meta information so that the existing git merge just works. git cherry is free to do the same. Having a git cherry that fully interoperates with git merge, is a feature. Even in those workflows, it's possible to have conflicts due to genuinely conflicting changes, even with no cherry-pick involved. I find the '[merge] conflictstyle = diff3' setting (see git-config(1)) and git-rerere(1) to be helpful in making that less painful. I think those two should be the default, but it is easy enough to turn them on that it doesn’t matter much. In my environment, I have both on.-- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cherry picking and merge
On Aug 1, 2014, at 1:12 PM, Sam Vilain s...@vilain.net wrote: Git merge has a notion of discrete merge strategies”. There's no particular reason that you couldn't implement a merge strategy which works more like SVN's approach, which essentially does an internal rebase and then commits the result. Well, being a simple user, wanting to do a simple thing, I want the default strategy to just work. The expert that wants it to work faster, but less well, well, they can use a merge -s faster, or cherry-pick -s faster. However, there are corresponding disadvantages to this strategy. It's just as easy to contrive a situation where this internal rebasing doesn't do the right thing, even without cheating by getting the metadata wrong. I sketched a solution using branches… A large portion of the failures that happen when a cherry is remerged are gone. I feel that benefit easily swamps the problem you sketched. And besides, there's already a way to do this: do an actual rebase. One problem is that rebase doesn’t work with co-workers nicely… The other is that it isn’t spelled merge. I am a simple user. I was curious if svn handles this better the same or worse, and it did it just fine. I know that a while ago, svn could not handle this, it would do what git does currently. Apparently they figured out it was a bug and fixed it. Have you guys figured out it is a bug yet? The first step in solving a problem, is admitting you have a problem. So, I have to chuckle when I read this indignant comment. :-) Yeah, a chuckle, good. Actually, no anger is involved. I’d just like for git to work better in this regard.-- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cherry picking and merge
On Aug 1, 2014, at 1:50 PM, Jonathan Nieder jrnie...@gmail.com wrote: And on the other hand a one-patch-at-a-time merge would try to apply X (with no effect, since it's already applied) and then try to apply the revert of X. The net effect would be to revert X from master (bad)! Yeah, I appreciate that. I know the type of user that would do that, and I understand why you would want to do that and even do that by default. However, as an expert user, I don’t need that particular type of hand holding given the cost of more conflicts and would like an option to let me to choose to have fewer conflicts when merging.-- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
cherry picking and merge
Cherry picking doesn’t work as well as it should. I was testing on git version 1.7.9.5. Put in a line in a file, call it: first version then cherry pick this into your branch. Then update on master and transform that into: second version then, merge that branch back to master. Death in the form of conflicts. In gcc land, I do this sort of thing all the time, and I need a merging subsystem to actually keep track of things. I can manage this will diff and patch and it works flawlessly. The point of using something better than diff and patch is for it to be better than diff and patch. I’d like for merge to merge in the work that has yet to be merged. Not that, plus blindly try and apply or reapply cherry picked items.-- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
