Re: Embracing git usage for Firefox/Gecko development?
On Thu, Jul 11, 2013 at 10:49 AM, Chris Peterson cpeter...@mozilla.com wrote: Can you describe your patch queue-like workflow using git? git rebase -i lets you reorder commits, but how do you quickly do the equivalent of hg qpopping and qpushing a bunch of commits? What I normally do is make the change in the working directory, commit it as a temporary commit, then use interactive rebase to fold it into the final commit I want the change to belong too. When working on multiple patches for a bug these are all separate commits and I'm committing/folding/resetting as needed. And if I make a mistake there's the reflog to recover from or I can just create a new branch if I'm going to do major surgery to the commits, allowing me to go back to the old branch if I change my mind. -- http://www.bluishcoder.co.nz ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Embracing git usage for Firefox/Gecko development?
On 7/11/13 12:05 AM, Justin Lebar wrote: On Wed, Jun 5, 2013 at 3:25 AM, Philipp Kewisch mozi...@kewis.ch wrote: git rebase --interactive. It is /far/ more powerful than mq for this use-case. I even have a |git qrebase| alias for this. https://github.com/jlebar/moz-git-tools Looks very interesting, lots of nice tools I could make use of :) I have the feeling git is very powerful, but the commands to use it are not very intuitive. You have a repository full of commands I would expect git to have included. Yes, somehow rebasing seems to be the solution here, but with git its quite common that you push your changes to your own remote and then do pull requests. This again will require me to do push -f almost always, since I often change the order of patches. This doesn't sound ideal to me. I don't understand the objection here. Yes, if you're changing remote history (e.g. updating a pull request after you re-ordered patches), you have to push -f. What's the problem with this? It just seems wrong, or unclean to me to force a push. Maybe its just me used to hg where I keep things for myself and only upload patches for review. Philipp ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Embracing git usage for Firefox/Gecko development?
Can you describe your patch queue-like workflow using git? git rebase -i lets you reorder commits, but how do you quickly do the equivalent of hg qpopping and qpushing a bunch of commits? qpop qpush is a nop, of course; what do you want to in-between those two commands? I have a |git qrebase| command which does |git rebase -i| on my whole patch queue. The patch queue is defined as the set of commits in the range $(git qparent)..HEAD, where |git qparent| is the first common ancestor between HEAD and |git tracks| (i.e., |git merge-base $(git tracks) HEAD|). |git tracks| is defined as the upstream branch that the current branch tracks (as set by |git branch -u| in newer versions of git). All of these commands are defined in my git-tools repository. https://github.com/jlebar/moz-git-tools As an example, to qpop your patch queue so that commit X is the new head, then add a new patch on top of X, then re-apply your patch queue, you'd do |git qrebase|, then mark commit X as edit, then |git commit| your new patch, then |git rebase --continue|. This is better than hg in a number of ways, but among them is the fact that the old state of your patch queue is not lost for 90 days or so. If you discover that you screwed up and want to go back to the old version of the patch queue, you merely need to look through |git reflog| and then |git reset --hard Z| where Z is the commit you were at before you rebased. This is in contrast to hg, where |hg qref| is destructive and can cause you to lose work (unless you're versioning your patch queue, which is a whole other can of worms). On Wed, Jul 10, 2013 at 6:49 PM, Chris Peterson cpeter...@mozilla.com wrote: On 7/10/13 3:01 PM, Justin Lebar wrote: I can't see how they are a good alternative. With patch queues, I can maintain a complex refactoring in a patch queue containing dozens of smallish patches. In particular, I can easily realize I made a mistake in patch 3 while working on patch 21 and make sure that the fix ends up in patch 3; I don't see how that is easily achievable in git branches. This is far easier to achieve in git than with mq. Can you describe your patch queue-like workflow using git? git rebase -i lets you reorder commits, but how do you quickly do the equivalent of hg qpopping and qpushing a bunch of commits? I use ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Embracing git usage for Firefox/Gecko development?
On 2013-07-11 11:26 AM, Philipp Kewisch wrote: On 7/11/13 12:05 AM, Justin Lebar wrote: On Wed, Jun 5, 2013 at 3:25 AM, Philipp Kewisch mozi...@kewis.ch wrote: git rebase --interactive. It is /far/ more powerful than mq for this use-case. I even have a |git qrebase| alias for this. https://github.com/jlebar/moz-git-tools Looks very interesting, lots of nice tools I could make use of :) I have the feeling git is very powerful, but the commands to use it are not very intuitive. You have a repository full of commands I would expect git to have included. git rebase -i is more powerful, *and* more flexible, than mq is. Now you may not like git or not want to learn how to use interactive rebase, but arguing that mq workflow is better than what git can offer is flawed. Cheers, Ehsan ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Embracing git usage for Firefox/Gecko development?
On 7/11/13 8:00 PM, Ehsan Akhgari wrote: On 2013-07-11 11:26 AM, Philipp Kewisch wrote: On 7/11/13 12:05 AM, Justin Lebar wrote: On Wed, Jun 5, 2013 at 3:25 AM, Philipp Kewisch mozi...@kewis.ch wrote: git rebase --interactive. It is /far/ more powerful than mq for this use-case. I even have a |git qrebase| alias for this. https://github.com/jlebar/moz-git-tools Looks very interesting, lots of nice tools I could make use of :) I have the feeling git is very powerful, but the commands to use it are not very intuitive. You have a repository full of commands I would expect git to have included. git rebase -i is more powerful, *and* more flexible, than mq is. Now you may not like git or not want to learn how to use interactive rebase, but arguing that mq workflow is better than what git can offer is flawed. Oh no, sorry if this came over wrong. I agree that git is much more powerful in quite a few aspects. I like git add -p much better than the qrecord extension, and the interactive rebase is quite nice too. git status is much more verbose, and there are likely a few other goodies I missed. I'm just saying that doing things in git has a higher entry barrier. hg and mq commands seem more logical at times (why does git checkout handle both reverting files to a revision and checking out branches?) or just require less typing (git log origin..HEAD vs hg out), which in turn means less need for the user to set up aliases. Philipp ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Embracing git usage for Firefox/Gecko development?
On 07/11/2013 11:00 AM, Ehsan Akhgari wrote: On 2013-07-11 11:26 AM, Philipp Kewisch wrote: On 7/11/13 12:05 AM, Justin Lebar wrote: On Wed, Jun 5, 2013 at 3:25 AM, Philipp Kewisch mozi...@kewis.ch wrote: git rebase --interactive. It is /far/ more powerful than mq for this use-case. I even have a |git qrebase| alias for this. https://github.com/jlebar/moz-git-tools Looks very interesting, lots of nice tools I could make use of :) I have the feeling git is very powerful, but the commands to use it are not very intuitive. You have a repository full of commands I would expect git to have included. git rebase -i is more powerful, *and* more flexible, than mq is. Now you may not like git or not want to learn how to use interactive rebase, but arguing that mq workflow is better than what git can offer is flawed. I may still be missing something, but afaict mq git rebase -i hg qcrecord (from the crecord extension.) This is speaking as someone who hasn't used git rebase -i much, but people who have seem to agree with me after seeing a qcrecord/qcrefresh demo. I wouldn't be surprised if something or other can give you something similar for git, but I don't know what that might be. Nor do I need to care... yet. I (of course) have my own hacked-up version that smooths over some of the friction in the workflow, so if someone forced me to use git it probably wouldn't be too hard to port the interesting bits of crecord to a git workflow as well. (Or at least to an stgit workflow. Or guilt, or whatever the current quilt-ish hotness is.) ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Embracing git usage for Firefox/Gecko development?
I may still be missing something, but afaict mq git rebase -i hg qcrecord (from the crecord extension.) This is speaking as someone who hasn't used git rebase -i much, but people who have seem to agree with me after seeing a qcrecord/qcrefresh demo. qcrecord is, as far as I'm aware (it's been a while), much more like git add -i than git rebase -i. I would definitely be interested in having a patch-off with you sometime. :) ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Embracing git usage for Firefox/Gecko development?
On 5/30/13 8:04 PM, Joshua Cranmer wrote: I can't see how they are a good alternative. With patch queues, I can maintain a complex refactoring in a patch queue containing dozens of smallish patches. In particular, I can easily realize I made a mistake in patch 3 while working on patch 21 and make sure that the fix ends up in patch 3; I don't see how that is easily achievable in git branches. Stacked Git (stgit) is a good compromise between git's lightweight branches and hq's patch queues. I wouldn't want to use git without it. I wrote about stgit here: http://www.cpeterso.com/blog/02013/03/stacked-git-mercurial-style-patch-queues-for-git/ chris peterson ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Embracing git usage for Firefox/Gecko development?
On 5/30/2013 7:56 PM, Johnny Stenback wrote: [TL;DR, I think we need to embrace git in addition to hg for Firefox/Gecko hacking, what do you think?] Personally, the more I use git, the more I hate it. But I also see that other people love it... * Developers can use git branches. They just work, and they're a good alternative to patch queues. I can't see how they are a good alternative. With patch queues, I can maintain a complex refactoring in a patch queue containing dozens of smallish patches. In particular, I can easily realize I made a mistake in patch 3 while working on patch 21 and make sure that the fix ends up in patch 3; I don't see how that is easily achievable in git branches. * Developers can benefit from the better merge algorithms used by git. I don't know what you mean by that. Note that git's octopus merges are not directly realizable in hg, so an hg-git bidirectional mirror may need to ban such things in the incoming git. * Git works well with Github, even though we're not switching to Github as the ultimate source of truth (more on that below). Outside of the community size, I personally can't see any benefits to Github for Mozilla; if all you want is community outreach, you pretty much just need a read-only git mirror on Github, which we already have. Some of the known issues with embracing git are: * Performance of git on windows is sub-optimal (we're already working on it). * Infrastructure changes needed... * Retraining Mercurial developers to use git (if we went git-only) * Porting all of our Mercurial customizations to git for a check-in head * Updating all of our developer build documentation * Adapting build processes to adjust to the git way of doing things Option 1 is where I personally think it's worth investing effort. It means we'd need to set up an atomic bidirectional bridge between hg and git (which I'm told is doable, and there are even commercial solutions for this out there that may solve this for us). This is harder than it sounds: hg and git are subtly different in ways that can seriously frustrate users if you don't pay attention. For example (one that bit me very hard): hg records file copy/moves as part of the changeset, while git attempts to reverse engineer them when it needs to do so. So my attempt to merge my changes with a remote git repository via hg-git failed miserably since the hg local repo didn't catch the git moved repo. Mercurial also treats history as immutable, so you can't push history revisions. If all you need is a read-only mirror, these kind of changes aren't as important (you're not going to damage official records, just inconvenience some developers); for a two-way mirror, they are very much worth considering. -- Joshua Cranmer Thunderbird and DXR developer Source code archæologist ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Embracing git usage for Firefox/Gecko development?
On 5/30/2013 5:56 PM, Johnny Stenback wrote: Some of the known issues with embracing git are: * Performance of git on windows is sub-optimal (we're already working on it). This has become a bit of an urban legend; I often see it repeated but seldom with actual measurements. I don't think it's a valid reason to avoid git unless there are specific cases where git's performance on Windows is insufficient compared to hg's performance on Windows. In some brief tests using hg.mozilla.org/mozilla-central versus github.com/mozilla/mozilla-central on a modern Core i7 laptop with SSD and 8GB RAM, after throwing away the first result (so I'm measuring warm cache times; cold times would be useful too but would take more work for me to measure correctly): log the last 10,000 changes: git 0.745s hg 2.570s blame mobile/android/chrome/content/browser.xul: git 1.015s hg 0.830s diff with no changes: git 2.136s hg 2.001s status: git 3.011s hg 1.680s commit one-line change to configure.in: git 2.420s hg 3.911s clone from remote: git 26m43s hg 19m01s pull from remote to an up-to-date clone: git 1.585s hg 0.875s update working dir from tip to FIREFOX_AURORA_23_BASE: git 16.008s hg 25.704s There *are* some cases where git is worse than hg on Windows, but hg is as bad or worse for many common operations like log, diff, and commit. Overall I find both painful on Windows, but neither noticeably better than the other. (And of course some of these tests are highly unfair because the git repo has a more complete history than the hg one, or because they test network or server performance that is unpredictable and may vary between github and hg.m.o.) ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Embracing git usage for Firefox/Gecko development?
On 5/31/13 10:14 PM, Johnny Stenback wrote: On 5/31/2013 12:32 AM, Mike Hommey wrote: [...] Option 1 is where I personally think it's worth investing effort. It means we'd need to set up an atomic bidirectional bridge between hg and git (which I'm told is doable, and there are even commercial solutions for this out there that may solve this for us). Assuming we solve the bridge problem one way or another, it would give us all the benefits listed above, plus developer tool choice, and we could roll this out incrementally w/o the need to change all of our infrastructure at once. I.e. our roll out could look something like this: 1. create a read only, official mozilla-central git mirror 2. add support for pushing to try with git and see the results in tbpl 3. update tbpl to show git revisions in addition to hg revisions 4. move to project branches, then inbound, then m-c, release branches, etc Another way to look at this would be to make the git repository the real central source, and keep the mercurial branches as clones of it, with hg-git (and hg-git supports pushing to git, too). This would likely make it easier to support pushing to both, although we'd need to ensure nobody pushes octopus merges in the git repo. Yup, could be, and IMO the main point is that we'd have a lot of flexibility here. Option 2 is where this discussion started (in the Tuesday meeting a few weeks ago, https://wiki.mozilla.org/Platform/2013-05-07#Should_we_switch_from_hg_to_git.3F). Since then I've had a number of conversations and have been convinced that a wholesale change is the less attractive option. The cost of a wholesale change will be *huge* on the infrastructure end, to a point where we need to question whether the benefits are worth the cost. I have also spoken with other large engineering orgs about git performance limitations, one of which is doing the opposite switch, going from git to hg. I bet this is facebook. Their usecase includes millions of changesets with millions of files (iirc, according to posts i've seen on the git list). I've promised not to mention names here, so I won't confirm nor deny... but the folks I've been talking to mostly have a repo that's a good bit less than a single order of magnitude larger than m-c, so a couple of hundred k files, not millions. And given the file count trend in m-c (see attached image for an approximation), that doesn't make me feel too good about a wholesale switch given the work involved in doing so. I wouldn't be surprised if depth of tree was an impacting perf, given how git stores directories, with trees, tree children and refs. I would think that 1000 files in the top level dir perform better than a 1000 files in 20 dirs depth. Axel ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Embracing git usage for Firefox/Gecko development?
I used to also really dislike mercurial, but the more I used it the more I started to realize its power. Gps summarized it much better than I ever could have in a blog post last month: http://gregoryszorc.com/blog/2013/05/12/thoughts-on-mercurial-%28and-git%29/ I think mercurial and git are both great, but maybe the easier way to solve this problem is through better mercurial education. Andrew On 05/30/2013 08:56 PM, Johnny Stenback wrote: [TL;DR, I think we need to embrace git in addition to hg for Firefox/Gecko hacking, what do you think?] Hello everyone, The question of whether Mozilla engineering should embrace git usage for Firefox/Gecko development has come up a number of times already in various contexts, and I think it's time to have a serious discussion about this. To me, this question has already been answered. Git is already a reality at Mozilla: 1. Git is in use exclusively for some of our significant projects (B2G, Gaia, Rust, Servo, etc) 2. Lots of Gecko hackers use git for their work on mozilla-central, through various conversions from hg to git. What we're really talking about is whether we should embrace git for Firefox/Gecko development in mozilla-central. IMO, the benefits for embracing git are: * Simplified on-boarding, most of our newcomers come to us knowing git (thanks to Github etc), few know hg. * We already mirror hg to git (in more ways than one), and git is already a necessary part of most of our lives. Having one true git repository would simplify developers' lives. * Developers can use git branches. They just work, and they're a good alternative to patch queues. * Developers can benefit from the better merge algorithms used by git. * Easier collaboration through shared branches. * We could have full history in git, including all of hg and CVS history since 1998! * Git works well with Github, even though we're not switching to Github as the ultimate source of truth (more on that below). Some of the known issues with embracing git are: * Performance of git on windows is sub-optimal (we're already working on it). * Infrastructure changes needed... So in other words, I think there's significant value in embracing git and I think we should make it easier to hack on Gecko/Firefox with git. I see two ways to do that: 1: Embrace both git and hg as a first class DVCS. 2: Switch wholesale to git. Option 1 is where I personally think it's worth investing effort. It means we'd need to set up an atomic bidirectional bridge between hg and git (which I'm told is doable, and there are even commercial solutions for this out there that may solve this for us). Assuming we solve the bridge problem one way or another, it would give us all the benefits listed above, plus developer tool choice, and we could roll this out incrementally w/o the need to change all of our infrastructure at once. I.e. our roll out could look something like this: 1. create a read only, official mozilla-central git mirror 2. add support for pushing to try with git and see the results in tbpl 3. update tbpl to show git revisions in addition to hg revisions 4. move to project branches, then inbound, then m-c, release branches, etc While doing all this, things like build infrastructure and l10n would be largely, if not completely, unaffected. Lots of details TBD there, but the point is we'd have a lot of flexibility in how we approach this while the amount of effort required before our git mirror is functional will be minimal compared to doing a wholesale switch as described below. We would of course need to run high availability servers for both hg and git, and eventually the atomic bidirectional bridge (all of which would likely be on the same hardware). Option 2 is where this discussion started (in the Tuesday meeting a few weeks ago, https://wiki.mozilla.org/Platform/2013-05-07#Should_we_switch_from_hg_to_git.3F). Since then I've had a number of conversations and have been convinced that a wholesale change is the less attractive option. The cost of a wholesale change will be *huge* on the infrastructure end, to a point where we need to question whether the benefits are worth the cost. I have also spoken with other large engineering orgs about git performance limitations, one of which is doing the opposite switch, going from git to hg. While I don't see us hitting those limitations any time soon, I also don't think the risk of hitting those limitations is one we want to take in a wholesale change at this point. One inevitable question that will arise here if we were to switch wholesale over to git is whether we're also considering hosting Firefox/Gecko development on Github, and the answer to that question at this point is no (but we will likely continue to mirror mozilla-central etc to Github). We've been in talks with Github, but we will not get the reliability guarantees we need nor the flexibility we need if we were to
Re: Embracing git usage for Firefox/Gecko development?
On 5/30/2013 7:56 PM, Johnny Stenback wrote: [TL;DR, I think we need to embrace git in addition to hg for Firefox/Gecko hacking, what do you think?] Personally, the more I use git, the more I hate it. But I also see that other people love it... jcramner++ :-) * Developers can use git branches. They just work, and they're a good alternative to patch queues. I can't see how they are a good alternative. With patch queues, I can maintain a complex refactoring in a patch queue containing dozens of smallish patches. In particular, I can easily realize I made a mistake in patch 3 while working on patch 21 and make sure that the fix ends up in patch 3; I don't see how that is easily achievable in git branches. I'm still totally at sea regarding the git development model - hg queues make a lot of sense to me. hg qqueue is very handy and I think covers a lot of what people think they need git for. I'm sure I'm missing things, but I keep accidentally adding files in git patches. * Git works well with Github, even though we're not switching to Github as the ultimate source of truth (more on that below). Outside of the community size, I personally can't see any benefits to Github for Mozilla; if all you want is community outreach, you pretty much just need a read-only git mirror on Github, which we already have. I honestly don't see any serious advantages other than some projects are using it anyways. -- Randell Jesup, Mozilla Corp remove .news for personal email ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Embracing git usage for Firefox/Gecko development?
Weirdly enough, I'm hoping we're using one or the other, and I think git is more promising. Yes, I need to rewrite a bunch of stuff l10n-wise, but still. I actually think that we should aim high. Don't bother about command lines, but what takes us to a system where people can just contribute to Firefox and Gecko on the web. Find the bug, click a button, start editing in your browser, try, review, merge. If we can get that working without involving a single thing but a browser, than we're making a change. git command line vs hg command line doesn't bring that change. The best model people have come up so far is a fork per bug (LegNeato at the time) or per user (github). I see how someone can serve a few 100k of forks (see our bugcount) on git storage, but I don't see that with hg. That to me is the compelling argument. Doing both hg and git sounds like we'll get the worst of both worlds. I'm also advocating for taking hosting dead serious. We're at least struggling with the amount of repos we're serving on hg right now, adding more complexity won't make the systems more stable. Also, Vicent (githubber) has a great talk about the mistakes they did based on git, http://vimeo.com/64716825 (30 mins). They're well past the approaches we're currently at on the hg side. If we don't have to be compatible with hg, we can also rethink the constraints that's putting on us for merge days, etc. I know that option 2 isn't a quick path, and I love the beauty of hg, but we've failed to use that beauty to make the next game changing infrastructure to support contributors, IMHO. I can see us having a better chance with git in the backend. Quick notes on hackability: In terms of stable and reliable hacking, hg and git are on par. Shell out to the command line tools, parse the output. hg being mostly in python is nice for python hacking, but the code paths you're hooking in to are far from stable. I do that extensively, and quite a few ports have been painful. Scripting hg outside of python, well, yes. git has libgit2 now, which is a very basic C impl, and jgit, a java implementation of git. Bindings for libgit2 exist in many languages, but only the ruby and C# are really good. In particular the python binding is far from being pythonic, and from being complete. If it's the right base to create a pythonic api is TBD. Regarding bugzilla integration, there are perl bindings that get modifications. I refuse to know perl good enough to make any statement on the value of the perl bindings, though. Axel On 5/31/13 2:56 AM, Johnny Stenback wrote: [TL;DR, I think we need to embrace git in addition to hg for Firefox/Gecko hacking, what do you think?] Hello everyone, The question of whether Mozilla engineering should embrace git usage for Firefox/Gecko development has come up a number of times already in various contexts, and I think it's time to have a serious discussion about this. To me, this question has already been answered. Git is already a reality at Mozilla: 1. Git is in use exclusively for some of our significant projects (B2G, Gaia, Rust, Servo, etc) 2. Lots of Gecko hackers use git for their work on mozilla-central, through various conversions from hg to git. What we're really talking about is whether we should embrace git for Firefox/Gecko development in mozilla-central. IMO, the benefits for embracing git are: * Simplified on-boarding, most of our newcomers come to us knowing git (thanks to Github etc), few know hg. * We already mirror hg to git (in more ways than one), and git is already a necessary part of most of our lives. Having one true git repository would simplify developers' lives. * Developers can use git branches. They just work, and they're a good alternative to patch queues. * Developers can benefit from the better merge algorithms used by git. * Easier collaboration through shared branches. * We could have full history in git, including all of hg and CVS history since 1998! * Git works well with Github, even though we're not switching to Github as the ultimate source of truth (more on that below). Some of the known issues with embracing git are: * Performance of git on windows is sub-optimal (we're already working on it). * Infrastructure changes needed... So in other words, I think there's significant value in embracing git and I think we should make it easier to hack on Gecko/Firefox with git. I see two ways to do that: 1: Embrace both git and hg as a first class DVCS. 2: Switch wholesale to git. Option 1 is where I personally think it's worth investing effort. It means we'd need to set up an atomic bidirectional bridge between hg and git (which I'm told is doable, and there are even commercial solutions for this out there that may solve this for us). Assuming we solve the bridge problem one way or another, it would give us all the benefits listed above, plus developer
Re: Embracing git usage for Firefox/Gecko development?
On 5/31/13 3:20 PM, Matt Brubeck wrote: blame mobile/android/chrome/content/browser.xul: git 1.015s hg 0.830s Was this a git blame -C (which would be more similar to hg blame), or just a git blame? -Boris ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Embracing git usage for Firefox/Gecko development?
On 5/31/2013 12:32 PM, Boris Zbarsky wrote: On 5/31/13 3:20 PM, Matt Brubeck wrote: blame mobile/android/chrome/content/browser.xul: git 1.015s hg 0.830s Was this a git blame -C (which would be more similar to hg blame), or just a git blame? Good catch. (Sorry, I missed your messages on IRC warning me about this.) The above numbers were without -C. git blame -C takes about 3.7 seconds on this file. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Embracing git usage for Firefox/Gecko development?
I also agree to Randell and Joshua. I've been using both lately and there are just a few things missing in git that I am used to in hg. Mercurial Queues is the most prominent. I am used to switching the order of patches in my queue, which seems like a pain to me in git. Or maybe I haven't quite found out how to do it reliably. Yes, somehow rebasing seems to be the solution here, but with git its quite common that you push your changes to your own remote and then do pull requests. This again will require me to do push -f almost always, since I often change the order of patches. This doesn't sound ideal to me. Philipp ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Embracing git usage for Firefox/Gecko development?
Johnny Stenback j...@mozilla.com wrote: [TL;DR, I think we need to embrace git in addition to hg for Firefox/Gecko hacking, what do you think?] 1: Embrace both git and hg as a first class DVCS. 2: Switch wholesale to git. For what it's worth, as someone who is happy to use whatever is thrown at me (I used hg until I worked on B2G for a while and switched to git for a while) I think option 1 is the way to go here. This question has already sucked up way too much time and, assuming the bridge between hg and git works as advertised, letting people simply use their preferred VCS seems like the easiest way to move forward. -- Blake Kaplan ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Embracing git usage for Firefox/Gecko development?
On Wed, Jun 5, 2013 at 3:25 AM, Philipp Kewisch mozi...@kewis.ch wrote: I also agree to Randell and Joshua. I've been using both lately and there are just a few things missing in git that I am used to in hg. Mercurial Queues is the most prominent. I am used to switching the order of patches in my queue, which seems like a pain to me in git. Or maybe I haven't quite found out how to do it reliably. git rebase --interactive. It is /far/ more powerful than mq for this use-case. I even have a |git qrebase| alias for this. https://github.com/jlebar/moz-git-tools Yes, somehow rebasing seems to be the solution here, but with git its quite common that you push your changes to your own remote and then do pull requests. This again will require me to do push -f almost always, since I often change the order of patches. This doesn't sound ideal to me. I don't understand the objection here. Yes, if you're changing remote history (e.g. updating a pull request after you re-ordered patches), you have to push -f. What's the problem with this? ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Embracing git usage for Firefox/Gecko development?
I can't see how they are a good alternative. With patch queues, I can maintain a complex refactoring in a patch queue containing dozens of smallish patches. In particular, I can easily realize I made a mistake in patch 3 while working on patch 21 and make sure that the fix ends up in patch 3; I don't see how that is easily achievable in git branches. This is far easier to achieve in git than with mq. I this to a room of experienced hg developers a few days ago and got a loud chorus of wows. But git isn't easy to use, and the fact that you think git can't do this is certainly evidence for that. * Developers can benefit from the better merge algorithms used by git. I don't know what you mean by that. Note that git's octopus merges are not directly realizable in hg, so an hg-git bidirectional mirror may need to ban such things in the incoming git. An octopus merge is not a merge algorithm. The merge algorithm is the thing that decides /how/ to merge two (or N) commits. Mercurial also treats history as immutable, so you can't push history revisions. History in git is no more or less immutable than in hg. Every time you hg qpop or qref, you're mutating history. On Thu, May 30, 2013 at 11:04 PM, Joshua Cranmer pidgeo...@gmail.com wrote: On 5/30/2013 7:56 PM, Johnny Stenback wrote: [TL;DR, I think we need to embrace git in addition to hg for Firefox/Gecko hacking, what do you think?] Personally, the more I use git, the more I hate it. But I also see that other people love it... * Developers can use git branches. They just work, and they're a good alternative to patch queues. I can't see how they are a good alternative. With patch queues, I can maintain a complex refactoring in a patch queue containing dozens of smallish patches. In particular, I can easily realize I made a mistake in patch 3 while working on patch 21 and make sure that the fix ends up in patch 3; I don't see how that is easily achievable in git branches. * Developers can benefit from the better merge algorithms used by git. I don't know what you mean by that. Note that git's octopus merges are not directly realizable in hg, so an hg-git bidirectional mirror may need to ban such things in the incoming git. * Git works well with Github, even though we're not switching to Github as the ultimate source of truth (more on that below). Outside of the community size, I personally can't see any benefits to Github for Mozilla; if all you want is community outreach, you pretty much just need a read-only git mirror on Github, which we already have. Some of the known issues with embracing git are: * Performance of git on windows is sub-optimal (we're already working on it). * Infrastructure changes needed... * Retraining Mercurial developers to use git (if we went git-only) * Porting all of our Mercurial customizations to git for a check-in head * Updating all of our developer build documentation * Adapting build processes to adjust to the git way of doing things Option 1 is where I personally think it's worth investing effort. It means we'd need to set up an atomic bidirectional bridge between hg and git (which I'm told is doable, and there are even commercial solutions for this out there that may solve this for us). This is harder than it sounds: hg and git are subtly different in ways that can seriously frustrate users if you don't pay attention. For example (one that bit me very hard): hg records file copy/moves as part of the changeset, while git attempts to reverse engineer them when it needs to do so. So my attempt to merge my changes with a remote git repository via hg-git failed miserably since the hg local repo didn't catch the git moved repo. Mercurial also treats history as immutable, so you can't push history revisions. If all you need is a read-only mirror, these kind of changes aren't as important (you're not going to damage official records, just inconvenience some developers); for a two-way mirror, they are very much worth considering. -- Joshua Cranmer Thunderbird and DXR developer Source code archæologist ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Embracing git usage for Firefox/Gecko development?
On Fri, May 31, 2013 at 4:07 PM, Matt Brubeck mbrub...@mozilla.com wrote: On 5/31/2013 12:32 PM, Boris Zbarsky wrote: On 5/31/13 3:20 PM, Matt Brubeck wrote: blame mobile/android/chrome/content/browser.xul: git 1.015s hg 0.830s Was this a git blame -C (which would be more similar to hg blame), or just a git blame? The git repository contains CVS history, so is often slower than hg blame. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Embracing git usage for Firefox/Gecko development?
On 7/10/2013 5:01 PM, Justin Lebar wrote: History in git is no more or less immutable than in hg. Every time you hg qpop or qref, you're mutating history. However, newer versions of hg make it much harder to mutate official history via the phases work. In effect, only the stuff in development is actually mutable. Also, hg has been working on http://mercurial.selenic.com/wiki/ChangesetEvolution, which is being billed as a much safer way to do rebasing than fully mutable history. -- Joshua Cranmer Thunderbird and DXR developer Source code archæologist ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Embracing git usage for Firefox/Gecko development?
On 7/10/13 3:01 PM, Justin Lebar wrote: I can't see how they are a good alternative. With patch queues, I can maintain a complex refactoring in a patch queue containing dozens of smallish patches. In particular, I can easily realize I made a mistake in patch 3 while working on patch 21 and make sure that the fix ends up in patch 3; I don't see how that is easily achievable in git branches. This is far easier to achieve in git than with mq. Can you describe your patch queue-like workflow using git? git rebase -i lets you reorder commits, but how do you quickly do the equivalent of hg qpopping and qpushing a bunch of commits? I've investigated git tools like StGit and and TopGit, but they are pretty heavyweight and get confused if you tweak your git branches without using their porcelain scripts. chris ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Embracing git usage for Firefox/Gecko development?
For me, the most appealing reason to switch to git is so that we can get back to having a single revision control system, but I think the benefit of that is probably outweighed by the churn. We'd need to rewrite a lot of stuff simultaneously, more or less. As for supporting both, I have a hard time forming an opinion on it without having a clue what the cost would be. Clearly, it would involve a lot of effort to support git's idiosyncrasies and failure modes (even if git ends up being better overall for our needs than hg.) What's the opportunity cost? What would get stalled, postponed, or scrapped? And what status quo-ish alternative are we talking about, given that we have to gradually support git in more and more places due to existing usage and external dependencies? In short, it's hard to have an opinion about a decision when it's unclear what the decision means. Is it that git becomes tier 1 and anything you cannot do with git that you can do with hg automatically becomes a blocker? Or is it just a request please prioritize things that will improve git support a little higher than you have in the past? You've laid out some concrete milestones in your original email, but I don't know how those milestones would be prioritized relative to the one or two other little things that releng and IT are working on already. On a different gear, would we want to keep m-c/m-i/projects etc. as separate git repos, or could we make them branches in a merged repo? Separate repos feel like a mercurial-induced nuisance to me. (I'd prefer aurora/beta/release to be in the same repo too, not that I truly understand the repercussions.) Also, note that a historical weakness of our hg usage has been onboarding education. Adding git as an alternative would improve this for people already accustomed to git, but would make the situation worse for anyone who isn't already comfortable with the more advanced usage of either git or mercurial. (I assert that nontrivial git is harder to learn, and we're already not doing a good job with nontrivial hg education.) It's somewhat tangential, but it would be cool if somebody could wave a magic wand and address education at the same time -- if only to prevent a bad situation from becoming worse. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Embracing git usage for Firefox/Gecko development?
On Thu, May 30, 2013 at 05:56:43PM -0700, Johnny Stenback wrote: [TL;DR, I think we need to embrace git in addition to hg for Firefox/Gecko hacking, what do you think?] Hello everyone, The question of whether Mozilla engineering should embrace git usage for Firefox/Gecko development has come up a number of times already in various contexts, and I think it's time to have a serious discussion about this. To me, this question has already been answered. Git is already a reality at Mozilla: 1. Git is in use exclusively for some of our significant projects (B2G, Gaia, Rust, Servo, etc) 2. Lots of Gecko hackers use git for their work on mozilla-central, through various conversions from hg to git. What we're really talking about is whether we should embrace git for Firefox/Gecko development in mozilla-central. IMO, the benefits for embracing git are: * Simplified on-boarding, most of our newcomers come to us knowing git (thanks to Github etc), few know hg. * We already mirror hg to git (in more ways than one), and git is already a necessary part of most of our lives. Having one true git repository would simplify developers' lives. * Developers can use git branches. They just work, and they're a good alternative to patch queues. * Developers can benefit from the better merge algorithms used by git. * Easier collaboration through shared branches. * We could have full history in git, including all of hg and CVS history since 1998! * Git works well with Github, even though we're not switching to Github as the ultimate source of truth (more on that below). Some of the known issues with embracing git are: * Performance of git on windows is sub-optimal (we're already working on it). * Infrastructure changes needed... So in other words, I think there's significant value in embracing git and I think we should make it easier to hack on Gecko/Firefox with git. I see two ways to do that: 1: Embrace both git and hg as a first class DVCS. 2: Switch wholesale to git. Option 1 is where I personally think it's worth investing effort. It means we'd need to set up an atomic bidirectional bridge between hg and git (which I'm told is doable, and there are even commercial solutions for this out there that may solve this for us). Assuming we solve the bridge problem one way or another, it would give us all the benefits listed above, plus developer tool choice, and we could roll this out incrementally w/o the need to change all of our infrastructure at once. I.e. our roll out could look something like this: 1. create a read only, official mozilla-central git mirror 2. add support for pushing to try with git and see the results in tbpl 3. update tbpl to show git revisions in addition to hg revisions 4. move to project branches, then inbound, then m-c, release branches, etc Another way to look at this would be to make the git repository the real central source, and keep the mercurial branches as clones of it, with hg-git (and hg-git supports pushing to git, too). This would likely make it easier to support pushing to both, although we'd need to ensure nobody pushes octopus merges in the git repo. Option 2 is where this discussion started (in the Tuesday meeting a few weeks ago, https://wiki.mozilla.org/Platform/2013-05-07#Should_we_switch_from_hg_to_git.3F). Since then I've had a number of conversations and have been convinced that a wholesale change is the less attractive option. The cost of a wholesale change will be *huge* on the infrastructure end, to a point where we need to question whether the benefits are worth the cost. I have also spoken with other large engineering orgs about git performance limitations, one of which is doing the opposite switch, going from git to hg. I bet this is facebook. Their usecase includes millions of changesets with millions of files (iirc, according to posts i've seen on the git list). Mike ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Embracing git usage for Firefox/Gecko development?
Thanks for starting this conversation, jst. As an advocate of git in general, I actually think option #1 (maintaining both git and hg as first class citizens) makes a lot more sense for us. The benefits of git in my opinion outweigh hg, but not to a degree that would warrant paying the cost of switching all of our infra to use git directly. Especially since that I believe a two-way mirroring which lets people push from either hg to git is extremely easy (the rest is a solved problem https://github.com/mozilla/mozilla-central.) Nicolas (nbp) built https://github.com/nbp/git-hg-bridge a while ago, and while the existing implementation is not good enough for a scale of multiple developers using it, I have had plans to fix that (unfortunately I never gotten around to document that...) I would be happy to share more details if needed. With this tool finalized, we can push to hg through git (in an entirely transparent manner). And we already have a read-only mirror which people have been using for quite a while. These two will let people who prefer git to live and breathe git, and for others to live and breathe hg, and will additionally let us to build other tools on top of git where it makes sense (gerrit, anyone?!), and is doable with a very low initial cost compared to the switch the world approach. Thus, I'd like to advocate for the first option. Cheers, Ehsan ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
