Re: [PATCH v2 00/18] remote-bzr: massive changes
On Wed, 01 May 2013 11:38:47 -0700 Junio C Hamano wrote: JCH> Felipe Contreras writes: >> On Wed, May 1, 2013 at 11:39 AM, Junio C Hamano wrote: >>> Felipe Contreras writes: >>> > So let's go ahead and apply these directly on top of 'master', once > we hear from Emacs folks and they are happy with it. I'll queue it > on 'pu' so that I do not have to go back to the list archive when it > happens. I already heard that everything seems to be working correctly, except one feature, the biggest change, which I screwed up with a one-liner commit. That's why I added a test. Anyway, I've fixed it in my github branch and in this patch series, and I've told them to try the fix. >>> >>> Let us know when they make progress on that front. >>> >>> If Emacs decides to switch to Git and decides to use this version of >>> remote-bzr for their conversion, or at least a nontrivial group of >>> developers favor to do so, without seeing concrete technical points >>> that say remote-bzr is not yet ready (e.g. "the conversion is still >>> wrong and X, Y and Z needs to be fixed"), that would be a very >>> welcome solid vote of confidence in favor of us going ahead with >>> this. >> >> Seems unlikely for political reasons (isn't it always for GNU?), since >> RMS is heavily involved in the decision. JCH> I am very aware of that discussion (and the original one when they JCH> decided to use bzr). That is exactly why I said "at least ... favor JCH> to do so". FYI, in case you're not aware, there's a pretty strong feeling on emacs-devel that the switch to Git will happen and RMS is not opposed. I don't know if they'll use remote-bzr, though. It's more likely they'll use one of the already-existing mirrors and sync it up, based on the feedback so far. It's a good time to bring remote-bzr up on emacs-devel if you want it to be considered. HTH Ted -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 00/18] remote-bzr: massive changes
Felipe Contreras writes: > On Wed, May 1, 2013 at 11:39 AM, Junio C Hamano wrote: >> Felipe Contreras writes: >> So let's go ahead and apply these directly on top of 'master', once we hear from Emacs folks and they are happy with it. I'll queue it on 'pu' so that I do not have to go back to the list archive when it happens. >>> >>> I already heard that everything seems to be working correctly, except >>> one feature, the biggest change, which I screwed up with a one-liner >>> commit. That's why I added a test. Anyway, I've fixed it in my github >>> branch and in this patch series, and I've told them to try the fix. >> >> Let us know when they make progress on that front. >> >> If Emacs decides to switch to Git and decides to use this version of >> remote-bzr for their conversion, or at least a nontrivial group of >> developers favor to do so, without seeing concrete technical points >> that say remote-bzr is not yet ready (e.g. "the conversion is still >> wrong and X, Y and Z needs to be fixed"), that would be a very >> welcome solid vote of confidence in favor of us going ahead with >> this. > > Seems unlikely for political reasons (isn't it always for GNU?), since > RMS is heavily involved in the decision. I am very aware of that discussion (and the original one when they decided to use bzr). That is exactly why I said "at least ... favor to do so". -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 00/18] remote-bzr: massive changes
On Wed, May 1, 2013 at 11:39 AM, Junio C Hamano wrote: > Felipe Contreras writes: > >>> So let's go ahead and apply these directly on top of 'master', once >>> we hear from Emacs folks and they are happy with it. I'll queue it >>> on 'pu' so that I do not have to go back to the list archive when it >>> happens. >> >> I already heard that everything seems to be working correctly, except >> one feature, the biggest change, which I screwed up with a one-liner >> commit. That's why I added a test. Anyway, I've fixed it in my github >> branch and in this patch series, and I've told them to try the fix. > > Let us know when they make progress on that front. > > If Emacs decides to switch to Git and decides to use this version of > remote-bzr for their conversion, or at least a nontrivial group of > developers favor to do so, without seeing concrete technical points > that say remote-bzr is not yet ready (e.g. "the conversion is still > wrong and X, Y and Z needs to be fixed"), that would be a very > welcome solid vote of confidence in favor of us going ahead with > this. Seems unlikely for political reasons (isn't it always for GNU?), since RMS is heavily involved in the decision. -- Felipe Contreras -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 00/18] remote-bzr: massive changes
Felipe Contreras writes: >> So let's go ahead and apply these directly on top of 'master', once >> we hear from Emacs folks and they are happy with it. I'll queue it >> on 'pu' so that I do not have to go back to the list archive when it >> happens. > > I already heard that everything seems to be working correctly, except > one feature, the biggest change, which I screwed up with a one-liner > commit. That's why I added a test. Anyway, I've fixed it in my github > branch and in this patch series, and I've told them to try the fix. Let us know when they make progress on that front. If Emacs decides to switch to Git and decides to use this version of remote-bzr for their conversion, or at least a nontrivial group of developers favor to do so, without seeing concrete technical points that say remote-bzr is not yet ready (e.g. "the conversion is still wrong and X, Y and Z needs to be fixed"), that would be a very welcome solid vote of confidence in favor of us going ahead with this. Thanks. -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 00/18] remote-bzr: massive changes
On Wed, May 1, 2013 at 12:44 AM, Junio C Hamano wrote: > Felipe Contreras writes: > >> After being contacted by the emacs developers and others who are stuck with >> Bazaar, which at this point seems to be utterly abandoned, I realized the >> current implementation is too crude. >> ... >> That is of course, if pushing actually worked (which in many cases doesn't). >> >> In short, our support for real-world projects suck. >> >> These patches fix all the issues I encountrered. >> ... >> Finally, after all these changes I was finally able to clone the whole emacs >> repository, all 130685 commits, and 56 branches without running out of memory >> in my modest laptop. > > Yay ;-) > > I assume that the trees at a handful of key points (e.g. releases) > were verified to be identical with the original history and the > conversion result. Not really. I don't think the users are that interested in the history being identical at this point, merely that they can use it as a proxy to interact with bazaar repositories. People have found discrepancies, so I assume they have compared at least the tip of the branches, and found them. This probably means that the history is correct, since bazaar deals with changesets (git is one of the few DSCMs that don't). Also, there's further news on this, pushes seem to work correctly to the emacs' repo[1]. >> I'll ask the emacs developers to give them a try, and let's see >> how it goes. > > Yeah, that's the least we can do for both existing and future users. > > Generally speaking, post -rc0 is too late for "if no issues are > found", simply because no existing user has enough time to find > corner case regressions in her work using the new software (I do > not expect a trivial bug that can be uncovered in a few weeks of use > would remain in a version that has successfully converted the Emacs > history; but real world users always have different needs than what > we anticipate). > > I however am finding myself moderately receptive to this series. > That is primarily because this series touches only two files that > are totally isolated from the rest of the system. Even if they did > not work at all, there is no risk for the remainder of Git. Nobody > other than existing users of remote-bzr will even notice if we > merged this by the final. > > For existing users of remote-bzr that we shipped in 1.8.2, the story > is a bit different, though. If this series makes things worse in a > way your tests did not reveal, and if such a regression is not > reported and/or cannot be fixed by 1.8.3 final, that will mean a > real regression in the released version for them. > > If that ever happens, that would be the time for us to regret the > hasty decision to merge remote-bzr in 1.8.2, justifying that with a > "There wasn't anything working for interoperating with bzr, and here > is one to do so; anything is better than nothing", and learn from > that mistake (it is not an option to say "the 1.8.2 users chose to > use contrib/ material that are clearly marked as sub-par quality > with their own risk". If we did not ship it in 1.8.2, they did not > have to get burned with any regression and could have kept working > with bzr a bit longer. "Anything" is not necessarily better than > "nothing"). Fortunately there seem to be at least some users that find what is in 1.8.2 working to some extent, not in all the repositories, and not all the features, but at least something, which is much better than the alternatives, even the best one has been blocked for years, even when a solution is known[2]. > Hopefully, such a regression will not have to happen (for one thing, > I would expect that the existing 1.8.2 remote-bzr user base would be > very small). Also I somehow have a feeling that it is very unlikely > to happen, especially given your report: > > (1) the series converts Emacs history without barfing; and > > (2) you have some confidence in the conversion result after > inspecting at least a handful of key release points and trees > and metainformation match between the original and the > converted history. > > So let's go ahead and apply these directly on top of 'master', once > we hear from Emacs folks and they are happy with it. I'll queue it > on 'pu' so that I do not have to go back to the list archive when it > happens. I already heard that everything seems to be working correctly, except one feature, the biggest change, which I screwed up with a one-liner commit. That's why I added a test. Anyway, I've fixed it in my github branch and in this patch series, and I've told them to try the fix. Let's see. Cheers. [1] http://bzr.savannah.gnu.org/lh/emacs/xwidget/revision/101292 [2] https://bugs.launchpad.net/bzr/+bug/541626 -- Felipe Contreras -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 00/18] remote-bzr: massive changes
Felipe Contreras writes: > After being contacted by the emacs developers and others who are stuck with > Bazaar, which at this point seems to be utterly abandoned, I realized the > current implementation is too crude. > ... > That is of course, if pushing actually worked (which in many cases doesn't). > > In short, our support for real-world projects suck. > > These patches fix all the issues I encountrered. > ... > Finally, after all these changes I was finally able to clone the whole emacs > repository, all 130685 commits, and 56 branches without running out of memory > in my modest laptop. Yay ;-) I assume that the trees at a handful of key points (e.g. releases) were verified to be identical with the original history and the conversion result. > Since the purpose of remote-bzr is to actually be usable for the > poor souls stucks in DSCMs that are not git, these changes are a > must. I propose they be merged for the next major version of git > (v1.8.3) if no issues are found. They changes pass all the tests, > and work on various repositories I've tried. Nice. > I'll ask the emacs developers to give them a try, and let's see > how it goes. Yeah, that's the least we can do for both existing and future users. Generally speaking, post -rc0 is too late for "if no issues are found", simply because no existing user has enough time to find corner case regressions in her work using the new software (I do not expect a trivial bug that can be uncovered in a few weeks of use would remain in a version that has successfully converted the Emacs history; but real world users always have different needs than what we anticipate). I however am finding myself moderately receptive to this series. That is primarily because this series touches only two files that are totally isolated from the rest of the system. Even if they did not work at all, there is no risk for the remainder of Git. Nobody other than existing users of remote-bzr will even notice if we merged this by the final. For existing users of remote-bzr that we shipped in 1.8.2, the story is a bit different, though. If this series makes things worse in a way your tests did not reveal, and if such a regression is not reported and/or cannot be fixed by 1.8.3 final, that will mean a real regression in the released version for them. If that ever happens, that would be the time for us to regret the hasty decision to merge remote-bzr in 1.8.2, justifying that with a "There wasn't anything working for interoperating with bzr, and here is one to do so; anything is better than nothing", and learn from that mistake (it is not an option to say "the 1.8.2 users chose to use contrib/ material that are clearly marked as sub-par quality with their own risk". If we did not ship it in 1.8.2, they did not have to get burned with any regression and could have kept working with bzr a bit longer. "Anything" is not necessarily better than "nothing"). Hopefully, such a regression will not have to happen (for one thing, I would expect that the existing 1.8.2 remote-bzr user base would be very small). Also I somehow have a feeling that it is very unlikely to happen, especially given your report: (1) the series converts Emacs history without barfing; and (2) you have some confidence in the conversion result after inspecting at least a handful of key release points and trees and metainformation match between the original and the converted history. So let's go ahead and apply these directly on top of 'master', once we hear from Emacs folks and they are happy with it. I'll queue it on 'pu' so that I do not have to go back to the list archive when it happens. Thanks. -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 00/18] remote-bzr: massive changes
Hi, This is the same as last series, except that I dropped a couple of conflicting patches, and added one test. After being contacted by the emacs developers and others who are stuck with Bazaar, which at this point seems to be utterly abandoned, I realized the current implementation is too crude. Bazaar branches are very simplistic, and our support for them is the same; we need to create one remote per branch. This works nicely if you work on small projects with few branches, but doesn't scale. Big projects like emacs take a lot of space, and creating a remote per branch is unrealistic, because each remote will have the whole Bazaar repository copied, wasting space, and time each time a remote (for a bzr branch) is set up. Moreoever, a developer needs to constantly reset the master branch to the commit he/she wants to push before pushing, since the transport-helper infraestructure doesn't support pushing with refspecs (xwidget:master). That is of course, if pushing actually worked (which in many cases doesn't). In short, our support for real-world projects suck. These patches fix all the issues I encountrered. 1) First of all, there are several improvements for pushing. Before, we failed when trying to push a merge, now, even tricky merges work. 2) Secondly, bzr branches are tied to a transport, so they time out if not used for a period of time, and importing/exporting huge chunks of a repository do take some time. So now they are only opened when they are about to be used. 3) Then the big one; now bzr repositories are supported. They are very simple: basically an object store with no notion of branches, so to find the branches we need to traverse a directory (sometimes) remote, to find them. This is how Bazaar does it, ableit very slowly. Naturally, a lot of code had to be changed to support more than one branch. 4) In addition, now remotes share all the objects, so adding a new remote doesn't imply fetchng a bunch of duplicated objects. They are just re-used automagically. 5) Since the bzr objects are shared now, it only makes sense to share the remote-bzr marks, so we don't have to fast-import them again. 6) The code was also reorganized to keep referenced as few objects as possible, since Bazaar seems to be need *a ton* of memory for them. Finally, after all these changes I was finally able to clone the whole emacs repository, all 130685 commits, and 56 branches without running out of memory in my modest laptop. Since the purpose of remote-bzr is to actually be usable for the poor souls stucks in DSCMs that are not git, these changes are a must. I propose they be merged for the next major version of git (v1.8.3) if no issues are found. They changes pass all the tests, and work on various repositories I've tried. I'll ask the emacs developers to give them a try, and let's see how it goes. Felipe Contreras (18): remote-bzr: cleanup CustomTree remote-bzr: delay blob fetching until the very end remote-bzr: fix order of locking in CustomTree remote-bzr: always try to update the worktree remote-bzr: add support to push merges remote-bzr: fixes for branch diverge remote-bzr: fix partially pushed merge remote-bzr: use branch variable when appropriate remote-bzr: add support for bzr repos remote-bzr: fix branch names remote-bzr: add support for shared repo remote-bzr: improve author sanitazion remote-bzr: add custom method to find branches remote-bzr: add option to specify branches remote-bzr: improve progress reporting remote-bzr: iterate revisions properly remote-bzr: delay peer branch usage remote-bzr: access branches only when needed contrib/remote-helpers/git-remote-bzr | 305 -- contrib/remote-helpers/test-bzr.sh| 72 2 files changed, 293 insertions(+), 84 deletions(-) -- 1.8.3.rc0.399.gc96a135 -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
