Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Sat, Jun 2, 2012 at 8:03 AM, Dirkjan Ochtman wrote: > On Sat, Jun 2, 2012 at 12:59 PM, Rich Freeman wrote: >> It appears that devs will have to add the remote for the live >> repository after they've cloned the bundle - otherwise they'll just >> keep pulling from the bundle which isn't all that convenient. > > I think you still misunderstand. As I understand Robin, we wouldn't > even offer up a clone of the full-history bundle, it would only be > offered as a normal download. The default workflow is cloning from the > shallow version, which will obviously give you the desired remote. I wasn't talking about full-history. I was talking about the fact that we're distributing a bundle. If you clone a bundle, you won't have a remote for the live repository. You would need to add it, unless you plan to never push a commit back to the gentoo repository, and you plan to manually download bundles anytime you want to update your local repository. I'm not sure how exactly Robin was planning on making the full history available, but it sounded like it would also be distributed as a bundle. That means that you can certainly clone it - just type git clone path-to-locally-saved-bundle-file . If it is in some other format like a pack file then you would import it into a repository via a different command. Rich
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Sat, Jun 2, 2012 at 12:59 PM, Rich Freeman wrote: > Looks useful. Wasn't aware that a bundle was something other than a tarball. > > We'll probably need to spell out the preferred process in the docs, > and reference it frequently in communications. Otherwise you'll get > quite a few clones instead. > > It appears that devs will have to add the remote for the live > repository after they've cloned the bundle - otherwise they'll just > keep pulling from the bundle which isn't all that convenient. I think you still misunderstand. As I understand Robin, we wouldn't even offer up a clone of the full-history bundle, it would only be offered as a normal download. The default workflow is cloning from the shallow version, which will obviously give you the desired remote. Cheers, Dirkjan
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Sat, Jun 2, 2012 at 12:04 AM, Robin H. Johnson wrote: > please look up git-bundle before suggesting things like tarballs of > repos/checkouts. > Looks useful. Wasn't aware that a bundle was something other than a tarball. We'll probably need to spell out the preferred process in the docs, and reference it frequently in communications. Otherwise you'll get quite a few clones instead. It appears that devs will have to add the remote for the live repository after they've cloned the bundle - otherwise they'll just keep pulling from the bundle which isn't all that convenient. Rich
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Fri, Jun 01, 2012 at 08:52:42PM -0400, Rich Freeman wrote: > On Fri, Jun 1, 2012 at 4:49 PM, Michael Weber wrote: > > Cloning the repo [1] takes 200seconds on 8cores (it's 2GB of data and > > 22 minutes of 3.4GHz cpus). > As others have pointed out, probably the best way to bootstrap this is > to offer tarballs of a shallow repository and a full repository. > Perhaps we'd offer the latter as a torrent. The shallow repository > should be light on the CPU too. no. i explicitly said before, there would be two repos: 1. main working repo, starts with ONLY initial commit. 40MB pack. 2. historical. read-only, you graft it to history to use. BOTH of these would have preferred download via git-bundle over HTTP/rsync. historical would NOT be available via clone due to cputime hit. please look up git-bundle before suggesting things like tarballs of repos/checkouts. -- Robin Hugh Johnson Gentoo Linux: Developer, Trustee & Infrastructure Lead E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Fri, Jun 1, 2012 at 4:49 PM, Michael Weber wrote: > > Cloning the repo [1] takes 200seconds on 8cores (it's 2GB of data and > 22 minutes of 3.4GHz cpus). > As others have pointed out, probably the best way to bootstrap this is to offer tarballs of a shallow repository and a full repository. Perhaps we'd offer the latter as a torrent. The shallow repository should be light on the CPU too. This would be a lot easier on the server than having everybody and their uncle doing a full 2GB clone. Devs could then do a pull to get the latest and greatest, and that would only transfer the delta. I imagine our mirror network can handle the bandwidth compared to what we're already doing with distfiles. Worst case we could take a one-time hit and use S3 or whatever to do the distribution (they even support bittorrent to cut down on the bill). Rich
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
> "WH" == William Hubbs writes: WH> My big complaint about merge commits is if you do a git show on WH> a merge commit, you get nothing, With current git and proper merge logs you get useful info. The headers contain the hashes, so you can get the list of commits pulled by that merge. The signed tag log is show. And merge conflicts also are shown. Based on the hashes in the Merge: header, you can use git log to see the individual commits or git diff to see the whole picture at once. Linus’ current tip is a good example: cd .../linux git show 1193755ac632 So, Gentoo shouldn't prohibit merges. Instead, it should demand that all merges be of signed tags. The plan includes signed commits anyway, so signed tags for pulls will be fully supported by any version of git which might be used. -JimC -- James Cloos OpenPGP: 1024D/ED7DAEA6
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 05/31/2012 02:04 PM, Aaron W. Swenson wrote: > The 6 hours it takes to clone the repo. afaik it's 6 hours to transform the whole cvs history into a git repo. Cloning the repo [1] takes 200seconds on 8cores (it's 2GB of data and 22 minutes of 3.4GHz cpus). [1] git clone git://git-exp.overlays.gentoo.org/exp/gentoo-x86.git [2] http://lore.xmw.de/gentoo/gentoo-exp/notes Michael - -- Gentoo Dev http://xmw.de/ -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.17 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iF4EAREIAAYFAk/JKucACgkQknrdDGLu8JAIxAEAhLZ4Tk6rXy1qAUbDDHS4UYiL gGUPj5PKUKSS1YJp6hkA/R9aEqjSNr/8iZ2novNFGjvbJ5CtDkvI+PsvMTUNDoDo =3t+7 -END PGP SIGNATURE-
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Fri, Jun 01, 2012 at 04:33:35PM +, Robin H. Johnson wrote: > On Fri, Jun 01, 2012 at 10:45:48AM -0500, William Hubbs wrote: > > Overlays are completely separate repositories. There is nothing stopping > > an overlay from using git right now even if the main tree isn't using > > git. They just work in their git repositories until they are ready to > > commit something to the main tree, then they move the changes to the > > main tree. > What about overlay repositories that elect to be a branch of the main > tree via git? > > Do we call that forbidden usage? No, it just would mean that they have to know how to rebase their changes on master before they commit. They would clone the main tree to their overlay location, then make a kde branch in their location. That branch would be where they did their work and they would keep master in sync with the main tree (upstream) by running git pull. To merge the overlay changes into the master, a developer would use a git remote to add the kde overlay to his tree, add a kde branch that tracked the remote kde branch, rebase that on current master, merge it into master to create a fast-forward merge then push. I think that probably sounds more complicated than it is, so let me know if you need clarification. William pgpglcTCfr3Mr.pgp Description: PGP signature
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Fri, Jun 1, 2012 at 12:33 PM, Robin H. Johnson wrote: > What about overlay repositories that elect to be a branch of the main > tree via git? > > Do we call that forbidden usage? I think that branches off of the main tree are mainly going to be useful for more eclass/profile/etc-related work. Work on a package or small group of packages will almost always go better in a completely independent overlay. If you try to do that kind of work in a branch then if you create an rsync tree from that branch it will contain all the other packages that you aren't working on, and they'll get stale very quickly. Anybody using that as an overlay will get a bunch of old ebuilds for who-knows-what in their tree. Now, branches in the main tree are going to be really useful for stuff like package moves, new virtuals, eclass api changes, or other messy changes that have big tree-wide impacts. You can stage the change and fix the 300 impacted ebuilds in a branch, get lots of people to test them, and then merge those in with a single transaction, pulling in updates from master all the while. That is more about portage tree maintenance than package maintenance per-se. All that said, having the tree in git is still a big help to proxy maintainers and others even with all these issues. I've worked as an outside contributor to other projects and it is a lot easier with git. I can easily work in my own PM, rebase against their master, and then easily submit a nice clean diff as a patch, even without doing any pushing at all. I don't have to have push rights to anything official to be more efficient in my contributions. Rich
Re: Re: Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Sat, Jun 02, 2012 at 04:56:33AM +1200, Kent Fredric wrote: > > Only in your local history. The push to the central repo would only send > > the commits in the active chain to your branch HEAD. > Any commits that are rebased, and then replicated somewhere after that > rebase, will be stripped of their signatures by the rebase process. I'm saying the old signatures will not be pushed out for verification, only the new signatures will. -- Robin Hugh Johnson Gentoo Linux: Developer, Trustee & Infrastructure Lead E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
Re: Re: Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
> Only in your local history. The push to the central repo would only send > the commits in the active chain to your branch HEAD. Any commits that are rebased, and then replicated somewhere after that rebase, will be stripped of their signatures by the rebase process. -- Kent perl -e "print substr( \"edrgmaM SPA NOcomil.ic\\@tfrken\", \$_ * 3, 3 ) for ( 9,8,0,7,1,6,5,4,3,2 );" http://kent-fredric.fox.geek.nz
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On 2 June 2012 04:33, Robin H. Johnson wrote: > On Fri, Jun 01, 2012 at 10:45:48AM -0500, William Hubbs wrote: >> Overlays are completely separate repositories. There is nothing stopping >> an overlay from using git right now even if the main tree isn't using >> git. They just work in their git repositories until they are ready to >> commit something to the main tree, then they move the changes to the >> main tree. > What about overlay repositories that elect to be a branch of the main > tree via git? > > Do we call that forbidden usage? You can't practically use any overlay foolish enough to publish these repositories for end user consumption. Its just a silly idea. There's no problem with having overlays cloned into a branch as a step towards it hitting mainline, but overlays being distributed to users as main tree branches is just a silly idea. Mostly, because end-users will still have ::gentoo via rsync, and the load of cloning a git repo of ::gentoo will be too much for the average user, doing that just to get an overlay is exhaustively execessive vs the current mechanism we have for overlays, and it comes at a penalty at being not as good as overlays in that you can't easily have >1 of them. -- Kent perl -e "print substr( \"edrgmaM SPA NOcomil.ic\\@tfrken\", \$_ * 3, 3 ) for ( 9,8,0,7,1,6,5,4,3,2 );" http://kent-fredric.fox.geek.nz
Re: Re: Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Fri, Jun 01, 2012 at 11:53:52AM -0400, Rich Freeman wrote: > However, Kent did point out the rebase function doesn't actually apply > new signatures to the "new old" commits anyway, so you'd end up with > unsigned commits in the history. Only in your local history. The push to the central repo would only send the commits in the active chain to your branch HEAD. -- Robin Hugh Johnson Gentoo Linux: Developer, Trustee & Infrastructure Lead E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Fri, Jun 01, 2012 at 10:45:48AM -0500, William Hubbs wrote: > Overlays are completely separate repositories. There is nothing stopping > an overlay from using git right now even if the main tree isn't using > git. They just work in their git repositories until they are ready to > commit something to the main tree, then they move the changes to the > main tree. What about overlay repositories that elect to be a branch of the main tree via git? Do we call that forbidden usage? -- Robin Hugh Johnson Gentoo Linux: Developer, Trustee & Infrastructure Lead E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Sat, Jun 02, 2012 at 03:56:43AM +1200, Kent Fredric wrote: > You can however merge dissimilar histories with no common parents if > you know what you're doing. It does warn you, but it still lets you do > it. > > … > > Yeah, selectively pulling in files with histories however is hard, > I've occasionally been fussed enough to create temporary copies of > branches, and then iterate over their history so the history is > reduced to changes only for a small selection of files, but its > largely lots of work, and I can't remember how to do it every time I > go to do it -_- I have to look it up every time I need it too, but the procedure involves `git filter-branch` [1,2], and actually is pretty easy to wrap your head around, even though the paricular commands are not so easy to remember. Trevor [1]: http://stackoverflow.com/questions/7375528/ [2]: http://gbayer.com/development/moving-files-from-one-git-repository-to-another-preserving-history/ -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy signature.asc Description: OpenPGP digital signature
Re: Re: Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On 2 June 2012 03:53, Rich Freeman wrote: > git-rebase is just a shell script, that ultimately just calls > git-commit as far as I can see, which means that implementing > re-signing is just a matter of adding the appropriate parameters, or > use default configuration (assuming it doesn't already do this). > > Rich > Oh that makes it slightly easier, I didn't realise it was just a blob of bash doing all that O.o grep -nR 'git commit' git-rebase* git-rebase--interactive.sh:152: warn " git commit --amend" git-rebase--interactive.sh:441: git commit --amend --no-post-rewrite || { git-rebase--interactive.sh:484: do_with_author output git commit --no-verify -F "$squash_msg" || git-rebase--interactive.sh:491: do_with_author git commit --no-verify -F "$fixup_msg" || git-rebase--interactive.sh:496: do_with_author git commit --no-verify -e || git-rebase--interactive.sh:699: git commit --amend git-rebase--interactive.sh:703: git commit git-rebase--interactive.sh:723: do_with_author git commit --no-verify -F "$msg" -e || { git-rebase--merge.sh:30:if ! git commit --no-verify -C "$cmt" git-rebase--merge.sh:32:echo "Commit failed, please do not call \"git commit\"" Throwing '-S' in there liberally aught to do the trick. -- Kent perl -e "print substr( \"edrgmaM SPA NOcomil.ic\\@tfrken\", \$_ * 3, 3 ) for ( 9,8,0,7,1,6,5,4,3,2 );" http://kent-fredric.fox.geek.nz
Re: Re: Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Fri, Jun 1, 2012 at 5:53 PM, Rich Freeman wrote: > If you want the tree to be traceable to Gentoo devs, then rewriting > the signatures is probably a good thing. I'd say that signing the merge commit is good enough. It says the Gentoo dev who merged it has reviewed the changes and can be held responsible for them. One could even say that he mediates a web-of-trust to the more casual contributor who signed the original csets. Cheers, Dirkjan
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On 2 June 2012 03:39, Rich Freeman wrote: > On Fri, Jun 1, 2012 at 9:25 AM, Nicolas Sebrecht wrote: >> So, like explained before your concern is clearly out of the current >> discussion. Importing commit history from Overlays is not supported and >> will probably never be. Gentoo doesn't forces (and doesn't want to) >> overlays maintainers to use Git. > > I'm not sure that git even supports this, unless the overlay is a > complete clone of the entire gentoo-x86.git repository. > > I think the way git operations work is by finding common parents in > the history of two heads, and then moving forward from there. If you > have two completely different repositories then they never will have a > common parent. You can however merge dissimilar histories with no common parents if you know what you're doing. It does warn you, but it still lets you do it. > Plus, even if it did work, to rebase the overlay on the gentoo-x86 > repository you'd have to import the full gentoo-x86 tree into it. > Then you'd have to push EVERYTHING in your overlay into gentoo-x86. I > don't think you can just push individual files from one tree to > another. Yeah, selectively pulling in files with histories however is hard, I've occasionally been fussed enough to create temporary copies of branches, and then iterate over their history so the history is reduced to changes only for a small selection of files, but its largely lots of work, and I can't remember how to do it every time I go to do it -_- Its reasonably straight forward to cherry-pick commits in though. > From what I've seen the various methods out there which do involve > moving only part of one branch into another basically involve a LOT of > history manipulation or are really no different than just copying the > files into the branch and adding them, losing all history in the > process. > > I'm not sure how important all that history preservation actually is - > the overlay would still possess it. If we did want to preserve it > then the only practical option I see is to have the overlay start out > as a clone of gentoo-x86 and keep around all the non-overlay packages, > which then means that anybody using the overlay will get all those old > gentoo-x86 packages as part of their portage tree. Plus you still > have the rebase+gpg-signatures=fail issue. > > Rich > Myself, I'd be inclined to do something like this: 1. Have an integration branch on gx86 2. Select a package or packages to migrate to gx86 from an overlay 3. create a branch in the overlay that only contains the history with regard to those packages. 4. copy the branch to a local checkout of gx86, and then rebase it on top of the integration branch 5. then delete it from the overlay. But the exact part of "Create a branch that contains only the history of those packages" is the part I need to get down to an art . -- Kent perl -e "print substr( \"edrgmaM SPA NOcomil.ic\\@tfrken\", \$_ * 3, 3 ) for ( 9,8,0,7,1,6,5,4,3,2 );" http://kent-fredric.fox.geek.nz
Re: Re: Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Fri, Jun 1, 2012 at 11:36 AM, Andreas K. Huettel wrote: >> On 2 June 2012 03:12, Andreas K. Huettel wrote: >> Yes. Which basically means, you *cannot* have both >> >> a) rebase only merges >> and >> b) every commit must be signed >> >> as policies. >> > > I would say that this is a very strong argument in favour of allowing merge > commits. One advantage of merge commits with signatures is that the history really does reflect who signed what. Proxy maintainer signs a bunch of ebuilds. I merge them in. The commits show that the proxy maintainer signed a bunch of work done against an old tree, and I signed a bunch of merge diffs that basically synced them up to the new tree. However, this is missing another issue. What is the value of preserving all those original signatures in the first place? I'd think that they'd mainly be used as some kind of web-of-trust. Well, would such a web-of-trust include proxy maintainers in the first place? If you want the tree to be traceable to Gentoo devs, then rewriting the signatures is probably a good thing. However, Kent did point out the rebase function doesn't actually apply new signatures to the "new old" commits anyway, so you'd end up with unsigned commits in the history. git-rebase is just a shell script, that ultimately just calls git-commit as far as I can see, which means that implementing re-signing is just a matter of adding the appropriate parameters, or use default configuration (assuming it doesn't already do this). Rich
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Fri, Jun 01, 2012 at 12:57:10PM +, Duncan wrote: > William Hubbs posted on Thu, 31 May 2012 15:57:14 -0500 as excerpted: > > Overlays aren't really part of this discussion; those are independent > > trees which we have no control over, so commiting changes from overlays > > to the main tree is the responsibility of the overlay maintainers. > > But it seems to me that overlays are the primary use case for commits to > public trees other than gentoo first. Otherwise, the whole rebase-vs- > merge problem goes away, because the first public commit is to the gentoo > tree. But especially with overlays (like kde) that have an overlay- > first, test, then gentoo-tree, policy, that public overlay tree (which is > already in git) is part of the process. Commits MUST go thru the overlay > to get to the tree, and that overlay is public, so constant rebasing is a > definite no-no. > > Which unless your workaround idea works, pretty much leaves us with merge- > commits as a necessity. (Which of course, as Ciaran pointed out, are > part of the point of git, such that running git without merge-commits > defeats part of the purpose of the whole exercise.) Overlays are completely separate repositories. There is nothing stopping an overlay from using git right now even if the main tree isn't using git. They just work in their git repositories until they are ready to commit something to the main tree, then they move the changes to the main tree. What the main tree on git would give them is the ability to create a branch from the main tree for their changes, but that would not be pushed to the main repository. All they would have to do when they are ready for their code to be merged back into the main repository is make sure that they are creating a fast-forward merge so that there is no merge commit on the master branch. William pgpmzpgQTqKJR.pgp Description: PGP signature
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Fri, Jun 1, 2012 at 9:25 AM, Nicolas Sebrecht wrote: > So, like explained before your concern is clearly out of the current > discussion. Importing commit history from Overlays is not supported and > will probably never be. Gentoo doesn't forces (and doesn't want to) > overlays maintainers to use Git. I'm not sure that git even supports this, unless the overlay is a complete clone of the entire gentoo-x86.git repository. I think the way git operations work is by finding common parents in the history of two heads, and then moving forward from there. If you have two completely different repositories then they never will have a common parent. Plus, even if it did work, to rebase the overlay on the gentoo-x86 repository you'd have to import the full gentoo-x86 tree into it. Then you'd have to push EVERYTHING in your overlay into gentoo-x86. I don't think you can just push individual files from one tree to another. >From what I've seen the various methods out there which do involve moving only part of one branch into another basically involve a LOT of history manipulation or are really no different than just copying the files into the branch and adding them, losing all history in the process. I'm not sure how important all that history preservation actually is - the overlay would still possess it. If we did want to preserve it then the only practical option I see is to have the overlay start out as a clone of gentoo-x86 and keep around all the non-overlay packages, which then means that anybody using the overlay will get all those old gentoo-x86 packages as part of their portage tree. Plus you still have the rebase+gpg-signatures=fail issue. Rich
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Sat, 2 Jun 2012 03:25:43 +1200 Kent Fredric wrote: > On 2 June 2012 03:12, Andreas K. Huettel wrote: > >> "git cat-file -p $sha" is as close as you can get to commit objects > >> without needing to write your own decompressing wrapper. But it > >> gives the same results. > > > > Now, does the "signed data" also contain the parent sha? > > > > If yes, our discussion about rebasing is moot, because a rebase > > will in every case destroy previous signatures. > > > > Yes. Which basically means, you *cannot* have both > > a) rebase only merges > and > b) every commit must be signed > > as policies. > > At very best, I think either > a) a future git might support signed rebases ( ie: replacing existing > signatures with new signatures in the name of the person performing > the rebase ) > or > b) somebody could write a wrapper that provides signed-rebase support > until git get around to implementing it natively. > > and even then, you're going to lose original signing info ( Though, > thats no worse than the signer of the manifest file changing every > sign ) Yes, it's no worse so we're practically considering implementing a very complex mechanism for no real benefit. As I see it, as good would be only requiring some kind of 'top-level' commits to be signed. For example, if one does merge a branch to the tree, a signed merge commit should already be good enough for us. Not sure if git is able to do that, however... -- Best regards, Michał Górny signature.asc Description: PGP signature
Re: Re: Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
> On 2 June 2012 03:12, Andreas K. Huettel wrote: > >> "git cat-file -p $sha" is as close as you can get to commit objects > >> without needing to write your own decompressing wrapper. But it gives > >> the same results. > > > > Now, does the "signed data" also contain the parent sha? > > > > If yes, our discussion about rebasing is moot, because a rebase will in > > every case destroy previous signatures. > > Yes. Which basically means, you *cannot* have both > > a) rebase only merges > and > b) every commit must be signed > > as policies. > I would say that this is a very strong argument in favour of allowing merge commits. -- Andreas K. Huettel Gentoo Linux developer kde, sci, arm, tex, printing signature.asc Description: This is a digitally signed message part.
Re: Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Fri, Jun 1, 2012 at 11:12 AM, Andreas K. Huettel wrote: > Now, does the "signed data" also contain the parent sha? > So, I was working on a lengthy email which now would be fairly repetitive of what Kent posted. Suffice it to say I managed to rip out a commit from the kde overlay, deflate it, and compared that the signature: -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.18 (GNU/Linux) iQEcBAABCgAGBQJPx+mcAAoJEO+t9ga+3I3aqLoH/0OrRA1+NPRHGfbbLoQrqMwl sB+2It2Pb9LfPjEme+lrQu5WgFY4j7k0qd2ZYdnXM7JdQjsqmpfAMloHh5JN4TAS 4vk8+u2GJCYgzL/SY5XlPl2l8dT91PhQJSN0yVt4Q9TsoN3nzVpFBjACJCy9R6j2 HrXvz/g3+MqY/9VesV8IiVgvQUTVgCdh8zBJ2rVyWAVH0bErsn518aiwEyfzNOxA 1qJxxgGJLMpXp+nI8rnmhqTAAKiNA+byAKAsTEl3LS7OvQZ51aOCwa4A2GLOn2ef 5JmuYQG5/FsS0RfXrqk72PiStTBWa3TakHYrgNXOXlslIR5AIB2tYnXqZcdEqYQ= =fucY -END PGP SIGNATURE- does in fact verify for the payload: --start-- tree 7d7f97cded40158d0f580ca6fbe97398d5c867f8 parent 14d7d9cb2219f64c7a715d8da0bbe48a32c9dad8 author Johannes Huber 1338501525 +0200 committer Johannes Huber 1338501525 +0200 [kde-base/kdelibs] Sync with live. (Portage version: 2.2.0_alpha108/git/Linux i686, unsigned Manifest commit) --end-- Dump those into a text file and run gpg for yourself... The full commit contains the gpg signature in a field as already posted by Kent. And while I appreciate the performance boost and space savings provided by all the compression/packing/etc, I've learned to almost hate those features with a passion this morning... Getting a cloned repo unpacked, and the commit decompressed was a bit pita. The other issue is that the header in the commit file is stripped before it is signed, the actual start of the commit is "commit 830tree..." Rich
Re: Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On 2 June 2012 03:12, Andreas K. Huettel wrote: >> "git cat-file -p $sha" is as close as you can get to commit objects >> without needing to write your own decompressing wrapper. But it gives >> the same results. > > Now, does the "signed data" also contain the parent sha? > > If yes, our discussion about rebasing is moot, because a rebase will in every > case destroy previous signatures. > Yes. Which basically means, you *cannot* have both a) rebase only merges and b) every commit must be signed as policies. At very best, I think either a) a future git might support signed rebases ( ie: replacing existing signatures with new signatures in the name of the person performing the rebase ) or b) somebody could write a wrapper that provides signed-rebase support until git get around to implementing it natively. and even then, you're going to lose original signing info ( Though, thats no worse than the signer of the manifest file changing every sign ) -- Kent perl -e "print substr( \"edrgmaM SPA NOcomil.ic\\@tfrken\", \$_ * 3, 3 ) for ( 9,8,0,7,1,6,5,4,3,2 );" http://kent-fredric.fox.geek.nz
Re: Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
> "git cat-file -p $sha" is as close as you can get to commit objects > without needing to write your own decompressing wrapper. But it gives > the same results. Now, does the "signed data" also contain the parent sha? If yes, our discussion about rebasing is moot, because a rebase will in every case destroy previous signatures. -- Andreas K. Huettel Gentoo Linux developer kde, sci, arm, tex, printing signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
> The purpose of overlays is to have ebuilds maintained outside of the > official Gentoo portage. Importing a ebuild from an overlay whether it > uses Git or not means importing the ebuild(s). In the Git context, it > means the Gentoo maintainer has to make an import commit the same way > it would be done to start a project with somthing like: > > cp 'files to be commited' . > git add . > git commit -m 'initial import' > > So, like explained before your concern is clearly out of the current > discussion. Importing commit history from Overlays is not supported and > will probably never be. Which does not mean that it's not desirable. The KDE ebuilds are mainly evolving inside the KDE overlay, where KDE betas and live ebuilds live. Being able (in the future) to see the history of an ebuild before it was imported into the main tree would certainly help in some cases. -- Andreas K. Huettel Gentoo Linux developer kde, sci, arm, tex, printing signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On 2 June 2012 00:42, Rich Freeman wrote: > On Fri, Jun 1, 2012 at 7:23 AM, Kent Fredric wrote: >> >> Nope, at least not as far as I can tell, and I just implemented commit >> signature verification >_> > > I've been trying to find an example of a signed commit, but can't find > one anywhere, so it is hard to tell what it is doing without going > through the git source carefully. If you have an example of a signed > commit feel free to send it to me. > > Note that I am NOT interested in the output of any git operation (such > as git log --show-signature, git show, etc) - these are all processed > and do not reveal what is actually going on. I want a copy of the > actual file containing the signature. If the signature is embedded in > the commit then I want the file in the objects directory tree that > contains the commit. They're just compressed text files (though it is > a bit of a pain to decompress them). "git cat-file -p $sha" is as close as you can get to commit objects without needing to write your own decompressing wrapper. But it gives the same results. Here is a sample signed git commit ( sans compression ) : https://gist.github.com/2852363 Line 5 to 21 are where the GPG signature is stored. ( Git uses the leading space on all the lines of the GPG sig to indicate that its part of the signature, line 22 does not lead with a space, so that line is after the GPG signature. Line 22 also is empty, which ends the "header" section of the commit, and there starts the "comment" section. If you don't believe me that "git cat-file -p $sha" is reliable, here is a simple script that has no git internals in it , its just Zlib decompression : https://gist.github.com/2852411 perl /tmp/deflate.pl .git/objects/66/50c09ad7b2a62e28476e639654443015ac5945 And that re-emits exactly the same thing : https://gist.github.com/2852363 And if you want to do that yourself, here's that object in compressed form base64 encoded https://gist.github.com/2852436 > The filesystem WON'T look the same after a rebase. > > quick example (operations done in this order): > > file in commit A in master: > 1 > 2 > 3 > 4 > 5 > > file in commit B in a branch off of master: > 1 > 2a > 3 > 4 > 5 > > file in commit C in master: > 1 > 2 > 3 > 4a > 5 > > if you merge master into the branch you'll end up with a new commit D > whose parent is B that looks like: > 1 > 2a > 3 > 4a > 5 > > if instead you rebase master into the branch you'll end up with a new > commit D whose parent is C that looks like: > 1 > 2a > 3 > 4a > 5 > > The history for the branch will no longer contain B. If there were 14 > commits B1..14 you'd end up with 14 commits D1.14 that each contain > the line 4a. None of them would use the same trees as B1..14, and > they can't use the same signatures as a result, even if only the tree > was signed. As far as the new history was concerned, line 4a was > there before the branch was started. It all depends really on how you do the rebase, if the changes in the rebase shadow all the competing changes , or the changes are shadowed out before the rebase, the rebase will be the same afterwards, just with a different history. ie: root -> a -> b -> c -> d( backout of c ) If you had started a branch off "b", and then rebased that branch on top of d, the tree would be in the exact same state, but the history would have changed. And likewise, if the change was : root -> a -> b -> c -> d -> e And you started a branch at b, with commits m, n, o, p , changing the same lines c, d and e changed, and when you rebased, you opted to let the changes in m, n, and o replace the changes in c, d and e, ( which has the same effect as if you had deleted c , d and e, except they're still there in the history ), when you move the branch from b to e, the last commit of that branch (p) will still be the same. The rebased commits m, n, and o will be different to what they were when they were on "b", because now they're not only making changes vs "b", but they're also replacing code added in c, d, and e. Sure, this only happens in certain circumstances, and usually when it happens, its because you want it to happen , but it does happen, and its quite useful sometimes. ( essentially, x + y + z == x can be true, as long as z = -y ) > At least, that is my understanding of rebasing. Again, I'm a bit of a > git novice, but I never really grokked git until I saw a presentation > that didn't start with commands, but instead started out with just > walking through the actual structure of the repository. Once you grok > the COW model I find it all clicks into place. Yeah, chapter 9 of the Pro Git book is where I recommend everyone with even a sliver of comp-sci knowledge to start. http://www.git-scm.com/book/en/Git-Internals Once you see its basically a bunch of text files, that refer to each other via sha1 , and are stored in files named after their sha1, ( after a bit of compression ), all that graph theory just clicks and you wond
[gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
The 01/06/12, Duncan wrote: > But it seems to me that overlays are the primary use case for commits to > public trees other than gentoo first. Otherwise, the whole rebase-vs- > merge problem goes away, because the first public commit is to the gentoo > tree. But especially with overlays (like kde) that have an overlay- > first, test, then gentoo-tree, policy, that public overlay tree (which is > already in git) is part of the process. Commits MUST go thru the overlay > to get to the tree, and that overlay is public, so constant rebasing is a > definite no-no. The purpose of overlays is to have ebuilds maintained outside of the official Gentoo portage. Importing a ebuild from an overlay whether it uses Git or not means importing the ebuild(s). In the Git context, it means the Gentoo maintainer has to make an import commit the same way it would be done to start a project with somthing like: cp 'files to be commited' . git add . git commit -m 'initial import' So, like explained before your concern is clearly out of the current discussion. Importing commit history from Overlays is not supported and will probably never be. Gentoo doesn't forces (and doesn't want to) overlays maintainers to use Git. -- Nicolas Sebrecht
[gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
William Hubbs posted on Thu, 31 May 2012 15:57:14 -0500 as excerpted: > On Thu, May 31, 2012 at 08:26:58PM +, Duncan wrote: >> William Hubbs posted on Thu, 31 May 2012 14:54:50 -0500 as excerpted: >> >> I don't know what's going to happen to all the overlays with the main >> tree switch to git, but won't that break various "overlay first" >> policies, say for the kde overlay? >> >> Of course, if all the official overlays are converted to git branches >> of the main tree... but won't they still require rebasing as they've >> already been pushed? (This assumes your workaround idea doesn't work. >> If it does, great!) > > Overlays aren't really part of this discussion; those are independent > trees which we have no control over, so commiting changes from overlays > to the main tree is the responsibility of the overlay maintainers. But it seems to me that overlays are the primary use case for commits to public trees other than gentoo first. Otherwise, the whole rebase-vs- merge problem goes away, because the first public commit is to the gentoo tree. But especially with overlays (like kde) that have an overlay- first, test, then gentoo-tree, policy, that public overlay tree (which is already in git) is part of the process. Commits MUST go thru the overlay to get to the tree, and that overlay is public, so constant rebasing is a definite no-no. Which unless your workaround idea works, pretty much leaves us with merge- commits as a necessity. (Which of course, as Ciaran pointed out, are part of the point of git, such that running git without merge-commits defeats part of the purpose of the whole exercise.) -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
Hi! Check kde overlay ;) we used signed commits here Rich Freeman писал 2012-06-01 16:42: On Fri, Jun 1, 2012 at 7:23 AM, Kent Fredric wrote: Nope, at least not as far as I can tell, and I just implemented commit signature verification >_> I've been trying to find an example of a signed commit, but can't find one anywhere, so it is hard to tell what it is doing without going through the git source carefully. If you have an example of a signed commit feel free to send it to me. Note that I am NOT interested in the output of any git operation (such as git log --show-signature, git show, etc) - these are all processed and do not reveal what is actually going on. I want a copy of the actual file containing the signature. If the signature is embedded in the commit then I want the file in the objects directory tree that contains the commit. They're just compressed text files (though it is a bit of a pain to decompress them). Not nessecarily. Given that : a file with a given content has a fixed SHA A tree is just a list of these SHA's , and that in turn is referenced by SHA, so if 2 commits have identical file content, their 'tree' sha will be the same ( in theory ). So that means, if you perform a rebase, assuming the filesystem looks the same as it did before the rebase, it will have the same SHA1 for the tree, regardless of the process of how it got to be that way. The filesystem WON'T look the same after a rebase. quick example (operations done in this order): file in commit A in master: 1 2 3 4 5 file in commit B in a branch off of master: 1 2a 3 4 5 file in commit C in master: 1 2 3 4a 5 if you merge master into the branch you'll end up with a new commit D whose parent is B that looks like: 1 2a 3 4a 5 if instead you rebase master into the branch you'll end up with a new commit D whose parent is C that looks like: 1 2a 3 4a 5 The history for the branch will no longer contain B. If there were 14 commits B1..14 you'd end up with 14 commits D1.14 that each contain the line 4a. None of them would use the same trees as B1..14, and they can't use the same signatures as a result, even if only the tree was signed. As far as the new history was concerned, line 4a was there before the branch was started. At least, that is my understanding of rebasing. Again, I'm a bit of a git novice, but I never really grokked git until I saw a presentation that didn't start with commands, but instead started out with just walking through the actual structure of the repository. Once you grok the COW model I find it all clicks into place. Rich -- Best Regards, Alexey 'Alexxy' Shvetsov Petersburg Nuclear Physics Institute, NRC Kurchatov Institute, Gatchina, Russia Department of Molecular and Radiation Biophysics Gentoo Team Ru Gentoo Linux Dev mailto:alexx...@gmail.com mailto:ale...@gentoo.org mailto:ale...@omrb.pnpi.spb.ru
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Fri, Jun 1, 2012 at 7:23 AM, Kent Fredric wrote: > > Nope, at least not as far as I can tell, and I just implemented commit > signature verification >_> I've been trying to find an example of a signed commit, but can't find one anywhere, so it is hard to tell what it is doing without going through the git source carefully. If you have an example of a signed commit feel free to send it to me. Note that I am NOT interested in the output of any git operation (such as git log --show-signature, git show, etc) - these are all processed and do not reveal what is actually going on. I want a copy of the actual file containing the signature. If the signature is embedded in the commit then I want the file in the objects directory tree that contains the commit. They're just compressed text files (though it is a bit of a pain to decompress them). > Not nessecarily. Given that : > > a file with a given content has a fixed SHA > A tree is just a list of these SHA's , and that in turn is referenced > by SHA, so if 2 commits have identical file content, their 'tree' sha > will be the same ( in theory ). > > So that means, if you perform a rebase, assuming the filesystem looks > the same as it did before the rebase, it will have the same SHA1 for > the tree, regardless of the process of how it got to be that way. The filesystem WON'T look the same after a rebase. quick example (operations done in this order): file in commit A in master: 1 2 3 4 5 file in commit B in a branch off of master: 1 2a 3 4 5 file in commit C in master: 1 2 3 4a 5 if you merge master into the branch you'll end up with a new commit D whose parent is B that looks like: 1 2a 3 4a 5 if instead you rebase master into the branch you'll end up with a new commit D whose parent is C that looks like: 1 2a 3 4a 5 The history for the branch will no longer contain B. If there were 14 commits B1..14 you'd end up with 14 commits D1.14 that each contain the line 4a. None of them would use the same trees as B1..14, and they can't use the same signatures as a result, even if only the tree was signed. As far as the new history was concerned, line 4a was there before the branch was started. At least, that is my understanding of rebasing. Again, I'm a bit of a git novice, but I never really grokked git until I saw a presentation that didn't start with commands, but instead started out with just walking through the actual structure of the repository. Once you grok the COW model I find it all clicks into place. Rich
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Fri, 1 Jun 2012 23:23:34 +1200 Kent Fredric wrote: > On 1 June 2012 22:54, Rich Freeman wrote: > > Rebasing re-applies the same diff to the new head to give you a new > > set of commits. When you apply the same diff to a different parent > > you end up with a different tree, so the tree signature won't be the > > same either. > > Not nessecarily. Given that : > > a file with a given content has a fixed SHA > A tree is just a list of these SHA's , and that in turn is referenced > by SHA, so if 2 commits have identical file content, their 'tree' sha > will be the same ( in theory ). > > So that means, if you perform a rebase, assuming the filesystem looks > the same as it did before the rebase, it will have the same SHA1 for > the tree, regardless of the process of how it got to be that way. I don't think that 'not necessarily' makes any difference here. Maybe in our particular case this is not as likely as with regular source code tree but while rebasing you can hit conflicts. And then files start to change... -- Best regards, Michał Górny signature.asc Description: PGP signature
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On 1 June 2012 22:54, Rich Freeman wrote: > On Fri, Jun 1, 2012 at 12:55 AM, Kent Fredric wrote: >> >> Hmm, thats annoying. Almost makes me wish it was the trees that were >> signed, not the commits. > > I think it is the tree that is signed, but that changes too. Nope, at least not as far as I can tell, and I just implemented commit signature verification >_> > Rebasing re-applies the same diff to the new head to give you a new > set of commits. When you apply the same diff to a different parent > you end up with a different tree, so the tree signature won't be the > same either. Not nessecarily. Given that : a file with a given content has a fixed SHA A tree is just a list of these SHA's , and that in turn is referenced by SHA, so if 2 commits have identical file content, their 'tree' sha will be the same ( in theory ). So that means, if you perform a rebase, assuming the filesystem looks the same as it did before the rebase, it will have the same SHA1 for the tree, regardless of the process of how it got to be that way. The only SHA that has to change is the 'parent', ( Demonstration here: https://gist.github.com/2851330 , note I have 2 commits with the same tree sha, and the tree sha only really refers to one file 3 times as all the empty files have same sha ) But unfortunately, with a rebase, even if the trees don't change, if the history order changes, the commits themselves have to change due to the "parent" sha needing to change ( yes, I know commits are immutable in reality, and they don't change, but are duplicated and the duplicate is given the new sha , but the effect is still the same , because those unreferenced commits will eventually get reaped ) -- Kent perl -e "print substr( \"edrgmaM SPA NOcomil.ic\\@tfrken\", \$_ * 3, 3 ) for ( 9,8,0,7,1,6,5,4,3,2 );" http://kent-fredric.fox.geek.nz
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Fri, Jun 1, 2012 at 12:55 AM, Kent Fredric wrote: > > Hmm, thats annoying. Almost makes me wish it was the trees that were > signed, not the commits. I think it is the tree that is signed, but that changes too. Rebasing re-applies the same diff to the new head to give you a new set of commits. When you apply the same diff to a different parent you end up with a different tree, so the tree signature won't be the same either. Keep in mind that git does not store a long train of diffs. It stores a long chain of complete trees, and the diffs get calculated if you ask for them. Since it is COW you only re-store files that actually change, and incorporate others by reference. However, if you have a 1MB file that you change 1 line on 100x, you'll end up with 100MB of files. Of course, when they get packed I'd hope that they'd compress well. Rich
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Fri, Jun 1, 2012 at 12:05 AM, Michał Górny wrote: > Yes, it isn't but such kind of work flow was presented in the message I > replied to. Yup, I wasn't aware that when rebasing you have the option to squash commits or not. They all get rewritten as if they were applied to the current head in the first place, but apparently unless you squash them you still get all the individual commits. Rich
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On 1 June 2012 20:16, Dirkjan Ochtman wrote: > Can you elaborate on why the cleaner history a no-merge policy > enforces is a good thing? I actually think that seeing merge commits > might clarify the history; it can be valuable to see that some mistake > was made in a merge instead, but you can only see that if there's an > explicit merge commit. > > I should note that I come at this from the Mercurial side, where the > immutability of (public) history has historically carried more value > than on the git side (and related to that, rebase-like tools were less > integrated until relatively recently). > > Cheers, > > Dirkjan > One of the perk of fast-forward only histories is merge problems don't happen. If a commit sequence isn't fast-fowardable, then the submitter has to rebase it, and a rebased history is a guaranteed clean merge. It also puts the onus on making the patch sequence mergable on the contributor of that patch sequence, by forcing them to resolve conflicts before they can submit their branch for inclusion, and this is good, because they understand their own changes best, and are the best person to decide how to deal with conflicts. The best example of this is 2 people contribute behaviourally identical code, with different syntax . In the case of a merge, the person performing the merge has to decide which one to take, and they might not be the best person to do so. ( And historically, it can be confusing if you look at the parents of the merge, and find the 2 competing implementations, of which only 1 is relevant ) If you request that a branch is rebased before it is integrated, then this problem solves itself in a way. Whoevers implementation gets in the tree first gets in, and then the second person rebases their branch on top of that. In the process of performing the rebase, the second person will discover the precise commit on their side that introduces a conflict, and be able to decide if they need to replace the existing code anyway, or if they need to destroy their own code. And when they're done, their entire commit series should be sane and logical as if the first persons code was there in the first place before the second person even started coding. a--->b--->c>d--->e--->f | \|/ \--->x--->y>z Essentially, in this diagram, if d and y are different, but equivalent, they will cause a collision when merge Z is performed. By performing a rebase of the second branch, the logical order becomes a--->b--->c>d--->e--->f--->x--->y--->z And when the rebaser is applying "y", they will notice it is likely redundant, and the history will become a--->b--->c>d--->e--->f--->x--->z And in both the merge and rebase examples, z will be the same, just the cognitive overhead of understanding "what the hell happened" in the latter case is simpler, as the biggest delta between any 2 commits will be 1 commit worth, whereas in the merge case, there will be a huge delta between the commit before the merge, and the commit after the merge, and looking at the merge itself diff-wise, you'll be comparing the aggregate result of 2 entire branches worth of commits, as opposed to only comparing one small and simple commit with another. -- Kent perl -e "print substr( \"edrgmaM SPA NOcomil.ic\\@tfrken\", \$_ * 3, 3 ) for ( 9,8,0,7,1,6,5,4,3,2 );" http://kent-fredric.fox.geek.nz
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, May 31, 2012 at 6:49 PM, Robin H. Johnson wrote: > Discussion on merge policy. Originally I thought we would disallow merge > commits, so that we would get a cleaner history. However, it turns out that if > the repo ends up being pushed to different places with slightly different > histories, merges are absolutely going to be required to prevent somebody from > having to rebase at least one of their sets of commits that are already > pushed. Can you elaborate on why the cleaner history a no-merge policy enforces is a good thing? I actually think that seeing merge commits might clarify the history; it can be valuable to see that some mistake was made in a merge instead, but you can only see that if there's an explicit merge commit. I should note that I come at this from the Mercurial side, where the immutability of (public) history has historically carried more value than on the git side (and related to that, rebase-like tools were less integrated until relatively recently). Cheers, Dirkjan
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On 1 June 2012 14:49, Rich Freeman wrote: > On Thu, May 31, 2012 at 5:52 PM, Kent Fredric wrote: >> Just I haven't worked out what happens when the SHA1 of the 'parent' >> header changes, which *will* change if the rebase is anything other >> than a fast-forward. >> >> If that SHA1 changes, the gpg signature will surely fail? > > Rebasing doesn't modify past commits - it creates new ones and the > past ones are no longer in the history of the current head. So, it > doesn't break the old signatures so much as discard them. You would > need to create new signatures in their place, presumably from whoever > performed the rebase. Hmm, thats annoying. Almost makes me wish it was the trees that were signed, not the commits. Although, I probably could brew up a prototype resigning tool ( based on Git::PurePerl ,... when they accept and publish my changes ) , just would be problematic because simply the act of signing a past commit means the SHA1 of the commit itself is different, so all successive commits after a re-signed commit will change and also need to be rebased and re-signed. -- Kent perl -e "print substr( \"edrgmaM SPA NOcomil.ic\\@tfrken\", \$_ * 3, 3 ) for ( 9,8,0,7,1,6,5,4,3,2 );" http://kent-fredric.fox.geek.nz
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, 31 May 2012 16:27:48 -0400 Michael Orlitzky wrote: > On 05/31/12 16:09, Michał Górny wrote: > > On Thu, 31 May 2012 15:58:43 -0400 > > Rich Freeman wrote: > > > >> On Thu, May 31, 2012 at 3:33 PM, Michał Górny > >> wrote: > >>> What would git signing work with rebased commits? Would all of > >>> them have to be signed once again? > >>> > >> > >> The whole point of rebasing is to throw away history (which is > >> either good or bad based on your perspective). > >> > >> So, if 14 devs spend 3 years and 2000 commits working on something > >> in a branch, and I commit it to master using a rebase, then all > >> you'll see in the master history is that rich0 committed 20k lines > >> of code to master on May 31st, and that would be signed by me. > >> > >> I think that rebasing before merging is a pretty typical workflow > >> anyway - when you submit a patch to Linus, he doesn't really care > >> that you spent six months tweaking it - he just is getting a big > >> blob of code that either works or doesn't. Does all that > >> sub-history really matter? You could always push the branch to > >> the repository if you wanted to keep it on the side. > > > > That sounds like a pretty poor workflow to me. If I tweak something > > for 3 years, I'm likely to have a larger set of logically organized > > commits. I'm not saying it's unlikely I'm going to rebase fixes for > > obvious mistakes but putting everything onto one blob just makes > > the changes harder to read and less maintainable. > > > > For example, if I first create a basic function and then add > > additional options to it, I'm likely to keep those as separate > > commits. If one of the changes was a really bad idea, I'd like to > > revert the appropriate commit rather than removing all traces of it > > by hand and mistakenly introducing even worse breakage. > > > > That isn't what rebasing does, unless you do an interactive rebase and > decide to squash the commits. Yes, it isn't but such kind of work flow was presented in the message I replied to. > The usual use case for a rebase is just to avoid the ugly merge > commit. Which devs should simply do whenever they use the scenario you mentioned. I agree we could block 'auto-merges' when pushing to a modified repo but not disallow a valid merges from devs who know what they're doing. -- Best regards, Michał Górny signature.asc Description: PGP signature
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, 31 May 2012 17:04:30 -0500 William Hubbs wrote: > On Thu, May 31, 2012 at 08:23:31PM +0100, Ciaran McCreesh wrote: > > On Thu, 31 May 2012 14:18:04 -0500 > > William Hubbs wrote: > > > > Not sure I'm following, but I will be the first to admit that > > > > I'm a git novice. Would this be aided by a convention, like > > > > only committing to master on the gentoo official repository, > > > > and any on-the-side work on places like github/etc stays in > > > > branches? Those repositories would just keep getting fed > > > > commits on master from the official repository. > > > > > > Iagree with this; I think we should ban merge commits on master. > > > That would force everyone to rebase their work on current master > > > before they commit to master which would make the history clean. > > > > So what's the point of switching to git if you want to ban the main > > reason git exists? > > To clarify: we should only allow fast-forward merges on master. > > My big complaint about merge commits is if you do a git show on > a merge commit, you get nothing, so there is no way to see what > actually changed in that commit. Or you use a graphical tool which shows the whole merge history and you see the exact changes happening rather than some blob with 'do foo, do bar, and some baz too'. -- Best regards, Michał Górny signature.asc Description: PGP signature
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, May 31, 2012 at 5:52 PM, Kent Fredric wrote: > Just I haven't worked out what happens when the SHA1 of the 'parent' > header changes, which *will* change if the rebase is anything other > than a fast-forward. > > If that SHA1 changes, the gpg signature will surely fail? Rebasing doesn't modify past commits - it creates new ones and the past ones are no longer in the history of the current head. So, it doesn't break the old signatures so much as discard them. You would need to create new signatures in their place, presumably from whoever performed the rebase. I'm trying to glean what I can from the little info out there about how the new commit signatures work, but I don't think that you can use signatures to convey authorship if later authors are going to rebase the branch. The situation is not unlike what we have now with manifests. As far as I can tell if you want to do work outside of master, and then get those commits into master but preserve all the past signatures in the history of master, then you'd need to do a merge commit, so that all the deltas to do the merge are in a separate commit which is then signed by the person doing the merge. Rich
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
William Hubbs wrote: > To clarify: we should only allow fast-forward merges on master. Not a dev yet, but +1 pgpYLlPixexJM.pgp Description: PGP signature
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On 1 June 2012 08:26, Duncan <1i5t5.dun...@cox.net> wrote: > William Hubbs posted on Thu, 31 May 2012 14:54:50 -0500 as excerpted: > Of course, if all the official overlays are converted to git branches of > the main tree... but won't they still require rebasing as they've already > been pushed? (This assumes your workaround idea doesn't work. If it > does, great!) > End users will still want to work with overlays that are not merged with the main tree, not merely git branches. Its foreseeable that there will be git branches that /track/ overlays and exist as an integration pipeline for content from the overlays joining core gentoo, but end users will not want to use that. For the simple reason of course, that as soon as you want >1 overlay, portage's way of doing it with separate repositories is far more effective. You don't want each user to have to maintain an octopus merge between all the branches they want to have commits from ;) -- Kent perl -e "print substr( \"edrgmaM SPA NOcomil.ic\\@tfrken\", \$_ * 3, 3 ) for ( 9,8,0,7,1,6,5,4,3,2 );" http://kent-fredric.fox.geek.nz
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, May 31, 2012 at 08:23:31PM +0100, Ciaran McCreesh wrote: > On Thu, 31 May 2012 14:18:04 -0500 > William Hubbs wrote: > > > Not sure I'm following, but I will be the first to admit that I'm a > > > git novice. Would this be aided by a convention, like only > > > committing to master on the gentoo official repository, and any > > > on-the-side work on places like github/etc stays in branches? > > > Those repositories would just keep getting fed commits on master > > > from the official repository. > > > > Iagree with this; I think we should ban merge commits on master. That > > would force everyone to rebase their work on current master before > > they commit to master which would make the history clean. > > So what's the point of switching to git if you want to ban the main > reason git exists? To clarify: we should only allow fast-forward merges on master. My big complaint about merge commits is if you do a git show on a merge commit, you get nothing, so there is no way to see what actually changed in that commit. William pgpLhNVh63xgN.pgp Description: PGP signature
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On 1 June 2012 07:58, Rich Freeman wrote: > On Thu, May 31, 2012 at 3:33 PM, Michał Górny wrote: >> What would git signing work with rebased commits? Would all of them >> have to be signed once again? >> > > The whole point of rebasing is to throw away history (which is either > good or bad based on your perspective). > > So, if 14 devs spend 3 years and 2000 commits working on something in > a branch, and I commit it to master using a rebase, then all you'll > see in the master history is that rich0 committed 20k lines of code to > master on May 31st, and that would be signed by me. > > I think that rebasing before merging is a pretty typical workflow > anyway - when you submit a patch to Linus, he doesn't really care that > you spent six months tweaking it - he just is getting a big blob of > code that either works or doesn't. Does all that sub-history really > matter? You could always push the branch to the repository if you > wanted to keep it on the side. > > Rich > I think you're conflating "rebasing" and "squashing commits". You should rebase a long commit sequence and squash pointless fixup commits, and to make the commit sequence logical and ordered, possibly divided by logical changes that one may wish to later revert. ( That way, backing out a problem is simply reversing that commit and applying the reversal patch ) You should not rebase for the purpose of squashing an entire history of changes into a single scattered commit. Rebasing is more to make the history itself linear and non-complex, as when walking backwards through history, there being 2 parallel histories that generated a merged commit can be confusing for humans, so eliminating the parallel histories is one of the primary purposes I advocate use of rebase for. Squashed commits are a handy feature of rebase, but I wouldn't want to see an entire overlay squashed into the main tree as a single squashed commit. Another bad reason for squashed commits: if you're getting rid of the Changes files, you'll have no history on anything if you just group long histories into a single commit. None. -- Kent perl -e "print substr( \"edrgmaM SPA NOcomil.ic\\@tfrken\", \$_ * 3, 3 ) for ( 9,8,0,7,1,6,5,4,3,2 );" http://kent-fredric.fox.geek.nz
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On 1 June 2012 07:52, Alexey Shvetsov wrote: >> >> What would git signing work with rebased commits? Would all of them >> have to be signed once again? > > > Commits itsels still will be signed Do you know how git does this? Do you have experience/information you can cite as to that this works? Commit signing seems poorly documented at present, and I've been looking at the git internals, and it would *APPEAR* that the content that is signed is the blob of text you normally get when you git cat-file -p $SHA1 And indeed, if you git cat-file -p $SHA1 > file, extract the SIGNATURE part into its own file (removing the leading spaces), and remove the "gnupg" section from the commit headers, gpg --verify $sigfile $file # tells me I have a good signature. Just I haven't worked out what happens when the SHA1 of the 'parent' header changes, which *will* change if the rebase is anything other than a fast-forward. If that SHA1 changes, the gpg signature will surely fail? -- Kent perl -e "print substr( \"edrgmaM SPA NOcomil.ic\\@tfrken\", \$_ * 3, 3 ) for ( 9,8,0,7,1,6,5,4,3,2 );" http://kent-fredric.fox.geek.nz
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, May 31, 2012 at 03:58:43PM -0400, Rich Freeman wrote: > On Thu, May 31, 2012 at 3:33 PM, Michał Górny wrote: > > What would git signing work with rebased commits? Would all of them > > have to be signed once again? > > > > The whole point of rebasing is to throw away history (which is either > good or bad based on your perspective). > > So, if 14 devs spend 3 years and 2000 commits working on something in > a branch, and I commit it to master using a rebase, then all you'll > see in the master history is that rich0 committed 20k lines of code to > master on May 31st, and that would be signed by me. You don't commit to master with a rebase,; it is always a merge. The type of merge is what controls what you see in the logs. If you rebase your branch on master, merge to master then run "git pull --rebase" then push, you will get a fast forward merge, which shows the individual commits. If you don't include the rebasing, you get another type of merge which just shows up in the logs as one commit afaik. William pgpGov9QxG7kv.pgp Description: PGP signature
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, May 31, 2012 at 08:26:58PM +, Duncan wrote: > William Hubbs posted on Thu, 31 May 2012 14:54:50 -0500 as excerpted: > > I don't know what's going to happen to all the overlays with the main > tree switch to git, but won't that break various "overlay first" > policies, say for the kde overlay? > > Of course, if all the official overlays are converted to git branches of > the main tree... but won't they still require rebasing as they've already > been pushed? (This assumes your workaround idea doesn't work. If it > does, great!) Overlays aren't really part of this discussion; those are independent trees which we have no control over, so commiting changes from overlays to the main tree is the responsibility of the overlay maintainers. William pgpnUoF15VbKy.pgp Description: PGP signature
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On 05/31/12 16:09, Michał Górny wrote: > On Thu, 31 May 2012 15:58:43 -0400 > Rich Freeman wrote: > >> On Thu, May 31, 2012 at 3:33 PM, Michał Górny >> wrote: >>> What would git signing work with rebased commits? Would all of them >>> have to be signed once again? >>> >> >> The whole point of rebasing is to throw away history (which is either >> good or bad based on your perspective). >> >> So, if 14 devs spend 3 years and 2000 commits working on something in >> a branch, and I commit it to master using a rebase, then all you'll >> see in the master history is that rich0 committed 20k lines of code to >> master on May 31st, and that would be signed by me. >> >> I think that rebasing before merging is a pretty typical workflow >> anyway - when you submit a patch to Linus, he doesn't really care that >> you spent six months tweaking it - he just is getting a big blob of >> code that either works or doesn't. Does all that sub-history really >> matter? You could always push the branch to the repository if you >> wanted to keep it on the side. > > That sounds like a pretty poor workflow to me. If I tweak something for > 3 years, I'm likely to have a larger set of logically organized commits. > I'm not saying it's unlikely I'm going to rebase fixes for obvious > mistakes but putting everything onto one blob just makes the changes > harder to read and less maintainable. > > For example, if I first create a basic function and then add additional > options to it, I'm likely to keep those as separate commits. If one of > the changes was a really bad idea, I'd like to revert the appropriate > commit rather than removing all traces of it by hand and mistakenly > introducing even worse breakage. > That isn't what rebasing does, unless you do an interactive rebase and decide to squash the commits. The usual use case for a rebase is just to avoid the ugly merge commit. For example, 1. I clone your portage tree 2. I start making commits locally 3. You add a new package 4. I try to push my changes to you Step 4 doesn't work because your repo has changed. I can either, a) pull from you again (merge) b) pull with a rebase And then I should be able to push to you, assuming there were no conflicts. The merge option works fine but generates an ugly merge commit. The rebase takes our two histories and combines them into a nice linear history. In this case, a rebase would (essentially) take all of my commits and stick them on top of your "new package" commit. Afterwards, it looks like there's a nice linear history: you added a package, and then I did a bunch of other stuff. All of the commits are there. Since that history is linear and it looks like just a bunch of stuff on top of your existing tree, I can push it to you without problems. The only downside to the rebase is that it modifies my local history. So, if somebody cloned *my* repo in the middle of that, they could get screwed up.
[gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
William Hubbs posted on Thu, 31 May 2012 14:54:50 -0500 as excerpted: > On Thu, May 31, 2012 at 07:13:42PM +, Robin H. Johnson wrote: >> - You have a commit, that you want to put into the Gentoo tree. >> - You have already pushed it to your github, signed > > If I have a github tree, that would probably be because I didn't have > push access to the official tree, so signing the commit probably > isn'tgoing to matter; I would expect that a gentoo dev who has push > access to the tree would sign the commit when it is put into the gentoo > tree. I don't know what's going to happen to all the overlays with the main tree switch to git, but won't that break various "overlay first" policies, say for the kde overlay? Of course, if all the official overlays are converted to git branches of the main tree... but won't they still require rebasing as they've already been pushed? (This assumes your workaround idea doesn't work. If it does, great!) -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, 31 May 2012 15:58:43 -0400 Rich Freeman wrote: > On Thu, May 31, 2012 at 3:33 PM, Michał Górny > wrote: > > What would git signing work with rebased commits? Would all of them > > have to be signed once again? > > > > The whole point of rebasing is to throw away history (which is either > good or bad based on your perspective). > > So, if 14 devs spend 3 years and 2000 commits working on something in > a branch, and I commit it to master using a rebase, then all you'll > see in the master history is that rich0 committed 20k lines of code to > master on May 31st, and that would be signed by me. > > I think that rebasing before merging is a pretty typical workflow > anyway - when you submit a patch to Linus, he doesn't really care that > you spent six months tweaking it - he just is getting a big blob of > code that either works or doesn't. Does all that sub-history really > matter? You could always push the branch to the repository if you > wanted to keep it on the side. That sounds like a pretty poor workflow to me. If I tweak something for 3 years, I'm likely to have a larger set of logically organized commits. I'm not saying it's unlikely I'm going to rebase fixes for obvious mistakes but putting everything onto one blob just makes the changes harder to read and less maintainable. For example, if I first create a basic function and then add additional options to it, I'm likely to keep those as separate commits. If one of the changes was a really bad idea, I'd like to revert the appropriate commit rather than removing all traces of it by hand and mistakenly introducing even worse breakage. -- Best regards, Michał Górny signature.asc Description: PGP signature
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, May 31, 2012 at 3:33 PM, Michał Górny wrote: > What would git signing work with rebased commits? Would all of them > have to be signed once again? > The whole point of rebasing is to throw away history (which is either good or bad based on your perspective). So, if 14 devs spend 3 years and 2000 commits working on something in a branch, and I commit it to master using a rebase, then all you'll see in the master history is that rich0 committed 20k lines of code to master on May 31st, and that would be signed by me. I think that rebasing before merging is a pretty typical workflow anyway - when you submit a patch to Linus, he doesn't really care that you spent six months tweaking it - he just is getting a big blob of code that either works or doesn't. Does all that sub-history really matter? You could always push the branch to the repository if you wanted to keep it on the side. Rich
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, May 31, 2012 at 07:13:42PM +, Robin H. Johnson wrote: > - You have a commit, that you want to put into the Gentoo tree. > - You have already pushed it to your github, signed If I have a github tree, that would probably be because I didn't have push access to the official tree, so signing the commit probably isn'tgoing to matter; I would expect that a gentoo dev who has push access to the tree would sign the commit when it is put into the gentoo tree. > - It needs to be merged/rebased so that it applies on the Gentoo tree. > - If you force it to be a rebase so it applies on the tip, then you may > have changed the history of your github tree, and broken any further > forks. > - If you permit a merge instead, nobody gets broken. If you do this: git rebase master mybranch git checkout master git merge mybranch <-- this is a fast-forward merge git pull --rebase git push I think that covers this concern doesn't it? > > > 2. > > > Git-SVN breakage. Why does this matter you're wondering? > > > We need the newer Git for the commit signing, but it comes with a > > > price, the git-svn binary has some major failures with SVN 1.7. > > > Git since 1.7.8 has been broken this way. > > To clarify - these won't be issues for gentoo per se, but there is a > > sense that we can't stabilize the latest git because it will break it > > for people using git-svn on non-gentoo work? > As the Git maintainer, I will not keyword it for anybody until I know > it's not going to lose/corrupt data, regardless of what they are using > it for. Why not keyword it and use package.use.mask for the git-svn flag? William pgpwkzFYxv1GM.pgp Description: PGP signature
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
Michał Górny писал 2012-05-31 23:33: On Thu, 31 May 2012 14:18:04 -0500 William Hubbs wrote: On Thu, May 31, 2012 at 01:48:29PM -0400, Rich Freeman wrote: > On Thu, May 31, 2012 at 12:49 PM, Robin H. Johnson > wrote: > > 1. > > Discussion on merge policy. Originally I thought we would > > disallow merge commits, so that we would get a cleaner history. > > However, it turns out that if the repo ends up being pushed to > > different places with slightly different histories, merges are > > absolutely going to be required to prevent somebody from having > > to rebase at least one of their sets of commits that are already > > pushed. > > Not sure I'm following, but I will be the first to admit that I'm a > git novice. Would this be aided by a convention, like only > committing to master on the gentoo official repository, and any > on-the-side work on places like github/etc stays in branches? > Those repositories would just keep getting fed commits on master > from the official repository. Iagree with this; I think we should ban merge commits on master. That would force everyone to rebase their work on current master before they commit to master which would make the history clean. What would git signing work with rebased commits? Would all of them have to be signed once again? Commits itsels still will be signed -- Best Regards, Alexey 'Alexxy' Shvetsov Petersburg Nuclear Physics Institute, NRC Kurchatov Institute, Gatchina, Russia Department of Molecular and Radiation Biophysics Gentoo Team Ru Gentoo Linux Dev mailto:alexx...@gmail.com mailto:ale...@gentoo.org mailto:ale...@omrb.pnpi.spb.ru
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, 31 May 2012 14:18:04 -0500 William Hubbs wrote: > On Thu, May 31, 2012 at 01:48:29PM -0400, Rich Freeman wrote: > > On Thu, May 31, 2012 at 12:49 PM, Robin H. Johnson > > wrote: > > > 1. > > > Discussion on merge policy. Originally I thought we would > > > disallow merge commits, so that we would get a cleaner history. > > > However, it turns out that if the repo ends up being pushed to > > > different places with slightly different histories, merges are > > > absolutely going to be required to prevent somebody from having > > > to rebase at least one of their sets of commits that are already > > > pushed. > > > > Not sure I'm following, but I will be the first to admit that I'm a > > git novice. Would this be aided by a convention, like only > > committing to master on the gentoo official repository, and any > > on-the-side work on places like github/etc stays in branches? > > Those repositories would just keep getting fed commits on master > > from the official repository. > > Iagree with this; I think we should ban merge commits on master. That > would force everyone to rebase their work on current master before > they commit to master which would make the history clean. What would git signing work with rebased commits? Would all of them have to be signed once again? -- Best regards, Michał Górny signature.asc Description: PGP signature
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, May 31, 2012 at 3:13 PM, Robin H. Johnson wrote: > - You have a commit, that you want to put into the Gentoo tree. > - You have already pushed it to your github, signed > - It needs to be merged/rebased so that it applies on the Gentoo tree. > - If you force it to be a rebase so it applies on the tip, then you may > have changed the history of your github tree, and broken any further > forks. > - If you permit a merge instead, nobody gets broken. Maybe the best compromise is to tell people that if you push to "master" on other repositories, you get to deal with the mess. If we try to keep side overlays/etc working on branches and not on master then there will be no history to rewrite, as the merge will be rebased when it hits the official master, and from there it will get pulled by other repositories. We can perhaps allow merge commits on other branches, where the continuity of history is less important. Does that make sense? > You'd be excluding me entirely, I need to use git-svn for other work > projects, and emerging between two different versions of git would be > very annoying (I switch constantly between the sides of work as they > overlap). I'm a big proponent of letting the people doing the work scratch their own itches first! However, this does make us dependent on upstream - is there any sense of when they'll be ready, or what their own priority is for this issue. If this is becoming a deprecated feature then I'm not sure we can tie our future to it. I wasn't sure if any of the existing git-svn bugs pertained to this issue. Either way we should add this as a blocker to the git migration tracker. I think that even if we made a big push it would still take us a month or two to be ready with docs/infra/etc, and that might be optimistic. So, this might not be rate-limiting if upstream is actively working on it. Rich
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, 31 May 2012 14:18:04 -0500 William Hubbs wrote: > > Not sure I'm following, but I will be the first to admit that I'm a > > git novice. Would this be aided by a convention, like only > > committing to master on the gentoo official repository, and any > > on-the-side work on places like github/etc stays in branches? > > Those repositories would just keep getting fed commits on master > > from the official repository. > > Iagree with this; I think we should ban merge commits on master. That > would force everyone to rebase their work on current master before > they commit to master which would make the history clean. So what's the point of switching to git if you want to ban the main reason git exists? -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, May 31, 2012 at 01:48:29PM -0400, Rich Freeman wrote: > On Thu, May 31, 2012 at 12:49 PM, Robin H. Johnson wrote: > > 1. > > Discussion on merge policy. Originally I thought we would disallow merge > > commits, so that we would get a cleaner history. However, it turns out that > > if > > the repo ends up being pushed to different places with slightly different > > histories, merges are absolutely going to be required to prevent somebody > > from > > having to rebase at least one of their sets of commits that are already > > pushed. > > Not sure I'm following, but I will be the first to admit that I'm a > git novice. Would this be aided by a convention, like only committing > to master on the gentoo official repository, and any on-the-side work > on places like github/etc stays in branches? Those repositories would > just keep getting fed commits on master from the official repository. Iagree with this; I think we should ban merge commits on master. That would force everyone to rebase their work on current master before they commit to master which would make the history clean. William pgpA0ksxOrWmJ.pgp Description: PGP signature
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, May 31, 2012 at 01:48:29PM -0400, Rich Freeman wrote: > On Thu, May 31, 2012 at 12:49 PM, Robin H. Johnson wrote: > > 1. > > Discussion on merge policy. Originally I thought we would disallow merge > > commits, so that we would get a cleaner history. However, it turns out that > > if > > the repo ends up being pushed to different places with slightly different > > histories, merges are absolutely going to be required to prevent somebody > > from > > having to rebase at least one of their sets of commits that are already > > pushed. > Not sure I'm following, but I will be the first to admit that I'm a > git novice. Would this be aided by a convention, like only committing > to master on the gentoo official repository, and any on-the-side work > on places like github/etc stays in branches? Those repositories would > just keep getting fed commits on master from the official repository. Ok, let me try and reword my statement. - You have a commit, that you want to put into the Gentoo tree. - You have already pushed it to your github, signed - It needs to be merged/rebased so that it applies on the Gentoo tree. - If you force it to be a rebase so it applies on the tip, then you may have changed the history of your github tree, and broken any further forks. - If you permit a merge instead, nobody gets broken. > > 2. > > Git-SVN breakage. Why does this matter you're wondering? > > We need the newer Git for the commit signing, but it comes with a > > price, the git-svn binary has some major failures with SVN 1.7. > > Git since 1.7.8 has been broken this way. > To clarify - these won't be issues for gentoo per se, but there is a > sense that we can't stabilize the latest git because it will break it > for people using git-svn on non-gentoo work? As the Git maintainer, I will not keyword it for anybody until I know it's not going to lose/corrupt data, regardless of what they are using it for. I don't think there are many SVN repos left in Gentoo that haven't converted to using Git directly, so it's probably not a problem from that side. > If that is the case, what is our sense of how important this feature > even is to gentoo developers? They're the only ones who really have > to have the latest git to commit to the official tree. You'd be excluding me entirely, I need to use git-svn for other work projects, and emerging between two different versions of git would be very annoying (I switch constantly between the sides of work as they overlap). -- Robin Hugh Johnson Gentoo Linux: Developer, Trustee & Infrastructure Lead E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, May 31, 2012 at 12:49 PM, Robin H. Johnson wrote: > 1. > Discussion on merge policy. Originally I thought we would disallow merge > commits, so that we would get a cleaner history. However, it turns out that if > the repo ends up being pushed to different places with slightly different > histories, merges are absolutely going to be required to prevent somebody from > having to rebase at least one of their sets of commits that are already > pushed. Not sure I'm following, but I will be the first to admit that I'm a git novice. Would this be aided by a convention, like only committing to master on the gentoo official repository, and any on-the-side work on places like github/etc stays in branches? Those repositories would just keep getting fed commits on master from the official repository. > > 2. > Git-SVN breakage. Why does this matter you're wondering? > We need the newer Git for the commit signing, but it comes with a > price, the git-svn binary has some major failures with SVN 1.7. > Git since 1.7.8 has been broken this way. To clarify - these won't be issues for gentoo per se, but there is a sense that we can't stabilize the latest git because it will break it for people using git-svn on non-gentoo work? I think the general conclusion was that we would not be supporting any mixed git+cvs/svn/etc workflows for gentoo itself. If that is the case, what is our sense of how important this feature even is to gentoo developers? They're the only ones who really have to have the latest git to commit to the official tree. Rich
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, May 31, 2012 at 08:04:10AM -0400, Aaron W. Swenson wrote: > On 05/30/2012 04:31 PM, Dirkjan Ochtman wrote: > > On Wed, May 30, 2012 at 8:33 PM, Robin H. Johnson > > wrote: > >> No, the last mock conversion is still live and updating fairly > >> often: > >> http://git-exp.overlays.gentoo.org/gitweb/?p=exp/gentoo-x86.git;a=summary > The 6 hours it takes to clone the repo. To directly clone that repo yes. Which you should NEVER be doing in reality. The final conversion repo we have already stated will only be 40MB, with no history. History will be available separately to graft on if you need it, just like the Linux kernel did with historical data. -- Robin Hugh Johnson Gentoo Linux: Developer, Trustee & Infrastructure Lead E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Wed, May 30, 2012 at 10:31:06PM +0200, Dirkjan Ochtman wrote: > On Wed, May 30, 2012 at 8:33 PM, Robin H. Johnson wrote: > > No, the last mock conversion is still live and updating fairly often: > > http://git-exp.overlays.gentoo.org/gitweb/?p=exp/gentoo-x86.git;a=summary > Since you seem to know most about this project, can you give a short > summary on what *you* think are hard blockers for the migration? There are only two items from my perspective. 1. Discussion on merge policy. Originally I thought we would disallow merge commits, so that we would get a cleaner history. However, it turns out that if the repo ends up being pushed to different places with slightly different histories, merges are absolutely going to be required to prevent somebody from having to rebase at least one of their sets of commits that are already pushed. 2. Git-SVN breakage. Why does this matter you're wondering? We need the newer Git for the commit signing, but it comes with a price, the git-svn binary has some major failures with SVN 1.7. Git since 1.7.8 has been broken this way. You can see them best with the testsuite. I haven't keyworded those ebuilds for git at all, because git-svn in some of my tests ended up being destructive of the repository - it actually lost data. The error sometimes turns up like this. Initialized empty Git repository in /dev/shm/portage/dev-vcs/git-/work/git-/t/t d.t9155/git_project/.git/ svn: E235000: In file 'subversion/libsvn_subr/dirent_uri.c' line 2291: assertion failed (svn_uri_is_canonical(url, pool)) This is seemingly due to a change in SVN 1.7 that is stricter about all paths (both URIs and to files/directories) being properly escaped. Upstream Git has made no progress on it in more than 6 months. Both ferringb and I put several weeks of work into it without succeeding. If you want to reproduce it: - Upgrade your SVN to 1.7 - Build & run the git testsuite - Most of the t91xx tests will fail, because the working dir is 'trash directory'. - If you patch the working dir to be 'trash_directory', you'll be left with failures in at least: t9100 t9118 t9120 Because those are the tests that have whitespace or other characters that need escaping. The last version of my patch is files/git-1.7.8-git-svn-1.7-canonical-path.patch but it caused more problems than it fixed. -- Robin Hugh Johnson Gentoo Linux: Developer, Trustee & Infrastructure Lead E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, May 31, 2012 at 2:04 PM, Aaron W. Swenson wrote: > The 6 hours it takes to clone the repo. IIRC someone already proposed that the packed repo could be offered via normal download (or even BitTorrent). Cheers, Dirkjan
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
Aaron W. Swenson wrote: > > what *you* think are hard blockers for the migration? > > The 6 hours it takes to clone the repo. Maybe clone on server and distribute the initial repo as tarball. //Peter
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 05/30/2012 04:31 PM, Dirkjan Ochtman wrote: > On Wed, May 30, 2012 at 8:33 PM, Robin H. Johnson > wrote: >> No, the last mock conversion is still live and updating fairly >> often: >> http://git-exp.overlays.gentoo.org/gitweb/?p=exp/gentoo-x86.git;a=summary > >> > Since you seem to know most about this project, can you give a > short summary on what *you* think are hard blockers for the > migration? > > Cheers, > > Dirkjan > The 6 hours it takes to clone the repo. - - Aaron -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.17 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iF4EAREIAAYFAk/HXjoACgkQVxOqA9G7/aB2DwD+JqdcMJDIfu30Oht8rB6H/pMY Wr1RjhujZvVGIDQG55QA/jPlsWh3c8El7HjxzhjoCNIPkV1Vj3b7VZVc/D6y6oq9 =FxGY -END PGP SIGNATURE-
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Wed, May 30, 2012 at 8:33 PM, Robin H. Johnson wrote: > No, the last mock conversion is still live and updating fairly often: > http://git-exp.overlays.gentoo.org/gitweb/?p=exp/gentoo-x86.git;a=summary Since you seem to know most about this project, can you give a short summary on what *you* think are hard blockers for the migration? Cheers, Dirkjan
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Wed, May 30, 2012 at 01:06:45PM +, Duncan wrote: > Of course, those previous trial runs are probably similarly dated, now, > so a new one's probably in order with the new perspective on those bug > priorities, etc, but at least getting the input of someone that was > involved with them should speed progress over some already covered > ground, in any case. No, the last mock conversion is still live and updating fairly often: http://git-exp.overlays.gentoo.org/gitweb/?p=exp/gentoo-x86.git;a=summary -- Robin Hugh Johnson Gentoo Linux: Developer, Trustee & Infrastructure Lead E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On 30.05.2012 18:58, Aaron W. Swenson wrote: > On 05/30/2012 12:53 PM, Tobias Klausmann wrote: >> Hi! > >> On Wed, 30 May 2012, Rich Freeman wrote: >>> On Wed, May 30, 2012 at 6:16 AM, Dirkjan Ochtman >>> wrote: Yeah... this is why I was asking about access to infra to test the conversion; so far, I haven't had any replies, though. >>> >>> A mock conversion would probably help with creating >>> procedures/docs/etc as well. It is nice to say that we're "just >>> going to use git" but I think everybody has a slightly different >>> picture of how that is going to work. > >> I recommend having a smallish set of willing alpha/beta testers >> for this. This usually helps with some of the near-edge cases. >> You'll still find a thousand other bugs once things go live for >> everybody. Still, it turns a million into a thousand. It also gives >> you slightly more realistic load test. Regards, Tobias > > > As one that's relatively new to Git, I'm a perfect candidate for > testing. If we go this way, I volunteer. > > - Aaron > As I am a hater of CVS and my current checkout is repeatability spiting weird errors, I would join as a tester here. justin signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 05/30/2012 12:53 PM, Tobias Klausmann wrote: > Hi! > > On Wed, 30 May 2012, Rich Freeman wrote: >> On Wed, May 30, 2012 at 6:16 AM, Dirkjan Ochtman >> wrote: >>> Yeah... this is why I was asking about access to infra to test >>> the conversion; so far, I haven't had any replies, though. >> >> A mock conversion would probably help with creating >> procedures/docs/etc as well. It is nice to say that we're "just >> going to use git" but I think everybody has a slightly different >> picture of how that is going to work. > > I recommend having a smallish set of willing alpha/beta testers > for this. This usually helps with some of the near-edge cases. > You'll still find a thousand other bugs once things go live for > everybody. Still, it turns a million into a thousand. It also gives > you slightly more realistic load test. Regards, Tobias > As one that's relatively new to Git, I'm a perfect candidate for testing. If we go this way, I volunteer. - - Aaron -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.17 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iF4EAREIAAYFAk/GUZkACgkQVxOqA9G7/aDxogEAjEsS6k0M9TejvCAoYISvuk1u odo5ecLsyshwvwRlk3sA+wUeVAfGrafgH1gKpSrEedk1OT5x7HfsMquK0owq0/Np =mzDU -END PGP SIGNATURE-
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
Hi! On Wed, 30 May 2012, Rich Freeman wrote: > On Wed, May 30, 2012 at 6:16 AM, Dirkjan Ochtman wrote: > > Yeah... this is why I was asking about access to infra to test the > > conversion; so far, I haven't had any replies, though. > > A mock conversion would probably help with creating > procedures/docs/etc as well. It is nice to say that we're "just going > to use git" but I think everybody has a slightly different picture of > how that is going to work. I recommend having a smallish set of willing alpha/beta testers for this. This usually helps with some of the near-edge cases. You'll still find a thousand other bugs once things go live for everybody. Still, it turns a million into a thousand. It also gives you slightly more realistic load test. > If we could set up an "official unofficial" portage tree in git based > on a one-time migration (maybe refreshing it from time to time) that > could be a sandbox used to work things out, and it would then be > replaced with the official tree. When the official migration comes > along we'd already be experts in doing it. This is a good idea that goes nicely with what I wrote above. > All we need to do is execute the migration, and just not point the > rsync generation process at it. Maybe it won't be perfectly right at > first, and that would basically be the point of doing it. Devs could > update tools to work against it, and the docs could be written > alongside. The scientist in me wonders how big the dent in productivity will be, actually. After all, there's going to be a lot of people that will hammer the new setup just because of the New! Shiny! appeal. Regards, Tobias
[gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
Rich Freeman posted on Wed, 30 May 2012 07:32:49 -0400 as excerpted: > On Wed, May 30, 2012 at 6:16 AM, Dirkjan Ochtman wrote: >> Yeah... this is why I was asking about access to infra to test the >> conversion; so far, I haven't had any replies, though. >> >> > A mock conversion would probably help with creating procedures/docs/etc > as well. It is nice to say that we're "just going to use git" but I > think everybody has a slightly different picture of how that is going to > work. I /believe/ such mock conversions had been done, based on previous threads here, tho NOT on the SCM-migration list, which would certainly have the greater detail. AFAIK the previous sticking points were all those still-open bugs, not the general conversion. But the bugs, other than documentation, either seem mostly fixed or not so important, any more. Of course, those previous trial runs are probably similarly dated, now, so a new one's probably in order with the new perspective on those bug priorities, etc, but at least getting the input of someone that was involved with them should speed progress over some already covered ground, in any case. (I've considered following scmm myself, but there's apparently some sort of issue between gmane, pan, and lists commonly crossposted between, that has already killed project for me, header-fetch does nothing, and scmm would almost certainly be similarly dead to me.) -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Wed, May 30, 2012 at 6:16 AM, Dirkjan Ochtman wrote: > Yeah... this is why I was asking about access to infra to test the > conversion; so far, I haven't had any replies, though. > A mock conversion would probably help with creating procedures/docs/etc as well. It is nice to say that we're "just going to use git" but I think everybody has a slightly different picture of how that is going to work. If we could set up an "official unofficial" portage tree in git based on a one-time migration (maybe refreshing it from time to time) that could be a sandbox used to work things out, and it would then be replaced with the official tree. When the official migration comes along we'd already be experts in doing it. All we need to do is execute the migration, and just not point the rsync generation process at it. Maybe it won't be perfectly right at first, and that would basically be the point of doing it. Devs could update tools to work against it, and the docs could be written alongside. Rich
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Wed, May 30, 2012 at 11:38 AM, Duncan <1i5t5.dun...@cox.net> wrote: > Of course, there's a much larger infra component to the git migration, so > either having that someone being an infra person, or at least having > someone from infra have the time and be willing to work closely with > them, is going to be critical. But again, given a council "*priority*, > let's move on it!" decision, I'd at least /hope/ that's not a blocker. Yeah... this is why I was asking about access to infra to test the conversion; so far, I haven't had any replies, though. Cheers, Dirkjan
[gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
Rich Freeman posted on Tue, 29 May 2012 21:55:04 -0400 as excerpted: > So, what is the big issue? Is there something not being tracked, or is > one of those items a lot harder than it looks? I'd suggest that it's like openrc stabilization. The biggest problem with it is that it's a BIG job, and the simplest solution is to simply have someone commit to it, with full council *priority* backing, and push and push until it's done. There *is* one huge don't-do-it-that-way lesson to take from the openrc stabilization, tho: Get the documentation in place BEFORE "throwing the switch". I'm still not sure what happened with openrc. The documentation bug was a blocker... until it was the last one and then suddenly it went live without proper upgrade documentation, or that's the way it seemed from here, anyway. The fact that we had essentially no doc- project to work on the documentation was unfortunately a problem, the one guy still trying to hang on in docs so backlogged and burnt out that it was about hopeless, action time stretching toward infinity, but swift coming back on board dramatically improved that situation so at least it shouldn't be an infinity blocker, now. But really what that means is that whoever ends up taking charge of that final push, needs to be prepared to learn gentoo's docs CSS definitions and do it themself, if it comes to that. Meanwhile, the positive takeaway from the openrc stabilization is that someone suitably determined, along with council backing and everyone else rowing the same way where their little part of gentoo comes into contact with the job at hand, goes a long way! Of course, there's a much larger infra component to the git migration, so either having that someone being an infra person, or at least having someone from infra have the time and be willing to work closely with them, is going to be critical. But again, given a council "*priority*, let's move on it!" decision, I'd at least /hope/ that's not a blocker. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 05/24/2012 03:37 PM, Kent Fredric wrote: Kent, this is of topic, stop it. - -- - -- Gentoo Dev http://xmw.de/ -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.17 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iF4EAREIAAYFAk++PO8ACgkQknrdDGLu8JC2cAD/WknV6DJEzYEsuXJD0TuDU99I arDTkrBHNXydYVdaxGoBAJmmVm3o7YWlMAvFBz2Eu4ma/VXEHdqocFwl0NK+FEAh =Esu3 -END PGP SIGNATURE-
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On 25 May 2012 00:05, Dirkjan Ochtman wrote: > On Thu, May 24, 2012 at 1:43 PM, Duncan <1i5t5.dun...@cox.net> wrote: >> In that regard, git is nothing like for instance svn, where branches come >> at a much higher cost, as does merging between them. > > That's wrong. SVN branches are just about as cheap as git branches, > although merges used to be much more painful. I'm not sure how good > merging in recent SVN is. Cheapness ... maybe in binary disk utilization ( need an actual comparison here I think ), but in cognitive overheads, I'd argue git's branching system is definitely cheaper. Going from Git back to SVN, the mentality of "copy a directory and you have a new branch!!!" seems a bit crazy. And switching between branches in-place at a fixed disk location is definitely cheaper ( mentally at least ) than SVN. I hope I never have to use svn switch again :/ -- Kent perl -e "print substr( \"edrgmaM SPA NOcomil.ic\\@tfrken\", \$_ * 3, 3 ) for ( 9,8,0,7,1,6,5,4,3,2 );" http://kent-fredric.fox.geek.nz
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, 24 May 2012 14:05:50 +0200 Dirkjan Ochtman wrote: > On Thu, May 24, 2012 at 1:43 PM, Duncan <1i5t5.dun...@cox.net> wrote: > > In that regard, git is nothing like for instance svn, where > > branches come at a much higher cost, as does merging between them. > > That's wrong. SVN branches are just about as cheap as git branches, [citation needed] -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, May 24, 2012 at 1:43 PM, Duncan <1i5t5.dun...@cox.net> wrote: > In that regard, git is nothing like for instance svn, where branches come > at a much higher cost, as does merging between them. That's wrong. SVN branches are just about as cheap as git branches, although merges used to be much more painful. I'm not sure how good merging in recent SVN is. Let's please stay a little on-topic? The migration will get there much faster if we don't succumb to feature creep. Cheers, Dirkjan
[gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
Dan Douglas posted on Thu, 24 May 2012 01:51:28 -0500 as excerpted: > I don't think doing a branch of the entire tree is a good idea (well > maybe...). I was thinking more along the lines of subtree merges into a > local overlay, or perhaps submodules. To do that currently (I think) > would require taking the rsync tree and putting that into a repo, and > trying to keep it synchronized. Plus in the process you lose all > correspondance with upstream commits so that logs and diffs become > meaningless. The thing about git branches is that they cost almost nothing. If the whole main tree is a single git tree, and you already have that available, maintaining a branch consisting of what we now call an overlay is trivial, compared to simply maintaining the separate files, as is normally done now. In that regard, git is nothing like for instance svn, where branches come at a much higher cost, as does merging between them. (CVS... I don't actually know enough about to make an informed comparison. It'd be a real shame not to expose the read-only git tree to the users who want it. Git was /designed/ to be distributed in that manner. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
2012/5/24 Dan Douglas > On Thursday, May 24, 2012 06:33:53 AM Duncan wrote: > > Dan Douglas posted on Thu, 24 May 2012 01:04:48 -0500 as excerpted: > > > On Thursday, May 24, 2012 07:56:58 AM Michał Górny wrote: > > >> On Wed, 23 May 2012 16:14:53 -0500 > > >> > > >> Dan Douglas wrote: > > >> > If not I will be leaving Gentoo for Funtoo in the near future, > though > > >> > there are disadvantages to doing this I don't look forward to > dealing > > >> > with. > > >> > > >> Most of us will probably be doing that :P. > > > > > > Eh sorry that wasn't meant to be antagonistic. I'll still have Gentoo > > > boxen to deal with. I just need to be able to use git on the tree (even > > > without the full history is perfectly fine) to ease the difficulty of > > > local overlay management. Glad to hear that will be possible, or at > > > least somewhat easier. > > > > FWIW, I as a user would sure like a git-based tree. Doing git > whatchanged > > searches on individual files and being able to track my last checkout and > > roll back to it, or to a point between it and current HEAD, are extremely > > useful. I haven't thought of it much until now, but I think maintaining > > overlays as simple branches would be great, as well. > > I don't think doing a branch of the entire tree is a good idea (well > maybe...). I was thinking more along the lines of subtree merges into a > local > overlay, or perhaps submodules. To do that currently (I think) would > require > taking the rsync tree and putting that into a repo, and trying to keep it > synchronized. Plus in the process you lose all correspondance with upstream > commits so that logs and diffs become meaningless. > -- > Dan Douglas git++ -- Vítor Brandão (noisebleed)
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thursday, May 24, 2012 06:33:53 AM Duncan wrote: > Dan Douglas posted on Thu, 24 May 2012 01:04:48 -0500 as excerpted: > > On Thursday, May 24, 2012 07:56:58 AM Michał Górny wrote: > >> On Wed, 23 May 2012 16:14:53 -0500 > >> > >> Dan Douglas wrote: > >> > If not I will be leaving Gentoo for Funtoo in the near future, though > >> > there are disadvantages to doing this I don't look forward to dealing > >> > with. > >> > >> Most of us will probably be doing that :P. > > > > Eh sorry that wasn't meant to be antagonistic. I'll still have Gentoo > > boxen to deal with. I just need to be able to use git on the tree (even > > without the full history is perfectly fine) to ease the difficulty of > > local overlay management. Glad to hear that will be possible, or at > > least somewhat easier. > > FWIW, I as a user would sure like a git-based tree. Doing git whatchanged > searches on individual files and being able to track my last checkout and > roll back to it, or to a point between it and current HEAD, are extremely > useful. I haven't thought of it much until now, but I think maintaining > overlays as simple branches would be great, as well. I don't think doing a branch of the entire tree is a good idea (well maybe...). I was thinking more along the lines of subtree merges into a local overlay, or perhaps submodules. To do that currently (I think) would require taking the rsync tree and putting that into a repo, and trying to keep it synchronized. Plus in the process you lose all correspondance with upstream commits so that logs and diffs become meaningless. -- Dan Douglas signature.asc Description: This is a digitally signed message part.
[gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
Dan Douglas posted on Thu, 24 May 2012 01:04:48 -0500 as excerpted: > On Thursday, May 24, 2012 07:56:58 AM Michał Górny wrote: >> On Wed, 23 May 2012 16:14:53 -0500 >> >> Dan Douglas wrote: >> > If not I will be leaving Gentoo for Funtoo in the near future, though >> > there are disadvantages to doing this I don't look forward to dealing >> > with. >> >> Most of us will probably be doing that :P. > > Eh sorry that wasn't meant to be antagonistic. I'll still have Gentoo > boxen to deal with. I just need to be able to use git on the tree (even > without the full history is perfectly fine) to ease the difficulty of > local overlay management. Glad to hear that will be possible, or at > least somewhat easier. FWIW, I as a user would sure like a git-based tree. Doing git whatchanged searches on individual files and being able to track my last checkout and roll back to it, or to a point between it and current HEAD, are extremely useful. I haven't thought of it much until now, but I think maintaining overlays as simple branches would be great, as well. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
[gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On 2012-05-23 22:42, Michael Weber wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi, i've looked at the blockers of "[TRACKER] portage migration to git" [1] and want to discuss "testing git-cvsserver" [2]. There are two proposed scenarios how to migrate the developers write access to the portage tree. "Clean cut" turns of cvs access on a given and announced timestamp, rsync-generation/updates is suspended (no input -> no changes), some magic scripts prepare the git repo (according to [3], some hours duration) and we all checkout the tree (might be some funny massive load). "testing git-cvsserver" proses "Clean cut" with the additional ability to continue using cvs update/commit, - in best case - on the old checkout w/o alteration on the developers side. "Clean cut" forces us to clean up out dirty checkouts (I have some added directories, added ebuilds i hesitated to `repoman commit`). Plus we have to alter all our hot-wired portage mangling scripts from cvs'ish to git'ish (I use my read/write checkout as portage tree (cvs checkout + egencache for checkout) and have an automated google-chrome bump script). But this can be accomplished on a per developer basis, and slackers don't stall the process. "testing git-cvsserver" forces us all to test these cvs'ish scripts and behaviours against a git-cvsserver and report. We all know that this test-runs will never happen, stalling this bug till infinity. Plus infra/"subset of devs marshalling the migration" get stuck between fixing git issues and git-cvsserver. *if you still read this* *wow* Please discuss my arguments and come to the conclusions to RESO/WONT-FIX "testing git-cvsserver", make a "clean cut" and remove this bug from the blockers of "[TRACKER] portage migration to git". My lengthy 2 cents. [1] https://bugs.gentoo.org/333531 [2] https://bugs.gentoo.org/333699 [3] https://bugs.gentoo.org/333705#c2 - -- Gentoo Dev http://xmw.de/ -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.17 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iF4EAREIAAYFAk+82z0ACgkQknrdDGLu8JBUWAD/dmuqyES/mYDrMam+/txnHmgd VaQaqwHMlwzzqQwbpY4A/0h+5Vp8sLbOE78k4SCaGE2dCQtmeOz0jd1YxkDzP+YW =jXLQ -END PGP SIGNATURE- Another vote for clean cut.