Re: [gentoo-dev] [PATCH eutils] Move remove_libtool_files() from autotools-utils for wider use.
On Thursday 31 May 2012 01:46:41 Michał Górny wrote:
On Wed, 30 May 2012 17:19:49 -0400 Mike Frysinger wrote:
On Monday 28 May 2012 03:58:56 Michał Górny wrote:
+# @USAGE: [all]
this is incorrect. the usage is:
all | files to remove
No, it's perfectly valid. Moreover, it even explains what the function
actually does rather than your imagination.
why are you so angry all the time ? try being less confrontational for once.
going from the usage:
remove_libtool_files [all]
that means this may be called in only two ways:
1) remove_libtool_files
2) remove_libtool_files all
yet, if you read the actual code, you'll see:
+ [[ ${#} -le 1 ]] || die Invalid number of args to ${FUNCNAME}()
+ if [[ ${#} -eq 1 ]]; then
+ ...
+ fi
that means if more than 1 argument is passed, no error is thrown. i thought
you were intending to parse $@ further on because of it (hence the suggestion
of updating the @USAGE), but it looks merely like your arg parsing is
incorrect and needs fixing. probably easiest by doing:
case $#:$1 in
0:'') ;;
1:all) removing_all=1 ;;
*) die invalid usage ;;
esac
although, since we don't call die or anything, we can pipeline it to
speed things up a bit:
pc_libs=( $(
tpc=${T}/.pc
find ${D} -name '*.pc' -type f | \
while read pc ; do
sed -e '/^Requires:/d' ${pc} ${tpc}
$(tc-getPKG_CONFIG) --libs ${tpc}
done | tr ' ' '\n' | sort -u | \
sed -n '/^-l/{s:^-l:lib:;s:$:.la:;p}'
rm -f ${tpc}
) )
Could you remind me, please, what performance-critical use of this
function does justify making it so harsh?
looks perfectly fine to me, and it has the bonus of working
+ rm -f ${archivefile} || die
`rm -f` almost never fails. in the edge cases where it does, you've
got bigger problems.
And that problem is good enough to die here.
more like the system at large is going to be falling over independently
-mike
signature.asc
Description: This is a digitally signed message part.
Re: [gentoo-dev] [PATCH eutils] Move remove_libtool_files() from autotools-utils for wider use.
On Thu, 31 May 2012 02:09:11 -0400
Mike Frysinger vap...@gentoo.org wrote:
yet, if you read the actual code, you'll see:
+ [[ ${#} -le 1 ]] || die Invalid number of args to
${FUNCNAME}()
+ if [[ ${#} -eq 1 ]]; then
+ ...
+ fi
that means if more than 1 argument is passed, no error is thrown.
The exact opposite. If more than a single argument is passed, error is
thrown.
thought you were intending to parse $@ further on because of it
(hence the suggestion of updating the @USAGE), but it looks merely
like your arg parsing is incorrect and needs fixing. probably
easiest by doing: case $#:$1 in
0:'') ;;
1:all) removing_all=1 ;;
*) die invalid usage ;;
esac
Just a little reverse logic in spirit of makefiles. But the case
variant would be probably more readable indeed.
--
Best regards,
Michał Górny
signature.asc
Description: PGP signature
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 05/30/2012 04:31 PM, Dirkjan Ochtman wrote: On Wed, May 30, 2012 at 8:33 PM, Robin H. Johnson robb...@gentoo.org wrote: No, the last mock conversion is still live and updating fairly often: http://git-exp.overlays.gentoo.org/gitweb/?p=exp/gentoo-x86.git;a=summary Since you seem to know most about this project, can you give a short summary on what *you* think are hard blockers for the migration? Cheers, Dirkjan The 6 hours it takes to clone the repo. - - Aaron -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.17 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iF4EAREIAAYFAk/HXjoACgkQVxOqA9G7/aB2DwD+JqdcMJDIfu30Oht8rB6H/pMY Wr1RjhujZvVGIDQG55QA/jPlsWh3c8El7HjxzhjoCNIPkV1Vj3b7VZVc/D6y6oq9 =FxGY -END PGP SIGNATURE-
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
Aaron W. Swenson wrote: what *you* think are hard blockers for the migration? The 6 hours it takes to clone the repo. Maybe clone on server and distribute the initial repo as tarball. //Peter
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, May 31, 2012 at 2:04 PM, Aaron W. Swenson titanof...@gentoo.org wrote: The 6 hours it takes to clone the repo. IIRC someone already proposed that the packed repo could be offered via normal download (or even BitTorrent). Cheers, Dirkjan
[gentoo-dev] [PATCH eutils] Introduce prune_libtool_files().
A function which determines correct .la files for removal and removes
them.
---
gx86/eclass/eutils.eclass | 92 -
1 file changed, 91 insertions(+), 1 deletion(-)
diff --git a/gx86/eclass/eutils.eclass b/gx86/eclass/eutils.eclass
index c88ef35..b0399ac 100644
--- a/gx86/eclass/eutils.eclass
+++ b/gx86/eclass/eutils.eclass
@@ -18,7 +18,7 @@
if [[ ${___ECLASS_ONCE_EUTILS} != recur -_+^+_- spank ]] ; then
___ECLASS_ONCE_EUTILS=recur -_+^+_- spank
-inherit multilib user
+inherit multilib toolchain-funcs user
DESCRIPTION=Based on the ${ECLASS} eclass
@@ -1330,6 +1330,96 @@ makeopts_jobs() {
echo ${jobs:-1}
}
+# @FUNCTION: prune_libtool_files
+# @USAGE: [--all]
+# @DESCRIPTION:
+# Locate unnecessary libtool files (.la) and libtool static archives
+# (.a) and remove them from installation image.
+#
+# By default, .la files are removed whenever the static linkage can
+# either be performed using pkg-config or doesn't introduce additional
+# flags.
+#
+# If '--all' argument is passed, all .la files are removed. This is
+# usually useful when the package installs plugins and does not use .la
+# files for loading them.
+#
+# The .a files are only removed whenever corresponding .la files state
+# that they should not be linked to, i.e. whenever these files
+# correspond to plugins.
+#
+# Note: this function implicitly calls pkg-config. You should add it to
+# your DEPEND when using it.
+prune_libtool_files() {
+ debug-print-function ${FUNCNAME} $@
+
+ local removing_all opt
+ for opt; do
+ case ${opt} in
+ --all)
+ removing_all=1
+ ;;
+ *)
+ die Invalid argument to ${FUNCNAME}(): ${opt}
+ esac
+ done
+
+ # Create a list of all .pc-covered libs.
+ local pc_libs=()
+ if [[ ! ${removing_all} ]]; then
+ local f
+ local tf=${T}/prune-lt-files.pc
+ local pkgconf=$(tc-getPKG_CONFIG)
+
+ while IFS= read -r -d '' f; do # for all .pc files
+ sed -e '/^Requires:/d' ${f} ${tf}
+ for arg in $(${pkgconf} --libs ${tf}); do
+ [[ ${arg} == -l* ]] pc_libs+=(
lib${arg#-l}.la )
+ done
+ done (find ${D} -type f -name '*.pc' -print0)
+ fi
+
+ local f
+ while IFS= read -r -d '' f; do # for all .la files
+ local archivefile=${f/%.la/.a}
+
+ [[ ${f} != ${archivefile} ]] || die 'regex sanity check failed'
+
+ # Remove static libs we're not supposed to link against.
+ if grep -q '^shouldnotlink=yes$' ${f}; then
+ einfo Removing unnecessary ${archivefile#${D%/}}
+ rm -f ${archivefile}
+
+ # The .la file may be used by a module loader, so avoid
removing it
+ # unless explicitly requested.
+ [[ ${removing_all} ]] || continue
+ fi
+
+ # Remove .la files when:
+ # - user explicitly wants us to remove all .la files,
+ # - respective static archive doesn't exist,
+ # - they are covered by a .pc file already,
+ # - they don't provide any new information (no libs no flags).
+ local reason
+ if [[ ${removing_all} ]]; then
+ reason='requested'
+ elif [[ ! -f ${archivefile} ]]; then
+ reason='no static archive'
+ elif has ${f##*/} ${pc_libs[@]}; then
+ reason='covered by .pc'
+ elif [[ ! $(sed -nre \
+
s/^(dependency_libs|inherited_linker_flags)='(.*)'$/\2/p \
+ ${f}) ]]; then
+ reason='no libs flags'
+ fi
+
+ if [[ ${reason} ]]; then
+ einfo Removing unnecessary ${f#${D%/}} (${reason})
+ rm -f ${f}
+ fi
+ done (find ${D} -type f -name '*.la' -print0)
+}
+
check_license() { die you no longer need this as portage supports
ACCEPT_LICENSE itself; }
fi
--
1.7.10.2
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Wed, May 30, 2012 at 10:31:06PM +0200, Dirkjan Ochtman wrote: On Wed, May 30, 2012 at 8:33 PM, Robin H. Johnson robb...@gentoo.org wrote: No, the last mock conversion is still live and updating fairly often: http://git-exp.overlays.gentoo.org/gitweb/?p=exp/gentoo-x86.git;a=summary Since you seem to know most about this project, can you give a short summary on what *you* think are hard blockers for the migration? There are only two items from my perspective. 1. Discussion on merge policy. Originally I thought we would disallow merge commits, so that we would get a cleaner history. However, it turns out that if the repo ends up being pushed to different places with slightly different histories, merges are absolutely going to be required to prevent somebody from having to rebase at least one of their sets of commits that are already pushed. 2. Git-SVN breakage. Why does this matter you're wondering? We need the newer Git for the commit signing, but it comes with a price, the git-svn binary has some major failures with SVN 1.7. Git since 1.7.8 has been broken this way. You can see them best with the testsuite. I haven't keyworded those ebuilds for git at all, because git-svn in some of my tests ended up being destructive of the repository - it actually lost data. The error sometimes turns up like this. Initialized empty Git repository in /dev/shm/portage/dev-vcs/git-/work/git-/t/t d.t9155/git_project/.git/ svn: E235000: In file 'subversion/libsvn_subr/dirent_uri.c' line 2291: assertion failed (svn_uri_is_canonical(url, pool)) This is seemingly due to a change in SVN 1.7 that is stricter about all paths (both URIs and to files/directories) being properly escaped. Upstream Git has made no progress on it in more than 6 months. Both ferringb and I put several weeks of work into it without succeeding. If you want to reproduce it: - Upgrade your SVN to 1.7 - Build run the git testsuite - Most of the t91xx tests will fail, because the working dir is 'trash directory'. - If you patch the working dir to be 'trash_directory', you'll be left with failures in at least: t9100 t9118 t9120 Because those are the tests that have whitespace or other characters that need escaping. The last version of my patch is files/git-1.7.8-git-svn-1.7-canonical-path.patch but it caused more problems than it fixed. -- Robin Hugh Johnson Gentoo Linux: Developer, Trustee Infrastructure Lead E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, May 31, 2012 at 08:04:10AM -0400, Aaron W. Swenson wrote: On 05/30/2012 04:31 PM, Dirkjan Ochtman wrote: On Wed, May 30, 2012 at 8:33 PM, Robin H. Johnson robb...@gentoo.org wrote: No, the last mock conversion is still live and updating fairly often: http://git-exp.overlays.gentoo.org/gitweb/?p=exp/gentoo-x86.git;a=summary The 6 hours it takes to clone the repo. To directly clone that repo yes. Which you should NEVER be doing in reality. The final conversion repo we have already stated will only be 40MB, with no history. History will be available separately to graft on if you need it, just like the Linux kernel did with historical data. -- Robin Hugh Johnson Gentoo Linux: Developer, Trustee Infrastructure Lead E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
Re: [gentoo-dev] Should packages auto-eselect alternative implementation on removal?
Michał Górny wrote: There is a number of virtuals in Gentoo which switching active implementation via eselect. However, most of the packages being 'alternative providers' don't seem to care about eselect at all. Is that the correct thing to do, or maybe should every package ensure that after its removal another implementation is eselected (or a cleanup is done)? we have been using for quite a while now the alternatives-2 eclass in the science overlay for many virtual packages. the nice thing about it apart from implementing the issues you raised is we do not have to write a new eselect package for the virtual. sebastien signature.asc Description: PGP signature
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, May 31, 2012 at 12:49 PM, Robin H. Johnson robb...@gentoo.org wrote: 1. Discussion on merge policy. Originally I thought we would disallow merge commits, so that we would get a cleaner history. However, it turns out that if the repo ends up being pushed to different places with slightly different histories, merges are absolutely going to be required to prevent somebody from having to rebase at least one of their sets of commits that are already pushed. Not sure I'm following, but I will be the first to admit that I'm a git novice. Would this be aided by a convention, like only committing to master on the gentoo official repository, and any on-the-side work on places like github/etc stays in branches? Those repositories would just keep getting fed commits on master from the official repository. 2. Git-SVN breakage. Why does this matter you're wondering? We need the newer Git for the commit signing, but it comes with a price, the git-svn binary has some major failures with SVN 1.7. Git since 1.7.8 has been broken this way. To clarify - these won't be issues for gentoo per se, but there is a sense that we can't stabilize the latest git because it will break it for people using git-svn on non-gentoo work? I think the general conclusion was that we would not be supporting any mixed git+cvs/svn/etc workflows for gentoo itself. If that is the case, what is our sense of how important this feature even is to gentoo developers? They're the only ones who really have to have the latest git to commit to the official tree. Rich
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, May 31, 2012 at 01:48:29PM -0400, Rich Freeman wrote: On Thu, May 31, 2012 at 12:49 PM, Robin H. Johnson robb...@gentoo.org wrote: 1. Discussion on merge policy. Originally I thought we would disallow merge commits, so that we would get a cleaner history. However, it turns out that if the repo ends up being pushed to different places with slightly different histories, merges are absolutely going to be required to prevent somebody from having to rebase at least one of their sets of commits that are already pushed. Not sure I'm following, but I will be the first to admit that I'm a git novice. Would this be aided by a convention, like only committing to master on the gentoo official repository, and any on-the-side work on places like github/etc stays in branches? Those repositories would just keep getting fed commits on master from the official repository. Ok, let me try and reword my statement. - You have a commit, that you want to put into the Gentoo tree. - You have already pushed it to your github, signed - It needs to be merged/rebased so that it applies on the Gentoo tree. - If you force it to be a rebase so it applies on the tip, then you may have changed the history of your github tree, and broken any further forks. - If you permit a merge instead, nobody gets broken. 2. Git-SVN breakage. Why does this matter you're wondering? We need the newer Git for the commit signing, but it comes with a price, the git-svn binary has some major failures with SVN 1.7. Git since 1.7.8 has been broken this way. To clarify - these won't be issues for gentoo per se, but there is a sense that we can't stabilize the latest git because it will break it for people using git-svn on non-gentoo work? As the Git maintainer, I will not keyword it for anybody until I know it's not going to lose/corrupt data, regardless of what they are using it for. I don't think there are many SVN repos left in Gentoo that haven't converted to using Git directly, so it's probably not a problem from that side. If that is the case, what is our sense of how important this feature even is to gentoo developers? They're the only ones who really have to have the latest git to commit to the official tree. You'd be excluding me entirely, I need to use git-svn for other work projects, and emerging between two different versions of git would be very annoying (I switch constantly between the sides of work as they overlap). -- Robin Hugh Johnson Gentoo Linux: Developer, Trustee Infrastructure Lead E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, May 31, 2012 at 01:48:29PM -0400, Rich Freeman wrote: On Thu, May 31, 2012 at 12:49 PM, Robin H. Johnson robb...@gentoo.org wrote: 1. Discussion on merge policy. Originally I thought we would disallow merge commits, so that we would get a cleaner history. However, it turns out that if the repo ends up being pushed to different places with slightly different histories, merges are absolutely going to be required to prevent somebody from having to rebase at least one of their sets of commits that are already pushed. Not sure I'm following, but I will be the first to admit that I'm a git novice. Would this be aided by a convention, like only committing to master on the gentoo official repository, and any on-the-side work on places like github/etc stays in branches? Those repositories would just keep getting fed commits on master from the official repository. Iagree with this; I think we should ban merge commits on master. That would force everyone to rebase their work on current master before they commit to master which would make the history clean. William pgpA0ksxOrWmJ.pgp Description: PGP signature
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, 31 May 2012 14:18:04 -0500 William Hubbs willi...@gentoo.org wrote: Not sure I'm following, but I will be the first to admit that I'm a git novice. Would this be aided by a convention, like only committing to master on the gentoo official repository, and any on-the-side work on places like github/etc stays in branches? Those repositories would just keep getting fed commits on master from the official repository. Iagree with this; I think we should ban merge commits on master. That would force everyone to rebase their work on current master before they commit to master which would make the history clean. So what's the point of switching to git if you want to ban the main reason git exists? -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, May 31, 2012 at 3:13 PM, Robin H. Johnson robb...@gentoo.org wrote: - You have a commit, that you want to put into the Gentoo tree. - You have already pushed it to your github, signed - It needs to be merged/rebased so that it applies on the Gentoo tree. - If you force it to be a rebase so it applies on the tip, then you may have changed the history of your github tree, and broken any further forks. - If you permit a merge instead, nobody gets broken. Maybe the best compromise is to tell people that if you push to master on other repositories, you get to deal with the mess. If we try to keep side overlays/etc working on branches and not on master then there will be no history to rewrite, as the merge will be rebased when it hits the official master, and from there it will get pulled by other repositories. We can perhaps allow merge commits on other branches, where the continuity of history is less important. Does that make sense? You'd be excluding me entirely, I need to use git-svn for other work projects, and emerging between two different versions of git would be very annoying (I switch constantly between the sides of work as they overlap). I'm a big proponent of letting the people doing the work scratch their own itches first! However, this does make us dependent on upstream - is there any sense of when they'll be ready, or what their own priority is for this issue. If this is becoming a deprecated feature then I'm not sure we can tie our future to it. I wasn't sure if any of the existing git-svn bugs pertained to this issue. Either way we should add this as a blocker to the git migration tracker. I think that even if we made a big push it would still take us a month or two to be ready with docs/infra/etc, and that might be optimistic. So, this might not be rate-limiting if upstream is actively working on it. Rich
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, 31 May 2012 14:18:04 -0500 William Hubbs willi...@gentoo.org wrote: On Thu, May 31, 2012 at 01:48:29PM -0400, Rich Freeman wrote: On Thu, May 31, 2012 at 12:49 PM, Robin H. Johnson robb...@gentoo.org wrote: 1. Discussion on merge policy. Originally I thought we would disallow merge commits, so that we would get a cleaner history. However, it turns out that if the repo ends up being pushed to different places with slightly different histories, merges are absolutely going to be required to prevent somebody from having to rebase at least one of their sets of commits that are already pushed. Not sure I'm following, but I will be the first to admit that I'm a git novice. Would this be aided by a convention, like only committing to master on the gentoo official repository, and any on-the-side work on places like github/etc stays in branches? Those repositories would just keep getting fed commits on master from the official repository. Iagree with this; I think we should ban merge commits on master. That would force everyone to rebase their work on current master before they commit to master which would make the history clean. What would git signing work with rebased commits? Would all of them have to be signed once again? -- Best regards, Michał Górny signature.asc Description: PGP signature
[gentoo-dev] last rites: games-util/nforenum
# Michael Sterrett mr_bon...@gentoo.org (31 May 2012) # No longer needed. # Masked for removal on 20120630 games-util/nforenum
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
Michał Górny писал 2012-05-31 23:33: On Thu, 31 May 2012 14:18:04 -0500 William Hubbs willi...@gentoo.org wrote: On Thu, May 31, 2012 at 01:48:29PM -0400, Rich Freeman wrote: On Thu, May 31, 2012 at 12:49 PM, Robin H. Johnson robb...@gentoo.org wrote: 1. Discussion on merge policy. Originally I thought we would disallow merge commits, so that we would get a cleaner history. However, it turns out that if the repo ends up being pushed to different places with slightly different histories, merges are absolutely going to be required to prevent somebody from having to rebase at least one of their sets of commits that are already pushed. Not sure I'm following, but I will be the first to admit that I'm a git novice. Would this be aided by a convention, like only committing to master on the gentoo official repository, and any on-the-side work on places like github/etc stays in branches? Those repositories would just keep getting fed commits on master from the official repository. Iagree with this; I think we should ban merge commits on master. That would force everyone to rebase their work on current master before they commit to master which would make the history clean. What would git signing work with rebased commits? Would all of them have to be signed once again? Commits itsels still will be signed -- Best Regards, Alexey 'Alexxy' Shvetsov Petersburg Nuclear Physics Institute, NRC Kurchatov Institute, Gatchina, Russia Department of Molecular and Radiation Biophysics Gentoo Team Ru Gentoo Linux Dev mailto:alexx...@gmail.com mailto:ale...@gentoo.org mailto:ale...@omrb.pnpi.spb.ru
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, May 31, 2012 at 07:13:42PM +, Robin H. Johnson wrote: - You have a commit, that you want to put into the Gentoo tree. - You have already pushed it to your github, signed If I have a github tree, that would probably be because I didn't have push access to the official tree, so signing the commit probably isn'tgoing to matter; I would expect that a gentoo dev who has push access to the tree would sign the commit when it is put into the gentoo tree. - It needs to be merged/rebased so that it applies on the Gentoo tree. - If you force it to be a rebase so it applies on the tip, then you may have changed the history of your github tree, and broken any further forks. - If you permit a merge instead, nobody gets broken. If you do this: git rebase master mybranch git checkout master git merge mybranch -- this is a fast-forward merge git pull --rebase git push I think that covers this concern doesn't it? 2. Git-SVN breakage. Why does this matter you're wondering? We need the newer Git for the commit signing, but it comes with a price, the git-svn binary has some major failures with SVN 1.7. Git since 1.7.8 has been broken this way. To clarify - these won't be issues for gentoo per se, but there is a sense that we can't stabilize the latest git because it will break it for people using git-svn on non-gentoo work? As the Git maintainer, I will not keyword it for anybody until I know it's not going to lose/corrupt data, regardless of what they are using it for. Why not keyword it and use package.use.mask for the git-svn flag? William pgpwkzFYxv1GM.pgp Description: PGP signature
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, May 31, 2012 at 3:33 PM, Michał Górny mgo...@gentoo.org wrote: What would git signing work with rebased commits? Would all of them have to be signed once again? The whole point of rebasing is to throw away history (which is either good or bad based on your perspective). So, if 14 devs spend 3 years and 2000 commits working on something in a branch, and I commit it to master using a rebase, then all you'll see in the master history is that rich0 committed 20k lines of code to master on May 31st, and that would be signed by me. I think that rebasing before merging is a pretty typical workflow anyway - when you submit a patch to Linus, he doesn't really care that you spent six months tweaking it - he just is getting a big blob of code that either works or doesn't. Does all that sub-history really matter? You could always push the branch to the repository if you wanted to keep it on the side. Rich
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, 31 May 2012 15:58:43 -0400 Rich Freeman ri...@gentoo.org wrote: On Thu, May 31, 2012 at 3:33 PM, Michał Górny mgo...@gentoo.org wrote: What would git signing work with rebased commits? Would all of them have to be signed once again? The whole point of rebasing is to throw away history (which is either good or bad based on your perspective). So, if 14 devs spend 3 years and 2000 commits working on something in a branch, and I commit it to master using a rebase, then all you'll see in the master history is that rich0 committed 20k lines of code to master on May 31st, and that would be signed by me. I think that rebasing before merging is a pretty typical workflow anyway - when you submit a patch to Linus, he doesn't really care that you spent six months tweaking it - he just is getting a big blob of code that either works or doesn't. Does all that sub-history really matter? You could always push the branch to the repository if you wanted to keep it on the side. That sounds like a pretty poor workflow to me. If I tweak something for 3 years, I'm likely to have a larger set of logically organized commits. I'm not saying it's unlikely I'm going to rebase fixes for obvious mistakes but putting everything onto one blob just makes the changes harder to read and less maintainable. For example, if I first create a basic function and then add additional options to it, I'm likely to keep those as separate commits. If one of the changes was a really bad idea, I'd like to revert the appropriate commit rather than removing all traces of it by hand and mistakenly introducing even worse breakage. -- Best regards, Michał Górny signature.asc Description: PGP signature
[gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
William Hubbs posted on Thu, 31 May 2012 14:54:50 -0500 as excerpted: On Thu, May 31, 2012 at 07:13:42PM +, Robin H. Johnson wrote: - You have a commit, that you want to put into the Gentoo tree. - You have already pushed it to your github, signed If I have a github tree, that would probably be because I didn't have push access to the official tree, so signing the commit probably isn'tgoing to matter; I would expect that a gentoo dev who has push access to the tree would sign the commit when it is put into the gentoo tree. I don't know what's going to happen to all the overlays with the main tree switch to git, but won't that break various overlay first policies, say for the kde overlay? Of course, if all the official overlays are converted to git branches of the main tree... but won't they still require rebasing as they've already been pushed? (This assumes your workaround idea doesn't work. If it does, great!) -- Duncan - List replies preferred. No HTML msgs. Every nonfree program has a lord, a master -- and if you use the program, he is your master. Richard Stallman
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On 05/31/12 16:09, Michał Górny wrote: On Thu, 31 May 2012 15:58:43 -0400 Rich Freeman ri...@gentoo.org wrote: On Thu, May 31, 2012 at 3:33 PM, Michał Górny mgo...@gentoo.org wrote: What would git signing work with rebased commits? Would all of them have to be signed once again? The whole point of rebasing is to throw away history (which is either good or bad based on your perspective). So, if 14 devs spend 3 years and 2000 commits working on something in a branch, and I commit it to master using a rebase, then all you'll see in the master history is that rich0 committed 20k lines of code to master on May 31st, and that would be signed by me. I think that rebasing before merging is a pretty typical workflow anyway - when you submit a patch to Linus, he doesn't really care that you spent six months tweaking it - he just is getting a big blob of code that either works or doesn't. Does all that sub-history really matter? You could always push the branch to the repository if you wanted to keep it on the side. That sounds like a pretty poor workflow to me. If I tweak something for 3 years, I'm likely to have a larger set of logically organized commits. I'm not saying it's unlikely I'm going to rebase fixes for obvious mistakes but putting everything onto one blob just makes the changes harder to read and less maintainable. For example, if I first create a basic function and then add additional options to it, I'm likely to keep those as separate commits. If one of the changes was a really bad idea, I'd like to revert the appropriate commit rather than removing all traces of it by hand and mistakenly introducing even worse breakage. That isn't what rebasing does, unless you do an interactive rebase and decide to squash the commits. The usual use case for a rebase is just to avoid the ugly merge commit. For example, 1. I clone your portage tree 2. I start making commits locally 3. You add a new package 4. I try to push my changes to you Step 4 doesn't work because your repo has changed. I can either, a) pull from you again (merge) b) pull with a rebase And then I should be able to push to you, assuming there were no conflicts. The merge option works fine but generates an ugly merge commit. The rebase takes our two histories and combines them into a nice linear history. In this case, a rebase would (essentially) take all of my commits and stick them on top of your new package commit. Afterwards, it looks like there's a nice linear history: you added a package, and then I did a bunch of other stuff. All of the commits are there. Since that history is linear and it looks like just a bunch of stuff on top of your existing tree, I can push it to you without problems. The only downside to the rebase is that it modifies my local history. So, if somebody cloned *my* repo in the middle of that, they could get screwed up.
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, May 31, 2012 at 08:26:58PM +, Duncan wrote: William Hubbs posted on Thu, 31 May 2012 14:54:50 -0500 as excerpted: I don't know what's going to happen to all the overlays with the main tree switch to git, but won't that break various overlay first policies, say for the kde overlay? Of course, if all the official overlays are converted to git branches of the main tree... but won't they still require rebasing as they've already been pushed? (This assumes your workaround idea doesn't work. If it does, great!) Overlays aren't really part of this discussion; those are independent trees which we have no control over, so commiting changes from overlays to the main tree is the responsibility of the overlay maintainers. William pgpnUoF15VbKy.pgp Description: PGP signature
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, May 31, 2012 at 03:58:43PM -0400, Rich Freeman wrote: On Thu, May 31, 2012 at 3:33 PM, Michał Górny mgo...@gentoo.org wrote: What would git signing work with rebased commits? Would all of them have to be signed once again? The whole point of rebasing is to throw away history (which is either good or bad based on your perspective). So, if 14 devs spend 3 years and 2000 commits working on something in a branch, and I commit it to master using a rebase, then all you'll see in the master history is that rich0 committed 20k lines of code to master on May 31st, and that would be signed by me. You don't commit to master with a rebase,; it is always a merge. The type of merge is what controls what you see in the logs. If you rebase your branch on master, merge to master then run git pull --rebase then push, you will get a fast forward merge, which shows the individual commits. If you don't include the rebasing, you get another type of merge which just shows up in the logs as one commit afaik. William pgpGov9QxG7kv.pgp Description: PGP signature
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On 1 June 2012 07:52, Alexey Shvetsov ale...@gentoo.org wrote: What would git signing work with rebased commits? Would all of them have to be signed once again? Commits itsels still will be signed Do you know how git does this? Do you have experience/information you can cite as to that this works? Commit signing seems poorly documented at present, and I've been looking at the git internals, and it would *APPEAR* that the content that is signed is the blob of text you normally get when you git cat-file -p $SHA1 And indeed, if you git cat-file -p $SHA1 file, extract the SIGNATURE part into its own file (removing the leading spaces), and remove the gnupg section from the commit headers, gpg --verify $sigfile $file # tells me I have a good signature. Just I haven't worked out what happens when the SHA1 of the 'parent' header changes, which *will* change if the rebase is anything other than a fast-forward. If that SHA1 changes, the gpg signature will surely fail? -- Kent perl -e print substr( \edrgmaM SPA NOcomil.ic\\@tfrken\, \$_ * 3, 3 ) for ( 9,8,0,7,1,6,5,4,3,2 ); http://kent-fredric.fox.geek.nz
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On 1 June 2012 07:58, Rich Freeman ri...@gentoo.org wrote: On Thu, May 31, 2012 at 3:33 PM, Michał Górny mgo...@gentoo.org wrote: What would git signing work with rebased commits? Would all of them have to be signed once again? The whole point of rebasing is to throw away history (which is either good or bad based on your perspective). So, if 14 devs spend 3 years and 2000 commits working on something in a branch, and I commit it to master using a rebase, then all you'll see in the master history is that rich0 committed 20k lines of code to master on May 31st, and that would be signed by me. I think that rebasing before merging is a pretty typical workflow anyway - when you submit a patch to Linus, he doesn't really care that you spent six months tweaking it - he just is getting a big blob of code that either works or doesn't. Does all that sub-history really matter? You could always push the branch to the repository if you wanted to keep it on the side. Rich I think you're conflating rebasing and squashing commits. You should rebase a long commit sequence and squash pointless fixup commits, and to make the commit sequence logical and ordered, possibly divided by logical changes that one may wish to later revert. ( That way, backing out a problem is simply reversing that commit and applying the reversal patch ) You should not rebase for the purpose of squashing an entire history of changes into a single scattered commit. Rebasing is more to make the history itself linear and non-complex, as when walking backwards through history, there being 2 parallel histories that generated a merged commit can be confusing for humans, so eliminating the parallel histories is one of the primary purposes I advocate use of rebase for. Squashed commits are a handy feature of rebase, but I wouldn't want to see an entire overlay squashed into the main tree as a single squashed commit. Another bad reason for squashed commits: if you're getting rid of the Changes files, you'll have no history on anything if you just group long histories into a single commit. None. -- Kent perl -e print substr( \edrgmaM SPA NOcomil.ic\\@tfrken\, \$_ * 3, 3 ) for ( 9,8,0,7,1,6,5,4,3,2 ); http://kent-fredric.fox.geek.nz
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, May 31, 2012 at 08:23:31PM +0100, Ciaran McCreesh wrote: On Thu, 31 May 2012 14:18:04 -0500 William Hubbs willi...@gentoo.org wrote: Not sure I'm following, but I will be the first to admit that I'm a git novice. Would this be aided by a convention, like only committing to master on the gentoo official repository, and any on-the-side work on places like github/etc stays in branches? Those repositories would just keep getting fed commits on master from the official repository. Iagree with this; I think we should ban merge commits on master. That would force everyone to rebase their work on current master before they commit to master which would make the history clean. So what's the point of switching to git if you want to ban the main reason git exists? To clarify: we should only allow fast-forward merges on master. My big complaint about merge commits is if you do a git show hash on a merge commit, you get nothing, so there is no way to see what actually changed in that commit. William pgpLhNVh63xgN.pgp Description: PGP signature
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On 1 June 2012 08:26, Duncan 1i5t5.dun...@cox.net wrote: William Hubbs posted on Thu, 31 May 2012 14:54:50 -0500 as excerpted: Of course, if all the official overlays are converted to git branches of the main tree... but won't they still require rebasing as they've already been pushed? (This assumes your workaround idea doesn't work. If it does, great!) End users will still want to work with overlays that are not merged with the main tree, not merely git branches. Its foreseeable that there will be git branches that /track/ overlays and exist as an integration pipeline for content from the overlays joining core gentoo, but end users will not want to use that. For the simple reason of course, that as soon as you want 1 overlay, portage's way of doing it with separate repositories is far more effective. You don't want each user to have to maintain an octopus merge between all the branches they want to have commits from ;) -- Kent perl -e print substr( \edrgmaM SPA NOcomil.ic\\@tfrken\, \$_ * 3, 3 ) for ( 9,8,0,7,1,6,5,4,3,2 ); http://kent-fredric.fox.geek.nz
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, May 31, 2012 at 5:52 PM, Kent Fredric kentfred...@gmail.com wrote: Just I haven't worked out what happens when the SHA1 of the 'parent' header changes, which *will* change if the rebase is anything other than a fast-forward. If that SHA1 changes, the gpg signature will surely fail? Rebasing doesn't modify past commits - it creates new ones and the past ones are no longer in the history of the current head. So, it doesn't break the old signatures so much as discard them. You would need to create new signatures in their place, presumably from whoever performed the rebase. I'm trying to glean what I can from the little info out there about how the new commit signatures work, but I don't think that you can use signatures to convey authorship if later authors are going to rebase the branch. The situation is not unlike what we have now with manifests. As far as I can tell if you want to do work outside of master, and then get those commits into master but preserve all the past signatures in the history of master, then you'd need to do a merge commit, so that all the deltas to do the merge are in a separate commit which is then signed by the person doing the merge. Rich
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, 31 May 2012 17:04:30 -0500 William Hubbs willi...@gentoo.org wrote: On Thu, May 31, 2012 at 08:23:31PM +0100, Ciaran McCreesh wrote: On Thu, 31 May 2012 14:18:04 -0500 William Hubbs willi...@gentoo.org wrote: Not sure I'm following, but I will be the first to admit that I'm a git novice. Would this be aided by a convention, like only committing to master on the gentoo official repository, and any on-the-side work on places like github/etc stays in branches? Those repositories would just keep getting fed commits on master from the official repository. Iagree with this; I think we should ban merge commits on master. That would force everyone to rebase their work on current master before they commit to master which would make the history clean. So what's the point of switching to git if you want to ban the main reason git exists? To clarify: we should only allow fast-forward merges on master. My big complaint about merge commits is if you do a git show hash on a merge commit, you get nothing, so there is no way to see what actually changed in that commit. Or you use a graphical tool which shows the whole merge history and you see the exact changes happening rather than some blob with 'do foo, do bar, and some baz too'. -- Best regards, Michał Górny signature.asc Description: PGP signature
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, 31 May 2012 16:27:48 -0400 Michael Orlitzky mich...@orlitzky.com wrote: On 05/31/12 16:09, Michał Górny wrote: On Thu, 31 May 2012 15:58:43 -0400 Rich Freeman ri...@gentoo.org wrote: On Thu, May 31, 2012 at 3:33 PM, Michał Górny mgo...@gentoo.org wrote: What would git signing work with rebased commits? Would all of them have to be signed once again? The whole point of rebasing is to throw away history (which is either good or bad based on your perspective). So, if 14 devs spend 3 years and 2000 commits working on something in a branch, and I commit it to master using a rebase, then all you'll see in the master history is that rich0 committed 20k lines of code to master on May 31st, and that would be signed by me. I think that rebasing before merging is a pretty typical workflow anyway - when you submit a patch to Linus, he doesn't really care that you spent six months tweaking it - he just is getting a big blob of code that either works or doesn't. Does all that sub-history really matter? You could always push the branch to the repository if you wanted to keep it on the side. That sounds like a pretty poor workflow to me. If I tweak something for 3 years, I'm likely to have a larger set of logically organized commits. I'm not saying it's unlikely I'm going to rebase fixes for obvious mistakes but putting everything onto one blob just makes the changes harder to read and less maintainable. For example, if I first create a basic function and then add additional options to it, I'm likely to keep those as separate commits. If one of the changes was a really bad idea, I'd like to revert the appropriate commit rather than removing all traces of it by hand and mistakenly introducing even worse breakage. That isn't what rebasing does, unless you do an interactive rebase and decide to squash the commits. Yes, it isn't but such kind of work flow was presented in the message I replied to. The usual use case for a rebase is just to avoid the ugly merge commit. Which devs should simply do whenever they use the scenario you mentioned. I agree we could block 'auto-merges' when pushing to a modified repo but not disallow a valid merges from devs who know what they're doing. -- Best regards, Michał Górny signature.asc Description: PGP signature
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On 1 June 2012 14:49, Rich Freeman ri...@gentoo.org wrote: On Thu, May 31, 2012 at 5:52 PM, Kent Fredric kentfred...@gmail.com wrote: Just I haven't worked out what happens when the SHA1 of the 'parent' header changes, which *will* change if the rebase is anything other than a fast-forward. If that SHA1 changes, the gpg signature will surely fail? Rebasing doesn't modify past commits - it creates new ones and the past ones are no longer in the history of the current head. So, it doesn't break the old signatures so much as discard them. You would need to create new signatures in their place, presumably from whoever performed the rebase. Hmm, thats annoying. Almost makes me wish it was the trees that were signed, not the commits. Although, I probably could brew up a prototype resigning tool ( based on Git::PurePerl ,... when they accept and publish my changes ) , just would be problematic because simply the act of signing a past commit means the SHA1 of the commit itself is different, so all successive commits after a re-signed commit will change and also need to be rebased and re-signed. -- Kent perl -e print substr( \edrgmaM SPA NOcomil.ic\\@tfrken\, \$_ * 3, 3 ) for ( 9,8,0,7,1,6,5,4,3,2 ); http://kent-fredric.fox.geek.nz
