Re: [gentoo-dev] CVS headers in ebuilds

[James Le Cuirot]

On Sun, 10 Apr 2016 18:21:44 -0400
Michael Orlitzky  wrote:

> On 04/10/2016 05:36 PM, Gordon Pettey wrote:
> > Or you could just use a branching workflow like Git has great
> > support for, and create your overlay as a branch of the main tree
> > you're copying ebuilds from. With recent versions, you can even
> > have checkouts of different branches from the same tree in
> > different directories, so you're not duplicating the the whole
> > repository. Then you could just git diff from the master branch,
> > and no need for preserving CVS misbehavior. 
> 
> I suggested this on the bug, but it's not so clear-cut. With a branch,
> you have a million other ebuilds sitting around in "your overlay" that
> you don't care about. I'm also not sure how the metadata updates would
> work; like, if it would be efficient to pull the rsync metadata and
> then run `emerge --metadata` or something like that. It wouldn't be
> fair to say "just use a branch!" if that's going to mean that
> somebody's day-to-day work gets a lot slower.

Would a sparse checkout help here? I've used this feature in a
different context. You could add the packages you do care about to the
sparse checkout list.

http://stackoverflow.com/questions/600079/is-there-any-way-to-clone-a-git-repositorys-sub-directory-only/13738951#13738951

-- 
James Le Cuirot (chewi)
Gentoo Linux Developer


pgprLYJIC7D0v.pgp
Description: OpenPGP digital signature

Re: [gentoo-dev] CVS headers in ebuilds

[Michael Orlitzky]

On 04/10/2016 05:36 PM, Gordon Pettey wrote:
> Or you could just use a branching workflow like Git has great support
> for, and create your overlay as a branch of the main tree you're copying
> ebuilds from. With recent versions, you can even have checkouts of
> different branches from the same tree in different directories, so
> you're not duplicating the the whole repository. Then you could just git
> diff from the master branch, and no need for preserving CVS misbehavior.
> 

I suggested this on the bug, but it's not so clear-cut. With a branch,
you have a million other ebuilds sitting around in "your overlay" that
you don't care about. I'm also not sure how the metadata updates would
work; like, if it would be efficient to pull the rsync metadata and then
run `emerge --metadata` or something like that. It wouldn't be fair to
say "just use a branch!" if that's going to mean that somebody's
day-to-day work gets a lot slower.

I would also feel a little guilty saying that the supported way of
extending/modifying the tree (overlays) won't work and you have to
branch off our VCS. I promised to think about it, but haven't had much
inspiration. Here's the best I've got so far.

  1. Use git for your ::gentoo tree.

  2. Before every sync (git pull), save the current HEAD:

   OLD=$(git rev-parse --verify HEAD)

  3. git pull

  4. Save the new HEAD:

   NEW=$(git rev-parse --verify HEAD)

  5. Search through the log (between the old HEAD and the new one) for
 any files that were modified:

 git log $OLD..$NEW --format='' --name-status \
   | grep '^M.*ebuild' \
   | cut -f2 \
   | sort \
   | uniq

That gives you a nice list at least, and you can compare it to the files
present in your overlay.

If you add another

  | xargs git log --stat $OLD..$NEW

onto the end of that last command, you can see statistics on what
changes were made. (Holy crap do we have a lot of people editing
dependencies without making revision bumps.)

Re: [gentoo-dev] CVS headers in ebuilds

[Gordon Pettey]

Or you could just use a branching workflow like Git has great support for,
and create your overlay as a branch of the main tree you're copying ebuilds
from. With recent versions, you can even have checkouts of different
branches from the same tree in different directories, so you're not
duplicating the the whole repository. Then you could just git diff from the
master branch, and no need for preserving CVS misbehavior.

On Sun, Apr 10, 2016 at 12:28 PM, Robin H. Johnson 
wrote:

> On Sun, Apr 10, 2016 at 06:16:05PM +0200, Ulrich Mueller wrote:
> > Why would you need $Id$ feature for this? "git ls-files -s" gives you
> > the hash of the blob as well, is more efficient than grep, and even
> > works recursively on a directory tree.
> >
> >$ git ls-files -s -- www-client/seamonkey/seamonkey-2.40.ebuild
> >100644 5ecd7709c6c8a316d9f005b4e4a0a54da81eb048 0
> www-client/seamonkey/seamonkey-2.40.ebuild
> Your ls-files doesn't let you track what blob the modified ebuild came
> from, when it's copied out of the git repo where expansion was
> happening.
>
> If the $Id$ is expanded in rsync, or your local environment, then you
> copy the file with the expanded $Id$ to an overlay (or another Git
> environment without expansion enabled), you have preserved the $Id$.
>
> Now the user edits it in their overlay, and since the original $Id$ is
> preserved, when you ask on bugzilla, please submit it as a diff; they
> can simply do:
> # diff <(git show $IdHash) $OVERLAY/pn/pv.ebuild
>
>
> --
> Robin Hugh Johnson
> Gentoo Linux: Developer, Infrastructure Lead, Foundation Trustee
> E-Mail : robb...@gentoo.org
> GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
>
>

Re: [gentoo-dev] CVS headers in ebuilds

[Robin H. Johnson]

On Sun, Apr 10, 2016 at 06:16:05PM +0200, Ulrich Mueller wrote:
> Why would you need $Id$ feature for this? "git ls-files -s" gives you
> the hash of the blob as well, is more efficient than grep, and even
> works recursively on a directory tree.
> 
>$ git ls-files -s -- www-client/seamonkey/seamonkey-2.40.ebuild 
>100644 5ecd7709c6c8a316d9f005b4e4a0a54da81eb048 0 
> www-client/seamonkey/seamonkey-2.40.ebuild
Your ls-files doesn't let you track what blob the modified ebuild came
from, when it's copied out of the git repo where expansion was
happening.

If the $Id$ is expanded in rsync, or your local environment, then you
copy the file with the expanded $Id$ to an overlay (or another Git
environment without expansion enabled), you have preserved the $Id$.

Now the user edits it in their overlay, and since the original $Id$ is
preserved, when you ask on bugzilla, please submit it as a diff; they
can simply do: 
# diff <(git show $IdHash) $OVERLAY/pn/pv.ebuild


-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Infrastructure Lead, Foundation Trustee
E-Mail : robb...@gentoo.org
GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85

Re: [gentoo-dev] CVS headers in ebuilds

[Ulrich Mueller]

> On Sat, 9 Apr 2016, Lars Wendler wrote:

>>> > Yes, I still use these lines to check for ebuild changes between
>>> > portage and my personal overlay. So please keep this line.

> Enable the ident feature for *.ebuild files in git:

>   $ cat ~/gentoo/.git/info/attributes
>   *.ebuild ident

> Now re-checkout every ebuild you wanna track.

>   $ git checkout --
>   ~/gentoo/www-client/seamonkey/seamonkey-2.40.ebuild

> Once you have done that those ebuilds will have some hash in the
> $Id$ field:

>   $ grep '$Id' ~/gentoo/www-client/seamonkey/seamonkey-2.40.ebuild
>   # $Id: 5ecd7709c6c8a316d9f005b4e4a0a54da81eb048 $

> The same hash is in each corresponding ebuild in my personal overlay
> as well. Occasionally I run a script to compare ebuilds from my
> overlay with the one from the git tree. When the hash is different
> something in the gentoo ebuild has changed and I can decide if I
> want to apply these changes to ebuilds in my overlay as well.

Why would you need $Id$ feature for this? "git ls-files -s" gives you
the hash of the blob as well, is more efficient than grep, and even
works recursively on a directory tree.

   $ git ls-files -s -- www-client/seamonkey/seamonkey-2.40.ebuild 
   100644 5ecd7709c6c8a316d9f005b4e4a0a54da81eb048 0 
www-client/seamonkey/seamonkey-2.40.ebuild

Ulrich


pgpOFBY7awgmg.pgp
Description: PGP signature

Re: [gentoo-dev] CVS headers in ebuilds

[Robin H. Johnson]

On Mon, Apr 04, 2016 at 03:42:37PM +1200, Kent Fredric wrote:
> On 4 April 2016 at 14:43, Robin H. Johnson  wrote:
> > We'd have to find all of those files and explicitly create .gitattribute
> > files, per directory, for them.
> I was under the impression that a .gitattribute file in the root
> directory sufficed?
> 
> ( I maybe have misinterpreted what you said, but the writing implied
> ">1 .gitattribute files proliferated into the tree" )
> 
> I'd personally think */*/*.ebuild would be a satisfactory start, and
> the rest could be turned on in an as-deemed-necessary basis.
This discussion is for files to _exclude_ from expansion.
If they were in CVS with the -ko or -kb flags, then they'd need to be
excluded. This applies mostly to patches in the files subdir.

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Infrastructure Lead, Foundation Trustee
E-Mail : robb...@gentoo.org
GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85

Re: [gentoo-dev] CVS headers in ebuilds

[Lars Wendler]

On Sun, 3 Apr 2016 22:57:42 +0200 Ulrich Mueller wrote:

>Does anyone still use the CVS $Id$ keywords that are in all ebuilds'
>headers, or are they being expanded anywhere? Or is there any other
>reason why they should be kept?
>
>In fact, the council had already voted to drop them in its 20141014
>meeting:
>
>   Can we drop CVS headers post-migration?
>   Aye: blueness, creffett (proxy for ulm), dberkholz, dilfridge,
>   radhermit, rich0, williamh 
>
>Ulrich

Yes, I still use these lines to check for ebuild changes between
portage and my personal overlay. So please keep this line.

Thanks.

Lars

-- 
Lars Wendler
Gentoo package maintainer
GPG: 21CC CF02 4586 0A07 ED93  9F68 498F E765 960E 9B39

Attention! New gpg key! See (self signed server cert for now)
http://www.gentoofan.org/blog/index.php?/archives/9-New-gpg-keys.html


pgpclkVFNS3fO.pgp
Description: Digitale Signatur von OpenPGP

Re: [gentoo-dev] CVS headers in ebuilds

[Ulrich Mueller]

> On Mon, 4 Apr 2016, Robin H Johnson wrote:

> On Mon, Apr 04, 2016 at 09:03:59AM +1200, Kent Fredric wrote:
>> Last time this came up, a sole example case was mentioned, but it
>> might have been buried.
>> https://bugs.gentoo.org/show_bug.cgi?id=557386

> Infra left the $Id$ stubs in place for this use case, but did not
> turn on global expansion due to the related issue mentioned in the
> bug: CVS allows easy individual file control of disabling keyword
> expansion (and setting a file as binary which also implicitly
> disables keyword expansion).
> We'd have to find all of those files and explicitly create
> .gitattribute files, per directory, for them.

I wonder if the small benefit gained from expanding keywords would be
worth this effort. Seems that the feature wasn't missed much in the
8 months since the conversion to git.

Ulrich


pgpFCCbfJcbDk.pgp
Description: PGP signature

Re: [gentoo-dev] CVS headers in ebuilds

[Kent Fredric]

On 4 April 2016 at 14:43, Robin H. Johnson  wrote:
> We'd have to find all of those files and explicitly create .gitattribute
> files, per directory, for them.


I was under the impression that a .gitattribute file in the root
directory sufficed?

( I maybe have misinterpreted what you said, but the writing implied
">1 .gitattribute files proliferated into the tree" )

I'd personally think */*/*.ebuild would be a satisfactory start, and
the rest could be turned on in an as-deemed-necessary basis.


-- 
Kent

KENTNL - https://metacpan.org/author/KENTNL

Re: [gentoo-dev] CVS headers in ebuilds

[Robin H. Johnson]

On Mon, Apr 04, 2016 at 09:03:59AM +1200, Kent Fredric wrote:
> On 4 April 2016 at 08:57, Ulrich Mueller  wrote:
> > Does anyone still use the CVS $Id$ keywords that are in all ebuilds'
> > headers, or are they being expanded anywhere? Or is there any other
> > reason why they should be kept?
> Last time this came up, a sole example case was mentioned, but it
> might have been buried.
> https://bugs.gentoo.org/show_bug.cgi?id=557386
Infra left the $Id$ stubs in place for this use case, but did not turn
on global expansion due to the related issue mentioned in the bug:
CVS allows easy individual file control of disabling keyword expansion
(and setting a file as binary which also implicitly disables keyword
expansion).
We'd have to find all of those files and explicitly create .gitattribute
files, per directory, for them.

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Infrastructure Lead, Foundation Trustee
E-Mail : robb...@gentoo.org
GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85

Re: [gentoo-dev] CVS headers in ebuilds

[Mike Gilbert]

On Sun, Apr 3, 2016 at 4:57 PM, Ulrich Mueller  wrote:
> Does anyone still use the CVS $Id$ keywords that are in all ebuilds'
> headers, or are they being expanded anywhere? Or is there any other
> reason why they should be kept?
>
> In fact, the council had already voted to drop them in its 20141014
> meeting:
>
>Can we drop CVS headers post-migration?
>Aye: blueness, creffett (proxy for ulm), dberkholz, dilfridge,
>radhermit, rich0, williamh

Please get a fix for repoman lined up before dropping the line from ebuilds.

floppym@naomi systemd % head -n 4 systemd-.ebuild
# Copyright 1999-2016 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2

EAPI=6
floppym@naomi systemd % repoman full

RepoMan scours the neighborhood...
 ebuild.badheader  1
  sys-apps/systemd/systemd-.ebuild: Malformed Id header on line: 3

Re: [gentoo-dev] CVS headers in ebuilds

[Rich Freeman]

On Sun, Apr 3, 2016 at 5:57 PM, William Hubbs  wrote:
> On Mon, Apr 04, 2016 at 09:03:59AM +1200, Kent Fredric wrote:
>> On 4 April 2016 at 08:57, Ulrich Mueller  wrote:
>> > Does anyone still use the CVS $Id$ keywords that are in all ebuilds'
>> > headers, or are they being expanded anywhere? Or is there any other
>> > reason why they should be kept?
>>
>>
>> Last time this came up, a sole example case was mentioned, but it
>> might have been buried.
>>
>> https://bugs.gentoo.org/show_bug.cgi?id=557386
>
> Tracking this type of value to "track upstream" doesn't seem to make
> much sense, and I think if infra was going to be able to fix it, it
> would have happened by now.
>
> My vote is to nuke it and be done with it.
>

++ - though as was pointed out this is a vote I've already cast before.

-- 
Rich

Re: [gentoo-dev] CVS headers in ebuilds

[Daniel Campbell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 04/03/2016 02:57 PM, William Hubbs wrote:
> On Mon, Apr 04, 2016 at 09:03:59AM +1200, Kent Fredric wrote:
>> On 4 April 2016 at 08:57, Ulrich Mueller  wrote:
>>> Does anyone still use the CVS $Id$ keywords that are in all
>>> ebuilds' headers, or are they being expanded anywhere? Or is
>>> there any other reason why they should be kept?
>> 
>> 
>> Last time this came up, a sole example case was mentioned, but
>> it might have been buried.
>> 
>> https://bugs.gentoo.org/show_bug.cgi?id=557386
> 
> Tracking this type of value to "track upstream" doesn't seem to
> make much sense, and I think if infra was going to be able to fix
> it, it would have happened by now.
> 
> My vote is to nuke it and be done with it.
> 
> William
> 
+1

- -- 
Daniel Campbell - Gentoo Developer
OpenPGP Key: 0x1EA055D6 @ hkp://keys.gnupg.net
fpr: AE03 9064 AE00 053C 270C  1DE4 6F7A 9091 1EA0 55D6
-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iQIcBAEBCAAGBQJXAZaPAAoJEAEkDpRQOeFwnP0P/Al+A36ls1DMQjEPu5U11vYn
L9aQPtckfeztfBYq4EB94Xazrno+yZI9kPg+vhUXA577H7h+OxM06wGd+DH4Q8ZJ
tChD+HrJleKZVV4lcT0e11FRKjzMm4ImKXXcEXFoy8asV76nT+U/Qd2cc+I10EnF
rgapKsIPpzUjOmmESmEX2ZxMcT5sDbUY2/4qE6veaS9HJbkjyoUChfv13SwiFydf
5jB2LQmru+A8Eq9u1ju21dQGTmkhLQCGcSBTmPvscpzrDmmqofpCeWo2rrn8Jz+l
4uWOcsBmnWOnJ6GzlZ+oY7iQ/2p+5Suzom61egKe2moNQyP2Fxgj58KhRZPRYvZ3
GJrkN/CGJOpc+usQVNk0ggSaewR8PRVBITT7LeRxyz/PukNaH6leP75+TtiBSnIJ
6UDKsRh/aMaZixKq8hcLUlhITaVQOI/zbYZmXGWcbxFQ8348kFGCdXXDFJf8g3CA
Bq1Z50fBaCDDEQ/3dVV479nuT3su/XCoI2VL3E5BwWVhra5nFh56jf3uSZ9ihGgd
IQFNWRIZ7ZEep2rciSl8ohFvEFgyP9lc6uod6Iukef3QPDPWK0YIVSyBgQGKzju/
qZqnHXUYv6vBvJxKtIVWNFWUzEnPLH6WrW+ec4m+bhCekHnJOpLxMLhCn0TySVtD
OQNW+DngQ8gI3q8pYngk
=GkAE
-END PGP SIGNATURE-

Re: [gentoo-dev] CVS headers in ebuilds

[William Hubbs]

On Mon, Apr 04, 2016 at 09:03:59AM +1200, Kent Fredric wrote:
> On 4 April 2016 at 08:57, Ulrich Mueller  wrote:
> > Does anyone still use the CVS $Id$ keywords that are in all ebuilds'
> > headers, or are they being expanded anywhere? Or is there any other
> > reason why they should be kept?
> 
> 
> Last time this came up, a sole example case was mentioned, but it
> might have been buried.
> 
> https://bugs.gentoo.org/show_bug.cgi?id=557386

Tracking this type of value to "track upstream" doesn't seem to make
much sense, and I think if infra was going to be able to fix it, it
would have happened by now.

My vote is to nuke it and be done with it.

William



signature.asc
Description: Digital signature

Re: [gentoo-dev] CVS headers in ebuilds

[Kent Fredric]

On 4 April 2016 at 08:57, Ulrich Mueller  wrote:
> Does anyone still use the CVS $Id$ keywords that are in all ebuilds'
> headers, or are they being expanded anywhere? Or is there any other
> reason why they should be kept?


Last time this came up, a sole example case was mentioned, but it
might have been buried.

https://bugs.gentoo.org/show_bug.cgi?id=557386

-- 
Kent

KENTNL - https://metacpan.org/author/KENTNL