Re: [gentoo-dev] rfc: revisiting our stabilization policy

[Dirkjan Ochtman]

On Wed, Jan 15, 2014 at 5:49 AM, William Hubbs willi...@gentoo.org wrote:
 Also, there is a substantial number of packages which contain only python
 code (or perl, ruby), or only LaTeX classes, or only documentation. It
 makes no sense to test them on each arch separately. I think maintainers
 should be allowed to stabilize such packages (with no compiled code) on
 all arches.

 There is a reason we don't do this, back in Gentoo history somewhere, but  I
 don't remember what it was.

 If someone can tell us why this isn't allowed I am all ears. Otherwise,
 I could agree on this point as well.

Yeah, as the python team lead, I feel we could definitely stick some
policy bits in (almost) all python packages that says stable for one
arch means stable for all arches. Sure, there'd be some fallout, but I
suspect that would be very limited, and in return only one arch tester
(or the maintainer!) can stabilize for all architectures.

Cheers,

Dirkjan

Re: [gentoo-dev] rfc: revisiting our stabilization policy

[Hans de Graaff]

On Tue, 2014-01-14 at 22:49 -0600, William Hubbs wrote:
  Also, there is a substantial number of packages which contain only python 
  code (or perl, ruby), or only LaTeX classes, or only documentation. It 
  makes no sense to test them on each arch separately. I think maintainers 
  should be allowed to stabilize such packages (with no compiled code) on 
  all arches.
 
 There is a reason we don't do this, back in Gentoo history somewhere, but  I
 don't remember what it was.
 
 If someone can tell us why this isn't allowed I am all ears. Otherwise,
 I could agree on this point as well.

Speaking for ruby I have seen various arch-related bugs in pure ruby
code. It doesn't happen a lot (maybe 1% of stable requests) but it is
also not predictable.

I also like the second set of eyes verifying what we've done as part of
marking a package stable, so I probably would still file bugs rather
than marking stuff stable myself.

Hans

Re: [gentoo-dev] rfc: revisiting our stabilization policy

[Michał Górny]

Dnia 2014-01-14, o godz. 15:37:19
William Hubbs willi...@gentoo.org napisał(a):

 I want comments wrt two ideas:
 
 1. I think maintainers should be able to stabilize their packages on arch's
 they have access to. I think this is allowed by some arch teams, but I
 think it would be good to formalize it.

I think we'd use more feedback from the 'other' arch teams before
agreeing on this. Some arches may have a pretty tricky issues that
could affect stabilization but which average developer may be not aware
of. Maybe it'd be good if each arch team had a wiki page explaining
the testing process for their arch.

We should also make it clear that the developer is supposed to test
the package on a pure stable system to avoid misunderstandings.

 2. I would like to see the policy below applied to all arch's [2].

 [2] http://www.gentoo.org/proj/en/council/meeting-logs/20130917-summary.txt

Honestly, this sounds like a very bad idea to me. Even on the minor
architectures.

TLDR: users will end up running unsupported mixed-arch systems
and stabilizing the package again sometime wouldn't make much sense.

Dropping the stable keyword on a package means that user either:

1) has to remove the package and either find an alternative or lose
particular features completely. And unlike with regular package.mask,
he won't get any tips from us. In fact, this policy makes it possible
to kill, say, the last graphical word processor on the arch.

2) has to add package.accept_keywords entry for the package. Which
means turning a pure stable system into an unsupported mixed-keyword
system.

Considering portage behavior, I think that 2) is much more likely. Now,
the keyword may be added per-version or per-package. If it's added
per-version, user simply ends up sticking to another single version
until he thinks of upgrading the package manually.

If it's added per-package, the keyword usually persists on the user's
system. When we bring the stable keywords to the package again, user
would have to notice that and remove his override. How likely is that
going to happen?

So, in the end once we remove stable keyword from a package, most users
add ~arch keyword and future stable keyword on the package becomes
meaningless.

I'd rather go for removing stable keywords from all packages. This
would at least make turning the architecture back stable easy for users.

-- 
Best regards,
Michał Górny


signature.asc
Description: PGP signature

Re: [gentoo-dev] rfc: revisiting our stabilization policy

[Sergey Popov]

15.01.2014 01:37, William Hubbs пишет:
 I want comments wrt two ideas:
 
 1. I think maintainers should be able to stabilize their packages on arch's
 they have access to. I think this is allowed by some arch teams, but I
 think it would be good to formalize it.
 
 2. I would like to see the policy below applied to all arch's [2].
 
 Thoughts?
 
 William
 
 [1] http://bugs.gentoo.org/487332
 [2] http://www.gentoo.org/proj/en/council/meeting-logs/20130917-summary.txt
 

amd64 and x86 arch teams have agreement, that maintainers can stabilize
their packages if they know how to properly test them.

-- 
Best regards, Sergey Popov
Gentoo developer
Gentoo Desktop Effects project lead
Gentoo Qt project lead
Gentoo Proxy maintainers project lead



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] rfc: revisiting our stabilization policy

[Sergey Popov]

15.01.2014 03:11, William Hubbs пишет:
 The status quo is not good, because we are forced to keep old, and
 potentially buggy, versions of software around longer than necessary.

arch team member opinion
But both of suggested solutions will break the whole idea of stabling.
Dropping packages to unstable on regular basis will annoy our users.

If we have old stable package, it builds and works correctly in new
environment(new gcc, glibc etc), then i do not see the point in rushing
things up, unless there are some more critical things, that needs new
version of this package. Stable should be reasonable. Each new version
of package contains bugfixes, it is true. But we should note, that new
versions(even so-called bugfix releases) can also bring new bugs.
/arch team member


-- 
Best regards, Sergey Popov
Gentoo developer
Gentoo Desktop Effects project lead
Gentoo Qt project lead
Gentoo Proxy maintainers project lead



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] rfc: revisiting our stabilization policy

[Sergey Popov]

15.01.2014 01:37, William Hubbs пишет:
 All,
 
 It is becoming more and more obvious that we do not have enough manpower
 on the arch teams, even some of the ones we consider major arch's, to
 keep up with stabilization requests. For example, there is this bug [1],
 which is blocking the stabilization of several important packages.

And by the way, the only arches left there are ppc and ppc64, which are
NOT major ones.

-- 
Best regards, Sergey Popov
Gentoo developer
Gentoo Desktop Effects project lead
Gentoo Qt project lead
Gentoo Proxy maintainers project lead



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] rfc: revisiting our stabilization policy

[Sergey Popov]

15.01.2014 06:42, Tom Wijsman пишет:
 And for that occasional mis-guess, *boohoo*, the user can just file a
 bug; which ironically even happens occasionally for stable packages.

If we blindly approves increasing of such mis-guesses, then our QA level
in arch teams will down below the apropriate level IMO. And this is not
good first of all for our stable users.

-- 
Best regards, Sergey Popov
Gentoo developer
Gentoo Desktop Effects project lead
Gentoo Qt project lead
Gentoo Proxy maintainers project lead



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] rfc: revisiting our stabilization policy

[Rich Freeman]

On Wed, Jan 15, 2014 at 4:54 AM, Michał Górny mgo...@gentoo.org wrote:

 2) has to add package.accept_keywords entry for the package. Which
 means turning a pure stable system into an unsupported mixed-keyword
 system.

As opposed to an unsupported pure stable system or an unsupported pure
unstable system?  I doubt anybody runs a pure stable system, and if
they do I doubt their experience is much different from those who run
mixed keywords.

Of course, having random system packages drop stable keywords from
time to time isn't going to help anything in that department.

Obviously maintainers should keep stable system packages around as
long as they can.  However, there needs to be some way to deal with
cases where their stabilization gets held up on a major arch.  If we
don't have the manpower to stabilize newer versions, what makes
anybody think that maintainers have the manpower to support ancient
versions?

The have our cake and eat it too solution is to obtain more manpower.
That should be done without question, but for whatever reason it never
actually happens, so we need to think about what the alternative is.
If talking about it scares more people into volunteering so that it
isn't necessary all the better...

Given constrained manpower the options basically are some variation on:
1.  Status quo - for some packages stable gets REALLY old, and likely
has problems that maintainers ignore.  You can't force somebody to
maintain something any more than you can force somebody to test
something.

2.  We become more liberal with stabilizing newer versions.  Lots of
variations on this, but the bottom line is that packages get
stabilized that would otherwise not be.

3.  We drop stable keywords on packages.  Lots of variations on this
as well, but the result is mixed-arch systems or dropping stable for
the whole arch.

I don't think #1 is really the answer - it just results in
less-visible problems and a stable that is probably works worse than
either #2 or #3 would yield.

#2 has the advantage that it gives us more control over what gets
installed on stable systems and you don't end up with mixed keywords.
The downside to #2 is that the user doesn't have as much visibility
into which packages are really stable.  Maybe an in-between quality
tier would help (if you're going to run a mixed keyword system, at
least use this version).

#3 has the advantage that it makes the problem directly visible to the
user.  However, the solution ends up being entirely user-discretion.
Some users end up on ~arch for life, others do it version-by-version
in which case they end up on various versions.  Personally I run
stable and when I need to run unstable on a package I tend to use
cat/foo-major+1 syntax so that I'm tracking unstable until the next
major version and then I get a chance to revisit the decision -
usually stable catches up by then.

The only nice solution is more manpower.  I'd like a pony to go with
that as well...

Rich

Re: [gentoo-dev] rfc: revisiting our stabilization policy

[William Hubbs]

On Wed, Jan 15, 2014 at 03:30:39PM +0400, Sergey Popov wrote:
 15.01.2014 01:37, William Hubbs пишет:
  All,
  
  It is becoming more and more obvious that we do not have enough manpower
  on the arch teams, even some of the ones we consider major arch's, to
  keep up with stabilization requests. For example, there is this bug [1],
  which is blocking the stabilization of several important packages.
 
 And by the way, the only arches left there are ppc and ppc64, which are
 NOT major ones.

Sparc is also still on that bug, and according to the council decision I
sited, these arch's are still treated like major arch's.

Wrt your comment about x86 and amd64 having agreements that maintainers
can stabilize packages on those arch's, I thought amd64 did, but I
didn't know about x86.

Formal policy says that all stabilizations must be done by arch teams
unless you have special arrangements with them [1], so my questions
still stand.

1. Should we make it policy that maintainers can stabilize packages on
arch's they have access to?

2. See Rich's message in this thread for my other concern; he spells it
out pretty well -- what should we do about architectures the maintainer
does not have access to?

3. Also, another interesting question has come up in this thread, that of
non-binary packages. Should we give maintainers the option of
stabilizing them on all arch's themselves?

William

[1] http://devmanual.gentoo.org/keywording/index.html


signature.asc
Description: Digital signature

Re: [gentoo-dev] rfc: revisiting our stabilization policy

[Tom Wijsman]

On Wed, 15 Jan 2014 15:33:28 +0400
Sergey Popov pinkb...@gentoo.org wrote:

 15.01.2014 06:42, Tom Wijsman пишет:
  And for that occasional mis-guess, *boohoo*, the user can just file
  a bug; which ironically even happens occasionally for stable
  packages.
 
 If we blindly approves increasing of such mis-guesses, then our QA
 level in arch teams will down below the apropriate level IMO. And
 this is not good first of all for our stable users.

What I'm saying is that even on arch team stabilized ebuilds bugs
getting filed happens as well; so, it happening for a maintainer
stabilized ebuild wouldn't be so problematic from that perspective.

But, indeed, it depends on which arch team procedure efforts the
maintainer actually applies; on an own arch it is quite possible for
the maintainer to be near the QA level of the arch team, whereas not
having access to a certain architecture can indeed become problematic.

So, for the arches the maintainers do have, it depends on what the
maintainers do as much as what the arch teams do; and to be realistic
arch teams might not always do everything as intended, especially under
time pressure. In my opinion, a maintainer would even spend more time.

As for arches the maintainer does not have, the available machines
might be usable; though the doubt is whether they have enough power.

Most of this discussion is hypothetical assuming stabilization stays
(or continues to grow to be more) problematic; who knows we might get
to see the opposite effect that this thread yields some new arch team
members, which could make what we've discussed here not necessary in the
short term run. It is clear everyone wants to hold on to the arch teams.

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : tom...@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D


signature.asc
Description: PGP signature

Re: [gentoo-dev] rfc: revisiting our stabilization policy

[Tom Wijsman]

On Wed, 15 Jan 2014 15:40:20 +0400
Sergey Popov pinkb...@gentoo.org wrote:

 As i said earlier for similar proposals - the one option that i see
 here to make all things going better - to recruit more people in arch
 teams/arch testers. Other options lead us to nowhere, when stable
 will be eliminated or transformed into fake.

If eventually our existing approach yields no or worsening results, it
would leads us nowhere as well; we can pick that option a few times,
but if it doesn't improve anything we'll need to start reconsidering.

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : tom...@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D


signature.asc
Description: PGP signature

Re: [gentoo-dev] rfc: revisiting our stabilization policy

[Matthew Thode]

On 01/15/2014 10:57 AM, Tom Wijsman wrote:
 On Wed, 15 Jan 2014 15:33:28 +0400
 Sergey Popov pinkb...@gentoo.org wrote:
 
 15.01.2014 06:42, Tom Wijsman пишет:
 And for that occasional mis-guess, *boohoo*, the user can just file
 a bug; which ironically even happens occasionally for stable
 packages.

 If we blindly approves increasing of such mis-guesses, then our QA
 level in arch teams will down below the apropriate level IMO. And
 this is not good first of all for our stable users.
 
 What I'm saying is that even on arch team stabilized ebuilds bugs
 getting filed happens as well; so, it happening for a maintainer
 stabilized ebuild wouldn't be so problematic from that perspective.
 
 But, indeed, it depends on which arch team procedure efforts the
 maintainer actually applies; on an own arch it is quite possible for
 the maintainer to be near the QA level of the arch team, whereas not
 having access to a certain architecture can indeed become problematic.
 
 So, for the arches the maintainers do have, it depends on what the
 maintainers do as much as what the arch teams do; and to be realistic
 arch teams might not always do everything as intended, especially under
 time pressure. In my opinion, a maintainer would even spend more time.
 
 As for arches the maintainer does not have, the available machines
 might be usable; though the doubt is whether they have enough power.
 
 Most of this discussion is hypothetical assuming stabilization stays
 (or continues to grow to be more) problematic; who knows we might get
 to see the opposite effect that this thread yields some new arch team
 members, which could make what we've discussed here not necessary in the
 short term run. It is clear everyone wants to hold on to the arch teams.
 
I would like to see the ability for devs to become arch testers for the
packages they own on the architectures they have access to.  The hard
part is enforcing the arch testing guidelines, so maybe this can be
considered 'arch tester jr.'.

-- 
-- Matthew Thode (prometheanfire)



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] rfc: revisiting our stabilization policy

[Thomas Sachau]

William Hubbs schrieb:

 Thoughts?
 
 William
 
 [1] http://bugs.gentoo.org/487332
 [2] http://www.gentoo.org/proj/en/council/meeting-logs/20130917-summary.txt
 

I see 2 cases here:

1. specific or all arch teams allow maintainers to stabilize packages on
their own, when they follow the arch team stabilization rules (e.g.
having a system running with stable keywords for testing the package).
This should not reduce the quality of the stable tree (or only to the
small amount, that some arch testers do additional checks the maintainer
does not do). Reading through this thread, it seems like amd64 and x86
arch teams already use this policy. This sounds like a reasonable
agreement, so i am supporting this too.

2. for arches with no such agreement or where the maintainer does not
have the needed hardware to test, no action for a certain amount of time
usually means, that the arch team is overloaded with work so the only
short- to mid-term solution is to reduce the amount of work resulting in
smaller amount of stable packages. So i am voting for maintainers
dropping stable keywords after a certain amount of time with no actions
(maybe with some notice beforehand). This might result in a mixed arch
user setup by default, but imho it is still better to have a smaller
stable set of core packages and testing packages on top then having
either everything on testing or broken/untested/unsupported packages,
which are still claimed to be the opposite with the stable keyword.

short summary:

-in agreement with arch teams, following stabilization policy and having
the needed hardware, maintainers should be able to add stable keywords
themselves
-if either agreement of arch team or needed hardware is missing,
keywords should be dropped, so that after some time the workload matches
the abilities of the arch team again.

-- 

Thomas Sachau
Gentoo Linux Developer



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] rfc: revisiting our stabilization policy

[William Hubbs]

On Wed, Jan 15, 2014 at 07:33:45PM +0100, Thomas Sachau wrote:
 William Hubbs schrieb:
 
  Thoughts?
  
  William
  
  [1] http://bugs.gentoo.org/487332
  [2] http://www.gentoo.org/proj/en/council/meeting-logs/20130917-summary.txt
  
 
 I see 2 cases here:
 
 1. specific or all arch teams allow maintainers to stabilize packages on
 their own, when they follow the arch team stabilization rules (e.g.
 having a system running with stable keywords for testing the package).
 This should not reduce the quality of the stable tree (or only to the
 small amount, that some arch testers do additional checks the maintainer
 does not do). Reading through this thread, it seems like amd64 and x86
 arch teams already use this policy. This sounds like a reasonable
 agreement, so i am supporting this too.
 
 2. for arches with no such agreement or where the maintainer does not
 have the needed hardware to test, no action for a certain amount of time
 usually means, that the arch team is overloaded with work so the only
 short- to mid-term solution is to reduce the amount of work resulting in
 smaller amount of stable packages. So i am voting for maintainers
 dropping stable keywords after a certain amount of time with no actions
 (maybe with some notice beforehand). This might result in a mixed arch
 user setup by default, but imho it is still better to have a smaller
 stable set of core packages and testing packages on top then having
 either everything on testing or broken/untested/unsupported packages,
 which are still claimed to be the opposite with the stable keyword.
 
 short summary:
 
 -in agreement with arch teams, following stabilization policy and having
 the needed hardware, maintainers should be able to add stable keywords
 themselves
 -if either agreement of arch team or needed hardware is missing,
 keywords should be dropped, so that after some time the workload matches
 the abilities of the arch team again.

When you say drop keywords do you mean:

1) revert the old version back to ~arch or
2) remove the old version.

As a maintainer, I would rather do 2, because I do not want to backport
fixes to the old version.

William



signature.asc
Description: Digital signature

Re: [gentoo-dev] rfc: revisiting our stabilization policy

[Ruud Koolen]

On Tuesday 14 January 2014 22:37:19 William Hubbs wrote:
 I think we need a global policy that either helps keep the stable tree
 up to date or reverts an architecture to ~ over time if the arch team
 can't keep up.

As a compromise solution for minor archs, it would be nice if there were a 
portage feature allowing me to ACCEPT_KEYWORDS those packages that are 
keyworded both ~arch, and stable on some major arch. For example, on m68k, it 
would select packages that are keyworded ~m68k  amd64, or ~m68k  x86, 
etc. This would allow me to run m68k kinda stable.

[Speculation as to applying a similar system more broadly witheld.]

-- Ruud

[gentoo-dev] Re: rfc: revisiting our stabilization policy

[Duncan]

Rich Freeman posted on Wed, 15 Jan 2014 07:51:49 -0500 as excerpted:

 Given constrained manpower the options basically are some variation on:

 1. Status quo - for some packages stable gets REALLY old, and likely has
 problems that maintainers ignore.  You can't force somebody to maintain
 something any more than you can force somebody to test something.
 
 2. We become more liberal with stabilizing newer versions.  Lots of
 variations on this, but the bottom line is that packages get stabilized
 that would otherwise not be.
 
 3. We drop stable keywords on packages.  Lots of variations on this as
 well, but the result is mixed-arch systems or dropping stable for the
 whole arch.
 
 I don't think #1 is really the answer - it just results in less-visible
 problems and a stable that is probably works worse than either #2 or #3
 would yield.
 
 #2 has the advantage that it gives us more control over what gets
 installed on stable systems and you don't end up with mixed keywords.
 The downside to #2 is that the user doesn't have as much visibility into
 which packages are really stable.  Maybe an in-between quality tier
 would help (if you're going to run a mixed keyword system, at least use
 this version).

What about a third target-stable keyword?  Either arch-teams or package-
maintainers (with maintainers getting priority in case of differing 
viewpoints, just as arch-teams already have priority over full-stable) 
could set this, and it would let users who wanted to be almost stable 
but hasn't gotten thru all the hoops just yet arch-testers do just that, 
see and test almost-stable packages.

An open stabilization bug would be required for any target-stable 
package, and only one target-stable per-slot per-arch would be allowed -- 
if there's already a target-stable in that slot for that arch, it would 
need to be removed and either that package fully stabilized or the new 
one substituted in the target-stable slot.

* Note however that different archs could however have different target-
stable packages within a slot.  This would allow a maintainer to set a 
minimal version target-stable keyword for lagging archs, while setting a 
preferred version target-stable keyword for current archs.

A maintainer facing lagging archs could then set target-stable 
appropriately, and could re-assign any bugs on the old-stable version to 
the arch in question, with comment boilerplate to the effect that the 
arch is lagging and hasn't updated stable, so they get to deal with bugs 
for their ancient-stable version.

A variant on the theme would be to split the current stable keyword in 
half, arch-stable and maintainer stable.  Any package version keyworded 
for both would be considered fully stable, but the two halves would then 
be fully independent, no longer interlocked, and for half-stable packages 
bugs would be assigned to the stable side with the other side CCed.  In 
this variant, there'd be no limit on the number of package versions that 
could be half-stabled by either half, just as there can now be multiple 
stable-keyworded versions, and for that matter, multiple testing versions 
as well.

 #3 has the advantage that it makes the problem directly visible to the
 user.  However, the solution ends up being entirely user-discretion.

Either target-stable keyword variant above would be directly visible to 
the end user as well, and of course they could choose full-stable or half-
stable on either side as they wished.

 Some users end up on ~arch for life, others do it version-by-version in
 which case they end up on various versions.  Personally I run stable and
 when I need to run unstable on a package I tend to use cat/foo-major+1
 syntax so that I'm tracking unstable until the next major version and
 then I get a chance to revisit the decision - usually stable catches
 up by then.

Interesting.  Nominally I do ~arch as for me stab?le== , but I do more or 
less the same thing at the overlay-~arch/unkeyworded/masked/live- 
level.

For my kde packages, for example, I run the overlay and normally 
package.keyword/package.unmask 4.y. as soon as it's available in the 
kde overlay (I have scripts that do git log --color --stat --graph 
$branch ORIG_HEAD..  on the overlays, and routinely run them after 
syncing so I know what's going on).  But since upstream kde doesn't split 
a branch until the rcs and thus those branches and the kde overlay 
packages based on them don't appear for the betas, I have to switch to 
- during the kde betas, and downgrade back to 4.y. when 
upstream branches and the 4.y. ebuilds appear.  I suppose one of 
these days I'll setup kde-frameworks and be doing about the same for it, 
too...

What's nice is that for kde 4.12. for example, when gentoo/kde was 
still sorting out the masking/dependencies issues in the overlay due to 
plasma/workspace being locked to 4.11.x, I was able to grab the 4.11. 
unmask files from the overlay, do a global search and replace 

[gentoo-dev] Re: rfc: revisiting our stabilization policy

[Duncan]

Michael Orlitzky posted on Tue, 14 Jan 2014 19:50:30 -0500 as excerpted:

 As I mentioned in a reply to William, right now I can decide when stuff
 is broken and keyword the newer versions. The proposal is to force me
 onto the new versions, which is strictly worse from my perspective.

Force??

As I discovered when gentoo/kde forced me into taking semantic-desktop 
up the beep with early kde 4.11, there's rather less force on gentoo 
than many realize, certainly as long as upstream is still supporting the 
options, anyway, one of the reasons I run gentoo.[1] =:^)

Every once in awhile I drag an ebuild out of /var/db/pkg/ to put in my 
overlay, because the ~arch I normally run has moved on and my current 
version is gone, but the new version is broken (or simply hugely changed 
and I don't have time to reconfigure ATM), while the stab?le version is 
just that, stale.

Of course the kde-sunset overlay is perhaps the ultimate example of that.

Yes, ultimately there will eventually be some forcing as in-tree deps 
change and keeping the old/overlaid version building and running becomes 
more of an issue over time, but it'll be a gradual process over a number 
of years, and the gentooer remains free to pick his pain point and do the 
migration in his own time, which at minimum, makes it a substantially 
softer force than would be the case on /most/ distributions.

---
[1] In the kde/semantic-desktop case, I diffed package versions with and 
without the flag and figured out which changes were related to it and 
which not, creating my own ebuild patches, which I dropped in a tree 
under /etc/portage/patches.ebuild/, similar to the /etc/portage/patches/ 
tree.  I then hacked up a script to apply those ebuild patches and re-
manifest, and added that step to my sync-script.  This was all possible, 
and actually surprisingly easy, because (1) upstream kde still supports 
the configure options and AFAIK intends to thru all of kde4, and (2) 
gentoo/kde had the options available at one point, so all I had to do was 
diff the before and after, and reverse the effect, hard-coding the flag 
off, where gentoo/kde was was effectively hard-coding it on.

Fortunately, before 4.11 went stable, gentoo/kde decided to keep the 
flags after all, and reintroduced them.  So I didn't have to carry my own 
patches for as long as I had feared I might.  But regardless, their 
forcing semantic-desktop on ~arch and overlay users didn't force /me/ 
to take it, because I'm an empowered gentooer and one way or another I 
wasn't taking any such forcing!  There efforts underway to do a user-
controlled kde-sunset overlay thing, possibly calling it kde-lite, too, 
thus spreading the work around a bit, but fortunately that ultimately 
wasn't needed.  And if it had come to it, I was beginning to look at 
other desktops too, as I had tried it previously and was done with kde 
with semantic-desktop, period, but fortunately that migration didn't have 
to happen either. =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
Every nonfree program has a lord, a master --
and if you use the program, he is your master.  Richard Stallman

[gentoo-dev] Re: rfc: revisiting our stabilization policy

[Duncan]

Tom Wijsman posted on Wed, 15 Jan 2014 01:28:09 +0100 as excerpted:

 Another option (and I don't mean to step on any toes or call anyone out
 here, these are just examples) may be to just deprecate stabilizing
 certain software. Packages such as the stuff in app-vim/ or app-emacs/
 or some games or some scientific software. For the editor plugins, most
 people do not get them from the package manager I feel and a Vim plugin
 requires almost as much arch testing work as a new version of grep, for
 example...
 
 Sounds like a good idea, but how do we translate that to the user;
 always mark them stable, or always mark them unstable? Do we want users
 to explicitly accept keywords on these packages?

As a ~arch/masked/overlay/live user here (who additionally doesn't even 
use gentoo kernel ebuilds, preferring direct upstream git and my own 
scripts), I haven't even checked if it actually happened (looks like 
gentoo-sources-3.10.x is still stable on x86/amd64, so maybe not), but...

There was previous discussion of destable-keywording the kernel.  How has 
that gone?

I've always thought that having a stable policy exception that the user 
actually has to deal with for certain packages, particularly core 
packages such as the kernel, would be confusing at best.  Still, given 
the upstream development pattern, I couldn't think of a reasonable 
alternative for the kernel, and agreed with the thread that it may have 
to be, for packages like that and perhaps google-chrome and firefox, 
where upstream releases are too close to 30-day and updates are very 
likely to be security-critical on packages that are net-exposed.

So it seemed it had to be, for them, and if that has gone well, perhaps 
expanding that no-stable policy precedent to things like editor plugins 
could work better than I might have imagined.

The other question then becomes, since ~arch packages are normally masked 
to stable, how are users exposed to them?  What about a file somewhere in 
profiles that lists all these no-stable packages, such that the PM can 
(perhaps optionally, I could imagine a setting in make.conf...) list all 
~arch versions of those packages on an otherwise stable system as if they 
were stable, tho possibly marked in some way to indicate that this 
package isn't a stable-keyword candidate?

-- 
Duncan - List replies preferred.   No HTML msgs.
Every nonfree program has a lord, a master --
and if you use the program, he is your master.  Richard Stallman

Re: [gentoo-dev] Re: rfc: revisiting our stabilization policy

[Tom Wijsman]

On Wed, 15 Jan 2014 23:59:49 + (UTC)
Duncan 1i5t5.dun...@cox.net wrote:

 There was previous discussion of destable-keywording the kernel.  How
 has that gone?

That was for vanilla-sources only, because that has restricted to only
the latest upstream version; as that makes the version change almost
weekly, the package can't undergo our stabilization procedure.
 
 I've always thought that having a stable policy exception that the
 user actually has to deal with for certain packages, particularly
 core packages such as the kernel, would be confusing at best.

Yes, if this would ever happen to gentoo-sources; I'd think the
handbook would then need to be updated to mention the necessary extra
step, but I think it is not bound to happen any time soon.

 Still,
 given the upstream development pattern, I couldn't think of a
 reasonable alternative for the kernel, and agreed with the thread
 that it may have to be, for packages like that and perhaps
 google-chrome and firefox, where upstream releases are too close to
 30-day and updates are very likely to be security-critical on
 packages that are net-exposed.

What we do now appears to work fine, critical security bugs cause fast
track stabilization if needed; I've backported some security fixes in
the past for less critical CVEs in the past, but the main problem here
for keeping this up is the lack of manpower on the kernel team.

 So it seemed it had to be, for them, and if that has gone well,
 perhaps expanding that no-stable policy precedent to things like
 editor plugins could work better than I might have imagined.

I think it needs to put the accept keywords in a more prominent place
if we're going to do this at a wider scale; currently it's in one of
those sections that people often don't read due to focusing on
continuing with there install instead, eg. they move to some DE guide.

 The other question then becomes, since ~arch packages are normally
 masked to stable, how are users exposed to them?

They aren't unless they accept keywords for them; which can either be
done globally using package.accept_keywords, or locally
by listing the package atom in /etc/portage/package.accept_keywords

 What about a file
 somewhere in profiles that lists all these no-stable packages, such
 that the PM can (perhaps optionally, I could imagine a setting in
 make.conf...) list all ~arch versions of those packages on an
 otherwise stable system as if they were stable, tho possibly marked
 in some way to indicate that this package isn't a stable-keyword
 candidate?

If we drop stable versions on a wider scale, we could indeed make the
~arch versions more visible where they currently aren't; we don't want
to give the impression that we are removing everything.

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : tom...@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D


signature.asc
Description: PGP signature

Re: [gentoo-dev] rfc: revisiting our stabilization policy

[Steev Klimaszewski]

On Wed, 2014-01-15 at 13:07 -0600, William Hubbs wrote:
 When you say drop keywords do you mean:
 
 1) revert the old version back to ~arch or
 2) remove the old version.
 
 As a maintainer, I would rather do 2, because I do not want to backport
 fixes to the old version.
 
 William
 

I'm not sure what he meant by drop keywords either, however, something
that I would like to see (with my ARM hat on here) - is, if something is
taking a while to stable for a certain arch, remove the keywords except
for that existing arch.  

We actually ran into something along this issue with git.

Now, arm is an interesting keyword, because for arm, when something
needs to be stabled, we have to test armv4, armv5, armv6, armv6
hardfloat, armv7, armv7 hardfloat, armv7 uclibc.

In my testing, one known issue was that git on uclibc did (and still
doesn't) work properly starting with git 1.8 - so I noted in the bug
that this was the case, and to NOT stable it for arm.  Unfortunately,
someone else on the ARM team disregarded the note and stabled the new
git, then the git maintainers dropped the old versions.  Now on arm
uclibc, git is entirely broken and unusable.

And no, adding more people to the arch team doesn't particularly help,
as people are buying more and more armv7 so they test that, but not the
rest of the versions - and no one wants to buy the older hardware
because it's so slow - we know it's slow, that's why it takes time.

I'd have definitely preferred that for git, that the 1.7 version stuck
around with just KEYWORDS=-* arm (and maybe even stabling 1.8 but
leaving 1.7 in masked?) - I realize it was a bit of a special case
because of the new git eclass.  Unfortunately, debugging what's going
on, is a bit above me, and the only other person I know who can/does
work on it, is blueness, and he's quite busy.

Re: [gentoo-dev] rfc: revisiting our stabilization policy

[Robin H. Johnson]

On Wed, Jan 15, 2014 at 06:58:27PM -0600, Steev Klimaszewski wrote:
 We actually ran into something along this issue with git.
 
 Now, arm is an interesting keyword, because for arm, when something
 needs to be stabled, we have to test armv4, armv5, armv6, armv6
 hardfloat, armv7, armv7 hardfloat, armv7 uclibc.
 
 In my testing, one known issue was that git on uclibc did (and still
 doesn't) work properly starting with git 1.8 - so I noted in the bug
 that this was the case, and to NOT stable it for arm.  Unfortunately,
 someone else on the ARM team disregarded the note and stabled the new
 git, then the git maintainers dropped the old versions.  Now on arm
 uclibc, git is entirely broken and unusable.
Ugh, this does suck.

Wasn't there a proposal years ago to include the libc in the keyword?

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Infrastructure Lead
E-Mail : robb...@gentoo.org
GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85

Re: [gentoo-dev] rfc: revisiting our stabilization policy

[Steev Klimaszewski]

On Thu, 2014-01-16 at 02:32 +, Robin H. Johnson wrote:
  In my testing, one known issue was that git on uclibc did (and still
  doesn't) work properly starting with git 1.8 - so I noted in the bug
  that this was the case, and to NOT stable it for arm.  Unfortunately,
  someone else on the ARM team disregarded the note and stabled the new
  git, then the git maintainers dropped the old versions.  Now on arm
  uclibc, git is entirely broken and unusable.
 Ugh, this does suck.
 

It does, though it's specific to arm uclibc, as it works fine on
amd64/x86 uclibc.  And unfortunately, it seems like this type of thing
is what people are proposing we move towards.  Instead of working but
old, not having access to the software at all.  I know it's not the
norm, nor is it typical, but the chance of this happening does exist,
and I can't see how anyone would say, well, that's just the chance that
people should take, unless they've never been bitten by a bug like this.


 Wasn't there a proposal years ago to include the libc in the keyword?
 

There may have been, I'm not sure that's really the right step either
though.  

Re: [gentoo-dev] rfc: revisiting our stabilization policy

[Sergey Popov]

15.01.2014 19:30, William Hubbs пишет:
 On Wed, Jan 15, 2014 at 03:30:39PM +0400, Sergey Popov wrote:
 15.01.2014 01:37, William Hubbs пишет:
 All,

 It is becoming more and more obvious that we do not have enough manpower
 on the arch teams, even some of the ones we consider major arch's, to
 keep up with stabilization requests. For example, there is this bug [1],
 which is blocking the stabilization of several important packages.

 And by the way, the only arches left there are ppc and ppc64, which are
 NOT major ones.
 
 Sparc is also still on that bug, and according to the council decision I
 sited, these arch's are still treated like major arch's.

Well, to be honest, personally i consider only amd64 and x86(and maybe
arm) as major arches, other are minor in my eyes. Council decision is
more about arches, that crucially lacks manpower.

 Wrt your comment about x86 and amd64 having agreements that maintainers
 can stabilize packages on those arch's, I thought amd64 did, but I
 didn't know about x86.

It's not mentioned, yeah, i was not aware about it for some time.
Probably it should be mentioned in Gentoo Development Guide.

 Formal policy says that all stabilizations must be done by arch teams
 unless you have special arrangements with them [1], so my questions
 still stand.
 
 1. Should we make it policy that maintainers can stabilize packages on
 arch's they have access to?
 
 2. See Rich's message in this thread for my other concern; he spells it
 out pretty well -- what should we do about architectures the maintainer
 does not have access to?
 
 3. Also, another interesting question has come up in this thread, that of
 non-binary packages. Should we give maintainers the option of
 stabilizing them on all arch's themselves?

1. If you know how to test it properly, know arch-specific
problems(aligning, endianness, ABI breakage) and how to fix it - then,
probably yes. But usually maintainers are bored to do proper testing.
2. I think - no. You can not test it - you can not stabilize it, period.
3. If code is interpreted rather then compiled, it does not matter that
it is properly ported on minor arches. I knew dozens of examples with
Perl and Python packages(not sure about Ruby, but Hans said that it
happens with it too). So, i would not treat such packages differently.


-- 
Best regards, Sergey Popov
Gentoo developer
Gentoo Desktop Effects project lead
Gentoo Qt project lead
Gentoo Proxy maintainers project lead



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] rfc: revisiting our stabilization policy

[Sergey Popov]

15.01.2014 21:04, Tom Wijsman пишет:
 On Wed, 15 Jan 2014 15:40:20 +0400
 Sergey Popov pinkb...@gentoo.org wrote:
 
 As i said earlier for similar proposals - the one option that i see
 here to make all things going better - to recruit more people in arch
 teams/arch testers. Other options lead us to nowhere, when stable
 will be eliminated or transformed into fake.
 
 If eventually our existing approach yields no or worsening results, it
 would leads us nowhere as well; we can pick that option a few times,
 but if it doesn't improve anything we'll need to start reconsidering.
 

It can not go to no result, unless we have no breakages in stable,
stable REMAINS stable. If it contains old, but working software - then
it is stable.

As i said earlier, problem begins when we NEED to stabilize something to
prevent breakages and arch teams are slow.

-- 
Best regards, Sergey Popov
Gentoo developer
Gentoo Desktop Effects project lead
Gentoo Qt project lead
Gentoo Proxy maintainers project lead



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] rfc: revisiting our stabilization policy

[Sergey Popov]

15.01.2014 22:33, Thomas Sachau пишет:
 William Hubbs schrieb:
 
 Thoughts?

 William

 [1] http://bugs.gentoo.org/487332
 [2] http://www.gentoo.org/proj/en/council/meeting-logs/20130917-summary.txt

 
 I see 2 cases here:
 
 1. specific or all arch teams allow maintainers to stabilize packages on
 their own, when they follow the arch team stabilization rules (e.g.
 having a system running with stable keywords for testing the package).
 This should not reduce the quality of the stable tree (or only to the
 small amount, that some arch testers do additional checks the maintainer
 does not do). Reading through this thread, it seems like amd64 and x86
 arch teams already use this policy. This sounds like a reasonable
 agreement, so i am supporting this too.
 
 2. for arches with no such agreement or where the maintainer does not
 have the needed hardware to test, no action for a certain amount of time
 usually means, that the arch team is overloaded with work so the only
 short- to mid-term solution is to reduce the amount of work resulting in
 smaller amount of stable packages. So i am voting for maintainers
 dropping stable keywords after a certain amount of time with no actions
 (maybe with some notice beforehand). This might result in a mixed arch
 user setup by default, but imho it is still better to have a smaller
 stable set of core packages and testing packages on top then having
 either everything on testing or broken/untested/unsupported packages,
 which are still claimed to be the opposite with the stable keyword.
 
 short summary:
 
 -in agreement with arch teams, following stabilization policy and having
 the needed hardware, maintainers should be able to add stable keywords
 themselves
 -if either agreement of arch team or needed hardware is missing,
 keywords should be dropped, so that after some time the workload matches
 the abilities of the arch team again.
 

Thanks, for the proposal. IIRC, there was similar backroom agreement in
some minor arches, look at how armin76 stabilized packages earlier -
sometimes he drops vast amount of keywords on ia64/sparc/m68k to
unstable in stabilization requests.

And i think we should continue this practice.

-- 
Best regards, Sergey Popov
Gentoo developer
Gentoo Desktop Effects project lead
Gentoo Qt project lead
Gentoo Proxy maintainers project lead



signature.asc
Description: OpenPGP digital signature

[gentoo-dev] Re: rfc: revisiting our stabilization policy

[Michael Palimaka]

On 01/16/2014 05:27 PM, Sergey Popov wrote:
 
 Thanks, for the proposal. IIRC, there was similar backroom agreement in
 some minor arches, look at how armin76 stabilized packages earlier -
 sometimes he drops vast amount of keywords on ia64/sparc/m68k to
 unstable in stabilization requests.
 
 And i think we should continue this practice.
 
I agree completely. The keywords on many packages are historical and do
not necessarily represent the current reality. Is it really a good use
of our resources to maintain stable keywords for some small auxiliary
package?
I think every stable request is a good opportunity for each minor arch
team to review their keywords for that package.

Re: [gentoo-dev] rfc: revisiting our stabilization policy

[Christopher Head]

On Tue, 14 Jan 2014 20:46:04 -0600
William Hubbs willi...@gentoo.org wrote:

 s/month/year/
 
 Do you feel the same way then? I have heard of stabilizations taking
 this long before. I just had to try to pick something reasonable for
 the discussion.
 
 I suppose a compromise would be, instead of removing the old versions,
 move them back to ~arch for a month then remove them, but that still
 implies action on your part.
 
 William

If I need or want a feature or bugfix which isn’t in the newer version,
I always have the choice to use ~. If I don’t, why do I care if the
package is a year old? I lose none of my time to use the old version,
since it does all I want; I lose a nonzero amount of time if I get a
version which breaks things (even if the only time I lose is the time
it takes to downgrade), so it’s in my best interest to use the stable
versions of such packages, even if they’re a month, a year, or three
years old.
-- 
Christopher Head


signature.asc
Description: PGP signature

[gentoo-portage-dev] [PATCH v2] Add repoman check to warn if src_prepare/src_configure are used in EAPI 0/1 and if pkg_pretend is used in EAPI 4. Fixes bug 379491.

[Chris Reffett]

---
 pym/repoman/checks.py | 29 +
 1 file changed, 29 insertions(+)

diff --git a/pym/repoman/checks.py b/pym/repoman/checks.py
index 85aa065..5c55b0d 100644
--- a/pym/repoman/checks.py
+++ b/pym/repoman/checks.py
@@ -731,6 +731,21 @@ class DeprecatedHasq(LineCheck):
re = re.compile(r'(^|.*\b)hasq\b')
error = errors.HASQ_ERROR
 
+# EAPI 2 checks
+class Eapi01UndefinedPhases(LineCheck):
+   repoman_check_name = 'EAPI.incompatible'
+   src_configprepare_re = re.compile(r'\s*src_(configure|prepare)\s*\(\)')
+
+   def check_eapi(self, eapi):
+   return eapi in ('0', '1')
+
+   def check(self, num, line):
+   m = self.src_configprepare__re.match(line)
+   if m is not None:
+   return (%s % m.group(1)) + \
+phase is not defined in EAPI=0/1 on line: %d
+
+
 # EAPI-3 checks
 class Eapi3DeprecatedFuncs(LineCheck):
repoman_check_name = 'EAPI.deprecated'
@@ -745,6 +760,20 @@ class Eapi3DeprecatedFuncs(LineCheck):
return ('%s' % m.group(1)) + \
 has been deprecated in EAPI=3 on line: %d
 
+# EAPI 4 checks
+class Eapi0123UndefinedPhases(LineCheck):
+   repoman_check_name = 'EAPI.incompatible'
+   pkg_pretend_re = re.compile(r'\s*pkg_pretend\s*\(\)')
+
+   def check_eapi(self, eapi):
+   return eapi in ('0', '1', '2', '3')
+
+   def check(self, num, line):
+   m = self.pkg_pretend_re.match(line)
+   if m is not None:
+   return (%s % m.group(1)) + \
+phase is not defined in EAPI  4 on line: %d
+
 # EAPI-4 checks
 class Eapi4IncompatibleFuncs(LineCheck):
repoman_check_name = 'EAPI.incompatible'
-- 
1.8.5.1

Re: [gentoo-portage-dev] [PATCH] Document bugzilla workflow

[Sebastian Luther]

Am 15.01.2014 17:20, schrieb Tom Wijsman:
 On Wed, 15 Jan 2014 07:29:19 +0100
 Sebastian Luther sebastianlut...@gmx.de wrote:
 
 Am 15.01.2014 04:11, schrieb Tom Wijsman:


 I send the first mail with this wording 8 days ago. Enough time to
 comment on it. I'd prefer to discuss it on the list.
 
 Yes, but not all comments were discussed yet, therefore (dis)agreement
 on them is missing; and this last thing rather became a topic of
 discussion due to the work clashes that we saw happen twice.
 
I'd say the clashes occurred because nobody mentioned at all what they
are working on. Since people started using IN_PROGRESS to mean I'm
working on it, this shouldn't happen again.
 
 Yes, I see some commit messages not refer to bugs which is something we
 will want to avoid; think this might need to go into the commit policy.
 
There's nothing wrong with fixing/implementing something that nobody
filed a bug about.


 The way it was is to not care about them at all. There was no
 agreement on the the other thread if these things should be used or
 not. So I left it vague so everyone could use it, but they are not
 forced to.
 
 Hmm, could this result in conflicting usage of these?

Maybe, but I'd first see if the usage patterns converge to something
that makes everyone happy.
 
 +There are a number of bugs named [TRACKER] * that collect bugs
 +for specific topics. Confirmed bugs should be marked as blocking
 +these tracker bugs if appropriate.

 For clarity, it should be mentioned that this does not mean to block
 the tracker for the next version; this could be misinterpreted.

 Considering that the tracker gets renamed, I'm not sure what you mean
 here.
 
 As you are confused yourself by misinterpreting what you have written,
 you demonstrate the case for the need of clarity here; this is not
 about the next version tracker or it being renamed at all, it's about
 all other trackers that are not version trackers. The part of the
 policy quoted here doesn't make that clear, it had me puzzling for a
 moment too when I first read that; I think you were puzzled too now...
 

Sorry, I failed to properly read what you quoted.

I think once you know that these other trackers exist, it's clear. If
you want something added there, that's fine with me too.


Sebastian

Re: [gentoo-portage-dev] [PATCH] Document bugzilla workflow

[Sebastian Luther]

Am 15.01.2014 19:41, schrieb Tom Wijsman:
 Yes, I see some commit messages not refer to bugs which is 
 something we will want to avoid; think this might need to go
 into the commit policy.
 
 There's nothing wrong with fixing/implementing something that
 nobody filed a bug about.
 
 Sorry, consider the common case where a bug was filed but the
 commit message does not refer to that bug. Also note that I am
 trying to refer to the ChangeLog of Portage itself, not that of the
 ebuild; thus I mean the commit messages for the Portage repo, not
 to the Portage tree.
 
Not sure if we're talking about the same things.

1) If you fix something that has a bug, you should refer to that in
the git commit message.
2) There's nothing wrong with a git commit message that does not refer
to a bug, if there is no bug filed.

 The whole point of documenting it in a workflow is to make it
 converge;

Not the converge I meant.

What I meant was to allow people to test different styles and hope
that the one that works best will be adopted by everyone at some
point. Once that happens you can document that style.

 if you instead leave things unclear or undocumented, you have no 
 guaranteed convergence and might even see a disconvergence.

Yes, maybe. One then needs to see if that is a problem and if it is
then force everyone to use one style.

 
 It's already making people unhappy right now; because as it is 
 documented now, it is turned from the meaningful experience that
 the previous Portage team had before to something that is
 meaningless. It is a regression in checking the list of bugs that
 block the tracker, as the states of the bugs no longer have a value
 as it is documented now.
 
Previously the bug state was not used at all. There is no regression.

[gentoo-portage-dev] [PATCH v3] Add repoman check to warn if src_prepare/src_configure are used in EAPI 0/1 and if pkg_pretend is used in EAPI 4. Fixes bug 379491.

[Chris Reffett]

---
 pym/repoman/checks.py | 29 +
 1 file changed, 29 insertions(+)

Ignore v2, I apparently didn't commit all of my changes and so that patch
won't work. Undid the compression of the src_prepare/src_configure regex,
because the way that the regex is set up means that it would output prepare
phase is not defined in EAPI... instead of src_prepare and I feel that
it's more intuitive to match the full name of the function instead of tacking
src_ in front of the output.


diff --git a/pym/repoman/checks.py b/pym/repoman/checks.py
index 85aa065..c6860d8 100644
--- a/pym/repoman/checks.py
+++ b/pym/repoman/checks.py
@@ -731,6 +731,21 @@ class DeprecatedHasq(LineCheck):
re = re.compile(r'(^|.*\b)hasq\b')
error = errors.HASQ_ERROR
 
+# EAPI 2 checks
+class Eapi01UndefinedPhases(LineCheck):
+   repoman_check_name = 'EAPI.incompatible'
+   src_configprepare_re = 
re.compile(r'\s*(src_configure|src_prepare)\s*\(\)')
+
+   def check_eapi(self, eapi):
+   return eapi in ('0', '1')
+
+   def check(self, num, line):
+   m = self.src_configprepare_re.match(line)
+   if m is not None:
+   return ('%s' % m.group(1)) + \
+phase is not defined in EAPI  2 on line: %d
+
+
 # EAPI-3 checks
 class Eapi3DeprecatedFuncs(LineCheck):
repoman_check_name = 'EAPI.deprecated'
@@ -745,6 +760,20 @@ class Eapi3DeprecatedFuncs(LineCheck):
return ('%s' % m.group(1)) + \
 has been deprecated in EAPI=3 on line: %d
 
+# EAPI 4 checks
+class Eapi0123UndefinedPhases(LineCheck):
+   repoman_check_name = 'EAPI.incompatible'
+   pkg_pretend_re = re.compile(r'\s*(pkg_pretend)\s*\(\)')
+
+   def check_eapi(self, eapi):
+   return eapi in ('0', '1', '2', '3')
+
+   def check(self, num, line):
+   m = self.pkg_pretend_re.match(line)
+   if m is not None:
+   return ('%s' % m.group(1)) + \
+phase is not defined in EAPI  4 on line: %d
+
 # EAPI-4 checks
 class Eapi4IncompatibleFuncs(LineCheck):
repoman_check_name = 'EAPI.incompatible'
-- 
1.8.5.1

[gentoo-portage-dev] [PATCH v4] Add repoman check to warn if src_prepare/src_configure are used in EAPI 0/1 and if pkg_pretend is used in EAPI 4. Fixes bug 379491.

[Chris Reffett]

---
 pym/repoman/checks.py | 31 ++-
 1 file changed, 30 insertions(+), 1 deletion(-)

diff --git a/pym/repoman/checks.py b/pym/repoman/checks.py
index 85aa065..c814fa7 100644
--- a/pym/repoman/checks.py
+++ b/pym/repoman/checks.py
@@ -15,7 +15,7 @@ import repoman.errors as errors
 import portage
 from portage.eapi import eapi_supports_prefix, eapi_has_implicit_rdepend, \
eapi_has_src_prepare_and_src_configure, eapi_has_dosed_dohard, \
-   eapi_exports_AA
+   eapi_exports_AA, eapi_has_pkg_pretend
 
 class LineCheck(object):
Run a check on a line of an ebuild.
@@ -731,6 +731,21 @@ class DeprecatedHasq(LineCheck):
re = re.compile(r'(^|.*\b)hasq\b')
error = errors.HASQ_ERROR
 
+# EAPI 2 checks
+class UndefinedSrcPrepareSrcConfigurePhases(LineCheck):
+   repoman_check_name = 'EAPI.incompatible'
+   src_configprepare_re = 
re.compile(r'\s*(src_configure|src_prepare)\s*\(\)')
+
+   def check_eapi(self, eapi):
+   return not eapi_has_src_prepare_and_src_configure(eapi)
+
+   def check(self, num, line):
+   m = self.src_configprepare_re.match(line)
+   if m is not None:
+   return ('%s' % m.group(1)) + \
+phase is not defined in EAPI  2 on line: %d
+
+
 # EAPI-3 checks
 class Eapi3DeprecatedFuncs(LineCheck):
repoman_check_name = 'EAPI.deprecated'
@@ -745,6 +760,20 @@ class Eapi3DeprecatedFuncs(LineCheck):
return ('%s' % m.group(1)) + \
 has been deprecated in EAPI=3 on line: %d
 
+# EAPI 4 checks
+class UndefinedPkgPretendPhase(LineCheck):
+   repoman_check_name = 'EAPI.incompatible'
+   pkg_pretend_re = re.compile(r'\s*(pkg_pretend)\s*\(\)')
+
+   def check_eapi(self, eapi):
+   return not eapi_has_pkg_pretend(eapi)
+
+   def check(self, num, line):
+   m = self.pkg_pretend_re.match(line)
+   if m is not None:
+   return ('%s' % m.group(1)) + \
+phase is not defined in EAPI  4 on line: %d
+
 # EAPI-4 checks
 class Eapi4IncompatibleFuncs(LineCheck):
repoman_check_name = 'EAPI.incompatible'
-- 
1.8.5.1

[gentoo-portage-dev] [PATCH 2/3] Have repoman check that a package directory contains at least one ebuild (bug #245305).

[Tom Wijsman]

---
 bin/repoman   | 8 
 man/repoman.1 | 3 +++
 2 files changed, 11 insertions(+)

diff --git a/bin/repoman b/bin/repoman
index 9b703dc..3263ceb 100755
--- a/bin/repoman
+++ b/bin/repoman
@@ -330,6 +330,7 @@ qahelp = {
SRC_URI.mirror: A uri listed in profiles/thirdpartymirrors is found 
in SRC_URI,
ebuild.syntax: Error generating cache entry for ebuild; typically 
caused by ebuild syntax error or digest verification failure,
ebuild.output: A simple sourcing of the ebuild produces output; this 
breaks ebuild policy.,
+   ebuild.missing: A package directory must at least contain one ebuild 
or be treecleaned.,
ebuild.nesteddie: Placing 'die' inside ( ) prints an error, but 
doesn't stop the ebuild.,
variable.invalidchar: A variable contains an invalid character that 
is not part of the ASCII character set,
variable.readonly: Assigning a readonly variable,
@@ -1466,6 +1467,13 @@ for x in effective_scanlist:
can_force = False
continue
 
+   if len(ebuildlist) == 0:
+   stats[ebuild.missing] += 1
+   fails[ebuild.missing].append(%s must at least contain one  
% x + \
+   ebuild or be treecleaned.)
+   can_force = False
+   continue
+
# Sort ebuilds in ascending order for the KEYWORDS.dropped check.
ebuildlist = sorted(pkgs.values())
ebuildlist = [pkg.pf for pkg in ebuildlist]
diff --git a/man/repoman.1 b/man/repoman.1
index e739d56..2bf3765 100644
--- a/man/repoman.1
+++ b/man/repoman.1
@@ -301,6 +301,9 @@ Ebuilds that exist but have not been added to cvs
 .B ebuild.output
 A simple sourcing of the ebuild produces output; this breaks ebuild policy.
 .TP
+.B ebuild.missing
+A package directory must at least contain one ebuild or be treecleaned.
+.TP
 .B ebuild.patches
 PATCHES variable should be a bash array to ensure white space safety
 .TP
-- 
1.8.5.2

[gentoo-portage-dev] Repoman patches for bugs #205909, #245305 and #482084.

[Tom Wijsman]

In reply, you will find three repoman patches; PATCH 1 is a bit more complex
which I will detail here, the other two patches should be fairly trivial.

In the first patch I need to use the @system set, as I only want to check
DEPEND for packages not in the @system set; thus here is kept in mind that
the @system set could possible change, in which case the check continues to
work. After checking up with Arfrever, a first version that I came up with is

 +from portage._sets.profiles import PackagesSystemSet
 +system_set_atoms = PackagesSystemSet(portage.settings.profiles).getAtoms()

but I am not sure if this is appropriate when used in other repositories.
Arfrever warned me about this but I think I do not fully understand that.

Both archive_formats* variables are based on the PMS specifications.

The rest of the code has comments and should be trivial to understand.

For the check name we came up with unpack.DEPEND.missing; most of the
checks are two words, so, I don't know if three words is permitted. At least
repoman runs without complaining or bailing out because of this.

There's still a TODO left in the code that leaves me in doubt on how to
properly ask the keywords to Portage; seems that I still need to learn to
find my way through the documentation, but I guess after getting pointed to
it a few times it will become easier.

These are my first patches to the Repoman code, all three patches introduce a 
new warning / error, therefore it might be possible that I missed something.
Grepping on an existing warning I saw the man page needs to be updated; since
I never did that before, feel free to check if the syntax of that is right.

Thank you for taking your time to review these.

--
 bin/repoman   | 63 
+++
 man/repoman.1 | 10 ++
 pym/repoman/checks.py | 10 ++
 3 files changed, 83 insertions(+)

 [PATCH 1/3] Have repoman check if the packages to unpack rare archive
 formats from SRC_URI are present in DEPEND (bug #205909).

 [PATCH 2/3] Have repoman check that a package directory contains at least
 one ebuild (bug #245305).

 [PATCH 3/3] Have repoman deprecate G2CONF for the GNOME team. (bug #482084).

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : tom...@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D

Re: [gentoo-portage-dev] [PATCH 3/3] Have repoman deprecate G2CONF for the GNOME team. (bug #482084).

[Jesus Rivero (Neurogeek)]

On Jan 15, 2014 7:08 PM, Tom Wijsman tom...@gentoo.org wrote:

 ---
  bin/repoman   |  2 ++
  man/repoman.1 |  3 +++
  pym/repoman/checks.py | 10 ++
  3 files changed, 15 insertions(+)

 diff --git a/bin/repoman b/bin/repoman
 index 3263ceb..6754edd 100755
 --- a/bin/repoman
 +++ b/bin/repoman
 @@ -318,6 +318,7 @@ qahelp = {
 EAPI.incompatible: Ebuilds that use features that are only
available with a different EAPI,
 EAPI.unsupported: Ebuilds that have an unsupported EAPI
version (you must upgrade portage),
 SLOT.invalid: Ebuilds that have a missing or invalid SLOT
variable value,
 +   G2CONF.deprecated: G2CONF is deprecated, see Gentoo bug
#482084 and the GNOME team policies,
 HOMEPAGE.missing: Ebuilds that have a missing or empty
HOMEPAGE variable,
 HOMEPAGE.virtual: Virtuals that have a non-empty HOMEPAGE
variable,
 PDEPEND.suspect: PDEPEND contains a package that usually only
belongs in DEPEND.,
 @@ -382,6 +383,7 @@ qawarnings = set((
  dependency.badtilde,
  DESCRIPTION.toolong,
  EAPI.deprecated,
 +G2CONF.deprecated,
  HOMEPAGE.virtual,
  LICENSE.deprecated,
  LICENSE.virtual,
 diff --git a/man/repoman.1 b/man/repoman.1
 index 2bf3765..7ec43d5 100644
 --- a/man/repoman.1
 +++ b/man/repoman.1
 @@ -227,6 +227,9 @@ Syntax error in RESTRICT (usually an extra/missing
space/parenthesis)
  .B SLOT.invalid
  Ebuilds that have a missing or invalid SLOT variable value
  .TP
 +.B G2CONF.deprecated
 +G2CONF is deprecated, see Gentoo bug #482084 and the GNOME team policies
 +.TP
  .B SRC_URI.mirror
  A uri listed in profiles/thirdpartymirrors is found in SRC_URI
  .TP
 diff --git a/pym/repoman/checks.py b/pym/repoman/checks.py
 index 85aa065..c2608b0 100644
 --- a/pym/repoman/checks.py
 +++ b/pym/repoman/checks.py
 @@ -799,6 +799,16 @@ class PortageInternalVariableAssignment(LineCheck):
 e += ' on line: %d'
 return e

 +class DeprecateG2CONF(LineCheck):
 +   repoman_check_name = 'G2CONF.deprecated'
 +   re = re.compile(r'.*G2CONF.*')
 +
 +   def check(self, num, line):
 +   Run the check on line and return error if there is
one
 +   m = self.re.match(line)
 +   if m is not None:
 +   return (G2CONF on line %d is deprecated, see
Gentoo bug #482084.)
Are you missing the line number interpolation here? or %d is supposed to be
returned?
 +
  _base_check_classes = (InheritEclass, LineCheck, PhaseCheck)
  _constant_checks = None

 --
 1.8.5.2


Other than that, Looks good to me.

Re: [gentoo-portage-dev] [PATCH 1/3] Have repoman check if the packages to unpack rare archive formats from SRC_URI are present in DEPEND (bug #205909).

[Alec Warner]

On Wed, Jan 15, 2014 at 4:07 PM, Tom Wijsman tom...@gentoo.org wrote:

 ---
  bin/repoman   | 53 +
  man/repoman.1 |  4 
  2 files changed, 57 insertions(+)


I urge you to not author new checks like this. /usr/bin/repoman is already
a mess.

Write these checks as functions, put them in a different file, and just
call them from the giant messy loop. At least that way the checks are self
contained (great for avoiding things like variable re-use or shadowing).

-A



 diff --git a/bin/repoman b/bin/repoman
 index d1542e9..9b703dc 100755
 --- a/bin/repoman
 +++ b/bin/repoman
 @@ -36,6 +36,9 @@ pym_path =
 osp.join(osp.dirname(osp.dirname(osp.realpath(__file__))), pym)
  sys.path.insert(0, pym_path)
  import portage
  portage._internal_caller = True
 +
 +from portage._sets.profiles import PackagesSystemSet
 +system_set_atoms = PackagesSystemSet(portage.settings.profiles).getAtoms()
  portage._disable_legacy_globals()

  try:
 @@ -300,6 +303,7 @@ qahelp = {
 inherit.missing: Ebuild uses functions from an eclass but does
 not inherit it,
 inherit.unused: Ebuild inherits an eclass but does not use it,
 java.eclassesnotused: With virtual/jdk in DEPEND you must
 inherit a java eclass,
 +   unpack.DEPEND.missing: A rare archive format was used in
 SRC_URI, but its package to unpack it is missing in DEPEND.,
 wxwidgets.eclassnotused: Ebuild DEPENDs on x11-libs/wxGTK
 without inheriting wxwidgets.eclass,
 KEYWORDS.dropped: Ebuilds that appear to have dropped KEYWORDS
 for some arch,
 KEYWORDS.missing: Ebuilds that have a missing or empty KEYWORDS
 variable,
 @@ -399,6 +403,7 @@ qawarnings = set((
  metadata.warning,
  portage.internal,
  repo.eapi.deprecated,
 +unpack.DEPEND.missing,
  usage.obsolete,
  upstream.workaround,
  LIVEVCS.stable,
 @@ -479,6 +484,25 @@ ruby_deprecated = frozenset([
 ruby_targets_ree18,
  ])

 +# TODO: Add functionality to support checking for deb2targz on platforms
 where
 +#   GNU binutils is absent; see PMS 5, section 11.3.3.13.
 +archive_formats = {
 +   \.7[zZ]:app-arch/p7zip,
 +   \.(bz2?|tbz2):app-arch/bzip2,
 +   \.jar:app-arch/unzip,
 +   \.(LH[aA]|lha|lzh):app-arch/lha,
 +   \.lzma:app-arch/lzma-utils,
 +   \.(rar|RAR):app-arch/unrar,
 +   \.(tar(\.(bz2?|gz|Z))?|tbz2|t[bg]z)?:app-arch/tar,
 +   \.(gz|tar\.Z|t[bg]z|[zZ]):app-arch/gzip,
 +   \.(zip|ZIP):app-arch/unzip,
 +}
 +
 +archive_formats_eapi_3_to_5 = {
 +   \.tar.xz:app-arch/tar,
 +   \.xz:app-arch/xz-utils,
 +}
 +
  metadata_xml_encoding = 'UTF-8'
  metadata_xml_declaration = '?xml version=1.0 encoding=%s?' % \
 (metadata_xml_encoding,)
 @@ -1559,6 +1583,7 @@ for x in effective_scanlist:
 fetchlist_dict = portage.FetchlistDict(checkdir, repoman_settings,
 portdb)
 myfiles_all = []
 src_uri_error = False
 +   needed_unpack_depends = {}
 for mykey in fetchlist_dict:
 try:
 myfiles_all.extend(fetchlist_dict[mykey])
 @@ -1573,7 +1598,22 @@ for x in effective_scanlist:
 stats[SRC_URI.syntax] += 1
 fails[SRC_URI.syntax].append(
 %s.ebuild SRC_URI: %s % (mykey,
 e))
 +
 +   # Compare each SRC_URI entry against archive_formats; if
 one of the
 +   # extensions match, we remember which archive depends are
 needed to
 +   # check them later on.
 +   needed_unpack_depends[mykey] = []
 +   for file_extension in archive_formats or \
 +   ((re.match('[345]$', eapi) is not None) \
 +   and file_extension in
 archive_formats_eapi_3_to_5):
 +   for entry in fetchlist_dict[mykey]:
 +   if re.match('.*%s$' % file_extension,
 entry) is not None:
 +   format =
 archive_formats[file_extension]
 +
 +   if format not in
 needed_unpack_depends[mykey]:
 +
 needed_unpack_depends[mykey].append(format)
 del fetchlist_dict
 +
 if not src_uri_error:
 # This test can produce false positives if SRC_URI could
 not
 # be parsed for one or more ebuilds. There's no point in
 @@ -2010,6 +2050,17 @@ for x in effective_scanlist:
 atoms = None
 badsyntax.append(str(e))

 +   if atoms and mytype == 'DEPEND':
 +   # We check whether the needed archive
 dependencies are present
 +   # in DEPEND, which were determined from
 SRC_URI.
 +   for entry in needed_unpack_depends[catdir
 + '/' + y]:
 +   if entry not in system_set_atoms
 and entry \
 +  

Re: [gentoo-portage-dev] [PATCH 1/3] Have repoman check if the packages to unpack rare archive formats from SRC_URI are present in DEPEND (bug #205909).

[Sebastian Luther]

Am 16.01.2014 01:07, schrieb Tom Wijsman:
 ---
  bin/repoman   | 53 +
  man/repoman.1 |  4 
  2 files changed, 57 insertions(+)
 
 diff --git a/bin/repoman b/bin/repoman
 index d1542e9..9b703dc 100755
 --- a/bin/repoman
 +++ b/bin/repoman
 @@ -36,6 +36,9 @@ pym_path = 
 osp.join(osp.dirname(osp.dirname(osp.realpath(__file__))), pym)
  sys.path.insert(0, pym_path)
  import portage
  portage._internal_caller = True
 +
 +from portage._sets.profiles import PackagesSystemSet
 +system_set_atoms = PackagesSystemSet(portage.settings.profiles).getAtoms()
  portage._disable_legacy_globals()
  

You should be using repoman_settings instead of portage.settings.

Considering the later use, you don't need PackagesSystemSet set here,
just use a set. And use atom.cp instead of the atoms.

  try:
 @@ -300,6 +303,7 @@ qahelp = {
   inherit.missing: Ebuild uses functions from an eclass but does not 
 inherit it,
   inherit.unused: Ebuild inherits an eclass but does not use it,
   java.eclassesnotused: With virtual/jdk in DEPEND you must inherit a 
 java eclass,
 + unpack.DEPEND.missing: A rare archive format was used in SRC_URI, 
 but its package to unpack it is missing in DEPEND.,
   wxwidgets.eclassnotused: Ebuild DEPENDs on x11-libs/wxGTK without 
 inheriting wxwidgets.eclass,
   KEYWORDS.dropped: Ebuilds that appear to have dropped KEYWORDS for 
 some arch,
   KEYWORDS.missing: Ebuilds that have a missing or empty KEYWORDS 
 variable,
 @@ -399,6 +403,7 @@ qawarnings = set((
  metadata.warning,
  portage.internal,
  repo.eapi.deprecated,
 +unpack.DEPEND.missing,
  usage.obsolete,
  upstream.workaround,
  LIVEVCS.stable,
 @@ -479,6 +484,25 @@ ruby_deprecated = frozenset([
   ruby_targets_ree18,
  ])
  
 +# TODO: Add functionality to support checking for deb2targz on platforms 
 where
 +#   GNU binutils is absent; see PMS 5, section 11.3.3.13.
 +archive_formats = {
 + \.7[zZ]:app-arch/p7zip,
 + \.(bz2?|tbz2):app-arch/bzip2,
 + \.jar:app-arch/unzip,
 + \.(LH[aA]|lha|lzh):app-arch/lha,
 + \.lzma:app-arch/lzma-utils,
 + \.(rar|RAR):app-arch/unrar,
 + \.(tar(\.(bz2?|gz|Z))?|tbz2|t[bg]z)?:app-arch/tar,
 + \.(gz|tar\.Z|t[bg]z|[zZ]):app-arch/gzip,
 + \.(zip|ZIP):app-arch/unzip,
 +}
 +
 +archive_formats_eapi_3_to_5 = {
 + \.tar.xz:app-arch/tar,
 + \.xz:app-arch/xz-utils,
 +}
 +
  metadata_xml_encoding = 'UTF-8'
  metadata_xml_declaration = '?xml version=1.0 encoding=%s?' % \
   (metadata_xml_encoding,)
 @@ -1559,6 +1583,7 @@ for x in effective_scanlist:
   fetchlist_dict = portage.FetchlistDict(checkdir, repoman_settings, 
 portdb)
   myfiles_all = []
   src_uri_error = False
 + needed_unpack_depends = {}
   for mykey in fetchlist_dict:
   try:
   myfiles_all.extend(fetchlist_dict[mykey])
 @@ -1573,7 +1598,22 @@ for x in effective_scanlist:
   stats[SRC_URI.syntax] += 1
   fails[SRC_URI.syntax].append(
   %s.ebuild SRC_URI: %s % (mykey, e))
 +
 + # Compare each SRC_URI entry against archive_formats; if one of 
 the
 + # extensions match, we remember which archive depends are 
 needed to
 + # check them later on.
 + needed_unpack_depends[mykey] = []
 + for file_extension in archive_formats or \
 + ((re.match('[345]$', eapi) is not None) \

Use portage.eapi for the line above. You may have to add a new function
to portage.eapi.

 + and file_extension in 
 archive_formats_eapi_3_to_5):
 + for entry in fetchlist_dict[mykey]:
 + if re.match('.*%s$' % file_extension, entry) is 
 not None:
 + format = archive_formats[file_extension]

As these regex are used frequently, they should be compiled using
re.compile.

 +
 + if format not in 
 needed_unpack_depends[mykey]:
 + 
 needed_unpack_depends[mykey].append(format)

I'd make needed_unpack_depends[mykey] a set. Then you can just add()
instead of checking and appending.

   del fetchlist_dict
 +
   if not src_uri_error:
   # This test can produce false positives if SRC_URI could not
   # be parsed for one or more ebuilds. There's no point in
 @@ -2010,6 +2050,17 @@ for x in effective_scanlist:
   atoms = None
   badsyntax.append(str(e))
  
 + if atoms and mytype == 'DEPEND':

Use if atoms and buildtime: here.

 + # We check whether the needed archive 
 dependencies are present
 + # in DEPEND, which were determined from SRC_URI.
 + for entry in