Re: [gentoo-dev] EAPI 6 portage is out!

[Rich Freeman]

On Wed, Nov 18, 2015 at 6:12 AM, Alexander Berntsen  wrote:
> When I do QA in projects I'm involved with (at least outside of
> Gentoo), we don't do it live on end-user systems. I'll leave the
> details as an exercise for the Gentoo developer.
>

People who run ~arch are not really end-users - they're contributors
who have volunteered to test packages.

But, if you want to contribute a unit testing framework for portage
I'm sure it would be welcome!

-- 
Rich

Re: [gentoo-dev] EAPI 6 portage is out!

[Rich Freeman]

On Wed, Nov 18, 2015 at 6:05 AM, Ulrich Mueller  wrote:
>
> And on what basis would you stabilise Portage, when there are no
> ebuilds in the tree to test its EAPI 6 code?
>

As long as the EAPI6 code in the new portage is no more broken than
the EAPI6 code in the current stable version of portage (which doesn't
work/exist at all), it isn't much of a regression.  What would be more
of a pain is dealing with fixes in stable.

But, I don't have a problem with starting to use EAPI6 now, mainly
because the ~arch version of portage does not require any new ~arch
dependencies that would create a mess for stable users.  So, if a user
needs to switch to a newer portage for a month or two it shouldn't be
that big of a deal.

Actually, what is less clear to me is how portage versioning actually
works, or if we attach any meaning to the version numbers at all.
Both the stable and unstable series are on 2.2.x, but there are no
versions in the tree between 2.2.20 and 2.2.23.

The main thing I find painful in following ~arch on the odd package is
when maintainers drop versions quickly after bumping them.  That tends
to result in a situation where if you follow ~arch you end up having
to accept lots of updates because none of the versions stay in the
tree long enough to actually get stabilized.  Unless a ~arch package
version is so broken that it could never be stabilized it is probably
better to leave them there so that it is easier for users to drop back
from ~arch to stable without downgrading.

-- 
Rich

Re: [gentoo-dev] Re: [gentoo-commits] repo/gentoo:master commit in: eclass/

[Rich Freeman]

On Tue, Nov 17, 2015 at 3:37 PM, Raymond Jennings  wrote:
> As a possibly relevant side note, I've observed how api changes are handled
> in the linux kernel:
>
> You can change whatever you want if it's a good idea, but as part of proving
> it, you have to be willing to take over the warranty for anything you break.
>
> So basically you change what you please ONLY if you also take the burden of
> fixing everything that depended on what you screwed with.
>
> Is this how things are handled by gentoo as well?
>

For the most part, yes, though sometimes changes are posted well in
advance with the goal of getting everybody to pitch in.

This is why a change like this was implemented with an EAPI change.
There is no retroactive impact of the change, so nothing in the tree
is affected.  Developers just have to fix their ebuilds before they
move to the new EAPI.

-- 
Rich

Re: [gentoo-dev] Re: EAPI 6 portage is out!

[Rich Freeman]

On Tue, Nov 17, 2015 at 8:54 PM, Duncan <1i5t5.dun...@cox.net> wrote:
>
> Gentoo has never really guaranteed the stability of a mostly-stable
> system with a few ~arch accept-keyworded packages as that's simply not a
> properly testable setup.
>

True, but Gentoo has never really guaranteed much of anything at all.
In my experience stable with a few ~arch packages tends to work just
fine, and that is the configuration I typically use.  Certainly I know
that any packages that I maintain work fine on a stable system.  I
always figured that this should be the focus of testing, since stable
is the ultimate target for anything we deploy.  If a package happens
to break on ~arch that isn't as big a deal, since that is what ~arch
users are signing up for.  Such issues should be fixed, of course.

-- 
Rich

Re: [gentoo-dev] Re: [gentoo-commits] repo/gentoo:master commit in: eclass/

[Rich Freeman]

On Mon, Nov 16, 2015 at 4:52 AM, Alexis Ballier  wrote:
> On Mon, 16 Nov 2015 10:29:43 +0100
> "Justin Lecher (jlec)"  wrote:
>
>> -BEGIN PGP SIGNED MESSAGE-
>> Hash: SHA512
>>
>> On 16/11/15 10:14, Alexis Ballier wrote:
>> > Probably those that want to ban it should fix the(ir) tree so that
>> > developers have no pain in bumping to eapi6?
>>
>> Versioned APIs are made to have incompatible changes. What do you like
>> to see?
>
> deprecation warnings for some time or..
>

Well, it sounds like you've received and understood your deprecation
warning.  :)  EAPI5 won't be going away for a long time, and you can
move to EAPI6 whenever you're ready.

-- 
Rich

Re: [gentoo-dev] Re: ChangeLog

[Rich Freeman]

On Tue, Nov 3, 2015 at 1:43 AM, Duncan <1i5t5.dun...@cox.net> wrote:
>
> Tho I get Dale's somewhat confused and now belabored point as well.  For
> people who don't know how to do git on their own, as clearly he doesn't,
> the official signaling of what the stopgap changelog alternatives were
> until the scripts were running to regenerate them from git, simply wasn't
> there.  Without that, confusion reigned, and blaming the user for
> confusion in the absence of official signaling really doesn't help the
> situation, either.
>

Fair enough.  If we had outright decided to get rid of changelogs we'd
probably have published a news item with clear instructions on how to
just obtain this info from git.  Instead we opted to keep them, but
they broke, so there was no news.

-- 
Rich

Re: [gentoo-dev] Re: ChangeLog

[Rich Freeman]

On Tue, Nov 3, 2015 at 2:22 AM, Patrick Lauer  wrote:
> Apparently my complaining finally re-triggered some action, so sadly
> this looks like the currently best strategy.

You could have simply made a simple post pointing out that changelog
generation appears to be broken and likely had the same effect.

The thing with complaining to trigger action is that it can be like
taking one step forward and two steps back.  It might appear to have
the desired effect, and then nobody wants to work in that part of the
community, or in the community at all, so in the long run we're all
worse off.  It doesn't inspire people to contribute.  The areas where
contributions are lost may not even appear to be directly related to
the areas where the complaints are made, so it is hard to demonstrate
a clear relationship, and thus people still feel vindicated in their
complaining or reluctant to do anything about it.

Can I prove that the above is true?  No.  Does it concern me?
Absolutely.  I think the fact that so many are torn between getting
rid of key contributors who tend to create a lot of drama and
continuing to tolerate them just makes people not want to try to fix
it as well.

-- 
Rich

Re: [gentoo-dev] Re: ChangeLog

[Rich Freeman]

On Tue, Nov 3, 2015 at 10:04 AM, Chí-Thanh Christopher Nguyễn
 wrote:
> Matt Turner schrieb:
>>
>> The git transition had been 9 years in the making and has massively
>> improved Gentoo development. Look at the graph of contributions per month:
>> https://www.openhub.net/p/gentoo
>
> I'd like to point out that some stuff that has previously been done in a
> single commit is now several commits (e.g. bump + removal of old version).
> How much of the rise in commit activity is attributable to actual
> development increase is not clear to me.
>

How were previous cvs commit stats generated?  CVS has not concept of
a commit across multiple files in its data model.  You can of course
look for commits to multiple files that share the same timestamp and
author and infer that these are a single commit, but if you make two
commits at the same time with the same name and description in cvs
there is no way to distinguish that from one commit that hits both
files.  In git these would be captured differently.

For the historical migration to git commits were consolidated using a
window.  Otherwise you'd get a bazillion Manifest commits on top of
everything else, to say nothing of simultaneous commits to
filesdir/etc.

But, yours is a fair point all the same.  In any case, git should be a
lot more useful overall.  Not to mention that while we might be
arguing about which 3rd-party tools are the best for improving our
workflow at least we have a choice now.

-- 
Rich

Re: [gentoo-dev] Re: ChangeLog

[Rich Freeman]

On Mon, Nov 2, 2015 at 1:24 AM, Patrick Lauer <patr...@gentoo.org> wrote:
>
> On 11/02/2015 02:56 AM, Rich Freeman wrote:
>> On Sun, Nov 1, 2015 at 8:22 PM, Duncan <1i5t5.dun...@cox.net> wrote:
>>>  I know if I were still on rsync (or webrsync), I'd be raising hell about 
>>> the lack of
>>> changelogs well before now
>> Perhaps rather than raising hell you'd do better to raise money to
>> hire an infra team to fix the bug or something.
> Hire?
> I'm still willing to fix things, if I were given access. And I would
> presume that I'm not the only one.

Well, if somebody who has a good history of working with teams and is
generally well-liked wants to step up and volunteer for the job, they
could probably ping the council for endorsement.

>
> But since I don't have access, and can only affect things by motivating
> or upsetting people,

I think you'll find that the former works better than the latter with
volunteers.  Upsetting people generally tends to result in everybody
walking out in a huff and leaving users high and dry, so you're not
really doing anybody any favors with the attitude.

>>
>> I get the frustration, but we only have a few people who have the
>> necessary access to fix the problem.
> So fix that.

Proposals are welcome.

>
> But one of the conditions for tolerating the git migration was that we
> have no regressions.

I certainly never put that condition on it.  I think we were already
trying too hard to make things perfect vs accept the change and work
out some of the details later.  Changelogs being down for a few months
isn't a big deal IMO, especially since the data is all available in
git anyway.

If I didn't think git was ready to go I'd have voted to stop it, and
the Council probably could have done this.  Sometimes you just have to
embrace a change and deal with the fallout around minor issues like
this.  It isn't like Gentoo ground to a halt for a week.

> (And as a consequence, why doesn't it then get fixed in a reasonable time?)

I don't know, why don't you ask the responsible person's manager to
document this on their annual performance review?

> But at least now we get some good information what is broken how, and
> maybe someone can fix it. And then I won't have to be the stone in
> people's shoe anymore ;)

You don't have to be that now.

I get it, you don't like the state of affairs.  Join the club - I
think we're all in it already.  IMO part of the reason we're
struggling is that Gentoo was structured to work the way it does back
when we had at least 3x as much activity in some of the key projects.
The model doesn't scale down well, especially around key roles like
Infra where the design isn't open enough for others to effectively
contribute patches.

-- 
Rich

Re: [gentoo-dev] Re: ChangeLog

[Rich Freeman]

On Mon, Nov 2, 2015 at 1:08 AM, Dale  wrote:
>
> Then perhaps all this should have been worked out BEFORE switching to
> github?

We didn't switch to github.

>
> I don't mind change but it seem this one wasn't really ready to be done
> yet although most made it sound like it was.

IMO we took too long as it is.  I'd have pushed it faster, but just as
I don't have access to fix this bug I didn't have access to implement
git, and those with access wanted to keep Changelogs, so we did.

If you haven't already gotten the impression, Gentoo is mostly a
conglomeration of various associations of devs who tend to do their
own thing, each with different sets of knowledge/access.  When there
is a direct conflict between two groups we generally (if often not
enthusiastically) accept the votes of the Council to settle disputes.
For the most part the Council has the ability to tell devs to NOT do
things.  When something needs to be done somebody has to step up and
do it.  Complaining on the lists mainly seems to de-motivate people
from working on anything.  By all means point out that something isn't
working, but attitudes tend to be contagious.

-- 
Rich

Re: [gentoo-dev] Re: ChangeLog

[Rich Freeman]

On Mon, Nov 2, 2015 at 8:20 PM, Dale  wrote:
> I thought Gentoo was not depending on git/github either.

Take 5min and read the wikipedia articles on both git and github, please.

Gentoo is not going to depend on github, because of the social contract issues.

Gentoo absolutely does depend on git, and it is 100% FOSS.

If these statements seem contradictory, you really need to look up a
video on git 101/etc.  To be fair, I don't think you can truly use git
without first groking it, and you won't accomplish that until you
understand its data model.  Git is a terrific data model wrapped in a
mediocre command line utility.

-- 
Rich

Re: [gentoo-dev] Re: ChangeLog

[Rich Freeman]

On Mon, Nov 2, 2015 at 9:31 PM, Dale  wrote:
> That is why a link was posted for me to use github instead.  I
> do realize and understand that git and github are two different things
> but it seems they can work together as well.  It ended up that the info
> I needed was on github but not to be found on any Gentoo site at the time.

Anything in /usr/portage that you can find on github is also on
https://gitweb.gentoo.org/repo/gentoo.git/, which is a Gentoo site.

That's kind of the point of git - there are a bazillion tools
available for it and it makes it very easy to clone a full repository
with full history.  It also lowers the bar to contribution.

I get that you're frustrated with the change, and there are a few
others that are as well, but thousands of people use Gentoo, and
generally people only bother to post on lists when they're frustrated.
We can't go into panic mode every time somebody raises a complaint,
and ultimately everybody here is a volunteer.

The complaints don't really bother me much personally, but I do get
concerned that they'll discourage others from contributing.  I can
just ignore threads like these easily enough, but people get
frustrated when they contribute and others just criticize.

-- 
Rich

Re: [gentoo-dev] ChangeLog

[Rich Freeman]

On Sun, Nov 1, 2015 at 7:33 AM, Мисбах-Соловьёв Вадим  wrote:
> And why don't just only generate them on rsync mirrors, but remove them from 
> git repo (like was planned initially, AFAIRC)?
>

That is in fact how it works.  Or, at least how it is supposed to
work.  I don't use the rsync mirror, so I can't vouch for whether
they're producing ChangeLogs.

Personally I'd just as soon see them go away entirely, but if somebody
wants to make them work I won't stop them.

-- 
Rich

Re: [gentoo-dev] ChangeLog

[Rich Freeman]

On Sun, Nov 1, 2015 at 10:24 AM, Alexis Ballier <aball...@gentoo.org> wrote:
> On Sun, 1 Nov 2015 10:17:54 -0500
> Rich Freeman <ri...@gentoo.org> wrote:
>>
>> I haven't heard anybody propose a new plan.  I certainly am not
>> proposing one.
>
> The part you cut:
>
>>
>> You shouldn't use rsync anymore, it is inherently insecure. The git
>> tree is _properly_ gpg signed so you can verify it's correctness.
>>

That was just a random developer offering random advice.  People are
welcome to do that on the lists.  Nobody is preventing anybody from
fixing the bug.

There is no approved grandiose plan to obsolete rsync.

-- 
Rich

Re: [gentoo-dev] ChangeLog

[Rich Freeman]

On Sun, Nov 1, 2015 at 10:00 AM, Alexis Ballier <aball...@gentoo.org> wrote:
> On Sun, 1 Nov 2015 09:19:25 -0500
> Rich Freeman <ri...@gentoo.org> wrote:
>
> [...]
>> What discussion or decision is necessary?
>
> One that announces the initial and current plan has changed and
> describes the new plan maybe?
>

I haven't heard anybody propose a new plan.  I certainly am not proposing one.

Like I said, I don't have a problem with there being Changelogs in
rsync.  By all means go fix it.

-- 
Rich

Re: [gentoo-dev] ChangeLog

[Rich Freeman]

On Sun, Nov 1, 2015 at 8:47 AM, Alexis Ballier  wrote:
> Considering the original plan was to have changelogs auto-generated
> from git and still serving the tree via rsync, where's the relevant
> discussion and decision about this?

What discussion or decision is necessary?  As far as I'm aware nobody
has forbidden making changelogs available via rsync.  It just sounds
like there is a bug in their generation.  What is needed is for those
who want changelogs to fix the bug, not endless discussion.  If the
issue is infra access, just make your own mirror with working code and
I'm sure infra will borrow it, and if not nobody really HAS to use
infra.  I'm syncing from the github mirror that contains pre-generated
metadata for convenience, which is just one of the many options
available.

Nobody is actively preventing anybody from having what they want.
This isn't some corporation where we are paying people and we can
demand that the responsible parties fix things or lose their jobs.

Lots of effort has gone into making the git migration as seamless as
possible, but it was bound to be imperfect.  Personally I would have
been fine with less effort being spent on it than actually was.
Gentoo has never been a hand-holding distro.  I have nothing against
people who choose to invest their time into making it more helpful to
those who wish to use alternate tools (like changelogs), but I don't
favor telling those who are working on new features to not actually
deploy them unless THEY spend their time on such things as long as we
have a reasonable path forward for everybody.

So, if you want to see what has changed there are half a dozen ways of
doing it without using changelogs.

-- 
Rich

Re: [gentoo-dev] Re: ChangeLog

[Rich Freeman]

On Sun, Nov 1, 2015 at 3:33 PM, Martin Vaeth  wrote:
> Please do not break all these possibilities for users
> who do not have to waste the resources for a full git
> clone and want to see regularly ChangeLogs nevertheless!

I don't think anybody has proposed breaking anything.  It sounds like
it is already broken, and somebody needs to fix it.

Keep in mind that "resources" is a vague term and for some resources
either git or rsync can be cheaper.  Rsync will tend to require less
local disk space than git (unless you try to purge all the history out
of the repositories, which of course defeats the goal of having
Changelogs).  On the other hand, git should require far less disk io
and CPU to sync since it doesn't have to rely on stat-ing every file
(and if you want rsync to be truly reliable you need to tell it to
hash everything which adds a boatload of additional IO - git doesn't
rely on mtime).

-- 
Rich

Re: [gentoo-dev] Re: ChangeLog

[Rich Freeman]

On Sun, Nov 1, 2015 at 8:22 PM, Duncan <1i5t5.dun...@cox.net> wrote:
>  I know if I were still on rsync (or webrsync), I'd be raising hell about the 
> lack of
> changelogs well before now

Perhaps rather than raising hell you'd do better to raise money to
hire an infra team to fix the bug or something.

I get the frustration, but we only have a few people who have the
necessary access to fix the problem.  Infra is also a difficult
project to deal with in general because it is fairly closed due to the
implications of having random people messing with it.  I don't really
see anybody stepping up to try to change anything fundamental about it
either.  This isn't the sort of thing that will get better if the
council votes on something.

-- 
Rich

Re: [gentoo-dev] Re: ChangeLog

[Rich Freeman]

On Sun, Nov 1, 2015 at 10:29 AM, Martin Vaeth <mar...@mvath.de> wrote:
> Rich Freeman <ri...@gentoo.org> wrote:
>>
>> What discussion or decision is necessary?
>> What is needed is for those who want changelogs
>> to fix the bug
>
> The bug can only be fixed by somebody who knows
> the details how the rsync mirrors are set up.

And that is really the bigger problem here.  I think we really need to
minimize dependency on stuff that only a few people have access to,
since they're overworked.  I'd certainly encourage somebody to offer
up a fixed rsync server.  While you're at it, a gitlab/whatever server
so that we don't have to keep arguing over github would also be nice.

I'd really like to see our infrastructure get to a point where it is
all published FOSS minus maybe a few authentication tokens so that
anybody can just fork it on demand.  Even though we'd probably still
want the officially-designated servers it would make it far easier for
others to contribute if they could actually test it all out.
Authentication could use openid or whatever so that a clone could be a
first-class alternative.

In any case though, this isn't some vast conspiracy to get rid of
Changelogs.  More likely there are only a few people who can fix them,
and they simply haven't gotten around to it.

-- 
Rich

Re: [gentoo-dev] ssl vs openssl vs libressl vs gnutls USE flag foo

[Rich Freeman]

On Fri, Oct 30, 2015 at 5:16 PM, Anthony G. Basile  wrote:
> On 10/30/15 3:35 PM, hasufell wrote:
>>
>> On 10/30/2015 06:55 PM, Michał Górny wrote:
>>>
>>> We have no way of saying 'I prefer polarssl, then gnutls, then
>>> libressl, and never openssl'.
>>
>> I don't think this is something that can be reasonably supported and it
>> sounds awfully automagic. And I don't see how this is possible right
>> now, so I'm not really sure what you expect to get worse.
>>
>> E.g. -gnutls pulling in dev-libs/openssl is not really something you'd
>> expect. If we go for provider USE flags, then things become consistent,
>> explicit and unambiguous. The only problem is our crappy implementation
>> of providers USE flags via REQUIRED_USE.
>>
> I'm not sure what mgorny has in mind, but the problem I see with saying I
> want just X to be my provider system wide is that some pkgs build with X
> others don't, other pkgs might need a different provider.  So it might make
> sense to order them in terms of preference: X1 > X2 > X3 ... and then when
> emerging a package, the first provider in the preference list that works is
> pulled in for that package.

I think that would be useful in general.  It would probably not be
useful in this case, since it was somebody's bright idea to make it
essentially impossible to install two of the options on the same
system (and that wasn't directed at hasufell).  Users could of course
still express the preference, but the PM would need to be smart enough
to ignore that preference on 95% of packages that support both options
so that it can take the lower preference on the 5% of packages that
only support the option the user didn't really want.

-- 
Rich

Re: [gentoo-dev] ssl vs openssl vs libressl vs gnutls USE flag foo

[Rich Freeman]

On Fri, Oct 30, 2015 at 1:55 PM, Michał Górny  wrote:
>>
>> The pain is for a short time.  Then we have to live with this for a
>> long time.  USE flags should have one meaning.  The fact that this
>> isn't the case right now is already a bug.  We don't need to
>> perpetuate it.
>
> No, the pain is neverending. You define a number of flags which are
> scattered all over the place and there's practically no good value but
> the 'default'.
>

My response was intended as a comparison of the two options presented,
which so far are the only options that have been suggested by anybody
that don't require EAPI changes.

I wasn't suggesting that there wasn't room for improvement in general.
However, short of banning libressl until EAPI7 and actually doing
something in EAPI7 this is our current best option.

-- 
Rich

Re: [gentoo-dev] Re: ssl vs openssl vs libressl vs gnutls USE flag foo

[Rich Freeman]

On Wed, Oct 28, 2015 at 7:16 AM, hasufell  wrote:
>
> This is outside of the scope of this thread, but there are already
> distros that have fixed this:
> 1. NixOS [0] with truly declarative configuration format, e.g. something
> like:
> packages.ssl.provider = openssl;

Well, we can accomplish this in our syntax.  Just RDEPEND on openssl,
and set USE requirements for openssl on any dependencies that offer
both.

NixOS is still bound by the constraint that the two libraries have
colliding namespace, so a package needs to have a dependency chain
that exclusively uses one or the other.

However, assuming all your packages are able to work with either
library the thing NixOS does have going for it is that it would let
you have apache using openssl and postfix using libressl on the same
system, with side-by-side versions of any shared dependencies built
against each.

Their approach (as I understand it) is basically that every process is
almost containerized on the same filesystem.

>
> which is a lot cleaner than USE_EXPAND + REQUIRED_USE, which still can
> have arbitrary meanings.
>

Well, I think we can accomplish all of the above using the tools we
already have, but I agree that we tend to do it in one namespace while
other distros are using more than one.  That is probably a good idea
just to improve consistency.

We should probably pursue both.  For ssl we need the best solution we
can implement today.  However, for a future EAPI we should pursue a
better way to handle this.

-- 
Rich

Re: [gentoo-dev] ssl vs openssl vs libressl vs gnutls USE flag foo

[Rich Freeman]

On Tue, Oct 27, 2015 at 10:06 PM, hasufell  wrote:
>
> B) 1 feature flag, 3 strict provider flags
> * ssl: enable any sort of SSL/TLS support
> * gnutls: only to enable gnutls provided ssl support in case there
>   is a choice
> * openssl: only to enable openssl provided ssl support in case
>there is a choice (should not be implemented as !gnutls?)
> * libressl: only to enable libressl provided ssl support in case there
> is a choice, must conflict with 'openssl' USE flag
>
> consequences:
> * REQUIRED_USE="^^ ( openssl libressl )" is not only allowed, it is
>   _mandatory_
> * packages like media-video/ffmpeg _must_ switch the USE flag
>   openssl->ssl to avoid breaking global USE flags
> * !gnutls? ( dev-libs/openssl:0 ) will be bad form or even disallowed
>
> B will definitely be more work, but ofc is also a lot cleaner and
> totally unambigous.
>

++

The pain is for a short time.  Then we have to live with this for a
long time.  USE flags should have one meaning.  The fact that this
isn't the case right now is already a bug.  We don't need to
perpetuate it.

Honestly, this just seems like "the right thing" so if there isn't
opposition then I'd suggest to "just do it" and commit fixes to
ebuilds that need the fix (ie if maintainer doesn't respond to bug
quickly just take care of it).  If people object they should speak up
now, and we can take it up at the next council meeting if necessary
(which is right around the corner).

-- 
Rich

Re: [gentoo-dev] Why is my news item not showing up.

[Rich Freeman]

On Wed, Oct 21, 2015 at 4:44 AM, Anthony G. Basile  wrote:
>
> I pushed out my news item and it landed in /usr/portage/metadata on my
> hardened servers, but its not showing up with eselect news.  Does anyone
> know why?

1.  Do you have hardend-sources installed?
2.  Do you have either hardened or pax_kernel in your ACCEPT_KEYWORDS?
3.  Do you have one of the listed profiles selected?

If the answer to any of those questions is no, then that's your
problem - according to glep 42 the individual checks are ORs, and
they're combined by AND.

-- 
Rich

Re: [gentoo-dev] News Item: Future Support of hardened-sources Kernel

[Rich Freeman]

On Tue, Oct 20, 2015 at 4:23 AM, Daniel Campbell  wrote:
> However, does this mean the hardened kernel package must stay in ~arch
> since it's technically the testing version? Or would we keyword it
> based on our own findings of stability?

I'd recommend that the team does whatever adds the most value.  If it
doesn't want to do QA on released versions then I suggest it all stay
as ~arch.  If you're going to do your own QA I don't see why you can't
mark versions as stable - just make it clear to users what stable
means.

BTW, while they're only tracking the most recent stable branch of the
kernel, they ARE tracking a stable branch, and not mainline.

-- 
Rich

Re: [gentoo-dev] Re: [gentoo-dev-announce] EAPI 6 draft for review

[Rich Freeman]

On Tue, Oct 20, 2015 at 3:51 AM, Alexis Ballier <aball...@gentoo.org> wrote:
> On Mon, 19 Oct 2015 15:49:06 -0400
> Rich Freeman <ri...@gentoo.org> wrote:
>
> It's not about correctness vs convenience: eapply_user idempotent
> doesn't prevent from doing it correctly. It makes it possible to do it
> incorrectly though, just like any turing-complete language actually, but
> it is not clear at all what would be the ratio of "incorrect convenient
> usage" vs "correct convenient usage": if 99.9% of the tree is fine,
> then convenience is clearly worth it instead of changing everything to
> accommodate the remaining 0.1%.

Sure, but that is the point of correctness vs convenience.  The goal
of correctness is to make it not possible to do things incorrectly,
even if that makes it harder to do things correctly.  The goal of
convenience is to make it easier to do things correctly, even if it
makes it possible to do things incorrectly.  It is a tradeoff.

> eapply_user dying on second call, defined in PMS, OTOH, prevents
> such flexibility and thus raises a lot of design questions that, as seen
> here, don't seem to have a clear answer yet and for which it could
> indeed be worth refactoring.

It has a perfectly clear answer - just don't run eapply_user in eclasses.

It isn't a convenient answer, perhaps, but it will always work.

Apologies to the rest of the Council if this wasn't on everybody's
radar back when we were discussing EAPI6.  I thought we were mostly on
the same page about taking this sort of approach already.  I guess it
doesn't hurt to be more explicit and less clever in the wording.  I
don't think we wanted to outright ban eclass use of eapply_user though
- it is still appropriate to use in situations where ebuilds are just
wrappers around an eclass.

> I can't find an example of where the guideline "apply ebuild
> patches before eapply_user" would not be sufficient: After all,
> all subsequent phases will also run on user-patched sources and there
> are millions of ways to make a patch break a package.

How would you apply ebuild patches before eapply_user when you have
multiple eclasses all applying patches and all calling eapply_user?

Adding another phase may be the cleaner solution.  It is a bit late to
be talking about that, however, for EAPI6.

-- 
Rich

Re: [gentoo-dev] Re: [gentoo-dev-announce] EAPI 6 draft for review

[Rich Freeman]

On Tue, Oct 20, 2015 at 5:22 AM, Alexis Ballier  wrote:
>
> Ok, that's what I'd call "forced correctness" :)
> But again, theory tells you that if you want algorithmically checkable
> correctness then you have to seriously limit your possibilities, which
> is why I usually don't even consider this tradeoff :)
>

I'm not suggesting we're achieving perfection here.  However, I think
that a lot of how we do things encourages lazy ebuild writing.  That
does have a cost in QA, and maintainability.

Now, making it harder to write ebuilds also has a cost in fewer
ebuilds getting written.  So, this is a trade-off.  I do get that.

>
> First, eclasses shouldn't apply patches on their own but take what the
> ebuild tells it to apply: With multiple eclasses applying random
> patches on their own, you're already asking for trouble.
> Then, ebuild can just send all its patches through the first eclass,
> which will call the real eaplly_user.
>
> Sure, there can be imaginary cases where this would horribly break or
> not be so convenient, but do we have a real example ?

That is a fair point.  I think there are other problems with eclasses
exporting phase functions which are far more likely to cause issues
than patching at the wrong spot.

I guess the question is whether I/we/whoever is turning eapply_user
into a big issue more to force the general move away from exporting
phase functions than because it is a big issue on its own.  I do think
that is the direction we ought to be moving in, but I would agree that
we shouldn't be using eapply_user as a club to try to force people to
move that way.  If we want to discourage eclass phase function
exporting, we should just do that on its own merits.

So, perhaps it is a fair question to ask what is the specific harm
from allowing it to be a no-op on subsequent calls, other than
encouraging a coding practice that could possibly have other unrelated
effects?

-- 
Rich

Re: [gentoo-dev] stabilization commits and atomicity

[Rich Freeman]

On Mon, Oct 19, 2015 at 7:55 AM, Dirkjan Ochtman  wrote:
> On Mon, Oct 19, 2015 at 1:21 PM, hasufell  wrote:
>> I'd go so far to say allow people to do commits like:
>> """
>> amd64 stabilizations
>>
>> 
>> """
>> possibly pre-pending the rough domain like "kde", if any. I think kde
>> herd already does that, no?
>
> Sounds sane to me.

I think that standardizing how we comment on bulk-stabilization
commits makes more sense than making them less atomic.  Not getting
half a KDE update is actually one of the nice selling features of git.
Plus, in the event of a disaster it also makes rollback easier.

But, by all means we should update the wiki to recommend the standard
way to document these changes.

-- 
Rich

Re: [gentoo-dev] Re: [gentoo-dev-announce] EAPI 6 draft for review

[Rich Freeman]

On Mon, Oct 19, 2015 at 3:12 AM, Alexis Ballier  wrote:
> But there is something important we've overlooked: should eclasses that
> export src_prepare call eapply_user ? I think yes, otherwise they'd
> make packages inheriting them violate the 'at least once rule'.

This sort of thing has been discussed by the council, though we didn't
make a specific recommendation on the eclass authoring side.  The
exactly once language was specifically chosen to force eclass/ebuild
authors to consider this problem, because bad things happen if you
don't.

I'd argue that eclasses should fall into two categories:
1.  Utility eclasses.  These are things like git.eclass that export
useful functions that ebuilds can call.  They shouldn't export phase
functions, so ebuild authors can use as many of these as they want and
they'll never conflict.
2.  Do-everything eclasses.  These are things like the kde eclasses
which operate with the design that the packages themselves do almost
nothing, and all the work is in the eclass.  It allows things like
splitting up kde ebuilds without having a ton of duplicated code in
hundreds of packages.  These should export phase functions, and you
tend to run into all kinds of problems if you try to use more than one
of them, so you shouldn't.

So, if you're writing a utility eclass you shouldn't be exporting
src_prepare, and so you're fine.  By all means export a function
designed to be called during src_prepare, but don't export the phase
itself.

If you're writing a do-everything eclass then do whatever makes the
most sense.  Most likely you're co-maintianing every package that uses
the eclass anyway.  Just document how you're doing it.

The real problem is when people try to do something in-between, which
happens with a lot of language-based eclasses it seems.  Then when you
have some package that uses three different languages in its build
system you get to deal with a mess of which eclass should be doing
what.  We really need to get away from this.

I'd say the best approach for compatibility if you have an existing
eclass and it already exports src_prepare is to not call eapply_user
unless it firmly falls into the #2 category above.

However, IMO we should be trying to move away from these kinds of
eclasses (other than the two cases above).  It is really pretty to
just type inherit foo at the top of your ebuild and watch the magic
happen, until it isn't.

As has already been pointed out, if we allow eapply_user to be called
anything other than exactly once, then you run into various situations
where it ends up not being called correctly.  And we can hardly make
it a NOOP every time but the last time unless we do two-pass
src_prepare or solve the halting problem, neither of which is
appealing.

-- 
Rich

Re: [gentoo-dev] Re: [gentoo-dev-announce] EAPI 6 draft for review

[Rich Freeman]

(To avoid repeating the same exception over and over, please
understand that nothing said below is intended to apply to the
do-everything eclasses used by KDE/etc, where the eclass and ebuilds
are carefully maintained in conjunction with each other.)

On Mon, Oct 19, 2015 at 9:34 AM, Alexis Ballier  wrote:
>
> However, by somewhat throwing away the problem, you're just asking for
> a complete mess

The intent of the next paragraph was to not throw away the problem.

>
>> I'd say the best approach for compatibility if you have an existing
>> eclass and it already exports src_prepare is to not call eapply_user
>> unless it firmly falls into the #2 category above.
>
> Replace 'not call eapply_user' by 'not export src_prepare' and I'd
> agree with you if going a bit further by ensuring there is no hidden
> problem:

Well, taken together my recommendation does amount to:
1.  Avoid exporting src_prepare at all.
2.  If you do export src_prepare, then don't call eapply_user.

That means that anybody creating an ebuild that uses an eclass which
does export src_prepare should define the phase in the ebuild, call
the various eclass src_prepares in the appropriate orders, and call
eapply_user in the appropriate place.  Since the ebuild needs to be
modified to use EAPI6 it can be done at that time.

> Going through each eclass exporting src_prepare and defining
> which should export it and which should not. I hope what you say is
> sufficient, but it'd be a bad idea to set this in stone for realizing
> in a few months that this forces people to write crappy code for having
> some eclasses to co-exist nicely.

You already need to write crappy code to get some eclasses to co-exist
nicely, because they export conflicting phase functions assuming
they're the only eclass you'll use.

The eclass docs already indicate which ones export src_prepare.  If
you're using one of those you need to make sure you're overriding it,
and calling eapply_user.  As long as eclasses don't call eapply_user
we're fine.

>
> Some temptative list of which could be annoying (as in, do not seem to
> be 'do everything' eclasses):
>
> bzr.eclass
>   -> patches (now useless) and bootstrap script, dropping it might
>   encounter some resistance
> cuda.eclass
>   -> appends cflags, could easily be moved to src_configure and not
>   exported
> java-pkg-opt-2.eclass
>   -> sanity checks & autofixing, not sure how to handle
> subversion.eclass
>   -> patches (now useless) and bootstrap script, dropping it might
>   encounter some resistance
> vala.eclass
>   -> sets up some kind of virtual env, could very well be src_configure

The solution to these kinds of problems isn't to remove functionality
from the eclass, but rather just export a function that ebuilds can
call from src_prepare at the appropriate time, rather than just trying
to do it all magically.

This was discussed fairly extensively at:
http://article.gmane.org/gmane.linux.gentoo.devel/92581

I'm not attempting to tackle that now, but as a step in the right
direction I suggest that eclasses not try to call eapply_user in
general, and then we don't have to worry about the situation where you
want to use three eclasses which all try to call it.

I think the long-term solution is to stop exporting phase functions in
eclasses.  I'd recommend stripping the ability to do so at all from
PMS, except for the whole KDE exception which makes sense to me.

-- 
Rich

Re: [gentoo-dev] stabilization commits and atomicity

[Rich Freeman]

On Mon, Oct 19, 2015 at 1:13 PM, hasufell  wrote:
>
> We already know that. But if e.g. ago runs his scripts at 00:00 with
> ~300 packages stabilized, the history (without git command line) on
> github/gitweb will be fun to read (and people DO that).
>

It doesn't seem like it would have been any better in the cvs days,
but I guess that isn't a reason to reject this on its own.

If this was about changing the copyright headers in all the ebuilds in
the tree I'd say that this is a million related trivial changes that
can be merged.  Nobody needs to see that in the history broken out.

However, stabilizing a single package really is an impactful change.
The fact that you're doing 100 of them at one time doesn't really
diminish the impact of each one.  Any of them could break a system or
need to be reverted.

If they're being done at once because they're all part of some library
stabilization then I'd combine them into a commit, because they are
actually related.

Maybe what is needed is better tools for tagging/filtering history?

-- 
Rich

Re: [gentoo-dev] Re: [gentoo-dev-announce] EAPI 6 draft for review

[Rich Freeman]

On Mon, Oct 19, 2015 at 10:21 AM, Alexis Ballier <aball...@gentoo.org> wrote:
> On Mon, 19 Oct 2015 09:51:20 -0400
> Rich Freeman <ri...@gentoo.org> wrote:
> [...]
>> >
>> >> I'd say the best approach for compatibility if you have an existing
>> >> eclass and it already exports src_prepare is to not call
>> >> eapply_user unless it firmly falls into the #2 category above.
>> >
>> > Replace 'not call eapply_user' by 'not export src_prepare' and I'd
>> > agree with you if going a bit further by ensuring there is no hidden
>> > problem:
>>
>> Well, taken together my recommendation does amount to:
>> 1.  Avoid exporting src_prepare at all.
>> 2.  If you do export src_prepare, then don't call eapply_user.
>
> 2. sucks: an ebuild inheriting that eclass will have to redefine
> src_prepare in order not to break with eapi6, so  there's no point in
> exporting the function in the first place.

No argument.  I'm just saying that nothing stops us from using an
existing eclass with EAPI6 without changing function names all over
the tree.  Non-EAPI6 ebuilds can still use the existing function
automatically, and new ebuilds using EAPI6 have to work around the
issue until the eclass is revisioned.

>
> Also, since you seem to know well KDE: where would you call
> eapply_user, in kde eclasses or cmake-utils ?
>

Definitely in a kde eclass.  cmake-utils would fall into the category
of a utility eclass, so it ideally shouldn't export phase functions at
all.  Again, I'm not proposing forcing a change on that now, and it
needs more thought, but it would be a lousy place to put eapply_user
since it could be used in the same ebuild as another eclass that
performs a similar function.

That might require having the kde eclass call the cmake-utils eclass
function.  Since the kde eclass is only intended to be used by kde
ebuilds maintained by the same group that maintains the eclass, there
isn't a problem here.  The ebuilds themselves just set a bunch of
variables and leave the work to the eclass.

-- 
Rich

Re: [gentoo-dev] stabilization commits and atomicity

[Rich Freeman]

On Mon, Oct 19, 2015 at 1:40 PM, hasufell <hasuf...@gentoo.org> wrote:
> On 10/19/2015 07:37 PM, Rich Freeman wrote:
>>
>> However, stabilizing a single package really is an impactful change.
>> The fact that you're doing 100 of them at one time doesn't really
>> diminish the impact of each one.  Any of them could break a system or
>> need to be reverted.
>>
>
> Since when do we allow reverting stabilization? The package needs to be
> fixed and possibly revbumped instead.
>

It would really depend on the nature of the break.  If it is a serious
upstream problem and no fix is available, then reverting might be the
only practical solution.  It is of course not a preferred solution.

-- 
Rich

Re: [gentoo-dev] Re: [gentoo-dev-announce] EAPI 6 draft for review

[Rich Freeman]

On Mon, Oct 19, 2015 at 2:28 PM, Alexis Ballier  wrote:
>
> However, as you say, putting it in cmake-utils needs to be properly
> thought so that it doesn't conflict with other eclasses: Hence the need
> to properly define what eclasses should call eapply_user and apply
> patches and what should not.
>

That is my main concern.  Maybe a package uses cmake and ant.  Patches
could affect either.  With an automagic design you might have to run
half of each src_configure, then apply patches, then run the other
half of each src_configure.

> Your initial argument was very convincing, but under those conditions,
> it seems way much saner to make eapply_user idempotent and be done.
> (and maybe in addition require informally that eapply_user is called
> after applying patches, but that'd fall under good practices rather
> than rules)

The only danger here is that later phase functions to run would be
operating on already-patched sources, when they expect to be running
on unpatched sources.  I imagine that usually wouldn't be a problem,
but it of course could be one.

I do get your analogy to eapply_user not being in the default phase function.

This all falls into that general category of correctness vs
convenience.  It is more convenient if you can just call eapply_user
at a suboptimal point and not have to mess with your ebuilds.  The
same argument could be made for running all the inherited eclass phase
functions and not just one of them.  The issue is that this can break
in lots of hard-to-predict ways.  I'd rather see things refactored to
deal with this in a more sane manner.

That actually makes me wonder if the better solution is to add another
phase - such as src_postprepare or something like that, where you'd
run autotools or whatever.  If that were done then you could also make
hasufell happy by just having the PM do the patching after
src_prepare.  Maybe that can go in EAPI7.  :)

-- 
Rich

Re: [gentoo-dev] stabilization commits and atomicity

[Rich Freeman]

On Mon, Oct 19, 2015 at 1:55 PM, hasufell <hasuf...@gentoo.org> wrote:
> On 10/19/2015 07:52 PM, Rich Freeman wrote:
>> On Mon, Oct 19, 2015 at 1:40 PM, hasufell <hasuf...@gentoo.org> wrote:
>>> On 10/19/2015 07:37 PM, Rich Freeman wrote:
>>>>
>>>> However, stabilizing a single package really is an impactful change.
>>>> The fact that you're doing 100 of them at one time doesn't really
>>>> diminish the impact of each one.  Any of them could break a system or
>>>> need to be reverted.
>>>>
>>>
>>> Since when do we allow reverting stabilization? The package needs to be
>>> fixed and possibly revbumped instead.
>>>
>>
>> It would really depend on the nature of the break.  If it is a serious
>> upstream problem and no fix is available, then reverting might be the
>> only practical solution.  It is of course not a preferred solution.
>>
>
> I don't think we depend on 'git revert' in that case. KEYWORDS are
> trivial changes (in terms of file diffs).
>

True, as long as they're truly standalone.

Maybe the compromise is:
1.  Groups of related stabilizations should be grouped.
2.  Groups of only unrelated stabilizations can also be grouped.
3.  You must not try to mix #1 and #2 in the same commit.

As you say individual packages are easy to revert anyway.  However, we
wouldn't want 100 of those to be mixed in with 50 packages that all
needed to be coordinated, because those 50 aren't easy to revert.

-- 
Rich

Re: [gentoo-dev] stabilization commits and atomicity

[Rich Freeman]

On Mon, Oct 19, 2015 at 11:27 AM, Ian Stakenvicius  wrote:
>
> Ahh, so what you're referring to here is stabilization of multiple
> unrelated packages in a single commit..  ok..  i'm not so
> comfortable with that idea..

Nor am I.  A commit should be a set of related changes.  Stabilizing
all of KDE-n in one commit makes a lot of sense.  Stabilizing 5 random
packages in one commit doesn't make sense.  By all means push them all
at once, but don't commit them all at once.  It isn't like we have to
pay for each commit.

I also don't have a problem with fixing multiple bugs in one commit.
In an ideal world you'd do that on a branch and then do a merge, and I
don't have a problem with that either, but I get that we tend to not
work that way.  If you want every commit to stand on its own and
you're porting to a new EAPI and fixing 3 bugs at the same time, I
don't expect maintainers to rewrite their bugs into the new code to
port it to the new EAPI, then fix each bug in turn.

> BUT, nothing stopped us from doing this
> with CVS (although the mapping of commit between CVS and GIT aren't
> exactly 1:1), so i guess it's fine..?

In cvs all commits were at the file level, even if you happened to
create more than one as a single operation.  If you did one commit
that touched 100 ebuilds, you were actually doing 100 commits, and
there is nothing that really ties those 100 commits together and by
the time it gets to rsync you might only get 50 of them if the timing
is right.

So, this actually is a new problem, or rather benefit.

>
> What about simply keeping things as they are but not strictly
> enforcing them when they are used in a manner like this for special
> cases, such as ago's stabilizations or other security@ or arch team
> keyword-related commits?
>

I don't think we're strictly enforcing anything now but I could be
wrong.  I think we should have guidelines that recommend best
practices and try to stick to them.  If there is a really good reason
to do things differently, that is why we call them "guidelines."

-- 
Rich

Re: [gentoo-dev] Re: [gentoo-dev-announce] EAPI 6 draft for review

[Rich Freeman]

On Sun, Oct 18, 2015 at 8:44 AM, Ciaran McCreesh
 wrote:
>
> But the big gain for everyone is in replacing a weird, overly clever
> and highly fragile collection of weirdness that's designed to mostly
> accept any dodgy input, with one that just gets you to give it a sane
> input to begin with.
>

See also:
http://cr.yp.to/qmail/guarantee.html

> 5. Don't parse.
>
> I have discovered that there are two types of command interfaces in
> the world of computing: good interfaces and user interfaces.
>
> The essence of user interfaces is parsing: converting an unstructured
> sequence of commands, in a format usually determined more by
> psychology than by solid engineering, into structured data.
>
> When another programmer wants to talk to a user interface, he has
> to quote: convert his structured data into an unstructured sequence
> of commands that the parser will, he hopes, convert back into the
> original structured data.
>
> This situation is a recipe for disaster. The parser often has bugs: it
> fails to handle some inputs according to the documented interface. The
> quoter often has bugs: it produces outputs that do not have the right
> meaning. Only on rare joyous occasions does it happen that the parser
> and the quoter both misinterpret the interface in the same way.

When you have programs talking to programs it makes far more sense to
just spell things out correctly vs having the receiving program try to
guess what the calling program meant.

-- 
Rich

Re: [gentoo-dev] Re: [gentoo-dev-announce] EAPI 6 draft for review

[Rich Freeman]

On Sun, Oct 18, 2015 at 8:05 AM, hasufell  wrote:
>
> If you are messing with the build system in a patch, there is no
> guarantee that eautoreconf will be enough. It might or might not be true
> (see net-irc/hexchat for an example). Are we going to run eautoreconf
> unconditionally then (which is exceptionally bad)?

Nope.  It isn't perfect, and I'm fine with that.

> Maybe the ebuild
> author doesn't even provide a live ebuild and there's no example for
> doing the right thing wrt random build systems patches. How about other
> build systems? You simply cannot do this properly.

I think you can do it properly.  I don't think you can do it in a
manner that is guaranteed to never fail.  IMO there is a difference.

>
> I think we are encouraging bad practice with this feature by making it
> part of PMS, causing users to file bugs because their random patches
> don't work with someones ebuilds.

Such bugs can be closed.  The intent is to provide a useful feature to
users, not to support the resulting binaries.  This isn't magic - it
is a tool for those who know how to use it, much like the rest of
Gentoo.  Forking ebuilds is far more hassle, even if it is more
powerful.

> If people need to hack on ebuilds, there are already numerous ways to do
> that, including doing it properly. Why add another one that is still not
> consistently thought through?

I'm not sure what makes this "not consistently thought through."  The
fact is that every objection you're raising was raised the first time
this came up.  I at least was well aware of all of them when I voted
to add this to EAPI6.  The issue seemingly isn't that it wasn't
thought through, but rather that those who did think through it didn't
agree with you on the decision.  That's ok - we'll never agree on
everything.

Believe it or not it is possible for two intelligent people to
thoughtfully consider the same data and come to different decisions.
This isn't deductive logic, and it isn't science until something has
been tried.

-- 
Rich

Re: [gentoo-dev] Re: [gentoo-dev-announce] EAPI 6 draft for review

[Rich Freeman]

On Sun, Oct 18, 2015 at 6:17 AM, Ulrich Mueller  wrote:
>> On Sun, 18 Oct 2015, Michał Górny wrote:
>
>> On Sun, 18 Oct 2015 11:54:40 +0200
>> Ulrich Mueller  wrote:
>
>>> So the question is if we should add a sentence like the following to
>>> the spec:
>>>
>>> In EAPIs where it is supported, all ebuilds must run
>>> \t{eapply\_user} in the \t{src\_prepare} phase.
>
>> How about:
>
>> In EAPIs listed in table blah blah blah, \t{eapply\_user} must
>> be called exactly once in the \t{src\_prepare} phase.
>
>> Which emphasizes that eclass or default may do it instead of ebuild.
>
> Yeah, that's better actually. We need not reference the table again
> though, since we do it in the sentence before.
>
> In EAPIs where it is supported, \t{eapply\_user} must be called
> exactly once in the \t{src\_prepare} phase.

++

-- 
Rich

Re: [gentoo-dev] Re: [gentoo-dev-announce] EAPI 6 draft for review

[Rich Freeman]

On Sun, Oct 18, 2015 at 7:37 AM, hasufell  wrote:
> On 10/17/2015 08:03 PM, Ciaran McCreesh wrote:
>> On Sat, 17 Oct 2015 14:49:36 +0200
>> hasufell  wrote:
>>> You can apply the patches post_unpack or post_src_prepare witht hooks.
>>> What's the problem?
>>
>> Running autorecrap.
>>
>
> You can do that with hooks too (which is not very clean tbh). But at
> that point, you probably want to actually fork the ebuild in an overlay
> if you need to mess with phase internals instead of relying on some
> magic that the ebuild writer might or might not have done properly.
>

This sounds like "if it isn't done perfectly it isn't worth doing."
If you're going to start by assuming that devs will always design
ebuilds incorrectly, then you might as well just fork all of Gentoo
into your overlay.  :)

People already find epatch_user useful, and I think moving it to PMS
will just make it more consistently useful.  Sure, there will be
mistakes, but right now epatch_user is optional, and in the future it
could be considered a QA issue.

-- 
Rich

Re: [gentoo-dev] Re: [gentoo-dev-announce] EAPI 6 draft for review

[Rich Freeman]

On Sat, Oct 17, 2015 at 8:25 AM, hasufell  wrote:
> On 10/17/2015 02:19 PM, Jason A. Donenfeld wrote:
>>
>> The other question is more critical -- could you merge eapply and
>> eapply_user? Or add some hook to PMS so that eapply_user isn't needed?
>> IOW, it'd be nice if every package was, by default, patchable by the user.
>>
>
> IMO, eapply_user should not be in the eclass and not in PMS. patches are
> something that can easily be done via PM hooks, if the PM has proper
> hooks support.
>

The reason this was done was to give maintainers more control over
WHEN patches are applied, while still ensuring they are applyied.

The other feature that is supposed to be in EAPI6 (I didn't read the
draft yet) is that the PM should refuse to install the package if
eapply is never called (ie src_prepare is overridden and the ebuild
didn't call eapply).  It is required that all ebuilds call it once
unconditionally.  That way users don't get inconsistent behavior from
package to package and be dependent on maintainers to fix it.

We'd have to dig through the archives, but I'm sure there was
extensive discussion about whether this belonged in the PM or PMS.

-- 
Rich

Re: [gentoo-dev] Re: [gentoo-dev-announce] EAPI 6 draft for review

[Rich Freeman]

On Sat, Oct 17, 2015 at 8:49 AM, hasufell  wrote:
>
>> The other feature that is supposed to be in EAPI6 (I didn't read the
>> draft yet) is that the PM should refuse to install the package if
>> eapply is never called (ie src_prepare is overridden and the ebuild
>> didn't call eapply).  It is required that all ebuilds call it once
>> unconditionally.  That way users don't get inconsistent behavior from
>> package to package and be dependent on maintainers to fix it.
>>
>
> I hope that "feature" doesn't make it into EAPI6.
>

The council already voted it in, but of course the final spec requires
approval.  I don't intend to approve it without it, unless somebody
makes a REALLY good case for it.

Why wouldn't you want this, anyway?  You're advocating for having the
PM do it 100% of the time, and simultaneously arguing that if it is
done via a call in the ebuild it shouldn't be 100% consistent.  Those
positions do not seem consistent, unless you just want EAPI6 to be
broken so that you can argue for EAPI7 or whatever.

-- 
Rich

Re: [gentoo-dev] Re: [gentoo-dev-announce] EAPI 6 draft for review

[Rich Freeman]

On Sat, Oct 17, 2015 at 8:51 AM, Michał Górny <mgo...@gentoo.org> wrote:
> Dnia 2015-10-17, o godz. 08:38:51
> Rich Freeman <ri...@gentoo.org> napisał(a):
>
>> On Sat, Oct 17, 2015 at 8:25 AM, hasufell <hasuf...@gentoo.org> wrote:
>> > On 10/17/2015 02:19 PM, Jason A. Donenfeld wrote:
>> >>
>> >> The other question is more critical -- could you merge eapply and
>> >> eapply_user? Or add some hook to PMS so that eapply_user isn't needed?
>> >> IOW, it'd be nice if every package was, by default, patchable by the user.
>> >>
>> >
>> > IMO, eapply_user should not be in the eclass and not in PMS. patches are
>> > something that can easily be done via PM hooks, if the PM has proper
>> > hooks support.
>> >
>>
>> The reason this was done was to give maintainers more control over
>> WHEN patches are applied, while still ensuring they are applyied.
>>
>> The other feature that is supposed to be in EAPI6 (I didn't read the
>> draft yet) is that the PM should refuse to install the package if
>> eapply is never called (ie src_prepare is overridden and the ebuild
>> didn't call eapply).  It is required that all ebuilds call it once
>> unconditionally.  That way users don't get inconsistent behavior from
>> package to package and be dependent on maintainers to fix it.
>>
>> We'd have to dig through the archives, but I'm sure there was
>> extensive discussion about whether this belonged in the PM or PMS.
>
> I don't think this was really accepted. I think the best we can do is
> make repoman complain about it.

Well, the vote was:

User patches
The council endorses an eapply_user function in the PM to apply user
patches in EAPI6.  This will be called by the default src_prepare, and
must be called once if src_prepare is overrided by either a non-virtual
ebuild or eclass.
Aye: blueness, creffett (proxy for williamh), dilfridge, rich0, scarabeus, ulm
Nay: dberkholz
Passed
https://projects.gentoo.org/council/meeting-logs/20140617-summary.txt

We were voting on what was going into PMS, not what was going into
repoman, though there is no harm in repoman checking for things that
will make package managers fail.

I don't see the problem with having the PM enforce this.  It is
providing the function to apply the patches, so surely it can keep
track of whether it was called or not and just die before executing
src_configure.

My main concern with just doing it in repoman is that we already
routinely find packages in the tree that fail repoman tests.  If the
PMs check it then the packages won't even install, which makes this
pretty hard to miss during testing, and it makes anything that does
end up in the tree a candidate for treecleaning.  I guess my question
is what is the problem with having the PM perform this check?  And I'd
rather make that a part of PMS rather than having some PMs do the
check and others not do it, and then we get to have WORKSFORME debates
on bugs when the maintainer's preferred PM is lax on the rules (which
is something a few on this thread should be able to sympathize with).

Another general comment I have on this thread is that the scope of
this really ought to be finding areas where the PMS doesn't reflect
what we already approved going in, or major show-stoppers that simply
weren't brought up before.  Almost everything I'm seeing in this
thread are issues that were raised before the first time the council
approved the contents of PMS.  I don't really see much point in just
re-iterating the same arguments.

The whole point of pre-approving the contents of PMS at a high level
was to allow those doing implementation work to avoid wasting time
developing features only to watch the council throw their work away at
the last minute.  If we make changes now that is basically what it
amounts to.  If there really is some show-stopper that nobody
discovered before please speak up, but it isn't really productive to
re-hash the same arguments over and over.  This should be about
perfecting the specification, not about what is in and out.

-- 
Rich

Re: [gentoo-dev] Re: [gentoo-dev-announce] EAPI 6 draft for review

[Rich Freeman]

On Sat, Oct 17, 2015 at 8:52 AM, Ulrich Mueller  wrote:
>
> That eapply_user is called can be enforced by repoman, or by a QA
> warning.
>

I hate to reply again on the same topic, but how would repoman even
know whether eapply_user will always get called?  Isn't that
equivalent to the halting problem?

That would be another reason to have the PM do the check.  All it has
to do is set an internal flag when it is called, and then check the
flag before starting the next phase.  Then you can have as many levels
of conditionals and nested functions in eclasses as you want, and the
check still works.

-- 
Rich

Re: [gentoo-dev] ironing out release tarballs

[Rich Freeman]

On Fri, Oct 16, 2015 at 2:45 AM, Alexis Ballier  wrote:
>
> if it's just used by catalyst to pre-seed world then indeed pms
> doesn't have anything to do with it, but if it's meant to be some set
> that profiles add to 'world' set dynamically, then interoperability is
> probably desired
>

I'd suggest that we probably don't want anything dynamic here.

I think most of our users would appreciate a friendly "here, I went
ahead and pre-loaded screen for you but feel free to drop it" on the
first install.  They could review that world file and see what
is/isn't pre-loaded and just delete the whole file if they want a
blank slate.

On the other hand, three years into using their system they probably
wouldn't appreciate if portage just went and installed screen for
them, because it was trying to be helpful and we decided that new
installs should now include screen.

-- 
Rich

Re: [gentoo-dev] [rfc] enable USE=xattr by default

[Rich Freeman]

On Thu, Oct 15, 2015 at 7:58 AM, Alexander Tsoy  wrote:
>
> I was wrong. This patch was not merged upstream. It is still needed and
> included in latest genpatches for 4.2:
>
> $ tar tf genpatches-4.2-6.base.tar.xz | grep XATTR
> ./1500_XATTR_USER_PREFIX.patch

I suspect what we all have in common then is that we're using tmpfs to
do builds and we're not using genpatches.

If the warning isn't an issue for non-hardened users then I don't see
any need to change anything.  Is the patch (or something similar)
likely to get merged?  It doesn't really seem ideal to be dependent on
something not in mainline.

-- 
Rich

Re: [gentoo-dev] [rfc] drop iputils from @system (i.e. ping)

[Rich Freeman]

On Wed, Oct 14, 2015 at 11:39 PM, Mike Frysinger  wrote:
> iputils is currently in @system for everyone.  by default, it only
> installs `ping`.  do we feel strongly enough about this to require
> all systems include it ?  or should this wait for the long idea of
> releasing stage4's instead of stage3's ?

IMO this should certainly be installed by default, but it should also
not be a part of the @system set unless portage depends on it or
something like that.

Thus this should probably wait until we start releasing stage4s.

The fact that everybody wants a package to be installed shouldn't be a
reason to put it in the @system set.  It should instead be a reason to
have some tool other than the @system set to give everybody what they
want.  We need one set for the stuff that everybody agrees is
indispensable (@tbd), and a different set for the minimum set of
software (@system) you need to bootstrap the rest of the packages in
the repo.

-- 
Rich

Re: [gentoo-dev] [rfc] enable USE=xattr by default

[Rich Freeman]

On Thu, Oct 15, 2015 at 7:22 AM, Tobias Klausmann  wrote:
>
> So it's not a BTRFS problem, but one of tmpfs. So I wondered if I
> maybe had missed to activate xattr suport for tmpfs, but no:
>
> # zgrep -i tmpfs /proc/config.gz
> CONFIG_DEVTMPFS=y
> CONFIG_DEVTMPFS_MOUNT=y
> CONFIG_TMPFS=y
> CONFIG_TMPFS_POSIX_ACL=y
> CONFIG_TMPFS_XATTR=y
> #

Same here (but I don't enable DEVTMPFS_MOUNT).  I had also wondered if
this was btrfs-related but it might indeed be tmpfs related.

-- 
Rich

Re: [gentoo-dev] [rfc] enable USE=xattr by default

[Rich Freeman]

On Thu, Oct 15, 2015 at 6:56 AM, Jason Zaman  wrote:
>
> Can you try this:
>
> # getfattr -d -m- /bin/ping
> security.capability=0sAQAAAgAgAAA=
> # setfattr -n user.test -v "foo" ./ping
> # setfattr -n user.pax.flags -v "me" ./ping
> # getfattr -d -m- /bin/ping
> security.capability=0sAQAAAgAgAAA=
> user.pax.flags="me"
> user.test="foo"
>
> If this works then something else is causing those messages and we
> should look into it further.

This behaves exactly as described above for me on btrfs, but I still
do get all the error messages whenever I install stuff.

I assume the extra attributes are harmless and will get removed the
next time I update ping?

-- 
Rich

Re: [gentoo-dev] ironing out release tarballs

[Rich Freeman]

On Thu, Oct 15, 2015 at 4:10 PM, Zac Medico  wrote:
> What's probably desired is to create a stage3 profile which adds
> whatever extra stuff you want to @system, and to use the stage3 profile
> for to build stage3. After the stage3 is built, catalyst could set some
> other profile if we don't want users to have the stage3 profile by default.

Unless we seed @selected with the extra stuff, it will get removed the
first time the user runs --depclean, which probably isn't what we
want.

I'm fine with going the multi-profile route, but it seems to me that
we still need some kind of initial @selected set.

Basically the goal here is to give the user a bunch of useful stuff by
default (specified in some way in the profile), but make it easy for
the user to remove it if they wish.

A side goal is to do this without adding 14 more profiles, because we
don't have mixins.

-- 
Rich

Re: [gentoo-dev] ironing out release tarballs

[Rich Freeman]

On Thu, Oct 15, 2015 at 5:15 PM, Zac Medico <zmed...@gentoo.org> wrote:
> On 10/15/2015 02:03 PM, Rich Freeman wrote:
>> On Thu, Oct 15, 2015 at 4:57 PM, Zac Medico <zmed...@gentoo.org> wrote:
>>> Given the goals, having catalyst seed /var/lib/portage/world seems
>>> pretty reasonable to me.
>>
>> Then the question becomes how.  Does it diff @profile between the two
>> profiles and put the extra stuff in @selected?  Or, does the profile
>> just contain a special file containing the stuff that gets seeded?
>> That is really the gist of the two approaches, and if you just have a
>> special file full of stuff that gets seeded you really don't need
>> another profile, which is nice since profiles are a PITA right now.
>>
>
> We already have packages.build which is used to build stage1, so
> introducing a packages.stage3 that's used to seed @selected (aka
> /var/lib/portage/world) for stage3 seems reasonable.

Seems reasonable to me, and the nice thing is that it doesn't change
the behavior of anything at all besides catalyst, so other than
starting out with a non-empty /var/lib/portage/world users won't see a
thing happen.

I won't bikeshed on the name of the file.  Whatever seems reasonable
to the catalyst team works fine for me.

This then conserves @profile for stuff that is more essential to the
profile itself, such as a fancy firmware loader for an arm box or
whatever (in our future luxury world where we can spare new profiles
for specific boards and such).  Of course, it wouldn't hurt to
standardize on how such sets work if we're going to start using them
seriously if that isn't in PMS.  However, I see all of that as
off-topic for the present discussion other than the desire to not
interfere with it.

-- 
Rich

Re: [gentoo-dev] ironing out release tarballs

[Rich Freeman]

On Thu, Oct 15, 2015 at 4:57 PM, Zac Medico  wrote:
> Given the goals, having catalyst seed /var/lib/portage/world seems
> pretty reasonable to me.

Then the question becomes how.  Does it diff @profile between the two
profiles and put the extra stuff in @selected?  Or, does the profile
just contain a special file containing the stuff that gets seeded?
That is really the gist of the two approaches, and if you just have a
special file full of stuff that gets seeded you really don't need
another profile, which is nice since profiles are a PITA right now.

-- 
Rich

Re: [gentoo-dev] ironing out release tarballs

[Rich Freeman]

On Thu, Oct 15, 2015 at 6:00 PM, Zac Medico  wrote:
> On 10/15/2015 02:51 PM, Anthony G. Basile wrote:
>
>> The only change in moving it to @profile is the warning.
>
> What's the point of getting rid of the warning if the package is going
> to get pulled back in on the next @world update? Either way, the end
> result it that the user has to go through some hoops
> (/etc/portage/profile overrides) if they don't want that package
> installed again.

++

I think just pre-seeding the world file is a simpler solution.  Users
can just uninstall the files normally then.


-- 
Rich

Re: [gentoo-dev] ironing out release tarballs

[Rich Freeman]

On Thu, Oct 15, 2015 at 11:34 AM, Mike Frysinger  wrote:
>
> items to sort out:
> - should the list of packages be in catalyst or profile-stacked content
> -> imo it should be entirely in the profile

++

This would be really nice to combine with mix-ins so that instead of
special cases we could just use additional profiles (without the
normal cost of additional profiles), but absent that the approach you
have below seems fine.

>
> - should the packages list be in a new packages.default, or should we create a
>   new set to hold it, or should we just go with @profile ?
> -> @profile has the advantage of already existing.  we have to be careful so 
> as
>to make it difficult to uninstall packages that the user does not actually
>want.

I would be interested in use cases for @profile OTHER than convenience
packages (sshd, screen, etc).

Again, this is a case where having more profiles would get rid of the
need to have a compromise.  Just make it @profile, and be sure to have
a profile available that doesn't have any packages beyond @system.

However, if some profiles really do need to install fairly critical
packages then maybe we should also have a packages.default in addition
to @profile.

>
> - if the packages aren't in @profile, should they be seeded in @world ?
> -> imo yes as  we don't want all the default packages getting depcleaned as
>soon as you start using the new install.  if they're in @profile, then this
>is a moot point (assuming depclean does not clean out @profile).

If we end up with @system+@profile+packages.default then I'd say that
packages.default gets seeded in @world.  @profile becomes difficult to
uninstall.  This should be the solution if some profiles really do
need to add essential packages as well as convenience packages, but
the essential packages aren't assumed dependencies.

If we end up with just @system+@profile then I'd say that packages in
@profile get seeded in @world, and thus nothing special needs to be
done to uninstall them.  This should be done if there aren't essential
profile packages.

>
> - should stage3 be @system only, or @system+@profile, or
>   @system+@profile+packages.default ?
> -> this depends on the previous discussion a bit.  today, stage3's are
>@system, but imo @system+@profile is reasonable.  see next question too.

Agree it depends on the previous outcome.

>
> - should we release stage4's instead of stage3's ?
> -> if we keep stage3 as @system-only, then we'd build stage4's which would add
>@profile/whatever
> -> downside is that we've been training the world to download & install stage3
>for almost 15 years
> -> imo as long as the default @profile is kept slim, adjusting the definition 
> of
>a stage3 is OK

I don't have a strong opinion on this.  I don't see the need to
maintain separate stage3s in addition to the stage4s, so we're just
arguing semantics.

I think the real question I have is what would the @profile set be
used for OTHER than convenience packages?  While I did mention mix-ins
as being a better long-term solution I'm not suggesting that we should
hold off on a reasonable interim solution until we work that out.
Without mix-in support we don't really want to add more profiles,
which is the other way to go with this.

-- 
Rich

Re: [gentoo-portage-dev] Tools-Portage Lead

[Rich Freeman]

On Tue, Oct 13, 2015 at 1:47 PM, Paul Varner  wrote:
> Reading GLEP-39, it is not clear to me if a sub-project needs to have
> their leads elected or not.

No interest in interfering with whatever you guys come up with, but as
far as I'm aware subprojects are just projects as far as GLEP 39 goes
- the hierarchy is intended to be more for organization than anything.
So, you guys are welcome to and should of course consider selecting a
lead.

The motivation of the request was less about trying to dictate how
projects run themselves and more about having somebody people can
reach out to with questions/etc, and also to have a way to get a sense
for how alive projects are.

-- 
Rich

Re: [gentoo-dev] Re: [gentoo-commits] repo/gentoo:master commit in: sys-apps/portage/

[Rich Freeman]

On Sun, Oct 11, 2015 at 11:35 PM, Duncan <1i5t5.dun...@cox.net> wrote:
>
> It is, however, worth repeating that in git, commits are entirely
> separate from pushes and are very (as in, extremely!) cheap, while
> pushes, particularly if properly repoman-checked, are obviously much more
> expensive.

Each commit really should be able to stand on its own all the same.
They should all be able to pass a repoman check.

Otherwise when you do try to use tools like bisect you can get
breakage.  It is really frustrating when you're trying to track down a
change that causes an issue and find a large stretch of commits that
won't even build.

-- 
Rich

Re: [gentoo-dev] [RFC/announcement] Reviewers project

[Rich Freeman]

On Mon, Oct 12, 2015 at 9:29 AM, Ian Delaney  wrote:
>
> Not sure how to read this. The whole idea is for provider / client to
> communicate and negotiate a workable solution. At a glance this reads
> as the user needs to adapt to the service that the client is offering
> and appease the provider. What's wrong with this picture?

While I agree that this had a bit of a rough start, I don't think it
is realistic for any Gentoo project to tailor how it communicates to
each individual.  By all means find the 80% solution that works for
most, but if having bugs being opened seems to be the best solution,
we can't really have individual developers saying "don't open bugs for
me - just ping me on IRC/email/phone/the-bar-at-the-next-FOSDEM/etc."

I suggest we focus more on finding that common solution.

FWIW I don't see any issue with this stuff being public.  It shouldn't
be personal, and we should be making feedback as helpful as reasonably
possible.  That could be as simple as an email signature that says
"the above feedback is intended to be concise and is targeted at an
experienced Gentoo developer, if you have any questions about how to
handle it please do ABC to get help."  Just having an invitation for
support would probably go a long way, and we could have a separate set
of volunteers (perhaps overlapping) who volunteer to provide this
help.

To the extent that anything is said which shouldn't be said in public,
we probably shouldn't be saying it in private either, at least not in
the context of this project.

My concern with the -dev list was more that it ends up being a lot of
noise for most on the list, not that it is public.  That's why I
suggested a top-5 list or something like that, which would have weeded
out false positives and focus more on resolutions and trends than
individual incidents.

-- 
Rich

Re: [gentoo-dev] Re: [gentoo-commits] repo/gentoo:master commit in: sys-apps/portage/

[Rich Freeman]

On Mon, Oct 12, 2015 at 1:44 PM, Alec Warner <anta...@gentoo.org> wrote:
>
>
> On Mon, Oct 12, 2015 at 3:58 AM, Rich Freeman <ri...@gentoo.org> wrote:
>>
>> On Sun, Oct 11, 2015 at 11:35 PM, Duncan <1i5t5.dun...@cox.net> wrote:
>> >
>> > It is, however, worth repeating that in git, commits are entirely
>> > separate from pushes and are very (as in, extremely!) cheap, while
>> > pushes, particularly if properly repoman-checked, are obviously much
>> > more
>> > expensive.
>>
>> Each commit really should be able to stand on its own all the same.
>>
>> They should all be able to pass a repoman check.
>
>
> I guess in some ideal world I sort of agree; but in practice I think
> workflows exist that involve people messing around in their local repo until
> they get stuff right, then running repoman at the end and pushing the result
> (that is kind of the whole point of having a local staging repo.)

I think the best practice here really depends on the nature of what
you're working on.  For most software the right way to handle such
things is to either:
1.  If you're just doing multiple iterations on one true logical
change, rebase your commits into a single commit once you have it
right.  You could extend this into a few commits in some cases as long
as they can stand on their own.
2.  Do all the work on a side branch and do a merge commit into the
master branch.  Then master still has a history where every commit
(along one side) works.  This is the best approach for the development
of major features which necessarily will not be complete for extended
periods but where the details are still useful.

#1 is likely to be most appropriate for the Gentoo repo in most cases.

-- 
Rich

Re: [gentoo-dev] Re: Dynamic dependencies

[Rich Freeman]

On Fri, Oct 2, 2015 at 5:46 AM, Rich Freeman <ri...@gentoo.org> wrote:
> On Fri, Oct 2, 2015 at 2:22 AM, Martin Vaeth <mar...@mvath.de> wrote:
>> Rich Freeman <ri...@gentoo.org> wrote:
>>>
>>> Proposal 3a might be: Anytime an RDEPEND in an eclass is changed, the
>>> eclass must be revisioned unless all ebuilds in the gentoo repository
>>> will continue to work correctly with the old RDEPEND.
>>> Proposal 4a might be: Anytime an RDEPEND in an eclass is changed, all
>>> ebuilds that inherit the eclass in the gentoo repository must be
>>> revisioned if they will not continue to work correctly with the old
>>> RDEPEND.
>>
>> Adding an || alternative should be included here:
>> The installed package would continue to work without that alternative,
>> but without a revbump the user is not able to see that he might
>> possibly drop a package.
>>
>
> Perhaps add "or if the new RDEPEND allows the ebuild to work with
> additional dependencies."  Or maybe just straight out say "or if
> additional || atoms are added."  The first wording might allow for
> additional cases, which is probably good.
>

So, here is a consolidated list of the latest proposals:

RDEPEND changes directly in ebuilds (non-virtual and virtual)
Proposal 5: Anytime an RDEPEND in an ebuild is changed, the
ebuild must be revisioned.  This includes adding/removing inherited
eclasses which set RDEPENDs.


RDEPEND changes in eclasses
Proposal 3b: Anytime an RDEPEND in an eclass is changed, the
eclass must be revisioned unless:
1. all ebuilds in the gentoo repository will continue to work
correctly with the old RDEPEND,
2. and the new RDEPEND is a subset of the old RDEPEND


Proposal 4b: Anytime an RDEPEND in an eclass is changed, all
ebuilds that inherit the eclass in the gentoo repository must be
revisioned unless:
1. all ebuilds in the gentoo repository will continue to work
correctly with the old RDEPEND,
2. and the new RDEPEND is a subset of the old RDEPEND.


>From the tone of discussion the wording of the eclass proposals still
might be a bit conservative, and there may be other cases where we
could avoid bumps.  However, I think this does cover the examples that
actually came up.


Here are ones that i consider outdated:
RDEPEND changes directly in ebuilds
Proposal 1: Anytime an RDEPEND in a non-virtual ebuild is changed, the
ebuild must be revisioned.  This includes adding/removing inherited
eclasses which set RDEPENDs.

RDEPEND changes directly in virtuals
Proposal 2: Anytime an RDEPEND in a virtual ebuild is changed, the
ebuild must be revisioned.  This includes adding/removing inherited
eclasses which set RDEPENDs.

RDEPEND changes in eclasses
Proposal 3: Anytime an RDEPEND in an eclass is changed, the eclass
must be revisioned.
Proposal 4: Anytime an RDEPEND in an eclass is changed, all ebuilds
that inherit the eclass in the gentoo repository must be revisioned.
Proposal 3a: Anytime an RDEPEND in an eclass is changed, the
eclass must be revisioned unless all ebuilds in the gentoo repository
will continue to work correctly with the old RDEPEND.
Proposal 4a: Anytime an RDEPEND in an eclass is changed, all
ebuilds that inherit the eclass in the gentoo repository must be
revisioned if they will not continue to work correctly with the old
RDEPEND.

-- 
Rich

Re: [gentoo-dev] Dynamic dependencies

[Rich Freeman]

On Thu, Sep 17, 2015 at 7:57 AM, Kristian Fiskerstrand  wrote:
>
> Another thing that strikes me is separation of stable vs ~arch behavior.
>
> This applies in particular with in-place eclass alterations. Users on
> ~arch should normally expect more activity (in particular number of
> builds and changes, that is after all its definition). Stable users on
> the other hand might want slower update cycle for non-security upgrades.
>
> Reading this thread I got hit with a question whether this boundary is
> properly protected if doing in-place eclass changes?

This isn't about whether an eclass used by stable packages are allowed
to be modified or not.

This is about what has to be done AFTER an eclass is modified, so that
users won't run into problems down the road.

Today developers are modifying eclass RDEPENDs and not bumping
anything.  This impacts stable users, but it can do so in ways that
cause their systems to break months down the road in ways that are
hard to troubleshoot.

With the proposal when an eclass is changed the user will get a
rebuild, but they won't get the mysterious breakage.

I'm not sure that we're doing stable users a favor by not "disturbing"
them with rebuilds but instead we're just silently breaking their
systems in subtle ways.  I think that if anybody would benefit from a
more conservative approach it would be the stable users.

I'm all for having a separate discussion around when it is and isn't
appropriate to modify eclasses that are used by stable packages in the
first place.

-- 
Rich

Re: [gentoo-dev] Re: Dynamic dependencies

[Rich Freeman]

On Sun, Oct 11, 2015 at 7:09 AM, Rich Freeman <ri...@gentoo.org> wrote:
> So, here is a consolidated list of the latest proposals:
>

It has been suggested that there might be rare cases where the
exceptions in the eclass proposals might apply to ebuilds, so doing
some refactoring I think the latest proposals are:

RDEPEND changes directly in ebuilds (non-virtual and virtual)
Proposal 5: Anytime an RDEPEND in an ebuild is changed, the
ebuild must be revisioned.  This includes adding/removing inherited
eclasses which set RDEPENDs.


RDEPEND changes in eclasses
Proposal 3: Anytime an RDEPEND in an eclass is changed, the eclass
must be revisioned.
Proposal 4: Anytime an RDEPEND in an eclass is changed, all ebuilds
that inherit the eclass in the gentoo repository must be revisioned.

General exception to the above:
Proposal 6: Maintainers may avoid revbumps at their discretion if:
1. all ebuilds in the gentoo repository will continue to work
correctly with the old RDEPEND,
2. and the new RDEPEND is a subset of the old RDEPEND

(Ironically splitting that out just reverts us back to the original
proposals, but it is also simpler.)

-- 
Rich

Re: [gentoo-dev] Re: Dynamic dependencies

[Rich Freeman]

Ok, in the Council meeting today the following was made policy:
"Maintainers must not assume that dynamic dependencies will be applied
by the package manager. When changing runtime dependencies the
maintainer should revision the ebuild if the changes are likely to
cause problems for end users."

The wording intentionally left some room for maintainer discretion,
but this should be used with care.  Eclass changes were really the
main area of concern, and it was suggested that it would be a good
practice to talk about whether bumping makes sense when discussing the
eclass change on -dev as already required by policy.

The proposals I sent out earlier will be adopted as guidelines.  They
can go in the devmanual once they're worked out here, and can be
maintained as with anything else in the devmanual.  Adding additional
edge cases/etc them doesn't require explicit council approval
(obviously large controversies can always be appealed).

Guidelines:

RDEPEND changes directly in ebuilds
Anytime an RDEPEND in an ebuild is changed, the ebuild should be
revisioned.  This includes adding/removing inherited eclasses which
set RDEPENDs.

RDEPEND changes in eclasses
Anytime an RDEPEND in an eclass is changed, all ebuilds that inherit
the eclass in the gentoo repository should be revisioned.
It is generally going to be easier to do that all at once, but it may
make sense in some cases to revision the eclass itself to allow
gradual adoption of the change.  Ebuilds would be revisioned as they
adopt the new eclass.

General exception to the above:
There are cases where a change in runtime dependencies will not cause problems.

Maintainers may safely avoid revbumps if:
1. all ebuilds in the gentoo repository will continue to work
correctly with the old RDEPEND,
2. and the new RDEPEND is a subset of the old RDEPEND

A common example of a situation where a bump is not needed is when
increasing the minimum version of a dependency in an eclass, when the
old version was working for all ebuilds that used the eclass at the
change (i.e. the change is being made for the sake of ebuilds that are
about to be introduced to the tree).

Please reply if you see any need to improve these.  I would encourage
developers to take these guidelines seriously.  The purpose of
allowing discretion is so that we're not locked into rigid rules that
don't capture every nuance.  If you change an RDEPEND and don't bump
it can cause subtle problems that can show up weeks or months later,
when devs make changes that repoman considers valid but which are
inconsistent with what is recorded on user's vdbs.

-- 
Rich

Re: [gentoo-dev] Re: [gentoo-commits] repo/gentoo:master commit in: dev-libs/libgit2/

[Rich Freeman]

On Sat, Oct 10, 2015 at 10:15 AM, hasufell  wrote:
>
> Nevertheless, we'll try to continue, reduce public noise and keep the
> reviews useful.
>

Bugs would probably be helpful from the standpoint that they also are
a mechanism to keep track of whether the issue was corrected.

I don't think it is a bad thing if you wanted to publish the top-5
issues of the week/month/etc on the lists or something useful for
general education/improvement.  The key is to keep the general
relevance of the lists high.

Personally, if somebody pointed out a mistake I made I'd probably be a
bit embarrassed to give them a hard time about it no matter what the
mechanism.  Let's just try to keep things reasonably objective.

-- 
Rich

Re: [gentoo-dev] unnecessary revbump

[Rich Freeman]

On Tue, Oct 6, 2015 at 8:27 PM, Zac Medico  wrote:
>
> Yeah, there can be a benefit, as long as you're not one of the people
> who uses --changed-deps for all updates (revbumps for dependency changes
> are basically irrelevant to these people).
>

Presumably those inclined to do this would be likely to stop if our QA
standards eliminated the need for it.

-- 
Rich

Re: [gentoo-dev] unnecessary revbump

[Rich Freeman]

On Tue, Oct 6, 2015 at 12:44 PM, Zac Medico  wrote:
> On 10/06/2015 06:33 AM, William Hubbs wrote:
>> I don't think the revbump of net-misc/openconnect-7.06-r1 to -r2 was
>> necessary. When the change purely affects use flags, that is picked up
>> by the pm and there is no need to force everyone to rebuild the package.
>
> The same goes for dependency changes if the package manager has an
> option like emerge --changed-deps. So, apparently the assumption is that
> all relevant package managers implement behavior like emerge --newuse
> and/or --changed-use, but they don't necessarily implement --changed-deps?

Are there any negative consequences if you don't rebuild a package
that has new use flags, as there are if you don't rebuild one with new
dependencies (in some circumstances)?  In situations where there are,
we should revbump.

The discussion around bumping on dep changes isn't necessarily to bump
them ANYTIME a dep changes, but only under some circumstances.  (In
the more general cases you'd bump most of the time, but there are a
bunch of cases where you wouldn't have to, and some of them would
otherwise result in bumping dozens of packages at once.)  So, there is
benefit to bumping even if every PM had an option like --changed-deps.

-- 
Rich

Re: [gentoo-dev] LibreSSL switch-over progress

[Rich Freeman]

On Mon, Oct 5, 2015 at 11:28 AM, Jason A. Donenfeld  wrote:
> I assume there are developers hard
> at work adding the flag to each and every package.

Keep in mind that it isn't always a drop-in replacement.  If it were
we'd just make a virtual for libssl and you wouldn't need to mess with
flags at all.

Some upstreams may support libressl quickly, some might support it
more slowly, and some may or may not ever support it.  So, I suspect
that this will look a lot like trying to switch over to libav - you
might have to change what applications you're using in some cases if
you really want to do it.  As with libav you may see one library or
the other "win" in the end which should make things simpler, but I
suspect that in the meantime there may be a lot of bundling/etc.

When changes require patches and upstream hasn't committed to merging
them, that creates a potential maintenance burden and if package
maintainers aren't willing to undertake this then we should probably
figure out how that is going to work, unless we just plan to ignore
these packages for now.

If it is just a matter of sticking a simple sed in an ebuild and the
libressl team doesn't mind dealing with rare breakage that is probably
less of an issue.

-- 
Rich

Re: [gentoo-dev] Re: Dynamic dependencies

[Rich Freeman]

On Fri, Oct 2, 2015 at 2:22 AM, Martin Vaeth <mar...@mvath.de> wrote:
> Rich Freeman <ri...@gentoo.org> wrote:
>>
>> Proposal 3a might be: Anytime an RDEPEND in an eclass is changed, the
>> eclass must be revisioned unless all ebuilds in the gentoo repository
>> will continue to work correctly with the old RDEPEND.
>> Proposal 4a might be: Anytime an RDEPEND in an eclass is changed, all
>> ebuilds that inherit the eclass in the gentoo repository must be
>> revisioned if they will not continue to work correctly with the old
>> RDEPEND.
>
> Adding an || alternative should be included here:
> The installed package would continue to work without that alternative,
> but without a revbump the user is not able to see that he might
> possibly drop a package.
>

Ugh, I agree completely but this isn't going to make the wording prettier.

Perhaps add "or if the new RDEPEND allows the ebuild to work with
additional dependencies."  Or maybe just straight out say "or if
additional || atoms are added."  The first wording might allow for
additional cases, which is probably good.

Otherwise a change is made today without a revbump and a year later
somebody removes some package from the tree and random users run into
problems with it.

-- 
Rich

Re: [gentoo-dev] Dynamic dependencies

[Rich Freeman]

On Wed, Sep 16, 2015 at 1:09 PM, Rich Freeman <ri...@gentoo.org> wrote:
>
> I'll go ahead and start a tangent on this thread right here.  As a
> first step can we separately consider the proposal to require a
> revbump anytime a package's RDEPENDS changes?  I'm referring here to
> directly-specified RDEPENDS, not those inherited from an eclass or
> virtual.

Ok, for the purpose of the upcoming council meeting, I'd like to make
a few separate proposals around dynamic dependencies.
There are three categories of policies:
1.  RDEPEND changes directly in ebuilds.
2.  RDEPEND changes directly in virtuals.
3.  RDEPEND changes in eclasses.

Honestly, I'm not really convinced virtuals need special treatment,
but I split them off in case it is useful in discussion.

RDEPEND changes directly in ebuilds
Proposal 1: Anytime an RDEPEND in a non-virtual ebuild is changed, the
ebuild must be revisioned.  This includes adding/removing inherited
eclasses which set RDEPENDs.

RDEPEND changes directly in virtuals
Proposal 2: Anytime an RDEPEND in a virtual ebuild is changed, the
ebuild must be revisioned.  This includes adding/removing inherited
eclasses which set RDEPENDs.

RDEPEND changes in eclasses
Proposal 3: Anytime an RDEPEND in an eclass is changed, the eclass
must be revisioned.
Proposal 4: Anytime an RDEPEND in an eclass is changed, all ebuilds
that inherit the eclass in the gentoo repository must be revisioned.

Please speak up if you have any issues with any of these.

I'm leaning towards adopting 1, 2, and 4.  I would actually prefer 3
to 4 on principle, but we don't have a good way of retiring obsolete
eclasses and I don't want to see a huge multiplication in them.  If we
did adopt #3 we should probably start thinking about more consistent
version numbering for eclasses since I could see multiple active
series of eclass versions being updated in parallel.

I'll also note that the wording of Proposal 4 doesn't preclude doing
proposal 3 if the eclass maintainer thinks it is appropriate (maybe
the RDEPEND should only change in some circumstances or there are
other changes happening at the same time).  The wording of Proposal 4
refers to when an eclass is changed, and if you make the change in a
new revision you aren't changing the previous one, and so you don't
need to bump all the ebuilds.  Of course the ebuilds would still need
to be bumped if they were modified to use the new eclass, per proposal
1.

Proposal 3 is superior to proposal 4 from the standpoint of overlays
that use eclasses in the main repository, since nobody will be going
around and revbumping their ebuilds.  Of course, any eclass change
should be posted on -dev and so there would be notice.

Adopting 1, 2, and at least one of 3 or 4 would eliminate all the
dynamic dependency issues in the tree.  If anybody is aware of any
that I missed and I'll add them.

Also, in the event proposal 1 and 2 are both adopted, here is proposed
streamlined wording:
Proposal 5: Anytime an RDEPEND in an ebuild is changed, the ebuild
must be revisioned.  This includes adding/removing inherited eclasses
which set RDEPENDs.

However, the council doesn't have to approve the literal wording of
everything in the devmanual, so that might be overkill.  The devmanual
can of course explain the rationale for avoiding dynamic deps/etc.

-- 
Rich

Re: [gentoo-dev] Dynamic dependencies

[Rich Freeman]

On Thu, Oct 1, 2015 at 3:08 PM, Ian Stakenvicius  wrote:
>
>>
>> RDEPEND changes in eclasses Proposal 3: Anytime an RDEPEND in an
>> eclass is changed, the eclass must be revisioned. Proposal 4:
>> Anytime an RDEPEND in an eclass is changed, all ebuilds that
>> inherit the eclass in the gentoo repository must be revisioned.
>
> This one is trickier to deal with.  For starters, after thread
> between yourself and mgorny and I, I think we figured out that there
> wouldn't be an end-user breakage from people that have emerged a
> package eclass-rdepend'ing on one depset, when that depset is
> changed; it just ends up with everyone after the time of the change
> needing the new depset.
>
> There are specific cases where the revbump to rdeps will be
> necessary (and the eclass itself too, *if* keeping the old eclass
> around or differentiating which eclass version is inherited actually
> matters):
>
> - - slotmove operations on a dep in *DEPEND
> - - the addition of blocker atoms due to the introduction of
> conflicting packages
> - - the addition of atoms to address implicit dependencies that were
> missed before (and weren't in the ebuilds)
> - - adjustments to atoms based on changes made to the dependent
> package, with the changes now necessary to prevent breakage.  (IE:
> useflags changed in-place on a dep and the packages inheriting the
> eclass now need to ensure that the new useflag is set)
>
> ...there are likely other cases but I can't think of any right now.
>
> At any rate, the point here being that a simple minimum-version-bump
> in an eclass RDEPEND does indicate a divergence will occur between
> end-user VDB and the repo, but that doesn't necessarily mean it's
> something we need to avoid, or rather, we don't necessarily -need-
> to enforce convergence between the repo and end-user VDB.
>
>
> Once we get a bit more hashing out of the above I can try writing up
> the complicated proposal(5?,6?) wording...

Agreed.  Thanks for bringing this up again.  If you're adding an RDEP
to an eclass in anticipation of it being necessary for new ebuilds but
all the ebuilds in the tree still work with the old RDEP, then the
bumping can be selective.

Rather than trying to identify specific cases, why not identify the
intent, and then we can build out the individual cases on the
devmanual, and revise it over time as necessary.

Proposal 3a might be: Anytime an RDEPEND in an eclass is changed, the
eclass must be revisioned unless all ebuilds in the gentoo repository
will continue to work correctly with the old RDEPEND.
Proposal 4a might be: Anytime an RDEPEND in an eclass is changed, all
ebuilds that inherit the eclass in the gentoo repository must be
revisioned if they will not continue to work correctly with the old
RDEPEND.

There is no need to have the council approve revisions to the
devmanual every time somebody points out a new case where this
happens.  The goal here is to just let everybody air their opinions,
and set a general direction so that those implementing it don't get
caught in the crossfire.

It sounds like pkgmove should be added to all the proposals as an exception.

-- 
Rich

Re: [gentoo-dev] LibreSSL import plan

[Rich Freeman]

On Wed, Sep 30, 2015 at 3:29 PM, Anthony G. Basile  wrote:
>
> Yes you could use symbol versioning, and you can do the side by side by
> renaming the library but that's a real pita for us since we'd have to hack
> build systems to link against the correct library name.  Ths should have
> been done upstream.
>

Agree, though to be fair it was a failing in openssl before it was a
failing in libressl:
readelf -sW /usr/lib64/libssl.so | grep "@"
(output: nothing that didn't come from glibc)

> You'd have to name the libraries differently and you'd have to hack the
> LDLFAGS to aim the build to the correct library.  That, in my opinion, is
> the killer to this idea.  There is also the issue that some libc's like
> uclibc don't do symbol versioning, but I would deal with that in other ways.

++  There might be some solutions to automate this, but it would be a
PITA.  I think the better solution is to fix C itself.

>
> @rich0.  Just a side comment.  You said somewhere that maybe apache will
> choose openssl and postfix libressl and then we'll be in trouble.  No.  The
> incompatibility is at the abi not api level.  So, for example, some struct
> size might be different between the two because of internal implementation
> details, but both should provide a definition of the same struct in their
> header with the same members.  ie. apache should compile against either
> openssl or libressl and work, you just can't swap out your libssl without
> recompiling apache which you could do if you had full api compat.

I agree with this as long as both projects maintain API compatibility.
Whether that happens remains to be seen.  If openssl adds a new
feature and libressl decides that is a "bad feature" or libressl adds
a new feature and openssl doesn't have the manpower to keep up, or
whatever, then we'll start seeing things break, and then everybody
gets to pick sides.

As may be happening with ffmpeg/libav I suspect that eventually one
side or the other will become dominant, and at that point users will
have a clean solution again.  In the meantime they get to see WW3
unfurl on their desktops as they are forced to pick a side and decide
which packages they want to install.  I guess that would be a good
time to plug containers again.  (You know that symbol versioning is
broken when the solution is basically to install a completely
independent userspace for every process you run.)

-- 
Rich

Re: [gentoo-dev] LibreSSL import plan

[Rich Freeman]

On Wed, Sep 30, 2015 at 2:35 AM, Paweł Hajdan, Jr.
<phajdan...@gentoo.org> wrote:
> On 9/29/15 3:32 PM, Rich Freeman wrote:
>> The thing is that I think the libressl authors are shooting themselves
>> in the feet.  When upstreams do this sort of thing they think they're
>> making the upgrade path easier by not changing their symbol names.  In
>> reality, they're making the upgrade path harder by preventing
>> side-by-side adoption of the new solution.
>
> Yeah, it's not that obvious how to handle it best.
>
> Curious - how would the alternative look like? My reasoning is that if
> upstream changes symbols, that makes it easy for a distro to install it
> side-by-side. However, for anything to use such modified lib, they'd
> need to change all callers to use the alternative function names,
> wouldn't they?

Essentially, or somebody has to write a wrapper library.  But, once
you start changing the APIs everybody has to start tweaking their code
anyway to use the modified function prototypes.  Of course, they only
need to tweak the 10% of functions that changed, and not all of them.

Effectively it would mean that the new library would start out with
zero users, which is why nobody does it that way.  However, I think
the end result is worse, unless you maintain strict compatibility.

It hasn't been as much of a problem with mariadb because they haven't
gone changing all their APIs/protocols/etc.

This is the sort of thing that Java was trying to stop with their
compatibility requirements, and what a lot of companies try to do with
trademark.  The problem is that trademark doesn't really extend well
into things like symbol names and APIs.

Perhaps the in-between solution would be for forking upstreams to
preserve the same symbol names as long as the APIs are identical, and
change them when they are not.  I don't really see that having any
more impact on downstream consumers than silently changing the APIs
and it would probably get rid of the symbol collision problem.

-- 
Rich

Re: [gentoo-dev] LibreSSL import plan

[Rich Freeman]

On Wed, Sep 30, 2015 at 7:29 AM, Kristian Fiskerstrand  wrote:
>
> The way I see it this is relevant to the discussion at hand.

Admittedly it is a bit tangential, but it didn't seem worth forking
the thread over.  Certainly I'm not going to invent my own mailing
list and post it there, and then post here to advertise it.  I doubt
such a discussion will be all that welcome on the upstream mailing
list.

> Or is this just increasing our maintenance, and security tracking, etc
> burdens without any strong benefits?

I don't think that it is necessary to have a cost/benefit analysis
anytime somebody wants to introduce a new package in the tree.

Gentoo tends to be about making new alternatives available to users.
As long as hasufell is willing to do the work necessary to add the
necessary USE flags and blockers I don't see the harm in having this
in the tree.  If upstreams switch to requiring this library
exclusively and thus become incompatible with other upstreams which do
not, that is something that will affect us whether or not we allow
libressl in the tree (see ffmpeg/libav).

I think it was fair to pause to see if somebody could come up with a
better solution that allows co-existence, but absent that I don't see
any benefit from keeping libressl out of the tree.  We'll just
experience all the downsides of the fork without the upsides.

It might very well cost some of hasufell's time to maintain it, but
that is time he is freely offering, and it isn't like turning him away
is going to encourage him to spend more time on other Gentoo features.
Cost/benefit for a volunteer distro isn't a zero-sum game the way it
is if you're a manager of a 50-person development team.

I'd love to see somebody come out with a better solution for this sort
of thing, and it probably would need to be bigger than Gentoo to be
truly effective.  However, until such a solution comes along I don't
see the benefit of further delay.  That's just my two cents.

-- 
Rich

Re: [gentoo-dev] LibreSSL import plan

[Rich Freeman]

On Wed, Sep 30, 2015 at 8:10 AM, Kristian Fiskerstrand <k...@gentoo.org> wrote:
>
> On 09/30/2015 01:51 PM, Rich Freeman wrote:
>>
>> I think it was fair to pause to see if somebody could come up with
>>  a better solution that allows co-existence, but absent that I
>> don't see any benefit from keeping libressl out of the tree.
>> We'll just experience all the downsides of the fork without the
>> upsides.
>
> This is what worries me as well, as it increase workload and
> complexity affecting multiple projects without any immediate and
> obvious gain.

True, but that is the consequence of the decision to fork openssl in
this manner, and a possible future decision of downstream projects to
follow the fork or not (which may or may not happen).  It isn't
actually the result of a decision to allow libressl in the tree or
not.

That is, if apache decides to stick with openssl but postfix decides
to move to libressl, then users who want to use both with ssl support
are going to have a really hard time no matter what actions we take as
a distro, unless it involves patching the living daylights out of one
of the two pairs of software.

>
> Immediately I would think we'd need namespace isolation inspired by
> NixOS etc for this to work, but that isn't something that would easily
> be implemented and quite frankly would look scarily similar to Go's
> static linking and issues.
>

I thought a bit about that, but it isn't actually super-clean even if
you go sticking uuids on everything.

Suppose apache uses libfoo and libbar.  Libfoo switches to libressl,
and libbar sticks with openssl.  That is going to create a mess no
matter what you do with isolating their namespaces, because you're
forced to bring it all back together and not all software is designed
to handle that today (especially when you're not using --as-needed,
etc).  Flameeyes's blog entry keeps coming up:
https://blog.flameeyes.eu/2008/06/a-few-risks-i-see-related-to-the-new-portage-2-2-preserve-libs-behaviour

It really seems to me that the proper solution to symbol versioning
should be somehow built into the spec and should take into account
situations like these.  When I look around for solutions it seems like
everything comes down to hacks because the original design of C left
all of this out.

-- 
Rich

Re: [gentoo-dev] LibreSSL import plan

[Rich Freeman]

On Tue, Sep 29, 2015 at 8:22 AM, hasufell  wrote:
> No useful comments, so I will proceed as outlined in the transition plan.
>

I don't think your attitude is going to win you a lot of friends, and
I don't think that we're better off for it.

That said, I've yet to hear a workable alternative, and I don't have
one to offer myself.  I don't really like what has happened with
kerberos and ffmpeg and mysql, and I'm not looking forward to what is
going to happen with libressl.  I fear that as with the other
situations we'll end up with one solution used by 99% of systems, and
another solution used by 1% of systems, and no happy compromise that
lets people mix and match software that relies on either.  That really
doesn't strike me as the Gentoo way.

However, unless people stop promoting and/or using competing solutions
that share the same namespaces we're going to have these problems, and
we have to live with reality and not pretend that it doesn't exist.

If somebody can come up with a better solution, I'm all ears.  What
hasufell proposes isn't any worse than what we already have.  It just
fails to be any better.

As was pointed out there are some fundamental issues with just trying
to slot something like this unless you go patching the living
daylights out of the library and everything that uses it.

The thing is that I think the libressl authors are shooting themselves
in the feet.  When upstreams do this sort of thing they think they're
making the upgrade path easier by not changing their symbol names.  In
reality, they're making the upgrade path harder by preventing
side-by-side adoption of the new solution.

-- 
Rich

Re: [gentoo-dev] LibreSSL import plan

[Rich Freeman]

On Tue, Sep 29, 2015 at 9:43 AM, hasufell <hasuf...@gentoo.org> wrote:
> On 09/29/2015 03:32 PM, Rich Freeman wrote:
>> [...]
>
> I have waited 9 days. I don't see a reason to wait another few weeks,
> just because you like to bikeshed a lot.

I don't recall suggesting that you should wait longer.  That might be
why you didn't actually have any text quoted in your reply...

-- 
Rich

Re: [gentoo-dev] LibreSSL import plan

[Rich Freeman]

On Sun, Sep 20, 2015 at 7:14 AM, hasufell  wrote:
> On 09/20/2015 08:07 AM, Andrew Savchenko wrote:
>> Greetings,
>>
>> On Sat, 19 Sep 2015 23:04:14 +0200 hasufell wrote:
>>> Friends,
>>>
>>> I think it is time to import LibreSSL[0]. There are not many packages
>>> left that don't compile OOTB and those can be patched (e.g. dev-lang/ruby).
>>>
>>> My idea would be:
>>>
>>> 1. import "dev-libs/libressl" (this will block dev-libs/openssl) and
>>> introduce the global USE flag "libressl" with the following description:
>>
>> Please try to avoid such block, e.g. install libressl to a separate
>> location.
>
> No. I'm not going to hack downstream.
>

I don't think that it is the responsibility of the libressl maintainer
to actually get other packages to use libressl - though they should
document how it is done to facilitate this.

However, if your point is that you want to be able to use libressl
yourself as a drop-in replacement and this doesn't work if none of the
other packages you use actually build against it when you move it,
that is a fair point.

I do think this is a good topic for discussion, however.  It seems
like forks that keep the original namespace is all the rage these days
(kerberos, ffmpeg, openssl, mysql, etc).

You've been a proponent of something like nixos for a while, and I'd
think that this is exactly the sort of approach they take.  They don't
even keep the same namespace for a minor update of the same library.
Of course, they have designed their build systems with this in mind,
and perhaps this is a direction we need to evolve in as this sort of
thing is coming up more and more.

-- 
Rich

Re: [gentoo-dev] LibreSSL import plan

[Rich Freeman]

On Sun, Sep 20, 2015 at 9:27 AM, Manuel Rüger  wrote:
> On 19.09.2015 23:04, hasufell wrote:
>> Friends,
>>
>> I think it is time to import LibreSSL[0]. There are not many packages
>> left that don't compile OOTB and those can be patched (e.g. dev-lang/ruby).
>>
>> My idea would be:
>>
>> 1. import "dev-libs/libressl" (this will block dev-libs/openssl) and
>> introduce the global USE flag "libressl" with the following description:
>> """
>> libressl - Use dev-libs/libressl as SSL provider (might need ssl USE
>> flag), packages should not depend on this USE flag
>> """
>>
>
> Devmanual says to discuss global useflags here before introducing them.
> IMO merely announcing them is not enough. See
> 288d8cd90fca12fafd021d86837851d8cb5c6efe.
>

++

I'm not hostile to the proposed plan, but you should at least allow a
few days for things to settle down if we're not talking about security
patches/etc.

This isn't directed at anybody in particular, but I've been noticing a
bit of a recent trend towards wanting to discuss a change at all ==
bikeshedding for years -> time to pout and threaten to walk out.  It
isn't really helpful.  Gentoo is a fairly large project - this isn't
some little community of 5 developers who chat at the bar and then go
write their code.

Discussing change serves many purposes:
1.  It provides notice of the change.
2.  It is possible that others will notice something the authors missed.
3.  It provides an opportunity for education.  The way junior
contributors become senior contributors is by interacting with those
doing changes.  That might mean making well-intentioned suggestions
that don't work.

SSL is pretty important, and over time I could see this becoming
bigger than kerberos/ffmpeg/etc.  It makes sense to at least give this
change some thought.  My concern with holding it up is that we don't
actually have a practical alternative that we can implement today, at
least not that I'm aware of.  So, I'm more inclined to just let this
move forward and let somebody with a grandiose plan for managing
symbol collisions/etc to demonstrate it before holding everything else
up.  But, if people have plans that work in either the short- or
long-term please speak up.

-- 
Rich

Re: [gentoo-dev] LibreSSL import plan

[Rich Freeman]

On Sun, Sep 20, 2015 at 5:50 AM, Alexis Ballier  wrote:
>
> Yes, that's what gnome team is doing with gtk2 vs gtk3; however, I'm
> not sure how much work it is. Only package I know of providing
> different slots depending on what it's built upon is webkit-gtk.
>
> I can't imagine every library using {open,libre}ssl provide two slots,
> two different libraries, two different pkg-config and the like files,
> etc. And every package using a library that uses a library that uses a
> library that uses {open,libre}ssl to have to chose what ssl library to
> use.
>

I don't think the suggestion is to make it so that any package can be
built against either, though individual maintainers can support this.

I think the suggestion is to make it so that the libraries themselves
can be installed side-by-side, so that packages can depend exclusively
on one or the other and not effectively block each other.

When other distros do it, that is essentially what they're doing.  If
apache on debian uses libssl, then it depends on it, and you can't
install it using libressl (or if you can that is a separate pacakge).

-- 
Rich

Re: [gentoo-dev] LibreSSL import plan

[Rich Freeman]

On Sun, Sep 20, 2015 at 12:57 PM, hasufell  wrote:
> On 09/20/2015 06:47 PM, Manuel Rüger wrote:
>> On 20.09.2015 16:26, hasufell wrote:
>>> On 09/20/2015 03:27 PM, Manuel Rüger wrote:
 Please stop introducing further tree-wide changes regarding libressl.
>>>
>>> That's not possible, because in order to introduce the USE flag, we have
>>> to break the dep-graph on ~arch temporarily (for 'libressl' USE flag
>>> only ofc), because of circular deps.
>>>
>>> I am working on restoring it now. This does not affect stable branch at
>>> all and no one who is not using 'libressl' USE flag (which is
>>> practically impossible currently).
>>>
>>
>> Yet the way you execute your plan now violates several devmanual
>> policies. Is there any reason for that rush?
>>
>
> Any reason to bother me? There have been several threads about libressl
> and the overlay has been up for more than one year I think. If you have
> a suggestions, say it.

Here's a suggestion: go do something else for 48h.

Seriously.  This bickering isn't helpful.  You posted an idea on -dev.
It might be a great idea.  Let's at least allow it to be discussed
before implementing it.

This isn't about the stable branch.  It is about how we do things.  If
everybody just started making global changes less than a day after
posting about it on -dev there would be chaos.

-- 
Rich

Re: [gentoo-dev] Inconsistent and messy layout of team maintainership in Gentoo

[Rich Freeman]

On Sat, Sep 19, 2015 at 7:32 PM, Raymond Jennings  wrote:
> Is it possible for projects to be nested, possibly within multiple
> super-projects?
>
> Like, for example, a project dealing with a gnome chat client itself being
> members of both the gnome and the chat projects (hypothetically speaking)?

Yes, many projects are already nested in such a manner.
For example, the website project is a sub-project of PR.


-- 
Rich

Re: [gentoo-dev] Inconsistent and messy layout of team maintainership in Gentoo

[Rich Freeman]

On Sat, Sep 19, 2015 at 5:07 PM, Daniel Campbell  wrote:
> +1 in general, but I'm a little pensive about allowing non-devs to
> become official project members. Becoming a developer can be a
> grueling process, so I understand that some don't have the time or
> motivation, and still want to help out. So perhaps we could have
> contributors who wish to be project members pass our ebuild test, or
> some other litmus test to prove themselves, like a developer proxying
> them or something. Non-devs don't have direct push permissions to our
> main repo, so to my knowledge they'd still have to go through a dev.
> I'd just like to see some sort of documentation that sets expectations
> for non-dev project members so that a new contributor understands what
> would be expected.

I don't think that project member and commit access have to be
all-or-nothing together.

I'd suggest leaving it up to each team to decide who is allowed to be
a member if they're a non-dev, and the rest are just contributor.  The
team can use whatever rules seems best.

Project members don't necessarily have formal powers, though typically
they participate in elections for the lead.

As always, if there is trouble there is always comrel or council.  I
think most teams should be able to figure out who should and shouldn't
be acknowledged as a member.

But, there is still the GLEP 39 issue.  I'd suggest the "contributor"
label for things like alias members until that is sorted out.  There
isn't really much distinction in reality.

-- 
Rich

Re: [gentoo-dev] Inconsistent and messy layout of team maintainership in Gentoo

[Rich Freeman]

On Fri, Sep 18, 2015 at 1:10 AM, Michał Górny  wrote:
> Dnia 2015-09-17, o godz. 17:19:17
> Michael Orlitzky  napisał(a):
>
>> Replying somewhere randomly with an idea.
>>
>> Since projects are now on the wiki, why don't we use that as the
>> canonical source of project members? It's machine readable, although not
>> so nice to have it located outside the repo/ldap.
>
> How about... because:
>
> 1. You can't list developers who are not subscribed on the wiki. I've
> asked for some solution multiple times, and so far people are just
> INVALID-ating the bugs and pointing fingers. Of course, it could be
> partially related to the fact that we still don't have any SSO for
> Gentoo services, and have to ask people to sign up manually everywhere.

I'd have to dig up whether we actually took a vote, but I'm pretty
sure the council doesn't consider this a problem.  If people want to
contribute, they can sign up for the wiki.  It isn't like it is one of
those even non-FOSS tools.

However, I think the rest of your concerns are valid, especially the
concern about being able to include non-devs on aliases (whether you
consider them "members" or not).  I don't have a problem with asking
them to sign up on the wiki, but we'd need to change the templates/etc
so that they can still be listed somewhere on the project.  And of
course somebody has to build it.

-- 
Rich

Re: [gentoo-dev] Inconsistent and messy layout of team maintainership in Gentoo

[Rich Freeman]

On Thu, Sep 17, 2015 at 7:20 AM, Anthony G. Basile  wrote:
> On 9/17/15 7:05 AM, James Le Cuirot wrote:
>>
>> On Thu, 17 Sep 2015 06:57:08 -0400
>> "Anthony G. Basile"  wrote:
>>
>>> Totally rethink the idea of emails aliases as something that is
>>> created on the fly.  We just need to know who should get emails for a
>>> package when it comes to bug reports.  Why can't that be calculated
>>> on the fly from the metadata.xml?
>>
>> I've not read every last part of this thread but I think I like where
>> this is going. I just want to be sure that people besides those in the
>> Java herd/project/whatever can continue to receive emails for
>> j...@gentoo.org. For instance, gnu_andrew is not a dev and does not
>> intend to be but he still likes to be CC'd on all Java mail. I would
>> not like to have to add his address to the metadata.xml of every Java
>> package.
>>
> Yes.  If the metadata.xml contained both  and   tags,
> it would go to all of j...@gentoo.org and to gnu_andrew.  Already not all
>  are devs.
>

Can you clarify where you intend to stick gnu_andrew?  I think the
concern was NOT listing him as a maintainer on every java package, but
allowing him to still get mail.

I think there are a few levels of indirection involved in your proposal.

Packages have entities who are interested, who can be projects,
maintainers, and observers.
Projects have members and observers.

It sounds like your proposal is basically that every package gets an
email alias, and every project gets an email alias, and they're all
dynamically populated based on both project member/observership and
package metadata.  A package alias may very well end up including one
or more project aliases.

Example.  Bug logged against media-tv/mythtv
Bugzilla assigns to and emails to media-tv/myt...@g.org
Email to media-tv/myt...@g.org goes to myt...@g.org, which goes to
cardoe,rich0@g.o.

Another bug gets logged against media-plugins/mythplugins
Bugzilla assigns to and emails to media-plugins/mythplug...@g.org
Email to media-plugins/mythplug...@g.org goes to myt...@g.org, which
goes to cardoe,rich0@g.o.

Was that what you were getting at?  The obvious concern is whether
there is an issue with all those dyanically-created/destroyed bugzilla
and email accounts.

If you were hoping to cut out a layer I'm not sure that you'd be able
to as easily accommodate cases where both the project and the package
both have multiple email addresses listed.  However, I guess you could
just have bugzilla assign the first project it sees and if not the
first member and if not the first observer and then CC everybody else,
and then any projects or observers in the list could be email aliases.
That is closer to what we're doing today.

-- 
Rich

Re: [gentoo-dev] Re: Dynamic dependencies

[Rich Freeman]

On Thu, Sep 17, 2015 at 8:22 PM, Duncan <1i5t5.dun...@cox.net> wrote:
> So council was called in, and it asked the portage folks to take some
> steps that, portage development being what it is, had the effect of
> slowing down and delaying things for long enough that, hopefully, people
> have had time to come to terms with the changes, and with a bit of
> familiarity, see static-deps aren't so bad, after all.

To be clear, the only thing the council did was ask the portage team
to clarify whether they intended to make it a default, and to provide
a plan/policy for virtuals/eclasses/etc.  There was a lot of the usual
panic on the lists and it wasn't actually clear whether anybody
intended to change anything or if we were making a lot of ado over
just an idea.

The purpose of the discussions on-list are mostly to try to go ahead
and figure out what we want to do with virtuals/eclasses/etc so that
the portage team can make the change when they're ready.  My
understanding is that they're now fairly eager to do so, but perhaps a
bit gun-shy about dealing with all the likely bikeshedding.   So, a
few council members broached the subject so that people can throw
their stones at us and maybe wear themselves out.  In this way we also
protect our generous salaries by making the job sound even less
enviable than it must already seem.  :)

A year ago this got an huge outcry.  Of late I'm barely hearing a
whimper of protest.  I think that people have been dealing with broken
dependency resolution long enough with subslots now that they just
want to see the pain go away.  From what I've heard it hasn't been too
painful to disable dynamic deps, and I never really had issues with it
with paludis when I was using it.  I did take a look at the results of
an emerge --changed-deps world and it came out to 388 packages to
rebuild, much of it being kde.

-- 
Rich

Re: [gentoo-dev] Dynamic dependencies

[Rich Freeman]

On Wed, Sep 16, 2015 at 11:49 AM, Andreas K. Huettel
 wrote:
> Hi all,
>
> here's a quote from the Council 20140826 summary:
>
>> Dynamic dependencies in Portage
>> ===
>> During discussion, is was remarked that some changes, e.g. to
>> dependencies in eclasses, could require mass rebuilds of packages.
>>
>> Vote:
>> - "The council asks the Portage team to first outline their long-term
>>   plan regarding removal or replacement of dynamic dependencies,
>>   before they remove this feature. In particular, tree policies and
>>   the handling of eclasses and virtuals need to be clarified."
>>   Accepted unanimously.
>
> Since there seems to be interest in the Portage team to go ahead with that
> plan, I'd like to ask about the tree policies and the handling of eclasses and
> virtuals.

I'll go ahead and start a tangent on this thread right here.  As a
first step can we separately consider the proposal to require a
revbump anytime a package's RDEPENDS changes?  I'm referring here to
directly-specified RDEPENDS, not those inherited from an eclass or
virtual.

I agree completely that we need to solve the eclass and virtual issue
and that is by far the stickier part of the mess.  However, can we at
least get ebuild authors to stop making changes to their RDEPENDS
without revbumps?  If nothing else that will hopefully provide some
immediate relief to users with dependency breakage, and it doesn't
entail the amount of mass-rebuilding you can potentially get with the
eclass and virtual side of the problem.

And I acknowledge that this is not my original idea...

-- 
Rich

Re: [gentoo-dev] Dynamic dependencies

[Rich Freeman]

On Wed, Sep 16, 2015 at 3:31 PM, Michał Górny  wrote:
>
> As for virtuals and eclasses, I don't really understand why anyone
> thinks they are special in any regard. In both cases, we're talking
> about regular dependency change in metadata, and we need to understand
> the consequences. And they're going to be the same whether we change
> dep A to B, A to virtual, and whether we change it directly or via
> eclass.

Sure, but a developer KNOWS when their RDEPENDS change when they're
modifying it directly in an ebuild.  If they inherit an eclass and it
sets an RDEPEND and the eclass changes, then it effectively changes
the dependency for every ebuild that inherits it even though there
weren't any commits to any of those ebuilds.

So, we need to think about what kinds of changes we allow to eclasses.
This also applies to virtuals, but those don't have the same kind of
indirect impact to packages that RDEPEND on them any more than changes
in any other RDEPEND of an RDEPEND.

> 2. Dependency changes that don't need to apply immediately don't need
> revbump. For example, if foo.eclass raises minimal required version of
> a dependency but all packages built so far will work with the old one.
>

Are we talking about a build dependency or a run-time dependency?  I
don't get why we'd increase the minimal required version of a run-time
dependency if everything built so far still works with the old
version.

Also, assuming that there is a case where this actually happens, does
this have any impact on running --depclean or any other obscure
blocker issues because the version in VDB is no longer in the tree?

When the policy is just a simple "always revbump when you change
RDEPEND, whether you're an ebuild, an eclass, or a virtual" then I can
see how it is painful, but I can also see how it is fairly
bulletproof.

-- 
Rich

Re: [gentoo-dev] Dynamic dependencies

[Rich Freeman]

On Wed, Sep 16, 2015 at 4:17 PM, Michał Górny  wrote:
>
> If you modify an eclass, you're responsible for the outcome. Even if
> means revbumping hundreds of ebuilds for the sake of it. Note that this
> is the kind of revbump that wouldn't require resetting stable keywords
> as long as deps are satisfied.

Ok, so it sounds like there are two eclass proposals out there:
1.  Modify the eclass in place and then revbump all ebuilds that use
it for which the new RDEPEND matters (the second part of this email).
2.  Don't modify the eclass in place, and then update the eclass
reference and revbump any ebuilds for which the new RDEPEND matters,
and for the ones that don't matter there really is no change since
they use the old eclass.

#1 results in less eclass propagation, but requires mass-commits.  #2
seems safer and allows the eclass and ebuilds to be modified
separately, but still requires lots of ebuild changing.

>
> Runtime. The minver can be raised for developer's convenience -- like
> when a large number of packages is expected to require a new version
> soon, and the developers would otherwise have to override the eclass-
> specified versions. Instead, the eclass is updated and new requirements
> apply.

Does not revbumping in this situation actually save us much other than
the need to do the revbumps themselves?

If the dependency uses a slot operator then everything is going to get
rebuilt anyway as soon as the first package comes along and triggers
an update.  Then again, it does save us a bunch of revbumps.

-- 
Rich

Re: [gentoo-dev] Dynamic dependencies

[Rich Freeman]

On Wed, Sep 16, 2015 at 4:44 PM, Ian Stakenvicius  wrote:
>
> - - emerge -uD @world would update the dep anyhow
>
> - - emerge -u @world wouldn't rebuild the package if that package
> didn't change, and if the package did change then the new dep would
> get built.

Just to be clear, my point was that if the reason the eclass was
updated was because some new package version in the tree needed it,
and you update that other package in the tree, then that will update
the dependency, and that would in turn rebuild anything with a slot
operator dep on it.

Which is of course exactly what should happen if the soname/ABI needs to change.

>
> So I think I can safely drop my concern.  Care still needs to be
> given, certainly, as if the in-tree package isn't revbumped and gets
> a patch that only supports the new dep, then suddenly systems will
> fail when said package re-emerges.  But that seems bad practice anyhow
> .

Right, a patch change would always require a revbump and that is
nothing new.  The only case that doesn't require a revbump is a
build-time change.  And if somebody makes a build-time change with the
assumption that the new minimum depencency version is in effect it is
fine, since anybody with it already installed won't be rebuilding it,
and if anybody does rebuild it then portage will check and force the
dependency update so everything is fine at build time.

So, assuming we want to entertain the "only revbump if necessary"
eclass wording, do we think we can actually come up with something
that not only makes technical sense, but where we can actually expect
developers to get it right 99% of the time?  Are we encouraging
dangerous behavior just because a few of us might or might not be able
to get it right?

I suppose if somebody does mess up we can go in and revbump a bunch of
ebuilds.  The thing is that such problems are very hard to detect -
they usually manifest as users posting bizarre portage output on lists
and forums and being showered with a lot of bad advice before somebody
who knows what they're talking about notices the thread, and it can
happen a while after the commit that introduced the problem.

So, part of me really wonders if it is worth it just to save a bunch
of revbumps that probably could be done by a script and with git it
can even happen atomically and with the possibility of review or
testing.

-- 
Rich

Re: [gentoo-dev] Inconsistent and messy layout of team maintainership in Gentoo

[Rich Freeman]

On Wed, Sep 16, 2015 at 5:25 PM, Michał Górny  wrote:
>
> So, what are your thoughts for unmessing this?
>

This sounds familiar.  :)

Honestly, I wouldn't mind combining them all.  Just today I was pinged
by an !expn in IRC and thought to myself, "how many times have I typed
the wrong command to translate a group name into a list of members?"

Is there any way to just have one list of groups, and by all means we
can have a group-type as an attribute if that serves any purpose at
all, and we can either name the group by an email alias, or assign an
alias to a group?  Ideally the same list would be used to populate the
mail aliases.  That seems like it would make life much easier on
anybody writing software that accesses such lists, and perhaps we'd
stop worrying about what exactly a herd or a project or an alias is,
and which ones need to elect leads, and which ones don't, and maybe
just let each do whatever makes the most sense for the job at hand.

-- 
Rich

Re: [gentoo-dev] www-apps/otrs: needs new maintainer

[Rich Freeman]

On Tue, Sep 15, 2015 at 3:56 PM, Robin H. Johnson  wrote:
> On Tue, Sep 15, 2015 at 02:13:34PM +0200, hasufell wrote:
>> On 09/15/2015 02:00 PM, Peter Stuge wrote:
>> > If you have interest in this package then you can do one or more of:
>> > * become a Gentoo developer (ha-ha)
>> hmm
> For background, SGW did work on becoming a developer once before, but he
> mainly found he was short on time to handle more than the packages that
> he had a business interest in.
>
> I used to proxy-commit changes from him to the Amanda package, but found
> myself short on time to test them as well in the long run.
>

To be constructive, a github pull-request is probably the most
effective way to get a proxy maintainer to notice your work right now
and commit it for you.  Bugs assigned to proxy-maintainers@ might also
work.  Believe it or not people will actually use ebuilds you attach
whether or not they get into the tree.  Another option is to publish
your own overlay, and that makes a good foundation for a pull-request
anyway and it is easy to use for those who do want to use the overlay
directly.

And do consider signing up as a proxied maintainer for a package
you're interested in.  If you do establish a long-term relationship
with a package it probably will make devs more willing to commit your
changes without a lot of testing on their part.

-- 
Rich

Re: [gentoo-dev] RFI: A better workflow for github pull requests

[Rich Freeman]

On Sun, Sep 13, 2015 at 9:02 PM, Raymond Jennings  wrote:
> I agree.  I think that any "master" version of whatever repo we use should
> be hosted on gentoo owned infrastructure.
>
> Github might be allowed to take pull requests but I think it should be a
> slave to whatever's hosted on gentoo.
>
> That way if anything gets screwed up on github gentoo could always hit the
> big fat reset button on gitub

That is essentially all that is being proposed.  The pull request
system already exists on github.  Really all the proposal is about is
better integrating it with bugzilla, which if anything makes the
content more accessible via FOSS tools.

Counter-intuitively, the repo itself is actually the part that
concerns me the least.  With git the local clone you keep on your
desktop is just as suitable as any other copy of the repo should we
ever have to hit the reset button.  The part that isn't distributed is
all the comments, discussion, issues, review, etc which generally goes
into bugzilla.  If that were also a distributed database I'd have few
concerns about where that was hosted, since we're essentially talking
about the hosting only.

-- 
Rich

Re: [gentoo-dev] www-client/chromium gtk3 support

[Rich Freeman]

On Sat, Sep 12, 2015 at 12:55 AM, Raymond Jennings <shent...@gmail.com> wrote:
> On Fri, Sep 11, 2015 at 5:13 AM, Rich Freeman <ri...@gentoo.org> wrote:
>>
>> On Fri, Sep 11, 2015 at 5:03 AM, Daniel Campbell <z...@gentoo.org> wrote:
>> >
>> > I like the general 'gtk' flag we generally use to choose *which*
>> > toolkit, and local USE flags for specific versions, if they are
>> > supported. But in that case, the general gtk flag should be
>> > interpreted as the latest version supported, so users don't come
>> > across weirdly behaving packages that default to gtk2 (unless that
>> > version is the most stable).
>> >
>> >...
>> >
>> > For starters, versioned USE flags more than likely don't belong in
>> > make.conf's USE variable and shouldn't be global.
>
> Personally i disagree with this.
>
> Versioned use flags for widely used dependencies (like a windowing toolkit)
> IMO qualify as global USE flags because they have a common effect across
> many packages.

He wasn't suggesting that they have different meanings for different
packages.  By saying that they shouldn't be global he meant that users
should not typically be manipulating them at a global level, such as
in make.conf.

Back in the day it was common to stick flags like these in make.conf
or in profiles, since if you didn't packages wouldn't build GUIs and
such.  That was before USE defaults and it caused a lot of headaches
when multiple versions of toolkits started coming along and setting
these flags started causing harm.

But, the way we use the terms local/global USE flags is confusing.
They can mean that a flag has a package-specific vs global meaning, or
the terms can mean that it is recommended that the flag be enabled at
the package.use level vs at the make.conf level.  To be fair to you,
until very recently the first meaning was the most common.  People are
talking more about the second meaning of late because of problems that
happen when people try to tweak fairly detailed settings like gtk3 at
the global level.

>
>> I'd be tempted to even say to not have gtk3 but instead call the flag
>> chromium-gtk3 or whatever so that it becomes very difficult to put in
>> the global config.  However, that goes against our general principle
>> of letting the user break their system and keep the pieces if they
>> think they know what they're doing.  If somebody WANTS to test out a
>> gtk3-only system or whatever they should have the freedom to do so,
>> understanding that testing sometimes uncovers problems.
>
> I actually also think that there should be a single USE flag for building on
> gtk3, called gtk3.  calling it "(packagename)-gtk3" is a bit redundant, and
> also flies in the face of having a single global flag with a coherent
> purpose.
>

The only reason for doing it the other way would be to make it harder
for users to shoot themselves in the foot by setting these flags in
make.conf.  They'd have to put 50 flags in make.conf and not just one.
However, in general Gentoo operates under the principle that while we
should avoid surprising the user, we shouldn't actually make it hard
for the user to override our decisions when they feel it is best.

-- 
Rich

Re: [gentoo-dev] Re: www-client/chromium gtk3 support

[Rich Freeman]

On Sat, Sep 12, 2015 at 6:00 AM, Duncan <1i5t5.dun...@cox.net> wrote:
>
> Just bite the bullet and create entire USE flag families such that an
> ebuild can choose the flag appropriate to how it actually uses it.  AFAIK
> this would need EAPI help, for reasons given below, but it should be
> doable.
>
> gui
> gui-gtk
> gui-gtk-2
> gui-gtk-3
> gui-qt
> gui-qt-4
> gui-qt-5

I'm going to have to read the rest of your email about 14 times to
fully grok it, but one thing that strikes me about this approach, and
perhaps mine as well, is that this assumes that USE=-gui should imply
USE="-gtk -qt" or similar.

What if qt or gtk is used for more than just the GUI.  What if the
console version of the program still uses other functions in these
toolkits besides window decoration/etc?

Granted, I suspect that in such a case turning off any of this stuff
is probably not build-time-configurable.  If you want the console-only
version you'd just pass the appropriate command line option (like
deluge's --ui option).

However, it is conceivable that you could design a build system so
that the GUI requires qt and libfoo, spell check requires qt only, and
you could build the program with GUI support, spell check support, or
neither.  If you built with USE="-gui spellcheck" then you'd want qt
enabled but libfoo disabled.  That is obviously a highly contrived
example and perhaps we don't need to worry about this scenario.
However, it does illustrate the general danger of making USE flags a
strict hierarchy.  A hierarchy like qt/qt4 is probably fairly safe,
but when you generalize to gui/qt/qt4 you're making the assumption
that qt is only used for guis.

But I do like the concept, well, conceptually.

-- 
Rich

Re: [gentoo-dev] USE="gui"

[Rich Freeman]

On Fri, Sep 11, 2015 at 2:03 PM, Ian Stakenvicius  wrote:
> I'm just wondering if we're jumping the gun a little bit on
> IUSE="gui"..  yes it'll be nice to have one flag that "just works"
> for anyone not caring about the details, but it'll also mean
> propagating a slew of REQUIRED_USE=" {X,wayland,gtk,qt4,qt5}? ( gui
> )" entries and a lot of extra use-defaults which may or may not
> cleanup the sub-profiles of desktop/ 

A completely valid concern.  Of course, there is no requirement that
all this stuff happen overnight.

> Also, I believe we need to have the conversation about the pros and
> cons of IUSE=gui here before the council meeting, yes?

You can read my original post to -project:
http://article.gmane.org/gmane.linux.gentoo.project/4776

I did start it out with my reservation that this probably wouldn't
come to a vote.  However, I did want to at least toss out a specific
proposal so that we have something to throw darts at, vs just going
around the room saying "sounds like something that might need some
attention."

This is of course an opportunity to have that conversation, but I
recognize that we're starting pretty late considering the timing of
the meeting.  I didn't really intend to actually push for a vote on
this.

Most likely we'll express thoughts both pro and con, and then take it
back to the lists and maybe try to finalize something next month.

My sense is that this has been going on for a long time though and
we're seeing problems over and over.  I agree that wayland is still a
bit off in the future, but I can see it coming up again there.  In the
meantime both qt and gtk have run into this.  I don't want to do
something just to do something, but it seems like my proposal is along
the lines of what most have been talking about.

-- 
Rich

Re: [gentoo-dev] Re: www-client/chromium gtk3 support

[Rich Freeman]

On Fri, Sep 11, 2015 at 1:11 PM, Duncan <1i5t5.dun...@cox.net> wrote:
> Rich Freeman posted on Fri, 11 Sep 2015 08:13:48 -0400 as excerpted:
>
>> USE=gui or something like that if the main effect is to have a gui or
>> not.
>>  That is the sort of thing that SHOULD go in make.conf or in a profile.
>> If disabling gtk makes it a console-only application then use the gui
>> flag.
>
> I like the general proposal, but since it's going to council, can we try
> to kill another bird with the same stone?  This USE=gui helps...
>
> Wayland's coming, and to the extent that USE=X has previously indicated a
> GUI, much like USE=gtk and USE=qt indicating the same thing, we're going
> to have problems.
>
> Can we make USE=gui the generic policy for that, and deprecate more
> specific forms for choosing /any/ gui, so they can be used for choosing
> /which/ gui?

That was exactly why I used "gui" and not "X".  We're going to run
into the exact same problem once Wayland comes along with the way
things have been done so far.

>
> The question then remains whether ncurses, etc, should be treated as a
> gui. Maybe make mention of that one way or the other in the policy as
> well.
>

I actually was pondering that and left it out of my email.  My gut
feeling is that ncurses should be left alone for now.  USE=-gui would
mean console-only, whether that happens to include support for
ncurses, aa, or whatever.

Are there any applications out there which behave dramatically
differently if they do/don't have ncurses support built-in?  If you
set TERM=dumb I imagine that software which actually supports this
would just behave accordingly (ie if it is just using colors or moving
progress bars or whatever it will drop the decorations).  Usually
though dumb terminal software tends to be somewhat dedicated (for
things like editors and the like).

I don't want to complicate things if nobody really cares about them.
However, in theory you could treat various console-enhancing libraries
in the same way.  There is also svgalib and the like.

-- 
Rich

Re: [gentoo-dev] www-client/chromium gtk3 support

[Rich Freeman]

On Fri, Sep 11, 2015 at 5:03 AM, Daniel Campbell  wrote:
>
> I like the general 'gtk' flag we generally use to choose *which*
> toolkit, and local USE flags for specific versions, if they are
> supported. But in that case, the general gtk flag should be
> interpreted as the latest version supported, so users don't come
> across weirdly behaving packages that default to gtk2 (unless that
> version is the most stable).
>
>...
>
> For starters, versioned USE flags more than likely don't belong in
> make.conf's USE variable and shouldn't be global.
>

That was roughly my proposal.

USE=gui or something like that if the main effect is to have a gui or
not.  That is the sort of thing that SHOULD go in make.conf or in a
profile.  If disabling gtk makes it a console-only application then
use the gui flag.

USE=gtk if the main effect is to select /which/ toolkit is used if
more than one is optionally supported.  That /might/ go in a make.conf
or profile, but probably shouldn't in general.  It is more appropriate
for something like the desktop/gnome profile than the desktop profile.

USE=gtk# if you're picking which version to use.  That should /almost
never/ go in a profile (unless you're talking about a testing profile
of some kind, such as on an overlay), or in a global config unless you
REALLY know what you're getting into.  Users setting this globally
should expect to run into bugs.  The package should default these
flags to whatever is most appropriate for the specific package.

I'd be tempted to even say to not have gtk3 but instead call the flag
chromium-gtk3 or whatever so that it becomes very difficult to put in
the global config.  However, that goes against our general principle
of letting the user break their system and keep the pieces if they
think they know what they're doing.  If somebody WANTS to test out a
gtk3-only system or whatever they should have the freedom to do so,
understanding that testing sometimes uncovers problems.

Of course any change will need a transition period, news, handbook
updates, etc.  For the person who wants the "just works" experience
they can pick a profile and it will do the right thing, and if they
want to tailor things a bit more the USE=(-)gui flag will do what it
would be expected to do.

-- 
Rich

Re: [gentoo-dev] www-client/chromium gtk3 support

[Rich Freeman]

On Thu, Sep 10, 2015 at 8:13 AM, hasufell <hasuf...@gentoo.org> wrote:
> On 09/10/2015 02:03 PM, Rich Freeman wrote:
>>
>> Suppose you want to run on a non-embedded system with limited RAM and the
>> ability to choose means you can use one of the two libraries
>> exclusively, thus eliminating the need to load the other library?
>> Being able to control what libraries are in use is a key feature of
>> Gentoo, IMO.
>>
>
> Any reference that gtk3 has a higher memory footprint?
>

gtk2+gtk3 in RAM at the same time has a higher memory footprint than
either one alone.  If any package uses one or the other, it will end
up being loaded into RAM, so there is potentially value in using one
of them exclusively.

I'm not suggesting that package maintainers should be forced to
support both whenever possible.  I just don't think they should be
discouraged from doing so.

-- 
Rich

Re: [gentoo-dev] www-client/chromium gtk3 support

[Rich Freeman]

On Thu, Sep 10, 2015 at 6:50 AM, hasufell <hasuf...@gentoo.org> wrote:
> On 09/10/2015 12:45 PM, Rich Freeman wrote:
>> On Thu, Sep 10, 2015 at 4:47 AM, hasufell <hasuf...@gentoo.org> wrote:
>>> On 09/10/2015 08:21 AM, Daniel Campbell wrote:
>>>>
>>>> For me to not support gtk2 in the spacefm ebuild would be providing a
>>>> package inferior to upstream.
>>>
>>> That sounds like spacefm with gtk3 is lacking anything. It is not.
>>> Providing choice for the sake of choice is not always a good idea.
>>>
>>
>> Suppose you want to run on an embedded system with limited RAM and the
>> ability to choose means you can use one of the two libraries
>> exclusively, thus eliminating the need to load the other library?
>> Being able to control what libraries are in use is a key feature of
>> Gentoo, IMO.
>>
>
> We are not optimizing GUI desktop systems for embedded systems . That's
> totally unrealistic and not a real use case.
>

Suppose you want to run on a non-embedded system with limited RAM and the
ability to choose means you can use one of the two libraries
exclusively, thus eliminating the need to load the other library?
Being able to control what libraries are in use is a key feature of
Gentoo, IMO.

-- 
Rich

Re: [gentoo-dev] www-client/chromium gtk3 support

[Rich Freeman]

On Thu, Sep 10, 2015 at 8:33 AM, hasufell  wrote:
>
> So this makes no sense, since it's already an unsupported corner case.

Just what use of Gentoo do you not consider an unsupported corner
case, which isn't already better supported by some other distro?

The whole point of using Gentoo is having "support" for all those
"unsupported corner cases."  If you just want everything to support
doing things in the one way which is most supportable, you're
basically doing a really bad job at re-inventing Debian.

I use quotes around support since all support on Gentoo is
best-effort, and that is all I'm getting at here.  If a package
maintainer can support multiple configurations and are willing to do
so, they should be encouraged to do so.

>
>> I'm not suggesting that package maintainers should be forced to
>> support both whenever possible.  I just don't think they should be
>> discouraged from doing so.
>>
>
> Yes, they should be discouraged. It's a QA matter.
>

Well, I'm glad we've all aired our opinions on the matter.  :)

I just fail to see the QA issue here, unless it again boils down to
that it is easier to do QA when you have one configuration (like
Debian) and not many (like Gentoo).

The other issue that keeps coming up is that we don't have good
standards for USE flag naming in these situations, and the solution to
that is to come up with a good uniform practice.

-- 
Rich

Re: [gentoo-dev] www-client/chromium gtk3 support

[Rich Freeman]

On Thu, Sep 10, 2015 at 9:07 AM, Michał Górny  wrote:
>
> The happy end result is, sometimes user has choice between 'working
> package' and 'package randomly segfaulting when you least expect it'.
> Of course, it's all hidden nicely under USE=gtk2 and USE=gtk3, so just
> *maybe* if you have the time to read local flag descriptions for every
> single package you may notice which of the flags should be enabled to
> get a working app.

I'd propose that anybody sticking USE=gtk2 or USE=gtk3 in their USE
flags should expect to have a less-supportable system.  The problem
isn't the fact that the library is configurable, but rather that we
don't provide users an easy way to just install the package in the
best-supported configuration with a GUI.  Users shouldn't have to read
all the local descriptions to figure out how to install a package -
there should be a reasonable default that does what they want.  That
doesn't necessitate only supporting a single toolkit version.

This is on the agenda for discussion at the next council meeting, and
has been the subject of numerous threads in various contexts.  I think
the biggest problem here is that everybody does things a little
differently.  That, and we know a lot more than we did back when devs
were first adding these kinds of versioned flags.

>
> I hope you are ready to pay the developers who will waste their time
> figuring out what goes wrong when you report a bug, until they figure
> out it's because you have forced GTK+ version which upstream thought
> they're supporting but they do not. That's certainly a better
> alternative than paying for hardware that can handle loading two
> libraries.

Again, I'm suggesting this should be up to maintainer's discretion.
If they choose to support two gtk versions, and upstream chooses to do
the same, then why should we worry about filing bugs against a version
they chose to support?  If they don't want to support two versions,
they shouldn't.

This seems a bit like arguing that we shouldn't have
package-I-don't-use in the tree because the dev effort required to
support it could be better spent on package-I-use which isn't in the
tree.  That sounds nice, but I don't get to dictate what people spend
their time on, and the most we can do from a policy standpoint is
forbid contribution, not force contribution.

-- 
Rich