Re: [gentoo-dev] Unused profiles

[Michał Górny]

On Fri, 27 Jan 2017 23:42:04 +
"Robin H. Johnson"  wrote:

> On Thu, Jan 19, 2017 at 06:08:26PM +0100, Michał Górny wrote:
> > default/linux/amd64/dev/32bit-userland  
> Is this still supported? I've mulled over making a hardened version,
> specifically to support woodpecker.gentoo.org, as it's 64-bit hardware
> now, but the userspace is much older, and dates from when it was 32-bit
> hardware.

Doubt it. Dates back to 2009, seems incomplete and/or stupid,
and the README says you require a special hackery. You are probably
better off with plain x86 profile.

-- 
Best regards,
Michał Górny



pgpmZsvuLdxQu.pgp
Description: OpenPGP digital signature

[gentoo-dev] Re: berkdb and gdbm in global USE defaults

[Duncan]

Matt Turner posted on Fri, 27 Jan 2017 10:40:16 -0800 as excerpted:

> On Fri, Jan 27, 2017 at 8:22 AM, Mike Gilbert 
> wrote:
>> On Fri, Jan 27, 2017 at 2:54 AM, Mart Raudsepp  wrote:
>>> Then there is no need to think about what is enabled globally or not.
>>> Point being, use REQUIRED_USE sparingly, and rarely a good idea to
>>> block things with common global USE flags, or demand a local USE flag
>>> based on a default enabled global USE flag without locally USE
>>> defaulting that global flag too - and other such cases.
>>
>> I didn't really mean for this to turn into a thread about the merits of
>> REQUIRED_USE; in hindsight I should have left out that first sentence.
>>
>> Regardless of the REQUIRED_USE discussion, I don't think it makes sense
>> to have berkdb and gdbm in USE in make.defaults. I would like to move
>> them to IUSE defaults or package.use if necessary.
> 
> I think you should feel free to proceed with such a change.
> 
> FWIW, disabling these USE flags (and fortran) are among the first
> changes I made to a new make.conf.

TL;DR: see last sentence/paragraph.

FWIW, changes like that, either at profile upgrade time (with over a 
decade on gentoo on the same system, both hardware and software 
incrementally updated over time, I've done a few of those in my time) or 
worse made arbitrarily to existing profiles, are a big reason I 
ultimately decided USE="-* ..." worked best for me.  Because if I depend 
on the profile to make the decision for me, eventually that decision is 
going to change, and I'll have to dig into what and why and decide what I 
want to do in any case, so it's better to simply deal with all that up 
front, and USE="-* ..." is the way that's done on gentoo.

That way, global changes to my USE flags are only made when I make them, 
and I'll generally know at least the high-level why (say an update from 
qt4/kde4 to qt5/plasma5, with requisite USE flag changes) in that case, 
instead of having to deduce someone else's reason from the git log.  (I 
may still need to research the lower level why qt5/plasma5 require the 
new flags and decide whether I can/want/need to override, via manual 
semantic-desktop patchout or the like, potentially, but that's an 
entirely different focus that unlike global profile default-use changes 
the normal user won't need to deal with.)

Not saying I disagree with the change, but please at least make it a new 
profile only change if at all possible, so as not to disturb existing 
users until they decide to switch profiles, at which point they should be 
expecting at least some level of system adjustment to the new profile.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Re: Last rites: x11-drivers/ati-drivers, x11-libs/amd-adl-sdk, x11-libs/xvba-video

[Matt Turner]

On Fri, Jan 27, 2017 at 7:36 PM, Duncan <1i5t5.dun...@cox.net> wrote:
> Matt Turner posted on Thu, 26 Jan 2017 18:16:12 -0800 as excerpted:
>
>> # Matt Turner  (26 Jan 2017)
>> # Dead and replaced by media-libs/mesa[video_cards_radeonsi]
>> # (or the proprietary amdgpu-pro, which is not in tree).
>> # Masked for removal in 30 days.
>> # Bug #582406
>> x11-drivers/ati-drivers
>> x11-libs/amd-adl-sdk
>> x11-libs/xvba-video
>
> What about folks running pre-si radeons?  Radeon southern-island
> generation isn't that old, you know.

I highly recommend using the Mesa drivers.

Re: [gentoo-dev] Requirements for UID/GID management

[Rich Freeman]

On Fri, Jan 27, 2017 at 9:37 PM, Patrick McLean  wrote:
>
> I don't think we need to have stable UIDs/GIDs in the "normal" case of
> standalone users with a single Gentoo system at home.

Of course, but as you point out the enterprise case has more
sophisticated solutions.  I think the case of somebody running in a
home setting or a very small server setting is the main issue.

It isn't like inconsistent UIDs are the end of the world.  However,
IMO it still makes sense to at least try to standardize such things.
Really, if you have a package always installing the same user simply
sticking a default UID without any effort to avoid collisions is
better than nothing, but having a wiki page where people can register
UIDs isn't that big a deal.

If it fails, it fails...

-- 
Rich

[gentoo-dev] Re: Last rites: x11-drivers/ati-drivers, x11-libs/amd-adl-sdk, x11-libs/xvba-video

[Duncan]

Matt Turner posted on Thu, 26 Jan 2017 18:16:12 -0800 as excerpted:

> # Matt Turner  (26 Jan 2017)
> # Dead and replaced by media-libs/mesa[video_cards_radeonsi]
> # (or the proprietary amdgpu-pro, which is not in tree).
> # Masked for removal in 30 days.
> # Bug #582406
> x11-drivers/ati-drivers
> x11-libs/amd-adl-sdk
> x11-libs/xvba-video

What about folks running pre-si radeons?  Radeon southern-island 
generation isn't that old, you know.

Even if anything older is abandoned by the proprietary drivers (I 
wouldn't know as I won't do proprietary drivers), people may still be 
running older versions, with older kernels if required, and media-libs/
mesa with radeonsi isn't the correct migration path for them.

FWIW I was running Turks (northern-islands, pre-southern) until a few 
weeks ago when I upgraded to a 4k TV as a primary display and decided to 
upgrade the graphics to better handle it while I was at it.

(Tho for anyone thinking about it, the rx460, aka polaris11, I purchased, 
ended up being a poor choice as despite the hardware apparently 
supporting 3840x2160@60Hz, the freedomware drivers at least, apparently 
don't yet, so at that resolution I'm stuck at the same 30Hz that I 
believe I would have been stuck with on the older Turks card, which works 
better otherwise as the drivers for it have had longer to mature.)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Requirements for UID/GID management

[Michael Orlitzky]

On 01/27/2017 09:37 PM, Patrick McLean wrote:
> 
> To make something to solve our problem (and I suspect everyone
> else who cares about this), it would be sufficient to have a mechanism
> to override the default random assignment with a fixed UID/GID.

What I had in mind for this is that a "normal" user ebuild would look
like...

  EAPI=6
  # SYS_USER_NAME defaults to ${PN}
  SYS_USER_GROUPS="foo" # defaults empty or to ${PN}, not sure
  # SYS_USER_UID left undefined to get the next available
  # SYS_USER_HOME left undefined to get the default
  # SYS_USER_SHELL left undefined... etc.
  inherit sys-user.eclass
  ...
  # DEPEND set to "sys-group/foo" by the eclass
  # RDEPEND set to "sys-group/foo" by the eclass
  ...
  # src_install and pkg_prerm from the eclass.

To override the UID behavior, I think package.env would work, but an
overlay where the ebuild has SYS_USER_UID=42 is guaranteed to.

[gentoo-dev] Re: Requirements for UID/GID management

[Duncan]

Rich Freeman posted on Fri, 27 Jan 2017 16:23:02 -0500 as excerpted:

> On Fri, Jan 27, 2017 at 3:09 PM, Michael Orlitzky 
> wrote:
>> My first impression is that any package that doesn't care about its UID
>> should default to "first available", but if that causes problems, then
>> that's exactly the sort of use case I'm looking for.
>>
>>
> The ones I listed before were filesystems shared by multiple hosts,
> such as with nfs, containers, and chroots.  Granted, there are ways to
> deal with this sort of thing, but if you want to share your /var/www
> across a bunch of apache servers it would be nice if they all had the
> same UID for apache.

That's what an admin should be taking care of... if they have reason 
(like the given multiple machines accessing the same filesystem reason) 
to care.

And the way they'd do it under this proposal is simple enough.  Simply 
stick the admin-uid/apache (or whatever) ebuild in their overlay, 
uncomment the line in it that sets a specific UID instead of picking the 
next one in sequence, and change that specific UID if necessary for their 
installation.

The admin-uid/* ebuilds, meanwhile, could be pretty much empty save for 
two assignment lines, the commented specific UID assignment and an 
uncommented one listing the user name, and an eclass inherit, with the 
eclass simply reading the assigned name and picking a UID randomly if 
it's not already assigned, either by the user uncommenting the assignment 
line in the ebuild in their overlay, or a previous installation.

Of course the eclass could also check for an override variable, which 
would allow the user a second way of specifying UIDs -- via package.env 
or the like, similar to the way git-r3 allows environmental override of 
commit, etc.

(I say UID above, GID would be handled similarly, presumably in the same 
ebuilds and eclass, with different vars, of course.)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Requirements for UID/GID management

[Patrick McLean]

On Fri, 27 Jan 2017 14:53:18 -0500
Rich Freeman  wrote:

> On Fri, Jan 27, 2017 at 2:35 PM, Michael Orlitzky 
> wrote:
> > On 01/27/2017 01:52 PM, Rich Freeman wrote:  
> >>
> >> This doesn't really seem like a problem though.  Just have a table
> >> somewhere (wiki?) to track who is using what UID/GID and encode
> >> those defaults into the ebuild that creates those users.
> >>  
> >
> > It should be possible to have two different users with the same UID
> > in the tree, just like we can have two different packages that
> > install the same file. Eventually somebody will notice a file
> > collision, and then we add a blocker per usual.
> >  
> 
> I'm not saying we can't have random assignment for things where it
> doesn't matter, or fall back to random assignment, but it seems rather
> silly to go to all the trouble to have blockers when it would be just
> as easy to not have a conflict in the first place.  Now, if we have
> some longstanding issue from the past that might be another matter,
> but what's wrong with just picking an unused ID (again, for stuff that
> needs it)?
> 
> Telling users that they can't have postfix and apache on the same box
> because nobody can be bothered to pick IDs that don't collide seems
> like an arbitrary imposition.  Sometimes upstream creates conflicts
> that are just hard to work around (same SONAME, different ABI or even
> API, and so on).  And if we were talking about some binary-only
> upstream package that relies on hardcoded UIDs I guess blockers might
> be our only option, and users will just have to be happy that we're
> able to support it at all.  However, we shouldn't be the ones creating
> these kinds of conflicts.
> 

I don't think we need to have stable UIDs/GIDs in the "normal" case of
standalone users with a single Gentoo system at home. The people who
need predictable UIDs/GIDs are the "enterprise" users or the home users
who use things such as NFS. I work for a company that uses Gentoo, we
have a bunch of workarounds to make sure that UIDs and GIDs are
stable. To make something to solve our problem (and I suspect everyone
else who cares about this), it would be sufficient to have a mechanism
to override the default random assignment with a fixed UID/GID.
Possibly some file in /etc/portage or in the profile (or both) that
allows one to configure what UID/GID a user will get when the user is
being created. One advantage of this is that user.eclass could be
modified to support it, so we don't have to wait for a new EAPI before
taking advantage of it.

Re: [gentoo-dev] Requirements for UID/GID management

[Michael Orlitzky]

On 01/27/2017 04:15 PM, Michał Górny wrote:
>
>>   * users-update: cleanup can be done with --depclean now.
> 
> Err, cleanup is never easy. You shouldn't really remove a user if it
> owns any files. I guess you could abuse pkg_prerm() for that but
> depclean will be terribly slow then.
> 

What are our options if the user uninstalls a package that had its own
system user, and that system user still owns some files? Here are the
first few that come to my mind:

  1) Leave the user there but uninstall the sys-user/foo package. This
 is basically what we do now.

  2) Bail out of pkg_prerm and tell the user to clean up those files.

  3) Kill off the user anyway and leave the files.

The problem with (1) is that every user on the system is a liability,
and they never go away. If you test-compiled postgresql once five years
ago, there's still a "postgres" user with a shell on your system today.
Is the password locked? If not, is it strong? I have to ask the same
question about every other junk user in my /etc/passwd. There's also no
easy way to figure out which users are still needed at a later point.

The problem with (2) is that it's slow and annoying. Telling the user to
call "chown" on those files isn't safe, but I guess we could output a
"find ... -delete" command to be copy/pasted. We could also arrange
things so that if the user gets rid of the system user/group himself,
then the uninstall would succeed: basically a shut-up-portage override.
It'd still be horribly slow though.

The problem with (3) is that if some new user -- let's say "nobody" --
gets added to the system, it could be assigned a (previously removed)
UID that owns sensitive files. This would be abated by hard-coding UIDs
instead of taking the first available.

Hard-coding UIDs comes with its own set of problems though. It doesn't
totally solve the problem in (3), because users added outside of portage
could still be assigned a UID that owns files. And how do we guarantee
that a hard-coded UID won't be usurped? Some crappy build system could
throw a UID 80 onto the system and the user might never know until it
conflicts with apache. And right now we've got UID_MIN=1000 in
login.defs -- are we sure that we'll never have more than 1,000 sys-user
packages?

Re: [gentoo-dev] Unused profiles

[Robin H. Johnson]

On Thu, Jan 19, 2017 at 06:08:26PM +0100, Michał Górny wrote:
> default/linux/amd64/dev/32bit-userland
Is this still supported? I've mulled over making a hardened version,
specifically to support woodpecker.gentoo.org, as it's 64-bit hardware
now, but the userspace is much older, and dates from when it was 32-bit
hardware.

-- 
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation Trustee & Treasurer
E-Mail   : robb...@gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136

Re: [gentoo-dev] Pre-GLEP for review: mix-in profiles

[Magnus Granberg]

måndag 23 januari 2017 kl. 13:56:02 CET skrev  Rich Freeman:
> On Mon, Jan 23, 2017 at 4:23 AM, Michał Górny  wrote:
> > I've written a short proposal that aims to provide basic infrastructure
> > for defining mix-in profiles in Gentoo. I've tried to keep it simple,
> > and backwards compatible. The main goal is to be able to start defining
> > some mix-ins without having to reinvent the whole profile tree.
> 
> Would it actually make sense to reinvent more of the profile tree
> while we're at it?  So, have a few categories of mixins like kernel,
> arch, and some category that covers really invasive stuff like
> hardened/libc/etc?
> 
i think it would make sense to reinvent/split it up to a few categories/mixins 
like
base
arch
  abi
libc
kernel
init/udev
desktop
server
security

> Those might be 1-of-n selections.
> 
> Then we could have the fluff that sits on top and just set some rules
> about what they can do.
> 
> Part of me wonders if some of this could also fit in with the use of
> virtuals (think foo-meta virtuals but bigger).  A virtual can of
> course pull in USE dependencies, and a lot of other stuff.  We could
> have convenience virtuals that all the profiles pull in by default but
> which gets stuff like openssh out of @system.
> 
> I'm only suggesting the last bit to the extent where we see tie-ins to
> improve the initial mix-in implementation.  A lot of that is probably
> an expansion in scope, and to that extent I'm not suggesting that it
> needs to be addressed.  I just want to think about it broadly at first
> to make sure we're not missing something.

Re: [gentoo-dev] Requirements for UID/GID management

[Rich Freeman]

On Fri, Jan 27, 2017 at 3:09 PM, Michael Orlitzky  wrote:
> My first impression is that any package that doesn't care
> about its UID should default to "first available", but if that causes
> problems, then that's exactly the sort of use case I'm looking for.
>

The ones I listed before were filesystems shared by multiple hosts,
such as with nfs, containers, and chroots.  Granted, there are ways to
deal with this sort of thing, but if you want to share your /var/www
across a bunch of apache servers it would be nice if they all had the
same UID for apache.

-- 
Rich

Re: [gentoo-dev] Requirements for UID/GID management

[Michał Górny]

On Fri, 27 Jan 2017 12:54:07 -0500
Michael Orlitzky  wrote:

> We approved GLEP 27 (https://wiki.gentoo.org/wiki/GLEP:27) in 2004 but
> never implemented it. I'm wondering what are the explicit requirements
> that we have for user and group management?

I don't think GLEP 27 could be really considered approved if it hasn't
seen any action for 12 years.

> What I'm really wondering is, instead of the proposal in GLEP27, if we
> couldn't simply handle users like any other package. For example,
> net-dns/djbdns needs,
> 
>   pkg_preinst() {
> # The nofiles group is no longer provided by baselayout.
> # Share it with qmail if possible.
> enewgroup nofiles 200
> 
> enewuser dnscache -1 -1 -1 nofiles
> enewuser dnslog -1 -1 -1 nofiles
> enewuser tinydns -1 -1 -1 nofiles
>   }
> 
> Instead of that, why couldn't we have something like,
> 
>   (R)DEPEND="sys-user/dnscache
>  sys-user/dnslog
>  sys-user/tinydns"
> 
> and then in each of those packages,
> 
>   (R)DEPEND="sys-group/nofiles"

At a first glance it seems like a lot of effort for a problem that's
already well-solved on the eclass level. However, it probably makes
sense when you consider users/groups created by multiple packages.

> That satisfies most of the requirements that *I* have for user and group
> management on the system. Compared to the GLEP:
> 
>   * EUSERS + EGROUPS: replaced by (R)DEPEND.
>   * Defining Accounts: anyone can add a new package already.
>   * FEATURES=noautoaccts: use package.provided instead.
>   * Local Overrides: use an overlay.
>   * users-update: cleanup can be done with --depclean now.

Err, cleanup is never easy. You shouldn't really remove a user if it
owns any files. I guess you could abuse pkg_prerm() for that but
depclean will be terribly slow then.

> You don't really have to care what UID/GID is assigned, because each
> user/group will only be created once and referenced by name (as $PN). By
> default, we could pick the first available UID in most packages.
> I haven't thought much about the src_install implementation, but it
> couldn't be *that* hard. Maybe install a $uid file to /var/lib/portage
> somewhere to catch UID conflicts, and keep doing what user.eclass is
> doing otherwise.

Let's invent /etc/passwd.d and /etc/group.d ;-P.

-- 
Best regards,
Michał Górny



pgpd4RPtWhChI.pgp
Description: OpenPGP digital signature

Re: [gentoo-dev] REQUIRED_USE, global USE flags, user-friendliness...

[NP-Hardass]

On 01/27/2017 12:25 PM, Michael Orlitzky wrote:
> Forked from the gdbm/berkdb thread, wall of text ensues...
> 
> 
> On 01/27/2017 03:32 AM, Fabian Groffen wrote:
>>
>> You mention REQUIRED_USE should be used sparingly, I think I see your
>> reasoning, but if so, then why did we add it in the first place?
> 
> There are a few conflicting interests at play. Before REQUIRED_USE, we
> would have a bunch of checks in pkg_pretend() to test if the user's
> configuration was invalid. If it was, we could output a nice explanation
> and tell him to try again. But, bash code in pkg_pretend can't be
> understood by the package manager, and requires execution to determine
> if a package can be installed. So we got REQUIRED_USE, which fixes those
> problems, and introduces a new one: no one knows WTF to do when portage
> outputs a REQUIRED_USE error. Now you get what looks like a core dump of
> the dependency graph instead of "this package only uses one database, so
> pick either mysql or sqlite."
> 
> Both approaches have another problem: USE flag constraints on packages
> simply don't work with global USE flags. Global USE flags don't work
> that well to begin with, since the same flag means different things to
> each package (and the fact that they're global means "enable foo" is all
> we get for documentation). Regardless, when you have 100 flags enabled
> globally and start installing thousands of packages with USE
> constraints, you're eventually going to get to a point where everything
> has conflicting requirements and you need to switch to package.use to
> sort it all out.
> 
> Both pkg_pretend and REQUIRED_USE have that problem and try to solve it
> in different ways. If you don't care about machine-readability, then in
> pkg_pretend you could try to choose "the best" of two conflicting flags
> and just silently go with it. There are a lot of problems with that,
> like what happens if you need to add a conditional dependency on those
> flags (you can't change DEPEND in pkg_pretend). With REQUIRED_USE, you
> instead need to set IUSE defaults to get it to do something without user
> interaction, but the tricks that you can do with IUSE don't solve every
> REQUIRED_USE conflict. In the harder cases, you can try to figure out
> what to do on behalf of the user in the ebuild, but then you're right
> back to the same set of problems that you had with pkg_pretend, because
> the decision is being made in code and not in metadata/flags.
> 
> I think a slow migration away from global USE flags is the only way out
> of the jam. We get better USE flag docs for free, and no REQUIRED_USE
> conflicts that the user didn't cause himself. We could probably also get
> rid of a lot of IUSE defaults that serve only to undo various profile
> defaults. It would be annoying at first, but once a few critical profile
> defaults are moved into package.use, better.
> 

I might be wrong, but my suspicion is that those that advocate for
pkg_pretend over REQUIRED_USE tend to do so because of the blocking
nature of REQUIRED_USE's current implementation rather than the
construct of describing USE flag inter-dependencies itself.

So, personally, I think that we should be discussing ways of adding
interactivity to the package manager for resolution of REQUIRED_USE
conflicts rather than discussing ways to work around or remove it.  It's
a good construct, we should take advantage of it, and work to make it
more user friendly.

My initial feeling is having flags, one for interactive handling, one
for current behavior.  Interactive has two modes, like --autounmask and
--autounmask-write (and could even reuse these if possible), which does
something similar to how Debian's apt handles dependency conflicts by
calculating the alternatives and prompting the user to select a numbered
option.  So the autounmask equivalent displays the changes to the config
files and the autounmask-write equivalent writes them to the appropriate
config files.  This is just a general idea that I just came up with
right now, so I haven't put too much thought into it.  It is mostly
meant to get solutions for interactive handling discussed on the ML.

-- 
NP-Hardass



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] Requirements for UID/GID management

[Michael Orlitzky]

On 01/27/2017 02:53 PM, Rich Freeman wrote:
> 
> I'm not saying we can't have random assignment for things where it
> doesn't matter, or fall back to random assignment, but it seems rather
> silly to go to all the trouble to have blockers when it would be just
> as easy to not have a conflict in the first place...
> 
> Telling users that they can't have postfix and apache on the same box
> because nobody can be bothered to pick IDs that don't collide seems
> like an arbitrary imposition.

Agreed, blockers should be a last resort. If neither postfix nor apache
cares about its UID (they don't), then certainly somebody should change
one of them. My first impression is that any package that doesn't care
about its UID should default to "first available", but if that causes
problems, then that's exactly the sort of use case I'm looking for.

If it's a big problem, we can have devs pick an unclaimed UID and stick
with it. Or, if there are 5 users total who need the "apache" user to
have UID 80 across every machine, then it might just make more sense to
tell them to use an overlay to override sys-user/apache.

Keep in mind that currently, if the UID you want isn't available,
user.eclass will shrug and give you a different one. No one can really
rely on consistent UIDs now, but it's not fair to dismiss the idea
because maybe that's one of the things the GLEP was supposed to fix.

Re: [gentoo-dev] REQUIRED_USE, global USE flags, user-friendliness...

[james]

On 01/27/2017 12:25 PM, Michael Orlitzky wrote:

Forked from the gdbm/berkdb thread, wall of text ensues...


On 01/27/2017 03:32 AM, Fabian Groffen wrote:


You mention REQUIRED_USE should be used sparingly, I think I see your
reasoning, but if so, then why did we add it in the first place?


There are a few conflicting interests at play. Before REQUIRED_USE, we
would have a bunch of checks in pkg_pretend() to test if the user's
configuration was invalid. If it was, we could output a nice explanation
and tell him to try again. But, bash code in pkg_pretend can't be
understood by the package manager, and requires execution to determine
if a package can be installed. So we got REQUIRED_USE, which fixes those
problems, and introduces a new one: no one knows WTF to do when portage
outputs a REQUIRED_USE error. Now you get what looks like a core dump of
the dependency graph instead of "this package only uses one database, so
pick either mysql or sqlite."

Both approaches have another problem: USE flag constraints on packages
simply don't work with global USE flags. Global USE flags don't work
that well to begin with, since the same flag means different things to
each package (and the fact that they're global means "enable foo" is all
we get for documentation). Regardless, when you have 100 flags enabled
globally and start installing thousands of packages with USE
constraints, you're eventually going to get to a point where everything
has conflicting requirements and you need to switch to package.use to
sort it all out.

Both pkg_pretend and REQUIRED_USE have that problem and try to solve it
in different ways. If you don't care about machine-readability, then in
pkg_pretend you could try to choose "the best" of two conflicting flags
and just silently go with it. There are a lot of problems with that,
like what happens if you need to add a conditional dependency on those
flags (you can't change DEPEND in pkg_pretend). With REQUIRED_USE, you
instead need to set IUSE defaults to get it to do something without user
interaction, but the tricks that you can do with IUSE don't solve every
REQUIRED_USE conflict. In the harder cases, you can try to figure out
what to do on behalf of the user in the ebuild, but then you're right
back to the same set of problems that you had with pkg_pretend, because
the decision is being made in code and not in metadata/flags.

I think a slow migration away from global USE flags is the only way out
of the jam. We get better USE flag docs for free, and no REQUIRED_USE
conflicts that the user didn't cause himself. We could probably also get
rid of a lot of IUSE defaults that serve only to undo various profile
defaults. It would be annoying at first, but once a few critical profile
defaults are moved into package.use, better.


Exactly:: simplify the flags, profiles and associated constructs down to 
the bare bones. Even embedded (arm/mips/etc) builds could benefit from a 
really minimized gentoo as a starting point. Freedom to build and fork 
Gentoo, per GLEP 70 is a wonderful idea, whose time has arrived. That 
would set the stage for quickly building highly specific gentoo installs 
(VMs, Containers, Clusters or unikernel); all via a 'recipe' from the 
minimal core-sed, which  could then take a variety of forms, such as 
bare-metal builds vi IPMI, ipxe, or more traditional formularies like 
chef, puppet, ansible as well as a myriad of other possibilities.



All of the traditional gentoo installs, such as the handbook, would 
benefit from simplification  and the natural clarity that simplification 
brings. Builds of things like simple standard gentoo-cluster-CI. Nothing 
would preclude the adventuresome from building up

a traditional, highly customized install, for a specific needs such as
currently is the (handbook) practice. But additional gentoo installs 
that are quick, streamlined and soot a specific itch.



That way a user, a proxy-dev or a dev could work on any and every aspect 
of gentoo, perfectly isolated from other devs with a select group of 
testers. Each team could maintain their build formulary very easily, and 
others could provide the bugzilla feedback on that particular 
fork/custom/project. Those bugs would be assign to the project 
fork-team.  Gentoo-proper would benefit being able to 'reap' the 
benefits of a streamlined develop, deploy, test, refine, stabilize 
efforts; and folks can each have their own fiefdom, whilst still sharing 
gentoo-proper as we currently do. And bgo would be a default place to 
seach out gentoo specific issue, both on gentoo-proper and these glep-70 
projects. Of a separate bugs-engine for these projects

that is a companion to bgo, but separated, if that is better?


There are at least a half dozen folks that have done this, their own 
gentoo-derived-distro, many were mainstream devs at some point. Why not 
streamline and optimize that systems, to the  benefit of gentoo-proper? 
Gentoo proper dev testing could occur either before a proj

Re: [gentoo-dev] Requirements for UID/GID management

[Rich Freeman]

On Fri, Jan 27, 2017 at 2:35 PM, Michael Orlitzky  wrote:
> On 01/27/2017 01:52 PM, Rich Freeman wrote:
>>
>> This doesn't really seem like a problem though.  Just have a table
>> somewhere (wiki?) to track who is using what UID/GID and encode those
>> defaults into the ebuild that creates those users.
>>
>
> It should be possible to have two different users with the same UID in
> the tree, just like we can have two different packages that install the
> same file. Eventually somebody will notice a file collision, and then we
> add a blocker per usual.
>

I'm not saying we can't have random assignment for things where it
doesn't matter, or fall back to random assignment, but it seems rather
silly to go to all the trouble to have blockers when it would be just
as easy to not have a conflict in the first place.  Now, if we have
some longstanding issue from the past that might be another matter,
but what's wrong with just picking an unused ID (again, for stuff that
needs it)?

Telling users that they can't have postfix and apache on the same box
because nobody can be bothered to pick IDs that don't collide seems
like an arbitrary imposition.  Sometimes upstream creates conflicts
that are just hard to work around (same SONAME, different ABI or even
API, and so on).  And if we were talking about some binary-only
upstream package that relies on hardcoded UIDs I guess blockers might
be our only option, and users will just have to be happy that we're
able to support it at all.  However, we shouldn't be the ones creating
these kinds of conflicts.

-- 
Rich

Re: [gentoo-dev] Requirements for UID/GID management

[Gregory Woodbury]

On Fri, Jan 27, 2017 at 1:52 PM, Rich Freeman  wrote:

> On Fri, Jan 27, 2017 at 12:54 PM, Michael Orlitzky  wrote:
> >
> > You don't really have to care what UID/GID is assigned, because each
> > user/group will only be created once and referenced by name (as $PN). By
> > default, we could pick the first available UID in most packages.
>
> I might be not following correctly, but due to how filesystems/etc
> work it is probably desirable to have consistent UID/GIDs as much as
> reasonably possible.  Things like NFS, chroots, containers, and so on
> can be a bit simpler if these are consistent, because they involve one
> system having visibility into a filesystem hosted on another, and
> usually in these cases the UID/GID is what is kept constant, not the
> name.  (IMO UID/GID namespace is one of those areas where
> Linux/POSIX/etc has some weaknesses.)
>
> This doesn't really seem like a problem though.  Just have a table
> somewhere (wiki?) to track who is using what UID/GID and encode those
> defaults into the ebuild that creates those users.--
>

There should be a division of the system managed UID space:
1)  constant/consistent UID/GID for major things (portage, etc.)
2)  variable space for per package groups/users that generally don't care
  about consistency

A quick look at /etc/passwd shows that many of the system UIDs are
under 250 (portage) and a few scattered above 400. GIDs are similar,
though some are "fixed" and some are assigned going down from 999.

Some eclasses may need to be scrutinized for what behavior they are using.

-- 
G.Wolfe Woodbury
redwo...@gmail.com

Re: [gentoo-dev] Requirements for UID/GID management

[Michael Orlitzky]

On 01/27/2017 01:52 PM, Rich Freeman wrote:
> 
> This doesn't really seem like a problem though.  Just have a table
> somewhere (wiki?) to track who is using what UID/GID and encode those
> defaults into the ebuild that creates those users.
> 

It should be possible to have two different users with the same UID in
the tree, just like we can have two different packages that install the
same file. Eventually somebody will notice a file collision, and then we
add a blocker per usual.

Re: [gentoo-dev] Requirements for UID/GID management

[Rich Freeman]

On Fri, Jan 27, 2017 at 12:54 PM, Michael Orlitzky  wrote:
>
> You don't really have to care what UID/GID is assigned, because each
> user/group will only be created once and referenced by name (as $PN). By
> default, we could pick the first available UID in most packages.

I might be not following correctly, but due to how filesystems/etc
work it is probably desirable to have consistent UID/GIDs as much as
reasonably possible.  Things like NFS, chroots, containers, and so on
can be a bit simpler if these are consistent, because they involve one
system having visibility into a filesystem hosted on another, and
usually in these cases the UID/GID is what is kept constant, not the
name.  (IMO UID/GID namespace is one of those areas where
Linux/POSIX/etc has some weaknesses.)

This doesn't really seem like a problem though.  Just have a table
somewhere (wiki?) to track who is using what UID/GID and encode those
defaults into the ebuild that creates those users.

Overall I like your proposal.

-- 
Rich

Re: [gentoo-dev] berkdb and gdbm in global USE defaults

[Matt Turner]

On Fri, Jan 27, 2017 at 8:22 AM, Mike Gilbert  wrote:
> On Fri, Jan 27, 2017 at 2:54 AM, Mart Raudsepp  wrote:
>> Then there is no need to think about what is enabled globally or not.
>> Point being, use REQUIRED_USE sparingly, and rarely a good idea to
>> block things with common global USE flags, or demand a local USE flag
>> based on a default enabled global USE flag without locally USE
>> defaulting that global flag too - and other such cases.
>
> I didn't really mean for this to turn into a thread about the merits
> of REQUIRED_USE; in hindsight I should have left out that first
> sentence.
>
> Regardless of the REQUIRED_USE discussion, I don't think it makes
> sense to have berkdb and gdbm in USE in make.defaults. I would like to
> move them to IUSE defaults or package.use if necessary.

I think you should feel free to proceed with such a change.

FWIW, disabling these USE flags (and fortran) are among the first
changes I made to a new make.conf.

Re: [gentoo-dev] Requirements for UID/GID management

[Alexis Ballier]

On Fri, 27 Jan 2017 12:54:07 -0500
Michael Orlitzky  wrote:

> That satisfies most of the requirements that *I* have for user and
> group management on the system. Compared to the GLEP:
> 
>   * EUSERS + EGROUPS: replaced by (R)DEPEND.
>   * Defining Accounts: anyone can add a new package already.
>   * FEATURES=noautoaccts: use package.provided instead.
>   * Local Overrides: use an overlay.
>   * users-update: cleanup can be done with --depclean now.

+1

except package.provided which will probably be a mess to handle, but I
dont see why disabling it is a good idea: I'm assuming package will
misbehave at runtime if the user/group is missing

note that i dont think it fixes ROOT!=/ missing users though but
probably paves the way for it

[gentoo-dev] Requirements for UID/GID management

[Michael Orlitzky]

We approved GLEP 27 (https://wiki.gentoo.org/wiki/GLEP:27) in 2004 but
never implemented it. I'm wondering what are the explicit requirements
that we have for user and group management?

What I'm really wondering is, instead of the proposal in GLEP27, if we
couldn't simply handle users like any other package. For example,
net-dns/djbdns needs,

  pkg_preinst() {
# The nofiles group is no longer provided by baselayout.
# Share it with qmail if possible.
enewgroup nofiles 200

enewuser dnscache -1 -1 -1 nofiles
enewuser dnslog -1 -1 -1 nofiles
enewuser tinydns -1 -1 -1 nofiles
  }

Instead of that, why couldn't we have something like,

  (R)DEPEND="sys-user/dnscache
 sys-user/dnslog
 sys-user/tinydns"

and then in each of those packages,

  (R)DEPEND="sys-group/nofiles"

That satisfies most of the requirements that *I* have for user and group
management on the system. Compared to the GLEP:

  * EUSERS + EGROUPS: replaced by (R)DEPEND.
  * Defining Accounts: anyone can add a new package already.
  * FEATURES=noautoaccts: use package.provided instead.
  * Local Overrides: use an overlay.
  * users-update: cleanup can be done with --depclean now.

You don't really have to care what UID/GID is assigned, because each
user/group will only be created once and referenced by name (as $PN). By
default, we could pick the first available UID in most packages.
I haven't thought much about the src_install implementation, but it
couldn't be *that* hard. Maybe install a $uid file to /var/lib/portage
somewhere to catch UID conflicts, and keep doing what user.eclass is
doing otherwise.

There isn't a ton of motivation in that GLEP, so I'm not sure what other
use cases I might have overlooked.

[gentoo-dev] REQUIRED_USE, global USE flags, user-friendliness...

[Michael Orlitzky]

Forked from the gdbm/berkdb thread, wall of text ensues...


On 01/27/2017 03:32 AM, Fabian Groffen wrote:
>
> You mention REQUIRED_USE should be used sparingly, I think I see your
> reasoning, but if so, then why did we add it in the first place?

There are a few conflicting interests at play. Before REQUIRED_USE, we
would have a bunch of checks in pkg_pretend() to test if the user's
configuration was invalid. If it was, we could output a nice explanation
and tell him to try again. But, bash code in pkg_pretend can't be
understood by the package manager, and requires execution to determine
if a package can be installed. So we got REQUIRED_USE, which fixes those
problems, and introduces a new one: no one knows WTF to do when portage
outputs a REQUIRED_USE error. Now you get what looks like a core dump of
the dependency graph instead of "this package only uses one database, so
pick either mysql or sqlite."

Both approaches have another problem: USE flag constraints on packages
simply don't work with global USE flags. Global USE flags don't work
that well to begin with, since the same flag means different things to
each package (and the fact that they're global means "enable foo" is all
we get for documentation). Regardless, when you have 100 flags enabled
globally and start installing thousands of packages with USE
constraints, you're eventually going to get to a point where everything
has conflicting requirements and you need to switch to package.use to
sort it all out.

Both pkg_pretend and REQUIRED_USE have that problem and try to solve it
in different ways. If you don't care about machine-readability, then in
pkg_pretend you could try to choose "the best" of two conflicting flags
and just silently go with it. There are a lot of problems with that,
like what happens if you need to add a conditional dependency on those
flags (you can't change DEPEND in pkg_pretend). With REQUIRED_USE, you
instead need to set IUSE defaults to get it to do something without user
interaction, but the tricks that you can do with IUSE don't solve every
REQUIRED_USE conflict. In the harder cases, you can try to figure out
what to do on behalf of the user in the ebuild, but then you're right
back to the same set of problems that you had with pkg_pretend, because
the decision is being made in code and not in metadata/flags.

I think a slow migration away from global USE flags is the only way out
of the jam. We get better USE flag docs for free, and no REQUIRED_USE
conflicts that the user didn't cause himself. We could probably also get
rid of a lot of IUSE defaults that serve only to undo various profile
defaults. It would be annoying at first, but once a few critical profile
defaults are moved into package.use, better.

Re: [gentoo-dev] Fwd: Cron /usr/local/bin/pidlock -s rsync-gen /bin/bash /usr/local/bin/mastermirror/rsync-gen.sh

[james]

On 01/27/2017 07:41 AM, Rich Freeman wrote:

On Fri, Jan 27, 2017 at 7:06 AM, Dirkjan Ochtman  wrote:


On Fri, Jan 27, 2017 at 8:26 AM, Michał Górny  wrote:

I should point out that:

1) CI is detecting this kind of issues much faster than you are,
and reporting them both to the committer and to a *dedicated* mailing
list, so your mail is completely redundant and delayed.


That sounds like a somewhat better solution, although sometimes it can
make sense to send email to where the developers are already, rather
than putting the onus on them to join a separate mailing list.



I don't think the idea is to put the onus on people to join a separate
list so much as to give people the freedom to NOT join that list.


Hey thanks guys for pointing out that list concerned with CI.


Why is it necessary to notify every developer that somebody has not
run repoman?  For everybody who does what they're supposed to do,
there is no lesson to learn, and it is just noise.

For people interested in building tree-wide QA tools or who are
interested in overall trends, then they can mine the list archives.
If they have significant observations they can always post here or on
planet or whatever, and that would have a much higher S/N ratio.


2) Spamming the developer mailing list is completely unprofessional
here. If you are unhappy about those mails, just disable them, and stop
blaming people for your misery. Trying to prove others incompetent
helps nobody.


Come on... I think it's fair game to share news about people breaking
things on the gentoo-dev mailing list. Naming & shaming can be useful
sometimes.


I think naming and shaming is a short-term game.  It might have
immediate effects, but it tends to create a culture where nobody wants
to get involved, because they don't want to be the person getting
named and shamed.

We should certainly provide information to people about their errors
so that they can fix them.  We should certainly have this information
available to people making tools that can help people avoid errors,
since error is human nature.  There is no need to hide this
information, but we shouldn't have a culture where we're making it an
emphasis so that we can all go around pointing fingers.


H. Sure I agree that folks with aspiring skills, particular to 
gentoo, might need a bit of coddling and encouragement at times, as 
opposed to the back of the hand sort of management. Me, I'm rather 
thick-skinned, so a good public 'smacking' only means a dev cares and 
has higher expectations from his comrade? But, what comes around, goes 
around for this old fart. Kids now-a-days, not so much.



I have avoided the gentoo-CI project (building my own server-cluster) 
pretty much for the reason to avoid this sort of thread focused on 
myself (an ounce of prevention). My take is that this thread is 
justification that the gentoo-CI project needs more documentation, 
participation and dissemination so as to make it a community tool for 
devs and strong users alike. In fact, could not repoman be an option for 
a gentoo-cluster-CI configuration?




If somebody is a consistent problem and is impervious to attempts to
work with them (whatever the ultimate reason), we don't need to make
them a social pariah until they decide to quit.  We just need to have
QA revoke their commit rights.

I'm a little concerned that stuff like this starts to end up working
like collective punishment.  Fred over here broke the tree, so nobody
gets to have desert or recess today; you all know what to do with Fred
when he's looking to sit next to somebody at lunch and when the bus
drops you off at home later today.

I don't think that was the original motivation; I think frustration
with this being a frequent problem is more the issue and is quite
understandable.  I just don't think this is the right solution.



I have posted several times (I think) about the Gentoo CI being tuned 
into a straight forward project, where anyone with a few systems (or VMs 
or containers) can build a gentoo cluster and run it local. Folks, 
whether dev or experience user, could then run their own 
"gentoo-cluster-CI" leveraging the work of the devs involved and then 
selecting packages that they care most about. Automating much of what a 
proxy-dev does noting the results and studying the components, would 
make an excellent training system and help infuse folks with gentoo 
expertise into the wider software intensive industries. Thus it would 
greatly enhance gentoo's appeal and tend to subsequently attract younger 
folks with aspirations, imho.



This thread only reinforces that this idea, to make a gentoo-cluster-CI 
better documented and available for all of us with an interest. Granted, 
I'm still waiting a bit more to attempt that sort effort; but I do 
believe there would be strong interest, particularly if packaged up as 
stage-4 for the user base, proxy folks and those with not quite dev 
level skills. I also realize the gentoo-CI project may 

Re: [gentoo-dev] berkdb and gdbm in global USE defaults

[Mart Raudsepp]

Ühel kenal päeval, R, 27.01.2017 kell 11:22, kirjutas Mike Gilbert:
> On Fri, Jan 27, 2017 at 2:54 AM, Mart Raudsepp 
> wrote:
> > Then there is no need to think about what is enabled globally or
> > not.
> > Point being, use REQUIRED_USE sparingly, and rarely a good idea to
> > block things with common global USE flags, or demand a local USE
> > flag
> > based on a default enabled global USE flag without locally USE
> > defaulting that global flag too - and other such cases.
> 
> I didn't really mean for this to turn into a thread about the merits
> of REQUIRED_USE; in hindsight I should have left out that first
> sentence.
> 
> Regardless of the REQUIRED_USE discussion, I don't think it makes
> sense to have berkdb and gdbm in USE in make.defaults. I would like
> to
> move them to IUSE defaults or package.use if necessary.

That sounds great to me when done with proper care as an interim
solution.
If they aren't both global enabled, the concerns about REQUIRED_USE
here are lessened.


Mart

Re: [gentoo-dev] berkdb and gdbm in global USE defaults

[Michael Orlitzky]

On 01/26/2017 10:33 PM, Mike Gilbert wrote:
> 
> Is there any reason to have these USE flags enabled globally?

They are quite uncritical.


> These USE seem pretty package-specific in scope. On my system, they
> are used by around a dozen of 1000+ installed packages. I think it
> might make sense to migrate them to appropriate IUSE defaults, or
> leave them disabled where they do not provide critical functionality.
> 

+1

Re: [gentoo-dev] berkdb and gdbm in global USE defaults

[Kristian Fiskerstrand]

On 01/27/2017 05:46 PM, William Hubbs wrote:
>> It breaks the highly sought after "Gentoo is about choice" mantra.
>> In this case, choice to not care and have the best chosen for me.
> Actually it doesn't. In this case the user should make a choice rather
> than the maintainer silently making a choice behind their back.

There is an argument to be made for sane defaults in profiles as well as
default IUSE specification in this though to provide a better user
experience, but the underlying mechanism should be explicit.

-- 
Kristian Fiskerstrand
OpenPGP keyblock reachable at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] berkdb and gdbm in global USE defaults

[William Hubbs]

On Fri, Jan 27, 2017 at 06:27:09PM +0200, Mart Raudsepp wrote:
> Ühel kenal päeval, R, 27.01.2017 kell 13:08, kirjutas Kristian
> Fiskerstrand:
> > On 01/27/2017 01:01 PM, Dirkjan Ochtman wrote:
> > > On Fri, Jan 27, 2017 at 8:54 AM, Mart Raudsepp 
> > > wrote:
> > > > Ühel kenal päeval, N, 26.01.2017 kell 22:33, kirjutas Mike
> > > > Gilbert:
> > > > > I recently ran into a REQUIRED_USE constraint that required I
> > > > > select
> > > > > between berkdb and gdbm for an email client.
> > > > 
> > > > There shouldn't be a REQUIRED_USE constraint that forces you to
> > > > select
> > > > one or the other. The maintainer should be giving the choice of
> > > > both,
> > > > but if only one can be chosen, the maintainer should make the
> > > > choice
> > > > for you by preferring one of them. Likely gdbm, given berkdb
> > > > licensing
> > > > saga.
> > > 
> > > I'm not sure this makes sense to me. If the package will actually
> > > select one implementation out of a set, it makes sense to me that
> > > the
> > > maintainer for that package makes that choice explicit towards the
> > > user. In that case, setting REQUIRED_USE accordingly seems exactly
> > > right. The maintainer should set a good default, but if the user's
> > > USE
> > > settings are inconclusive in getting to the choice of
> > > implementation,
> > > it's better to whine explicitly than try to guess implicitly what
> > > the
> > > user wanted.
> > 
> > I tend to agree with this sentiment, explicit over implicit behavior
> > ensures better debugging ability and security considerations.

I agree, I prefer explicit strongly over implicit.

> It breaks the highly sought after "Gentoo is about choice" mantra.
> In this case, choice to not care and have the best chosen for me.

Actually it doesn't. In this case the user should make a choice rather
than the maintainer silently making a choice behind their back.

William


signature.asc
Description: Digital signature

Re: [gentoo-dev] berkdb and gdbm in global USE defaults

[Mart Raudsepp]

Ühel kenal päeval, R, 27.01.2017 kell 13:08, kirjutas Kristian
Fiskerstrand:
> On 01/27/2017 01:01 PM, Dirkjan Ochtman wrote:
> > On Fri, Jan 27, 2017 at 8:54 AM, Mart Raudsepp 
> > wrote:
> > > Ühel kenal päeval, N, 26.01.2017 kell 22:33, kirjutas Mike
> > > Gilbert:
> > > > I recently ran into a REQUIRED_USE constraint that required I
> > > > select
> > > > between berkdb and gdbm for an email client.
> > > 
> > > There shouldn't be a REQUIRED_USE constraint that forces you to
> > > select
> > > one or the other. The maintainer should be giving the choice of
> > > both,
> > > but if only one can be chosen, the maintainer should make the
> > > choice
> > > for you by preferring one of them. Likely gdbm, given berkdb
> > > licensing
> > > saga.
> > 
> > I'm not sure this makes sense to me. If the package will actually
> > select one implementation out of a set, it makes sense to me that
> > the
> > maintainer for that package makes that choice explicit towards the
> > user. In that case, setting REQUIRED_USE accordingly seems exactly
> > right. The maintainer should set a good default, but if the user's
> > USE
> > settings are inconclusive in getting to the choice of
> > implementation,
> > it's better to whine explicitly than try to guess implicitly what
> > the
> > user wanted.
> 
> I tend to agree with this sentiment, explicit over implicit behavior
> ensures better debugging ability and security considerations.
> 

It breaks the highly sought after "Gentoo is about choice" mantra.
In this case, choice to not care and have the best chosen for me.

Re: [gentoo-dev] berkdb and gdbm in global USE defaults

[Mike Gilbert]

On Fri, Jan 27, 2017 at 2:54 AM, Mart Raudsepp  wrote:
> Then there is no need to think about what is enabled globally or not.
> Point being, use REQUIRED_USE sparingly, and rarely a good idea to
> block things with common global USE flags, or demand a local USE flag
> based on a default enabled global USE flag without locally USE
> defaulting that global flag too - and other such cases.

I didn't really mean for this to turn into a thread about the merits
of REQUIRED_USE; in hindsight I should have left out that first
sentence.

Regardless of the REQUIRED_USE discussion, I don't think it makes
sense to have berkdb and gdbm in USE in make.defaults. I would like to
move them to IUSE defaults or package.use if necessary.

Re: [gentoo-dev] Fwd: Cron /usr/local/bin/pidlock -s rsync-gen /bin/bash /usr/local/bin/mastermirror/rsync-gen.sh

[M. J. Everitt]

On 27/01/17 12:41, Rich Freeman wrote:
> 
> I'm a little concerned that stuff like this starts to end up working
> like collective punishment.  Fred over here broke the tree, so nobody
> gets to have desert or recess today; you all know what to do with Fred
> when he's looking to sit next to somebody at lunch and when the bus
> drops you off at home later today.
>
>
Purely in the interests of pedantry (!) ..

I don't think anyone truly wants a /_desert_/ Rich, but perhaps you
meant a /*dessert*/ ?!

:]



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] berkdb and gdbm in global USE defaults

[Fabian Groffen]

On 27-01-2017 13:08:41 +0100, Kristian Fiskerstrand wrote:
> > On Fri, Jan 27, 2017 at 9:32 AM, Fabian Groffen  wrote:
> >> Replying here because I think said email client is the one I recently
> >> added REQUIRED_USE constraints for.
> >>
> >> Reason I added it is because it greatly simplified the ebuild: it's not
> >> just bdb and gdbm, but also tokyocabinet, qdbm and lmdb, with as result
> >> a lot of if-else-casing which implemented the implicit defaults before.
> >> I didn't realise changing this to REQUIRED_USE resulted in a conflict on
> >> default profiles, because I (obviously) have a package.use entry for the
> >> package.
> > I don't see Mike saying you got it wrong here. Reading your email, I
> > think you did the right thing.
> 
> Yup

That blurb was more directed at Mart ;)  I think I just explained why I
did what I did.  The scenario in older ebuilds (without REQUIRED_USE)
was basically the scenario that Mart suggested to be perferable over the
new REQUIRED_USE scenario.

I'm not looking for wrong/right.  I'm looking for concensus on this
topic, then I will likely change the ebuild to match concensus.

Fabian


-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: Digital signature

Re: [gentoo-dev] Fwd: Cron /usr/local/bin/pidlock -s rsync-gen /bin/bash /usr/local/bin/mastermirror/rsync-gen.sh

[Rich Freeman]

On Fri, Jan 27, 2017 at 7:06 AM, Dirkjan Ochtman  wrote:
>
> On Fri, Jan 27, 2017 at 8:26 AM, Michał Górny  wrote:
>> I should point out that:
>>
>> 1) CI is detecting this kind of issues much faster than you are,
>> and reporting them both to the committer and to a *dedicated* mailing
>> list, so your mail is completely redundant and delayed.
>
> That sounds like a somewhat better solution, although sometimes it can
> make sense to send email to where the developers are already, rather
> than putting the onus on them to join a separate mailing list.
>

I don't think the idea is to put the onus on people to join a separate
list so much as to give people the freedom to NOT join that list.

Why is it necessary to notify every developer that somebody has not
run repoman?  For everybody who does what they're supposed to do,
there is no lesson to learn, and it is just noise.

For people interested in building tree-wide QA tools or who are
interested in overall trends, then they can mine the list archives.
If they have significant observations they can always post here or on
planet or whatever, and that would have a much higher S/N ratio.

>> 2) Spamming the developer mailing list is completely unprofessional
>> here. If you are unhappy about those mails, just disable them, and stop
>> blaming people for your misery. Trying to prove others incompetent
>> helps nobody.
>
> Come on... I think it's fair game to share news about people breaking
> things on the gentoo-dev mailing list. Naming & shaming can be useful
> sometimes.

I think naming and shaming is a short-term game.  It might have
immediate effects, but it tends to create a culture where nobody wants
to get involved, because they don't want to be the person getting
named and shamed.

We should certainly provide information to people about their errors
so that they can fix them.  We should certainly have this information
available to people making tools that can help people avoid errors,
since error is human nature.  There is no need to hide this
information, but we shouldn't have a culture where we're making it an
emphasis so that we can all go around pointing fingers.

If somebody is a consistent problem and is impervious to attempts to
work with them (whatever the ultimate reason), we don't need to make
them a social pariah until they decide to quit.  We just need to have
QA revoke their commit rights.

I'm a little concerned that stuff like this starts to end up working
like collective punishment.  Fred over here broke the tree, so nobody
gets to have desert or recess today; you all know what to do with Fred
when he's looking to sit next to somebody at lunch and when the bus
drops you off at home later today.

I don't think that was the original motivation; I think frustration
with this being a frequent problem is more the issue and is quite
understandable.  I just don't think this is the right solution.

-- 
Rich

Re: [gentoo-dev] berkdb and gdbm in global USE defaults

[Kristian Fiskerstrand]

On 01/27/2017 01:01 PM, Dirkjan Ochtman wrote:
> On Fri, Jan 27, 2017 at 8:54 AM, Mart Raudsepp  wrote:
>> Ühel kenal päeval, N, 26.01.2017 kell 22:33, kirjutas Mike Gilbert:
>>> I recently ran into a REQUIRED_USE constraint that required I select
>>> between berkdb and gdbm for an email client.
>> There shouldn't be a REQUIRED_USE constraint that forces you to select
>> one or the other. The maintainer should be giving the choice of both,
>> but if only one can be chosen, the maintainer should make the choice
>> for you by preferring one of them. Likely gdbm, given berkdb licensing
>> saga.
> I'm not sure this makes sense to me. If the package will actually
> select one implementation out of a set, it makes sense to me that the
> maintainer for that package makes that choice explicit towards the
> user. In that case, setting REQUIRED_USE accordingly seems exactly
> right. The maintainer should set a good default, but if the user's USE
> settings are inconclusive in getting to the choice of implementation,
> it's better to whine explicitly than try to guess implicitly what the
> user wanted.

I tend to agree with this sentiment, explicit over implicit behavior
ensures better debugging ability and security considerations.

> 
> On Fri, Jan 27, 2017 at 9:32 AM, Fabian Groffen  wrote:
>> Replying here because I think said email client is the one I recently
>> added REQUIRED_USE constraints for.
>>
>> Reason I added it is because it greatly simplified the ebuild: it's not
>> just bdb and gdbm, but also tokyocabinet, qdbm and lmdb, with as result
>> a lot of if-else-casing which implemented the implicit defaults before.
>> I didn't realise changing this to REQUIRED_USE resulted in a conflict on
>> default profiles, because I (obviously) have a package.use entry for the
>> package.
> I don't see Mike saying you got it wrong here. Reading your email, I
> think you did the right thing.

Yup

-- 
Kristian Fiskerstrand
OpenPGP keyblock reachable at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] Fwd: Cron /usr/local/bin/pidlock -s rsync-gen /bin/bash /usr/local/bin/mastermirror/rsync-gen.sh

[Dirkjan Ochtman]

Wow, Michał, this email comes off as pretty harsh to me... Is that
really necessary?

On Fri, Jan 27, 2017 at 8:26 AM, Michał Górny  wrote:
> I should point out that:
>
> 1) CI is detecting this kind of issues much faster than you are,
> and reporting them both to the committer and to a *dedicated* mailing
> list, so your mail is completely redundant and delayed.

That sounds like a somewhat better solution, although sometimes it can
make sense to send email to where the developers are already, rather
than putting the onus on them to join a separate mailing list.

> 2) Spamming the developer mailing list is completely unprofessional
> here. If you are unhappy about those mails, just disable them, and stop
> blaming people for your misery. Trying to prove others incompetent
> helps nobody.

Come on... I think it's fair game to share news about people breaking
things on the gentoo-dev mailing list. Naming & shaming can be useful
sometimes. And even if you wanted to make this point, you should have
made it more constructively.

> That said, if you forward yet another mail with the same purpose,
> I will file a formal complaint at ComRel and request suspending your
> mailing list access.

This is completely out of line for me. It seems to me that Doug has
the best intentions. If you disagree with his approach, can we not
have a civilized discussion about it rather than going straight into
attack mode?

Cheers,

Dirkjan

Re: [gentoo-dev] berkdb and gdbm in global USE defaults

[Dirkjan Ochtman]

On Fri, Jan 27, 2017 at 4:33 AM, Mike Gilbert  wrote:
> Looking through our profiles, I see we have both berkdb and gdbm
> enabled "globally".
>
> default/linux/make.defaults:USE="berkdb crypt ipv6 ncurses nls pam
> readline ssl tcpd zlib"
> releases/make.defaults:USE="acl gdbm nptl unicode"
>
> Is there any reason to have these USE flags enabled globally?

Good question... I already disable them, I think, as it doesn't really
make sense from my perspective to enable them globally. I think
letting packages set their own defaults with IUSE would probably be a
better solution.

On Fri, Jan 27, 2017 at 8:54 AM, Mart Raudsepp  wrote:
> Ühel kenal päeval, N, 26.01.2017 kell 22:33, kirjutas Mike Gilbert:
>> I recently ran into a REQUIRED_USE constraint that required I select
>> between berkdb and gdbm for an email client.
>
> There shouldn't be a REQUIRED_USE constraint that forces you to select
> one or the other. The maintainer should be giving the choice of both,
> but if only one can be chosen, the maintainer should make the choice
> for you by preferring one of them. Likely gdbm, given berkdb licensing
> saga.

I'm not sure this makes sense to me. If the package will actually
select one implementation out of a set, it makes sense to me that the
maintainer for that package makes that choice explicit towards the
user. In that case, setting REQUIRED_USE accordingly seems exactly
right. The maintainer should set a good default, but if the user's USE
settings are inconclusive in getting to the choice of implementation,
it's better to whine explicitly than try to guess implicitly what the
user wanted.

On Fri, Jan 27, 2017 at 9:32 AM, Fabian Groffen  wrote:
> Replying here because I think said email client is the one I recently
> added REQUIRED_USE constraints for.
>
> Reason I added it is because it greatly simplified the ebuild: it's not
> just bdb and gdbm, but also tokyocabinet, qdbm and lmdb, with as result
> a lot of if-else-casing which implemented the implicit defaults before.
> I didn't realise changing this to REQUIRED_USE resulted in a conflict on
> default profiles, because I (obviously) have a package.use entry for the
> package.

I don't see Mike saying you got it wrong here. Reading your email, I
think you did the right thing.

Cheers,

Dirkjan

Re: [gentoo-dev] berkdb and gdbm in global USE defaults

[Mart Raudsepp]

Ühel kenal päeval, R, 27.01.2017 kell 13:16, kirjutas Mart Raudsepp:
> If anything, I think this is a suggestion that *maybe* we should a
> > way to
> > specify a mechanism for allowing a default to be chosen from a
> > mutually
> > exclusive set, and then:
> 
> Sure, I have some thoughts for this and a rough draft, at least in my
> head :)
> I don't have it as a priority to sketch it out well alone, but if
> someone is honestly interested, I could braindump my ideas in
> realtime
> medium. Or someone thinks of them themselves :)

I wrote a bit about it in my USE_EXPAND=gui proposal.
https://archives.gentoo.org/gentoo-dev/message/cc39dfafcb33ac0d345c26c0
475723b8

That's the copy of my piratepad draft on the subject. In the end
there's a section on it under future ideas.

The initial thread start was
https://archives.gentoo.org/gentoo-dev/message/a2de58fcbf9f214362cd5fdb5f67e6b2
These were the only 2 e-mails in that thread, everyone was busy
bikeshedding my USE=gui proposal at
https://archives.gentoo.org/gentoo-dev/message/eecad370248118c474a0d819fa7f3576
with some of the multi-choice aspects coming up there.

Any future complaints by QA or otherwise about versioned gtk USE flags
and whatnot get pointed to these, to pick up and actually go forward
with this, not just complain. Anyhow, that's off-topic for the
REQUIRED_USE stuff here.


Mart

Re: [gentoo-dev] berkdb and gdbm in global USE defaults

[Mart Raudsepp]

Ühel kenal päeval, R, 27.01.2017 kell 23:58, kirjutas Kent Fredric:
> On Fri, 27 Jan 2017 09:32:23 +0100
> Fabian Groffen  wrote:
> 
> > I'm interested to hear how other people feel about this.
> 
> Yeah. Pretty much my reaction to 
> 
> Mart Raudsepp  wrote:
> 
> > The maintainer should be giving the choice of both,
> > but if only one can be chosen, the maintainer should make the
> > choice
> > for you by preferring one of them. Likely gdbm, given berkdb
> > licensing
> > saga.
> 
> Brought the same question to me:
> 
> If the design is intended to force your hand when you have both, what
> is indeed
> the point of a REQUIRED_USE feature at all?

It can be very useful in some cases, especially when these cases
involve local USE flags in a way that the errors come after enabling
something locally in an unsuitable way.
But yes, ideally the package manager would have a clue about what
happened for the cases like the one in question, but REQUIRED_USE
provided a faster solution to some of the problems that could be
implemented in package managers in a reasonable time for the EAPI this
was introduced in.
We could work on top of this in a future EAPI.

> If "choose a useflag for the user" is something that is happening, it
> should
> at least be *visible* to the user that this is happening, not being a
> silent
> decision that didn't allow the user to have any say in the matter.
> 
> What if the feature you chose instead, was contrary to the one they
> wanted?
> 
> If anything, I think this is a suggestion that *maybe* we should a
> way to
> specify a mechanism for allowing a default to be chosen from a
> mutually
> exclusive set, and then:

Sure, I have some thoughts for this and a rough draft, at least in my
head :)
I don't have it as a priority to sketch it out well alone, but if
someone is honestly interested, I could braindump my ideas in realtime
medium. Or someone thinks of them themselves :)

> a. Inform the user via pretend output that this automatic conflict
> reduction
>    has been performed
> 
> b. Define a portage option that disables automatic conflict
> resolution for
>    required USE, so users who hate (a) can turn it off.
> 
> 
> But as it stands, Mart's suggestion of "Hey, just don't use required
> use,
> decide for the user" stands essentially as a regression against
> portage itself.

Re: [gentoo-dev] berkdb and gdbm in global USE defaults

[Kent Fredric]

On Fri, 27 Jan 2017 09:32:23 +0100
Fabian Groffen  wrote:

> I'm interested to hear how other people feel about this.

Yeah. Pretty much my reaction to 

Mart Raudsepp  wrote:

> The maintainer should be giving the choice of both,
> but if only one can be chosen, the maintainer should make the choice
> for you by preferring one of them. Likely gdbm, given berkdb licensing
> saga.

Brought the same question to me:

If the design is intended to force your hand when you have both, what is indeed
the point of a REQUIRED_USE feature at all?

If "choose a useflag for the user" is something that is happening, it should
at least be *visible* to the user that this is happening, not being a silent
decision that didn't allow the user to have any say in the matter.

What if the feature you chose instead, was contrary to the one they wanted?

If anything, I think this is a suggestion that *maybe* we should a way to
specify a mechanism for allowing a default to be chosen from a mutually
exclusive set, and then:

a. Inform the user via pretend output that this automatic conflict reduction
   has been performed

b. Define a portage option that disables automatic conflict resolution for
   required USE, so users who hate (a) can turn it off.


But as it stands, Mart's suggestion of "Hey, just don't use required use,
decide for the user" stands essentially as a regression against portage itself.


pgpmoqwpQFHxO.pgp
Description: OpenPGP digital signature

Re: [gentoo-dev] berkdb and gdbm in global USE defaults

[Fabian Groffen]

Replying here because I think said email client is the one I recently
added REQUIRED_USE constraints for.

Reason I added it is because it greatly simplified the ebuild: it's not
just bdb and gdbm, but also tokyocabinet, qdbm and lmdb, with as result
a lot of if-else-casing which implemented the implicit defaults before.
I didn't realise changing this to REQUIRED_USE resulted in a conflict on
default profiles, because I (obviously) have a package.use entry for the
package.

You mention REQUIRED_USE should be used sparingly, I think I see your
reasoning, but if so, then why did we add it in the first place?  Since
the ebuild will only use one of the db backends, when multiple are
selected, the package manager will falsely think both are in use (and
trigger rebuilds, etc.).  Isn't the point to express the actual
situation as correctly as possible for the PM to do a better job?
I also like the ebuild being way less convoluted, but I can overcome
that is necessary.

I'm interested to hear how other people feel about this.

Thanks,
Fabian


On 27-01-2017 09:54:00 +0200, Mart Raudsepp wrote:
> Ühel kenal päeval, N, 26.01.2017 kell 22:33, kirjutas Mike Gilbert:
> > I recently ran into a REQUIRED_USE constraint that required I select
> > between berkdb and gdbm for an email client.
> 
> There shouldn't be a REQUIRED_USE constraint that forces you to select
> one or the other. The maintainer should be giving the choice of both,
> but if only one can be chosen, the maintainer should make the choice
> for you by preferring one of them. Likely gdbm, given berkdb licensing
> saga.
> Though I guess this is a little bit more problematic when that DB is
> long living, but I think it should still work good enough with this
> guideline.
> 
> Then there is no need to think about what is enabled globally or not.
> Point being, use REQUIRED_USE sparingly, and rarely a good idea to
> block things with common global USE flags, or demand a local USE flag
> based on a default enabled global USE flag without locally USE
> defaulting that global flag too - and other such cases.
> 
> I'd talk to the maintainer(s) of such package(s) via bugzilla or other
> means and discuss such REQUIRED_USE potential overuse.
> 
> > Looking through our profiles, I see we have both berkdb and gdbm
> > enabled "globally".
> > 
> > default/linux/make.defaults:USE="berkdb crypt ipv6 ncurses nls pam
> > readline ssl tcpd zlib"
> > releases/make.defaults:USE="acl gdbm nptl unicode"
> > 
> > Is there any reason to have these USE flags enabled globally?
> > 
> > These USE seem pretty package-specific in scope. On my system, they
> > are used by around a dozen of 1000+ installed packages. I think it
> > might make sense to migrate them to appropriate IUSE defaults, or
> > leave them disabled where they do not provide critical functionality.
> 
> 

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: Digital signature