Re: [gentoo-dev] Should we join the which hunt?

2022-05-13 Thread Michael Orlitzky
On Fri, 2022-05-13 at 11:44 -0400, Mike Gilbert wrote:
> 
> "which" is a built-in command in bash, but not in dash. For most
> users, /bin/sh points at bash and I don't expect to see much breakage
> when /usr/bin/which is removed. The bug reports will come from people
> who like pain and run their systems with /bin/sh pointed at dash.
> 

Debian did that first, too. If your oldest and most boring friend jumps
off a bridge, it's fine. And it makes each ./configure run like 15%
faster!

Who is Gentoo for if not for people who would sacrifice essential
stability for a little temporary performance, and deserve neither?




Re: [gentoo-dev] Should we join the which hunt?

2022-05-13 Thread Michael Orlitzky
On Fri, 2022-05-13 at 09:11 +0200, Ulrich Mueller wrote:
> 
> So, should we join the "which hunt", with the goal of removing
> sys-apps/which from the system set and from stage1?

Yes, although I would suggest "command -v" as a POSIX replacement that
can be sent upstream. The "type" utility is also standard, but the "-p"
flag is not, so "type -p" creates some pointless bashisms. Both are
built-in to bash so there's no performance penalty in ebuilds either
way.




Re: [gentoo-dev] [PATCH] metadata/install-qa-check.d: add 60libtool-la (check for unnecessary .la files)

2022-04-15 Thread Michael Orlitzky
On Fri, 2022-04-15 at 09:40 +0100, Sam James wrote:
> +
> +   if grep -q "dev-libs/libltdl" <<<${RDEPEND}; then
> +   # Nothing to do here
> +   return
> +   fi

We should probably have delimiters around that atom to future-proof it
against (say) dev-libs/libltdl2. Would "has" suffice?



Re: [gentoo-dev] Feature request: auto-CC for bugs modified via commit tags

2022-01-25 Thread Michael Orlitzky
On Tue, 2022-01-25 at 21:59 +0100, Agostino Sarubbo wrote:
> 
> - If you are CC'ed by the hook and you are part of the alias that is the 
> assignee of the bug, 
> you will receive two emails unless the hook integrates the alias.
> 
> - Based on the previous point, I'd suggest to use a wrapper if you want to be 
> cc'ed on the 
> bug you are resolving:
> 

I don't have enough information about the environment to know if it's
possible, but obviously it would be nice if the implementation could
avoid double emails. A pass through projects.xml might suffice, since
non-devs can't commit directly. It might take an extra HTTP request
though.

An alias or wrapper isn't much better than checking the box on the
webpage, at least for me personally:

  * I still have to remember to use the wrapper and not the "repoman 
commit" I've typed thousands of times.

  * I need two wrappers; one for repoman and one for pkgdev.

  * I only want to be CCed if I actually modify the bug (not merely by 
committing), since that's what prompts follow-up comments.

  * A few other people have said they would like the feature, and 
we'd all need to reimplement the same thing on a bunch of machines.

  * It needs www-client/pybugz installed =)






Re: [gentoo-dev] Feature request: auto-CC for bugs modified via commit tags

2022-01-25 Thread Michael Orlitzky
On Tue, 2022-01-25 at 11:29 -0800, Alec Warner wrote:
> 
> Just to clarify here:
> For your own commits (e.g. fixing a package you own) you are already
> typically on the bug..right?

Right.


> I assume the major use case here is proxying commits for others (where
> they are on the bug, but you are not, either directly, or via an
> alias?)
> 
> 

Take maintainer-needed packages for example. No one else is maintaining
them, and I don't want to add myself to the alias or take full
responsibility for the package. But if I have ten free minutes I might
pick an open bug from the 24h recent list on bugzilla and do a drive-by
commit to fix a simple build failure. I'd like to be CCed on the
relevant bug in case someone comments on my fix.





[gentoo-dev] Feature request: auto-CC for bugs modified via commit tags

2022-01-25 Thread Michael Orlitzky
Can I request that Bug: and Closes: tags in our commits automatically
CC the committer on the bug that is modified?

Use case: I often fix (sci-*) bugs that I'm not CCed on, and a user
will leave a comment like "it still crashes on x86" that I never see.
Of course, I could manually CC myself on every bug. But that will send
everyone an extra email, and is forgettable. Plus, avoiding the manual
step is kind of the point of the automation, right?

One potential downside is that the commit author could wind up CCed
twice via an alias, but that could be solved with a sufficiently clever
implementation. Or disregarded if it's not too much of a problem in
practice; the bugs will usually be closed, after all.





Re: [gentoo-dev] [RFC] making rust-bin ordered first in virtual/rust

2022-01-20 Thread Michael Orlitzky
On 2022-01-20 16:32:30, Brian Evans wrote:
> 
> GNOME and Mozilla products still pull in spidermonkey but other users 
> will have a much reduced requirement for rust.
> 

Avoiding librsvg used to be difficult because it's required by our GTK
icon packages to render PNGs from SVGs. Luckily dilfridge's binrepo
now makes it easy to download pre-rendered icons, and there's no
security/performance/legal concerns with pre-rendered PNGs, so using
them shouldn't present any moral dilemmata. You mainly just have to
replace Firefox, Thunderbird, and GIMP.



Re: [gentoo-dev] Rationalizing USE flags by narrowing the scope of them.

2022-01-04 Thread Michael Orlitzky
On Tue, 2022-01-04 at 19:26 +0100, Piotr Karbowski wrote:
> 
> And none of which happens unless you intentionally trigger it.
> 
> ...
> 
> Sure, acl and how chmod manipulate mask on ACL-enabled entities is not 
> very simple, but nothing will break by itself just because you have acl 
> support enabled, you would need to try very hard to run into problems.
> 
> 

Even if you're right, and if no other tools invoke tar, and the user is
smart enough not to copy/paste commands from the web, and if no other
archivers can extract ACLs when invoked directly or indirectly...
you're still burdening the user to either have faith that this is all
true, or to verify it himself. Repeat the argument for other flags like
ipv6, and you wind up requiring either a lot of faith, or a lot of
diligence, both of which are antithetical to basic principles of
security.

You may not buy the argument, but it's why people disable this stuff,
and the ability to disable it is why a lot of our users user Gentoo to
begin with.





Re: [gentoo-dev] Rationalizing USE flags by narrowing the scope of them.

2022-01-04 Thread Michael Orlitzky
On Tue, 2022-01-04 at 12:03 -0500, Mike Gilbert wrote:
> 
> I disagree with the claim that "most people" should disable ACL
> support at build time. That just gives you partially functional tools.
> The ACL behavior can generally be controlled using runtime options.

I understand why people would disagree in this case, but isn't that a
an argument for having the flag?

There are plenty of great uses for ACLs, but unless you're extremely
knowledgeable, they also add a million new ways to compromise your
system. For example, if you untar a file with a default-ACL'd directory
in it and don't notice the little plus sign, you might wind up
unknowingly creating world-writable files. Even if you do notice the
ACL, you have to be an expert in the interaction between umask,
permission bits, the ACL mask, effective permissions, conflicting ACLs,
and all of the tools you're using to understand what will actually
happen or how to properly fix it. It's not something normal people can
handle.

If you don't need them for anything, it's just nice not to have to
worry about those issues.





Re: [gentoo-dev] Rationalizing USE flags by narrowing the scope of them.

2022-01-03 Thread Michael Orlitzky
On Tue, 2022-01-04 at 03:38 +, Sam James wrote:
> 
> ACL is kind of similar to what Ionen said for PAM, i.e. sometimes
> people may want to turn it off and it makes sense to expose
> this option for those who do, but we don't need to try support it.
> 

This is another important one. It has security implications, is highly
confusing, requires kernel support, and is nonstandard as a USE flag
and as an implementation. Most people should have it off to avoid
surprises, but disabling it in the kernel can make the userland
software complain when explicitly built with ACL support.





Re: [gentoo-dev] Rationalizing USE flags by narrowing the scope of them.

2022-01-03 Thread Michael Orlitzky
On Mon, 2022-01-03 at 16:51 -0800, Alec Warner wrote:
> > 
> > 
> > Many packages need their ipv6 code disabled if the kernel has no ipv6
> > support, and enabling ipv6 in the kernel is a pointless security risk
> > for pretty much anyone in the United States.
> 
> https://www.google.com/intl/en/ipv6/statistics.html#tab=per-country-ipv6-adoption
> 
> Go troll somewhere else?
> 

Anyway, leave the USE flag please.





Re: [gentoo-dev] Rationalizing USE flags by narrowing the scope of them.

2022-01-03 Thread Michael Orlitzky
On Mon, 2022-01-03 at 21:29 +0100, Piotr Karbowski wrote:
> 
> Perhaps the 'pam' example was extreme, but ipv6, or threads as Sam 
> shared, does not make sense to be togglable.
> 

Many packages need their ipv6 code disabled if the kernel has no ipv6
support, and enabling ipv6 in the kernel is a pointless security risk
for pretty much anyone in the United States.





Re: [gentoo-portage-dev] [PATCH] portage.output: Replace darkblue colors with teal

2021-12-04 Thread Michael Orlitzky
On Sat, 2021-12-04 at 10:47 +0100, Fabian Groffen wrote:
> 
> Now, if you would make a supported claim that all terminals we install
> use a black background by default, your change becomes more valid.
> 
> 

For one, when you're working through the handbook to install Gentoo,
you're doing it on a black background.

Or if you have any physical servers. The dark-blue-on-black is even
more fun to read from three feet away on the CRT monitor from 1995 in
the back of a poorly lit server room.





Re: [gentoo-dev] Common options missed in OpenRC declarative scripts and how to improve them

2021-12-02 Thread Michael Orlitzky
On 2021-12-02 08:12:55, Alec Warner wrote:
> 
> Can we automate any of it? Emit QA warnings? etc.
> 

I would love to be proven wrong, but I don't think so. We have two
main problems. First, The service scripts are POSIX sh, which is
better than bash, but still can't easily be parsed for semantic
information.

Second, if the daemon is "special," then the service script is
justified in being similarly unconventional. Unusual runtime behavior
can't be statically detected, and I doubt that the well-behaved
portion of daemons in the tree is large enough that we can warn about
every script that smells a little bit fishy.



Re: [gentoo-dev] Common options missed in OpenRC declarative scripts and how to improve them

2021-12-02 Thread Michael Orlitzky
On 2021-12-01 21:02:20, Brian Evans wrote:
> After a cursory scan of the Gentoo repository, I've noticed an
> overabundance of start_stop_daemon_args being declared in scripts committed.
> 
> I would like to draw attention and see if we can clean these up together.

A lot of this is covered in the service script guide:

  https://github.com/OpenRC/openrc/blob/master/service-script-guide.md

There's a 2.5-year old bug to mention it in the devmanual:

  https://bugs.gentoo.org/684354



Re: [gentoo-dev] rfc: allow -1 for ACCT_USER_ID and ACCT_GROUP_ID in ::gentoo

2021-12-01 Thread Michael Orlitzky
On Tue, 2021-11-30 at 22:45 -0800, Alec Warner wrote:
> 
> So questions from my side are:
> Does your cluster not have human users?
> Do the userids for the human users also not have to match between
> hosts in the cluster?
> 
> 

You can easily create ebuilds for the human users who access the
system, and sets like @admins or @developers, so that all important
user information is ultimately recorded by the package manager.





Re: [gentoo-dev] rfc: allow -1 for ACCT_USER_ID and ACCT_GROUP_ID in ::gentoo

2021-11-30 Thread Michael Orlitzky
On Tue, 2021-11-30 at 19:32 -0600, William Hubbs wrote:
> 
> This is the part of this that I don't understand. If we aren't enforcing
> an ID, why do we care which ID to try first? It seems to be an
> unnecessary step since users can pick the IDs they want by putting
> settings in make.conf.
> 

This way, all new installs have consistent IDs by default. Users having
to manually specify every ID on every system to have them be consistent
is exactly what we are trying to avoid.






Re: [gentoo-dev] rfc: allow -1 for ACCT_USER_ID and ACCT_GROUP_ID in ::gentoo

2021-11-29 Thread Michael Orlitzky
On Mon, 2021-11-29 at 05:05 +, Sam James wrote:
> 
> What I wish we had done (and there's still time to do, albeit belated --
> it's still useful for the remaining big bits like Apache and nginx) is
> write a news item explaining the implications and linked to a page
> like https://wiki.gentoo.org/wiki/Practical_guide_to_the_GLEP_81_migration 
> 
> (which ConiKost created after we discussed how to inform users better)
> that explains how to work around/express their preferences/give their own 
> hints.
> 

I agree, and there's still room to improve the user interface as well.
For example, I run postfix on all of my machines. A few of them also
need to run OpenDKIM, and need the postfix user to be a member of a
group that is shared with OpenDKIM, to access a socket.

To best integrate with the PM, the solution to this problem is to
override acct-user/postfix in an overlay. Ok, no problem. But since the
additional group should be optional, it needs to be controlled by a USE
flag if you want to use the same ebuild across your entire
organization. The implementation details make this a bit ugly:

  RDEPEND+=" dkimsocket? ( acct-group/dkimsocket )"

  pkg_setup() {
if use dkimsocket; then
  ACCT_USER_GROUPS+=( dkimsocket )
fi
  }

The extra noise is necessary because "dkimsocket" needs to be added to
two arrays, and that can't be done declaratively at the moment. It
would be a nice UI improvement if the best solution was less of a
headache.



> Sorry, I should've been explicit. The main thing I'd like to understand better
> from your POV is:
> 
> this isn't new, but you're quite clear you feel that the UID/GID range 
> limitations
> are completely arbitrary and without merit(?).
> 
> Whissi essentially says the opposite: 
> https://archives.gentoo.org/gentoo-dev/message/17a22877f5f18dae44a2f0859d807450
>  
> .
> 
> I'd like to understand if this is just a result of beliefs about what the PM 
> should/shouldn't do
> or if there's genuine problems with continuing to extend the range?

Using the rest of the system range (500-999) should pose no problems
for anyone, since that's exactly what the old user.eclass will do. I
also see no problem with 60001+. I'm skeptical that any daemons have
those ranges hard-coded; and even if they do, they're buggy and we
shouldn't handicap the distribution over it.

More fundamentally, the "user" and "system" limits themselves not
written in stone: they are written in login.defs, and are only hints
for a few standard tools. Software should not be guessing at them. I
think we should try to keep things as consistent as possible with other
distributions, operating systems, and standards -- but if it comes down
to it, the numbers in the acct-* ebuilds are only suggestions, and it
doesn't hurt anything in the long run to suggest a number that might
already be taken because login.defs allowed useradd to take it in the
past.





Re: [gentoo-dev] rfc: allow -1 for ACCT_USER_ID and ACCT_GROUP_ID in ::gentoo

2021-11-28 Thread Michael Orlitzky
On Sun, 2021-11-28 at 23:39 +, Sam James wrote:
> 
> Whissi and others raised some points that I think you may have some views on
> (and I'm interested in hearing them).
> 

I don't want to put words in his mouth, but I think Whissi takes issue
with using the package manager to manage users, period. Not
specifically with our use of a UID/GID hint.

I didn't respond to the first thread because I didn't want to pick a
fight when the correct conclusion (IMO) was already reached. In the
first thread I see only hypothetical problems raised (and a bunch of
people who didn't realize the numbers are only a hint). If any of those
problems are real and solved by allowing ACCT_USER_ID=-1 in ::gentoo,
you'll have to point them out.





Re: [gentoo-dev] rfc: allow -1 for ACCT_USER_ID and ACCT_GROUP_ID in ::gentoo

2021-11-28 Thread Michael Orlitzky
On Sun, 2021-11-28 at 16:31 -0600, William Hubbs wrote:
> All,
> 
> I want to discuss why we ban -1 as the ACCT_USER_ID and ACCT_GROUP_ID setting
> for all acct-user and acct-group packages in ::gentoo.
> 
> Here are my thoughts about it.
> 
> - As Gordon pointed out, it isn't necessary for us to care about UIDS/GIDS
>   most of the time.

It's not for you. It's for end users. And you don't have to care about
them. Just pick any old number.


> - I realize that our settings are suggestions, but the values we can
>   suggest are not infinite. We have run out once, and it is only a matter of
>   time until we do again.

We did not run out. The council placed an arbitrary limit on them once,
and then had to raise their own arbitrary limit.

Nobody complaining about "running out" understands what the GLEP says.
If we ever hit 2^16 acct-group packages, feel free to reuse them, or
keep counting. Nothing bad will happen. The worst case scenario is
still better than if no hint was given at all.


> - If an end user needs to care about the UID/GID, they can easily override
>   the settings in make.conf.

The point of the feature is to encourage all new installs to have
consistent UIDs/GIDs by default, without user intervention. Your
suggestion does not solve the same problem, and requires more work to
not solve it.


> 
> Thoughts? In particular, I want to hear from folks who disagree with me
> about using -1 in the main tree for most packages.
> 

The only problem that anyone has put forth is one that does not exist.
UIDs and GIDs are still assigned dynamically in Gentoo. The number you
type in the ebuild is only a hint: it's the first number that will be
tried during the dynamic assignment. There is no limit on the number of
hints, and we will never run out because a conflict is never possible,
because the damned things are assigned dynamically.

Is there an actual problem?





Re: [gentoo-dev] Don't use UIDs and GIDs below 100 without QA approval

2021-11-28 Thread Michael Orlitzky
On 2021-11-28 11:06:36, Ulrich Mueller wrote:
> 
> While the rationale for static allocation that made it into GLEP 81 [1]
> is rather weak, several people had argued in favour of it on the mailing
> list [2].
> 

We don't even do static allocation. The UIDs and GIDs in the ebuilds
are suggestions, meant to benefit the people who will benefit from
them, and be ignored by everyone else.

There are a few exceptional cases where a user or group needs a
specific identifier; but those were always statically allocated and
nothing has changed in that regard.



Re: [gentoo-dev] Common Lisp project is empty

2021-11-03 Thread Michael Orlitzky
On Wed, 2021-11-03 at 16:25 +0100, Max Zettlmeißl wrote:
> On Wed, 3 Nov 2021 at 14:15, Michael Orlitzky  wrote:
> > If no one intends to join, we should drop the following to maintainer-
> > needed:
> 
> I use some of those packets daily. Is there a way to join the Common
> Lisp project without currently having maintainer status or is that not
> intended?

I don't think you can join the project, but you could still be added as
an additional proxy maintainer for those packages you care about. That
would let you take care of them using pull requests once everyone is
made aware that common-lisp@ isn't going to respond to RFCs.

Then if the common lisp project is ever disbanded, you would remain as
the sole maintainer of those packages rather than having them go to
maintainer-needed.





[gentoo-dev] Common Lisp project is empty

2021-11-03 Thread Michael Orlitzky
If no one intends to join, we should drop the following to maintainer-
needed:
 
app-misc/rlwrap
dev-libs/ffcall
dev-libs/libsigsegv
dev-lisp/alexandria
dev-lisp/asdf
dev-lisp/cl-ppcre
dev-lisp/cl-ppcre-unicode
dev-lisp/clisp
dev-lisp/clozurecl
dev-lisp/clx
dev-lisp/cmucl
dev-lisp/ecls
dev-lisp/flexi-streams
dev-lisp/gcl
dev-lisp/hyperspec
dev-lisp/sbcl
dev-lisp/trivial-gray-streams
dev-lisp/uiop
virtual/commonlisp

The full package list is available at p.g.o but those above have no
other maintainer.





Re: [gentoo-portage-dev] [RFC] LTS branch of Portage

2021-10-05 Thread Michael Orlitzky
On Tue, 2021-10-05 at 17:13 +0200, Michał Górny wrote:
> 
> >   2. What happens to the LTS branch when the next EAPI is released?
> > 
> 
> I haven't really thought about it.  Are you suggesting that we could
> bump 'master' Portage to newer EAPI earlier or...?
> 

I just mean that, a priori, the LTS branch won't be able to install
ebuilds that use the next EAPI. Backporting an entire EAPI doesn't seem
appropriate for a stable series; but maintaining an increasingly-
obsolete branch at that point doesn't make much sense either.





Re: [gentoo-portage-dev] [RFC] LTS branch of Portage

2021-10-05 Thread Michael Orlitzky
On Tue, 2021-10-05 at 10:31 +0200, Michał Górny wrote:
> Hi, everyone.
> 
> I've been thinking about this for some time already, and the recent
> FILESDIR mess seems to confirm it: I'd like to start a more stable LTS
> branch of Portage.
> 

I think this is healthy for most software projects, but two things
stand out to me for portage specifically:

  1. Most portage users can fake this already by telling portage to 
 accept only stable keywords for sys-apps/portage. I know it's not 
 exactly the same, but it provides many of the same benefits.

  2. What happens to the LTS branch when the next EAPI is released?





Re: [gentoo-dev] Guidance on adding kernel config checks to ebuilds

2021-09-28 Thread Michael Orlitzky
On Mon, 2021-09-27 at 15:44 -0400, Mike Gilbert wrote:
> On Mon, Sep 27, 2021 at 3:11 PM Peter Stuge  wrote:
> > 
> > Mike Gilbert wrote:
> > > It's a waste of time and effort to pepper random ebuilds with checks
> > > for options that everyone should have enabled in the first place.
> > 
> > It's not for you to say what everyone should have enabled in their kernel.
> 
> Do you not agree that there are some options that should always be
> enabled, or at least that we can assume are enabled?
> 

There is a distro/Kconfig in gentoo-sources with,

  config GENTOO_LINUX
bool "Gentoo Linux support"
...

that could be extended to include a set of assumed features; otherwise
a sub-setting (default enabled) could be added.





Re: [gentoo-dev] [PATCH 02/44] apache-module.eclass: Set @PROVIDES

2021-09-02 Thread Michael Orlitzky
On Thu, 2021-09-02 at 14:58 +0200, Michał Górny wrote:
> > 
> Apparently, need_apache* is the problem.  Most of the ebuilds in www-
> apache/* are calling it:
> 

The apache-module eclass tells you to do that because it needs some of
those APACHE_* paths. It should really be figuring them out itself
rather than telling the user to call a function that does it in global
scope. It's an easy fix:

  APXS=apxs
  APACHE_MODULESDIR=$(apxs -q libexecdir)
  APACHE_MODULES_CONFDIR=$(apxs -q sysconfdir)/modules.d
  APACHE_VHOSTS_CONFDIR=$(apxs -q sysconfdir)/vhosts.d

On the one hand, we probably don't want ebuild maintainers trying to
solve that themselves when the eclass could do it. But for whatever
such a promise is worth, it also feels wrong to promise that apache-
module will provide all of depend.apache -- what if we someday apply
the fix above to apache-module and want to drop the depend.apache
inherit?

If we do ever update apache-module, updating ebuilds and retiring
depend.apache would be a lot easier if you could find the candidates
with `git grep depend.apache`, rather than having to look through all
consumers of apache-module for implicit uses of depend.apache.





Re: [gentoo-dev] [PATCH 02/44] apache-module.eclass: Set @PROVIDES

2021-09-02 Thread Michael Orlitzky
On Thu, 2021-09-02 at 12:46 +0200, Michał Górny wrote:
> Signed-off-by: Michał Górny 
> ---
>  eclass/apache-module.eclass | 1 +
>  1 file changed, 1 insertion(+)
> ...
>  # @SUPPORTED_EAPIS: 5 6 7
> +# @PROVIDES: depend.apache

I'm not sure about this one. The depend.apache eclass is junk and we
should be encouraging people to move away from it (bug 616612):

  * If you want to depend on apache, depend on apache.

  * If you need paths like APACHE_MODULESDIR, the "apxs" tool is now
in PATH so you can get it from $(apxs -q libexecdir)

  * If you need paths like APACHE_MODULES_CONFDIR, the eclass doesn't 
work anyway. You can hard-code those paths yourself (relative to
apxs -q sysconfdir), or if anyone feels up to the task, they could
write a greatly simplified apache-paths.eclass that provides these
paths via functions that are to be called outside of global scope.

If people are using anything in depend.apache, I think a warning is
appropriate. Making a promise that apache-module (which is not junk)
provides depend.apache will moreover make it harder to disentangle them
in the future if anyone decides to fix things.

Disclaimer: I'm not the apache maintainer.





[gentoo-dev] Infra support for mail submission with implicit TLS on port 465

2021-08-14 Thread Michael Orlitzky
There have been some attacks on STARTTLS lately, so I'm finally getting
around to using implicit TLS for mail submission on port 465.

I tried this on dev.gentoo.org, and it seems to work. For example: I
just switched Evolution to port 465, with always-on TLS, and am sending
this message.

Is this supported? I don't see it in the infra docs anywhere.





Re: [gentoo-dev] [PATCH 2/3] qmail.eclass: remove magic to query root group

2021-08-12 Thread Michael Orlitzky
On Thu, 2021-08-12 at 17:22 +0200, Rolf Eike Beer wrote:
> The default owner is root:root anyway.
> 

This is only true of you don't call insopts earlier with some other
value. I see "insopts -o root -g qmail -m 700" in there so you might
want to double check.





Re: [gentoo-dev] [PATCH] eclass/postgres.eclass: migrate to GLEP 81

2021-07-22 Thread Michael Orlitzky
On Fri, 2021-07-23 at 00:20 +0200, Conrad Kostecki wrote:
> This update drops the function 'postgres_new_user', which was used to
> create the 'postgres' user and group.
> 
> ...
> 
> Since many other packages depend on the 'postgres' and 'postgres-multi'
> eclass, adding the core acct-*/postgres packages here to [R]DEPEND.

Not all packages using the eclass necessarily need the "postgres"
user/group to build/run. You should only add (R)DEPEND in the packages
that actually call postgres_new_user, I think?


> 
> Before merging this eclass patch, acct-* packages will be added to
> the tree.

Before doing this, please double-check that SHELL=/bin/bash and
HOME=/var/lib/postgres are needed. It's likely that the default HOME
can be used, and that dev-db/postgresql (which actually *uses*
/var/lib/postgres) should control its permissions.

You should also get an ACK from titanofold, since he's been maintaining
postgresql essentially by himself forever (and may have some insight
into the SHELL/HOME items).





Re: [gentoo-dev] [PATCH] profiles/default/linux: Add USE="bzip2 lzma zstd" to defaults

2021-07-12 Thread Michael Orlitzky
On Mon, 2021-07-12 at 10:46 -0500, Ben Kohler wrote:
> 
> Nobody is "disabling choice" here, a change in defaults doesn't remove 
> your ability to choose something else.

I think what you're suggesting is that default-on is not any worse for
choice than default-off, since both can be changed?

Consider the one legitimate example given: sys-apps/kmod. How can we
disable lzma for everything except packages that have +lzma defaulted?
(Ignoring the open pull request, that is usually done for a good
reason.) In other words, how do people undo this patch, without
potentially breaking their systems?

I hesitate to speak for anyone else, but all I personally want is to be
reasonably sure what my configurations are going to do without having
to list every individual package and USE flag explicitly. I don't think
it's written in stone anywhere, but the repo relies on the fact that
USE flags are disabled by default. As a result, it's much easier for
users to add things to USE than it is to remove them.

If we assume that most IUSE default-ons exist for a good reason
(roughly true), then you can imagine two groups of people.

Person 1: wants everything enabled by default.
Person 2: wants only important things (determined by chosen profile and
IUSE defaults) enabled by default.

Before the patch,

Person 1: adds USE="bzip2 lzma zstd" to make.conf. Requires no ongoing
maintenance.
Person 2: does nothing.

After the patch,

Person 1: does nothing.
Person 2: lists a hundred different packages in all of his package.use
files, after checking each of them to see which ones have important
IUSE defaults. Requires ongoing maintenance as new packages are added.

We've kept things the same level of difficulty for one group of people,
but made them much harder for another. In no situation can anyone who
wants everything enabled have a harder time than 'adds USE="bzip2 lzma
zstd" to make.conf', but everyone else suffers to some degree. That's
discouraging choice overall.





Re: [gentoo-dev] [PATCH] 2021-07-09-systemd-tmpfiles: re-add news item

2021-07-12 Thread Michael Orlitzky
On Sun, 2021-07-11 at 15:53 +0200, Thomas Deutschmann wrote:
> 
> Furthermore, tmpfiles.d settings are only supposed for creation, 
> deletion and cleaning of volatile and temporary files. Any package which
> will install tmpfiles.d settings which will create files in persistent 
> locations should be treated like a bug in the package itself (for Gentoo
> packagers for example we have keepdir [3] function).

Not crucial to your main point, but packages that use keepdir under
/var/cache (which is persistent) get prodded to use tmpfiles instead:

  https://bugs.gentoo.org/692736





Re: [gentoo-dev] [PATCH] profiles/default/linux: Add USE="bzip2 lzma zstd" to defaults

2021-07-09 Thread Michael Orlitzky
On Fri, 2021-07-09 at 12:05 +0200, Ulrich Mueller wrote:
> > > > > > 
> 
> Sorry, but that doesn't make sense. These are global USE flags, and
> the aim here is to set a _global_ default for them, based on the fact
> that these libraries are present on every Gentoo system.
> 
> So if we agree that e.g. zlib should be on by default, then this
> belongs
> in profiles.

We don't agree that it belongs on by default. And I never said they
shouldn't be enabled in a profile. I said they shouldn't be enabled in
the BASE profile. If you enable them in, say, the desktop profile that
I have the option of not using, I'm fine with that.





Re: [gentoo-dev] [PATCH] profiles/default/linux: Add USE="bzip2 lzma zstd" to defaults

2021-07-08 Thread Michael Orlitzky
On Thu, 2021-07-08 at 13:17 -0700, Matt Turner wrote:
> 
> I hear you, and I appreciate the theoretical concerns.
> 
> I could maybe even support this position if you were actively working
> towards this new and glorious future, but the only time I hear
> anything about it is when you're arguing that someone else should do
> something the way you want.

I've spent a great deal of time in the past moving flags from the base
profiles into package defaults. Not that I see what that has to do with
anything. I'm not arguing that you should do things the way I want
except insofar as I'd prefer you to choose the way of doing things that
has all of the benefits for you and none of the downsides for the rest
of us.


> > 
> > I don't have them enabled for any packages where they're not IUSE-
> > defaulted, and haven't noticed any problems. Not not having a reason to
> > do something isn't a good justification to do it. If it ain't broke and
> > all that. If anyone wants them set, it's as easy as USE="lzma bzip2
> > zstd", and we are apparently all okay without them.
> 
> Well, you're okay without them until you need them: E.g., enable
> CONFIG_MODULE_COMPRESS / MODULE_COMPRESS_XZ without knowing that you
> need sys-apps/kmod[lzma] and your system becomes unbootable.
> 
> But you're of course right that some die-hards might rather accept
> this risk and save the 8 KiB of disk space (424 KiB → 436 KiB) they'd
> lose out on by enabling USE=lzma for the package. But the good news
> is.. they still can!

No, this is a great case for the package default. Save some people a
headache, and include the batteries for kmod. I'll disable that one
manually if I know what I'm doing. But, the kmod situation is not a
good reason to enable lzma support in FFmpeg.

Given the above, it is maybe not surprising that sys-apps/kmod already
has IUSE="+lzma +zlib".


> 
> > But, if you really look, I think you'll find that most of these flags
> > do completely useless things. Do you REALLY need libpcre to build and
> > install you a special executable called "pcregrep-libbz2" that just
> > pipes bunzip2 to pcregrep? No, nobody does. And most other uses are
> > comparably stupid.
> 
> I mean, you could make that argument for bzless or any other version
> of these tools. I don't find that compelling.

You don't find it compelling that changing the default globally has no
additional benefits but a bunch of negative side effects? I *am* making
that argument about all of the other tools. The bzip2/lzma/zstd support
is mostly junk and should not be enabled by default, especially not
globally. (If individual maintainers want to enable junk features by
default, there's not much I can do except point out how unusually
difficult it is to override.)




> Maybe you'd like to audit the compression USE flags and make
> recommendations for which you think do completely useless things? I
> can pretty easily replace this patch with a set of
> automatically-generated patches that enable these USE flags by default
> for all packages but on a per-package basis.

You're the one trying to do something here but you haven't really said
what yet. No, I don't want to spend hours reverse engineering what
you're trying to accomplish when you could just tell us. Which packages
do you think need these flags on by default, and why? If their
maintainers agree, you don't need anyone else's approval.





Re: [gentoo-dev] [PATCH] profiles/default/linux: Add USE="bzip2 lzma zstd" to defaults

2021-07-08 Thread Michael Orlitzky
On Thu, 2021-07-08 at 11:15 -0700, Matt Turner wrote:
> 
> So, the thing about running a minimal system is... you already have
> these dependencies installed. This doesn't change that...
> 
> I could of course change the default of every bzip2/lzma/zstd in IUSE
> (and might as well handle zlib too so we can remove it from
> make.defaults!) but what practical advantage does that bring?

There's more to being a dependency than just being installed. Not all
of these packages have e.g. USE=zstd so that they can run
/usr/bin/zstd. Some link against libzstd, which now bloats them to use
a tiny bit more space and memory, as well as exposing you to any
security problems in the library. Your dependency tree gets a little
bit more complicated, and your package manager has to figure out how to
do subslot rebuilds for everything when libzstd gets upgraded.

Per-package defaults are easier to override, since I don't have to undo
everything before setting the USE-flags that I wanted to enable in the
first place.

Per-package defaults are easier to revert if we change our collective
minds later, since we don't have to test the entire tree for breakage
first.

Global flags have essentially undefined behaviour, since e.g. USE=bzip2
does different things for each package. As an extreme example, global
flags affect packages that aren't even in the tree yet and that may use
USE=bzip2 in a way you don't expect. As a less extreme example,
USE=bzip2 may open your crypto library up to compression attacks. It
definitely makes your dev-lang/php more vulnerable. (The same goes for
USE=zlib, which should be removed in favor of per-package defaults.)

Global flag settings increase complexity because they all interact with
each other, creating a combinatorial explosion of interaction points.
Figuring out how to turn a global flag off for a subset of packages can
be a nightmare. Do you change the file where it's enabled? Or the arch
profile where the enabling is reverted? Or the arch/desktop profile
where the disabling is disabled? Or the hardened profile where the
disabled disabling is disabled? Any argument against global variables
in a program is an argument against global profile changes.


> I doubt there's a sensible reason to build without any of these USE
> flags enabled. I think the claim that most people will want them
> enabled is not really a question. So we should enable them by default.
> I think that logic is pretty straightforward. If someone wants to
> disable the flags for some reason, they of course still have that
> option.
> 
> If you can find a case where you wouldn't want to enable one of these
> USE flags, please let me know and I'll reconsider my position.
> 

I don't have them enabled for any packages where they're not IUSE-
defaulted, and haven't noticed any problems. Not not having a reason to
do something isn't a good justification to do it. If it ain't broke and
all that. If anyone wants them set, it's as easy as USE="lzma bzip2
zstd", and we are apparently all okay without them.

If you have a good reason to do it for certain packages, setting per-
package defaults is the way to do it. The base profile defaults are
only there because we didn't always have per-package defaults.

But, if you really look, I think you'll find that most of these flags
do completely useless things. Do you REALLY need libpcre to build and
install you a special executable called "pcregrep-libbz2" that just
pipes bunzip2 to pcregrep? No, nobody does. And most other uses are
comparably stupid.




Re: [gentoo-dev] [PATCH] profiles/default/linux: Add USE="bzip2 lzma zstd" to defaults

2021-07-08 Thread Michael Orlitzky
On Thu, 2021-07-08 at 07:19 -0500, Ben Kohler wrote:
> On 7/8/21 6:54 AM, Michael Orlitzky wrote:
> > On Wed, 2021-07-07 at 22:01 -0700, Matt Turner wrote:
> > > Enable these flags by default, since they effectively add no additional
> > > dependencies:
> > Why? This list should be getting smaller, not larger.
> 
> That's a valid opinion/viewpoint, but it's not a fact.
> 
> 

It is a fact. If you want to enable these features by default in some
packages (WHY?), there are better ways to do that now. Adding global
defaults into the profiles is sloppy and should be avoided.





Re: [gentoo-dev] Declare the type of source

2021-06-28 Thread Michael Orlitzky
On Mon, 2021-06-28 at 16:52 +0200, Michał Górny wrote:
> > 
> > This is long overdue, for many reasons, but in particular it would
> > force us to declare a dependency on a C compiler if one is needed and
> > allow you to re-test only those packages that use a C compiler.
> > 
> 
> Which C compiler?  The one I have in CC for the package in question, or
> the one selected via gcc-config, or any random C compiler that might not
> be used at all?  Does that mean that if clang is pulled via depgraph,
> Portage will insist on depcleaning gcc by default?
> 

If the package declares a dependency on e.g. virtual/c-compiler, ago
would want to re-test it whenever a new version of any compiler is
released that satisfies virtual/c-compiler. Conversely, if the package
doesn't require virtual/c-compiler, we may assume that it doesn't
compile C code, and is not affected by the new version.

I know switching toolchains is a mess at the moment, but independently
adding (correct) dependency information should make many things easier
without harming anything else. The precise nature of the dependencies
would of course need some thought.





Re: [gentoo-dev] Declare the type of source

2021-06-28 Thread Michael Orlitzky
On Mon, 2021-06-28 at 16:31 +0200, Gerion Entrup wrote:
> 
> This should only build, if _all_ build dependencies are present
> (including every compiler and base system tool). Of course, it needs a
> bigger rework of the portage build process.

We had a GSoC project that aimed to do something like this back in
2011:

  https://gitweb.gentoo.org/proj/autodep.git/

At this point, we'd be starting over from scratch, but in short: it's a
good idea. Filling out the list of dependencies should be boring and
fool-proof.






Re: [gentoo-dev] Declare the type of source

2021-06-28 Thread Michael Orlitzky
On Mon, 2021-06-28 at 15:00 +0200, Agostino Sarubbo wrote:
> 
> Instead, imagine that each ebuild declares a variable called SOURCETYPE ( or 
> similar, or in metadata.xml if you prefer ) and with a tool like equery/eix 
> we 
> are able to get the list of all packages that compiles C code.
> 

I think all you are really asking for is that we stop omitting a random
subset of @system from *DEPEND.

This is long overdue, for many reasons, but in particular it would
force us to declare a dependency on a C compiler if one is needed and
allow you to re-test only those packages that use a C compiler.





Re: [gentoo-dev] [PATCH] qmail.eclass: simplify is_prime()

2021-06-18 Thread Michael Orlitzky
> 
> This depends on the actual domain of numbers. If the primes involved
> have 20 digits as in your example, then factor should be used of course.
> 
> I suspect though that we're talking about small numbers (below 100?)
> here, in which case a solution in pure bash would be preferable.
> 

If so, we could just list them all.






Re: [gentoo-dev] [PATCH v2 3/3] gnuconfig.eclass: use BDEPEND, BROOT where available (drop support for EAPI <4)

2021-04-09 Thread Michael Orlitzky
On Sat, 2021-04-10 at 00:32 +0100, Sam James wrote:
> 
> 
> Yes, this is the part I find difficult too. The important
> distinction here was *bootstrapping* (which I missed)
> but I think at least we should make a list of packages generally considered
> critical for bootstrap.
> 

What is a bootstrap package?

There is some chicken-and-egg problem to be solved, but I don't think
that we should be assuming that e.g. GNU grep is always present just
because, during the base case of some recursive process, POSIX grep
must be available temporarily.

Anyway, https://bugs.gentoo.org/485356 awaits reopening if you make any
progress on this.





Re: [gentoo-dev] [PATCH v2 3/3] gnuconfig.eclass: use BDEPEND, BROOT where available (drop support for EAPI <4)

2021-04-07 Thread Michael Orlitzky
On Tue, 2021-04-06 at 20:32 +0100, Sam James wrote:
> 
> > 
> > We usually don't add basic tools like grep to dependencies.
> 
> Few points:
> 
> ...

5) There are no clear rules about what @system packages can be left out
of *DEPEND and when, so their presence is wildly inconsistent.





Re: [gentoo-dev] New tool: merge-driver-ekeyword automatically resolves git merge conflicts involving KEYWORDS=...

2021-03-03 Thread Michael Orlitzky
On Wed, 2021-03-03 at 13:09 +0100, Ulrich Mueller wrote:
> > > > > > On Tue, 02 Mar 2021, Michael Orlitzky wrote:
> > > Are you volunteering to fix all the tools to support the new format
> > > correctly?
> > The PMS says that KEYWORDS is whitespace-separated. Probably only
> > repoman/pkgcheck would require trivial changes.
> 
> No, there are other tools as well, e.g. ekeyword and ebuild-mode.
> 

But only repoman/pkgcheck should be enforcing ::gentoo QA policy. The
others are already buggy if they don't support newlines and should be
fixed independently. Then GOTO 1: only repoman/pkgcheck require trivial
updates.





Re: [gentoo-dev] New tool: merge-driver-ekeyword automatically resolves git merge conflicts involving KEYWORDS=...

2021-03-02 Thread Michael Orlitzky
On Tue, 2021-03-02 at 20:42 +0100, Michał Górny wrote:
> 
> Are you volunteering to fix all the tools to support the new format
> correctly?
> 

When you already spend all day fixing other peoples' software, this
doesn't sound that scary.

The PMS says that KEYWORDS is whitespace-separated. Probably only
repoman/pkgcheck would require trivial changes. Anything not enforcing
::gentoo policy should (ha ha) already support the PMS format.





Re: [gentoo-dev] New tool: merge-driver-ekeyword automatically resolves git merge conflicts involving KEYWORDS=...

2021-03-02 Thread Michael Orlitzky
On Tue, 2021-03-02 at 14:02 -0500, Matt Turner wrote:
> On Tue, Mar 2, 2021 at 12:11 AM Michael Orlitzky  wrote:
> > why don't we just enforce putting each
> > keyword on a separate line instead, so that we don't have this problem
> > in the first place?
> 
> Why don't we change 30 thousand ebuilds rather than use this script?
> 

If we're going to enforce a format, it makes more sense to sed things
into the easy format once than it does to leave things in the hard
format forever and maintain a custom tool that makes the hard format
more like the easy format.

With everything currently on one line, QA could easy change them all at
once in one big commit.





Re: [gentoo-dev] New tool: merge-driver-ekeyword automatically resolves git merge conflicts involving KEYWORDS=...

2021-03-01 Thread Michael Orlitzky
On Mon, 2021-03-01 at 22:54 -0500, Matt Turner wrote:
> tl;dr: In app-portage/gentoolkit-0.5.1 there's a new tool I wrote,
> called merge-driver-ekeyword that can automatically resolve git merge
> conflicts involving the KEYWORDS=... line in ebuilds.
> 
> Since the KEYWORDS=... assignment is a single line,

Is that enforced? If not, will the merge driver handle other formats
correctly? And if it is... why don't we just enforce putting each
keyword on a separate line instead, so that we don't have this problem
in the first place?





Re: [gentoo-dev] Re: portage reliance on GNU objcopy ownership perseverance behavior in strip

2021-02-09 Thread Michael Orlitzky
On Tue, 2021-02-09 at 21:44 -0500, Michael Orlitzky wrote:
> 
> ... games-arcade/xboing are also suspect.
> 

(This one's fine, that's the documented way to do things now. Although
I don't see why we couldn't use a separate group for each game's score
file.)





Re: [gentoo-dev] Re: portage reliance on GNU objcopy ownership perseverance behavior in strip

2021-02-09 Thread Michael Orlitzky
On Tue, 2021-02-09 at 18:25 -0800, Manoj Gupta wrote:
> > 
> > Root is the owner but often there is also a group that has access to the
> files.
> After stripping with llvm-strip, new ownership is root:root instead of
> root:.
> Therefore, the members of the group lose access to the files post stripping.
> 
> We found this issue in Chrome OS when we tried to switch the defaults to
> llvm's objcopy/strip.

Thanks, I still agree that some consistent behavior is needed. The only
reason I asked is because the fact that you *noticed* the problem is a
red flag to me. A suid group is a valid use case, but a few of these
examples don't set any special group permissions on the executables
whose group they change. For example...

> ./net-analyzer/netselect/netselect-.ebuild: fowners root:wheel
> /usr/bin/netselect

This ebuild does,

  fowners root:wheel /usr/bin/netselect
  fperms 4711 /usr/bin/netselect

which makes you wonder why it changed the group in the first place. The
ebuilds for mail-filter/procmail and games-arcade/xboing are also
suspect.





Re: [gentoo-dev] Re: portage reliance on GNU objcopy ownership perseverance behavior in strip

2021-02-09 Thread Michael Orlitzky
On Tue, 2021-02-09 at 17:53 -0800, Fāng-ruì Sòng wrote:
> (I replied via https://groups.google.com/g/linux.gentoo.dev/c/WG-OLQe3yng
> "Reply all" (which only replied to the list AFAICT) but I did not
> subscribe to gentoo-dev via the official
> https://www.gentoo.org/get-involved/mailing-lists/ so my reply is
> missing)
> 

Apologies for hijacking your post with a tangential question, but you
reminded me to ask: how did you notice this problem? Ultimately all
system executables (in $PATH) should be owned by (and writable only by)
root anyway; otherwise you get silly security vulnerabilities like "cat
~/virus > /usr/bin/foo" as a regular user.





Re: [gentoo-dev] dev-python/cryptography to use rust, effectively killing alpha, hppa, ia64, m68k, s390

2021-02-09 Thread Michael Orlitzky
On Wed, 2021-02-10 at 08:51 +0800, Benda Xu wrote:
> > 
> > The first big blocker we're going to hit is trustme [3] package that
> > relies on cryptography API pretty heavily to generate TLS certs for
> > testing.  If we managed to convince upstream to support an alternate
> > crypto backend, we'd be able to retain minor keywords a lot of packages
> > without too much pain.
> 
> I could feel the pain.  
> 
> Bootstraping Rust on Prefix is somewhere between alpha, hppa, ia64,
> m68k, s390 and amd64[1].  The problem was exposed by
> gnome-base/librsvg[2].
> 
> I am wondering how useable pkgcore is on alpha, hppa, etc.  Maybe it's
> time for us to plan for a Gentoo without essential Python dependency.

It's not usable anywhere. We keep updating the PMS, and the council
keeps voting to approve the new versions, and we teach all new
developers that they need to respect both the PMS and council
decisions... and then from that day on, everyone completely, publicly,
ignores it, adding thousands of packages across entire ecosystems that
don't work properly without portage. I'd like to say it's one of the
craziest things I've ever seen, but, there was 2020.

We "started" with three package managers, and now we're down to one.
The council needs to grow some balls and enforce the PMS before that
can change. Pkgcore could be salvaged if you could actually update your
system with it.

For my "me too," I've been told by upstream that the next major version
of clamav will require rust. There are a lot of "real" UNIX machines
relying on clamav for e.g. PCI compliance that will be screwed by that,
not to mention all of the small business routers and mail gateways on
obscure or limited hardware. Personally, I just haven't spent the last
20 years contributing to free software to be casually migrated to a
system of bundled binary blobs; nor am I able to set aside 8GB of RAM
for hours to rebuild the latest version of rust and its myriad
unofficial dependencies every day on a production mail server.
Eventually clamav will disappear, and we'll likely move a few big
customers to Microsoft O365 as a result.





Re: [gentoo-dev] dev-python/cryptography to use rust, effectively killing alpha, hppa, ia64, m68k, s390

2021-02-08 Thread Michael Orlitzky
On Mon, 2021-02-08 at 12:19 +0100, Michał Górny wrote:
> 
> Since cryptography is a very important package in the Python ecosystem,
> and it is an indirect dependency of Portage, this means that we will
> probably have to entirely drop support for architectures that are not
> supported by Rust.
> 

Not only that, but you will be dropping support for at least two of my
machines that are literally incapable of building the 10+ GiB bundled
rust package due to the amount of disk space and RAM required.





Re: [gentoo-dev] portage reliance on GNU objcopy ownership perseverance behavior in strip

2021-02-04 Thread Michael Orlitzky
On Thu, 2021-02-04 at 16:09 -0800, Manoj Gupta wrote:
> 
> What does everyone think of modifying usages of calls to strip and
> objcopy
> inside estrip so that file ownership is manually restored. e.g
> 
> owner=$(stat -U file)
> group=$(stat -G file)
> strip 
> chown owner:group file
> 

This is probably safe in portage because the temporary directory is no
longer user-accessible, but it seems perverse to seek feature parity by
adding the security vulnerability into the implementations that don't
have it, rather than by removing it from the ones that do. Hopefully
LLVM just accepts the patch.





Re: [gentoo-dev] [RFC] Moving subslot-only virtuals to a separate category to reduce confusion

2021-01-30 Thread Michael Orlitzky
On Sat, 2021-01-30 at 18:35 +0100, Michał Górny wrote:
> 
> To make this SOVERSION-virtual concept more visible and easily
> distinguishable from regular virtuals, I'd like to propose that we
> start
> moving them into a dedicated category.  For example, 'lib-sover'
> comes
> to my mind.  While this ofc doesn't guarantee that people will do
> things
> right, it will at least make it clear that these packages are
> somewhat
> different from regular virtuals and hopefully avoid making wrong
> assumptions.
> 
> 

I would go all the way and move (say) the libjpeg provider to media-
libs/libjpeg.





Re: [gentoo-dev] Re: [PATCH] acct-user.eclass: don't modify existing user by default

2021-01-04 Thread Michael Orlitzky

On 1/4/21 1:20 PM, Michał Górny wrote:


Most importantly, it doesn't resolve the core issue of 'we need to
update home before merging reverse dependencies'.



Quoth the devmanual, "if your package requires a user, you can no longer 
be sure of that user's home directory or its ownership and permissions."
Any package depending on a new revision of an acct-user ebuild to change 
its home directory is broken.


This was the exact problem I was trying to avoid while you made rude 
comments about how I was wasting everyone's time =P




Re: [gentoo-dev] Re: [PATCH] acct-user.eclass: don't modify existing user by default

2021-01-04 Thread Michael Orlitzky

On 1/4/21 1:23 PM, Thomas Deutschmann wrote:


People like me could just ignore changed users if changes won't go live
until you run said users-update command or make use of INSTALL_MASK.



Changes wouldn't go live until you ran etc-update, and *then* users-update.



Re: [gentoo-dev] Re: [PATCH] acct-user.eclass: don't modify existing user by default

2021-01-04 Thread Michael Orlitzky

On 1/4/21 11:45 AM, James Cloos wrote:

"RHJ" == Robin H Johnson  writes:


RHJ> The best I can come up with at the moment, is that any packaging should
RHJ> detect if there are user modifications, and provide control to users
RHJ> based on that fact.

Exactly.  Akin to etc-update.



We could implement this with something like an /etc/users.d directory 
that would be populated with entries by either the admin or package 
manager with CONFIG_PROTECT enabled. Then the system database would be 
updated by running something like "users-update" (cf. env-update). The 
essential problem that we need to work around is that e.g. /etc/passwd 
is "owned" by multiple system packages.


I think this would accomplish what you and Robin are talking about, but 
it wouldn't solve whissi's problem since it's still a Gentoo-specific 
solution.




Re: [gentoo-dev] [PATCH] acct-user.eclass: don't modify existing user by default

2021-01-04 Thread Michael Orlitzky

On 1/4/21 9:46 AM, Thomas Deutschmann wrote:



So the main problem I see with doing this is that it becomes
impossible to reliably make changes to a user in later ebuild
revisions.


He is obviously looking for a way to allow maintainers to change users
afterwards. But if we tell people, "If you need customization, fork the
user/group ebuild in your overlay" we will disconnect these users from
future changes.


There's pretty much no reason to change a user's settings unless you've 
completely fucked them up the first time around. This is precisely why 
the original GLEP had mailing list reviews.


If a package depends on a user having e.g. a specific home directory so 
that upgrading the package requires a corresponding revision bump of the 
user, then the package is broken. I tried really hard to document this, 
and to enforce it back when we had mailing list reviews. It's all in the 
devmanual now.




...
When you will get LPIC certification one can expect that you have some
basic knowledge in Linux stuff allowing you to do common tasks on all
different Linux systems. Now there comes Gentoo where you aren't allowed
to use standard Linux tools like 'usermod' when deploying another
service if you don't want to risk that your service will go down when
following best practice and do regular world upgrades. Really?


You also can't use the standard linux tools to edit scripts in /usr/bin 
without your changes being overwritten. This is no different... some 
things need to belong to the package manager if you want package 
management to work.




3) More important, the idea of forking acct-* packages whenever you need
to make modifications don't scale. Like I already outlined in February
2020, you cannot create overlays for each different user configuration:

I.e. using memcached/redis: You grant permission to socket via group. So
you put other services belonging to that application you are deploying
into your user running the key value store. Do you really expect that
one would create multiple overlays per application using one of these
services? How would you maintain hundreds of overlays? How would you
keep track that each box will use the correct overlay to get the
specific customized acct-* package? How do you deal with scenarios where
you don't just deploy single instances?


This is literally the example I gave. Our acct-user ebuilds can be added 
to additional groups based on USE flags. Every server uses the same 
overlay/ebuilds, but different machines get different package.use files, 
pushed out by the configuration management tool.


I understand that creating an overlay with acct-user overrides will not 
be for everyone, so I have no problem with adding an escape hatch. I do 
think it should be off by default though, and that missing future 
::gentoo changes will not be a problem unless some other error has been 
committed first.




Re: [gentoo-dev] [PATCH] acct-user.eclass: don't modify existing user by default

2021-01-03 Thread Michael Orlitzky

On 1/3/21 8:35 PM, Thomas Deutschmann wrote:

Modifying an existing user is a bad default and makes Gentoo
special because it is common for system administrators to make
modifications to user (i.e. putting an user into another service's
group to allow that user to access service in question) and it
would be unexpected to see these changes reverted during normal
world upgrade (which could break services).


It would be nice if this was well-supported by the official way of 
modifying system users/groups; that is, by using an overlay with 
modified user/group ebuilds.


Right now it's awkward to do because of the way the eclasses are 
structured. For example, some of our servers allow the "postfix" user to 
write to OpenDKIM's socket, but only on our *outgoing* mail servers (not 
on the incoming MX, where no signing takes place.) This is accomplished 
by creating an acct-group/dkimsocket ebuild (ok so far), and then by 
overriding the acct-user/postfix ebuild:


  EAPI=7

  inherit acct-user

  DESCRIPTION="user for postfix daemon"
  IUSE="dkimsocket"
  ACCT_USER_ID=207
  ACCT_USER_GROUPS=( postfix mail )
  acct-user_add_deps

  # This needs to be done outside of acct-user_add_deps because we can't
  # test use flags in global scope, and therefore we can't add groups
  # to ACCT_USER_GROUPS before calling acct-user_add_deps.
  RDEPEND+=" dkimsocket? ( acct-group/dkimsocket )"

  pkg_setup() {
  # https://wiki.gentoo.org/wiki/OpenDKIM
  #
  # Even though we added the group to RDEPEND manually, we still
  # need to add it to the array.
  if use dkimsocket; then
  ACCT_USER_GROUPS+=( dkimsocket )
  fi
  }

That's the common case of adding a system user to a group, and it's 
pretty ugly, so it's no wonder that people want to use "usermod" and 
then ignore subsequent changes by the PM.


And there's probably a backwards-compatible way we could support 
USE-conditional supplementary groups.




Re: [gentoo-dev] Slotted Lua + revdeps to be unmasked on Christmas Eve

2020-12-23 Thread Michael Orlitzky

On 12/23/20 6:18 PM, Michael Orlitzky wrote:


Using pkg-config has a related problem. If lua-5.1 is eselected and if
the upstream build system runs $(pkg-config ... lua), it's going to get
the information for lua-5.1, even if you're trying to build against
lua-5.2. So, first I have to patch upstream to use pkg-config, and then
I have to hack it in Gentoo to avoid using pkg-config.


Sorry, I was wrong about this. Patching the upstream build system to 
support pkg-config does work if you use the eclasses.





Re: [gentoo-dev] Slotted Lua + revdeps to be unmasked on Christmas Eve

2020-12-23 Thread Michael Orlitzky

On 12/23/20 4:51 PM, Mart Raudsepp wrote:

Ühel kenal päeval, K, 23.12.2020 kell 07:49, kirjutas Michael Orlitzky:

    AC_SEARCH_LIBS([whatever], [lua], [lua_found="yes"])

How do I pass the name "lua5.2" to that, without hacking configure.ac
and running autoreconf? The only option that comes to mind is to
build
the entire project with LIBS="-llua5.2", but that's not right if only
some of the executables are supposed to be linked with lua.


You either fix upstream to use pkg-config, or you set up the cache
variable for your configure call to tell configure what the answer is,
without making it do the work wrong.


Using pkg-config has a related problem. If lua-5.1 is eselected and if 
the upstream build system runs $(pkg-config ... lua), it's going to get 
the information for lua-5.1, even if you're trying to build against 
lua-5.2. So, first I have to patch upstream to use pkg-config, and then 
I have to hack it in Gentoo to avoid using pkg-config.


Overriding the cached check variable does work, but you have to override 
every single call to AC_SEARCH_LIBS(function...) with 
ac_cv_search_ which depends highly on the implementation 
details of configure.ac. This is roughly as invasive and hard to 
maintain as patching, and should just not be the default way to link 
against a library IMO.




Re: [gentoo-dev] Re: Slotted Lua + revdeps to be unmasked on Christmas Eve

2020-12-23 Thread Michael Orlitzky

On 12/23/20 6:04 PM, Aisha Tammy wrote:


Yes, this sounds doable and should work >
Only problem is that if there is an actual liblua.so with the proper
SONAME available in /usr/lib64 (I dunno if Gentoo has any provider of
liblua.so with that SONAME, IIRC SLOT=0 currently does that) then it
will ignore the wrong SONAMEd even with the -L/usr/lib64/lua5.2/



eselect-lua creates a symlink named /usr/lib64/liblua.so, but the -L 
path should take precedence when looking for liblua.so. When it's 
looking for e.g. liblua5.2.so at runtime, there is only one copy to 
find, so no problem there.




Re: [gentoo-dev] Re: Slotted Lua + revdeps to be unmasked on Christmas Eve

2020-12-23 Thread Michael Orlitzky

On 12/23/20 1:14 PM, Aisha Tammy wrote:


I've recently had the same problem for TACC/Lmod which uses
autotools to get lua versions and lua.cpath and lua.path and did infact manage
to push the horrendously large patch upstream -
https://github.com/TACC/Lmod/commit/0913bf05dd7e8f478f69d5297e26d744ddb5073a

Maybe something similar can work for your use case?


Yes, patching the build system works.



The problem with just using -L/path/to/lua/lib/ -llua would be loading
library at runtime, unless autotools is smart enough to actually change this
CFLAGs into a -Wl,-rpath,/path/to/lua/lib -L/path/to/lua/lib -llua.
I am not sure how intelligent autotools is to be able to do this or not.



We already add entries to /etc/ld.so.conf to make things like slotted 
LLVM work.




Re: [gentoo-dev] Re: Slotted Lua + revdeps to be unmasked on Christmas Eve

2020-12-23 Thread Michael Orlitzky

On 12/23/20 12:13 PM, Jonathan Callen wrote:


One way this could be done without breaking things might be to create a
subdirectory of /usr/$(get_libdir) for each version of Lua and creating
a liblua.a and/or liblua.so symlink in that directory pointing to the
liblua${VERSION}.* file in /usr/$(get_libdir).  Then you could simply
point those build systems at that directory and they would still use the
correct version.  As a hack in an ebuild, you probably could just create
such a setup in (subdirectories of) ${T}, pointing liblua.a, liblua.so,
and lua.pc at the appropriately-versioned files.


All of these packages are masked, so breakage isn't an issue.

And since these hacks are needed in every package that will link against 
liblua, doesn't it sound like something the eclass should be doing? Even 
the CMake packages would benefit, no longer having to pass in 
-DLUA_INCLUDE_DIR and -DLUA_LIBRARY by hand.




Re: [gentoo-dev] Slotted Lua + revdeps to be unmasked on Christmas Eve

2020-12-23 Thread Michael Orlitzky

On 12/23/20 8:35 AM, Jaco Kroon wrote:

Michael,

I'm busy disecting what Marek has done for asterisk as I need to make
that work for multiple versions of net-misc/asterisk-16.14.0-r100, not
merely the one he did it for - perhaps that would be helpful for you as
well?

I don't know lua at all.

Anyway, I'm on IRC as jkroon if you'd like to exchange notes.  I don't
particularly like the patch Marek did as I'll NEVER be able to push that
upstream.  The patch will work for us, but as a policy I highly prefer
patches which I can get unstreamed such that we don't need to carry them
more than one or two versions.


Agreed. This is more of a system library packaging issue than anything 
to do with Lua specifically.




I'd prefer to patch upstream to keep it's behaviour of detecting a lua
version, but have a --with-lua(version)? type argument that can take an
optional argument which will then be the version, but --with-* generally
takes a prefix where the include and lib folders can be found ... and
I'd prefer to depend on pkg-config as primary and fallback to the
./configure mess which tries to guess at things ...



This is exactly what I'm trying to get across. Support for multiple 
versions of libraries in weird locations is not a new thing. Generally, 
you can put CPPFLAGS="-I/foo/bar" and LIBS="-L/baz/quux" before running 
./configure and everything will just work with your header files and 
libraries in the non-standard paths /foo/bar and /baz/quux, 
respectively. Those paths take precedence over the defaults (if used 
correctly...), so you can use them to override the default version of a 
library and compile/link against a specific copy if you need to do that. 
Likewise, one could prepend a path to PKG_CONFIG_PATH to override the 
version that will be detected via pkg-config.


The problem that we've gotten ourselves into is that we've copied Debian 
by not only relocating the library, but renaming it (and its *.pc file) 
entirely. There is no good way to tell an autotools build system that 
you've renamed a library completely, and that it should prefer the 
renamed version over the standard one if the standard one also exists.


If you want to build against the eselected version of Lua on Gentoo, 
then eselect-lua makes everything OK. It creates symlinks from the 
standard names to the nonstandard ones, and everything just works. But 
if you want to build against a specific, not-eselected version of Lua? 
You have to tell the build system what to do. The eselected version 
lives in the correct locations (via those symlinks), so the build system 
is going to want to find that first. If the eselected version is not the 
one you want to build against, oops. Then, since the version you 
*do* want to link against has the wrong name, you have to tell the build 
system to link against it somehow. You can't simply override the 
PKG_CONFIG_PATH, because our *.pc files have the wrong names. You can't 
just override the library search path, because our libraries have the 
wrong names.


The work I've done so far for OpenDKIM does nothing to address these 
larger problems. If lua.pc is on the system, it will get used first, 
regardless of the version you actually want to build against. AFAIK 
there's no good way around this without patching.


If the libraries (and pkg-config files?) were installed to 
subdirectories instead, then the standard method of overrides could be 
used to build against a specific version, rather than requiring every 
upstream build system to support our weird layout.




Re: [gentoo-dev] Slotted Lua + revdeps to be unmasked on Christmas Eve

2020-12-23 Thread Michael Orlitzky

On 12/23/20 4:09 AM, Marek Szuba wrote:


I think what you are looking for is lua_get_shared_lib() from
lua-utils.eclass. We have already got ebuilds in the tree which use
it.



Knowing the library name only helps if I patch the build system; that's 
what I'm getting at. The few packages where this works use CMake, and 
CMake already tries to guess[0] the name of the lua library (thanks 
Debian) and so it has a pre-defined variable to store the result.


Not all build systems are going to work like that. Suppose I know that 
the name of the lua library is "lua5.2". An autotools build system is 
going to run something like,


  AC_SEARCH_LIBS([whatever], [lua], [lua_found="yes"])

How do I pass the name "lua5.2" to that, without hacking configure.ac 
and running autoreconf? The only option that comes to mind is to build 
the entire project with LIBS="-llua5.2", but that's not right if only 
some of the executables are supposed to be linked with lua.


For contrast, if the lua library was stored in /usr/lib64/lua5.2, then I 
could just set LIBS="-L/usr/lib64/lua5.2" and let ./configure do its thing.



[0] https://github.com/Kitware/CMake/blob/master/Modules/FindLua.cmake



Re: [gentoo-dev] Slotted Lua + revdeps to be unmasked on Christmas Eve

2020-12-22 Thread Michael Orlitzky

On 12/22/20 6:15 PM, Marek Szuba wrote:

Dear all,

mail-filter/opendkim - committed ebuild needs one extra fix


One last design issue that I ran into during the migration.

The slotted lua ebuilds install the headers into subdirectories like 
/usr/include/lua5.2, but otherwise with their upstream names. The 
libraries, on the other hand, all get installed directly to $libdir.. 
but with NON-standard names like liblua5.2.a (as opposed to simply 
liblua.a).


This makes it impossible to build against a specific version of Lua 
without using pkg-config. The usual way would be to add 
/usr/include/lua5.2 to the compiler's include path, and $libdir/lua5.2 
to its library path (via -I and -L)... but there's no way to tell the 
build system to look for a library of an entirely different name. Things 
like AC_SEARCH_LIBS are going to try -llua, and that's it.




Re: [gentoo-dev] GPG key refresh

2020-12-15 Thread Michael Orlitzky

On 12/15/20 11:11 AM, Thomas Deutschmann wrote:


What do you mean exactly?

For Gentoo tooling, only Gentoo keyservers are important and Gentoo no longer 
synchronizes with any other pool.

"The Gentoo developer tooling explicitly checks the Gentoo keyserver 
pool with a much higher frequency" strongly implies that we check the 
non-Gentoo pools with a non-zero frequency.





Re: [gentoo-dev] GPG key refresh

2020-12-15 Thread Michael Orlitzky

On 12/15/20 10:27 AM, Thomas Deutschmann wrote:

 
https://wiki.gentoo.org/wiki/Project:Infrastructure/Generating_GLEP_63_based_OpenPGP_keys#Submit_your_new_key_to_the_keyserver



And FWIW this sentence is a little misleading if the SKS refresh 
frequency is zero =)


  The SKS keyserver pool can take much longer to replicate over the
  keyserver network, while the Gentoo developer tooling explicitly
  checks the Gentoo keyserver pool with a much higher frequency.



Re: [gentoo-dev] GPG key refresh

2020-12-15 Thread Michael Orlitzky

On 12/15/20 10:27 AM, Thomas Deutschmann wrote:

Hi,

what exactly did you do already?

Did you uploaded to our internal key server? You can only upload
through dev.gentoo.org, see
https://wiki.gentoo.org/wiki/Project:Infrastructure/Generating_GLEP_63_based_OpenPGP_keys#Submit_your_new_key_to_the_keyserver

 However, you can pull from this server.

I tried to test your key but I am currently getting a failure from
our keyserver. Waiting for infra to check.



It's working now. I hadn't pushed directly to the internal server, 
unaware that it was still internal. An update to the error message (git 
hook) that mentions that wiki page could mitigate similar headaches in 
the future. GLEP63 still says,


  == Bare minimum requirements ==
  ...
7. Upload your key to the SKS keyserver rotation before usage!

which should also be amended if there's no imminent plan to resume 
pulling keys from there.




Re: [gentoo-dev] GPG key refresh

2020-12-14 Thread Michael Orlitzky

On 12/14/20 2:17 PM, Alec Warner wrote:


I think you need to push to hkps://keys.gentoo.org 



Ok, did that. The error message specifically states,

  remote: *** None of your keys comply with GLEP 63.

and GLEP63 says to upload your keys to the SKS keyservers. Does the GLEP 
need an update? I guess we never resumed fetching from them after the 
incident?




[gentoo-dev] GPG key refresh

2020-12-14 Thread Michael Orlitzky

I'm still getting this,

  $ git push --signed
  ...
  remote: 1C49724D229E93A2 [Michael Orlitzky ] [E]
  expire:short Expiration date is too close, please renew (is 2020-12-26
  23:30:47, less than 14 days)
  remote: 1C49724D229E93A2:6F48D3DA05C2DADB [Michael Orlitzky
  ] [E] expire:short Expiration date is too close,
  please renew (is 2020-12-26 23:31:43, less than 14 days)

over a week after I've renewed my keys. The answer I get back from the 
keyserver(s) looks OK:


  $ gpg --search-keys 0x6F48D3DA05C2DADB
  gpg: data source: http://85.25.207.23:11371
  (1)   Michael Orlitzky 
Michael Orlitzky 
  4096 bit RSA key 0x1C49724D229E93A2, created: 2010-03-17,
  expires: 2022-12-05


Did I forget a step? How long should it take for infra to refresh?



Re: [gentoo-dev] Slotted Lua: 2020-12-04 status update

2020-12-09 Thread Michael Orlitzky

On 12/9/20 10:10 AM, Marek Szuba wrote:


LUA_DEPS itself will not but the change of LUA_SINGLE_TARGET in the
package in question will, same way other packages can be rebuilt on
USE-flag changes.



So lua has inherited the python approach of requiring everyone to use 
portage? =/




Re: [gentoo-dev] Slotted Lua: 2020-12-04 status update

2020-12-09 Thread Michael Orlitzky

On 12/7/20 9:11 AM, Marek Szuba wrote:

On 2020-12-04 13:16, Marek Szuba wrote:


Since a week ago the number of open bugs blocking the slotted-Lua
tracker has been reduced from 119 to under 80.


Updated count as of a few minutes ago: 64 open tickets! Full list:

https://dev.gentoo.org/~marecki/open_blocking_lua_eclass_bugs-20201207135752.txt



I think the slotted lua ebuilds should be doing some eselect-lua stuff 
in pkg_postinst() and pkg_postrm(). For example:


  * I have lua-5.1 and lua-5.2 installed
  * Lua-5.2 is eselected
  * I uninstall lua-5.2
  * Now /usr/lib64/pkgconfig/lua.pc is a dangling symlink

Related to the above: is there a way to trigger rebuilds of consumers 
when the "single implementation" changes? If I have a package that links 
against liblua and if I upgrade from lua-5.1 to lua-5.2, then my package 
should rebuild. I don't think $LUA_DEPS from lua-single.eclass can do that?




Re: [gentoo-dev] PSA: switching default tmpfiles virtual provider

2020-11-26 Thread Michael Orlitzky
On 11/26/20 5:57 PM, Thomas Deutschmann wrote:
> 
> I disagree here: Packages installing tmpfiles configs requiring
> recursive chown on each boot are doing something wrong from  my P.O.V.

No argument there, but me thinking they're wrong doesn't stop people
from doing it.


> Note that hardlinks aren't even fixed for systemd's tmpfiles provider.
> It will always rely on fs.protected_hardlinks for example. And checking
> for hardlinks like happened to address CVE-2017-18078 will create
> another TOCTOU race. Where is the follow-up report for this?

Systemd only supports Linux, and sets fs.protected_hardlinks=1 itself.
There's not much more we can ask from them.

I normally err on the side of caution, but if someone goes out of their
way to disable a security setting, I don't consider it CVE-worthy if the
thing that setting was preventing is now exploitable.


> In short: As long as it is possible for attacker to write to directory
> you are working on you can never do mentioned things in a safe way. You
> first have to revoke access for everyone except you and then you can
> start checking file per file... but *no* implementation is doing
> something like that.

This came up in the old (late 1990s, early 2000s) LKML discussions about
the protected_* sysctls. The Right Thing To Do is to drop privileges to
the user who owns the directory if you need to do stuff in a directory
that a user owns or can write to.

To quote your earlier message:

> Rule of thumb: Just make sure that you only create top level directories.

...and then drop privileges.





Re: [gentoo-dev] PSA: switching default tmpfiles virtual provider

2020-11-26 Thread Michael Orlitzky
On 11/26/20 5:37 PM, Peter Stuge wrote:
> Georgy Yakovlev wrote:
>> I'll be switching default tmpfiles provider to sys-apps/systemd-tmpfiles
>> by the end of the week by updating virtual/tmpfiles ebuild.
> 
> Michael Orlitzky wrote:
>> Corollary: the tmpfiles.d specification can only be implemented (safely)
>> on Linux after all.
> 
> So should virtual/tmpfiles differentiate based on system?
> 

There's no scenario where opentmpfiles is preferable.

systemd-tmpfiles with the fs.protected_hardlinks=1 sysctl is secure on
Linux. On other kernels, you're out of luck -- none of the options are
secure. Securing the service manager on other kernels would require
dropping tmpfiles entirely, and major changes to OpenRC.



Re: [gentoo-dev] PSA: switching default tmpfiles virtual provider

2020-11-26 Thread Michael Orlitzky
On 11/26/20 10:07 AM, Thomas Deutschmann wrote:
> 
> Only root is allowed to write to these directories. In other words: To
> exploit this, a malicious local user (or a remote attacker who already
> gained user access) would have to trick root into creating specially
> crafted tmpfiles config allowing for race conditions first (according to
> the 10 immutable laws of security, if this is already possible, you are
> already lost).

Most of these security issues were fixed in systemd-tmpfiles years ago,
and you can easily find upstream tmpfiles.d entries that contain e.g.
"Z" entries. In that case, the upstream file is not in error, and root
doesn't have to be actively tricked into installing anything -- it will
just happen.

Opentmpfiles literally cannot fix this. There is no POSIX API to safely
handle hardlinks. At best it can be reduced to the same race condition
we have in checkpath, but the entire project would have to be rewritten
in C to accomplish even that.

Corollary: the tmpfiles.d specification can only be implemented (safely)
on Linux after all.



Re: [gentoo-dev] [PATCH 0/1] depend.apache.eclass: support EAPI-7

2020-11-03 Thread Michael Orlitzky

On 11/3/20 11:25 AM, Marek Szuba wrote:

The fact this eclass does not support EAPI-7 yet blocks migration
of www-apache/mod_security to Lua eclasses. Seems simple enough to
address though, likely simpler than adding EAPI-6 support to lua.eclass.



It's likely broken in EAPI=7, because it was in EAPI=6:

  https://bugs.gentoo.org/616612

I left a comment there with a proposal for how to fix it, but I haven't 
thought about it much since then.




Re: [gentoo-dev] [infra] Anti-spam changes: removal of malware patrol and other older ClamAV rules

2020-09-11 Thread Michael Orlitzky
On 2020-09-11 15:09, Robin H. Johnson wrote:
> Hi,
> 
> As a result of a recent high-impact [1] false positive spam detection in
> Gentoo mail, we've disabled using the MalwarePatrol ruleset in Clamav
> for spam detection for all inbound mail to @gentoo.org
> 

All of these services produce killer false positives eventually. If
you're using amavisd-new, you can score them instead of reject outright:

  @virus_name_to_spam_score_maps =
(new_RE(
  [ qr'^MBL_.*' => 4.0 ],
  ));

That doesn't totally fix the problem, but if the message is otherwise
pristine (no blacklists, etc.) then a MalwarePatrol hit won't be fatal.




Re: [gentoo-dev] [PATCH] ebuild-maintenance/removal: Process for virtual removal

2020-09-07 Thread Michael Orlitzky
On 2020-09-07 08:47, Alessandro Barbieri wrote:
> Being consistent in decision is hard I see.

You're missing some context. In October of last year, a QA team member
broke dependency resolution on a lot of systems by making the same sort
of change that this patch proposes:

https://archives.gentoo.org/gentoo-dev/message/64c42804eb4cf4bc7d1161a2e9222c6a

Last month, someone brought up that example and named the QA team as
partly responsible for the --changed-deps requirement, which goes
against the PMS and a council decision:

https://archives.gentoo.org/gentoo-dev/message/dcebabbd6f13aed6622424d439f7becc

Shortly thereafter, another QA member opened a pull request that would
retroactively make what the first QA member did OK:

https://github.com/gentoo/devmanual/pull/177

And now, we are having a third QA team member in charge of approving
that change to the devmanual, which will later be cited as "policy."

Your problem is that you're not a member of the right gang.



Re: [gentoo-dev] [PATCH] ebuild-maintenance/removal: Process for virtual removal

2020-09-07 Thread Michael Orlitzky
On 2020-09-07 08:10, Ulrich Mueller wrote:
>> This has caused dependency resolution problems in the past. The PMS
>> implies a new revision,
> 
> PMS says nothing about new revisions or revision bumps:
> 

That is indeed what the word "implies" implies.


> The devmanual [1] says that a revbump should be done when a new runtime
> dependency is added to an ebuild, but it doesn't say that for removal of
> a dependency.
> 

One dependency is removed, and another is added. All completely besides
the point that this breaks things.



Re: [gentoo-dev] [PATCH] ebuild-maintenance/removal: Process for virtual removal

2020-09-07 Thread Michael Orlitzky
On 2020-09-07 02:14, Michał Górny wrote:
> +  
> +Update all ebuilds not to reference the virtual. Since there is
> +no urgent need to remove the virtual from user systems
> +and the resulting rebuilds would be unnecessary, do not bump ebuilds
> +when replacing the dependency.
> +  

This has caused dependency resolution problems in the past. The PMS
implies a new revision, the council said make a new revision, and the
devmanual already says make a new revision. Documenting this behavior
might make people feel better about ignoring all of that, but your lack
of guilt doesn't do me any good when portage starts crashing.



Re: [gentoo-dev] Please port your packages to Python 3.8

2020-09-04 Thread Michael Orlitzky
On 2020-09-04 09:22, Rich Freeman wrote:
> 
> Current Gentoo policy:
> 
> ...if the changes are likely to cause problems for end users." 

If you're willing to ignore the user reports of problems, and ignore the
mailing list threads telling you that it will cause problems, and avoid
running any tests, then it becomes possible to construct a fortress of
flawed reasoning to defend the claim that "this won't cause problems for
end users."



Re: [gentoo-dev] Please port your packages to Python 3.8

2020-09-04 Thread Michael Orlitzky
On 2020-09-04 08:54, Alexis Ballier wrote:
> 
> py37 will (*) still be installed as it cannot be depcleaned because of
> 1. emerge won't fail since deps are satisfied.
> 
> 
> (*) or rather should, but I think the only case that matters is a valid
> system state where noone forced uninstall of a needed package or
> manually rm'ed some random files
> 

There's no need to speculate; use pkgcore for a month and come back and
tell me how much comfort these hypotheticals were.


>> or..
>>
>>   3b. Some reverse dependency of foo-1.2.3 gets python-3.8 support.
>>   4b. A user tries to install that revdep, but the PM sees that
>>   the latest version of foo is already installed, and it (the
>>   installed version) doesn't support python-3.8. Mysterious
>>   error messages and manual intervention ensue.
> 
> 
> precisely the case I wrote above: unsatisfied dep -> pull ebuild.
> non-issue too.

It's easy to say "well this is not an issue because it can be solved by
..."

If it's easy, get it added to the PMS and I'll agree with you.



Re: [gentoo-dev] Re: Please port your packages to Python 3.8

2020-09-04 Thread Michael Orlitzky
On 2020-09-04 04:39, Martin Vaeth wrote:
> 
>> That's completely legal according to the PMS, and also the
>> smart thing to do:
> 
> s/smart/dumb/, but necessary for a dumb PM
Word games notwithstanding, these are the package managers described by
the PMS.


>> sourcing a few thousand lines of bash *just in case* there was an
>> important change in some ebuild is a huge waste of users' time.
> 
> That's wrong: The metadata has to be (re)generated anyway,
> independent of whether this is a revision bump or just a modification.
> For most syncing methods the bash sourcing does not happen on the
> user's machine, and on those machines where the metadata is actually
> calculated, there are checksums and timestamps used to minimize the
> effort. From this perspective there is no difference between a
> modification or a revision bump.

This is Hollywood accounting =)

Who generates the metadata when I `git pull`? Or in overlays? It's me,
but if you record that wasted time in a different "metadata generation"
ledger, then it looks like I've saved a bunch of time because the "bash
sourcing" ledger reads zero.

Even if I believe in a metadata angel and if we pretend that the PMS
requires the metadata to be there, then rebuilding whenever metadata
changes is still not 100% correct (as you point out), because it often
rebuilds pointlessly. But that's getting into a harder problem.


> ... costs thousands of users a lot of compilation time, although
> the recompilation is completely pointless for each of them, but
> done only to keep the PM simpler.

The recompilation isn't always pointless. In the present scenario, the
rebuild is what installs the python-3.8 copy of the program.

I'm not arguing that this is the best way to do things, but that it's
the only way to do them given what's in the PMS today. The foundation is
always looking for ways to spend money; maybe it should pay a few people
to sit around and think about this for a week. In any case, until a
better solution finds its way into the PMS, we should make the revisions.



Re: [gentoo-dev] Please port your packages to Python 3.8

2020-09-03 Thread Michael Orlitzky
On 2020-09-03 12:38, Alexis Ballier wrote:
> 
> if some upgrade wants a package with unmatched deps (e.g. not installed
> at all or py38 usedep not satisfied), $PM will surely try to satisfy
> it by installing an ebuild. I don't think PMS specifies this, nor should
> it.
> 

It's not an upgrade per se. The situation is roughly this:

  1. User installs foo-1.2.3.ebuild, which supports python-3.7.
  2. Developer ninja-edits the ebuild to support python-3.8.
  3a. (crickets)
  4a. Developer removes python-3.7 support from foo-1.2.3.ebuild.
  5a. The next time something pulls foo-1.2.3 into the depgraph,
  the PM will see that the installed version of foo-1.2.3 depends on
  python-3.7, but that there is no python-3.7 in the tree or that
  it's masked. Now emerge always fails.

or..

  3b. Some reverse dependency of foo-1.2.3 gets python-3.8 support.
  4b. A user tries to install that revdep, but the PM sees that
  the latest version of foo is already installed, and it (the
  installed version) doesn't support python-3.8. Mysterious
  error messages and manual intervention ensue.

What's happening is that the PM is using the metadata from the installed
version of the package, rather than the ninja-edited metadata in the
repo (how would it know which ebuilds were edited meaningfully?). That's
completely legal according to the PMS, and also the smart thing to do:
sourcing a few thousand lines of bash *just in case* there was an
important change in some ebuild is a huge waste of users' time.

Developers have an easy way to signal that there was an important
change: to bump the "r" number at the end of the ebuild. This forces any
upgrade involving e.g. foo-1.2.3 to pull in foo-1.2.3-r1, and the fact
that an upgrade is available is evident from `ls`, rather than from
sourcing a mountain of bash. One developer makes a change, and it saves
thousands of users each a lot of time. That's what we're all here for.

A package manager that uses the installed metadata is acting within its
rights (both pkgcore and portage without dynamic deps do it), so we
shouldn't be doing anything to break them in ::gentoo. Requiring
--changed-use is both insisting that users' waste time finding out
something that the developer could have told them, and also precluding
the use of a package manager that doesn't implement the (unspecified)
--changed-use flag in the same way portage does.

tl;dr the name of an ebuild is the only sensible (and PMS-compliant) way
to determine if something important changed inside it. The PMS says that
a new revision will be noticed by the PM, and it doesn't specify any
heuristics like --changed-use that ask the PM to (slowly) guess at it.
So, we should use revisions for these things.



Re: [gentoo-dev] Please port your packages to Python 3.8

2020-09-02 Thread Michael Orlitzky
On 2020-09-02 14:08, Andreas Sturmlechner wrote:
> On Wednesday, 2 September 2020 19:42:33 CEST Michael Orlitzky wrote:
>> New USE flags generally change dependencies (as is the case here), so a
>> new revision ensures that people are forced to install the ebuild that
>> supports python-3.8. Otherwise, you will eventually find a lot of people
>> stuck unable to upgrade because they have an ebuild installed that only
>> supports <=python-3.7, and were never prompted to install the copy that
>> supports python-3.8.
> 
> Python target changes must be done with -U, also documented by the 
> accompanying repository news item, not really a problem.
> 

If you want to write the GLEP that obsoletes the PMS, I might even
support it at this point. But until then, requiring --changed-use to
have a functional system is not allowed. Any PMS-compliant package
manager must be able to use ::gentoo, including one that does not
implement portage-only heuristics.



Re: [gentoo-dev] Please port your packages to Python 3.8

2020-09-02 Thread Michael Orlitzky
On 2020-09-02 13:23, Sam James wrote:
> 
> Please request stabilisation of your Python 3.8+ changes at the 30 days 
> point, or earlier if it’s a trivial revbump
> (as new Python targets are equivalent to new USE flags, there is no real need 
> for a revbump unless doing other tidying
> whilst there).
> 

New USE flags generally change dependencies (as is the case here), so a
new revision ensures that people are forced to install the ebuild that
supports python-3.8. Otherwise, you will eventually find a lot of people
stuck unable to upgrade because they have an ebuild installed that only
supports <=python-3.7, and were never prompted to install the copy that
supports python-3.8.



Re: [gentoo-dev] RFC: pgo - a command line interface for packages.g.o

2020-09-01 Thread Michael Orlitzky


On 2020-09-01 08:38, Max Magorsch wrote:
> Do you think this is something you would be interested in? Do you
> have any features you like to see included in this case?
Yes!

Here's a trick that bugzilla cannot do: show me the bugs that are
assigned to me **or a project that I'm a member of**. Killer feature
right there.



[gentoo-portage-dev] [PATCH 1/1] Revert "repoman: deprecate netsurf.eclass."

2020-08-14 Thread Michael Orlitzky
This reverts commit a73024729860f9224b8d1660d24c450080b67d9f. This
eclass was successfully purged from the tree, so the deprecation is no
longer needed. And eventually, to address an eblit infestation,
another eclass with the same name will return. The new one will not be
deprecated.

Signed-off-by: Michael Orlitzky 
---
 repoman/lib/repoman/modules/linechecks/deprecated/inherit.py | 1 -
 1 file changed, 1 deletion(-)

diff --git a/repoman/lib/repoman/modules/linechecks/deprecated/inherit.py 
b/repoman/lib/repoman/modules/linechecks/deprecated/inherit.py
index 60410347b..5848a0c37 100644
--- a/repoman/lib/repoman/modules/linechecks/deprecated/inherit.py
+++ b/repoman/lib/repoman/modules/linechecks/deprecated/inherit.py
@@ -33,7 +33,6 @@ class InheritDeprecated(LineCheck):
"gst-plugins10": "gstreamer",
"ltprune": False,
"mono": "mono-env",
-   "netsurf": False,
"python": "python-r1 / python-single-r1 / python-any-r1",
"ruby": "ruby-ng",
"user": "GLEP 81",
-- 
2.26.2




Re: [gentoo-dev] xorg-x11 RDEPEND changes without revisions

2020-08-07 Thread Michael Orlitzky
On 2020-08-07 14:43, Toralf Förster wrote:
> On 8/7/20 8:25 PM, Michael Orlitzky wrote:
>>
>> I have too many other things to do to waste time reverse-engineering
>> these fuck-ups. Get it together.
> 
> I'm just curious if you refer to commit d8c442ba8 - b/c that was made by 
> someone at Wed Oct 16 19:41:02 2019 + and I do wonder why nobody else run 
> into that issue since that time?
> 

Beats me. I run a stable system, so probably it just needed the right
combination of packages to stabilize. Or maybe it was the change in
xorg-2.eclass that triggered it. Or maybe no one else has noticed that
there's a useless package on his system (that will be stranded until all
of the affected packages get upgrades/revisions) and tried to remove it.
Or maybe everyone has set USE="-gentoo-dev" for portage to avoid this
perpetual clown show. Or...

what difference does it make? It already took me 100x longer to figure
out what went wrong than it would have taken to make the revisions in
the first place. I don't want to waste any more. I'm still tracking down
more packages that need to be rebuilt by hand because RDEPEND was
changed in an eclass too.



[gentoo-dev] xorg-x11 RDEPEND changes without revisions

2020-08-07 Thread Michael Orlitzky
When you ignore the devmanual and the pkgcheck warning and the 10+
threads I've started about the issue, and make changes like...

  --- a/x11-base/xorg-x11/xorg-x11-7.4-r3.ebuild
  +++ b/x11-base/xorg-x11/xorg-x11-7.4-r3.ebuild
  @@ -1,4 +1,4 @@
  -# Copyright 1999-2018 Gentoo Foundation
  +# Copyright 1999-2019 Gentoo Authors
   # Distributed under the terms of the GNU General Public License v2

   EAPI=6
  @@ -21,8 +21,7 @@ RDEPEND="${RDEPEND}
  x11-apps/bitmap
  x11-apps/iceauth
  x11-apps/luit
  -   x11-apps/mkfontdir
  -   x11-apps/mkfontscale
  +   >=x11-apps/mkfontscale-1.2.0
  x11-apps/sessreg


This is what portage does:

  $ sudo emerge -uDN @world
  Password:
  Calculating dependencies... done!

  emerge: there are no ebuilds to satisfy "x11-apps/mkfontdir".
  (dependency required by "x11-base/xorg-x11-7.4-r3::gentoo"
  [installed])
  (dependency required by "@selected" [set])
  (dependency required by "@world" [argument])


I have too many other things to do to waste time reverse-engineering
these fuck-ups. Get it together.



Re: [gentoo-dev] How to set CXX compiler?

2020-07-03 Thread Michael Orlitzky
On 2020-07-03 18:35, Xianwen Chen (陈贤文) wrote:
> 
> In the Makefile, it is written that
> 
> cc = g++
> 
> I would like to use sed to patch it so that it ebuild knows which g++ to
> use. For example, depending on whether ccache is set, ebuild shall know
> whether to use ccache.

First, you should suggest to upstream that they don't force a particular
compiler in their Makefile. One uncontroversial way to do this is by
setting the default only if the user does not already have one set, with
the "?=" assignment operator:

  cc ?= g++

Then, you should suggest that they change the name of that variable to
the relatively-standard "CXX" that is used for the C++ compiler:

  CXX ?= g++

Finally, in your ebuild, you should run

  emake CXX=$(tc-getCXX) ...

to override the default CXX in the Makefile. Even if upstream doesn't
respond right away, you can patch the Makefile with these suggestions in
Gentoo and send them the patch.



Re: [gentoo-dev] RFC: Standard build environment variables

2020-07-01 Thread Michael Orlitzky
On 2020-06-30 12:22, Matthew Thode wrote:
> 
> I'd like to suggest allowing only approved variables in the build
> environment, having portage unset all variables and setting only what is
> needed (or configured).

I think this is orthogonal to the problem I'm trying to solve. Even if
all environment variables had to be whitelisted, ebuilds would still
need to know how to use them when they happen to be defined.

I basically just want to write down things like "If set, CC is assumed
to contain the name of a compiler driver such as /usr/bin/gcc." That way
ebuilds can be written to pass $CC to the build system in places that
are expecting a compiler driver. Conversely, if LD is documented to
contain a dynamic linker such as /bin/ld, then ebuilds must mangle LD
whenever the upstream build system (e.g. pari, perl) interprets it
otherwise.

These meanings are already enshrined in the tc-getFOO() functions and
the various de-facto standards, but there's no user or developer
documentation promising that the variables will be used in any
particular way.



[gentoo-dev] RFC: Standard build environment variables

2020-06-28 Thread Michael Orlitzky
As many of you probably know, ago@ has been expanding the scope of our
CFLAGS/CC support to include some other common build variables:

  * CC
  * CXX
  * AR
  * CPP
  * NM
  * RANLIB
  * AS
  * LD

Some of those are POSIX standards[0],

  * CC
  * AR

Others are de-facto GNU make standards[1],

  * CXX
  * CPP
  * AS

and a few are de-facto GNU libtool standards[2]:

  * NM
  * RANLIB
  * LD

If we expect them all to work properly in Gentoo, we have to agree on
what they mean, and thus how they should be injected into build systems.
For example, we had a problem with sci-mathematics/pari, whose upstream
is using the LD environment variable for something other than what GNU
libtool uses it for. With LD set to something libtooly in the
environment, the pari build fails. We can solve that by unsetting LD in
the ebuild, but for that to be The Right Thing To Do, we should be
expecting LD to contain something libtooly, and thus something
inappropriate to be passed to the pari build.

To avoid these issues, I suggest creating a list of "Gentoo environment
variables" in the devmanual with descriptions of how they should be used
and pointers to the references (for why we chose that meaning). That way
a user can export LD, for example, and know that it will be used how he
thinks it will be used.


[0] https://pubs.opengroup.org/onlinepubs/009695399/utilities/make.html

[1]
https://www.gnu.org/software/make/manual/html_node/Implicit-Variables.html

[2] https://www.gnu.org/software/libtool/manual/libtool.html



Re: [gentoo-dev] */*: Mask Py2 only packages

2020-06-25 Thread Michael Orlitzky
On 2020-06-24 16:08, Michał Górny wrote:
> 
> $ git grep -l mgo...@gentoo.org '**/metadata.xml' | cut -d/ -f1-2 |
> xargs gpy-py2 2>/dev/null
> 

The big problem with this is that it misses any aliases (like graphics@)
that you're a member of. But let's golf; this is POSIX sh, doesn't use
grep to parse XML, and takes the maintainer's email address as an argument:

REPO=/var/db/repos/gentoo
XPATH="/pkgmetadata/maintainer/email[normalize-space(text()) = '${1}']"

find -L "${REPO}" \
  -mindepth 3 \
  -maxdepth 3 \
  -name 'metadata.xml' \
  -exec sh -c "
for f in \"\${@}\"; do
  xmllint --xpath \"${XPATH}\" \"\${f}\" >/dev/null 2>&1 && \
echo \"\$(dirname -- \"\${f}\")\" | sed \"s:${REPO%/}/::\"
done
  " - {} +



[gentoo-portage-dev] [PATCH 1/1] repoman: deprecate netsurf.eclass.

2020-06-16 Thread Michael Orlitzky
While investigating bug 489542, it became clear that the netsurf
eclass is deprecated in spirit if not in fact. All of the newer
netsurf ebuilds use a custom function loaded from a script that is
shipped in netsurf-buildsystem's $FILESDIR to do (some of) what the
eclass used to do. That function probably does belong in an eclass,
but at this point, we should throw this thing out and start from
scratch.

By deprecating the eclass, we make sure that no one else inherits it
during the time it takes to purge the two remaining consumers. Then,
once those are gone, the build system script magic can be put back
into an eclass, and its consumers updated one-at-a-time.

Bug: https://bugs.gentoo.org/489542
Bug: https://bugs.gentoo.org/637824
Signed-off-by: Michael Orlitzky 
---
 repoman/lib/repoman/modules/linechecks/deprecated/inherit.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/repoman/lib/repoman/modules/linechecks/deprecated/inherit.py 
b/repoman/lib/repoman/modules/linechecks/deprecated/inherit.py
index 5848a0c37..60410347b 100644
--- a/repoman/lib/repoman/modules/linechecks/deprecated/inherit.py
+++ b/repoman/lib/repoman/modules/linechecks/deprecated/inherit.py
@@ -33,6 +33,7 @@ class InheritDeprecated(LineCheck):
"gst-plugins10": "gstreamer",
"ltprune": False,
"mono": "mono-env",
+   "netsurf": False,
"python": "python-r1 / python-single-r1 / python-any-r1",
"ruby": "ruby-ng",
"user": "GLEP 81",
-- 
2.26.2




[gentoo-dev] Re: [PATCH] netsurf.eclass: remove EROOT from PREFIX

2020-06-16 Thread Michael Orlitzky
On 2020-06-14 16:30, a...@freemail.hu wrote:
> 
> Suggested fix for: https://bugs.gentoo.org/show_bug.cgi?id=489542
> Bug 489542 - netsurf.eclass should not include EROOT in PREFIX
> 

Well, I've applied this as well as some other fixes for the eclass, only
to find that the problem has been relocated from netsurf.eclass into
/usr/share/netsurf-buildsystem/gentoo-helpers.sh (which it looks like is
just an out-of-tree eclass?).

There are only two ebuilds left in the tree using netsurf.eclass:

  * dev-libs/libparserutils-0.2.3 (superseded by v0.2.4-r1)
  * net-libs/libhubbub-0.3.3 (superseded by v0.3.6)

The first one needs a stabilization, but then the old versions can be
removed and netsurf.eclass can be last-rited. That leaves us with a
similar problem in gentoo-helpers.sh, which does...

  NSSHARED="${EROOT}"/usr/share/netsurf-buildsystem
  LIBDIR="$(get_libdir)"
  PREFIX="${EROOT}/usr"

Digging into the netsurf buildsystem it looks like PREFIX is already
pre/appended to the uses of NSSHARED and DESTDIR. So I think we should
have instead,

  LIBDIR="$(get_libdir)"
  PREFIX="${EPREFIX}/usr"

and then the ebuilds themselves need to be fixed. For example,

  _emake TARGET=framebuffer DESTDIR="${ED}" install

should have DESTDIR="${D}" instead, because the buildsystem already
installs to $(DESTDIR)$(PREFIX). As it is, you're getting the "E" twice.

Removing the EROOT (as in the proposed patch) would also fix this, but
the standard pattern for ebuilds is to use the "E" variables at
configure-time, and only $D at install-time.



Re: [gentoo-portage-dev] [PATCH] Change BINPKG_COMPRESS default from bzip2 to xz

2020-04-26 Thread Michael Orlitzky
On 4/26/20 3:25 PM, Ulrich Mueller wrote:
>>>>>> On Sun, 26 Apr 2020, Michael Orlitzky wrote:
> 
>> Fuel for the fire:
> 
>>  * https://www.nongnu.org/lzip/lzip_benchmark.html
>>  * https://www.nongnu.org/lzip/xz_inadequate.html
> 
> Yep. That's why lzip is the dominant compression format now, and nobody
> is using xz any more.
> 

If you believed that argument, you would be using Windows =P

Dude has GRAPHS, it must be true.



Re: [gentoo-portage-dev] [PATCH] Change BINPKG_COMPRESS default from bzip2 to xz

2020-04-26 Thread Michael Orlitzky
On 4/26/20 12:55 PM, Matt Turner wrote:
> Bug: https://bugs.gentoo.org/715108
> Signed-off-by: Matt Turner 
> ---
> Strawman patch. Bikeshed away.
> 

Fuel for the fire:

 * https://www.nongnu.org/lzip/lzip_benchmark.html
 * https://www.nongnu.org/lzip/xz_inadequate.html



  1   2   3   4   5   6   7   8   9   >