Re: [gentoo-dev] [PATCH 1/3] dune.eclass: restrict strip

[Alexis Ballier]

On Mon, 2021-10-11 at 06:07 +0100, Sam James wrote:
> This breaks at least ocamlopt.
> 
> Closes: https://bugs.gentoo.org/803047
> Closes: https://bugs.gentoo.org/811315
> Signed-off-by: Sam James 
> ---
>  eclass/dune.eclass | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/eclass/dune.eclass b/eclass/dune.eclass
> index 02a8a870ef43e..8a6e01893932a 100644
> --- a/eclass/dune.eclass
> +++ b/eclass/dune.eclass
> @@ -28,6 +28,9 @@ esac
>  # Do not complain about CFLAGS etc since ml projects do not use
> them.
>  QA_FLAGS_IGNORED='.*'
>  
> +# Breaks with ocamlopt
> +RESTRICT="strip"
> +



err this is a terrible idea to do at such a large scale, esp. since
nowadays you can use dostrip -x and provide proper comments as to what
and why something breaks when stripped on a case by case basis

Re: [gentoo-dev] Re: [PATCH 1/2] acct-group.eclass: declare the missing dependency on shadow

[Alexis Ballier]

On Wed, 9 Sep 2020 09:48:04 -0400
David Michael  wrote:

> On Wed, Sep 9, 2020 at 5:37 AM Alexis Ballier 
> wrote:
> > On Tue, 8 Sep 2020 15:54:14 -0400
> > David Michael  wrote:
> >
> > > Hi,
> > >
> > > This fix might not be so straightforward.  A configuration I
> > > tested hit a dependency loop with shadow -> pambase -> systemd ->
> > > a bunch of groups -> shadow.  It is possible to bootstrap around
> > > by emerging shadow with no USE flags first, but I don't know how
> > > acceptable it is to introduce new dep loops like this.
> >
> > what happens without your change ?
> 
> Someone reported an issue in https://bugs.gentoo.org/720948 that
> showed shadow and groups are not ordered during installation.  I am
> not sure what environment produced those symptoms since I never
> encountered it, but you can rage-clean shadow and a group, delete the
> group, then reinstall it to reproduce the problem.
> 

Yep, that's exactly why one needs the change you are proposing.
The dependency loop needs to be resolved, but introducing it like that
is IMHO better than failing like in the above bug because it is not
resolved properly.

Re: [gentoo-dev] Re: [PATCH 1/2] acct-group.eclass: declare the missing dependency on shadow

[Alexis Ballier]

On Tue, 8 Sep 2020 15:54:14 -0400
David Michael  wrote:

> Hi,
> 
> This fix might not be so straightforward.  A configuration I tested
> hit a dependency loop with shadow -> pambase -> systemd -> a bunch of
> groups -> shadow.  It is possible to bootstrap around by emerging
> shadow with no USE flags first, but I don't know how acceptable it is
> to introduce new dep loops like this.


what happens without your change ?

my understanding is that this will be merged in that order:

1. a bunch of groups
2. systemd
3. pambase
4. shadow


in which case, the groups from 1. will fail if shadow is not present at
that point


PS: we have the 'build' useflag for this when building stage1's

Re: [gentoo-dev] [PATCH] ebuild-maintenance/removal: Process for virtual removal

[Alexis Ballier]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On Mon, 7 Sep 2020 21:39:54 +1200
Kent Fredric  wrote:

> On Mon,  7 Sep 2020 08:14:52 +0200
> Michał Górny  wrote:
> 
> > However, please
> > +do not include it in the package.mask entry as users do
> > not need
> > +to be forced to proactively unmerge it.
> 
> I can think of a utilitarian value of doing this anyway.
> 
> Namely, it gives a window during `emerge -uD @world` where portage
> tells you that they have a masked package installed, and the reason.

I think portage also warns for installed packages with no corresponding
ebuild; the reason is somewhat generic though (sth like 'it is gone')

IMHO last rites windows have only one purpose: give time to people to
step up and fix the reason why it is going away
-BEGIN PGP SIGNATURE-

iHUEAREIAB0WIQSpOxaxaZikKNVNlsYOJUi7xgflrgUCX1YK/gAKCRAOJUi7xgfl
rrooAQCBK/WXgwMl1wm8jQh+e2oyD8WIAzSrzFFBCE7xWL1WGgEAgWSGBmG9ug8/
ZNfrHnOhBL2o6hZupzMy/84dX7DFBvk=
=pwzN
-END PGP SIGNATURE-

Re: [gentoo-dev] Please port your packages to Python 3.8

[Alexis Ballier]

On Fri, 4 Sep 2020 09:06:46 -0400
Michael Orlitzky  wrote:

> On 2020-09-04 08:54, Alexis Ballier wrote:
> > 
> > py37 will (*) still be installed as it cannot be depcleaned because
> > of 1. emerge won't fail since deps are satisfied.
> > 
> > 
> > (*) or rather should, but I think the only case that matters is a
> > valid system state where noone forced uninstall of a needed package
> > or manually rm'ed some random files
> > 
> 
> There's no need to speculate; use pkgcore for a month and come back
> and tell me how much comfort these hypotheticals were.

there's no speculation in assuming a consistent set of installed
packages (consistent as specified in... PMS ;) ); there is, however,
speculation in the hypothetical error messages when the installed set
of packages is inconsistent


> >> or..
> >>
> >>   3b. Some reverse dependency of foo-1.2.3 gets python-3.8 support.
> >>   4b. A user tries to install that revdep, but the PM sees that
> >>   the latest version of foo is already installed, and it (the
> >>   installed version) doesn't support python-3.8. Mysterious
> >>   error messages and manual intervention ensue.
> > 
> > 
> > precisely the case I wrote above: unsatisfied dep -> pull ebuild.
> > non-issue too.
> 
> It's easy to say "well this is not an issue because it can be solved
> by ..."

are you kidding ? are you seriously suggesting adding to PMS that PM
needs to pull ebuilds to install packages ? good luck with that ;)

Re: [gentoo-dev] Please port your packages to Python 3.8

[Alexis Ballier]

On Thu, 3 Sep 2020 14:17:06 -0400
Michael Orlitzky  wrote:

> On 2020-09-03 12:38, Alexis Ballier wrote:
> > 
> > if some upgrade wants a package with unmatched deps (e.g. not
> > installed at all or py38 usedep not satisfied), $PM will surely try
> > to satisfy it by installing an ebuild. I don't think PMS specifies
> > this, nor should it.
> > 
> 
> It's not an upgrade per se. The situation is roughly this:
> 
>   1. User installs foo-1.2.3.ebuild, which supports python-3.7.
>   2. Developer ninja-edits the ebuild to support python-3.8.
>   3a. (crickets)
>   4a. Developer removes python-3.7 support from foo-1.2.3.ebuild.
>   5a. The next time something pulls foo-1.2.3 into the depgraph,
>   the PM will see that the installed version of foo-1.2.3 depends
> on python-3.7, but that there is no python-3.7 in the tree or that
>   it's masked. Now emerge always fails.


py37 will (*) still be installed as it cannot be depcleaned because of
1. emerge won't fail since deps are satisfied.


(*) or rather should, but I think the only case that matters is a valid
system state where noone forced uninstall of a needed package or
manually rm'ed some random files

> 
> or..
> 
>   3b. Some reverse dependency of foo-1.2.3 gets python-3.8 support.
>   4b. A user tries to install that revdep, but the PM sees that
>   the latest version of foo is already installed, and it (the
>   installed version) doesn't support python-3.8. Mysterious
>   error messages and manual intervention ensue.


precisely the case I wrote above: unsatisfied dep -> pull ebuild.
non-issue too.


> What's happening is that the PM is using the metadata from the
> installed version of the package, rather than the ninja-edited
> metadata in the repo (how would it know which ebuilds were edited
> meaningfully?). That's completely legal according to the PMS, and
> also the smart thing to do: sourcing a few thousand lines of bash
> *just in case* there was an important change in some ebuild is a huge
> waste of users' time.

you seem to be confusing dynamic deps and unsatisfied deps here. A
package installed with py38 disabled (e.g. after a revbump) or no py38
support at all (e.g. without revbump) will not satisfy a [py38] usedep
in any case so will work just the same


> Developers have an easy way to signal that there was an important
> change: to bump the "r" number at the end of the ebuild. This forces
> any upgrade involving e.g. foo-1.2.3 to pull in foo-1.2.3-r1, and the
> fact that an upgrade is available is evident from `ls`, rather than
> from sourcing a mountain of bash. One developer makes a change, and
> it saves thousands of users each a lot of time. That's what we're all
> here for.

fixing non-issues does not seem so important to me :/

[...]

Re: [gentoo-dev] Please port your packages to Python 3.8

[Alexis Ballier]

On Wed, 2 Sep 2020 15:00:27 -0400
Michael Orlitzky  wrote:

> On 2020-09-02 14:08, Andreas Sturmlechner wrote:
> > On Wednesday, 2 September 2020 19:42:33 CEST Michael Orlitzky wrote:
> >> New USE flags generally change dependencies (as is the case here),
> >> so a new revision ensures that people are forced to install the
> >> ebuild that supports python-3.8. Otherwise, you will eventually
> >> find a lot of people stuck unable to upgrade because they have an
> >> ebuild installed that only supports <=python-3.7, and were never
> >> prompted to install the copy that supports python-3.8.
> > 
> > Python target changes must be done with -U, also documented by the 
> > accompanying repository news item, not really a problem.
> > 
> 
> If you want to write the GLEP that obsoletes the PMS, I might even
> support it at this point. But until then, requiring --changed-use to
> have a functional system is not allowed. Any PMS-compliant package
> manager must be able to use ::gentoo, including one that does not
> implement portage-only heuristics.
> 

?

if some upgrade wants a package with unmatched deps (e.g. not installed
at all or py38 usedep not satisfied), $PM will surely try to satisfy
it by installing an ebuild. I don't think PMS specifies this, nor should
it.

Re: [gentoo-dev] Improving warnings on packages.g.o

[Alexis Ballier]

On Wed, 2020-08-26 at 18:57 +, Max Magorsch wrote:
> Hi all,
> 
> Good news regarding packages.g.o!
> 
> While the new packages.g.o version went into production some time
> ago,
> this also led to some false warnings about outdated package versions.
> This is because we currently take information about outdated package
> versions from repology.org, which are not always accurate.
> 


Any plans on using the remoteid from metadata.xml ?
It's likely to be much more accurate since people have been filling
those manually for some time now ;)

Alexis

Re: [gentoo-dev] rfc: switching default udev provider for new systems to udev

[Alexis Ballier]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On Sat, 8 Aug 2020 13:51:41 -0500
William Hubbs  wrote:

> All,
> 
> I would like to propose that we switch the default udev provider on
> new systems from eudev to udev.
> 
> This is not a lastrites, and it will not affect current systems since
> they have to migrate manually. Also, this change can be overridden at
> the profile level if some profile needs eudev (the last time I
> checked, this applies to non-glibc configurations).
> 
> What do people think?

No opinion on which to choose, I use the default one at the time I do
an install and have been happy with both.

My main concern is that since the change won't be "live" until a
switched virtual reaches stable, eudev will still be much better tested
in our environment at this point, and people might uncover corner cases
when it's too late. Maybe a compromise could be to provide and
primarily advertise udev stages before making the switch ?

Alexis.
-BEGIN PGP SIGNATURE-

iHUEAREIAB0WIQSpOxaxaZikKNVNlsYOJUi7xgflrgUCXzFsKAAKCRAOJUi7xgfl
rkDGAP9no3aFUEIPFr3mPHp9lUmIk7ZUl+njCpQo0+GsgoFVuQD+OG2zf3SVSOPs
hrYNa/PYEHKujS/Rfk2m180it41yDwM=
=/0De
-END PGP SIGNATURE-

Re: [gentoo-dev] Last rites: */*: More Py2 stuff

[Alexis Ballier]

On Wed, 29 Jul 2020 10:03:47 -0400
Aaron Bauman  wrote:

> Adjust the mask, drop the ebuild, or simply remove the mask. I would
> happily apologize for a mistake, but reverting something that is
> largely not in error seems silly. 
> 
> Again, this is a massive commit, but it should be the last time. Look
> at the previous sets of masks... impact vs inconvenience was pretty
> low. 


I think you've been told several times already, but impacting users
with a mask (that can't do anything useful about it) and not bothering
to file bugs for developers (that *can* do something about it) is not
the proper way to achieve anything here.

I believe there are a few quiz questions that closely relate to that
kind of impact.

Re: [gentoo-dev] [PATCH 2/2] kernel-install.eclass: Warn about linux-firmware in pkg_pretend()

[Alexis Ballier]

On Wed, 17 Jun 2020 10:58:03 -0400
Mike Gilbert  wrote:

> On Wed, Jun 17, 2020 at 7:42 AM Ulrich Mueller  wrote:
> >
> > > On Wed, 17 Jun 2020, Michał Górny wrote:
> >
> > > Can we please put users above silly politics?  Gentoo 'does not
> > > depend' on any non-free package to print the warning.  If people
> > > have hardware that requires non-free firmware, the least we can
> > > do is point out where to get this firmware from.
> >
> > s/If/If and only if/ and I'll be fine with it. :)
> 
> Are you proposing that the ebuild inspect the user's hardware and/or
> kernel config before printing a warning? That sounds like a terrible
> idea to me. Who is going to maintain the problematic hardware list?

The kernel. Try 'modinfo -F firmware iwlwifi'.
e.g. something like that could work:
modinfo -F firmware $(lsmod | sed '1d' | awk '{print $1}')


or if there is a need to be extra protective, iterate over all
installed modules.

Re: [gentoo-dev] [PATCH 2/2] kernel-install.eclass: Warn about linux-firmware in pkg_pretend()

[Alexis Ballier]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On Wed, 17 Jun 2020 13:08:56 +0200
Michał Górny  wrote:

> On Wed, 2020-06-17 at 12:57 +0200, Ulrich Mueller wrote:
> > > > > > > On Wed, 17 Jun 2020, Michał Górny wrote:
> >  
> > > +# @FUNCTION: kernel-install_pkg_pretend
> > > +# @DESCRIPTION:
> > > +# Check for missing optional dependencies and output warnings.
> > > +kernel-install_pkg_pretend() {
> > > + debug-print-function ${FUNCNAME} "${@}"
> > > +
> > > + if ! has_version -d sys-kernel/linux-firmware; then
> > > + ewarn "sys-kernel/linux-firmware not found
> > > installed on your system."
> > > + ewarn "This package provides various firmware
> > > files that may be needed"
> > > + ewarn "for your hardware to work.  If in doubt,
> > > it is recommended"
> > > + ewarn "to pause or abort the build process and
> > > install it before"
> > > + ewarn "resuming."
> > > +
> > > + if use initramfs; then
> > > + elog
> > > + elog "If you decide to install
> > > linux-firmware later, you can rebuild"
> > > + elog "the initramfs via issuing a
> > > command equivalent to:"
> > > + elog
> > > + elog "emerge --config
> > > ${CATEGORY}/${PN}"
> > > + fi
> > > + fi
> > > +}
> > 
> > Should we really warn about a package that (in its default
> > configuration) can only be installed if the user accepts non-free
> > licenses?
> 
> That's one of the reasons it's only a warning and not a USE flag.
> 
> > I would think that even without such a warning, users will be well
> > aware if some devices of their system will need additional
> > firmware. Also, some people prefer the separate packages from
> > sys-firmware which tend to be more lightweight (though I am aware
> > that some of them may be considered legacy packages).
> > 
> 
> This has been requested by users, some of whom apparently forget that
> they need to manually install one more package for their system to
> even boot.  If it saves a few people from having to go through
> recovery, it's worth it.
> 

You're probably better with a proper check like what debian's scripts
do:
https://salsa.debian.org/kernel-team/initramfs-tools/-/blob/master/hook-functions#L101

so that there is only a warning when the real file is missing; this
would also handle cases with non-linux-firmware firmwares or
USE=savedconfig there
-BEGIN PGP SIGNATURE-

iHUEAREIAB0WIQSpOxaxaZikKNVNlsYOJUi7xgflrgUCXun+HwAKCRAOJUi7xgfl
rlAIAP97XGdJwGvbWcDwMfZ5lptj/TRdhS77mzCAl2OY2CZ12gD+P8Csx0bnQnk0
4NvZwkoBl+fFiaGWvUFkva08A8MKOoo=
=ayj9
-END PGP SIGNATURE-

Re: [gentoo-dev] [RFC] Codec project

[Alexis Ballier]

On Fri, 12 Jun 2020 10:58:24 -0400
Rich Freeman  wrote:

> On Fri, Jun 12, 2020 at 10:33 AM Alexis Ballier 
> wrote:
> >
> > What about /j #gentoo-media, discuss, join the current projects,
> > get a few things done (there is a lot of choice there ;) ), maybe
> > orphan unmaintained players/viewers, or check if they are
> > maintained and hand them to a specific maintainer, and then see
> > about merging or splitting all those projects *after* gaining
> > experience and knowledge about the peculiarities of some of those
> > packages ?
> >
> 
> Probably best to leave the details up to those doing the work, but I
> would suggest this as a guideline:
> 
> Keep the scope reasonable.  If you expand the scope to a point where
> 90% of the project members don't care about 50% of the project scope,
> then you're setting yourself up for a repeat of the past.  You want
> 80% of the project members to be interested in 80% of the packages
> being maintained ideally.  Sure, there will be little niches that a
> subset are more interested in, but you want to keep it focused around
> a core where coordination makes sense.  You can have different roles
> in the project but it should still be one project.

Totally agree here. Back in the days we had split proaudio from sound
for this very reason.

> If most of the project members aren't talking to each other about most
> of the things they're doing in the project, then it isn't really a
> project - it is just a category tag.  The point of a project is to
> coordinate things that actually NEED to be coordinated or at least
> benefit from it in some way.

It is not like a KDE, gnome or gstreamer project that has very tight
coordination needs, so we shouldn't judge those on the same grounds, but
from time to time, when a core library changes its API it needs a
project-wide coordination (plus a few extra revdep here and there that
you get to know over time). That's more how I see coordination there.
It is not as critical nor as frequent as it used to be but still
happens from time to time. Having a codec project goes against that
IMHO, we'd end up with a category tag where there's no relation between
any of the package except they're media libraries.

Alexis.

Re: [gentoo-dev] [RFC] Codec project

[Alexis Ballier]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On Wed, 10 Jun 2020 20:25:20 +0200
Michał Górny  wrote:

> Hi,
> 
> Let's split this from [1] as I suppose having it in middle of
> high-noise 'up for grabs' might prevent some interested people from
> seeing it.
> 
> The general purpose of codec project [2] is to maintain core libraries
> for various multimedia format encoder/decoder libraries.  It's like
> gfx+sound+video except only for core packages and not every possible
> single viewer, player, editor, frontend...  

Not that I'm against the idea, but seeing the project has 3 members
already and none of them were part of the gfx+sound+video projects
AFAIK, I am wondering if this is the proper way to do it.

What about /j #gentoo-media, discuss, join the current projects, get a
few things done (there is a lot of choice there ;) ), maybe orphan
unmaintained players/viewers, or check if they are maintained and hand
them to a specific maintainer, and then see about merging or splitting
all those projects *after* gaining experience and knowledge about the
peculiarities of some of those packages ?

Speaking about sound, I am under the impression that the library,
or "codec" part, of those projects is far less lacking manpower than the
"leaf" part (players, frontends, etc.).

Alexis.
-BEGIN PGP SIGNATURE-

iHUEAREIAB0WIQSpOxaxaZikKNVNlsYOJUi7xgflrgUCXuOSHQAKCRAOJUi7xgfl
rp68AP9hzTRmZoDoderMZkLt5HsRCcfwemcVl2kvytPInVPvxQD+LnBcBOGaQy7Y
9iOhbi0fkwt9YBoq4rsBk3rzguHjvm0=
=f3JV
-END PGP SIGNATURE-

Re: [gentoo-dev] Re: [gentoo-commits] repo/gentoo:master commit in: media-libs/x265/

[Alexis Ballier]

On Tue, 26 May 2020 23:41:28 +0200
zo...@gentoo.org wrote:

> tisdag 26 maj 2020 kl. 19:54:51 CEST skrev  Alexis Ballier:
> > On Tue, 26 May 2020 10:45:39 -0400
> > 
> > Mike Gilbert  wrote:
> > > > Note that having the 'pic' useflag should be considered
> > > > something to be fixed: rewrite the asm in a PIC way. But these
> > > > days nobody has the will to do it since this is mostly an issue
> > > > on x86+pax, both being slowly decreasing.
> > > 
> > > Given that PaX has been stripped out of official Gentoo kernels
> > > due to the grsecurity licensing issue, I wonder if there is any
> > > other good reason to keep the "pic" USE flag today. Surely this
> > > affects a very small population of users.
> > 
> > Yeah that was my thought when I saw pax/grsec beginning to be more
> > hostile to open source. That's not my call but the hardened team's,
> > however I'm all for removing these workarounds entirely if there's
> > no point in having them anymore.
> Is not only PaX/Grsec that don't allow textrel. SELinux do it to and
> that is manline kernel.


k so that means there's no dropping the pic useflag then i guess

Re: [gentoo-dev] Re: [gentoo-commits] repo/gentoo:master commit in: media-libs/x265/

[Alexis Ballier]

On Tue, 2020-05-26 at 14:14 -0400, Mike Gilbert wrote:
[...]
> 
> > 
> > Assuming that the pic performance penalty is really only relevant
> > on
> > legacy arches (like x86), here are a couple of options:
> > 
> > 1. Disable pic on arches where tie performance penalty is small.
> > 2. Force pic everywhere, performance be damned.
> 
> Option 1 should say "Disable pic on arches where the performance
> penalty is large."

Performance penalty is not really a matter of arch. Of course -fPIC
from C/C++ code has a small performance impact on x86 (bigger than on
amd64), but the gain is worth it. Here it is a matter of hand written
assembly that is not relocatable. The performance impact thus depends
on the gain of said assembly, which in the case of multimedia apps/libs
is huge because that's usually about vectorizing inner loops bodys
(somewhere between x4 or x8 speedups is not unusual for the part in
question).

That's why the policy has always been to have PIC everywhere with
exceptions/workarounds tied to the pic useflag, giving users and
profiles the choice.

Again, the real fix is to rewrite the assembly in a PIC way, or at
least have ifdefs to allow it, but that is hard. Packages that have the
option to have PIC or slightly faster non-PIC assembly do/should not
have a pic useflag and always use the PIC variant.

Re: [gentoo-dev] Re: [gentoo-commits] repo/gentoo:master commit in: media-libs/x265/

[Alexis Ballier]

On Tue, 26 May 2020 10:45:39 -0400
Mike Gilbert  wrote:
> > Note that having the 'pic' useflag should be considered something
> > to be fixed: rewrite the asm in a PIC way. But these days nobody
> > has the will to do it since this is mostly an issue on x86+pax,
> > both being slowly decreasing.
> 
> Given that PaX has been stripped out of official Gentoo kernels due to
> the grsecurity licensing issue, I wonder if there is any other good
> reason to keep the "pic" USE flag today. Surely this affects a very
> small population of users.


I couldn't find any recent reference, but PIC shared libs used to be a
QA policy. There's mainly two reasons to it: First is W^X enforcement;
non PIC shared libs are refused by the x86_64 linker so a non-issue
there, on x86 you need pax to emulate it because the mmu doesn't support
the X part; I don't know about other arches.
Then there is the small memory waste done because those libs will be
loaded COW and thus their "code" is not shared anymore between
processes. And the small startup performance hit to
perform the relocations.

The latter part affects everyone, and the rule of thumb for having a
pic useflag (instead of always pic) is that the gain for non-pic asm is
better than the loss of non-pic shared libs. This is subjective but
usually a no-brainer for multimedia packages.

This is probably something to bring up to QA too.

Re: [gentoo-dev] Re: [gentoo-commits] repo/gentoo:master commit in: media-libs/x265/

[Alexis Ballier]

On Tue, 26 May 2020 10:45:39 -0400
Mike Gilbert  wrote:

> > Note that having the 'pic' useflag should be considered something
> > to be fixed: rewrite the asm in a PIC way. But these days nobody
> > has the will to do it since this is mostly an issue on x86+pax,
> > both being slowly decreasing.
> 
> Given that PaX has been stripped out of official Gentoo kernels due to
> the grsecurity licensing issue, I wonder if there is any other good
> reason to keep the "pic" USE flag today. Surely this affects a very
> small population of users.
> 

Yeah that was my thought when I saw pax/grsec beginning to be more
hostile to open source. That's not my call but the hardened team's,
however I'm all for removing these workarounds entirely if there's no
point in having them anymore.

Re: [gentoo-dev] Re: [gentoo-commits] repo/gentoo:master commit in: media-libs/x265/

[Alexis Ballier]

[...]
> > If I understand you correctly, we should just drop the USE="pic"
> > logic
> > from the remaining packages that have it? Or are you trying to say
> > something else?
> 

[...]

> Note that having the 'pic' useflag should be considered something to
> be
> fixed: rewrite the asm in a PIC way. But these days nobody has the
> will
> to do it since this is mostly an issue on x86+pax, both being slowly
> decreasing.


As a side note on this: if USE=-pic has no textrel then obviously the
useflag should be removed. This can happen over time if we don't pay
enough attention.

Re: [gentoo-dev] Re: [gentoo-commits] repo/gentoo:master commit in: media-libs/x265/

[Alexis Ballier]

On Mon, 2020-05-25 at 21:09 -0400, Mike Gilbert wrote:
> On Mon, May 25, 2020 at 7:35 PM Alexis Ballier 
> wrote:
> > On Mon, 2020-05-25 at 17:04 -0400, Mike Gilbert wrote:
> > > On Mon, May 25, 2020 at 3:18 PM Michał Górny 
> > > wrote:
> > > > On Mon, 2020-05-25 at 19:49 +0200, Alexis Ballier wrote:
> > > > > On Mon, 25 May 2020 11:26:26 -0400
> > > > > Mike Gilbert  wrote:
> > > > > 
> > > > > > On Mon, May 25, 2020 at 9:13 AM Alexis Ballier <
> > > > > > aball...@gentoo.org>
> > > > > > wrote:
> > > > > > > On Sun, 24 May 2020 20:25:11 + (UTC)
> > > > > > > "Thomas Deutschmann"  wrote:
> > > > > > > 
> > > > > > > > commit: 6e149596cc76f1bbcee6720828c8c8c92420f2a3
> > > > > > > > Author: Thomas Deutschmann  gentoo
> > > > > > > > 
> > > > > > > > org>
> > > > > > > > AuthorDate: Sun May 24 19:47:08 2020 +
> > > > > > > > Commit: Thomas Deutschmann  gentoo
> > > > > > > > 
> > > > > > > > org>
> > > > > > > > CommitDate: Sun May 24 20:23:53 2020 +
> > > > > > > > URL:
> > > > > > > > https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=6e149596
> > > > > > > > 
> > > > > > > > media-libs/x265: drop USE=pic
> > > > > > > > 
> > > > > > > > Gentoo's toolchain uses PIC by default. Since USE=asm
> > > > > > > > was
> > > > > > > > added,
> > > > > > > > we no longer need a USE flag to control that behavior.
> > > > > > > 
> > > > > > > You got it wrong here it seems: USE=pic does not control
> > > > > > > whether
> > > > > > > the toolchain produces PIC or not. Shared libs always
> > > > > > > are,
> > > > > > > and have
> > > > > > > always been, built that way on Gentoo.
> > > > > > > In this case, USE=pic means "no matter what it costs, I
> > > > > > > do
> > > > > > > not want
> > > > > > > textrels", for the cases of hand written assembly that
> > > > > > > has to
> > > > > > > be
> > > > > > > rewritten to support PIC. And, still in this case, this
> > > > > > > costs
> > > > > > > a lot
> > > > > > > of performance, so it is enabled by default on hardened
> > > > > > > profiles
> > > > > > > and not others.
> > > > > > > Textrels work fine (on some architectures), they disallow
> > > > > > > W^X
> > > > > > > and
> > > > > > > force each process using the shared lib to make a "copy"
> > > > > > > at
> > > > > > > runtime
> > > > > > > in order to resolve relocations, so are not desirable but
> > > > > > > sometimes
> > > > > > > the cost outweights the gain.
> > > > > > > 
> > > > > > > Plus, profiles/features/hardened enables pic by default
> > > > > > > but
> > > > > > > knows
> > > > > > > nothing about USE=asm so this is a regression for them.
> > > > > > 
> > > > > > The USE flag toggles use of assembly, not use of PIC. The
> > > > > > default USE
> > > > > > value in the hardened profile should not drive decisions on
> > > > > > what we
> > > > > > name USE flags.
> > > > > 
> > > > > ... but using a global well documented useflag instead of a
> > > > > local
> > > > > invention should drive such decisions.
> > > > 
> > > > What 'global well documented useflag'?
> > > 
> > > It's neither global, nor well-documented, but several packages do
> > > define it locally.
> > > 
> > 
> > https://gitweb.gentoo.org/repo/gentoo/historical.git/commit/profiles/use.desc?id=103236c295aa30e5e42cfc8a7429e4eea5f0d680
> > 
> > https://gitweb.gentoo.org/repo/gentoo/historical.git/commit/profiles/use.desc?id=784deb7134b9d430546557a8f8a0877bf35c02ba
> > 
> > I guess this hasn't been really discussed back then.
> > 
> > It is also used in a global way in profiles (make.defaults).
> > 
> > > Personally, I think it should be renamed to "asm" or something
> > > similar
> > > in the majority of cases where it actually disables all use of
> > > assembly code.
> > 
> > Thankfully these days there's usually no need to disable asm to
> > have
> > pic. hardened has no mention of that flag, and I think that e.g.
> > for
> > openssl they would have noticed long ago.
> > And again, 'asm' as a useflag makes no sense: if it works and
> > simply
> > replaces a C function by a faster one then it shouldn't even be an
> > useflag. 'pic' on the other hand conveys the tradeoff idea.
> 
> If I understand you correctly, we should just drop the USE="pic"
> logic
> from the remaining packages that have it? Or are you trying to say
> something else?


Drop USE=asm unless there's a real reason to it: Such a useflag is,
IMHO, at the same level of a useflag on dev-lang/python that would
toggle dict's underlying implementations but not the semantics of the
language.
Have USE=pic for its historical meaning, aka, sacrificing everything to
have PIC shared libs because your system enforces this (pax).

Note that having the 'pic' useflag should be considered something to be
fixed: rewrite the asm in a PIC way. But these days nobody has the will
to do it since this is mostly an issue on x86+pax, both being slowly
decreasing.

Re: [gentoo-dev] Re: [gentoo-commits] repo/gentoo:master commit in: media-libs/x265/

[Alexis Ballier]

On Mon, 2020-05-25 at 17:04 -0400, Mike Gilbert wrote:
> On Mon, May 25, 2020 at 3:18 PM Michał Górny 
> wrote:
> > On Mon, 2020-05-25 at 19:49 +0200, Alexis Ballier wrote:
> > > On Mon, 25 May 2020 11:26:26 -0400
> > > Mike Gilbert  wrote:
> > > 
> > > > On Mon, May 25, 2020 at 9:13 AM Alexis Ballier <
> > > > aball...@gentoo.org>
> > > > wrote:
> > > > > On Sun, 24 May 2020 20:25:11 + (UTC)
> > > > > "Thomas Deutschmann"  wrote:
> > > > > 
> > > > > > commit: 6e149596cc76f1bbcee6720828c8c8c92420f2a3
> > > > > > Author: Thomas Deutschmann  gentoo 
> > > > > > org>
> > > > > > AuthorDate: Sun May 24 19:47:08 2020 +
> > > > > > Commit: Thomas Deutschmann  gentoo 
> > > > > > org>
> > > > > > CommitDate: Sun May 24 20:23:53 2020 +
> > > > > > URL:
> > > > > > https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=6e149596
> > > > > > 
> > > > > > media-libs/x265: drop USE=pic
> > > > > > 
> > > > > > Gentoo's toolchain uses PIC by default. Since USE=asm was
> > > > > > added,
> > > > > > we no longer need a USE flag to control that behavior.
> > > > > 
> > > > > You got it wrong here it seems: USE=pic does not control
> > > > > whether
> > > > > the toolchain produces PIC or not. Shared libs always are,
> > > > > and have
> > > > > always been, built that way on Gentoo.
> > > > > In this case, USE=pic means "no matter what it costs, I do
> > > > > not want
> > > > > textrels", for the cases of hand written assembly that has to
> > > > > be
> > > > > rewritten to support PIC. And, still in this case, this costs
> > > > > a lot
> > > > > of performance, so it is enabled by default on hardened
> > > > > profiles
> > > > > and not others.
> > > > > Textrels work fine (on some architectures), they disallow W^X
> > > > > and
> > > > > force each process using the shared lib to make a "copy" at
> > > > > runtime
> > > > > in order to resolve relocations, so are not desirable but
> > > > > sometimes
> > > > > the cost outweights the gain.
> > > > > 
> > > > > Plus, profiles/features/hardened enables pic by default but
> > > > > knows
> > > > > nothing about USE=asm so this is a regression for them.
> > > > 
> > > > The USE flag toggles use of assembly, not use of PIC. The
> > > > default USE
> > > > value in the hardened profile should not drive decisions on
> > > > what we
> > > > name USE flags.
> > > 
> > > ... but using a global well documented useflag instead of a local
> > > invention should drive such decisions.
> > 
> > What 'global well documented useflag'?
> 
> It's neither global, nor well-documented, but several packages do
> define it locally.
> 

https://gitweb.gentoo.org/repo/gentoo/historical.git/commit/profiles/use.desc?id=103236c295aa30e5e42cfc8a7429e4eea5f0d680

https://gitweb.gentoo.org/repo/gentoo/historical.git/commit/profiles/use.desc?id=784deb7134b9d430546557a8f8a0877bf35c02ba

I guess this hasn't been really discussed back then.

It is also used in a global way in profiles (make.defaults).

> Personally, I think it should be renamed to "asm" or something
> similar
> in the majority of cases where it actually disables all use of
> assembly code.

Thankfully these days there's usually no need to disable asm to have
pic. hardened has no mention of that flag, and I think that e.g. for
openssl they would have noticed long ago.
And again, 'asm' as a useflag makes no sense: if it works and simply
replaces a C function by a faster one then it shouldn't even be an
useflag. 'pic' on the other hand conveys the tradeoff idea.

Re: [gentoo-dev] Re: [gentoo-commits] repo/gentoo:master commit in: media-libs/x265/

[Alexis Ballier]

On Mon, 25 May 2020 11:26:26 -0400
Mike Gilbert  wrote:

> On Mon, May 25, 2020 at 9:13 AM Alexis Ballier 
> wrote:
> >
> > On Sun, 24 May 2020 20:25:11 + (UTC)
> > "Thomas Deutschmann"  wrote:
> >
> > > commit: 6e149596cc76f1bbcee6720828c8c8c92420f2a3
> > > Author: Thomas Deutschmann  gentoo  org>
> > > AuthorDate: Sun May 24 19:47:08 2020 +
> > > Commit: Thomas Deutschmann  gentoo  org>
> > > CommitDate: Sun May 24 20:23:53 2020 +
> > > URL:
> > > https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=6e149596
> > >
> > > media-libs/x265: drop USE=pic
> > >
> > > Gentoo's toolchain uses PIC by default. Since USE=asm was added,
> > > we no longer need a USE flag to control that behavior.
> >
> >
> > You got it wrong here it seems: USE=pic does not control whether
> > the toolchain produces PIC or not. Shared libs always are, and have
> > always been, built that way on Gentoo.
> > In this case, USE=pic means "no matter what it costs, I do not want
> > textrels", for the cases of hand written assembly that has to be
> > rewritten to support PIC. And, still in this case, this costs a lot
> > of performance, so it is enabled by default on hardened profiles
> > and not others.
> > Textrels work fine (on some architectures), they disallow W^X and
> > force each process using the shared lib to make a "copy" at runtime
> > in order to resolve relocations, so are not desirable but sometimes
> > the cost outweights the gain.
> >
> > Plus, profiles/features/hardened enables pic by default but knows
> > nothing about USE=asm so this is a regression for them.
> 
> The USE flag toggles use of assembly, not use of PIC. The default USE
> value in the hardened profile should not drive decisions on what we
> name USE flags.

... but using a global well documented useflag instead of a local
invention should drive such decisions.

> 
> You can add the flag to package.use or package.use.mask in the
> hardened profile if necessary.


Sure but it was not added there and it's not really relevant here since
 USE=asm does not make any sense either. (aka: this was better
 before this series of commits)

[gentoo-dev] Re: [gentoo-commits] repo/gentoo:master commit in: media-libs/x265/

[Alexis Ballier]

On Sun, 24 May 2020 20:25:11 + (UTC)
"Thomas Deutschmann"  wrote:

> commit: 6e149596cc76f1bbcee6720828c8c8c92420f2a3
> Author: Thomas Deutschmann  gentoo  org>
> AuthorDate: Sun May 24 19:47:08 2020 +
> Commit: Thomas Deutschmann  gentoo  org>
> CommitDate: Sun May 24 20:23:53 2020 +
> URL:
> https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=6e149596
> 
> media-libs/x265: drop USE=pic
> 
> Gentoo's toolchain uses PIC by default. Since USE=asm was added,
> we no longer need a USE flag to control that behavior.


You got it wrong here it seems: USE=pic does not control whether
the toolchain produces PIC or not. Shared libs always are, and have
always been, built that way on Gentoo.
In this case, USE=pic means "no matter what it costs, I do not want
textrels", for the cases of hand written assembly that has to be
rewritten to support PIC. And, still in this case, this costs a lot of
performance, so it is enabled by default on hardened profiles and not
others.
Textrels work fine (on some architectures), they disallow W^X and force
each process using the shared lib to make a "copy" at runtime in order
to resolve relocations, so are not desirable but sometimes the cost
outweights the gain.

Plus, profiles/features/hardened enables pic by default but knows
nothing about USE=asm so this is a regression for them.


Alexis.

[gentoo-dev] Re: [gentoo-commits] repo/gentoo:master commit in: media-libs/x265/

[Alexis Ballier]

On Sun, 24 May 2020 20:25:11 + (UTC)
"Thomas Deutschmann"  wrote:

> commit: eba596db8a926adb18595549c89294ed0a1e929e
> Author: Thomas Deutschmann  gentoo  org>
> AuthorDate: Sun May 24 15:07:04 2020 +
> Commit: Thomas Deutschmann  gentoo  org>
> CommitDate: Sun May 24 20:23:50 2020 +
> URL:
> https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=eba596db
> 
> media-libs/x265: rework assembly support
> 
> Closes: https://bugs.gentoo.org/681878
> Package-Manager: Portage-2.3.99, Repoman-2.3.22
> Signed-off-by: Thomas Deutschmann  gentoo.org>
> 
>  media-libs/x265/metadata.xml|  1 +
>  media-libs/x265/x265-3.3.ebuild | 66
> - 2 files changed, 34
> insertions(+), 33 deletions(-)
> 
> diff --git a/media-libs/x265/metadata.xml
> b/media-libs/x265/metadata.xml index 22a07293b83..c585d553631 100644
> --- a/media-libs/x265/metadata.xml
> +++ b/media-libs/x265/metadata.xml
> @@ -5,6 +5,7 @@
>  media-vi...@gentoo.org
>
>
> +Enable x86_64 assembly optimizations.



This should not even be an useflag.
Either it works or does not. Individual features support is controlled
by cpu_flags_* (or built-in and autodetected at runtime).
Please fix.



>  Add support for producing 10bits HEVC.
>  Add support for producing 12bits HEVC.
>  Build with support for NUMA nodes.
> 
> diff --git a/media-libs/x265/x265-3.3.ebuild
> b/media-libs/x265/x265-3.3.ebuild index 9fc0159bc00..f5c4fee6d97
> 100644 --- a/media-libs/x265/x265-3.3.ebuild
> +++ b/media-libs/x265/x265-3.3.ebuild
> @@ -19,15 +19,17 @@ HOMEPAGE="http://x265.org/
> https://bitbucket.org/multicoreware/x265/wiki/Home; LICENSE="GPL-2"
>  # subslot = libx265 soname
>  SLOT="0/188"
> -IUSE="+10bit +12bit cpu_flags_arm_neon numa pic power8 test"
> +IUSE="+asm +10bit +12bit cpu_flags_arm_neon numa pic power8 test"
>  
>  # Test suite requires assembly support and is known to be broken
>  RESTRICT="test"
>  
>  ASM_DEPEND=">=dev-lang/yasm-1.2.0"
>  
> -BDEPEND="abi_x86_32? ( ${ASM_DEPEND} )
> - abi_x86_64? ( ${ASM_DEPEND} )"
> +BDEPEND="asm? (
> + abi_x86_32? ( ${ASM_DEPEND} )
> + abi_x86_64? ( ${ASM_DEPEND} )
> + )"
>  
>  RDEPEND="numa? ( >=sys-process/numactl-2.0.10-r1[${MULTILIB_USEDEP}]
> )" 
> @@ -85,17 +87,6 @@ x265_variant_src_configure() {
>   -DENABLE_CLI=OFF
>   -DMAIN12=ON
>   )
> - if [[ ${ABI} = x86 ]] ; then
> - mycmakeargs+=( -DENABLE_ASSEMBLY=OFF
> )
> - fi
> - if [[ ${ABI} = arm ]] ; then
> - # 589674
> - mycmakeargs+=( -DENABLE_ASSEMBLY=OFF
> )
> - fi
> - if [[ ${ABI} = ppc64 ]] ; then
> - #
> https://bugs.gentoo.org/show_bug.cgi?id=607802#c5
> - mycmakeargs+=( -DENABLE_ASSEMBLY=OFF
> -DENABLE_ALTIVEC=OFF )
> - fi
>   ;;
>   "main10")
>   mycmakeargs+=(
> @@ -104,17 +95,6 @@ x265_variant_src_configure() {
>   -DENABLE_SHARED=OFF
>   -DENABLE_CLI=OFF
>   )
> - if [[ ${ABI} = x86 ]] ; then
> - mycmakeargs+=( -DENABLE_ASSEMBLY=OFF
> )
> - fi
> - if [[ ${ABI} = arm ]] ; then
> - # 589674
> - mycmakeargs+=( -DENABLE_ASSEMBLY=OFF
> )
> - fi
> - if [[ ${ABI} = ppc64 ]] ; then
> - #
> https://bugs.gentoo.org/show_bug.cgi?id=607802#c5
> - mycmakeargs+=( -DENABLE_ASSEMBLY=OFF
> -DENABLE_ALTIVEC=OFF )
> - fi
>   ;;


What are you trying to fix here ?
This sounds like a regression to me: some asm will not work for
10/12bits variants while 8bit is fine. You are removing this work here.



>   "main")
>   if (( "${#MULTIBUILD_VARIANTS[@]}" > 1 )) ;
> then @@ -146,7 +126,6 @@ multilib_src_configure() {
>   append-cxxflags -fPIC
>  
>   local myabicmakeargs=(
> - -DENABLE_TESTS=$(usex test ON OFF)
>   $(multilib_is_native_abi || echo "-DENABLE_CLI=OFF")
>   -DENABLE_LIBNUMA=$(usex numa ON OFF)
>   -DCPU_POWER8=$(usex power8 ON OFF)
> @@ -154,18 +133,39 @@ multilib_src_configure() {
>   -DLIB_INSTALL_DIR="$(get_libdir)"
>   )
>  
> + local supports_asm=yes
> +
>   if [[ ${ABI} = x86 ]] ; then
> - # Bug #528202
> - if use pic ; then
> + if use asm && use pic ; then
> + # Bug #528202
>   ewarn "PIC has been requested but x86 asm is
> not 

Re: [gentoo-dev] [RFC] Should NATTkA reject keywordreqs for packages with -arch (-*) keywords?

[Alexis Ballier]

On Wed, 2020-05-06 at 03:41 +0200, Thomas Deutschmann wrote:
> On 2020-05-06 00:52, James Le Cuirot wrote:
> > On Tue, 05 May 2020 22:19:59 +0200
> > Michał Górny  wrote:
> > > WDYT?
> > 
> > Play it safe. -* is frequently used for binary packages where an
> > arch
> > will simply either work or it won't, with little likelihood of the
> > situation changing. -arch is so rare that I don't recall ever
> > seeing
> > it. In either case, restoring an arch should be an explicit action.
> 
> +1
> 


+1

with the addition that -arch (by opposition to -*) is usually used for
specifying "this has been tested and is broken, don't waste your time
on it". Or at least used to be used that way.

If a package relies on arch specific support, then we could make a case
that it should be '-*' + a whitelist of arches having said support.
So, IMHO, -arch is quite version specific: package being broken is
likely due to a bug that may or may not be fixed in later versions,
hence, to me it makes total sense to have keyword reqs even for -arch
if this is a newer version that the one that initially had its -arch
added.

I also believe tools like ekeyword or repoman should reset -arch to
nothing, or at least issue a warning when adding new ebuilds with it,
which would make this simpler for nattka.


Alexis.

Re: [gentoo-dev] musl doesn't provide execinfo.h

[Alexis Ballier]

On Fri, 2020-03-27 at 19:07 -0400, Anthony G. Basile wrote:
> On 3/27/20 3:17 PM, Alexis Ballier wrote:
> > On Fri, 2020-03-27 at 08:03 -0400, Anthony G. Basile wrote:
> > > On 3/26/20 9:25 PM, Joshua Kinard wrote:
> > > > On 3/23/2020 04:21, Jaco Kroon wrote:
> > > > > Hi,
> > > > > 
> > > > > https://bugs.gentoo.org/713668 relates.
> > > > > 
> > > > >  * Searching for /usr/include/execinfo.h ...
> > > > > sys-libs/glibc-2.29-r7 (/usr/include/execinfo.h)
> > > > > 
> > > > > As I see I can either add an explicit depend on glibc which
> > > > > I'd
> > > > > prefer
> > > > > not to.  Or someone from the musl team could possibly assist
> > > > > on
> > > > > how to
> > > > > get the backtrace() set of calls on musl please?
> > > > > 
> > > > > Alternatively I need to add a test and simply path debug.c to
> > > > > only
> > > > > provide stub function for print_backtrace(FILE *fp) that just
> > > > > does
> > > > > fprintf(fp, "No backtrace() available to print a
> > > > > backtrace.\n");
> > > > > 
> > > > > Suggestions?
> > > > > 
> > > > > Kind Regards,
> > > > > Jaco
> > > > 
> > > > Some quick searching on google, it looks like the cleanest fix
> > > > for
> > > > that bug
> > > > is dahdi-tools needs to be patched to only include execinfo.h
> > > > or
> > > > only use
> > > > backtrace() on glibc-based systems, and that patch then sent to
> > > > the
> > > > dahdi-tools upstream developers for inclusion in a future
> > > > release.  That
> > > > way, we're not dragging that patch around forever in the tree
> > > > or in
> > > > the musl
> > > > overlay.
> > > > 
> > > > It also doesn't look like musl itself will ever implement
> > > > execinfo.h or
> > > > backtrace(), per this message in 2015 from the lead musl
> > > > developer:
> > > > https://www.openwall.com/lists/musl/2015/04/09/3
> > > > 
> > > 
> > > Correct.  I've been adding -standalone packages to provide for
> > > features
> > > like fts, obstack, argp,etc. which are bundled into glibc but not
> > > really
> > > under the POSIX standard.
> > > 
> > > So either we patch packages to turn off backtrace() or we add
> > > libunwind-standalone to the tree.
> > > 
> > 
> > BTW, we had libexecinfo for fbsd, which seems also present in
> > alpine:
> > https://pkgs.alpinelinux.org/package/edge/main/x86/libexecinfo
> > 
> > 
> 
> Had?  Is it in the tree now or should I look into adding it?

Restoring it: https://bugs.gentoo.org/683284

Re: [gentoo-dev] musl doesn't provide execinfo.h

[Alexis Ballier]

On Fri, 2020-03-27 at 08:03 -0400, Anthony G. Basile wrote:
> On 3/26/20 9:25 PM, Joshua Kinard wrote:
> > On 3/23/2020 04:21, Jaco Kroon wrote:
> > > Hi,
> > > 
> > > https://bugs.gentoo.org/713668 relates.
> > > 
> > >  * Searching for /usr/include/execinfo.h ...
> > > sys-libs/glibc-2.29-r7 (/usr/include/execinfo.h)
> > > 
> > > As I see I can either add an explicit depend on glibc which I'd
> > > prefer
> > > not to.  Or someone from the musl team could possibly assist on
> > > how to
> > > get the backtrace() set of calls on musl please?
> > > 
> > > Alternatively I need to add a test and simply path debug.c to
> > > only
> > > provide stub function for print_backtrace(FILE *fp) that just
> > > does
> > > fprintf(fp, "No backtrace() available to print a backtrace.\n");
> > > 
> > > Suggestions?
> > > 
> > > Kind Regards,
> > > Jaco
> > 
> > Some quick searching on google, it looks like the cleanest fix for
> > that bug
> > is dahdi-tools needs to be patched to only include execinfo.h or
> > only use
> > backtrace() on glibc-based systems, and that patch then sent to the
> > dahdi-tools upstream developers for inclusion in a future
> > release.  That
> > way, we're not dragging that patch around forever in the tree or in
> > the musl
> > overlay.
> > 
> > It also doesn't look like musl itself will ever implement
> > execinfo.h or
> > backtrace(), per this message in 2015 from the lead musl developer:
> > https://www.openwall.com/lists/musl/2015/04/09/3
> > 
> 
> Correct.  I've been adding -standalone packages to provide for
> features
> like fts, obstack, argp,etc. which are bundled into glibc but not
> really
> under the POSIX standard.
> 
> So either we patch packages to turn off backtrace() or we add
> libunwind-standalone to the tree.
> 


BTW, we had libexecinfo for fbsd, which seems also present in alpine:
https://pkgs.alpinelinux.org/package/edge/main/x86/libexecinfo

Re: [gentoo-dev] [PATCH 0/4] elt-patches: support wrapped Win32 MSVC toolchain

[Alexis Ballier]

On Thu, 2020-03-12 at 20:59 +, James Le Cuirot wrote:
> On Thu, 12 Mar 2020 11:23:04 +0100
> Alexis Ballier  wrote:
> 
> > On Thu, 2020-03-12 at 09:06 +0100, ha...@gentoo.org wrote:
> > > As this native Win32 support is considered highly experimental
> > > still,
> > > I
> > > would like to apply the libtool patches for parity via
> > > elibtoolize
> > > only,
> > > without applying them in sys-devel/libtool itself yet.
> > >   
> > 
> > IIRC you need to do it this way, experimental or not: elibtoolize
> > is
> > needed for packages whose autotools have been generated with an old
> > libtool (ie all of them for now). eautoreconf should call
> > elibtoolize,
> > so, after having this in elt-patches, better focus on upstreaming
> > this
> > in libtool itself so that the need for elibtoolize fades away with
> > time.
> > 
> > You will probably run into the same issues as in the old days with
> > BSD:
> > not all packages run elibtoolize and you do not have a sane way to
> > force this besides editing ebuilds.
> 
> I've long wanted to automatically apply elibtoolize to fix other
> cross-compile issues. I did come up with a rough prototype and it did
> work though I imagine it might break some packages. Maybe it should
> be
> opt-out rather than opt-in?

If a patch in elibtoolize might break something then it should not be
there in the first place: the function is called by a lot of packages.
However, we can assume that such packages have been tested whereas by
magically calling elibtoolize they have not. A good solution to avoid
this could be to modify the default src_prepare in EAPI8 to call it.

I think some hacks were implemented for fbsd via profile.bashrc because
the pain caused by *not* calling elibtoolize (soname changes) was worse
than having untested packages that might break under a red moon.


Alexis.

Re: [gentoo-dev] [PATCH 0/4] elt-patches: support wrapped Win32 MSVC toolchain

[Alexis Ballier]

On Thu, 2020-03-12 at 09:06 +0100, ha...@gentoo.org wrote:
> As this native Win32 support is considered highly experimental still,
> I
> would like to apply the libtool patches for parity via elibtoolize
> only,
> without applying them in sys-devel/libtool itself yet.
> 

IIRC you need to do it this way, experimental or not: elibtoolize is
needed for packages whose autotools have been generated with an old
libtool (ie all of them for now). eautoreconf should call elibtoolize,
so, after having this in elt-patches, better focus on upstreaming this
in libtool itself so that the need for elibtoolize fades away with
time.

You will probably run into the same issues as in the old days with BSD:
not all packages run elibtoolize and you do not have a sane way to
force this besides editing ebuilds.


Alexis.

Re: [gentoo-dev] [EAPI 8 RFC] Install-time dependencies

[Alexis Ballier]

On Fri, 2019-12-20 at 18:55 +0100, Ulrich Mueller wrote:
> > > > > > On Fri, 20 Dec 2019, Alexis Ballier wrote:
> > Should we use this to drop RDEPEND from pkg_*inst/rm phases ?
> > PMS states "RDEPEND (unless the particular dependency results in a
> > circular dependency, in which case it may be installed later)"
> > which is
> > kind of "maybe maybe not" and I'm not sure how one can rely on
> > RDEPEND
> > for those phases in their ebuilds if another random ebuild can
> > trigger
> > a case where it's not satisfied.
> 
> This wording has already been updated in git. It just says "RDEPEND"
> now
> for these four phases:
> 
> https://dev.gentoo.org/~ulm/pms/head/pms.html#x1-720008.1
> 
> 

Ok.
My question still stands though, so that we can avoid mutual RDEPEND
cycles and also, as you noted on the bug, in a x-compile setting, CHOST
deps are not really needed at installation.

Re: [gentoo-dev] [EAPI 8 RFC] Install-time dependencies

[Alexis Ballier]

On Thu, 2019-12-19 at 20:40 +0100, Michał Górny wrote:
> Hello,
> 
> Here's another potential EAPI 8 feature I'd like to discuss.  Please
> note that this is about *new dependency type*, so please don't hijack
> it
> into the big 'let's steal Exherbo syntax' debate.
> 
> Bug: https://bugs.gentoo.org/660306
> 
> 
> The problem
> ===
> 
> Right now we don't really have a clean way of specifying dependencies
> that are used during pkg_*inst (and pkg_*rm?) phases.  So far RDEPEND
> was used as a 'close enough' alternative (except for a few developers
> who rejected it as 'invalid' and used DEPEND which is even more
> wrong). 
> However, this is no longer sufficient with EAPI 7 cross support.
> 
> By design, pkg_*inst phases are run in build host's environment when
> cross is used (because obviously you can't run target host
> executables).
> Therefore, the relevant dependencies need to be installed into CBUILD
> root, while RDEPEND is installed into CHOST root.
> 
> 
> The proposed solution
> =
> 
> The proposal is to add a new dependency type (codename: IDEPEND)
> which
> indicates dependencies used for pkg_*inst (and pkg_*rm?)
> phases.  Those
> dependencies would be installed into CBUILD root (like BDEPEND), and
> therefore would be runnable from build host.  Similarly to RDEPEND,
> they
> would be installed for binary package installs but not for pure
> binpkg
> builds (without install).
> 
> Example:
> 
>   inherit xdg-utils
> 
>   IDEPEND="dev-util/desktop-file-utils"
> 
>   pkg_postinst() {
> xdg_desktop_database_update
>   }
> 
> 
> WDYT?
> 


Should we use this to drop RDEPEND from pkg_*inst/rm phases ?
PMS states "RDEPEND (unless the particular dependency results in a
circular dependency, in which case it may be installed later)" which is
kind of "maybe maybe not" and I'm not sure how one can rely on RDEPEND
for those phases in their ebuilds if another random ebuild can trigger
a case where it's not satisfied.

Re: [gentoo-dev] [PATCH] virtualx.eclass: Append RESTRICT="!test? ( test )" by default

[Alexis Ballier]

On Thu, 2019-12-12 at 12:14 -0500, Mike Gilbert wrote:
> On Thu, Dec 12, 2019 at 12:02 PM NP-Hardass 
> wrote:
> > On 12/11/19 9:58 AM, Michał Górny wrote:
> > > Append RESTRICT="!test? ( test )" in the default case when
> > > virtualx
> > > is conditional to USE=test.  This fixes 440 MissingTestRestrict
> > > warnings.
> > > 
> > > Signed-off-by: Michał Górny 
> > > ---
> > >  eclass/virtualx.eclass | 2 ++
> > >  1 file changed, 2 insertions(+)
> > > 
> > > diff --git a/eclass/virtualx.eclass b/eclass/virtualx.eclass
> > > index 40eeea5463bc..6aba6bf488dd 100644
> > > --- a/eclass/virtualx.eclass
> > > +++ b/eclass/virtualx.eclass
> > > @@ -89,6 +89,8 @@ case ${VIRTUALX_REQUIRED} in
> > >   fi
> > >   RDEPEND=""
> > >   IUSE="${VIRTUALX_REQUIRED}"
> > > + [[ ${VIRTUALX_REQUIRED} == test ]] &&
> > > + RESTRICT+=" !test? ( test )"
> > >   ;;
> > >  esac
> > > 
> > > 
> > 
> > Is there a better way to address this than editing a ton of
> > eclasses
> > independently?
> 
> Not really.
> 

Seems a good candidate for a future EAPI

Re: [gentoo-dev] unsanctioned python 2.7 crusade

[Alexis Ballier]

On Fri, 2019-12-06 at 21:28 +0100, Thomas Deutschmann wrote:
> On 2019-12-06 21:10, Andreas Sturmlechner wrote:
> > Just so we're on the same page, a recent example of what some
> > people 
> > suggesting to keep py27 ad nauseam are asking users to deal with:
> > [...]
> > WARNING: One or more updates/rebuilds have been skipped due to a
> > dependency 
> > conflict:
> 
> Yes, like said, at some point you cannot prevent that. Remember when
> I
> bumped libvpx to v1.8.0 and people yelled at me because they are now
> seeing that message all the time (If you are using gnome you probably
> know the same msg which triggers for unicode stuff which I am also
> responsible for) because I bumped that package but not everything
> supports that version yet?


For having maintained packages that often have this issue for years, I
must say this is a very bad idea: You are asking (or doing yourself)
consumer packages to have < deps on your package. This falsely gives
the impression that the non-latest version is still maintained. This
also makes removing this old version very error prone (we do have tools
to check for that but those are not in the standard workflow). Not sure
how portage handles this, but negative deps (<, =, ! & co) are much
harder to solve than purely positive ones -- PM probably uses some
heuristics but then this has some limitations and if the number of such
deps grows too much it may fail to solve them or do the right thing.

I think it is a much better way to package.mask the newest version of
your lib until all consumers work with it, or those that don't are
masked. This is how we handled such transitions before portage improved
its handling of negative deps and is IMHO still better.


Alexis.

Re: [gentoo-dev] [PATCH] package.deprecated: Create initial template

[Alexis Ballier]

On Fri, 6 Dec 2019 10:41:56 -0500
Matt Turner  wrote:

> On Fri, Dec 6, 2019 at 5:51 AM Alexis Ballier 
> wrote:
> >
> > On Fri, 6 Dec 2019 04:33:36 -0500
> > Tim Harder  wrote:
> >
> > > On 2019-12-06 Fri 04:03, Alexis Ballier wrote:
> > > > it's not just like repoman and cvs since repoman commit did
> > > > push ;) it will never be perfect but i really like repoman
> > > > commit to refuse to even commit if there's something obviously
> > > > wrong
> > >
> > > I'm more of the opinion (and am working towards that practicality
> > > in terms of runtime speed) that a subset of QA checks should be
> > > run as a git hook which would cause push failures on certain
> > > classes of bad commits.
> >
> >
> > There should be both. Amending the 23rd commit message because one
> > mistyped a semicolon is kind of a PITA.
> 
> It is?
> 
> git rebase -i HEAD~23
> 
> Is that what you think is a pain in the ass, or do you not know about
> interactive rebase? :)


You made me look at the doc and I indeed had never used the reword
option ;) got stuck at pick/squash/edit somehow and that's the edit I
did consider a PITA yes

Without good integration from the checker it is probably a PITA to
figure out that 23 too and also still doesn't help for broken commits
(not messages) that may or may not trigger later conflicts (unless we
decide we don't care about working commits, just working pushes, which
WFM)

Re: [gentoo-dev] [PATCH] package.deprecated: Create initial template

[Alexis Ballier]

On Fri, 6 Dec 2019 04:33:36 -0500
Tim Harder  wrote:

> On 2019-12-06 Fri 04:03, Alexis Ballier wrote:
> > it's not just like repoman and cvs since repoman commit did push ;)
> > it will never be perfect but i really like repoman commit to refuse
> > to even commit if there's something obviously wrong
> 
> I'm more of the opinion (and am working towards that practicality in
> terms of runtime speed) that a subset of QA checks should be run as a
> git hook which would cause push failures on certain classes of bad
> commits.


There should be both. Amending the 23rd commit message because one
mistyped a semicolon is kind of a PITA.

> > as you write below, it's just a matter of checking exit status and
> > using git, which can be done by scripting, but the script is
> > standard (*) and mandated to be part of the workflow
> 
> > it also allows to check or templatize commit messages to follow
> > policy
> 
> Technically pkgcheck supports more git-related checks than repoman
> last I checked, i.e. result keywords including BadCommitSummary,
> DirectStableKeywords, DroppedUnstableKeywords, DroppedStableKeywords,
> DirectNoMaintainer, and MissingSignOff; with possible future additions
> such as warning when modifying deps in an ebuild without revbumping.
> 
> Futhermore, one can scan against all commits in parallel via `pkgcheck
> scan --commits` which will enable potential commit results that are
> otherwise skipped.

All this seems post-commit, not pre-commit.

> Anyway, my main point is that if someone really wants commit
> functionality it's semi-trivial to script something similar to what
> repoman does (assuming exit status/api support exists) and if it's
> decent enough quality (including tests) I'd probably consider adding
> it to the pkgcheck repo.

It doesn't necessarily have to live in the pkgcheck repo, but it should
definitely not be "meh, script it yourself, it's trivial" since that
will probably lead to several scripts with varying degrees of quality
and brokeness.

Re: [gentoo-dev] [PATCH] package.deprecated: Create initial template

[Alexis Ballier]

On Fri, 2019-12-06 at 03:23 -0500, Tim Harder wrote:
> On 2019-12-05 Thu 17:00, Alexis Ballier wrote:
> > > > pkgcheck is mostly used by your CI checks for
> > > > producing huge reports, which is nice but addresses a different
> > > > problem
> > > There is nothing stopping you from running pkgcheck locally.  In
> > > fact,
> > > it should work out of the box these days.  If you have any
> > > problems,
> > > please report them and I'm sure they will be addressed promptly.
> > Sure I did that to get reports like what CI does for me now but
> > that's
> > always been a different usecase; I wasn't aware pkgcheck had the
> > equivalent of repoman commit
> 
> While I dislike contributing more to this off-topic tangent, since
> I've
> fielded this question/request in IRC a few times in the past I figure
> I
> might as well address it again here for the IRC-averse.
> 
> Personally I use pkgcheck as a QA tool and *git* (or another vcs
> tool)
> as a commit tool, just like how I used to use repoman and cvs a long
> time ago. I generally dislike when cli tools amalgamate disparate
> features that they weren't designed for so no one has been able to
> convince me why a tool designed to verify ebuilds and their related
> repos should support commit capabilities internally.

it's not just like repoman and cvs since repoman commit did push ;)
it will never be perfect but i really like repoman commit to refuse to
even commit if there's something obviously wrong

as you write below, it's just a matter of checking exit status and
using git, which can be done by scripting, but the script is standard
(*) and mandated to be part of the workflow

it also allows to check or templatize commit messages to follow policy

(*) and force the use of some handy git options like only commit paths
starting from cwd even if other files had been git added, which i never
remember what is the git cli option for this

Alexis.

Re: [gentoo-dev] [PATCH] package.deprecated: Create initial template

[Alexis Ballier]

On Thu, 2019-12-05 at 19:04 +0100, Michał Górny wrote:
> On Thu, 2019-12-05 at 18:59 +0100, Alexis Ballier wrote:
> > On Thu, 2019-12-05 at 18:39 +0100, Michał Górny wrote:
> > > On Thu, 2019-12-05 at 17:36 +0100, Alexis Ballier wrote:
> > > > On Thu, 2019-12-05 at 17:09 +0100, Michał Górny wrote:
> > > > > +
> > > > > 
> > > > > 
> > > > > +#
> > > > > +# This file specifies packages that are considered
> > > > > deprecated
> > > > > (but
> > > > > not
> > > > > +# masked yet).  It will trigger pkgcheck warnings whenever
> > > > > other
> > > > > +# packages depend on them.
> > > > > 
> > > > 
> > > > repoman would be more useful for this
> > > > 
> > > 
> > > Then feel free to take repoman over, and start maintaining
> > > it.  I've
> > > lost interest in contributing to the project after the last
> > > pointless
> > > refactoring made adding anything even more effort, and it doesn't
> > > seem
> > > that anyone else has.
> > > 
> > > Given that pkgcheck is a. faster by design, b. running checks
> > > in parallel, c. has sane API making contributing a pleasure, I
> > > don't
> > > really see a point in putting any more effort to support a dead
> > > repoman.
> > > 
> > 
> > it's not about who's maintaining what here...
> > just s/pkgcheck/QA tools/ and be done with it
> 
> Oh, I've listed pkgcheck there because it's the only tool
> implementing
> the file at the moment.  I'm happy to replace it with larger list or
> something more generic once there are other tools.  However, I
> believe
> that saying 'pkgcheck' right now has the advantage that devs know
> which
> tool to use to see the result.

IMHO maintaining such a list is better suited for devmanual or wiki;
just like skel.ebuild could be improved by removing portage references
and refer to PMS


> > pkgcheck is mostly used by your CI checks for
> > producing huge reports, which is nice but addresses a different
> > problem
> 
> There is nothing stopping you from running pkgcheck locally.  In
> fact,
> it should work out of the box these days.  If you have any problems,
> please report them and I'm sure they will be addressed promptly.

Sure I did that to get reports like what CI does for me now but that's
always been a different usecase; I wasn't aware pkgcheck had the
equivalent of repoman commit


> > i could see this file being useful for auto-generating lists on qa-
> > reports like for eapis too
> 
> I don't think there's really a point in duplicating this.

For now certainly not. Once someone wants to wipe a deprecated package
this could come in handy instead of searching a huge html page, but
sure this could be fixed the other way by having this in the per-check
reports like what is on the left side of the current CI reports

Re: [gentoo-dev] [PATCH] package.deprecated: Create initial template

[Alexis Ballier]

On Thu, 2019-12-05 at 18:39 +0100, Michał Górny wrote:
> On Thu, 2019-12-05 at 17:36 +0100, Alexis Ballier wrote:
> > On Thu, 2019-12-05 at 17:09 +0100, Michał Górny wrote:
> > > +
> > > 
> > > +#
> > > +# This file specifies packages that are considered deprecated
> > > (but
> > > not
> > > +# masked yet).  It will trigger pkgcheck warnings whenever other
> > > +# packages depend on them.
> > > 
> > 
> > repoman would be more useful for this
> > 
> 
> Then feel free to take repoman over, and start maintaining it.  I've
> lost interest in contributing to the project after the last pointless
> refactoring made adding anything even more effort, and it doesn't
> seem
> that anyone else has.
> 
> Given that pkgcheck is a. faster by design, b. running checks
> in parallel, c. has sane API making contributing a pleasure, I don't
> really see a point in putting any more effort to support a dead
> repoman.
> 

it's not about who's maintaining what here...
just s/pkgcheck/QA tools/ and be done with it


unless i missed something, repoman is still the standard for pre-commit 
checks and raising everyone's attention on potential
improvements/issues; pkgcheck is mostly used by your CI checks for
producing huge reports, which is nice but addresses a different problem

i could see this file being useful for auto-generating lists on qa-
reports like for eapis too


as for your current views on repoman vs pkgcheck, i have nothing
against stopping using repoman and switching to pkgcheck, but AFAIK
this has yet to happen at a policy level

Re: [gentoo-dev] [PATCH] package.deprecated: Create initial template

[Alexis Ballier]

On Thu, 2019-12-05 at 10:53 -0600, William Hubbs wrote:
> On Thu, Dec 05, 2019 at 05:36:58PM +0100, Alexis Ballier wrote:
> > On Thu, 2019-12-05 at 17:09 +0100, Michał Górny wrote:
> > > +
> > > 
> > > +#
> > > +# This file specifies packages that are considered deprecated
> > > (but
> > > not
> > > +# masked yet).  It will trigger pkgcheck warnings whenever other
> > > +# packages depend on them.
> > > 
> > 
> > repoman would be more useful for this
> 
> I wouldn't stop pkgcheck from supporting this, but repoman should as
> well.
> 
> 
> 

of course, that's what i meant ;)

Re: [gentoo-dev] [PATCH] package.deprecated: Create initial template

[Alexis Ballier]

On Thu, 2019-12-05 at 17:09 +0100, Michał Górny wrote:
> +
> +#
> +# This file specifies packages that are considered deprecated (but
> not
> +# masked yet).  It will trigger pkgcheck warnings whenever other
> +# packages depend on them.
> 


repoman would be more useful for this

Re: [gentoo-dev] Last rites: dev-python/* leaf packages

[Alexis Ballier]

On Wed, 2019-12-04 at 19:15 -0500, Aaron Bauman wrote:
> * Removal in 30 days
> 


IMHO masking with unfixed, or much later, removal date will better help
achieve your goal: You are making your point by having them masked so
that it will make enough noise for interested people to understand py2
only is not a thing anymore.

We already see a lot of "false positives", there are probably packages
that just work but are lacking attention. If after a longer time those
packages are still not fixed, you can probably safely remove them with
the "meh, nobody cares anyway" reason and not even bother having to
check hundreds (?) of packages yourself. The 30 days is usually a
guideline for packages that have known issues but seems a bit short for
checking if someone cares about a package using a deprecated but
working python.


Also, your list is missing dev-ros/* which is py2 only. I hope I'll be
able to update them soon, last time I tried I failed miserably though
since the whole stack is really python-single style so that one broken
package with py3 causes the whole stack to be py2...


Alexis.

Re: [gentoo-dev] [PMS] [PATCH] Correct the definition of ESYSROOT as EPREFIX isn't always applicable

[Alexis Ballier]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On Wed, 31 Jul 2019 21:40:19 +0100
James Le Cuirot  wrote:

> On Wed, 31 Jul 2019 15:51:58 +0200
> Alexis Ballier  wrote:
> 
> > On Tue, 30 Jul 2019 23:26:27 +0100
> > James Le Cuirot  wrote:
> > 
> > > > Admittedly without a full understanding of the problem, but this
> > > > looks wrong to me: SYSROOT, EPREFIX and BROOT are only relevant
> > > > in build phases (src_*); (EPREFIX is a little special here but
> > > > mostly for convenience). ROOT is only relevant in pkg_* phases.
> > > > I don't see how this can work. Say I build a binpkg with ROOT=/
> > > > then use that binpkg with ROOT=/somewhere, you can't go back
> > > > and change SYSROOT.  
> > > 
> > > ROOT is used to determine ESYSROOT, not the other way around. As
> > > you say, (E)SYSROOT is only relevant in src phases so it doesn't
> > > matter if ROOT has changed when installing a binpkg.  
> > 
> > I am missing something here: You are making ESYSROOT depend on the
> > value of ROOT, so how can it not matter ?
> 
> What I am trying to say (somewhat unsuccessfully!) is that the value
> of (E)SYSROOT only changes how the package is built, not what the
> resulting package looks like. It's where all the headers and libraries
> are sourced from at build time, which is irrelevant at runtime.

err, SYSROOT does change what the resulting package looks like: it
defines ABI (headers and needed entries for example).

> So why does ROOT affect it? Normally you install the packages for
> BDEPEND, DEPEND, and RDEPEND to the same location. If BDEPEND and
> RDEPEND are installed to different locations (ROOT!=/) then DEPEND
> will almost always be installed to one of the other two. If either of
> those two locations is prefixed then we need the prefix for DEPEND's
> location to match, otherwise it wouldn't actually be the same
> location. Using ROOT allows us to figure this out automatically in a
> way that covers all sensible use cases and avoids accidentally
> falling into an unsupported case.


So, now let's simplify this a bit and forget about prefix and crossdev
for a moment. Say I am building an atom chroot (for nfs root) on an
haswell desktop. I set ROOT=SYSROOT=/atomchroot. My desktop has CFLAGS
for haswell, and I have atom CFLAGS in
/atomchroot/etc/portage/make.conf. When I build something and portage
installs a BDEPEND to /, I want it to use my / configuration, and when
it is in /atomchroot I want it to use the configuration from there.
emerge has command line options to do that but I am not sure if this is
properly specified in PMS.


Now, say I am doing the same on a prefix haswell desktop. With your
algorithm, we have SYSROOT==ROOT so ESYSROOT is EPREFIX/SYSROOT.
However, what I want is a normal chroot with EPREFIX=/ here, so when
should one use ESYSROOT in an ebuild in that case ? where does this
EPREFIX comes from by the way ?


To me, this looks more of a general problem of defining where the
configuration is taken from, so that we can set EPREFIX independently
if it is SYSROOT or BROOT.



> I hope that's clearer.

Sorry :-(
-BEGIN PGP SIGNATURE-

iHUEAREIAB0WIQSpOxaxaZikKNVNlsYOJUi7xgflrgUCXUL2ygAKCRAOJUi7xgfl
rkwRAP9LDImZBCYxsWKMjT/ckCeK0hOLtctdpZuBwzjv5RIuiAD/cTUj7P7h9rBd
gYaEEI+pqdN25ZNbjao/8w/j7SsXUMo=
=WYef
-END PGP SIGNATURE-

Re: [gentoo-dev] [PMS] [PATCH] Correct the definition of ESYSROOT as EPREFIX isn't always applicable

[Alexis Ballier]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On Tue, 30 Jul 2019 23:26:27 +0100
James Le Cuirot  wrote:

> > Admittedly without a full understanding of the problem, but this
> > looks wrong to me: SYSROOT, EPREFIX and BROOT are only relevant in
> > build phases (src_*); (EPREFIX is a little special here but mostly
> > for convenience). ROOT is only relevant in pkg_* phases. I don't
> > see how this can work. Say I build a binpkg with ROOT=/ then use
> > that binpkg with ROOT=/somewhere, you can't go back and change
> > SYSROOT.
> 
> ROOT is used to determine ESYSROOT, not the other way around. As you
> say, (E)SYSROOT is only relevant in src phases so it doesn't matter if
> ROOT has changed when installing a binpkg.

I am missing something here: You are making ESYSROOT depend on the
value of ROOT, so how can it not matter ?

> I take your point that setting a src phase variable based on a pkg
> phase variable seems odd but we're only using ROOT to determine the
> applicable prefix. We're not taking the actual value of ROOT. When
> Portage works all this out, it has access to all the necessary
> variables. It only filters the variables based on the phase function
> later on.

The value does not really matter, the dependency of a variable used in
src_* from a variable that can change when installing a binpkg is what
worries me here.


Alexis.
-BEGIN PGP SIGNATURE-

iHUEAREIAB0WIQSpOxaxaZikKNVNlsYOJUi7xgflrgUCXUGc/gAKCRAOJUi7xgfl
rrlNAP9AEX0enc5Tzh++N6I+P4ZKp0hBdljlYBteYTuEipZLEAD/a3fT/pgOR3HJ
LyGvu98wHn6sCldtBMjoV/RgrxfaKO8=
=L9Hp
-END PGP SIGNATURE-

Re: [gentoo-dev] [PMS] [PATCH] Correct the definition of ESYSROOT as EPREFIX isn't always applicable

[Alexis Ballier]

On Sun, 28 Jul 2019 22:37:35 +0100
James Le Cuirot  wrote:

[...]
> -& \t{BDEPEND} &
> \t{DEPEND} & \t{RDEPEND}, \t{PDEPEND} \\
> +& \t{BDEPEND} &
> \t{DEPEND}& \t{RDEPEND}, \t{PDEPEND} \\
> \midrule


There seems to be unneeded whitespace only changes here that make the
diff less readable

[...]
>  \t{ESYSROOT} &
>  Ditto &
>  No &
> -Contains the concatenation of the paths in the \t{SYSROOT} and
> \t{EPREFIX} variables,
> -for convenience. See also the \t{EPREFIX} variable. Only for
> EAPIs listed
> -in table~\ref{tab:offset-env-vars-table} as supporting
> \t{ESYSROOT}. \\
> +Contains the concatenation of the \t{SYSROOT} path and
> applicable prefix value. The prefix value
> +is \t{EPREFIX}, \t{BROOT}, or blank if \t{SYSROOT} is equal to
> \t{ROOT}, \t{/}, or something
> +else respectively. Only for EAPIs listed in
> table~\ref{tab:offset-env-vars-table} as supporting
> +\t{ESYSROOT}. \\
>  \t{BROOT} &
>  Ditto &
>  No &

Admittedly without a full understanding of the problem, but this looks
wrong to me: SYSROOT, EPREFIX and BROOT are only relevant in build
phases (src_*); (EPREFIX is a little special here but mostly for
convenience). ROOT is only relevant in pkg_* phases. I don't see how
this can work. Say I build a binpkg with ROOT=/ then use that binpkg
with ROOT=/somewhere, you can't go back and change SYSROOT.

Also, I think the wording could be more "programmatic" (if ... then
ESYSROOT is ... else ... ) to avoid confusion.


Alexis.

Re: [gentoo-dev] Announcing RISC-V

[Alexis Ballier]

On Wed, 29 May 2019 10:27:34 -0700 (PDT)
Palmer Dabbelt  wrote:

> On Mon, 20 May 2019 02:44:18 PDT (-0700), aball...@gentoo.org wrote:
> > On Sat, 18 May 2019 20:47:28 +0200
> > Michał Górny  wrote:
> >
> >> On Fri, 2019-05-03 at 23:34 +0200, Andreas K. Huettel wrote:
> >> > * We will initially add two profiles to profile.desc: 
> >> >   default/linux/riscv/17.0/rv64gc/lp64d (non-multilib, 64bit
> >> > hardfloat) default/linux/riscv/17.0/rv64gc (multilib lp64d/lp64,
> >> > i.e. hard/softfloat)
> >> 
> >> I still don't understand the purpose of this multilib.  If you have
> >> a hardfloat CPU, why would you ever build some of the software
> >> softfloat?
> >
> > One reason I could imagine is that the hardfloat isn't IEEE 754
> > compliant. Searching through the RISC-V spec, it does not seem to be
> > the case here (ie: it is required to be compliant) so I'm also
> > wondering what is the point here.
> 
> The RISC-V floating-point extensions are IEEE-754 compliant, but
> they're optional.  We have chips without floating-point units, but
> right now all the Linux capable chips have FPUs.  As far as I know
> there are no Linux binaries that anyone cares about that are compiled
> for systems without hardware floating-point units, but I may be wrong
> about that one.


It was my understanding that FPU is not optional for rv64gc, is that
correct ?

> The non-FPU systems are much more interesting in embedded land, where
> lots of users don't have FPUs.  That's less relevant for Gentoo, but
> I do use crossdev embedded toolchains.


You'll probably not be using multilib here but rather a specific CHOST
and/or flags to enable softfloat everywhere.


[...]


Alexis.

Re: [gentoo-dev] Announcing RISC-V

[Alexis Ballier]

On Sat, 18 May 2019 20:47:28 +0200
Michał Górny  wrote:

> On Fri, 2019-05-03 at 23:34 +0200, Andreas K. Huettel wrote:
> > * We will initially add two profiles to profile.desc: 
> >   default/linux/riscv/17.0/rv64gc/lp64d (non-multilib, 64bit
> > hardfloat) default/linux/riscv/17.0/rv64gc (multilib lp64d/lp64,
> > i.e. hard/softfloat)
> 
> I still don't understand the purpose of this multilib.  If you have
> a hardfloat CPU, why would you ever build some of the software
> softfloat?


One reason I could imagine is that the hardfloat isn't IEEE 754
compliant. Searching through the RISC-V spec, it does not seem to be
the case here (ie: it is required to be compliant) so I'm also
wondering what is the point here.


Note that hard vs soft float will be the least of our concerns
here: Based on wikipedia, I count 4 base ISA (2 stable, 2 dev) and 13
optional extensions (6 stable, 7 dev) so we potentially have an
exponential number of ABIs here...

Re: [gentoo-dev] Rethinking multilib flags in Gentoo

[Alexis Ballier]

On Wed, 8 May 2019 11:08:47 -0700
Matt Turner  wrote:

> On Wed, May 8, 2019 at 3:19 AM Alexis Ballier 
> wrote:
> >
> > On Wed, 08 May 2019 12:01:21 +0200
> > Michał Górny  wrote:
> >
> > > On Wed, 2019-05-08 at 11:54 +0200, Alexis Ballier wrote:
> > > > On Wed, 08 May 2019 11:41:41 +0200
> > > > Michał Górny  wrote:
> > > >
> > > > > > There's multilib that adds a lot of flags with a single
> > > > > > eclass change, but I'd guess the number of packages and
> > > > > > flags is constantly growing, so sooner or later you'll be
> > > > > > hit by this again and no multilib killing will help you
> > > > > > then.
> > > > > >
> > > > > > I think it is more future proof to use the addition of
> > > > > > multilib flags to fix pkgcheck rather than actively
> > > > > > reducing the number of multilib flags to cope with its
> > > > > > limitations.
> > > > >
> > > > > Then please do it, by all means.  The reality is simple.  If
> > > > > the tool is broken, you either fix it or stop doing what you
> > > > > know that breaks it. Being unable to do the former, and
> > > > > having no good replacement, I'd go for the latter.
> > > >
> > > > Well, why is it slow ? IO ? CPU ? Did you collect profiling
> > > > data ?
> > >
> > > CPU definitely.  More detail than that, I don't and I don't have
> > > time to investigate.
> >
> > So you don't have time to change 3 lines to add cProfile but do have
> > time to send various emails and rework the entire multilib system ?
> > weird.
> 
> This isn't productive.
> 
> If you'd like to do the work you're suggesting, I'm sure Michał will
> support that, but as is you're just passive-aggressively questioning
> his choices in the regards to the multilib system he created and the
> CI system he created.
> 


You're right about the passive-aggressive part, sorry about it. But
maybe there's a single not so important check that's killing the
performance and could be, instead, disabled meanwhile. No data has been
shown except that adding useflags to packages make CI explode.
I am indeed questioning the choices of making tree wide changes because
a CI system is under performing and see nothing wrong about this.

Re: [gentoo-dev] Rethinking multilib flags in Gentoo

[Alexis Ballier]

On Wed, 08 May 2019 12:31:47 +0200
Michał Górny  wrote:

> On Wed, 2019-05-08 at 12:19 +0200, Alexis Ballier wrote:
> > On Wed, 08 May 2019 12:01:21 +0200
> > Michał Górny  wrote:
> > 
> > > On Wed, 2019-05-08 at 11:54 +0200, Alexis Ballier wrote:
> > > > On Wed, 08 May 2019 11:41:41 +0200
> > > > Michał Górny  wrote:
> > > > 
> > > > > > There's multilib that adds a lot of flags with a single
> > > > > > eclass change, but I'd guess the number of packages and
> > > > > > flags is constantly growing, so sooner or later you'll be
> > > > > > hit by this again and no multilib killing will help you
> > > > > > then.
> > > > > > 
> > > > > > I think it is more future proof to use the addition of
> > > > > > multilib flags to fix pkgcheck rather than actively
> > > > > > reducing the number of multilib flags to cope with its
> > > > > > limitations.
> > > > > 
> > > > > Then please do it, by all means.  The reality is simple.  If
> > > > > the tool is broken, you either fix it or stop doing what you
> > > > > know that breaks it. Being unable to do the former, and
> > > > > having no good replacement, I'd go for the latter.
> > > > 
> > > > Well, why is it slow ? IO ? CPU ? Did you collect profiling
> > > > data ?
> > > 
> > > CPU definitely.  More detail than that, I don't and I don't have
> > > time to investigate.
> > 
> > So you don't have time to change 3 lines to add cProfile but do have
> > time to send various emails and rework the entire multilib system ?
> > weird.
> 
> Instead of being so smug, you could have provided helpful instructions
> on how to do it.  Or did it yourself.

;)

I wasn't being smug, rather the contrary by assuming you would know
better than me.

So here we go:
https://docs.python.org/2/library/profile.html

$ python -m cProfile -o /full/path/to/file.log /usr/bin/command args

This will give you a profile log.

Then you need something to visualize it, I like runsnakerun.
http://www.vrplumber.com/programming/runsnakerun/


> Removing unused flags from multilib-build.eclass is certainly less
> work than that.

As a temporarily solution yes.

Re: [gentoo-dev] Rethinking multilib flags in Gentoo

[Alexis Ballier]

On Wed, 08 May 2019 12:01:21 +0200
Michał Górny  wrote:

> On Wed, 2019-05-08 at 11:54 +0200, Alexis Ballier wrote:
> > On Wed, 08 May 2019 11:41:41 +0200
> > Michał Górny  wrote:
> > 
> > > > There's multilib that adds a lot of flags with a single eclass
> > > > change, but I'd guess the number of packages and flags is
> > > > constantly growing, so sooner or later you'll be hit by this
> > > > again and no multilib killing will help you then.
> > > > 
> > > > I think it is more future proof to use the addition of multilib
> > > > flags to fix pkgcheck rather than actively reducing the number
> > > > of multilib flags to cope with its limitations.
> > > 
> > > Then please do it, by all means.  The reality is simple.  If the
> > > tool is broken, you either fix it or stop doing what you know
> > > that breaks it. Being unable to do the former, and having no good
> > > replacement, I'd go for the latter.
> > 
> > Well, why is it slow ? IO ? CPU ? Did you collect profiling data ?
> 
> CPU definitely.  More detail than that, I don't and I don't have time
> to investigate.

So you don't have time to change 3 lines to add cProfile but do have
time to send various emails and rework the entire multilib system ?
weird.


> > > > Also, remember that multilib is not entirely about skype or
> > > > slack, this was made with multibin in mind too: for example an
> > > > ABI may perform better than another one on specific workflows
> > > > (x32) and it may make sense to use this abi for a specific
> > > > binary (which would be manually built for now).
> > > 
> > > No, it weren't.  Just because someone found it convenient to use
> > > the design beyond its original purpose doesn't mean it had a
> > > different purpose.  Besides, how many 'multibin' packages do we
> > > have right now?
> > 
> > It was, because portage multilib was doing this and claimed to have
> > a use for this.
> 
> I am not talking of 'portage multilib'.  I am talking of
> multilib-build implementation.  If you are only concerned about the
> former, we can surely drop those abi_* flags that are irrelevant to
> it, can't we?

I am also talking about multilib-build and its way of doing cleanly and
replacing portage multilib.


> >  Its original purpose was generic enough to allow multibin
> > to be added easily. The number of multibin packages is only
> > relevant to show that this is not important enough for someone
> > to step up and do it: Everybody can easily build anything themselves
> > with the current implementation.
> 
> Exactly.  And that's why I believe having fast CI is more important
> than DIY multibin.  If someone can 'easily build anything
> themselves', they can also add needed flags locally.


It's a "little" more complex than just adding one flag to an eclass and
cannot really be done with an overlay; in the end, this requires to fork
gentoo.git more or less.

Re: [gentoo-dev] Rethinking multilib flags in Gentoo

[Alexis Ballier]

On Wed, 08 May 2019 11:41:41 +0200
Michał Górny  wrote:

> > There's multilib that adds a lot of flags with a single eclass
> > change, but I'd guess the number of packages and flags is
> > constantly growing, so sooner or later you'll be hit by this again
> > and no multilib killing will help you then.
> > 
> > I think it is more future proof to use the addition of multilib
> > flags to fix pkgcheck rather than actively reducing the number of
> > multilib flags to cope with its limitations.
> 
> Then please do it, by all means.  The reality is simple.  If the tool
> is broken, you either fix it or stop doing what you know that breaks
> it. Being unable to do the former, and having no good replacement,
> I'd go for the latter.


Well, why is it slow ? IO ? CPU ? Did you collect profiling data ?
Where are the scripts to repro the issue ?


> > Also, remember that multilib is not entirely about skype or slack,
> > this was made with multibin in mind too: for example an ABI may
> > perform better than another one on specific workflows (x32) and it
> > may make sense to use this abi for a specific binary (which would
> > be manually built for now).
> 
> No, it weren't.  Just because someone found it convenient to use the
> design beyond its original purpose doesn't mean it had a different
> purpose.  Besides, how many 'multibin' packages do we have right now?


It was, because portage multilib was doing this and claimed to have a
use for this. Its original purpose was generic enough to allow multibin
to be added easily. The number of multibin packages is only
relevant to show that this is not important enough for someone
to step up and do it: Everybody can easily build anything themselves
with the current implementation.


> That said, yes, I am also concerned about people proactively adding
> multilib to all random libraries which probably will never have any
> real multilib use case and only waste time of people who have
> abi_x86_32 enabled globally.

That last part is easily fixable: Have a useflag config option that
says "default off unless some package wants it on". This can be either
in the ebuild themselves (alike +/- iuse defaults) or, more simply, in
the package manager. None ever saw the light.

Re: [gentoo-dev] Rethinking multilib flags in Gentoo

[Alexis Ballier]

On Tue, 07 May 2019 23:47:30 +0200
Michał Górny  wrote:

> While the large number of flags is practically invisible to user with
> all the USE_EXPAND hiding, it negatively impacts pkgcheck.  When
> the number reached 10, CI became unusable.  We're currently back down
> to 8, thanks to powerpc team, but the problem is going to happen again
> sooner or later.  Ideally we'd improve pkgcheck but I'm not aware of
> anyone having a good idea how to do it.


While I don't disagree with your rationale below, I think this
motivation is the wrong one: What sort of algorithm does it use
to explode when going from 8 to 10 flags ?!?

There's multilib that adds a lot of flags with a single eclass change,
but I'd guess the number of packages and flags is constantly growing,
so sooner or later you'll be hit by this again and no multilib killing
will help you then.

I think it is more future proof to use the addition of multilib flags
to fix pkgcheck rather than actively reducing the number of multilib
flags to cope with its limitations.

Also, remember that multilib is not entirely about skype or slack,
this was made with multibin in mind too: for example an ABI may perform
better than another one on specific workflows (x32) and it may make
sense to use this abi for a specific binary (which would be manually
built for now).

Alexis.

Re: [gentoo-dev] [PATCH] opam.eclass: unbreak on EAPI=7

[Alexis Ballier]

On Mon, 04 Feb 2019 13:42:21 -0800
Georgy Yakovlev  wrote:

> On Monday, February 4, 2019 2:52:37 AM PST Alexis Ballier wrote:
> > On Sat,  2 Feb 2019 21:27:29 -0800
> > 
> > Georgy Yakovlev  wrote:
> > > Since D, ED, ROOT, EROOT no longer have a trailing slash in EAPI=7
> > > This eclass is terribly broken, installing things into
> > > imageusr/...
> > 
> > You might want to check https://github.com/aballier/ml-overlay
> Hi!
> 
> I don't really use this eclass in any way.
> A user reported breakage, I wrote a patch and submitted here for
> review.
> 
> there is a commit in overlay that adds slashes, but paths will end up
> having 
> 
> // two slashes on EAPI6 and single on EAPI7 
> 
> https://github.com/aballier/ml-overlay/commit/
> 98c0f16bc490349f17afdd7a7675b9b5264d267e
> 
> also probably the docdir subdir part of opam_src_install() will  also
> fail, as there are no slashes.
> 
> 
> ::gentoo version needs some kind of fix as any EAPI7 ebuild that uses 
> opam.eclass is broken.
> 
> If you are ok with my patches I can commit, just let me know.


yes go ahead (modulo list comments ofc =)

Re: [gentoo-dev] [PATCH] opam.eclass: unbreak on EAPI=7

[Alexis Ballier]

On Sat,  2 Feb 2019 21:27:29 -0800
Georgy Yakovlev  wrote:

> Since D, ED, ROOT, EROOT no longer have a trailing slash in EAPI=7
> This eclass is terribly broken, installing things into
> imageusr/...


You might want to check https://github.com/aballier/ml-overlay

Re: [gentoo-dev] Unmasking >=media-video/ffmpeg-4.0

[Alexis Ballier]

On Wed, 7 Nov 2018 09:57:36 -0500
Ian Stakenvicius  wrote:

> On 2018-11-06 11:21 a.m., Alexis Ballier wrote:
> > On Tue, 6 Nov 2018 11:09:17 -0500
> > Rich Freeman  wrote:
> >   
> >> On Tue, Nov 6, 2018 at 10:57 AM Alexis Ballier
> >>  wrote:  
> >>>
> >>> On Tue, 06 Nov 2018 17:08:22 +0200
> >>> Mart Raudsepp  wrote:
> >>>
> >>>> It is not GStreamer fault that ffmpeg breaks API and ABI without
> >>>> parallel installability, much less so the distro maintainers of
> >>>> it. If you/upstream don't make it parallel installable, then this
> >>>> is what you get.
> >>>
> >>> Are you, seriously, suggesting this is the solution to all
> >>> problems here ?
> >>>
> >>
> >> It isn't the only solution, but it is one sane upgrade path.  You
> >> can't expect everybody to update their software overnight when the
> >> API changes.  That means you have to support the old API for a
> >> while when you introduce a new one, otherwise you end up with some
> >> software that doesn't work with the old version, and some software
> >> that doesn't work with the new version.  
> > 
> > 
> > These days, only symbols/constants that have been deprecated (and
> > marked as such) for a couple of releases are removed. This means
> > people see warnings for more than one year before seeing them gone
> > for good. The problem here is not "overnight changes" but rather
> > consumers not paying attention to those warnings, or worse, nobody
> > ever seeing those because it's unmaintained.
> >   
> 
> But we aren't upstream most of the time, and if upstreams are pegging
> their ffmpeg to a single version they don't bother to try the newer
> one to find out the errors.  Take Kodi, v17.x is pegged to no newer
> than ffmpeg-3.3.x as I recall, and has been blocking even v3.4's
> installation for the year'ish it's been in the gentoo repo.
> 
> So this "people see warnings" thing, it really doesn't apply, unless
> you (A) have the desire and resources to build and maintain a patch
> for upstream, and (B) have an upstream with the desire and resources
> to support more than the one version of ffmpeg for a given release
> set.  Both, IMO, are in very short supply.
> 


Believe me, it's far easier to do this than maintaining several slotted
versions. See the amount of work done for e.g. gtk, qt, wxwidgets, etc.
and I'm not even counting backporting security fixes nor patching tens
of packages to "use the proper slot".

Note that the (B) desire is usually mostly killed by people claiming
multiple versions should be supported without any idea of the
implications this has. Fortunately, there's only very few upstreams
left that consider ffmpeg so special that they want to bundle or force
an old buggy and vulnerable version.

Re: [gentoo-dev] Unmasking >=media-video/ffmpeg-4.0

[Alexis Ballier]

On Tue, 6 Nov 2018 11:09:17 -0500
Rich Freeman  wrote:

> On Tue, Nov 6, 2018 at 10:57 AM Alexis Ballier 
> wrote:
> >
> > On Tue, 06 Nov 2018 17:08:22 +0200
> > Mart Raudsepp  wrote:
> >  
> > > It is not GStreamer fault that ffmpeg breaks API and ABI without
> > > parallel installability, much less so the distro maintainers of
> > > it. If you/upstream don't make it parallel installable, then this
> > > is what you get.  
> >
> > Are you, seriously, suggesting this is the solution to all problems
> > here ?
> >  
> 
> It isn't the only solution, but it is one sane upgrade path.  You
> can't expect everybody to update their software overnight when the API
> changes.  That means you have to support the old API for a while when
> you introduce a new one, otherwise you end up with some software that
> doesn't work with the old version, and some software that doesn't work
> with the new version.


These days, only symbols/constants that have been deprecated (and
marked as such) for a couple of releases are removed. This means people
see warnings for more than one year before seeing them gone for good.
The problem here is not "overnight changes" but rather consumers not
paying attention to those warnings, or worse, nobody ever seeing those
because it's unmaintained.


[...]

Alexis.

Re: [gentoo-dev] Unmasking >=media-video/ffmpeg-4.0

[Alexis Ballier]

On Tue, 06 Nov 2018 11:04:17 -0500
Craig Andrews  wrote:

> On 06.11.2018 06:00, Alexis Ballier wrote:
> > On Mon, 05 Nov 2018 20:38:58 -0500
> > Craig Andrews  wrote:
> >   
> >> I think it's time to remove the mask on >=media-video/ffmpeg-4.0
> >> 
> >> ffmpeg 4 is in many distros (including Debian, Arch) and FreeBSD.
> >> Upstreams are starting to require it, too. For example, Kodi 18 and
> >> media-plugins/gst-plugins-libav-16 will require it.
> >> 
> >> The blocker issue https://bugs.gentoo.org/653678 has only 5
> >> blockers against it right now:
> >> * 654628 has a pull request out to fix it:
> >> https://github.com/gentoo/gentoo/pull/10341
> >> * 655170 is for a package that (IMHO) should be last-rited:
> >> media-libs/mediastreamer
> >> * 666166 is for a package that (IMHO) should be last-rited:
> >> media-plugins/vdr-image
> >> * 655442 and 658128 are for media-video/mplayer and fixed upstream,
> >> requiring a new snapshot release
> >> 
> >> What do we say? Can we set a date when the hard mask will be
> >> removed?  
> > 
> > 
> > I think 6 months qualifies for ETIMEOUT, so go ahead and unmask
> > it :)
> > 
> > merge the github PR; I'll take care of mplayer  
> 
> Since Alexis added the mask I believe he can authorize its removal,
> so I have done so:
> https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=deb9d06fb461dc57291498b327e2eeb667444a5f


I forgot to mention it, but you should probably have checked the re-kw
bug if any and created one after dropping keywords with broken deps :/

Re: [gentoo-dev] Unmasking >=media-video/ffmpeg-4.0

[Alexis Ballier]

On Tue, 06 Nov 2018 17:08:22 +0200
Mart Raudsepp  wrote:

> Ühel kenal päeval, T, 06.11.2018 kell 12:00, kirjutas Alexis Ballier:
> > On Mon, 05 Nov 2018 20:38:58 -0500
> > Craig Andrews  wrote:
> >   
> > > I think it's time to remove the mask on >=media-video/ffmpeg-4.0
> > > 
> > > ffmpeg 4 is in many distros (including Debian, Arch) and FreeBSD.
> > > Upstreams are starting to require it, too. For example, Kodi 18
> > > and 
> > > media-plugins/gst-plugins-libav-16 will require it.
> > > 
> > > The blocker issue https://bugs.gentoo.org/653678 has only 5
> > > blockers 
> > > against it right now:
> > > * 654628 has a pull request out to fix it: 
> > > https://github.com/gentoo/gentoo/pull/10341
> > > * 655170 is for a package that (IMHO) should be last-rited: 
> > > media-libs/mediastreamer
> > > * 666166 is for a package that (IMHO) should be last-rited: 
> > > media-plugins/vdr-image
> > > * 655442 and 658128 are for media-video/mplayer and fixed
> > > upstream, 
> > > requiring a new snapshot release
> > > 
> > > What do we say? Can we set a date when the hard mask will be
> > > removed?  
> > 
> > 
> > I think 6 months qualifies for ETIMEOUT, so go ahead and unmask
> > it :)
> > 
> > merge the github PR; I'll take care of mplayer  
> 
> I'm sorry, but what?
> PR was filed yesterday, what 6 months are you talking about here?
> Please respect your fellow developers. We have other commitments and
> priorities and can't zero-day review a PR.


Oh, sorry, I did not even check what the PR was about nor its
uptime. It should definitely read as "get the PR merged" then.

6 months is when bugs started to be filled which by most standards a
package not fixed within that time can be considered dead/unmaintained.
I usually take no response for a couple of months as a green light to
do it myself.


> It is not GStreamer fault that ffmpeg breaks API and ABI without
> parallel installability, much less so the distro maintainers of it.
> If you/upstream don't make it parallel installable, then this is what
> you get.


Are you, seriously, suggesting this is the solution to all problems
here ?

Bests,

Alexis.

Re: [gentoo-dev] Unmasking >=media-video/ffmpeg-4.0

[Alexis Ballier]

On Mon, 05 Nov 2018 20:38:58 -0500
Craig Andrews  wrote:

> I think it's time to remove the mask on >=media-video/ffmpeg-4.0
> 
> ffmpeg 4 is in many distros (including Debian, Arch) and FreeBSD.
> Upstreams are starting to require it, too. For example, Kodi 18 and 
> media-plugins/gst-plugins-libav-16 will require it.
> 
> The blocker issue https://bugs.gentoo.org/653678 has only 5 blockers 
> against it right now:
> * 654628 has a pull request out to fix it: 
> https://github.com/gentoo/gentoo/pull/10341
> * 655170 is for a package that (IMHO) should be last-rited: 
> media-libs/mediastreamer
> * 666166 is for a package that (IMHO) should be last-rited: 
> media-plugins/vdr-image
> * 655442 and 658128 are for media-video/mplayer and fixed upstream, 
> requiring a new snapshot release
> 
> What do we say? Can we set a date when the hard mask will be removed?


I think 6 months qualifies for ETIMEOUT, so go ahead and unmask it :)

merge the github PR; I'll take care of mplayer

Re: [gentoo-dev] autotools.eclass and EAPI 7

[Alexis Ballier]

On Thu, 16 Aug 2018 21:36:56 +0300
Mart Raudsepp  wrote:

> Ühel kenal päeval, N, 16.08.2018 kell 14:27, kirjutas Brian Evans:
> > There are currently a handful of ebuilds using EAPI 7 and the
> > autotools
> > eclass.
> > 
> > I believe that this eclass should be reviewed for adding BDEPEND or
> > having BDEPEND replace the DEPEND statement as the default action of
> > the
> > eclass.
> > 
> > Other items might be needed, but that's doubtful.
> > 
> > Someone please advise the best course of action and either do it or
> > I will propose a patch based on the discussion.
> >   
> 
> Could or did someone also check through all the other eclasses that
> don't have any global EAPI compatibility checks?
> For the future, maybe it's better to add such a check - just accepting
> 0-7 or so, but unsure about all these custom EAPIs out there, might
> force more eclass copying to some overlays.

I don't really like that kind of checks: untested after usually small
changes != broken.

IMHO a better solution could be to have council members review all
eclasses prior to approving an eapi and either adding a comment why + a
die when used with the not-yet-approved-eapi if the eclass requires
changes or do nothing about it if it's fine. This has the double
advantage to give council members first hands experience with an eapi
before approving/voting it and ensures better migration when eapi is
approved.

Re: [gentoo-dev] rfc: [QA] Ban policy introduction SLIPUP

[Alexis Ballier]

On Tue, 31 Jul 2018 14:36:37 +0100
Amy Liffey  wrote:

> Hello folks,
> 
> I apologize to everyone for sending this proposal before it was
> finished. It was not voted on by the QA team hence it was not an
> official proposal by the QA team. There was probably some
> misunderstanding in communication.
> 
> After we finish the official draft and it is accepted by QA team
> members, we will be very happy to accept comments on the mailing list
> in the future.

During that drafting process, please clarify how this changes and
differs from the current GLEP48 wording and if the glep needs to be
updated:

If a particular developer persistently causes breakage, the QA team may
request that Comrel re-evaluates that developer's commit rights.
Evidence of past breakages will be presented with this request to
Comrel.





Alexis.

Re: [gentoo-dev] Re: What means bup?

[Alexis Ballier]

On Wed, 18 Jul 2018 14:20:56 +0200
Kristian Fiskerstrand  wrote:

> On 07/18/2018 02:10 PM, Alexis Ballier wrote:
> > I often find myself in the
> > need to use/invent some abbreviation in order to fit the limit.
> > Considering this is an error, this sends the message that short is
> > preferred over clear.  
> 
> Or that the summary should be concise and a longer proper description
> can be written in the body of the commit message instead. Potentially
> mixed in with multiple commits for different logical changes etc etc.
> 

Sure, but that somewhat defeats the point of a summary if it's not
possible to clearly express what it is about without relying on the
long description.

Re: [gentoo-dev] Re: What means bup?

[Alexis Ballier]

On Wed, 18 Jul 2018 10:35:41 +0200
Ulrich Mueller  wrote:

> > On Wed, 18 Jul 2018, Matthew Thode wrote:  
> 
> > On 18-07-18 09:16:07, Johannes Huber wrote:  
> >> english is not my mother language, so please clarify what bup
> >> means, just seen here:
> >> 
> >> https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=ee0e0401478cf30b3ced0405f6b89391e05d2397
> >> 
> >> Just checked if it was a typo, but can't be:
> >> git log --oneline --grep bup | count -l
> >> Expected 0 lines, got 1248.  
> 
> > It's similiar to a sound you make when you touch something's nose.
> > *boop*  
> 
> > I just prefer bup instead.  I generally only use it when doing
> > simple bumps of packages (copy ebuilds with only keyword edits).  
> 
> We used to have the rule that ChangeLog messages "should be well
> explained and written in clean English". I think this applies to
> commit messages, too.


While it does not really apply in this precise case, this rule's
weight has been lowered a lot with repoman enforcing 'cat/pkg: ...' +
hard limit on the commit message length. I often find myself in the
need to use/invent some abbreviation in order to fit the limit.
Considering this is an error, this sends the message that short is
preferred over clear.

Re: [gentoo-dev] How to provide a recent TeXLive for Gentoo for our users?

[Alexis Ballier]

On Mon, 26 Mar 2018 20:07:07 +0200
Jonas Stein  wrote:

> > [...]  
> >> This is not a very nice solution, but it works so far. One
> >> difficulty is, that there is no 1:1 relation between the texlive
> >> distribution and dev-texlive/* at the moment.  
> > There is a 1:1 relation.  
> 
> A full installation of TeXLive installs packages, which are in
> dev-tex/* and dev-texlive/* on gentoo.
> On the other hand single packages are bundled some times.
> Some programs/packages can be found in dev-tex/* and dev-texlive/*
> 
> I remember, that I last we had for example dev-tex/notoccite in the
> tree and dev-texlive/texlive-latexextra including shipped the same,
> but newer files.
> This is what I meant with "no 1:1 relation"

The idea with dev-tex/* packages is that they can be split out of
texlive and installed/updated independently as long as they are
actively maintained. While you might have a need for specific updates
on some packages, I seriously doubt you need (and can find the manpower
to maintain) a live ebuild for the whole CTAN and can live with yearly
texlive updates.

The notoccite example is what happens over the years: package is left
unmaintained, texlive catches up and the package becomes useless.
Worse: if done properly it will overlay the more recent version from
texlive! Those packages should be reported and removed (or updated).


> >> How can we enable our users to run a recent TeXLive in a clean
> >> way?  
> > /usr/local/share/texmf has been supported by texlive on Gentoo from
> > day one. You can use that. It's an overlay that takes precedence on
> > anything else, so per the above, you lose all the QA & testing done
> > behind the scenes if you use this.  
> 
> And packages which depend on a specific LaTeX package will not
> install. So the user has to provide a package.provided list, which is
> not so nice to maintain for so many packages.

They will install, only they will pull older unused files from
texlive. As said above, the proper way to avoid this is to step up as
maintainer of some dev-tex/* packages, fix the deps to add a || or
(probably after convincing me it's worth it because the package is
updated very frequently) remove it entirely from texlive and switch the
deps.


Alexis.

Re: [gentoo-dev] Proper dependencies for TeX related ebuilds

[Alexis Ballier]

On Mon, 26 Mar 2018 11:40:12 +0200
Jonas Stein  wrote:

> Hi,
> 
> I just read
> https://bugs.gentoo.org/625908#c5
> and saw that we do not have a good documentation on this and  many
> packages in the tree ship with strange tex dependencies.
> 
> I started a documentation on our wiki
> https://wiki.gentoo.org/wiki/Notes_on_TeX_related_ebuilds

Well, your example (inkscape) is wrong: dep on tl-core is *never*
sufficient and certainly does not provide a working latex command. First
step to get it right is to use the already existing virtuals
(virtual/{la,}tex-base).

Re: [gentoo-dev] How to provide a recent TeXLive for Gentoo for our users?

[Alexis Ballier]

On Mon, 26 Mar 2018 11:57:49 +0200
Jonas Stein  wrote:
> An installation via tlmgr provides many updates per day, but
> our distributed TeXLive is unfortunately always behind.
> Typically TeXLive on gentoo is 6-12 months behind upstream, because we
> have to bump a lot manually.

That is not the real reason. Most of it has been scripted for 10+ years.
The real reason is that we need to go through ~arch testing, fixing rev
deps that might need update to their .tex files because the underlying
packages they use has changed, adapt the deps for some potential
changes, and then a stablereq round. Following the yearly release cycle
leaves time for it to happen. It could move faster, but I don't see any
need for it as this would mean shortening stabilization cycles
potentially allowing for more bugs to enter stable and increasing the
load on arch teams.


[...]
> This is not a very nice solution, but it works so far. One difficulty
> is, that there is no 1:1 relation between the texlive distribution and
> dev-texlive/* at the moment.

There is a 1:1 relation.


> How can we enable our users to run a recent TeXLive in a clean way?

/usr/local/share/texmf has been supported by texlive on Gentoo from day
one. You can use that. It's an overlay that takes precedence on
anything else, so per the above, you lose all the QA & testing done
behind the scenes if you use this.


Alexis.

Re: [gentoo-dev] News Item: Portage Dynamic Deps

[Alexis Ballier]

On Sun, 21 Jan 2018 23:01:08 -0800
Zac Medico  wrote:

> Please review.
> 
> Title: Portage Dynamic Deps
> Author: Zac Medico 
> Posted: 2018-01-28
> Revision: 1
> News-Item-Format: 2.0
> Display-If-Installed:  
> Beginning with Portage 2.3.20, the previous default --dynamic-deps=y
> setting has changed to --dynamic-deps=n. Due to this change, some
> users may experience emerge dependency calculation failures triggered
> by installed packages that have outdated dependencies. In order to
> avoid problems of this nature, use the emerge --changed-deps=y option
> with your next deep @world update.

What's the rationale behind this ?

What I mean is: while '--dynamic-deps=n --changed-deps=n' is the
technically correct behavior, this just seems like throwing unbearable
dep calculation failure messages at users' faces while we could default
to '--dynamic-deps=n --changed-deps=y' and get the already
policy-mandated behavior of 'force a rebuild when you change deps'.

Re: [gentoo-dev] [PATCH 1/2] preserve-libs.eclass: Split off preserve_old_lib from eutils.

[Alexis Ballier]

On Thu, 4 Jan 2018 10:35:23 -0500
Mike Gilbert  wrote:

> On Thu, Jan 4, 2018 at 5:23 AM, Pacho Ramos  wrote:
> > I have seen this is only used by:
> > app-arch/xz-utils
> > dev-libs/gmp
> > dev-libs/libpcre
> > dev-libs/mpc
> > dev-libs/mpfr
> > net-nds/openldap
> > sys-libs/gdbm
> > sys-libs/ncurses
> > sys-libs/readline
> > sys-process/audit
> >
> > Maybe we could deprecate it and try to drop it in the future :/
> >  
> 
> As Soap touched on earlier, this should probably not be
> deprecated/removed until a solution compatible with Paludis and
> pkgcore is implemented.
> 
> A couple of options for that:
> 
> 1. Add functionality similar to preserve-libs to these alternate
> package managers. This is unlikely to happen.

IMHO having profiles mandate a preserve-libs behavior (and thus PMS
define this) for a few select packages (the ones above) is the way to go
here: preserve-libs is evil but really the least evil in this case.
If you miss the warning from the eclass, you can very easily end up
with binaries loading the two versions of the library; preserve-libs
has the advantage that portage is very loud about this.

Re: [gentoo-dev] [QA] New policy: 'files' directory must not be larger than 32 KiB

[Alexis Ballier]

On Wed, 20 Dec 2017 08:34:14 -0500
Rich Freeman <ri...@gentoo.org> wrote:

> On Wed, Dec 20, 2017 at 4:42 AM, Alexis Ballier <aball...@gentoo.org>
> wrote:
> > On Tue, 19 Dec 2017 16:00:16 -0500
> > "Aaron W. Swenson" <titanof...@gentoo.org> wrote:
> >  
> >> However, what alternative do we have to throwing the patches up in
> >> a devspace?  
> >
> > mirror://gentoo, aka /space/distfiles-local/
> >  
> 
> That isn't a great option.  This has been discussed previously:
> 
>https://lists.gt.net/gentoo/dev/224673?do=post_view_threaded
> 
>
> https://devmanual.gentoo.org/general-concepts/mirrors/index.html#suitable-download-hosts
> 
> 
> I would point out that a devspace isn't the only alternative -
> anything stable would be reasonable.

So... 6 years later... still no solution ? Kid me not.

I don't see what any other hosting style changes here: next time infra
checks pecker homedirs, everyone will spend time manually removing old
distfiles. For archival of sources, there has always been
gentoo/src/patchsets. For extreme cases, there's distfiles-whitelist.

Re: [gentoo-dev] [QA] New policy: 'files' directory must not be larger than 32 KiB

[Alexis Ballier]

On Tue, 19 Dec 2017 16:00:16 -0500
"Aaron W. Swenson"  wrote:

> On 2017-12-17 14:21, Michał Górny wrote:
> >   Total size of 'files' subdirectory of a package should not be
> > larger than 32 KiB. If the package needs more auxiliary files, they
> > should be put into SRC_URI e.g. via tarballs.  
> 
> I don’t have any strong opinions about this either way.

note that the announcement fails to mention why this has been a
self-imposed rule on some packages for a long time: tools like quilt or
git are made to track large patchsets. Once scripting is in place, it is
much more convenient to generate patch tarballs from those tools; epatch
supporting numbered patchsets and applying them all is great for that
too.

> However, what alternative do we have to throwing the patches up in a
> devspace?

mirror://gentoo, aka /space/distfiles-local/

> Having previously done so with dev-db/postgresql, it was annoying
> having to fix the SRC_URI because I wasn’t the one who did a slot
> bump.

or even that way: you can add several URIs for the same file, e.g.:
SRC_URI="pecker/~deva/patches.tar.bz2 pecker/~devb/patches.tar.bz2"

mirrors will pick the one available and usually users wont be impacted
by a 404 delay if devb hosts the patches; but I would still recommend
distfiles-local, esp. since it autocleans when the file is not used
anymore.

Re: [gentoo-dev] Reviving the Sandbox project

[Alexis Ballier]

On Fri, 22 Sep 2017 19:39:16 +0200
Michał Górny <mgo...@gentoo.org> wrote:

> W dniu pią, 22.09.2017 o godzinie 19∶15 +0200, użytkownik Alexis
> Ballier napisał:
> > On Fri, 22 Sep 2017 17:20:23 +0200
> > Michał Górny <mgo...@gentoo.org> wrote:
> >   
> > > W dniu pią, 22.09.2017 o godzinie 12∶57 +0200, użytkownik Alexis
> > > Ballier napisał:  
> > > > On Fri, 22 Sep 2017 06:07:18 +0200
> > > > Michał Górny <mgo...@gentoo.org> wrote:
> > > > 
> > > > > W dniu czw, 21.09.2017 o godzinie 15∶41 -0700, użytkownik Matt
> > > > > Turner napisał:
> > > > > > On Thu, Sep 21, 2017 at 2:25 PM, Michał Górny
> > > > > > <mgo...@gentoo.org> wrote:  
> > > > > > > Given that sandbox is utterly broken by design, I don't
> > > > > > > really want to put too much effort in trying to make it a
> > > > > > > little better. I'd rather put the minimal effort required
> > > > > > > to make it not-much-worse.  
> > > > > > 
> > > > > > You said in your initial email that you weren't an expert
> > > > > > in its internals, but here you say it's broken by design.
> > > > > > Why do you think that?
> > > > > >   
> > > > > 
> > > > > Because it uses LD_PRELOAD which is a huge hack and which
> > > > > causes guaranteed issues we can't really fix. All we can do
> > > > > is disable it for emacs, for compiler-rt and I'm afraid this
> > > > > list will grow because overriding random library functions is
> > > > > never a good idea. 
> > > > 
> > > > I think we're all ears for a better solution. There are probably
> > > > much better ways to do sandboxing these days than 15 years ago.
> > > > 
> > > > LD_PRELOAD does not work with static binaries. Hence the non
> > > > portable ptrace stuff. Hence bugs. Etc. The point is, that's the
> > > > best we have now.
> > > > 
> > > 
> > > I know of two obvious alternatives: ptrace and filesystem layer
> > > (e.g. FUSE).
> > > 
> > > For the former, there's sydbox. I'm going to look into
> > > integrating it into Portage when I have more time.  
> > 
> > From: https://github.com/alip/pinktrace/blob/master/configure.ac
> > case "$host_cpu" in
> > i[[3456]]86|pentium)
> > x86?64*|amd64)
> > ia64)
> > powerpc64*)
> > powerpc*)
> > arm*)
> >  [add support for those arches]
> > *)
> > AC_MSG_RESULT([NO!])
> > AC_MSG_ERROR([Architecture $host_cpu is not supported by
> > pinktrace]) ;;
> > 
> > sandbox keywords:
> > 2.11-r5:0: ~alpha ~amd64 ~arm ~arm64 ~hppa ~ia64 ~m68k ~mips ~ppc
> >  ~ppc64 ~s390 ~sh ~sparc ~sparc-fbsd ~x86 ~x86-fbsd
> > 
> > 
> > Good luck adding the missing bits!
> > 
> >   
> > > For the latter, I have writing one in TODO. But I'm not sure when
> > > I'll have enough time to do work on it.  
> > 
> > Not sure how that would work, but you'll likely need some kind of
> > chroot/container since you don't want to trust a random binary ran
> > as root to respect environment variables.
> >   
> 
> Environment variables? What for?
> 

I don't know :)
I have no idea how you would provide a virtual FS that would be the
only thing ever seen by all processes spawned.

Re: [gentoo-dev] Reviving the Sandbox project

[Alexis Ballier]

On Fri, 22 Sep 2017 17:20:23 +0200
Michał Górny <mgo...@gentoo.org> wrote:

> W dniu pią, 22.09.2017 o godzinie 12∶57 +0200, użytkownik Alexis
> Ballier napisał:
> > On Fri, 22 Sep 2017 06:07:18 +0200
> > Michał Górny <mgo...@gentoo.org> wrote:
> >   
> > > W dniu czw, 21.09.2017 o godzinie 15∶41 -0700, użytkownik Matt
> > > Turner napisał:  
> > > > On Thu, Sep 21, 2017 at 2:25 PM, Michał Górny
> > > > <mgo...@gentoo.org> wrote:
> > > > > Given that sandbox is utterly broken by design, I don't really
> > > > > want to put too much effort in trying to make it a little
> > > > > better. I'd rather put the minimal effort required to make it
> > > > > not-much-worse.
> > > > 
> > > > You said in your initial email that you weren't an expert in its
> > > > internals, but here you say it's broken by design. Why do you
> > > > think that?
> > > > 
> > > 
> > > Because it uses LD_PRELOAD which is a huge hack and which causes
> > > guaranteed issues we can't really fix. All we can do is disable
> > > it for emacs, for compiler-rt and I'm afraid this list will grow
> > > because overriding random library functions is never a good idea.
> > >   
> > 
> > I think we're all ears for a better solution. There are probably
> > much better ways to do sandboxing these days than 15 years ago.
> > 
> > LD_PRELOAD does not work with static binaries. Hence the non
> > portable ptrace stuff. Hence bugs. Etc. The point is, that's the
> > best we have now.
> >   
> 
> I know of two obvious alternatives: ptrace and filesystem layer (e.g.
> FUSE).
> 
> For the former, there's sydbox. I'm going to look into integrating it
> into Portage when I have more time.

From: https://github.com/alip/pinktrace/blob/master/configure.ac
case "$host_cpu" in
i[[3456]]86|pentium)
x86?64*|amd64)
ia64)
powerpc64*)
powerpc*)
arm*)
 [add support for those arches]
*)
AC_MSG_RESULT([NO!])
AC_MSG_ERROR([Architecture $host_cpu is not supported by
pinktrace]) ;;

sandbox keywords:
2.11-r5:0: ~alpha ~amd64 ~arm ~arm64 ~hppa ~ia64 ~m68k ~mips ~ppc
 ~ppc64 ~s390 ~sh ~sparc ~sparc-fbsd ~x86 ~x86-fbsd


Good luck adding the missing bits!


> For the latter, I have writing one in TODO. But I'm not sure when I'll
> have enough time to do work on it.

Not sure how that would work, but you'll likely need some kind of
chroot/container since you don't want to trust a random binary ran as
root to respect environment variables.

Alexis.

Re: [gentoo-dev] Reviving the Sandbox project

[Alexis Ballier]

On Fri, 22 Sep 2017 12:38:54 +0100
Sergei Trofimovich <sly...@gentoo.org> wrote:

> On Fri, 22 Sep 2017 12:57:21 +0200
> Alexis Ballier <aball...@gentoo.org> wrote:
> 
> > On Fri, 22 Sep 2017 06:07:18 +0200
> > Michał Górny <mgo...@gentoo.org> wrote:
> >   
> > > W dniu czw, 21.09.2017 o godzinie 15∶41 -0700, użytkownik Matt
> > > Turner napisał:
> > > > On Thu, Sep 21, 2017 at 2:25 PM, Michał Górny
> > > > <mgo...@gentoo.org> wrote:  
> > > > > Given that sandbox is utterly broken by design, I don't really
> > > > > want to put too much effort in trying to make it a little
> > > > > better. I'd rather put the minimal effort required to make it
> > > > > not-much-worse.  
> > > > 
> > > > You said in your initial email that you weren't an expert in its
> > > > internals, but here you say it's broken by design. Why do you
> > > > think that?
> > > >   
> > > 
> > > Because it uses LD_PRELOAD which is a huge hack and which causes
> > > guaranteed issues we can't really fix. All we can do is disable
> > > it for emacs, for compiler-rt and I'm afraid this list will grow
> > > because overriding random library functions is never a good idea.
> > > 
> > 
> > I think we're all ears for a better solution. There are probably
> > much better ways to do sandboxing these days than 15 years ago.
> > 
> > LD_PRELOAD does not work with static binaries. Hence the non
> > portable ptrace stuff. Hence bugs. Etc. The point is, that's the
> > best we have now.  
> 
> Some other distros try harder to isolate build environment either
> through chroot and/or private mount/user/network namespace that
> contains only explicitly specified files in build environment.
> 
> That would require more cooperation from package manager to fetch
> list of all visible depends.
> 
> Don't know if drop-in relacement could be implemented for sandbox
> that way. I like clear sandbox error reporting.


We definitely do need a kind of drop-in replacement since PMS
mandates some parts of the sandbox API (addwrite/addpredict & co for
instance)

Re: [gentoo-dev] Reviving the Sandbox project

[Alexis Ballier]

On Fri, 22 Sep 2017 06:07:18 +0200
Michał Górny  wrote:

> W dniu czw, 21.09.2017 o godzinie 15∶41 -0700, użytkownik Matt Turner
> napisał:
> > On Thu, Sep 21, 2017 at 2:25 PM, Michał Górny 
> > wrote:  
> > > Given that sandbox is utterly broken by design, I don't really
> > > want to put too much effort in trying to make it a little better.
> > > I'd rather put the minimal effort required to make it
> > > not-much-worse.  
> > 
> > You said in your initial email that you weren't an expert in its
> > internals, but here you say it's broken by design. Why do you think
> > that?
> >   
> 
> Because it uses LD_PRELOAD which is a huge hack and which causes
> guaranteed issues we can't really fix. All we can do is disable it for
> emacs, for compiler-rt and I'm afraid this list will grow because
> overriding random library functions is never a good idea.
> 

I think we're all ears for a better solution. There are probably much
better ways to do sandboxing these days than 15 years ago.

LD_PRELOAD does not work with static binaries. Hence the non
portable ptrace stuff. Hence bugs. Etc. The point is, that's the
best we have now.


Alexis.

Re: [gentoo-dev] glibc-2.26 and changes with SunRPC, libtirpc, ntirpc, libnsl (NIS and friends), ...

[Alexis Ballier]

On Mon, 18 Sep 2017 11:56:30 +0200
"Andreas K. Huettel"  wrote:

> sunrpc - build against glibc

Now that I think about it: What about other libcs ? musl, uclibc,
freebsd or even the prefix ones ?

[...]
> Porting a package means adding a dependency in the style of 
> || ( 

Re: [gentoo-dev] glibc-2.26 and changes with SunRPC, libtirpc, ntirpc, libnsl (NIS and friends), ...

[Alexis Ballier]

On Mon, 18 Sep 2017 11:56:30 +0200
"Andreas K. Huettel"  wrote:

> Porting a package means adding a dependency in the style of 
> || ( 

[gentoo-dev] Re: [gentoo-commits] repo/gentoo:master commit in: sys-libs/zlib/

[Alexis Ballier]

On Sat,  9 Sep 2017 21:07:02 + (UTC)
"Lars Wendler"  wrote:

> commit: b5f0471fe97de820f888702e1938989b9fd25522
> Author: David Seifert  gentoo  org>
> AuthorDate: Wed Sep  6 11:48:56 2017 +
> Commit: Lars Wendler  gentoo  org>
> CommitDate: Sat Sep  9 21:06:46 2017 +
> URL:
> https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=b5f0471f
> 
> sys-libs/zlib: Add sublot
[...]
> -SLOT="0"
> +SLOT="0/1" # subslot = SONAME

Not much to do in this case now, but can we please stop this insanity
in the future ?

You're not adding any subslot here, you're changing it from its default
${SLOT} (0) to 1, causing this:

 (sys-libs/zlib-1.2.11:0/1::gentoo, ebuild scheduled for merge) causes
 rebuilds for: (app-text/ghostscript-gpl-9.21:0/0::gentoo, ebuild
 scheduled for merge) (media-libs/libpng-1.6.32:0/16::gentoo, ebuild
 scheduled for merge) (dev-libs/libxml2-2.9.5:2/2::gentoo, ebuild
 scheduled for merge) (dev-db/mariadb-10.2.8:0/18::gentoo, ebuild
 scheduled for merge) (x11-libs/libpciaccess-0.13.5:0/0::gentoo, ebuild
 scheduled for merge) (dev-lang/python-2.7.13:2.7/2.7::gentoo, ebuild
 scheduled for merge) (media-libs/openjpeg-2.2.0:2/7::gentoo, ebuild
 scheduled for merge) (media-libs/tiff-4.0.8:0/0::gentoo, ebuild
 scheduled for merge) (net-misc/openssh-7.5_p1-r2:0/0::gentoo, ebuild
 scheduled for merge) (sys-devel/llvm-4.0.1:4/4::gentoo, ebuild
 scheduled for merge) (sys-auth/consolekit-1.2.0:0/0::gentoo, ebuild
 scheduled for merge) (dev-lang/python-3.4.6:3.4/3.4m::gentoo, ebuild
 scheduled for merge) (media-gfx/imagemagick-7.0.6.9:0/7.0.6.9::gentoo,
 ebuild scheduled for merge)



Thank you.


Alexis.

Re: [gentoo-dev] Of death and prerm

[Alexis Ballier]

On Tue, 29 Aug 2017 16:38:15 -0400
Michael Orlitzky  wrote:

> What should happen if an ebuild calls "die" in pkg_prerm?
> 
> The issue arose while trying to create a package that could not be
> uninstalled except as part of an upgrade. The first thing that came to
> mind was to have it die in pkg_prerm.
> 
> What portage does is *appear* to crash, but then continue along as if
> nothing happened.
> 
> Does the PMS cover this indirectly? (Is there a reliable way to make
> package removal fail?)

Is there any point in dying in any phase after (or during)
pkg_postinst ?
files are already live by then; what would this achieve ?

Re: [gentoo-dev] New eclass: opam.eclass

[Alexis Ballier]

> >   
> > >   "${pkg}.install" || die
> > >   done
> > > }
> > > 
> > > opam_src_install() {
> > >   opam-install "${PN}"
> > >   # Handle opam putting doc in a subdir
> > >   if [ -d "${ED}/usr/share/doc/${PF}/${PN}" ] ; then
> > 
> > Is PN always the correct subdirectory here?  
> 
> yes because opam-install is called for $PN only here


and guess what: $PN is not always the proper package name...

Re: [gentoo-dev] New eclass: opam.eclass

[Alexis Ballier]

On Tue, 25 Jul 2017 16:18:10 +0200
Michał Górny <mgo...@gentoo.org> wrote:

> On pon, 2017-07-24 at 17:20 +0200, Alexis Ballier wrote:
> > # Copyright 1999-2017 Gentoo Foundation
> > # Distributed under the terms of the GNU General Public License v2
> > 
> > # @ECLASS: opam.eclass
> > # @MAINTAINER:
> > # Gentoo ML Project <m...@gentoo.org>
> > # @AUTHOR:
> > # Alexis Ballier <aball...@gentoo.org>
> > # @BLURB: Provides functions for installing opam packages.
> > # @DESCRIPTION:
> > # Provides dependencies on opam and ocaml, opam-install and a
> > default # src_install for opam-based packages.
> > 
> > case ${EAPI:-0} in
> > 0|1|2|3|4) die "You need at least EAPI-5 to use opam.eclass";;  
> 
> Why not start straight with EAPI 6? You will have less to clean up
> later.


opam-based ebuilds are good with EAPI 6 but some other ocaml
eclasses are still waiting for patches from those deprecating
eclasses/features with EAPI6. So, EAPI5 is still the norm in ocaml
(it really is min for := though), and EAPI update can happen later, I'm
definitely not in a hurry to go for new EAPIs :)

> 
> > *) ;;
> > esac
> >   
> > RDEPEND=">=dev-lang/ocaml-4:="  
> > DEPEND="${RDEPEND}
> > dev-ml/opam"
> > 
> > # @FUNCTION: opam-install
> > # @USAGE: 
> > # @DESCRIPTION:
> > # Installs the opam packages given as arguments. For each "${pkg}"
> > element in # that list, "${pkg}.install" must be readable from
> > current working directory. opam-install() {  
> 
> local pkg

fixed

> 
> > for pkg ; do
> > opam-installer -i \
> > --prefix="${ED}/usr" \
> > --libdir="${D}/$(ocamlc -where)" \
> > --docdir="${ED}/usr/share/doc/${PF}" \
> > --mandir="${ED}/usr/share/man" \  
> 
> Both ED and D include the trailing slash, so either remove the extra
> slash or use ${ED%/}.

thx, removed /

by the way, is there any technical reason to use ${ED%/} ?
as in, let the shell do it when the OS can probably do it much better ?


> 
> > "${pkg}.install" || die
> > done
> > }
> > 
> > opam_src_install() {
> > opam-install "${PN}"
> > # Handle opam putting doc in a subdir
> > if [ -d "${ED}/usr/share/doc/${PF}/${PN}" ] ; then  
> 
> Is PN always the correct subdirectory here?

yes because opam-install is called for $PN only here

for multiple packages in one ebuild that won't work well, that is why I
didn't include this in opam-install itself.


the idea with the eclass is to use it to split all opam based package to
have only 1 ebuild per corresponding opam package though



And pushed in a few minutes


Thanks!

Re: [gentoo-dev] New eclass: opam.eclass

[Alexis Ballier]

On Mon, 24 Jul 2017 18:11:39 -0400
"Aaron W. Swenson" <titanof...@gentoo.org> wrote:

> On 2017-07-24 17:20, Alexis Ballier wrote:
> > Hey,
> > 
> > Here is an eclass that would allow me to factor quite a bit of
> > redundant code.
> > 
> > …
> > if [ -d "${ED}/usr/share/doc/${PF}/${PN}" ] ; then  
> 
> It’s always been recommended to me that we should use the [[ … ]]
> form.

Doesn't make much difference here and I've always been recommending the
other way :p
(the idea is that you don't get to write the [[ ]] in a /bin/sh script
just because you're used to it; if you only do ebuilds or bash, then
you don't care, but I definitely do other scripts)

[gentoo-dev] New eclass: opam.eclass

[Alexis Ballier]

Hey,

Here is an eclass that would allow me to factor quite a bit of
redundant code.

Potential users:
https://qa-reports.gentoo.org/output/genrdeps/dindex/dev-ml/opam


Examples of conversion:

diff --git a/dev-ml/ocaml-cstruct/ocaml-cstruct-3.1.1.ebuild
b/dev-ml/ocaml-cstruct/ocaml-cstruct-3.1.1.ebuild index
0acf2607860..5e238f762db 100644 ---
a/dev-ml/ocaml-cstruct/ocaml-cstruct-3.1.1.ebuild +++
b/dev-ml/ocaml-cstruct/ocaml-cstruct-3.1.1.ebuild @@ -3,7 +3,7 @@
 
 EAPI=5
 
-inherit findlib
+inherit findlib opam
 
 DESCRIPTION="Map OCaml arrays onto C-like structs"
 HOMEPAGE="https://github.com/mirage/ocaml-cstruct https://mirage.io;
@@ -57,18 +57,6 @@ src_test() {
jbuilder runtest -p $(get_targets) || die
 }
 
-oinstall() {
-   opam-installer -i \
-   --prefix="${ED}/usr" \
-   --libdir="${D}/$(ocamlc -where)" \
-   --docdir="${ED}/usr/share/doc/${PF}" \
-   ${1}.install || die
-}
-
 src_install() {
-   oinstall cstruct
-   oinstall cstruct-unix
-   use lwt && oinstall cstruct-lwt
-   use async && oinstall cstruct-async
-   use ppx && oinstall ppx_cstruct
+   opam-install $(get_targets | tr ',' ' ')
 }


diff --git a/dev-ml/utop/utop-2.0.1.ebuild
b/dev-ml/utop/utop-2.0.1.ebuild index 90056da08e8..c3bec3b1f94 100644
--- a/dev-ml/utop/utop-2.0.1.ebuild
+++ b/dev-ml/utop/utop-2.0.1.ebuild
@@ -3,7 +3,7 @@
 
 EAPI=5
 
-inherit findlib
+inherit findlib opam
 
 DESCRIPTION="A new toplevel for OCaml with completion and colorization"
 HOMEPAGE="https://github.com/diml/utop;
@@ -30,12 +30,3 @@ DEPEND="${DEPEND}
 
 DOCS=( "CHANGES.md" "README.md" )
 SITEFILE="50${PN}-gentoo.el"
-
-src_install() {
-   opam-installer -i \
-   --prefix="${ED}/usr" \
-   --libdir="${D}/$(ocamlc -where)" \
-   --docdir="${ED}/usr/share/doc/${PF}" \
-   --mandir="${ED}/usr/share/man" \
-   ${PN}.install || die
-}



Alexis.# Copyright 1999-2017 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2

# @ECLASS: opam.eclass
# @MAINTAINER:
# Gentoo ML Project <m...@gentoo.org>
# @AUTHOR:
# Alexis Ballier <aball...@gentoo.org>
# @BLURB: Provides functions for installing opam packages.
# @DESCRIPTION:
# Provides dependencies on opam and ocaml, opam-install and a default
# src_install for opam-based packages.

case ${EAPI:-0} in
0|1|2|3|4) die "You need at least EAPI-5 to use opam.eclass";;
*) ;;
esac

RDEPEND=">=dev-lang/ocaml-4:="
DEPEND="${RDEPEND}
dev-ml/opam"

# @FUNCTION: opam-install
# @USAGE: 
# @DESCRIPTION:
# Installs the opam packages given as arguments. For each "${pkg}" element in
# that list, "${pkg}.install" must be readable from current working directory.
opam-install() {
for pkg ; do
opam-installer -i \
--prefix="${ED}/usr" \
--libdir="${D}/$(ocamlc -where)" \
--docdir="${ED}/usr/share/doc/${PF}" \
--mandir="${ED}/usr/share/man" \
"${pkg}.install" || die
done
}

opam_src_install() {
opam-install "${PN}"
# Handle opam putting doc in a subdir
if [ -d "${ED}/usr/share/doc/${PF}/${PN}" ] ; then
mv "${ED}/usr/share/doc/${PF}/${PN}/"* 
"${ED}/usr/share/doc/${PF}/" || die
rmdir "${ED}/usr/share/doc/${PF}/${PN}" || die
fi
}

EXPORT_FUNCTIONS src_install

Re: [gentoo-dev] taking a break from arches stabilization

[Alexis Ballier]

On Mon, 10 Jul 2017 19:22:20 +0200
Agostino Sarubbo  wrote:

> Hi all.
> 
> every time that I attach my tmux session to see what happens on irc,
> I always see the same discussion about the 'minor' arches status.
> Since I joined gentoo(2011) I worked on all arches except hppa, I put
> more effort in amd64 and less where I saw other people work on it
> (arm,alpha) But every time the magic phrase is that those arches are
> unmaintained.
> 
> Now, since I work on these arches just to help, i.e. I don't have any
> business and I do non have any installation of those arches and the
> work I'm doing is not appreciated at all I decided to stop for now.
> If your dream is to put sparc and ia64 to ~arch, go ahead, I have no 
> objections.
> I will take a break also from amd64 and x86...let's see how things
> will change.
> 

Thanks for your work. It is appreciated. I respect your decision that
your time might be better spent by not trying to artificially maintain
some arches alive when you're clearly the only one doing the work and
caring about them.

There's an annoying side effect of open source communities that I've
came to witness a lot: When you do your work too well, nobody feels the
need to help by contributing, and you're essentially left alone.


Alexis.

Re: [gentoo-dev] About adding a *warning* to remind maintainers to check for new PYTHON_COMPAT values

[Alexis Ballier]

On Mon, 10 Jul 2017 11:55:30 -0500
William Hubbs  wrote:

> On Mon, Jul 10, 2017 at 01:04:10PM +0200, Pacho Ramos wrote:
> > Hello
> > 
> > Looking to the list of packages still not supporting python 3.5:
> > https://qa-reports.gentoo.org/output/gpyutils/34-to-35.txt
> > 
> > and considering that we should even start testing python 3.6, I
> > think it would be nice if we could make portage to warn when
> > PYTHON_COMPAT value is not updated. It's really frustrating to
> > still see new ebuilds being added with obsolete values for
> > PYTHON_COMPAT and relying on a few people looking to update this.
> > This is also causing huge delays to migrate to newer python
> > versions and I think it's responsibility of the maintainer to
> > ensure his/her package is supported on newer versions or, at least,
> > have a bug and ping upstream for the cases they need further fixing.
> > 
> > Of course, this wouldn't be a fatal check preventing you from
> > committing a package with outdated PYTHON_COMPAT, it would be a
> > warning to remind you to update it as soon as possible.
> > 
> > Any issues on trying to go further into implementing this
> > warning?   
> 
> What about the situation where a package is not compatible with newer
> versions of python so does not need a PYTHON_COMPAT change?
> 
> I don't think you can assume PYTHON_COMPAT is outdated for a package
> just because it doesn't have the latest versions of python
> listed. The only time you can know for sure that it is outdated is if
> it lists a version of python that no longer exists in the tree.

Maybe we should add some - modifier to PYTHON_COMPAT, just like
keywords, indicating it has been tested to *not* work.
PYTHON_COMPAT=( python{2_7,3_4,3_5} -python_3_6 )

Re: [gentoo-dev] Pre-GLEP RFC: Automated enforcing of REQUIRED_USE constraints

[Alexis Ballier]

On Sun, 09 Jul 2017 13:36:16 +0200
Michał Górny <mgo...@gentoo.org> wrote:

> On nie, 2017-07-09 at 11:29 +0200, Alexis Ballier wrote:
> > You don't seem to get how normalizing and constant
> > propagation/elimination works.
> > 
> > Basically, reordering would be:
> > And(list) -> And( forced(list) + free(list) + masked(list) )
> > Or(list)  -> Or( ... . )
> > etc.
> > 
> > While normalizing is:
> > And(list) -> False if False appears in a normalize(x) for x in list,
> >  True if normalize(x) for x in list is empty or all
> > True, And(normalize(x) for x in list if != True and != False)
> > etc.
> > 
> > That's described in one point: Apply boolean algebra rules.  
> 
> Last I checked, implementing a full-fledged boolean algebra processor
> with reductions and other magic is a non-goal here.


That is not a goal for auto solving required use, but catching
uninstallable packages is probably more important than that...


> > What I don't like about reordering is that it is too tightly
> > coupled to the following solving algorithm and the restricted
> > syntax, while really, having REQUIRED_USE constraints asking you to
> > enable a masked flag is something we ought to kill even without
> > solving as they hide broken deps and make all our QA checks less
> > useful.  
> 
> It's irrelevant since we kill the unsupported syntax anyway.

Your logic is flawed. You kill unsupported syntax because you fail to
get it right with all the complexity you're carrying, not the other way
around.


> > Finally, reordering, being essentially a local thing, does not have
> > the proper behavior in a general AST:
> > '|| ( ( a b ) c )' with 'a' and 'b' masked will be left invariant by
> > reordering and the resulting expanded form will be rejected while
> > constant propagation/elimination will reduce that to 'c' and be
> > good.  
> 
> Handling a rejected syntax is irrelevant.

Rejected by PMS ?

> > Hence, the reordering check cannot be used as a good input for
> > checking for broken REQUIRED_USE: I really think things like
> > 'vulkan? ( || ( video_cards_i965 video_cards_radeonsi ) )' should
> > be a repoman error on stable profiles where both those video cards
> > are masked and vulkan is not. For that, we need to support the
> > whole PMS-defined syntax, not a reduced one.  
> 
> No, we don't. It works just fine. The only difference is that it stops
> on the first erroneous constraint.

Don't you think it's a bit of an understatement saying that an
optional auto solver might need to enable masked useflag when in fact
there is a useflag that can never be enabled, auto solver or not ?

This also makes CI/repoman dep checks fail to catch broken cases: As of
today, nothing will catch a game depending on mesa[vulkan] and being
~arm. Good luck installing such a game on arm though.


> > > No, it is not. You do not have the values of all the items inside
> > > the group, just some of them. Depending on how many of them do you
> > > have and what are them, you need to transform the group
> > > appropriately, e.g. by removing items, replacing the group or
> > > failing entirely.  
> > 
> > Yes, that's boolean algebra rules. You propagate constants from
> > leaves to root in the AST and if some 'False' appears in your AST
> > when you've reached the root you fail. I agree one needs some
> > practice on recursive structures to understand that quickly and
> > easily though.  
> 
> Except that we're dealing with structures which don't strictly follow
> boolean algebra rules, as you've already noticed.

Hmmm. No ?


> > > > > > One big advantage of working on ASTs is that it becomes
> > > > > > trivial to suggest a proper reordering.  
> > > > > 
> > > > > Reordering is never a trivial problem. Unless I'm missing
> > > > > something, it is technically possible that a 'reordering' will
> > > > > actually require a sub- item being moved out of the containing
> > > > > group.
> > > > 
> > > > Not if done at the AST level.
> > > > 
> > > > > And to be honest, I find the output of the verification
> > > > > script in this regard quite useful. That is, it saying 'X
> > > > > affects Y, so it needs to go before it' is quite clear to me.
> > > > > I don't think most developers would actually need to script
> > > > > to pinpoint a specific location for every single
> > > > > constraint.
> > > > 
> > > > In most cases 

Re: [gentoo-dev] Pre-GLEP RFC: Automated enforcing of REQUIRED_USE constraints

[Alexis Ballier]

On Sat, 08 Jul 2017 23:56:07 +0200
Michał Górny <mgo...@gentoo.org> wrote:

> On sob, 2017-07-08 at 22:34 +0200, Alexis Ballier wrote:
> > On Sat, 08 Jul 2017 20:44:24 +0200
> > Michał Górny <mgo...@gentoo.org> wrote:
> >   
> > > On sob, 2017-07-08 at 16:12 +0200, Alexis Ballier wrote:  
> > > > On Sat, 08 Jul 2017 11:43:39 +0200
> > > > Michał Górny <mgo...@gentoo.org> wrote:
> > > > 
> > > > > Hi, everyone.
> > > > > 
> > > > > I think the affairs have settled enough and I've finished
> > > > > filling in the pre-GLEP for REQUIRED_USE auto-enforcing. It's
> > > > > got all the algorithms, rationale and separated reference
> > > > > implementation.
> > > > > 
> > > > > If there are no major concerns raised, I will soon start
> > > > > working on writing an optimized implementation for
> > > > > pkgcore/pkgcheck and integrating the verification algos with
> > > > > the CI.
> > > > > 
> > > > > The pre-GLEP for review is here:
> > > > > 
> > > > > https://wiki.gentoo.org/wiki/User:MGorny/GLEP:ReqUse
> > > > 
> > > > 
> > > > Constraint group reordering algorithm
> > > > 
> > > > I really think we should only consider REQUIRED_USE with
> > > > forced/masked useflags instantiated there. And ban (in repoman)
> > > > REQUIRED_USE that contain some "False": "a? ( b )" with 'a' free
> > > > and 'b' masked is perfectly ok now but it hides a serious
> > > > problem in the package/profile. Instantiating this would give:
> > > > "a? ( False )" and be an error just like we have depend.bad &
> > > > co. This is independent of auto solving or not, it's already
> > > > wrong.
> > > 
> > > As I've already explained you multiple times, this obtains
> > > *exactly the same* effect. However, it's much simpler when it's
> > > done like this because it makes it possible to reuse the already
> > > defined algorithms instead of having to precisely define how to
> > > preprocess REQUIRED_USE for this and cover all the corner cases.  
> > 
> > Simpler??? I don't think so. What I wrote clearly pinpoints that:
> > When you'll write the algorithm for "Verifying that the constraints
> > do not alter immutable flags" you'll notice this is exactly that
> > and can be put as a preprocessing step and then you can drop all
> > the corner cases considerations for immutable flags. I never
> > understood why you're insisting that much on immutables: they're
> > really not natural, not simple, not standard, and carrying them all
> > over seems to be a burden to me.  
> 
> I wrote the algorithms, and they're simple. This specific check is
> the combination of three simple steps:
> 
> a. reordering the groups based on immutables,
> 
> b. transforming the AST into flat form,
> 
> c. verifying each flat constraint.
> 
> The first step is trivial -- it's basically 'move true to front, false
> to back'. The second step is more complex but it's needed anyway,
> and quite well-defined, especially with the assumption that all
> the groups always have at least one flag inside. The third step is
> trivial again because it's just checking the conditions and
> constraints against a list.
> 
> The alternative to reordering is altering the groups. Altering means
> we need to have separate logic for every type of group while sorting
> works the same in all of them. Altering means we need to explicitly
> special case forcing 1 and >1 items, and masking all items, for each
> group. Again, sorting does not need to be concerned about that because
> the check following it (also trivial) will catch it anyway.

You don't seem to get how normalizing and constant
propagation/elimination works.

Basically, reordering would be:
And(list) -> And( forced(list) + free(list) + masked(list) )
Or(list)  -> Or( ... . )
etc.

While normalizing is:
And(list) -> False if False appears in a normalize(x) for x in list,
 True if normalize(x) for x in list is empty or all True,
 And(normalize(x) for x in list if != True and != False)
etc.

That's described in one point: Apply boolean algebra rules.

What I don't like about reordering is that it is too tightly coupled to
the following solving algorithm and the restricted syntax, while really,
having REQUIRED_USE constraints asking you to enable a masked flag is
something we ought to kill even without solving as they

Re: [gentoo-dev] Pre-GLEP RFC: Automated enforcing of REQUIRED_USE constraints

[Alexis Ballier]

On Sat, 08 Jul 2017 20:44:24 +0200
Michał Górny <mgo...@gentoo.org> wrote:

> On sob, 2017-07-08 at 16:12 +0200, Alexis Ballier wrote:
> > On Sat, 08 Jul 2017 11:43:39 +0200
> > Michał Górny <mgo...@gentoo.org> wrote:
> >   
> > > Hi, everyone.
> > > 
> > > I think the affairs have settled enough and I've finished filling
> > > in the pre-GLEP for REQUIRED_USE auto-enforcing. It's got all
> > > the algorithms, rationale and separated reference implementation.
> > > 
> > > If there are no major concerns raised, I will soon start working
> > > on writing an optimized implementation for pkgcore/pkgcheck
> > > and integrating the verification algos with the CI.
> > > 
> > > The pre-GLEP for review is here:
> > > 
> > > https://wiki.gentoo.org/wiki/User:MGorny/GLEP:ReqUse  
> > 
> > 
> > Constraint group reordering algorithm
> > 
> > I really think we should only consider REQUIRED_USE with
> > forced/masked useflags instantiated there. And ban (in repoman)
> > REQUIRED_USE that contain some "False": "a? ( b )" with 'a' free
> > and 'b' masked is perfectly ok now but it hides a serious problem
> > in the package/profile. Instantiating this would give: "a? ( False
> > )" and be an error just like we have depend.bad & co. This is
> > independent of auto solving or not, it's already wrong.  
> 
> As I've already explained you multiple times, this obtains *exactly
> the same* effect. However, it's much simpler when it's done like this
> because it makes it possible to reuse the already defined algorithms
> instead of having to precisely define how to preprocess REQUIRED_USE
> for this and cover all the corner cases.

Simpler??? I don't think so. What I wrote clearly pinpoints that:
When you'll write the algorithm for "Verifying that the constraints do
not alter immutable flags" you'll notice this is exactly that and can
be put as a preprocessing step and then you can drop all the corner
cases considerations for immutable flags. I never understood why you're
insisting that much on immutables: they're really not natural, not
simple, not standard, and carrying them all over seems to be a burden
to me.

> > Reordering is a dangerous path as we've already seen since it can
> > create unexpected loops for the solver.  
> 
> Freeform reordering is dangerous, and I've removed that already.
> Reordering restricted to immutables can not cause any issues that any
> other solution wouldn't cause.

You're very likely right there. Any proof? Esp. any proof that the
checker still guarantees the existence of a solution in all cases?
I'm not asking for a formal proof, but simply a bit more details than
just an assertion saying it's fine.

> > Working on instantiated REQUIRED_USE constraints would probably
> > simplify quite a bit your GLEP too: you already have the "Verifying
> > that the constraints do not alter immutable flags" part that roughly
> > does the same thing as instantiating, except if you assume it's
> > already true you can skip the reordering.  
> 
> Except that the reordering can be described in 2 points, and so can be
> the immutability verification. Please prove that you can provide
> a simpler explanation that doesn't fail in any of the corner cases.

Except reordering is an invention and immutable checking is simply
applying boolean logic rules to your implication and check that no
"False" can appear. You can simply start by applying boolean logic and
forget about reordering.


> > Concept for transforming REQUIRED_USE into implications
> > 
> > Ok, now I probably understand better the concept of common prefix.
> > I'm definitely biased here, but I would feel better with a more
> > recursive presentation of it. Assume we have 'validate(list of
> > clauses)'; basically, the common prefix idea is that for an
> > implication 'foo? ( consequences = list of clauses )' you first
> > validate the consequences as if it were a REQUIRED_USE (so that the
> > subtree rooted at foo is not self-conflicting) and then consider
> > the whole thing as a clause. The idea would then be to have similar
> > checks as to what you've written but working on trees (ASTs)
> > instead of flattened clauses. This would avoid having to deal with
> > unique identities (these would come for free) and IMHO would be
> > easier to understand. I'm not sure how to do this though, I'll ping
> > you when I have some idea.  
> 
> Well, the problem of common prefix is quite complex, and I'm not even
> sure if it's really worth more consideration. After all, we're
> prettych much talking about 

Re: [gentoo-dev] Pre-GLEP RFC: Automated enforcing of REQUIRED_USE constraints

[Alexis Ballier]

On Sat, 8 Jul 2017 21:05:57 +0200
Ulrich Mueller <u...@gentoo.org> wrote:

> >>>>> On Sat, 8 Jul 2017, Ciaran McCreesh wrote:  
> 
> > On Sat, 8 Jul 2017 16:39:29 +0200
> > Alexis Ballier <aball...@gentoo.org> wrote:  
> >> Indeed, makes sense. Would it also make sense to have some more
> >> logical meaning in a future EAPI ? I mean, in every context I've
> >> ever seen, applying a rule to the empty set is the neutral of that
> >> rule, so that it preserves associativity.
> >> That'd mean: || ( ) is false, && ( ) is true, ^^ ( ) is false,  
> 
> I have no strong opinion about this. Is it worth the effort of
> changing the spec?
> 
> >> ?? ( ) is false.  
> 
> I think ?? ( ) aka at-most-one-of should be true if empty.

Maybe; this one is annoying I think since it is not associative per
definition:
?? ( true ?? ( false false ) ) -> ?? ( true true ) -> false
?? ( ?? ( true false ) false ) -> ?? ( true false) -> true


> > The sensible thing to do is ban it, and additionally ban use? ( )
> > inside || and ^^ (if we've not done that already...).  
> 
> Right. We have to check if this will break any eclass generated
> dependencies, though.

That's probably the best course of action indeed.

Alexis.

Re: [gentoo-dev] Pre-GLEP RFC: Automated enforcing of REQUIRED_USE constraints

[Alexis Ballier]

On Sat, 8 Jul 2017 15:23:39 +0100
Ciaran McCreesh <ciaran.mccre...@googlemail.com> wrote:

> On Sat, 8 Jul 2017 16:14:09 +0200
> Alexis Ballier <aball...@gentoo.org> wrote:
> > On Sat, 8 Jul 2017 13:01:39 +0100
> > Ciaran McCreesh <ciaran.mccre...@googlemail.com> wrote:  
> > > On Sat, 8 Jul 2017 13:49:56 +0200
> > > Alexis Ballier <aball...@gentoo.org> wrote:
> > > > On Sat, 8 Jul 2017 12:26:59 +0200
> > > > Ulrich Mueller <u...@gentoo.org> wrote:  
> > > > > | * An any-of group (||) evaluates to true if at least one of
> > > > > the | items in it evaluates to true.
> > > > > | * An exactly-one-of group (^^) evaluates to true if exactly
> > > > > one of | the items in it evaluates to true, and all the
> > > > > remaining items | evaluate to false.
> > > > > | * An at-most-one-of group (??) evaluates to true if at most
> > > > > one of | the items in it evaluates to true.
> > > > > 
> > > > > It should be added that any empty group (|| or ^^ or ??)
> > > > > evalutates to true, because that's what PMS specifies:
> > > > > https://projects.gentoo.org/pms/6/pms.html#x1-780008.2
> > > > 
> > > > A bit OT, but that is *definitely* counter intuitive. What's the
> > > > rationale and usecase behind this ?  
> > > 
> > > Annoying special cases like || ( foo? ( ... ) bar? ( ... ) ) . The
> > > original reason was that old versions of Portage would simply
> > > remove unmet "flag? ( )" blocks internally. It was kept in EAPI 0
> > > because stuff in the tree used it back then.  
> > 
> > Wasn't REQUIRED_USE something completely new with no prior usage in
> > EAPI 3 ?  
> 
> Yes, but the spec defines dependency-like structures and their
> meanings once and consistently, rather than all over the place and
> inconsistently.
> 
> As much as I hate the weird || ( use ? ( ) ) and empty block rules, it
> would be worse to have them apply in some situations but not others.

Indeed, makes sense. Would it also make sense to have some more logical
meaning in a future EAPI ? I mean, in every context I've ever seen,
applying a rule to the empty set is the neutral of that rule, so that
it preserves associativity.
That'd mean: || ( ) is false, && ( ) is true, ^^ ( ) is false, ?? ( )
is false.

Alexis.

Re: [gentoo-dev] Pre-GLEP RFC: Automated enforcing of REQUIRED_USE constraints

[Alexis Ballier]

On Sat, 8 Jul 2017 13:01:39 +0100
Ciaran McCreesh <ciaran.mccre...@googlemail.com> wrote:

> On Sat, 8 Jul 2017 13:49:56 +0200
> Alexis Ballier <aball...@gentoo.org> wrote:
> > On Sat, 8 Jul 2017 12:26:59 +0200
> > Ulrich Mueller <u...@gentoo.org> wrote:  
> > > | * An any-of group (||) evaluates to true if at least one of the
> > > | items in it evaluates to true.
> > > | * An exactly-one-of group (^^) evaluates to true if exactly one
> > > of | the items in it evaluates to true, and all the remaining
> > > items | evaluate to false.
> > > | * An at-most-one-of group (??) evaluates to true if at most one
> > > of | the items in it evaluates to true.
> > > 
> > > It should be added that any empty group (|| or ^^ or ??)
> > > evalutates to true, because that's what PMS specifies:
> > > https://projects.gentoo.org/pms/6/pms.html#x1-780008.2
> > 
> > A bit OT, but that is *definitely* counter intuitive. What's the
> > rationale and usecase behind this ?  
> 
> Annoying special cases like || ( foo? ( ... ) bar? ( ... ) ) . The
> original reason was that old versions of Portage would simply remove
> unmet "flag? ( )" blocks internally. It was kept in EAPI 0 because
> stuff in the tree used it back then.
> 

Wasn't REQUIRED_USE something completely new with no prior usage in
EAPI 3 ?

Re: [gentoo-dev] Pre-GLEP RFC: Automated enforcing of REQUIRED_USE constraints

[Alexis Ballier]

On Sat, 08 Jul 2017 11:43:39 +0200
Michał Górny  wrote:

> Hi, everyone.
> 
> I think the affairs have settled enough and I've finished filling
> in the pre-GLEP for REQUIRED_USE auto-enforcing. It's got all
> the algorithms, rationale and separated reference implementation.
> 
> If there are no major concerns raised, I will soon start working
> on writing an optimized implementation for pkgcore/pkgcheck
> and integrating the verification algos with the CI.
> 
> The pre-GLEP for review is here:
> 
> https://wiki.gentoo.org/wiki/User:MGorny/GLEP:ReqUse


Constraint group reordering algorithm

I really think we should only consider REQUIRED_USE with forced/masked
useflags instantiated there. And ban (in repoman) REQUIRED_USE that
contain some "False": "a? ( b )" with 'a' free and 'b' masked is
perfectly ok now but it hides a serious problem in the package/profile.
Instantiating this would give: "a? ( False )" and be an error
just like we have depend.bad & co. This is independent of auto
solving or not, it's already wrong.

Reordering is a dangerous path as we've already seen since it can
create unexpected loops for the solver.

Working on instantiated REQUIRED_USE constraints would probably
simplify quite a bit your GLEP too: you already have the "Verifying
that the constraints do not alter immutable flags" part that roughly
does the same thing as instantiating, except if you assume it's already
true you can skip the reordering.





Concept for transforming REQUIRED_USE into implications

Ok, now I probably understand better the concept of common prefix. I'm
definitely biased here, but I would feel better with a more recursive
presentation of it. Assume we have 'validate(list of clauses)';
basically, the common prefix idea is that for an implication 'foo?
( consequences = list of clauses )' you first validate the consequences
as if it were a REQUIRED_USE (so that the subtree rooted at foo is
not self-conflicting) and then consider the whole thing as a clause.
The idea would then be to have similar checks as to what you've written
but working on trees (ASTs) instead of flattened clauses. This would
avoid having to deal with unique identities (these would come for free)
and IMHO would be easier to understand.
I'm not sure how to do this though, I'll ping you when I have some idea.

One big advantage of working on ASTs is that it becomes trivial to
suggest a proper reordering.

---

Restrictions on REQUIRED_USE format

I still fail to see the point here. One can simply apply the rewriting
you suggest below and be done with it. Rationale is not very convincing
to me:

- avoiding unpredictable results of automatic flag adjustments:
A deterministic algorithm is, by definition, predictable.
- improving readability of REQUIRED_USE constraints:
No need for a restriction for that. If people want to shoot
themselves in the foot, it is not a PMS problem. I see that
like proposing death penalty for those who commit suicide :)
- keeping the specification and implementation relatively simple:
You already define everything for working without restriction.
Plus, unlimited implication nesting has the same complexity.

---


Do you have numbers on the checker run on all inputs from gentoo-x86 ?
Since we're dealing with heuristics those are particularly important to
validate we're not rejecting too many constructs.


Alexis.

Re: [gentoo-dev] Pre-GLEP RFC: Automated enforcing of REQUIRED_USE constraints

[Alexis Ballier]

On Sat, 8 Jul 2017 12:26:59 +0200
Ulrich Mueller  wrote:

> | * An any-of group (||) evaluates to true if at least one of the
> | items in it evaluates to true.
> | * An exactly-one-of group (^^) evaluates to true if exactly one of
> | the items in it evaluates to true, and all the remaining items
> | evaluate to false.
> | * An at-most-one-of group (??) evaluates to true if at most one of
> | the items in it evaluates to true.
> 
> It should be added that any empty group (|| or ^^ or ??) evalutates to
> true, because that's what PMS specifies:
> https://projects.gentoo.org/pms/6/pms.html#x1-780008.2

A bit OT, but that is *definitely* counter intuitive. What's the
rationale and usecase behind this ?

Re: [gentoo-dev] The status of grsecurity upstream and hardened-sources downstream

[Alexis Ballier]

On Fri, 23 Jun 2017 12:28:27 -0400
"Anthony G. Basile"  wrote:

> Hardened Gentoo has two sides to it, kernel hardening (done via
> hardened-sources) and toolchain/executable hardening.  The two are
> interrelated but independent enough that toolchain hardening can
> continue on its own.  The hardened kernel, however, provided PaX
> protection for executables and this will be lost.  We did a lot of
> work to properly maintain PaX markings in our package management
> system and there was no part of Gentoo that wasn't touched by issues
> stemming from PaX support.


Good luck to them at providing a complete userland ecosystem for using
pax protection. Good luck at getting people accept and review their
often crashing asm patches at upstream projects that won't even be able
to test their benefits.

Maybe we should start a business for this ? :)
http://static.sstic.org/videos2015/SSTIC_2015-06-03_P08_CLIP.mp4
(This is for Patrice)



We'll need to decide what to do with things like USE=pic. For media
packages this is not something you usually want to enable as you can
bear the 10Mb relocations at startup to have 10% or more performance
improvement when reading your 2hours long movie.


Alexis.

Re: [gentoo-dev] Packages up for grabs

[Alexis Ballier]

On Tue, 20 Jun 2017 14:53:00 +0200
Pacho Ramos  wrote:

> This packages are now up for grabs:
> dev-ml/fort


added ml@g.o there


as a matter of fact: dev-ml/* should have ml@g.o at least as fallback
to have easier coordination & transitions for major ocaml releases

Re: [gentoo-dev] Hardening a default profile

[Alexis Ballier]

On Sat, 17 Jun 2017 14:43:24 +0300
Andrew Savchenko  wrote:

> On Thu, 15 Jun 2017 19:52:07 -0500 Matthias Maier wrote:
> > > there should be a way of turning these off systematically.  the
> > > advantage of the current hardened gcc specs is that one can switch
> > > between them using gcc-config.  if these are forced on for the
> > > default profile then there will be no easy way to systematically
> > > turn them off.  
> > 
> > No - there won't be an easy way for systematically turning off
> > SSP and PIE in 17.0 profiles [1,2].
> > 
> > The hardened toolchain with its different gcc profiles came from a
> > time where SSP and PIE were relatively new security features and a
> > certain amount of fine-grained control was needed. Further, at that
> > time we were talking about external patches against gcc. Nowadays
> > everything is upstreamed and (almost) no patches to gcc for
> > hardened profiles are applied any more.
> > 
> > Given the fact that all major linux distributions are following the
> > path of improved default hardening features (see for example [1])
> > and that we have been using ssp/pie in hardened profiles for years
> > now the purpose of fine-grained control over ssp/pie is also highly
> > questionable.
> > 
> > The consensus at the moment is that PIE and SSP (as well as stricter
> > linker flags) will soon be standard (or, actually *are* already
> > standard) compilation options. A per-package override (if
> > absoluetely needed) is fine - and, in fact, already in place
> > everywhere where needed.  
> 
> Gentoo is all about choice, remember? :)
> 
> It is really good to have them by default, it is bad to force them
> on everyone. Security is not always of paramount importance
> comparing to other factors, sometimes performance matters more,
> e.g. in isolated and restricted non-public HPC environment.
> 
> PIE, SSP may lead up to 8% of performance loss[1]. The
> stack-protector (especially stack-protector-all or -strong) may
> cause even more damage. For compute nodes this may be equivalent to
> millions USD loss (depends on the system scale of course).

This can probably be fixed by a gcc-config target disabling those as it
used to be the case on hardened

Re: [gentoo-dev] [RFC] toolchain-funcs.eclass / toolchain-glibc.eclass - gcc-6 bugfixes and updates

[Alexis Ballier]

On Fri, 16 Jun 2017 14:25:02 +0100
"M. J. Everitt"  wrote:

> On 16/06/17 09:27, Matthias Maier wrote:
> > On Wed, Jun 14, 2017, at 18:15 CDT, Matthias Maier
> >  wrote: 
> >> Hello all,
> >>
> >> this is a series of patches against the toolchian-funcs and
> >> toolchain-glibc eclasses, most notably
> >>  
> > Pushed.
> >
> > Best,
> > Matthias
> >  
> .. That was quick ...
> 
> I swore there was something in the devmanual about a nice long period
> of bikeshedding before changes to eclasses were approved ..
> 

Maintainers are not even required to send their changes to the ML
before committing. They do it because they think it makes sense to
have some review for eclasses having a wide usage. Sending trivial
changes here instead of b.g.o can be seen as spam.

Re: [gentoo-dev] [RFC] Forced/automatic USE flag constraints (codename: ENFORCED_USE)

[Alexis Ballier]

On Thu, 15 Jun 2017 18:48:42 +0100
Ciaran McCreesh <ciaran.mccre...@googlemail.com> wrote:

> On Thu, 15 Jun 2017 19:30:02 +0200
> Alexis Ballier <aball...@gentoo.org> wrote:
> > On Thu, 15 Jun 2017 18:04:35 +0100
> > Ciaran McCreesh <ciaran.mccre...@googlemail.com> wrote:  
> > > On Thu, 15 Jun 2017 18:55:45 +0200
> > > Alexis Ballier <aball...@gentoo.org> wrote:
> > > > The guarantee comes from the fact that the output is always in
> > > > the space of all possible inputs from the user. So, if some
> > > > output will kill a kitten, so does some input.  
> > > 
> > > USE=minimal
> > > USE=mips
> > > USE=-ssl
> > > 
> > 
> > So what?  
> 
> So, if the aim of this solution is to make things better for the user,
> what are you doing to establish that this will make things better for
> the user instead of recommending something awful?
> 

Considering that the way you write REQUIRED_USE defines how the solver
behaves, your problem is ill defined.

If I try to ask my crystal ball, I would say: USE=mips is either masked
or forced so never an option. Developer would not want USE=minimal to
be toggled randomly so would write a constraint so that it always
appears e.g. on the left part of an implication.

Re: [gentoo-dev] [RFC] Forced/automatic USE flag constraints (codename: ENFORCED_USE)

[Alexis Ballier]

On Thu, 15 Jun 2017 19:38:48 +0200
Michał Górny <mgo...@gentoo.org> wrote:

> On czw, 2017-06-15 at 18:07 +0200, Alexis Ballier wrote:
> > On Thu, 15 Jun 2017 17:59:13 +0200
> > Michał Górny <mgo...@gentoo.org> wrote:
> >   
> > > On śro, 2017-06-14 at 16:09 +0200, Alexis Ballier wrote:  
> > > > On Wed, 14 Jun 2017 15:57:38 +0200
> > > > Michał Górny <mgo...@gentoo.org> wrote:
> > > > [...]
> > > > > > [...]  
> > > > > > > > > > > [1]:https://wiki.gentoo.org/wiki/User:MGorny/GLEP:ReqUse  
> > > > > > > > > > > 
> > > > > > > > > > 
> > > > > > > > > > I really don't like the reordering thing. Even the
> > > > > > > > > > restricted syntax does not fix the issue with '^^
> > > > > > > > > > ( a b ) b? ( a )' already mentioned here. It'd be
> > > > > > > > > > much better and simpler for the spec just to assign
> > > > > > > > > > a fixed value and use the solving rules with
> > > > > > > > > > those.  
> > > > > > > > > 
> > > > > > > > > You're not going to convince me by providing examples
> > > > > > > > > that are utterly broken by design and
> > > > > > > > > meaningless ;-).
> > > > > > > > 
> > > > > > > > Well... if it's so obvious that the example is broken by
> > > > > > > > design that you don't even bother to explain why, I
> > > > > > > > assume you have an algorithm for that. Where is the
> > > > > > > > code ? What are the numbers ? How many ebuilds might
> > > > > > > > fail after reordering ? How can this be
> > > > > > > > improved ?
> > > > > > > 
> > > > > > > Are you arguing for the sake of arguing here? I just
> > > > > > > presumed that this example is so obviously broken there
> > > > > > > is no point wasting any more time on it. The code of
> > > > > > > nsolve clearly detects that, so I don't really understand
> > > > > > > what you're trying to prove here.  
> > > > > > 
> > > > > > Those are real questions. You should take breath, think a
> > > > > > bit about it, and try to run the 2 possible orderings of
> > > > > > the ^^ through nsolve or even solve.py. They both are very
> > > > > > happy (and are right to be) with the above ordering. You
> > > > > > might want to think a bit more about what is the relation
> > > > > > between this broken 10 chars example and the 10 lines
> > > > > > python targets one below.
> > > > > > 
> > > > > > You should also realize that all the above questions have
> > > > > > already been answered in length if you do as I
> > > > > > suggest.  
> > > > > 
> > > > > No. I have already spent too much time on this. We're already
> > > > > long past all useful use cases, and now I feel like you're
> > > > > going to argue to death just to find a perfect algorithm that
> > > > > supports every absurd construct anyone can even write, if
> > > > > only to figure out the construct is completely useless.
> > > > 
> > > > I'm not going to argue to death. It's already proven reordering
> > > > is broken.
> > > > 
> > > > > If you want to play with it more, then please by all means do
> > > > > so.
> > > > 
> > > > There is nothing to do for reordering. It's broken by design.
> > > > 
> > > > > However, do not expect me to waste any more of my time on it.
> > > > > I've done my part, the code works for all reasonable use
> > > > > cases and solves all the problems I needed solving. If you
> > > > > want more, then it's your job to do it and solve the
> > > > > resulting issues.
> > > > 
> > > > Like... writing code handling all the cases and describing how
> > > > it works ? We're past that. The only thing we're not past is
> > > > that you fail to understand it and attempt to block it.

Re: [gentoo-dev] [RFC] Forced/automatic USE flag constraints (codename: ENFORCED_USE)

[Alexis Ballier]

On Thu, 15 Jun 2017 18:04:35 +0100
Ciaran McCreesh <ciaran.mccre...@googlemail.com> wrote:

> On Thu, 15 Jun 2017 18:55:45 +0200
> Alexis Ballier <aball...@gentoo.org> wrote:
> > The guarantee comes from the fact that the output is always in the
> > space of all possible inputs from the user. So, if some output will
> > kill a kitten, so does some input.  
> 
> USE=minimal
> USE=mips
> USE=-ssl
> 

So what?

Re: [gentoo-dev] [RFC] Forced/automatic USE flag constraints (codename: ENFORCED_USE)

[Alexis Ballier]

On Thu, 15 Jun 2017 17:45:09 +0100
Ciaran McCreesh <ciaran.mccre...@googlemail.com> wrote:

> On Thu, 15 Jun 2017 18:37:16 +0200
> Alexis Ballier <aball...@gentoo.org> wrote:
> > > So you're saying that at the end of this, there's an ENFORCED_USE
> > > solver that spits out some answer that may or may not be in any
> > > way a sane solution to the conflict.
> > > 
> > > I don't see how that's helpful to a user.  
> > 
> > Define sane.
> > The definition of the solver is made to change the least possible of
> > the inputs and is completely and easily predictable by the person
> > writing the constraint. That is something I would call sane.  
> 
> The problem is not just writing a resolver that spits out a valid
> output. The problem is writing a resolver which will never go and
> uninstall bash as a result of unintended combinations of inputs (which
> Portage used to do, but there's now a special exception for system
> packages, so it will only occasionally unexpectedly uninstall critical
> packages that aren't explicitly in system due to virtuals instead).
> This is *hard*.

We're not talking about solving deps here.

> A bad suggestion to the user is worse than no suggestion at all.
> Unless you can safely determine that there aren't any unintended
> consequences of your rule, the focus needs to be on producing good
> error messages so the user can figure the problem out, not on
> producing bad solutions that will confuse things even more.

The guarantee comes from the fact that the output is always in the
space of all possible inputs from the user. So, if some output will
kill a kitten, so does some input.

Of course, implementation can decide to error out and propose a
solution, to continue but print big fat warnings, etc. I like the
initial proposal in that regard.


Alexis.

Re: [gentoo-dev] [RFC] Forced/automatic USE flag constraints (codename: ENFORCED_USE)

[Alexis Ballier]

On Thu, 15 Jun 2017 17:32:40 +0100
Ciaran McCreesh <ciaran.mccre...@googlemail.com> wrote:

> On Thu, 15 Jun 2017 18:30:10 +0200
> Alexis Ballier <aball...@gentoo.org> wrote:
> > On Thu, 15 Jun 2017 17:22:26 +0100
> > Ciaran McCreesh <ciaran.mccre...@googlemail.com> wrote:  
> > > On Thu, 15 Jun 2017 18:19:04 +0200
> > > Alexis Ballier <aball...@gentoo.org> wrote:
> > > > On Thu, 15 Jun 2017 17:13:57 +0100
> > > > Ciaran McCreesh <ciaran.mccre...@googlemail.com> wrote:  
> > > > > On Thu, 15 Jun 2017 18:07:00 +0200
> > > > > Alexis Ballier <aball...@gentoo.org> wrote:
> > > > > > > The best way to convince me is through valid
> > > > > > > examples.
> > > > > > 
> > > > > > It is also easier to be convinced when you try to understand
> > > > > > and ask for clarifications instead of just rejecting without
> > > > > > thinking :)  
> > > > > 
> > > > > The problem with this entire proposal is that it's still in
> > > > > "well I can't think of how it could possibly go wrong"
> > > > > territory. We need a formal proof that it's sound. History has
> > > > > shown that if something can be abused by Gentoo developers, it
> > > > > will be abused...
> > > > 
> > > > Had you read the thread you would have noticed that I provided
> > > > an algorithm giving sufficient conditions for the solver to
> > > > work. That is, if developers pay attention to repoman
> > > > warnings/errors, it will never fail. Obviously, since we're
> > > > still in the SAT space, you can ignore the errors and make it
> > > > fail, but it'll never be worse than what we currently
> > > > have.  
> > > 
> > > You have shown that you produce a solution, not the solution
> > > that's actually wanted.  
> > 
> > Since 'wanted' is still undefined, I'd say it produces the defined
> > solution and you can adapt to the definition to get what you want.  
> 
> So you're saying that at the end of this, there's an ENFORCED_USE
> solver that spits out some answer that may or may not be in any way a
> sane solution to the conflict.
> 
> I don't see how that's helpful to a user.
> 

Define sane.
The definition of the solver is made to change the least possible of
the inputs and is completely and easily predictable by the person
writing the constraint. That is something I would call sane.