Re: [gentoo-dev] [RFC] Forced/automatic USE flag constraints (codename: ENFORCED_USE)
On Sat, 03 Jun 2017 17:33:09 +0200
Michał Górny wrote:
> On sob, 2017-06-03 at 13:00 +0200, Alexis Ballier wrote:
> > This whole thing definitely needs more thought and feedback but for
> > now those extra restrictions seem quite natural to me, allow easy
> > solving on the PM side and allow to have useful feedback from
> > repoman.
>
> Well, I'll try to figure out the magic you were telling me later but
> as a quick note, my specific use case for this are Python targets, so
> I'm going throw a few basic concepts that need to work for you to
> play with ;-).
>
> In the following samples pt1,2,.. stands for PYTHON_TARGETS;
> pst1,2,... for PYTHON_SINGLE_TARGET. Eventually I'd like to kill the
> latter but that depends on how well the autosolving works.
>
> 1. ^^ ( pst1 pst2 pst3.. ) pst1? ( pt1 ) pst2? ( pt2 ) pst3? ( pt3 )..
the pt{1,2,...} part does not matter here: they all fail point 4 when
compared as 2nd clause to the others (each ptX appears only once in the
whole expression), so this boils down to solving n-ary ^^, and to how
those are translated: you can have several translations but some would
not work properly. Let's try:
(a) pst1? ( !pst2 !pst3 )
(b) pst2? ( !pst3 )
(c) !pst3? ( !pst2? ( pst1 ) )
Between (a) and (c), points 1 to 3 hold (those points are reflexive).
For point 4, "a q'_j is some p_i" will hold in both ways, so we do
actually have a cycle between them. Bad luck.
Let's write (c) as:
(c) !pst3? ( !pst2? ( !pst1? ( pst1 ) ) )
Now, (c) can't break (a) because of point 1: pst1 in (a) vs !pst1 in
(c).
(b) can't break (a) because of point 2: q_i=!pst2 is the negation of
p_j=pst2.
(c) can't break (b) because pst2 vs !pst2 (point 1).
So, this formulation works:
(a) pst1? ( !pst2 !pst3 )
(b) pst2? ( !pst3 )
(c) !pst3? ( !pst2? ( !pst1? ( pst1 ) ) )
USE="pst1 whatever" will enable only pst1. USE="-pst1 pst2 whatever"
will enable only pst2. USE="-pst1 -pst2 pst3" will leave it alone.
USE="-pst{1,2,3}" will enable pst1.
Note that "pst1? ( pt1 ) pst2? ( pt2 ) pst3? ( pt3 ) ^^ ( pst1 pst2
pst3.. )" is a different story: (c) expanded as above can break 'pst1?
( pt1 )' (point 4: a q'_j is some p_i) and you can actually check that
with USE="-pt* -pst*"; you'll need 2 passes to get the proper solution.
Fortunately, this is still a DAG and repoman would be able to propose an
ordering requiring only one pass.
[...]
> 2. ^^ ( pst1 pst2.. ) pst1? ( pt1 ) pst2? ( pt2 ).. ^^ ( pt1 pt2 )
>
> This is a possible extension of the above for the migration period.
> The idea is that exactly one PST must be selected, and only the
> matching PT must be selected (others are implicitly disabled).
If we expand '^^ ( pt1 pt2 )' as above we get:
(d) pt1? ( !pt2 )
(e) !pt2? ( !pt1? ( pt1 ) )
Here, (d) is annoying: it can break and be broken by 'pst2? ( pt2 )'.
There would be a cycle and this would be rejected/notified.
If you think about it, this would mean I have set USE="-* pt1 pst2";
pst2 forcing to enable pt2 but '^^ ( pt1 pt2 )' with pt1 enabled would
prefer pt1 and disable pt2 again... This hints the solution: You need
to define who wins between pt and pst.
Instead you could write it:
^^ ( pst1 pst2.. ) pst1? ( pt1 !pt2 ... ) pst2? ( !pt1 pt2 ... )..
But then 'pst2? ( !pt1 pt2 ... )' can break each other with 'pst1?
( pt1 !pt2 ... )' in the sense I defined because of point 4 (A q_i is
the negation of a q_'j); you'll get the repoman notification about a
cycle. This is a case of a perfectly valid constraint that is rejected
by the restriction.
It is valid because we know we are guaranteed exactly one pstX will
be enabled. We can hint the solver with that by writing:
^^ ( pst1 pst2.. )
pst1? ( pt1 !pt2 ... )
pst2? ( !pt1? ( pt2 !pt3 ... ) )
pst3? ( !pt1? ( !pt2? ( pt3 !pt4 ... ) ) )
Now we're good: For j>i, solving a pst{j} line does not break a pst{i}
one because of point 2: A q_i (pt{i}) is the negation of a p'_j
(!pt{i}).
It's getting a bit ugly but it's probably bearable with good reporting
from static checkers (repoman).
> 3. doc? ( || ( pt3 pt4 ) ) || ( pt1 pt2 pt3 pt4 )
>
> This is distutils-r1 with USE=doc requiring python2. Note that it's
> an example where the second || is added via eclass [NB: we've checked
> and PMS says eclass values are appended to ebuild value].
Much simpler here:
(a) doc? ( !pt4? ( pt3 ) )
(b) !pt4? ( !pt3? ( !pt2? ( pt1 ) )
(b) can't break (a) because of point 4: Neither 'a q_i is
the negation of a q_'j' nor 'a q'_j is some p_i' hold. We're good.
USE="-* doc" will enable pt3 only. USE="-* pt{whatever} doc" will
enable pt3 (if not enabled) unless pt{whatever} contains pt4.
Alexis.
Re: [gentoo-dev] [RFC] Forced/automatic USE flag constraints (codename: ENFORCED_USE)
On sob, 2017-06-03 at 13:00 +0200, Alexis Ballier wrote: > This whole thing definitely needs more thought and feedback but for now > those extra restrictions seem quite natural to me, allow easy solving > on the PM side and allow to have useful feedback from repoman. > Well, I'll try to figure out the magic you were telling me later but as a quick note, my specific use case for this are Python targets, so I'm going throw a few basic concepts that need to work for you to play with ;-). In the following samples pt1,2,.. stands for PYTHON_TARGETS; pst1,2,... for PYTHON_SINGLE_TARGET. Eventually I'd like to kill the latter but that depends on how well the autosolving works. 1. ^^ ( pst1 pst2 pst3.. ) pst1? ( pt1 ) pst2? ( pt2 ) pst3? ( pt3 ).. This is the standard constraint for PYTHON_SINGLE_TARGET -- it needs to ensure that exactly one PYTHON_SINGLE_TARGET is selected, and that the matching PYTHON_TARGETS value is (other PYTHON_TARGETS can be enabled or disabled -- doesn't matter). 2. ^^ ( pst1 pst2.. ) pst1? ( pt1 ) pst2? ( pt2 ).. ^^ ( pt1 pt2 ) This is a possible extension of the above for the migration period. The idea is that exactly one PST must be selected, and only the matching PT must be selected (others are implicitly disabled). 3. doc? ( || ( pt3 pt4 ) ) || ( pt1 pt2 pt3 pt4 ) This is distutils-r1 with USE=doc requiring python2. Note that it's an example where the second || is added via eclass [NB: we've checked and PMS says eclass values are appended to ebuild value]. -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] Lastrites: x11-misc/rednotebook
On Sat, Jun 3, 2017 at 9:45 AM, Michael Orlitzky wrote: > On 06/02/2017 04:38 AM, Pacho Ramos wrote: >> # Pacho Ramos (02 Jun 2017) >> # Relies on obsolete and vulnerable webkit-gtk version and bundles some libs >> # (#597532). Removal in a month. >> x11-misc/rednotebook >> > > The bundled libs wouldn't be too hard to eliminate, but we didn't have > packages for them when I checked. A user requested the last version > bump, so somebody's using it, and it isn't very outdated. I filed the > bundled-libs bug just so we wouldn't lose track of it. > > The latest version (2.0) supports python-3 and gtk-3: > > == 2.0 (2017-05-19) == > * Port to Python 3 and GTK 3. > * Add index of tags to LaTeX export (#324, thanks Alex Schickedanz). > * Use new CDN link for MathJax. > > So maybe it makes more sense to bump it. It has been without a maintainer since Aug 2016. If you want to maintain it, please do.
Re: [gentoo-dev] Lastrites: x11-misc/rednotebook
On 06/02/2017 04:38 AM, Pacho Ramos wrote: > # Pacho Ramos (02 Jun 2017) > # Relies on obsolete and vulnerable webkit-gtk version and bundles some libs > # (#597532). Removal in a month. > x11-misc/rednotebook > The bundled libs wouldn't be too hard to eliminate, but we didn't have packages for them when I checked. A user requested the last version bump, so somebody's using it, and it isn't very outdated. I filed the bundled-libs bug just so we wouldn't lose track of it. The latest version (2.0) supports python-3 and gtk-3: == 2.0 (2017-05-19) == * Port to Python 3 and GTK 3. * Add index of tags to LaTeX export (#324, thanks Alex Schickedanz). * Use new CDN link for MathJax. So maybe it makes more sense to bump it.
Re: [gentoo-dev] [RFC] Forced/automatic USE flag constraints (codename: ENFORCED_USE)
On Fri, 02 Jun 2017 15:55:17 +0200
Michał Górny wrote:
> On pią, 2017-06-02 at 13:27 +0200, Alexis Ballier wrote:
> > On Thu, 01 Jun 2017 23:31:25 +0200
> > Michał Górny wrote:
> > [...]
> > > > There are probably dozens of ways to make that non
> > > > deterministic. Here's one: USE='-*'. Apply '|| ( cli cgi fpm
> > > > apache2 embed phpdbg )' last; this enables 'cli'. Since it's
> > > > the last one, REQUIRED_USE is not satisfied because of 'cli?
> > > > ( ^^ ( readline libedit ) )'. If you process it first, then you
> > > > enable cli and readline and are good.
> > >
> > > You just take a second iteration, and things fall into place.
> > > That's not a problem.
> > >
> > > > > > Also, what happens if we applied all the constraints and
> > > > > > obtained some useflags setting that still fails REQUIRED_USE
> > > > > > check ?
> > > > >
> > > > > It can't happen. If you can apply all the constraints, then
> > > > > implicitly REQUIRED_USE is satisfied. If you can't apply all
> > > > > the constraints, then it just fails. Of course, we want to
> > > > > ultimately avoid that case.
> > > >
> > > > See the php example above. How do you ensure this does not
> > > > happen?
> > > >
> > > > Note that your assertion 'If you can apply all the constraints,
> > > > then implicitly REQUIRED_USE is satisfied.' is false: You're
> > > > changing the values of useflags, thus might violate some
> > > > previously checked constraint.
> > >
> > > So you reiterate. Either you reach a solution (preferably there's
> > > only a single valid solution you can reach) or you hit a previous
> > > state which indicates a loop and you fail.
> >
> >
> > So, PM, for every ebuild, will need to store the (at most) 2^n
> > states it has already seen while trying to solve REQUIRED_USE and
> > iterate until it cannot proceed anymore or falls into a previous
> > state (so that's 2^n time too). That way we go from a linear time &
> > linear space algorithm to one in exponential time & space. That's
> > probably not a good idea.
>
> I don't think that's actually going to happen. You'd have to try
> really hard to get over n-1 iterations. I mean, the only simple way I
> can think of is:
>
> b? ( a ) c? ( b ) d? ( c ) e? ( d ) ...
>
> and this only means n-1 iterations. Can you think of a better way to
> force multiple iterations that can be written without making
> REQUIRED_USE completely unreadable? How likely is that going to happen
> accidentally?
I don't see any reason why it should terminate after n iterations; I
don't see any example of how to do more either. I can try to think more
about it, but maybe it is not even needed, see below.
> > I think it's better to limit the number of iterations to some
> > constant. I'd say 1, then fail if REQUIRED_USE is still not
> > satisfied. Maybe 2 or 3 can be useful but I think it'd be much
> > harder to provide automated checks for that and much harder for
> > ebuild writers to understand what will happen with the REQUIRED_USE
> > they're about to write.
>
> Well, I don't mind setting that. All my tests (that weren't
> deliberately abusing the algorithm) were able to finish in a single
> iteration. 2 or 3 should probably be safe for all the reasonable
> uses. However, if we're not going to verify all possible values on
> repoman side, I think it would be better to have a larger limit for
> users, to not have things explode on them unnecessarily.
Look, if we assume left to right solving, only one iteration possible
and all the constraints to be of the form 'p_1? p_2? ... p_n? ( q_1 ...
q_m )' where p_i and q_i are either 'useflag' or '!useflag', then we
only have to check when such a clause can change a previously satisfied
clause to unsatisfied.
For two clauses:
"p_1? p_2? ... p_n? ( q_1 ... q_m )" and "p'_1? p'_2? ... p'_{n'}?
( q'_1 ... q'_{m'} )", assuming the first is satisfied, when can solving
the 2nd break the 1st?
It must be that:
1.The conditions are compatible: No p_i is the negation of a p'_j.
2.Solving the 1st does not make the 2nd trivially true: No q_i is
the negation of a p'_j.
3.Solving the 2nd does not make the 1st trivially true afterwards: No
p_i is the negation of a q'_j.
4.Solving the 2nd does break the 1st assumption: (A q_i is
the negation of a q_'j) or (a q'_j is some p_i and one of q_1 ... q_m
might be false).
The only a priori non polynomial part here is "one of q_1 ... q_m
might be false". If we do not check that (only check that some q'_j is
some p_i), we are still guaranteed that the 2nd clause cannot break the
1st but we might end up rejecting otherwise valid constructs.
Doing that, we have a check guaranteeing that the above algorithm will
always provide a valid answer for 'clause_1 clause_2' in
O(|clause_1|*|clause_2|) time.
For a complete REQUIRED_USE='clause_1 ... clause_k', it is sufficient
to check that, for any i>j, clause_i cannot break clause_j in the above
sense. This gives a polynomial algorithm fo
Re: [gentoo-dev] [RFC] Addition of a new field to metadata.xml
On Sat, 03 Jun 2017 09:58:28 +0200 Michał Górny wrote: > and that's a small one. I guess we could avoid this if you restricted > those remotes to the source package used to build them all. I think in the event they're a form of conventional foo foo-dev foo-dbg etc, under the knowledge that those things can't possibly map to other gentoo packages, we should codify only the first of those, and then have it placed on the iteroperating code to make logical inferences from that. "foo-dev" in a search query would map to "foo" if no "foo-dev" existed. But yeah, lots of complexity there. That's why I'd just say those facets shouldn't really be mapped explicitly. The general pattern being: "If a debian id can be conjugated from another debian id by guessing with a generic algorithm, only index the former" pgpJMn8ydNgDP.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] [RFC] Addition of a new field to metadata.xml
On sob, 2017-06-03 at 03:22 +1200, Kent Fredric wrote: > On Fri, 02 Jun 2017 16:51:25 +0200 > Michał Górny wrote: > > > ...so if a Gentoo package is split into 40 packages in Debian, are you > > going to list all of them? > > If it would be useful to do so, maybe. > > But its a text file, people are capable of making judgements about > adding 3.2k of text, I hope. ( worst-case, 40 lines of 80 characters each ) > > If the text at that size has a use, then, why not? > > As long as that 40-package example is an exceptional case, where it was deemed > useful, not the norm. It's not that exceptional in binary distributions where it's normal to split a single source into a few dozen packages (libraries, headers, tools, plugins...): https://packages.debian.org/source/sid/systemd and that's a small one. I guess we could avoid this if you restricted those remotes to the source package used to build them all. -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] [RFC] Addition of a new field to metadata.xml
On 06/01/2017 11:59 PM, Kent Fredric wrote: > On Thu, 1 Jun 2017 18:36:24 -0700 > Daniel Campbell wrote: > >> +1. Otherwise sounds good. But if we do this for Debian, will there be >> movement to add in package names for rpm-based distros? Arch? BSD? >> Slackware? Where do we draw the line? > > I'd say "as need be". Here we have a few extra benefits from using a > debian identifier that aren't strictly related to "package mapping". > > Its also easier for third party services to use our use of debian > identifiers to produce the other mappings for us where known ( that is, > non-gentoo entities can maintain a mapping of debian-to-foo, and we can > trivially hook into that by using the debian-id as the identifer ) > >> Will developers be expected to treat this like a mandated element? > > I'd imagine not, given not everything in debian exists in gentoo, or > vice versa. > > Similarly, I don't think there are any mandates that the other values > of remote-id be populated, only that its *encouraged* because that data > provides utility to an end user. > >> If >> not, which team will have authority to touch package metadata to make >> this change? > > I'd suggest it should stay within the controls of the package > maintainer for starters, where individual maintainers can provision it > as they feel fit, and we can review our stance on this later if we want > to make it a tree wide consistent thing. > > Partly because individual maintainers are more likely to understand > correctly how equivalent their dists are to the referenced debian one, > and be more equipped to decide whether to include/exclude a given ref. > That sounds very reasonable to me. Thanks for clarifying. -- Daniel Campbell - Gentoo Developer OpenPGP Key: 0x1EA055D6 @ hkp://keys.gnupg.net fpr: AE03 9064 AE00 053C 270C 1DE4 6F7A 9091 1EA0 55D6 signature.asc Description: OpenPGP digital signature
