Re: [gentoo-dev] Stable masks on multilib packages
On Wed, Apr 2, 2014 at 2:25 AM, Ben de Groot wrote: > I'm strongly considering reverting these changes in the packages I > maintain. I'm tired of having to deal time and again with multilib > breakage. > > Either that, or someone else can take over primary maintainership. I'd be happy for you to remove yourself as a maintainer of cairo. You've got one commit (not removing ebuilds) since July 2012, according to the changelog. At least three X11 team members have more commits than that, and I don't know how much sense it makes to have a non-X11-team member being the primary maintainer of a critical X11 package anyway.
Re: [gentoo-dev] Stable masks on multilib packages
On Tue, 1 Apr 2014 20:58:30 -0400 Rich Freeman wrote: > On Tue, Apr 1, 2014 at 8:13 PM, Patrick Lauer > wrote: > > Now let's just continue to ignore the existing multilib-portage > > work so we can claim it's irrelevant, while shifting the conditions > > for accepting it whenever it is convenient, while silently adding > > the competing method in-tree so it's all decided now anyway ... > > Conditions for accepting it? What are those, and how have they > changed? The conditions have always been "someone has to be able to explain exactly what the changes are"... -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Stable masks on multilib packages
On Wed, 2014-04-02 at 17:25 +0800, Ben de Groot wrote: > I'm strongly considering reverting these changes in the packages I > maintain. I'm tired of having to deal time and again with multilib > breakage. > > Either that, or someone else can take over primary maintainership. Ben, if you are uncomfortable with multilib changes in cairo and don't want to see it unmasked ~arch, I am volunteering to co-maintain it. Assign any multilib issues with it to me. It's worth it to me because I am a wine maintainer, and wine will benefit enormously once all of its dependencies have finally been multilibbed and we can unmask the flags in stable. signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] Stable masks on multilib packages
On Wed, 2014-04-02 at 17:14 +0800, Ben de Groot wrote: > On 1 April 2014 21:58, Alexandre Rostovtsev wrote: > > On Tue, 2014-04-01 at 13:13 +0800, Ben de Groot wrote: > >> On 1 April 2014 06:16, Michał Górny wrote: > >> > Hello, all. > >> > > >> > The late multilib ppc issues made me re-check our stable masks on > >> > abi_x86_* flags and, honestly, I'm not sure if we're doing things > >> > the right way. > >> > > >> > That said, I have an alternate idea inspired by the ppc breakage. > >> > > >> > Your thoughts? > >> > >> In my opinion your multilib approach introduces an unnecessary degree > >> of complexity, which --as has been shown here again-- is prone to > >> breakage. > >> > >> It would be best for our beloved distro to revert all the multilib > >> changes, and try a different approach, or leave this prone-to-breakage > >> implementation to an overlay for the few people who would actually > >> benefit from it. > > > > Speaking as a wine maintainer, the emul-linux-x86-* approach has many > > times been proven to be an embarrassing failure and the main source of > > pain and frustration for wine users. The sooner emul-linux-x86-* can be > > removed from the tree, the better for Gentoo. > > I would like to see an honest cost-benefit analysis of the > emul-linux-x86 approach compared to the multilib eclass approach. > Because in my experience the latter introduces more breakage and > higher maintenance costs. libfoo-1.5 becomes available ~arch. Users install it. Users realize that libfoo-1.5's headers are incompatible with libfoo-1.4 binaries from emul-linux-x86-bar after they try to build wine and get a failure 30 minutes into the compile. So users complain about gentoo's wine support. libfoo-1.5 is stabilized. Of course, emul-linux-x86-bar has not been updated immediately, because *all* parts of emul-linux-x86 need to be updated at once due to a complex network of library interdependencies, and then manually checked for sanity, and the folks taking care of emul-linux-x86 can only really afford to do it 2, max 3 times per year. So more users install libfoo-1.5, and see wine fail, and complain more about gentoo's wine support until a new emul-linux-x86 release is rolled out a couple months later. But meanwhile, libbaz-5.0 becomes available in ~arch, and its headers are backwards-incompatible with libbaz-4.2 binaries in emul-linux-x86-wombatlibs... And this happens, oh, only every time there are API changes in libpng, jpeg, llvm, gnutls, libxml2, pulseaudio, etc. signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] Stable masks on multilib packages
On Wed 02 Apr 2014 17:14:02 Ben de Groot wrote: > On 1 April 2014 21:58, Alexandre Rostovtsev wrote: > > On Tue, 2014-04-01 at 13:13 +0800, Ben de Groot wrote: > >> On 1 April 2014 06:16, Michał Górny wrote: > >> > Hello, all. > >> > > >> > The late multilib ppc issues made me re-check our stable masks on > >> > abi_x86_* flags and, honestly, I'm not sure if we're doing things > >> > the right way. > >> > > >> > That said, I have an alternate idea inspired by the ppc breakage. > >> > > >> > Your thoughts? > >> > >> In my opinion your multilib approach introduces an unnecessary degree > >> of complexity, which --as has been shown here again-- is prone to > >> breakage. > >> > >> It would be best for our beloved distro to revert all the multilib > >> changes, and try a different approach, or leave this prone-to-breakage > >> implementation to an overlay for the few people who would actually > >> benefit from it. > > > > Speaking as a wine maintainer, the emul-linux-x86-* approach has many > > times been proven to be an embarrassing failure and the main source of > > pain and frustration for wine users. The sooner emul-linux-x86-* can be > > removed from the tree, the better for Gentoo. > > I would like to see an honest cost-benefit analysis of the > emul-linux-x86 approach compared to the multilib eclass approach. > Because in my experience the latter introduces more breakage and > higher maintenance costs. the emul-linux-* approach is a terrible idea. it has obviously proven to: - not scale (sorry, but x86 is not the only ABI out there people care about) - be a huge pita to update/maintain - require shipping a lot of precompiled code the multilib eclasses aren't perfect, but they're a hell of a lot better than the emul-linux-* approach. we've been carrying that crap for over 10 years and it's gone literally nowhere. considering it as a viable alternative to the multilib eclasses is preposterous at best. -mike signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] Stable masks on multilib packages
On 2 April 2014 07:38, Patrick Lauer wrote: > On 04/01/2014 01:13 PM, Ben de Groot wrote: >> On 1 April 2014 06:16, Michał Górny wrote: >>> Hello, all. >>> >>> The late multilib ppc issues made me re-check our stable masks on >>> abi_x86_* flags and, honestly, I'm not sure if we're doing things >>> the right way. >>> >>> That said, I have an alternate idea inspired by the ppc breakage. >>> >>> Your thoughts? >> >> In my opinion your multilib approach introduces an unnecessary degree >> of complexity, which --as has been shown here again-- is prone to >> breakage. > > And it removes any chance of writing ebuilds - I seriously have no idea > how to fix those things now. They are multibuilds, not ebuilds. This is part of the complexity I mentioned. I significantly raises maintenance costs, and I'm not convinced of the benefit. >> >> It would be best for our beloved distro to revert all the multilib >> changes, and try a different approach, or leave this prone-to-breakage >> implementation to an overlay for the few people who would actually >> benefit from it. >> > As a temporary stage they are kinda okish, maybe ... but ... > > the whole transition strategy has been very very silly and should have > been staged in an overlay. I'd even build-test them and file bugs - just > don't do this salami tactic of one breakage a day. > I'm strongly considering reverting these changes in the packages I maintain. I'm tired of having to deal time and again with multilib breakage. Either that, or someone else can take over primary maintainership. -- Cheers, Ben | yngwin Gentoo developer
Re: [gentoo-dev] Stable masks on multilib packages
On 1 April 2014 21:58, Alexandre Rostovtsev wrote: > On Tue, 2014-04-01 at 13:13 +0800, Ben de Groot wrote: >> On 1 April 2014 06:16, Michał Górny wrote: >> > Hello, all. >> > >> > The late multilib ppc issues made me re-check our stable masks on >> > abi_x86_* flags and, honestly, I'm not sure if we're doing things >> > the right way. >> > >> > That said, I have an alternate idea inspired by the ppc breakage. >> > >> > Your thoughts? >> >> In my opinion your multilib approach introduces an unnecessary degree >> of complexity, which --as has been shown here again-- is prone to >> breakage. >> >> It would be best for our beloved distro to revert all the multilib >> changes, and try a different approach, or leave this prone-to-breakage >> implementation to an overlay for the few people who would actually >> benefit from it. > > Speaking as a wine maintainer, the emul-linux-x86-* approach has many > times been proven to be an embarrassing failure and the main source of > pain and frustration for wine users. The sooner emul-linux-x86-* can be > removed from the tree, the better for Gentoo. I would like to see an honest cost-benefit analysis of the emul-linux-x86 approach compared to the multilib eclass approach. Because in my experience the latter introduces more breakage and higher maintenance costs. > I am aware of only two solutions to the emul-linux-x86-* problems : > multilib-portage and multilib-build.eclass. The first requires everybody > to switch to a new package manager. The second allows us to keep using > portage, but requires library maintainers to add some simple boilerplate > to their ebuilds for multilib support. > > Do you have yet another alternative in mind? In my mind the emul-linux-x86 approach is more acceptable. I don't have experience with multilib-portage, as I don't have a use case for it. Another option, which seems to me to be more reasonable and which has greatly lower maintenance costs, is using a chroot. -- Cheers, Ben | yngwin Gentoo developer
Re: [gentoo-dev] Stable masks on multilib packages
On Tue, Apr 1, 2014 at 8:13 PM, Patrick Lauer wrote: > Now let's just continue to ignore the existing multilib-portage work so > we can claim it's irrelevant, while shifting the conditions for > accepting it whenever it is convenient, while silently adding the > competing method in-tree so it's all decided now anyway ... Conditions for accepting it? What are those, and how have they changed? Rich
Re: [gentoo-dev] Stable masks on multilib packages
On 04/01/2014 10:43 PM, Rich Freeman wrote: > On Tue, Apr 1, 2014 at 9:58 AM, Alexandre Rostovtsev > wrote: >> On Tue, 2014-04-01 at 13:13 +0800, Ben de Groot wrote: >>> >>> In my opinion your multilib approach introduces an unnecessary degree >>> of complexity, which --as has been shown here again-- is prone to >>> breakage. >>> >>> It would be best for our beloved distro to revert all the multilib >>> changes, and try a different approach, or leave this prone-to-breakage >>> implementation to an overlay for the few people who would actually >>> benefit from it. >> >> I am aware of only two solutions to the emul-linux-x86-* problems : >> multilib-portage and multilib-build.eclass. The first requires everybody >> to switch to a new package manager. The second allows us to keep using >> portage, but requires library maintainers to add some simple boilerplate >> to their ebuilds for multilib support. >> >> Do you have yet another alternative in mind? > > ++ > > I'm all for better solutions. I'm not in favor of abandoning > solutions that work moderately well in favor of talking about maybe > coming up with something better sometime down the road. > > The multilib eclass isn't a perfect solution. It will have issues in > concept, design, and implementation. These will come up from time to > time. I don't think it is productive that anytime any of these pop up > that we end up having a discussion about just reverting it entirely. So why do we have this discussion? > By all means work on a competing solution. Get it working and > supported by portage. When the day comes that we want to endorse one > solution or the other as the preferred solution we can have that > discussion. That is hilariously funny. Wow. Now let's just continue to ignore the existing multilib-portage work so we can claim it's irrelevant, while shifting the conditions for accepting it whenever it is convenient, while silently adding the competing method in-tree so it's all decided now anyway ...
Re: [gentoo-dev] Stable masks on multilib packages
On 04/01/2014 01:13 PM, Ben de Groot wrote: > On 1 April 2014 06:16, Michał Górny wrote: >> Hello, all. >> >> The late multilib ppc issues made me re-check our stable masks on >> abi_x86_* flags and, honestly, I'm not sure if we're doing things >> the right way. >> >> That said, I have an alternate idea inspired by the ppc breakage. >> >> Your thoughts? > > In my opinion your multilib approach introduces an unnecessary degree > of complexity, which --as has been shown here again-- is prone to > breakage. And it removes any chance of writing ebuilds - I seriously have no idea how to fix those things now. They are multibuilds, not ebuilds. > > It would be best for our beloved distro to revert all the multilib > changes, and try a different approach, or leave this prone-to-breakage > implementation to an overlay for the few people who would actually > benefit from it. > As a temporary stage they are kinda okish, maybe ... but ... the whole transition strategy has been very very silly and should have been staged in an overlay. I'd even build-test them and file bugs - just don't do this salami tactic of one breakage a day.
Re: [gentoo-dev] Stable masks on multilib packages
Dnia 2014-04-01, o godz. 00:16:17 Michał Górny napisał(a): > Your thoughts? Following the feedback on the mailing list, IRC and bugzilla, and some of my own thoughts, I'd like to put a few important extra points I've missed in the original mail: 1. playing with the masks will cause a large unnecessary --changed-use rebuild for stable users, including the users of no-multilib profiles, 2. prebuilt software ebuilds are allowed to unconditionally use abi_* flags in USE dependencies. Masking the native flags would break those dependencies. For those unaware, 2. here means that a pre-built x86 software can use a single dependency like: dev-libs/libfoo[abi_x86_32] that would satisfy both x86 (since abi_x86_32 is use.forced) and amd64 (by pulling in multilib). Considering the two points above, I'd like to withdraw the idea of adding global masks for abi_* and just leave it amd64-specific. Additionally, the following important points were raised: 1. ulm believes that the use.force implicit IUSE magic is a bug in portage and oughtn't be trusted upon, 2. keeping extra package.use.stable.masks for different arches seems pretty much unmaintainable. It would mean a lot of duplicate package lists in different profiles that need to be kept in sync. Instead, I'd like to limit the scope of stable masks necessity and just focus on stabilizing packages properly. That is, the new position is: 1. we keep the flags for packages that are in emul-linux stable-masked on amd64 until we decide to replace emul-linux, 2. we stabilize package dependencies properly so that there are no dependency tree breakage when new ABIs are added. Rationale: the multilib conversions are no longer moving in the fast pace they used to, and the stabilization interest conflicts are less likely than they were at the time of introducing package.use.stable.masks. Especially now that new multilib conversions are made package.masked, we can give them enough time for people to be ready to stabilize in some sane way. We'd like to still keep the emul-linux-replacing packages masked in order to keep emul-linux as single multilib provider for stable packages. This is in order to avoid exposing stable users to issues involved with partial conversions, and to confusing --autounmask behavior that can possibly be created through mixing emul-linux and multilib packages. If nobody minds, I'm doing a repoman run now and in a few hours I should have a list of packages that need to be stabilized in order to clean up the dependency trees for multilib packages. -- Best regards, Michał Górny signature.asc Description: PGP signature
Re: [gentoo-dev] Stable masks on multilib packages
Dnia 2014-04-01, o godz. 11:12:11
Ian Stakenvicius napisał(a):
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> On 31/03/14 06:16 PM, Michał Górny wrote:
> > Hello, all.
> >
> > The late multilib ppc issues made me re-check our stable masks on
> > abi_x86_* flags and, honestly, I'm not sure if we're doing things
> > the right way. First, a quick summary.
> >
> >
> > Let's consider dev-libs/elfutils that has a dependency on
> > app-arch/bzip2. If we're building 32-bit elfutils, it needs 32-bit
> > libbz2. This is enforced through a dep in form of:
> >
> > app-arch/bzip2[${MULTILIB_USEDEP}]
> >
> > that gets expanded into:
> >
> > app-arch/bzip2[abi_x86_32(-)?,abi_x86_64(-)?,...,abi_mips_o32(-)?,...]
> >
> > which means that any of the ABI_* flags that gets enabled on
> > elfutils, needs to be enabled on bzip2 as well. Of course, some of
> > use.forcing and masking gets applied here but that doesn't really
> > matter.
> >
> >
> > Now, since we're pretty much converting a lot of different
> > packages, some of them are eligible for stabilization earlier than
> > the others. However, the extra MULTILIB_USEDEPs enforce stabilizing
> > multilib dependencies before the actual packages which made a few
> > developers unhappy.
> >
> > That's why we decided to stable-mask the flags on affected
> > packages. Since the flags are masked on stable, people can
> > stabilize packages independently of whether their converted deps
> > are stable or not. We will be worrying about that consistency once
> > we decide to unmask the flags.
> >
> > The extra advantage of that is that we can avoid pushing stable
> > users into the mess involved with partial conversion of emul-linux.
> > The idea is pretty simple: we keep emul-linux for stable, and once
> > everything is ready, we stable-unmask the flags and let stable
> > users grok multilib.
> >
> >
> > Now, to the problem. Currently we're just stable-masking
> > abi_x86_32 on amd64. This serves the latter purpose well and seems
> > to work for the former. This is probably because the remaining flag
> > is use.forced (abi_x86_64 on amd64, and abi_x86_32 on x86) which
> > seems to add it to implicit IUSE somehow.
> >
> > floppym has done some research w/ stable elfutils and no stable
> > converted bzip2. It seems that if abi_x86_32 is stable.use.masked
> > and abi_x86_64 is use.forced, repoman groks it. However, if we
> > remove abi_x86_64 from use.force, it properly reports flag mismatch
> > error (since no stable bzip2 has IUSE=abi_x86_64).
> >
> > Now, I honestly have no idea if this implicit use.force behavior
> > is PMS-y or not, and how other PMs handle it. I can't find
> > something like this in the PMS but that doc is horribly hard to
> > cross-reference, so I might be missing something. I'd appreciate if
> > someone could help me with that.
> >
> >
> > That said, I have an alternate idea inspired by the ppc breakage.
> > I'm thinking of replacing the amd64 abi_x86_32 mask with a global
> > stable mask of all abi_*_* flags on the relevant packages.
> >
> > Differences:
> >
> > 1) old solution: native flag is forced, other flags are masked. new
> > solution: all flags are masked.
> >
> > 2) old solution: we need to replicate the masks properly for
> > different arches/profiles. new solution: we can keep a single mask
> > for all arches.
> >
> > 3) old solution: MULTILIB_USEDEP magically works (w/ portage at
> > least). new solution: since all flags are disabled,
> > MULTILIB_USEDEP is a no-op and old packages match correctly.
> >
> > 4) old solution: forced native flag runs the native build. new
> > solution: fallback code runs the native build (since no flags are
> > enabled).
> >
> >
> > Your thoughts?
> >
>
>
> ... how messy is this going to be on current systems, if implemented?
> This -is- going to cause use flag changes on a whackload of packages,
> which at minimum I expect means a bunch of needless rebuilds.
>
> I think it's probably best to keep with the current path, and just
> hurry up the migration on x86. If we switch to masking all abi_*_*'s,
> then not only do we rebuild a whackload of packages now, but we also
> end up rebuilding them again when we remove the abi_*_* mask, -even
> if- the alternative abi's aren't needed.
Note that the same issue is spilling into ppc now that vapier wants to
introduce multilib there. If we keep going the current way, we need to
mask abi_ppc_32 in the ppc profiles, and that means yet another copy of
the same mask to maintain in sync.
> A note on #3: if MULTILIB_USEDEP -isnt- working on other PMs, i hope
> we have bug reports and we should work to resolve those ASAP or
> abandon the project/find a different solution. We're over a year into
> it though (i think?) so i expect it's fine since I don't remember
> hearing of any issues yet...
I don't know how many people are using Gentoo with alternative PMs
and stable keywords. I think Paludis could be a case but then, I don't
know ho
Re: [gentoo-dev] Stable masks on multilib packages
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
On 31/03/14 06:16 PM, Michał Górny wrote:
> Hello, all.
>
> The late multilib ppc issues made me re-check our stable masks on
> abi_x86_* flags and, honestly, I'm not sure if we're doing things
> the right way. First, a quick summary.
>
>
> Let's consider dev-libs/elfutils that has a dependency on
> app-arch/bzip2. If we're building 32-bit elfutils, it needs 32-bit
> libbz2. This is enforced through a dep in form of:
>
> app-arch/bzip2[${MULTILIB_USEDEP}]
>
> that gets expanded into:
>
> app-arch/bzip2[abi_x86_32(-)?,abi_x86_64(-)?,...,abi_mips_o32(-)?,...]
>
> which means that any of the ABI_* flags that gets enabled on
> elfutils, needs to be enabled on bzip2 as well. Of course, some of
> use.forcing and masking gets applied here but that doesn't really
> matter.
>
>
> Now, since we're pretty much converting a lot of different
> packages, some of them are eligible for stabilization earlier than
> the others. However, the extra MULTILIB_USEDEPs enforce stabilizing
> multilib dependencies before the actual packages which made a few
> developers unhappy.
>
> That's why we decided to stable-mask the flags on affected
> packages. Since the flags are masked on stable, people can
> stabilize packages independently of whether their converted deps
> are stable or not. We will be worrying about that consistency once
> we decide to unmask the flags.
>
> The extra advantage of that is that we can avoid pushing stable
> users into the mess involved with partial conversion of emul-linux.
> The idea is pretty simple: we keep emul-linux for stable, and once
> everything is ready, we stable-unmask the flags and let stable
> users grok multilib.
>
>
> Now, to the problem. Currently we're just stable-masking
> abi_x86_32 on amd64. This serves the latter purpose well and seems
> to work for the former. This is probably because the remaining flag
> is use.forced (abi_x86_64 on amd64, and abi_x86_32 on x86) which
> seems to add it to implicit IUSE somehow.
>
> floppym has done some research w/ stable elfutils and no stable
> converted bzip2. It seems that if abi_x86_32 is stable.use.masked
> and abi_x86_64 is use.forced, repoman groks it. However, if we
> remove abi_x86_64 from use.force, it properly reports flag mismatch
> error (since no stable bzip2 has IUSE=abi_x86_64).
>
> Now, I honestly have no idea if this implicit use.force behavior
> is PMS-y or not, and how other PMs handle it. I can't find
> something like this in the PMS but that doc is horribly hard to
> cross-reference, so I might be missing something. I'd appreciate if
> someone could help me with that.
>
>
> That said, I have an alternate idea inspired by the ppc breakage.
> I'm thinking of replacing the amd64 abi_x86_32 mask with a global
> stable mask of all abi_*_* flags on the relevant packages.
>
> Differences:
>
> 1) old solution: native flag is forced, other flags are masked. new
> solution: all flags are masked.
>
> 2) old solution: we need to replicate the masks properly for
> different arches/profiles. new solution: we can keep a single mask
> for all arches.
>
> 3) old solution: MULTILIB_USEDEP magically works (w/ portage at
> least). new solution: since all flags are disabled,
> MULTILIB_USEDEP is a no-op and old packages match correctly.
>
> 4) old solution: forced native flag runs the native build. new
> solution: fallback code runs the native build (since no flags are
> enabled).
>
>
> Your thoughts?
>
... how messy is this going to be on current systems, if implemented?
This -is- going to cause use flag changes on a whackload of packages,
which at minimum I expect means a bunch of needless rebuilds.
I think it's probably best to keep with the current path, and just
hurry up the migration on x86. If we switch to masking all abi_*_*'s,
then not only do we rebuild a whackload of packages now, but we also
end up rebuilding them again when we remove the abi_*_* mask, -even
if- the alternative abi's aren't needed.
A note on #3: if MULTILIB_USEDEP -isnt- working on other PMs, i hope
we have bug reports and we should work to resolve those ASAP or
abandon the project/find a different solution. We're over a year into
it though (i think?) so i expect it's fine since I don't remember
hearing of any issues yet...
on a side note, to follow up -- when we do remove the masks, what're
the thoughts on perhaps changing the multilib profile on amd64 to
auto-enable abi_x86_32 flags on some packages? Indeed, if the
toolchain is converted to multilib-build, would this be enough to
differentiate between amd64-multilib and pure64? (bootstrapping may
of course still be a problem if users are trying to switch, but
probably not any more of a problem than it is right now, right?)
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)
iF4EAREIAAYFAlM610sACgkQ2ugaI38ACPAPoQD/TpzYLaAwLiRBSVLRII/lI1OK
fN9hOQ8wvMaHoKfuHK4A/RmUtt1G7k8HEsAhsMSWAYFUd0KdlbEd4snhcxaR6W+M
=n7yQ
--
Re: [gentoo-dev] Stable masks on multilib packages
On Tue, Apr 1, 2014 at 9:58 AM, Alexandre Rostovtsev wrote: > On Tue, 2014-04-01 at 13:13 +0800, Ben de Groot wrote: >> >> In my opinion your multilib approach introduces an unnecessary degree >> of complexity, which --as has been shown here again-- is prone to >> breakage. >> >> It would be best for our beloved distro to revert all the multilib >> changes, and try a different approach, or leave this prone-to-breakage >> implementation to an overlay for the few people who would actually >> benefit from it. > > I am aware of only two solutions to the emul-linux-x86-* problems : > multilib-portage and multilib-build.eclass. The first requires everybody > to switch to a new package manager. The second allows us to keep using > portage, but requires library maintainers to add some simple boilerplate > to their ebuilds for multilib support. > > Do you have yet another alternative in mind? ++ I'm all for better solutions. I'm not in favor of abandoning solutions that work moderately well in favor of talking about maybe coming up with something better sometime down the road. The multilib eclass isn't a perfect solution. It will have issues in concept, design, and implementation. These will come up from time to time. I don't think it is productive that anytime any of these pop up that we end up having a discussion about just reverting it entirely. By all means work on a competing solution. Get it working and supported by portage. When the day comes that we want to endorse one solution or the other as the preferred solution we can have that discussion. Dirty hack or not the multilib eclass seems to be working fairly well in practice, especially considering just how big a change it is. The emul-linux packages were an even dirtier hack and those worked well for a decade. Progress can be incremental. Rich
Re: [gentoo-dev] Stable masks on multilib packages
On Tue, 2014-04-01 at 13:13 +0800, Ben de Groot wrote: > On 1 April 2014 06:16, Michał Górny wrote: > > Hello, all. > > > > The late multilib ppc issues made me re-check our stable masks on > > abi_x86_* flags and, honestly, I'm not sure if we're doing things > > the right way. > > > > That said, I have an alternate idea inspired by the ppc breakage. > > > > Your thoughts? > > In my opinion your multilib approach introduces an unnecessary degree > of complexity, which --as has been shown here again-- is prone to > breakage. > > It would be best for our beloved distro to revert all the multilib > changes, and try a different approach, or leave this prone-to-breakage > implementation to an overlay for the few people who would actually > benefit from it. Speaking as a wine maintainer, the emul-linux-x86-* approach has many times been proven to be an embarrassing failure and the main source of pain and frustration for wine users. The sooner emul-linux-x86-* can be removed from the tree, the better for Gentoo. I am aware of only two solutions to the emul-linux-x86-* problems : multilib-portage and multilib-build.eclass. The first requires everybody to switch to a new package manager. The second allows us to keep using portage, but requires library maintainers to add some simple boilerplate to their ebuilds for multilib support. Do you have yet another alternative in mind? signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] Stable masks on multilib packages
On 04/01/2014 07:40 AM, Anthony G. Basile wrote: On 04/01/2014 01:54 AM, Michał Górny wrote: >Also, I don't see that it should be an issue, but do you think this >might affect catalyst runs --- I have to ask because >repairing/reconfiguring seeds is a lot of work. Well, I think this mostly depends on whether you want multiple multilib ABIs in stages -- or if you assume that the toolchain is enough, and people will build other ABIs as they need. Though I think that once toolchain switches to abi_* flags (vapier seems to show interest in that), we will use.force the necessary ABIs on it and its dependencies, so there should be no explicit need for change. However, I guess the toolchain changes will be sent out for testing anyway, so releng can check first hand. I have taken a non-standard approach to the hardened/linux/uclibc profiles where I do no inherit from arch. There was too much cruft in arch, so I just mirror the relevant stuff including the abi flag stuff. I understand my approach has pros/cons ... but give out the should when Sorry, my fingers and brains are not in sync. ... give out the SHOUT when ... the change is being tested and I'll see if I'm about to have any breakage. I currently seed at least 12 different arches/isa/abi/hard-soft float combinations which would be a pita to fix manually on native (read slow) hardware. -- Anthony G. Basile, Ph.D. Gentoo Linux Developer [Hardened] E-Mail: bluen...@gentoo.org GnuPG FP : 1FED FAD9 D82C 52A5 3BAB DC79 9384 FA6E F52D 4BBA GnuPG ID : F52D4BBA
Re: [gentoo-dev] Stable masks on multilib packages
On 04/01/2014 01:54 AM, Michał Górny wrote: >Also, I don't see that it should be an issue, but do you think this >might affect catalyst runs --- I have to ask because >repairing/reconfiguring seeds is a lot of work. Well, I think this mostly depends on whether you want multiple multilib ABIs in stages -- or if you assume that the toolchain is enough, and people will build other ABIs as they need. Though I think that once toolchain switches to abi_* flags (vapier seems to show interest in that), we will use.force the necessary ABIs on it and its dependencies, so there should be no explicit need for change. However, I guess the toolchain changes will be sent out for testing anyway, so releng can check first hand. I have taken a non-standard approach to the hardened/linux/uclibc profiles where I do no inherit from arch. There was too much cruft in arch, so I just mirror the relevant stuff including the abi flag stuff. I understand my approach has pros/cons ... but give out the should when the change is being tested and I'll see if I'm about to have any breakage. I currently seed at least 12 different arches/isa/abi/hard-soft float combinations which would be a pita to fix manually on native (read slow) hardware. -- Anthony G. Basile, Ph.D. Gentoo Linux Developer [Hardened] E-Mail: bluen...@gentoo.org GnuPG FP : 1FED FAD9 D82C 52A5 3BAB DC79 9384 FA6E F52D 4BBA GnuPG ID : F52D4BBA
Re: [gentoo-dev] Stable masks on multilib packages
Dnia 2014-03-31, o godz. 19:09:44 "Anthony G. Basile" napisał(a): > On 03/31/2014 06:16 PM, Michał Górny wrote: > > That said, I have an alternate idea inspired by the ppc breakage. I'm > > thinking of replacing the amd64 abi_x86_32 mask with a global stable > > mask of all abi_*_* flags on the relevant packages. > I'm not sure exactly what you mean here --- where/how does this global > stable masking happen? Right now you have a bunch of maskings of > abi_x86_32 on various packages in arch/amd64/package.use.stable.mask but > not on any other arches where you just have the use.mask/use.force > combination. Are you going to migrate the amd64 file to other multilib > arches and mask non-native abis? Like on mips64el/multilib/n32 would > you be masking abi_mips_o32 and abi_mips_n64 for all those packages? The new solution would be base/package.use.stable.mask that would mask all of multilib on all stable arches, on this long list of packages. So, not only the two ABIs but all of them. Which would effectively make ebuild behave like it had no multilib at all. > > Your thoughts? > > How does this "go live" once these flags are enabled? Do you just > remove the global mask all at once? That sounds a bit scarier than just > removing the masks one package + deps at a time. (Maybe a non-question > because I'm still now sure how this masking works.) Something like this, yes. Once all packages are migrated and some time passes, we unmask all the flags locally and do a repoman run. We find out what needs to go stable, report bugs, wait and repeat. Then we do a second stabilization request, this time for testing the tree (mostly emul-linux replacement) with multilib enabled. Once arch teams are done testing, they remove the stable masks for particular ABI. When all reverse dependencies are fixed to use || () deps instead of emul-linux (and rev-bumped) we can move away from emul-linux through the usual procedure of last rites with a proper announcement. This is likely the most fragile step of all but we will do our best to make it as simple as possible, and I think our users can handle that. > Also, I don't see that it should be an issue, but do you think this > might affect catalyst runs --- I have to ask because > repairing/reconfiguring seeds is a lot of work. Well, I think this mostly depends on whether you want multiple multilib ABIs in stages -- or if you assume that the toolchain is enough, and people will build other ABIs as they need. Though I think that once toolchain switches to abi_* flags (vapier seems to show interest in that), we will use.force the necessary ABIs on it and its dependencies, so there should be no explicit need for change. However, I guess the toolchain changes will be sent out for testing anyway, so releng can check first hand. -- Best regards, Michał Górny signature.asc Description: PGP signature
Re: [gentoo-dev] Stable masks on multilib packages
On 1 April 2014 06:16, Michał Górny wrote: > Hello, all. > > The late multilib ppc issues made me re-check our stable masks on > abi_x86_* flags and, honestly, I'm not sure if we're doing things > the right way. > > That said, I have an alternate idea inspired by the ppc breakage. > > Your thoughts? In my opinion your multilib approach introduces an unnecessary degree of complexity, which --as has been shown here again-- is prone to breakage. It would be best for our beloved distro to revert all the multilib changes, and try a different approach, or leave this prone-to-breakage implementation to an overlay for the few people who would actually benefit from it. -- Cheers, Ben | yngwin Gentoo developer
Re: [gentoo-dev] Stable masks on multilib packages
On 03/31/2014 06:16 PM, Michał Górny wrote:
Hello, all.
The late multilib ppc issues made me re-check our stable masks on
abi_x86_* flags and, honestly, I'm not sure if we're doing things
the right way. First, a quick summary.
Let's consider dev-libs/elfutils that has a dependency
on app-arch/bzip2. If we're building 32-bit elfutils, it needs 32-bit
libbz2. This is enforced through a dep in form of:
app-arch/bzip2[${MULTILIB_USEDEP}]
that gets expanded into:
app-arch/bzip2[abi_x86_32(-)?,abi_x86_64(-)?,...,abi_mips_o32(-)?,...]
which means that any of the ABI_* flags that gets enabled on elfutils,
needs to be enabled on bzip2 as well. Of course, some of use.forcing
and masking gets applied here but that doesn't really matter.
Now, since we're pretty much converting a lot of different packages,
some of them are eligible for stabilization earlier than the others.
However, the extra MULTILIB_USEDEPs enforce stabilizing multilib
dependencies before the actual packages which made a few developers
unhappy.
That's why we decided to stable-mask the flags on affected packages.
Since the flags are masked on stable, people can stabilize packages
independently of whether their converted deps are stable or not. We
will be worrying about that consistency once we decide to unmask
the flags.
The extra advantage of that is that we can avoid pushing stable users
into the mess involved with partial conversion of emul-linux. The idea
is pretty simple: we keep emul-linux for stable, and once everything is
ready, we stable-unmask the flags and let stable users grok multilib.
Now, to the problem. Currently we're just stable-masking abi_x86_32
on amd64. This serves the latter purpose well and seems to work for the
former. This is probably because the remaining flag is use.forced
(abi_x86_64 on amd64, and abi_x86_32 on x86) which seems to add it to
implicit IUSE somehow.
floppym has done some research w/ stable elfutils and no stable
converted bzip2. It seems that if abi_x86_32 is stable.use.masked
and abi_x86_64 is use.forced, repoman groks it. However, if we remove
abi_x86_64 from use.force, it properly reports flag mismatch error
(since no stable bzip2 has IUSE=abi_x86_64).
Now, I honestly have no idea if this implicit use.force behavior is
PMS-y or not, and how other PMs handle it. I can't find something like
this in the PMS but that doc is horribly hard to cross-reference, so I
might be missing something. I'd appreciate if someone could help me
with that.
That said, I have an alternate idea inspired by the ppc breakage. I'm
thinking of replacing the amd64 abi_x86_32 mask with a global stable
mask of all abi_*_* flags on the relevant packages.
I'm not sure exactly what you mean here --- where/how does this global
stable masking happen? Right now you have a bunch of maskings of
abi_x86_32 on various packages in arch/amd64/package.use.stable.mask but
not on any other arches where you just have the use.mask/use.force
combination. Are you going to migrate the amd64 file to other multilib
arches and mask non-native abis? Like on mips64el/multilib/n32 would
you be masking abi_mips_o32 and abi_mips_n64 for all those packages?
Differences:
1) old solution: native flag is forced, other flags are masked.
new solution: all flags are masked.
2) old solution: we need to replicate the masks properly for different
arches/profiles.
new solution: we can keep a single mask for all arches.
This sounds like a partial answer to my above question.
3) old solution: MULTILIB_USEDEP magically works (w/ portage at least).
new solution: since all flags are disabled, MULTILIB_USEDEP
is a no-op and old packages match correctly.
4) old solution: forced native flag runs the native build.
new solution: fallback code runs the native build (since no flags
are enabled).
Your thoughts?
How does this "go live" once these flags are enabled? Do you just
remove the global mask all at once? That sounds a bit scarier than just
removing the masks one package + deps at a time. (Maybe a non-question
because I'm still now sure how this masking works.)
Also, I don't see that it should be an issue, but do you think this
might affect catalyst runs --- I have to ask because
repairing/reconfiguring seeds is a lot of work.
--
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail: bluen...@gentoo.org
GnuPG FP : 1FED FAD9 D82C 52A5 3BAB DC79 9384 FA6E F52D 4BBA
GnuPG ID : F52D4BBA
[gentoo-dev] Stable masks on multilib packages
Hello, all.
The late multilib ppc issues made me re-check our stable masks on
abi_x86_* flags and, honestly, I'm not sure if we're doing things
the right way. First, a quick summary.
Let's consider dev-libs/elfutils that has a dependency
on app-arch/bzip2. If we're building 32-bit elfutils, it needs 32-bit
libbz2. This is enforced through a dep in form of:
app-arch/bzip2[${MULTILIB_USEDEP}]
that gets expanded into:
app-arch/bzip2[abi_x86_32(-)?,abi_x86_64(-)?,...,abi_mips_o32(-)?,...]
which means that any of the ABI_* flags that gets enabled on elfutils,
needs to be enabled on bzip2 as well. Of course, some of use.forcing
and masking gets applied here but that doesn't really matter.
Now, since we're pretty much converting a lot of different packages,
some of them are eligible for stabilization earlier than the others.
However, the extra MULTILIB_USEDEPs enforce stabilizing multilib
dependencies before the actual packages which made a few developers
unhappy.
That's why we decided to stable-mask the flags on affected packages.
Since the flags are masked on stable, people can stabilize packages
independently of whether their converted deps are stable or not. We
will be worrying about that consistency once we decide to unmask
the flags.
The extra advantage of that is that we can avoid pushing stable users
into the mess involved with partial conversion of emul-linux. The idea
is pretty simple: we keep emul-linux for stable, and once everything is
ready, we stable-unmask the flags and let stable users grok multilib.
Now, to the problem. Currently we're just stable-masking abi_x86_32
on amd64. This serves the latter purpose well and seems to work for the
former. This is probably because the remaining flag is use.forced
(abi_x86_64 on amd64, and abi_x86_32 on x86) which seems to add it to
implicit IUSE somehow.
floppym has done some research w/ stable elfutils and no stable
converted bzip2. It seems that if abi_x86_32 is stable.use.masked
and abi_x86_64 is use.forced, repoman groks it. However, if we remove
abi_x86_64 from use.force, it properly reports flag mismatch error
(since no stable bzip2 has IUSE=abi_x86_64).
Now, I honestly have no idea if this implicit use.force behavior is
PMS-y or not, and how other PMs handle it. I can't find something like
this in the PMS but that doc is horribly hard to cross-reference, so I
might be missing something. I'd appreciate if someone could help me
with that.
That said, I have an alternate idea inspired by the ppc breakage. I'm
thinking of replacing the amd64 abi_x86_32 mask with a global stable
mask of all abi_*_* flags on the relevant packages.
Differences:
1) old solution: native flag is forced, other flags are masked.
new solution: all flags are masked.
2) old solution: we need to replicate the masks properly for different
arches/profiles.
new solution: we can keep a single mask for all arches.
3) old solution: MULTILIB_USEDEP magically works (w/ portage at least).
new solution: since all flags are disabled, MULTILIB_USEDEP
is a no-op and old packages match correctly.
4) old solution: forced native flag runs the native build.
new solution: fallback code runs the native build (since no flags
are enabled).
Your thoughts?
--
Best regards,
Michał Górny
signature.asc
Description: PGP signature
