Re: [gentoo-dev] [PATCH] multilib eclass support for building binaries for none-default ABI

[Thomas Sachau]

Alexis Ballier schrieb:
 just to be clear: I prefer the 1st patch but I would give the variable
 (COMPLETE_MULTILIB) a more private name and document this is only for
 multilib-portage and it will not work with regular portage.
 
 

Since you only argued against such implementation in general, but did
not write any reasoning behind your choice, not much i could get out of
this.

I have been doing the second choice now, as written in my answer to Ian.

For your variable request:

I left it this way, since it is intended for end users, who use regular
portage. In addition, it is a cleaner solution for multilib-portage,
since i dont have to internally overwrite an eclass function, but that
is just a side effect, since this issue never blocked multilib-portage.

-- 

Thomas Sachau
Gentoo Linux Developer



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] [PATCH] multilib eclass support for building binaries for none-default ABI

[Thomas Sachau]

Ian Stakenvicius schrieb:
 On 25/08/13 10:15 AM, Ulrich Mueller wrote:
 On Sun, 25 Aug 2013, Thomas Sachau wrote:
 
 workaround: add a variable, which changes the return of the
 function checking for the current ABI (always true with variable,
 without only true, when $ABI == $DEFAULT_ABI)
 
 Would this variable be set by the user, in profiles, or in
 ebuilds?
 
 first version (multilib1.patch) directly changes the output of
 the currently used multilib_is_native_abi() function:
 
 I think this would be very misleading. If a function is called 
 multilib_is_native_abi then it should test for exactly that, not
 for something else.
 
 
 - From the discussion on this that we had ~2 weeks ago, it seems that
 'multilib_is_native_abi' is only used in the wild now (and only meant
 to be used) to handle the cases where we want to say, build everything
 for default_abi and only build certain bits for the others.
 
 If there was a common usage that is important for the actual native
 abi (for instance, some sort of check enabling alternate code in a
 build system for a specific CHOST or whatnot), then keeping
 multilib_is_native_abi as it is would make a lot of sense, but since
 there isn't any cases of this I'm not sure it matters -- changing it's
 functionality to essentially become multilib_is_default_abi() (whether
 we rename it or not) seems to make the most sense to me.

While there may currently only be one usecase for the function, it may
be used in other cases too with time and wider usage of the new multilib
eclasses. If i modify the function for the binaries part, this would
require a new function for other usecases. If this really does not
happen, the multilib_is_native_abi function could still be deprecated
and removed in the future.

So i decided to go with a new function.

-- 

Thomas Sachau
Gentoo Linux Developer



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] [PATCH] multilib eclass support for building binaries for none-default ABI

[Thomas Sachau]

Thomas Sachau schrieb:
 Ulrich Mueller schrieb:
 On Sun, 25 Aug 2013, Thomas Sachau wrote:

 workaround: add a variable, which changes the return of the function
 checking for the current ABI (always true with variable, without
 only true, when $ABI == $DEFAULT_ABI)

 Would this variable be set by the user, in profiles, or in ebuilds?
 
 This variable can be set by users and profiles, when they want binaries
 for a different ABI (e.g. 64bit toolchain with 32bit userland).
 

 first version (multilib1.patch) directly changes the output of the
 currently used multilib_is_native_abi() function:

 I think this would be very misleading. If a function is called
 multilib_is_native_abi then it should test for exactly that, not for
 something else.

 second version (multilib2.patch) creates a new function, which
 should then be used by ebuild authors to check, if they should build
 ABI-specific content or not (using build_binaries() function instead
 of multilib_is_native_abi() function)

 +build_binaries() {

 Name space pollution? Prefix with multilib please.
 
 i dont really care about the naming, so if you prefer some multilib in
 there, how about this:
 
 multilib_build_binaries()?
 

 +if [[ ${COMPLETE_MULTILIB} == yes ]] ; then
 +return 0
 +else
 +multilib_is_native_abi
 +fi

 This can be expressed much shorter (and clearer):

  [[ ${COMPLETE_MULTILIB} == yes ]] || multilib_is_native_abi

Thanks for the suggestion, this has now been added with my above
suggested name and your suggested shorter version.

I will give ebuild maintainers some days to adapt their ebuilds and will
likely start opening bugs against ebuilds next week. Anyone, who wants
to delegate that work to me, just tell me and i will modify your ebuilds
to use the new function.


-- 

Thomas Sachau
Gentoo Linux Developer



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] [PATCH] multilib eclass support for building binaries for none-default ABI

[Alexis Ballier]

On Tue, 17 Sep 2013 14:38:08 +0200
Thomas Sachau to...@gentoo.org wrote:

 Alexis Ballier schrieb:
  just to be clear: I prefer the 1st patch but I would give the
  variable (COMPLETE_MULTILIB) a more private name and document this
  is only for multilib-portage and it will not work with regular
  portage.
  
  
 
 Since you only argued against such implementation in general, but did
 not write any reasoning behind your choice, not much i could get out
 of this.


you simply ignored it...

http://article.gmane.org/gmane.linux.gentoo.devel/87862

 I have been doing the second choice now, as written in my answer to
 Ian.
 
 For your variable request:
 
 I left it this way, since it is intended for end users, who use
 regular portage. 

it's certainly not intended that way in its current state; maybe you
believe it is or want to make it that way but the truth is that if
regular users set this variable with regular portage they end up with
broken deps and packages failing to build...

 In addition, it is a cleaner solution for
 multilib-portage, since i dont have to internally overwrite an eclass
 function, but that is just a side effect, since this issue never
 blocked multilib-portage.

I understood this, hence the request for a more private name in order
not to make your work harder...

Re: [gentoo-dev] [PATCH] multilib eclass support for building binaries for none-default ABI

[Michał Górny]

Dnia 2013-08-25, o godz. 23:55:59
Thomas Sachau to...@gentoo.org napisał(a):

 Michał Górny schrieb:
  Dnia 2013-08-25, o godz. 21:34:09
  Thomas Sachau to...@gentoo.org napisał(a):
  
  Ulrich Mueller schrieb:
  On Sun, 25 Aug 2013, Thomas Sachau wrote:
 
  workaround: add a variable, which changes the return of the function
  checking for the current ABI (always true with variable, without
  only true, when $ABI == $DEFAULT_ABI)
 
  Would this variable be set by the user, in profiles, or in ebuilds?
 
  This variable can be set by users and profiles, when they want binaries
  for a different ABI (e.g. 64bit toolchain with 32bit userland).
  
  Where it simply won't work since executables for the native ABI will
  overwrite earlier ones.
  
 
 You obviously dont read my complete mails, otherwise you would have seen
 my later example, which works perfectly fine, once this patch applied:
 For the complete userland, set ABI_X86=32, so no 64bit target, no
 executables for the native ABI, nothing overwriting the 32bit binaries
 = it works.

This is forbidden by profiles and as you noticed above would cause more
screwup than benefit.

 Of course, if there are further issues with the ebuilds (like only parts
 of the dependencies converted with those needed for the binaries left
 out), it may cause issues with the current situation of the eclass and
 ebuild, but will again allow users of multilib-portage to do exactly,
 what i described, since they already are able to build the dependencies
 for their target ABI. :-)

multilib-portage can set DEFAULT_ABI=$ABI as I suggested and you
happily ignored. I can understand that this could cause issues but you
didn't seem to care to reply.

Instead, you are trying to mess up gx86 with more hacks for your fancy
out-of-tree project. Hacks that will convince people something works
while it doesn't. And didn't I write this already?

-- 
Best regards,
Michał Górny


signature.asc
Description: PGP signature

Re: [gentoo-dev] [PATCH] multilib eclass support for building binaries for none-default ABI

[Ian Stakenvicius]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 25/08/13 10:15 AM, Ulrich Mueller wrote:
 On Sun, 25 Aug 2013, Thomas Sachau wrote:
 
 workaround: add a variable, which changes the return of the
 function checking for the current ABI (always true with variable,
 without only true, when $ABI == $DEFAULT_ABI)
 
 Would this variable be set by the user, in profiles, or in
 ebuilds?
 
 first version (multilib1.patch) directly changes the output of
 the currently used multilib_is_native_abi() function:
 
 I think this would be very misleading. If a function is called 
 multilib_is_native_abi then it should test for exactly that, not
 for something else.
 

- From the discussion on this that we had ~2 weeks ago, it seems that
'multilib_is_native_abi' is only used in the wild now (and only meant
to be used) to handle the cases where we want to say, build everything
for default_abi and only build certain bits for the others.

If there was a common usage that is important for the actual native
abi (for instance, some sort of check enabling alternate code in a
build system for a specific CHOST or whatnot), then keeping
multilib_is_native_abi as it is would make a lot of sense, but since
there isn't any cases of this I'm not sure it matters -- changing it's
functionality to essentially become multilib_is_default_abi() (whether
we rename it or not) seems to make the most sense to me.


 second version (multilib2.patch) creates a new function, which 
 should then be used by ebuild authors to check, if they should
 build ABI-specific content or not (using build_binaries()
 function instead of multilib_is_native_abi() function)
 
 +build_binaries() {
 
 Name space pollution? Prefix with multilib please.
 
 + if [[ ${COMPLETE_MULTILIB} == yes ]] ; then +   return 0 +  
 else +
 multilib_is_native_abi +  fi
 
 This can be expressed much shorter (and clearer):
 
 [[ ${COMPLETE_MULTILIB} == yes ]] || multilib_is_native_abi
 
 But allow me a stupid question, why do you want to build binaries
 for other ABIs anyway? It's called multilib, not multibin.
 
 Ulrich
 

Some users want to have a toolchain that is 64bit and the rest of the
userspace that is 32bit.  Or, they want to just have certain packages
installed 32bits, to support specific use-cases.  Essentially, the
want to have multi-abi instead of multi-lib, and so they want the
current multi-lib to be expanded to allow them to do it.


-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.20 (GNU/Linux)

iF4EAREIAAYFAlIbYR8ACgkQ2ugaI38ACPBG8AEAgcED8DZxyN0c98nMKvkCwNRG
zO6AcwF83oBL0PzOErsA/0gPMFZsX0+sKOXHo557L9X0Ha3S+9V8ZQVDWBVVL0Xk
=pvlE
-END PGP SIGNATURE-

Re: [gentoo-dev] [PATCH] multilib eclass support for building binaries for none-default ABI

[Ian Stakenvicius]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 25/08/13 10:30 AM, Michał Górny wrote:
 Dnia 2013-08-25, o godz. 16:15:31 Ulrich Mueller u...@gentoo.org
 napisał(a):
 
 On Sun, 25 Aug 2013, Thomas Sachau wrote:
 
 first version (multilib1.patch) directly changes the output of
 the currently used multilib_is_native_abi() function:
 
 I think this would be very misleading. If a function is called 
 multilib_is_native_abi then it should test for exactly that, not
 for something else.
 
 It's for multilib-portage. It hacks over everything in the
 environment. None of multilib variables are really meaningful in
 this context, so I don't have a problem with hacking that
 behavior.
 

Multilib-portage could just override the definition of
multilib_is_native_abi so that it's always true -- this change isn't
necessary for that particular use case.  It *is* necessary to support
end-users that want to do what multilib-portage offers but just with
regular portage.


-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.20 (GNU/Linux)

iF4EAREIAAYFAlIbYWkACgkQ2ugaI38ACPDjKwEAq5Ocxg4NItUkZ+qS+4Hj2iag
zVqJpa0E2oDDAyXpK5IBAIR7S0NOfHuN3hd/5fVNKebxdlwUo4yPee8rWFHv8Q48
=dqDh
-END PGP SIGNATURE-

Re: [gentoo-dev] [PATCH] multilib eclass support for building binaries for none-default ABI

[Alexis Ballier]

On Sun, 25 Aug 2013 16:15:31 +0200
Ulrich Mueller u...@gentoo.org wrote:
  first version (multilib1.patch) directly changes the output of the
  currently used multilib_is_native_abi() function:
 
 I think this would be very misleading. If a function is called
 multilib_is_native_abi then it should test for exactly that, not for
 something else.

Native abi only really makes sense as 'the abi you want the binaries
for'; maybe it is misleading, but i don't think it's worth changing the
name.

However, these patches do not make sense as the only thing they do
is allowing building binaries twice so that the 'native abi' ones
override the previously built binaries. The justification for this is an
invalid, utterly broken, hypothetical case where people install
packages with invalid settings (disabling default/native abi).

Even if we want to support this approach, this will not work that
easily: Some packages install both binaries and libraries; usually the
binaries have more deps than the libraries, hence the correct dep
string for this packages is [MULTILIB_USEDEP] on the deps of the
libraries and a plain dep (assuming native abi) for the rest. Look at
e.g. jack-audio-connection-kit or lame. Anything else is bloated.

If we want to allow multibin, or rather a mixed system where some bins
have ABI A and some others have ABI B, this will require a more
thorough solution than these quick hacks: With one of these patches, if
you ask for building all the binaries and disable your native abi, you
get a binary with a non native ABI but also get broken deps in your
packages since they do not RDEPEND on the correct ABI of its deps.

Alexis.

Re: [gentoo-dev] [PATCH] multilib eclass support for building binaries for none-default ABI

[Alexis Ballier]

just to be clear: I prefer the 1st patch but I would give the variable
(COMPLETE_MULTILIB) a more private name and document this is only for
multilib-portage and it will not work with regular portage.

Re: [gentoo-dev] [PATCH] multilib eclass support for building binaries for none-default ABI

[Ulrich Mueller]

 On Sun, 25 Aug 2013, Thomas Sachau wrote:

 workaround: add a variable, which changes the return of the function
 checking for the current ABI (always true with variable, without
 only true, when $ABI == $DEFAULT_ABI)

Would this variable be set by the user, in profiles, or in ebuilds?

 first version (multilib1.patch) directly changes the output of the
 currently used multilib_is_native_abi() function:

I think this would be very misleading. If a function is called
multilib_is_native_abi then it should test for exactly that, not for
something else.

 second version (multilib2.patch) creates a new function, which
 should then be used by ebuild authors to check, if they should build
 ABI-specific content or not (using build_binaries() function instead
 of multilib_is_native_abi() function)

+build_binaries() {

Name space pollution? Prefix with multilib please.

+   if [[ ${COMPLETE_MULTILIB} == yes ]] ; then
+   return 0
+   else
+   multilib_is_native_abi
+   fi

This can be expressed much shorter (and clearer):

[[ ${COMPLETE_MULTILIB} == yes ]] || multilib_is_native_abi

But allow me a stupid question, why do you want to build binaries for
other ABIs anyway? It's called multilib, not multibin.

Ulrich

Re: [gentoo-dev] [PATCH] multilib eclass support for building binaries for none-default ABI

[Michał Górny]

Dnia 2013-08-25, o godz. 16:15:31
Ulrich Mueller u...@gentoo.org napisał(a):

  On Sun, 25 Aug 2013, Thomas Sachau wrote:
 
  first version (multilib1.patch) directly changes the output of the
  currently used multilib_is_native_abi() function:
 
 I think this would be very misleading. If a function is called
 multilib_is_native_abi then it should test for exactly that, not for
 something else.

It's for multilib-portage. It hacks over everything in the environment.
None of multilib variables are really meaningful in this context, so I
don't have a problem with hacking that behavior.

-- 
Best regards,
Michał Górny


signature.asc
Description: PGP signature

Re: [gentoo-dev] [PATCH] multilib eclass support for building binaries for none-default ABI

[Thomas Sachau]

Ulrich Mueller schrieb:
 On Sun, 25 Aug 2013, Thomas Sachau wrote:
 
 workaround: add a variable, which changes the return of the function
 checking for the current ABI (always true with variable, without
 only true, when $ABI == $DEFAULT_ABI)
 
 Would this variable be set by the user, in profiles, or in ebuilds?

This variable can be set by users and profiles, when they want binaries
for a different ABI (e.g. 64bit toolchain with 32bit userland).

 
 first version (multilib1.patch) directly changes the output of the
 currently used multilib_is_native_abi() function:
 
 I think this would be very misleading. If a function is called
 multilib_is_native_abi then it should test for exactly that, not for
 something else.
 
 second version (multilib2.patch) creates a new function, which
 should then be used by ebuild authors to check, if they should build
 ABI-specific content or not (using build_binaries() function instead
 of multilib_is_native_abi() function)
 
 +build_binaries() {
 
 Name space pollution? Prefix with multilib please.

i dont really care about the naming, so if you prefer some multilib in
there, how about this:

multilib_build_binaries()?

 
 + if [[ ${COMPLETE_MULTILIB} == yes ]] ; then
 + return 0
 + else
 + multilib_is_native_abi
 + fi
 
 This can be expressed much shorter (and clearer):
 
   [[ ${COMPLETE_MULTILIB} == yes ]] || multilib_is_native_abi
 
 But allow me a stupid question, why do you want to build binaries for
 other ABIs anyway? It's called multilib, not multibin.

I already wrote about a setup needing the above. If a user only enables
ab_x86_32 for ebuilds with binary-restrictions, they wont get any binary
at all.
Pretty bad, when you try a reboot with e.g. no udev binaries. ;-)

In addition, users previously had the choice with multilib-portage to
build binaries for some or all target ABIs. Restricting the building of
binaries to the default ABI removes this ability from users.


-- 

Thomas Sachau
Gentoo Linux Developer



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] [PATCH] multilib eclass support for building binaries for none-default ABI

[Michał Górny]

Dnia 2013-08-25, o godz. 21:34:09
Thomas Sachau to...@gentoo.org napisał(a):

 Ulrich Mueller schrieb:
  On Sun, 25 Aug 2013, Thomas Sachau wrote:
  
  workaround: add a variable, which changes the return of the function
  checking for the current ABI (always true with variable, without
  only true, when $ABI == $DEFAULT_ABI)
  
  Would this variable be set by the user, in profiles, or in ebuilds?
 
 This variable can be set by users and profiles, when they want binaries
 for a different ABI (e.g. 64bit toolchain with 32bit userland).

Where it simply won't work since executables for the native ABI will
overwrite earlier ones.

-- 
Best regards,
Michał Górny


signature.asc
Description: PGP signature

Re: [gentoo-dev] [PATCH] multilib eclass support for building binaries for none-default ABI

[Thomas Sachau]

Michał Górny schrieb:
 Dnia 2013-08-25, o godz. 21:34:09
 Thomas Sachau to...@gentoo.org napisał(a):
 
 Ulrich Mueller schrieb:
 On Sun, 25 Aug 2013, Thomas Sachau wrote:

 workaround: add a variable, which changes the return of the function
 checking for the current ABI (always true with variable, without
 only true, when $ABI == $DEFAULT_ABI)

 Would this variable be set by the user, in profiles, or in ebuilds?

 This variable can be set by users and profiles, when they want binaries
 for a different ABI (e.g. 64bit toolchain with 32bit userland).
 
 Where it simply won't work since executables for the native ABI will
 overwrite earlier ones.
 

You obviously dont read my complete mails, otherwise you would have seen
my later example, which works perfectly fine, once this patch applied:
For the complete userland, set ABI_X86=32, so no 64bit target, no
executables for the native ABI, nothing overwriting the 32bit binaries
= it works.

Of course, if there are further issues with the ebuilds (like only parts
of the dependencies converted with those needed for the binaries left
out), it may cause issues with the current situation of the eclass and
ebuild, but will again allow users of multilib-portage to do exactly,
what i described, since they already are able to build the dependencies
for their target ABI. :-)

-- 

Thomas Sachau
Gentoo Linux Developer



signature.asc
Description: OpenPGP digital signature