Re: RFC: x86-runtime packages (was: Re: libexec and multi-arch)

2018-02-14 Thread Tomasz Pala 
On Mon, Feb 12, 2018 at 09:04:07 +0100, Jacek Konieczny wrote:

> I was thinking about some other solution: Make some '32bit runtime'
> packages which would be built from our i686 packages and contain only
> the libraries and other necessary files needed to run x86 apps on x86_64
> system.
[...]
> Updating of the spec files could be partially automated.

If doing any magic on spec files, one could as well make them pure libs
(no other contents, except maybe for %_docdir) and the problem is solved.
However, this is not what should be done manually these days - I would
expect the package manager to either - build such pure libs subpackage
automatically (at most marking the spec files with some macro) or ignore
the conflicting contents during the install.

> This packages would be explicitly made not to conflict with anything
> from x86_64 packages and installing .i686 packages in x86_64 system
> would not be needed any more.

This is only part of the problem - there are other cases:
- installing multiple versions of some library in single arch (for legacy
  software),
- installing 32-bit apps in 64-bit environment - this MIGHT require
  installing other than pure-libs packages from 32-bit tree,
- installing packages from other distros, a PLD lacks MANY of useful
  software these days, or has some very old versions. I am really tired
  of compiling everything I want to use from sources, especially when
  time is a matter.

I am also tired of manually creating libs/devel/static subpackages (with 
dumb "%{name} devel files" descriptions and %post ldconfig repating all
over again) in spec files - this is SO automatable... Such development
is doomed.

> The packages could be:
> 
> x86-runtime-basesystem (glibc, libstdc++ with dependencies)
> x86-runtime-X11 (X11 libraries, maybe with the popular toolkits)
> x86-runtime-SDL (SDL_* ??? mostly for games)
> etc.

LDAP, kerberos, SASL, all-the-databases...? In fact, most of the
32-bit libs might be required.

This is much work for something, that should be handled by single tool.
What we need is some SysV->systemd-like transition for rpm, which is not
doing it's job. It was fine in '90s, but the world has changed.
Especially when there is appropriate code in core of rpm (colouring),
but the trivial solution is rejected by stucked in his own flat world
maintainer. Relying on such software has no future without tons of
package-maintainers.

-- 
Tomasz Pala 
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: RFC: x86-runtime packages (was: Re: libexec and multi-arch)

2018-02-14 Thread Michael Shigorin 
On Mon, Feb 12, 2018 at 09:04:07AM +0100, Jacek Konieczny wrote:
> >> Truly the best way out of all these discussions is to
> >> complete the 10+ year transition from 32bit to 64bit and
> >> stop the endless discussions and fiddle up retrofits.
> > Be my guest and just convert all the binary-only programs to 64 bit.
> > Including the commercial ones.
> I was thinking about some other solution: Make some '32bit runtime'
> packages which would be built from our i686 packages and
> contain only the libraries and other necessary files needed to
> run x86 apps on x86_64 system.

ALT did it through rpmrebuild and arepo (and rpmrebuild-arepo):
https://lists.altlinux.org/pipermail/devel/2012-April/193781.html
http://altlinux.org/biarch
(both links in Russian)

-- 
  WBR, Michael Shigorin / http://altlinux.org
  -- http://opennet.ru / http://anna-news.info
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

RFC: x86-runtime packages (was: Re: libexec and multi-arch)

2018-02-12 Thread Jacek Konieczny 
On 2018-02-12 07:42, Tomasz Pala wrote:
>> Truly the best way out of all these discussions is to complete the 10+ year 
>> transition from 32bit to 64bit and stop the endless discussions and fiddle 
>> up retrofits.
> 
> Be my guest and just convert all the binary-only programs to 64 bit.
> Including the commercial ones.

I was thinking about some other solution: Make some '32bit runtime'
packages which would be built from our i686 packages and contain only
the libraries and other necessary files needed to run x86 apps on x86_64
system.

This packages would be explicitly made not to conflict with anything
from x86_64 packages and installing .i686 packages in x86_64 system
would not be needed any more.

The packages could be:

x86-runtime-basesystem (glibc, libstdc++ with dependencies)
x86-runtime-X11 (X11 libraries, maybe with the popular toolkits)
x86-runtime-SDL (SDL_* – mostly for games)
etc.

'Source*' in those specs would point to .i686.rpm files built by our
i686 builders.

Updating of the spec files could be partially automated.


Jacek
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: libexec and multi-arch

2018-02-11 Thread Tomasz Pala 
On Sun, Feb 11, 2018 at 22:10:44 -0500, Jeff Johnson wrote:

>> How is rpm recognizing files that are subject to this special-handling
>> as same-path-executables?
> 
> RPM reads elf headers and sets a per file color flag in metadata:
> 0==no arch or not elf
> 1==elf32
> 2==elf64
> 4== mostly unused, except on big endian mips
> Bits were used instead of an enum in case some idiot decided to put both 
> elf32 and elf64 in the same static archive.

OK, so it is really just being too smart. If it happens, that there
are some simple sh-scripts in %_bindir, they won't be colored and
so the multilib would require general-rules approach (the same file
content or prepare for conflict).

> Choosing between multiple files with different flags but identical paths is 
> computed during install, and the preferred arch is used while other files are 
> marked "wrong color" and not installed
> 
>> How to extend this to all the files inside libexecdir?
> 
> There's nothing to extend: the confusion comes from mixtures of 
> libraries/modules/executables in libexecdir, not anything else.

I would explain this to you, but having a history of conversations in
mind I know this is pointless. So thank you for response to all my
questions.

> That is essentially what is implemented in rpm for files from different arch 
> (technically "color" as in elf32 or elf64) that are on the same path.

I just wanted the colors to be used without elf headers.

But I guess this is simply another misdesign choice for the colors to be
inheritly connected with specific _file_ architecture, not entire
_package_ architecture (at least as a fallback), so for non-ELF files rpm is 
simply not going to
be a help.

>> Currently, if such files exist, they need to have identical content for
>> the packages to be installed simultaneously, or be separated into
>> respective subpackages, which is unnecessary maintenance burden.
> 
> Um, I'm not sure how rpm is responsible for packaging policies (like 
> separated into sub packages) or maintenance burden.

Who says about responsibility? It simply doesn't help solving such cases.
And since they are happening frequently, this enforces to create some
decent packaging policy. It would be much better to solve this in some
single central tool, but there is no good package manager for Linux for
now, so we have to use what we have. I hope there would be some 21st
century package manager some day, handling ACLs, xattrs, file-versioning
etc.

-- 
Tomasz Pala 
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: libexec and multi-arch

2018-02-11 Thread Tomasz Pala 
On Sun, Feb 11, 2018 at 21:50:32 -0500, Jeff Johnson wrote:

> And then there is the /usr/lib/rpm path to "probably argue" about.

Not much to argue here until there are any _practical_ problems with
this location. Although for the sake of good order you might consider
using libexec here.

> Truly the best way out of all these discussions is to complete the 10+ year 
> transition from 32bit to 64bit and stop the endless discussions and fiddle up 
> retrofits.

Be my guest and just convert all the binary-only programs to 64 bit.
Including the commercial ones.

-- 
Tomasz Pala 
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: libexec and multi-arch

2018-02-11 Thread Jeff Johnson 


> On Feb 5, 2018, at 5:46 PM, Tomasz Pala  wrote:
> 
> On Mon, Feb 05, 2018 at 15:20:55 -0500, Jeff Johnson wrote:
> 
>>> No, it only hidden the problem behind getconf binary being handled
>>> _somehow_. I once even wondered how this is done, apparently rpm is
>>> trying to be way too smart.
>> 
>> RPM implements arch specific content generally as:
>>  Libraries on different paths.
>>  Executables on same path.
> 
> libexec directory should be treated as binaries, not libraries (even if
> technically some of the contents are libraries, like plugins) nor data,
> as the purpose of this is to carry private helpers of the binaries, not
> any shared code.
> 
>> This requires a resolution to a preferred arch (usually x86_64) when 
>> installing executables onto identical paths.
> 
> How is rpm recognizing files that are subject to this special-handling
> as same-path-executables?

RPM reads elf headers and sets a per file color flag in metadata:
0==no arch or not elf
1==elf32
2==elf64
4== mostly unused, except on big endian mips
Bits were used instead of an enum in case some idiot decided to put both elf32 
and elf64 in the same static archive.

Choosing between multiple files with different flags but identical paths is 
computed during install, and the preferred arch is used while other files are 
marked "wrong color" and not installed

> How to extend this to all the files inside libexecdir?

There's nothing to extend: the confusion comes from mixtures of 
libraries/modules/executables in libexecdir, not anything else.

> How to extend this to all the files in mandir maybe?
> 

There are no elf files in mandir, so an extension of what rpm does makes no 
sense.

Meanwhile there are many conventions to extend man pages, in subdirectories, 
and with suffixes, that can be used if necessary.

>> Whether RPM is too smart or the requested implementation is insufficiently 
>> general is arguable. For example, one might desire the ability for 
>> per-file-path configurable choice of architecture rather than per-system 
>> confIguration.
> 
> Or - to always use preferred arch, for ANY file that is overlapping in
> packages that differ only in arch, e.g. /etc or anything inside datadir.
> 

That is essentially what is implemented in rpm for files from different arch 
(technically "color" as in elf32 or elf64) that are on the same path.

> Currently, if such files exist, they need to have identical content for
> the packages to be installed simultaneously, or be separated into
> respective subpackages, which is unnecessary maintenance burden.
> 

Um, I'm not sure how rpm is responsible for packaging policies (like separated 
into sub packages) or maintenance burden.

hth

73 de Jeff
> -- 
> Tomasz Pala 
> ___
> pld-devel-en mailing list
> pld-devel-en@lists.pld-linux.org
> http://lists.pld-linux.org/mailman/listinfo/pld-devel-en
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: libexec and multi-arch

2018-02-11 Thread Jeff Johnson 


> On Feb 5, 2018, at 3:23 PM, Neal Gompa  wrote:
> 
> I'd probably argue that libexec should have the same multi-arch
> handling that (s)bin does (primary arch "wins" and secondary arches
> are ignored), though last I checked this isn't specific to paths and
> should Just Work(TM).
> 
> 

And then there is the /usr/lib/rpm path to "probably argue" about.

Truly the best way out of all these discussions is to complete the 10+ year 
transition from 32bit to 64bit and stop the endless discussions and fiddle up 
retrofits.

73 de Jeff
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: libexec and multi-arch

2018-02-05 Thread Tomasz Pala 
On Mon, Feb 05, 2018 at 15:23:25 -0500, Neal Gompa wrote:

>>> _somehow_. I once even wondered how this is done, apparently rpm is
>>> trying to be way too smart.
[...]
> I'd probably argue that libexec should have the same multi-arch
> handling that (s)bin does (primary arch "wins" and secondary arches
> are ignored), though last I checked this isn't specific to paths and
> should Just Work(TM).

Well, if not specific to paths, but autodetected per-file type, this
fulfils my definition of 'trying to be too smart'.
Because libexec files might be in fact of any type.

-- 
Tomasz Pala 
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: libexec and multi-arch

2018-02-05 Thread Tomasz Pala 
On Mon, Feb 05, 2018 at 15:20:55 -0500, Jeff Johnson wrote:

>> No, it only hidden the problem behind getconf binary being handled
>> _somehow_. I once even wondered how this is done, apparently rpm is
>> trying to be way too smart.
> 
> RPM implements arch specific content generally as:
>   Libraries on different paths.
>   Executables on same path.

libexec directory should be treated as binaries, not libraries (even if
technically some of the contents are libraries, like plugins) nor data,
as the purpose of this is to carry private helpers of the binaries, not
any shared code.

> This requires a resolution to a preferred arch (usually x86_64) when 
> installing executables onto identical paths.

How is rpm recognizing files that are subject to this special-handling
as same-path-executables?
How to extend this to all the files inside libexecdir?
How to extend this to all the files in mandir maybe?

> Whether RPM is too smart or the requested implementation is insufficiently 
> general is arguable. For example, one might desire the ability for 
> per-file-path configurable choice of architecture rather than per-system 
> confIguration.

Or - to always use preferred arch, for ANY file that is overlapping in
packages that differ only in arch, e.g. /etc or anything inside datadir.

Currently, if such files exist, they need to have identical content for
the packages to be installed simultaneously, or be separated into
respective subpackages, which is unnecessary maintenance burden.

-- 
Tomasz Pala 
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: libexec and multi-arch

2018-02-05 Thread Neal Gompa 
On Mon, Feb 5, 2018 at 3:20 PM, Jeff Johnson  wrote:
>
>
>> On Feb 5, 2018, at 10:13 AM, Tomasz Pala  wrote:
>>
>>> On Mon, Feb 05, 2018 at 10:47:32 +0100, Jacek Konieczny wrote:
>>>
>>> Using lib/lib64/libx32 instead of libexec was much smarter in this case.
>>
>> No, it only hidden the problem behind getconf binary being handled
>> _somehow_. I once even wondered how this is done, apparently rpm is
>> trying to be way too smart.
>>
>
> RPM implements arch specific content generally as:
>   Libraries on different paths.
>   Executables on same path.
> This requires a resolution to a preferred arch (usually x86_64) when 
> installing executables onto identical paths.
>
> Whether RPM is too smart or the requested implementation is insufficiently 
> general is arguable. For example, one might desire the ability for 
> per-file-path configurable choice of architecture rather than per-system 
> confIguration.
>

I'd probably argue that libexec should have the same multi-arch
handling that (s)bin does (primary arch "wins" and secondary arches
are ignored), though last I checked this isn't specific to paths and
should Just Work(TM).



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: libexec and multi-arch

2018-02-05 Thread Jeff Johnson 


> On Feb 5, 2018, at 10:13 AM, Tomasz Pala  wrote:
> 
>> On Mon, Feb 05, 2018 at 10:47:32 +0100, Jacek Konieczny wrote:
>> 
>> Using lib/lib64/libx32 instead of libexec was much smarter in this case.
> 
> No, it only hidden the problem behind getconf binary being handled
> _somehow_. I once even wondered how this is done, apparently rpm is
> trying to be way too smart.
> 

RPM implements arch specific content generally as:
  Libraries on different paths.
  Executables on same path.
This requires a resolution to a preferred arch (usually x86_64) when installing 
executables onto identical paths.

Whether RPM is too smart or the requested implementation is insufficiently 
general is arguable. For example, one might desire the ability for 
per-file-path configurable choice of architecture rather than per-system 
confIguration.

73 de Jeff
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: libexec and multi-arch

2018-02-05 Thread Tomasz Pala 
On Mon, Feb 05, 2018 at 10:47:32 +0100, Jacek Konieczny wrote:

> Using lib/lib64/libx32 instead of libexec was much smarter in this case.

No, it only hidden the problem behind getconf binary being handled
_somehow_. I once even wondered how this is done, apparently rpm is
trying to be way too smart.

> Are we dropping multi-arch support in PLD all together?

Multi-arch means ability to install multiple libraries, not binaries -
there is no /usr/bin64 for getconf using /usr/lib64/getconf files.


Anyway, our multi-arch usually requires the same versions of packages
due to various man pages or locales shipped with the libraries. Just
like in case of glibc.

-- 
Tomasz Pala 
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: libexec and multi-arch

2018-02-05 Thread Tomasz Pala 
On Mon, Feb 05, 2018 at 10:14:41 +0100, Arkadiusz Miśkiewicz wrote:
> 
> What's the approach for problems like:
> 
> error: Install/Erase problems:
> file /usr/libexec/getconf/POSIX_V6_ILP32_OFFBIG conflicts between 
> attempted installs of glibc-2.27-1.i686 and glibc-2.27-1.x32
> file /usr/libexec/getconf/POSIX_V7_ILP32_OFFBIG conflicts between 
> attempted installs of glibc-2.27-1.i686 and glibc-2.27-1.x32
> file /usr/libexec/getconf/XBS5_ILP32_OFFBIG conflicts between 
> attempted installs of glibc-2.27-1.i686 and glibc-2.27-1.x32
> 
> ?

The same as in the getconf binary itself:

$ rpm -qpilv glibc-2.27-2.i686.rpm glibc-2.27-2.x86_64.rpm  | grep 
/usr/bin/getconf
-rwxr-xr-x7 rootroot21988 Feb  5 11:55 /usr/bin/getconf
-rwxr-xr-x4 rootroot30968 Feb  5 11:56 /usr/bin/getconf

- they apparently differ, so which one do you want to have/use? How is
THIS problem resolved?

They should be either separated to subpackages that are not subject of
multiarch installations, or handled/colored/whatever by package manager
during the install process.

-- 
Tomasz Pala 
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: libexec and multi-arch

2018-02-05 Thread Jacek Konieczny 
On 2018-02-05 10:36, Jan Rękorajski wrote:
> On Mon, 05 Feb 2018, Arkadiusz Miśkiewicz wrote:
> 
>>
>> What's the approach for problems like:
>>
>> error: Install/Erase problems:
>> file /usr/libexec/getconf/POSIX_V6_ILP32_OFFBIG conflicts between 
>> attempted installs of glibc-2.27-1.i686 and glibc-2.27-1.x32
>> file /usr/libexec/getconf/POSIX_V7_ILP32_OFFBIG conflicts between 
>> attempted installs of glibc-2.27-1.i686 and glibc-2.27-1.x32
>> file /usr/libexec/getconf/XBS5_ILP32_OFFBIG conflicts between 
>> attempted installs of glibc-2.27-1.i686 and glibc-2.27-1.x32
>>
>> ?
> 
> Use the --force, Luke.
> 
> I haven't find anything smarter yet.

Using lib/lib64/libx32 instead of libexec was much smarter in this case.
Are we dropping multi-arch support in PLD all together?

Jacek
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: libexec and multi-arch

2018-02-05 Thread Jan Rękorajski 
On Mon, 05 Feb 2018, Arkadiusz Miśkiewicz wrote:

> 
> What's the approach for problems like:
> 
> error: Install/Erase problems:
> file /usr/libexec/getconf/POSIX_V6_ILP32_OFFBIG conflicts between 
> attempted installs of glibc-2.27-1.i686 and glibc-2.27-1.x32
> file /usr/libexec/getconf/POSIX_V7_ILP32_OFFBIG conflicts between 
> attempted installs of glibc-2.27-1.i686 and glibc-2.27-1.x32
> file /usr/libexec/getconf/XBS5_ILP32_OFFBIG conflicts between 
> attempted installs of glibc-2.27-1.i686 and glibc-2.27-1.x32
> 
> ?

Use the --force, Luke.

I haven't find anything smarter yet.

-- 
Jan Rękorajski| PLD/Linux
SysAdm | bagginspld-linux.org | http://www.pld-linux.org/
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

libexec and multi-arch

2018-02-05 Thread Arkadiusz Miśkiewicz 

What's the approach for problems like:

error: Install/Erase problems:
file /usr/libexec/getconf/POSIX_V6_ILP32_OFFBIG conflicts between 
attempted installs of glibc-2.27-1.i686 and glibc-2.27-1.x32
file /usr/libexec/getconf/POSIX_V7_ILP32_OFFBIG conflicts between 
attempted installs of glibc-2.27-1.i686 and glibc-2.27-1.x32
file /usr/libexec/getconf/XBS5_ILP32_OFFBIG conflicts between attempted 
installs of glibc-2.27-1.i686 and glibc-2.27-1.x32

?

-- 
Arkadiusz Miśkiewicz, arekm / ( maven.pl | pld-linux.org )
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en