Re: [oi-dev] GCC rebuilds

[Thomas Wagner]

On Tue, Aug 01, 2023 at 06:25:46PM +0100, Peter Tribble wrote:
> > On Tue, Aug 01, 2023 at 05:09:26PM +0200, Marcel Telka wrote:
> > > On Tue, Aug 01, 2023 at 02:38:32PM +0100, Peter Tribble wrote:
> > > > On Tue, Aug 1, 2023 at 6:41 AM Stephan Althaus <
> > > > stephan.alth...@duedinghausen.eu> wrote:
[...]
> > Placing a library in /usr/lib/ that caused version incompatibilties in the
> > past
> > and most likely will continue to do so every now an then is not the best
> > idea.
> > Despite the promised compatibility in newer versions of the runtime libs.
> > In rare cases we've seen binaries compiled with an old gcc version not
> > being
> > compatible with the latest gcc runtime libs. Especially for C++.
> >
> 
> That would be a plain and simple bug; the gcc team take binary
> compatibility very seriously,
> and actually understand things like shared library versioning properly. To
> the extent that we
> have had a forward-compatible libstdc++ that manages to cleanly handle the
> fact that the
> C++ ABI itself changed (leading to the library transparently handling the
> dual ABI from gcc
> 5.1 onwards) along with multiple versions of the C++ language since about
> GCC 3.4.
 
One thing is that gcc didn't get the promise delivered properly
in the past. The other thing is the forced recompile of all packages,
if the gcc compiler changes.

I've seen troubles with a OS provided gcc runtime since 2007.
Therefore the SFE packaging project got the best and most stable
result by not loading the gcc runtime from /usr/lib 
- instead from a compiled in path in /usr/gcc*/n.n/lib (LINK_LIBGCC_SPEC)
and that path correctly delivered by the matching gcc runtime lib 
packages of that version. 

Sure there are several solutions to that problem. I prefer the SFE style.


Regards
Thomas

> Therefore the SFE packaging project points libs and binaries to a
> > versioned directory to get the version of runtime libs loaded they have
> > been compiled with.
> > e.g. binaries look first in /usr/gcc-sfe/4.9/lib and /usr/gcc-sfe/11/lib
> > for the runtime libs in an early stage.
> >
> > Regards
> > Thomas

___
oi-dev mailing list
oi-dev@openindiana.org
https://openindiana.org/mailman/listinfo/oi-dev

Re: [oi-dev] GCC rebuilds

[Till Wegmüller]

This is what it was in the past and what is currently achieved by 
mediating links to /usr/lib. We had to change it because we could not 
resolve the openssl version problem with a little enough effort for us 
to achieve. So the proble is not the runtime changing places its 
application being compiled explicitly against gcc-7 in the outside of 
/usr/lib location. And somehow loading in the gcc7 runtime. So the only 
thing left to do is to bump the packages and try to fix them until it is 
fixed. Unfortunately with this specific version of virtualbox we the 
maintainers cannot test as we are missing hardware. So for some reason 
virtualbox and some other libs are still missing and it would be 
wonderfull if some people could bump the Revision if they find such 
libraries.


-Till

On 01.08.23 19:25, Peter Tribble wrote:



On Tue, Aug 1, 2023 at 5:28 PM Thomas Wagner > wrote:


On Tue, Aug 01, 2023 at 05:09:26PM +0200, Marcel Telka wrote:
 > On Tue, Aug 01, 2023 at 02:38:32PM +0100, Peter Tribble wrote:
 > > On Tue, Aug 1, 2023 at 6:41 AM Stephan Althaus <
 > > stephan.alth...@duedinghausen.eu
> wrote:
 > > > We are stumbling over some faults with regard to the GCC
Version change.
 > >
 > > Perhaps this would be an opportune moment to reconsider the way
that
 > > libstdc++
 > > (and generally the whole gcc/g++ runtime) is packaged, and to
go for the
 > > obvious
 > > and supported route of only shipping one copy of the runtime -
the one
 > > corresponding to
 > > the latest version of the compiler that you ship (gcc11 ?), and
putting it
 > > directly in
 > > /usr/lib.
 >
 > The obvious question now is:
 > Why it was not done that way since beginning?

Placing a library in /usr/lib/ that caused version incompatibilties
in the past
and most likely will continue to do so every now an then is not the
best idea.
Despite the promised compatibility in newer versions of the runtime
libs.
In rare cases we've seen binaries compiled with an old gcc version
not being
compatible with the latest gcc runtime libs. Especially for C++.


That would be a plain and simple bug; the gcc team take binary 
compatibility very seriously,
and actually understand things like shared library versioning properly. 
To the extent that we
have had a forward-compatible libstdc++ that manages to cleanly handle 
the fact that the
C++ ABI itself changed (leading to the library transparently handling 
the dual ABI from gcc
5.1 onwards) along with multiple versions of the C++ language since 
about GCC 3.4.


Therefore the SFE packaging project points libs and binaries to a
versioned directory to get the version of runtime libs loaded they have
been compiled with.
e.g. binaries look first in /usr/gcc-sfe/4.9/lib and /usr/gcc-sfe/11/lib
for the runtime libs in an early stage.

Regards
Thomas


___
oi-dev mailing list
oi-dev@openindiana.org 
https://openindiana.org/mailman/listinfo/oi-dev




--
-Peter Tribble
http://www.petertribble.co.uk/  - 
http://ptribble.blogspot.com/ 


___
oi-dev mailing list
oi-dev@openindiana.org
https://openindiana.org/mailman/listinfo/oi-dev


___
oi-dev mailing list
oi-dev@openindiana.org
https://openindiana.org/mailman/listinfo/oi-dev

Re: [oi-dev] GCC rebuilds

[Peter Tribble]

On Tue, Aug 1, 2023 at 5:28 PM Thomas Wagner 
wrote:

> On Tue, Aug 01, 2023 at 05:09:26PM +0200, Marcel Telka wrote:
> > On Tue, Aug 01, 2023 at 02:38:32PM +0100, Peter Tribble wrote:
> > > On Tue, Aug 1, 2023 at 6:41 AM Stephan Althaus <
> > > stephan.alth...@duedinghausen.eu> wrote:
> > > > We are stumbling over some faults with regard to the GCC Version
> change.
> > >
> > > Perhaps this would be an opportune moment to reconsider the way that
> > > libstdc++
> > > (and generally the whole gcc/g++ runtime) is packaged, and to go for
> the
> > > obvious
> > > and supported route of only shipping one copy of the runtime - the one
> > > corresponding to
> > > the latest version of the compiler that you ship (gcc11 ?), and
> putting it
> > > directly in
> > > /usr/lib.
> >
> > The obvious question now is:
> > Why it was not done that way since beginning?
>
> Placing a library in /usr/lib/ that caused version incompatibilties in the
> past
> and most likely will continue to do so every now an then is not the best
> idea.
> Despite the promised compatibility in newer versions of the runtime libs.
> In rare cases we've seen binaries compiled with an old gcc version not
> being
> compatible with the latest gcc runtime libs. Especially for C++.
>

That would be a plain and simple bug; the gcc team take binary
compatibility very seriously,
and actually understand things like shared library versioning properly. To
the extent that we
have had a forward-compatible libstdc++ that manages to cleanly handle the
fact that the
C++ ABI itself changed (leading to the library transparently handling the
dual ABI from gcc
5.1 onwards) along with multiple versions of the C++ language since about
GCC 3.4.

Therefore the SFE packaging project points libs and binaries to a
> versioned directory to get the version of runtime libs loaded they have
> been compiled with.
> e.g. binaries look first in /usr/gcc-sfe/4.9/lib and /usr/gcc-sfe/11/lib
> for the runtime libs in an early stage.
>
> Regards
> Thomas
>
>
> ___
> oi-dev mailing list
> oi-dev@openindiana.org
> https://openindiana.org/mailman/listinfo/oi-dev
>


-- 
-Peter Tribble
http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/
___
oi-dev mailing list
oi-dev@openindiana.org
https://openindiana.org/mailman/listinfo/oi-dev

Re: [oi-dev] GCC rebuilds

[Thomas Wagner]

On Tue, Aug 01, 2023 at 05:09:26PM +0200, Marcel Telka wrote:
> On Tue, Aug 01, 2023 at 02:38:32PM +0100, Peter Tribble wrote:
> > On Tue, Aug 1, 2023 at 6:41 AM Stephan Althaus <
> > stephan.alth...@duedinghausen.eu> wrote:
> > > We are stumbling over some faults with regard to the GCC Version change.
> > 
> > Perhaps this would be an opportune moment to reconsider the way that
> > libstdc++
> > (and generally the whole gcc/g++ runtime) is packaged, and to go for the
> > obvious
> > and supported route of only shipping one copy of the runtime - the one
> > corresponding to
> > the latest version of the compiler that you ship (gcc11 ?), and putting it
> > directly in
> > /usr/lib.
> 
> The obvious question now is:
> Why it was not done that way since beginning?

Placing a library in /usr/lib/ that caused version incompatibilties in the past
and most likely will continue to do so every now an then is not the best idea.
Despite the promised compatibility in newer versions of the runtime libs.
In rare cases we've seen binaries compiled with an old gcc version not being
compatible with the latest gcc runtime libs. Especially for C++.

Therefore the SFE packaging project points libs and binaries to a
versioned directory to get the version of runtime libs loaded they have
been compiled with.
e.g. binaries look first in /usr/gcc-sfe/4.9/lib and /usr/gcc-sfe/11/lib
for the runtime libs in an early stage.

Regards
Thomas


___
oi-dev mailing list
oi-dev@openindiana.org
https://openindiana.org/mailman/listinfo/oi-dev

Re: [oi-dev] GCC rebuilds

[Marcel Telka]

On Tue, Aug 01, 2023 at 02:38:32PM +0100, Peter Tribble wrote:
> On Tue, Aug 1, 2023 at 6:41 AM Stephan Althaus <
> stephan.alth...@duedinghausen.eu> wrote:
> > We are stumbling over some faults with regard to the GCC Version change.
> 
> Perhaps this would be an opportune moment to reconsider the way that
> libstdc++
> (and generally the whole gcc/g++ runtime) is packaged, and to go for the
> obvious
> and supported route of only shipping one copy of the runtime - the one
> corresponding to
> the latest version of the compiler that you ship (gcc11 ?), and putting it
> directly in
> /usr/lib.

The obvious question now is:
Why it was not done that way since beginning?

-- 
+---+
| Marcel Telka   e-mail:   mar...@telka.sk  |
|homepage: http://telka.sk/ |
+---+

___
oi-dev mailing list
oi-dev@openindiana.org
https://openindiana.org/mailman/listinfo/oi-dev

Re: [oi-dev] GCC rebuilds

[Peter Tribble]

On Tue, Aug 1, 2023 at 6:41 AM Stephan Althaus <
stephan.alth...@duedinghausen.eu> wrote:

> Hello!
>
> We are stumbling over some faults with regard to the GCC Version change.
>

Perhaps this would be an opportune moment to reconsider the way that
libstdc++
(and generally the whole gcc/g++ runtime) is packaged, and to go for the
obvious
and supported route of only shipping one copy of the runtime - the one
corresponding to
the latest version of the compiler that you ship (gcc11 ?), and putting it
directly in
/usr/lib.

To get there would involve rebuilding the versioned runtime packages (make
them empty)
and creating an unversioned runtime package that they all depend on, but
would obviate the
need to rebuild any of the consumers, and would largely eliminate any pain
in future.

-- 
-Peter Tribble
http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/
___
oi-dev mailing list
oi-dev@openindiana.org
https://openindiana.org/mailman/listinfo/oi-dev

Re: [oi-dev] GCC rebuilds

[Udo Grabowski (IMK)]

On 01/08/2023 10:59, Stephan Althaus wrote:

On 8/1/23 10:15, Till Wegmüller wrote:

Hi Stephan

Your idea is sound. The pkg file next to a Makefile in the same directory
indicate a component.

The pkg file contains a json with all required packages so the grep trick
works, you can also use jq to filter out the json syntax cleanly.

The rebuild is one thing we should do yes, so if you can bump the version
numbers I can run it through CI.

It's a good idea! keep going.

-Till

On 01.08.23 07:40, Stephan Althaus wrote:

Hello!

We are stumbling over some faults with regard to the GCC Version change.

i had the idea of a half-automation of these steps:
- find packages that need a rebuild
- patch Makefile and pkg5
- git checkout -b , git add, git commit, etc..

How to identify the appropriate packages ?
When i select packages by " (grep gcc-7-runtime" pkg) == true " i get some
candidates..

-> Do we have to rebuild all other packages that have been build prior to the
commit "switch default gcc to version 10"
because of the "ABI change" ??

That would be >200 packages..

Or is my idea "stupid" in the end?

Regards,
Stephan



___
oi-dev mailing list
oi-dev@openindiana.org
https://openindiana.org/mailman/listinfo/oi-dev


___
oi-dev mailing list
oi-dev@openindiana.org
https://openindiana.org/mailman/listinfo/oi-dev


Hello!

Thanks for your response!

The one thing left open in my mind,
does the "ABI change" indeed change the function calls in a way,
so that in situations like

someprog_compiled_with_gcc10 -> somelib-compiled-with-gcc7->func() ->
somelib-compiled-with-gcc10->func() ...

or-the-like are broken?

In this case we have to rebuild all packages that have been build before
december-2022.



I think this can lead to errors, when a function from the libs is
called from the gcc-10 compiled package after the gcc-7 compiled library
has been loaded down the chain (probably before the next gcc-10 compiled
library is loaded further down...). The possible f...up situations
here are probably uncountable. The top priority though should go to those
programs left that load both gcc7 and 10 themselves, if there are
still any.
--
Dr.Udo Grabowski   Inst.f.Meteorology & Climate Research IMK-ASF-SAT
http://www.imk-asf.kit.edu/english/sat.php
KIT - Karlsruhe Institute of Technology   http://www.kit.edu
Postfach 3640,76021 Karlsruhe,Germany T:(+49)721 608-26026 F:-926026



smime.p7s
Description: S/MIME Cryptographic Signature
___
oi-dev mailing list
oi-dev@openindiana.org
https://openindiana.org/mailman/listinfo/oi-dev

Re: [oi-dev] GCC rebuilds

[Stephan Althaus]

On 8/1/23 10:15, Till Wegmüller wrote:

Hi Stephan

Your idea is sound. The pkg file next to a Makefile in the same 
directory indicate a component.


The pkg file contains a json with all required packages so the grep 
trick works, you can also use jq to filter out the json syntax cleanly.


The rebuild is one thing we should do yes, so if you can bump the 
version numbers I can run it through CI.


It's a good idea! keep going.

-Till

On 01.08.23 07:40, Stephan Althaus wrote:

Hello!

We are stumbling over some faults with regard to the GCC Version change.

i had the idea of a half-automation of these steps:
- find packages that need a rebuild
- patch Makefile and pkg5
- git checkout -b , git add, git commit, etc..

How to identify the appropriate packages ?
When i select packages by " (grep gcc-7-runtime" pkg) == true " i get 
some candidates..


-> Do we have to rebuild all other packages that have been build 
prior to the commit "switch default gcc to version 10"

because of the "ABI change" ??

That would be >200 packages..

Or is my idea "stupid" in the end?

Regards,
Stephan



___
oi-dev mailing list
oi-dev@openindiana.org
https://openindiana.org/mailman/listinfo/oi-dev


___
oi-dev mailing list
oi-dev@openindiana.org
https://openindiana.org/mailman/listinfo/oi-dev


Hello!

Thanks for your response!

The one thing left open in my mind,
does the "ABI change" indeed change the function calls in a way,
so that in situations like

someprog_compiled_with_gcc10 -> somelib-compiled-with-gcc7->func() -> 
somelib-compiled-with-gcc10->func() ...


or-the-like are broken?

In this case we have to rebuild all packages that have been build before 
december-2022.


Regards,
Stephan



___
oi-dev mailing list
oi-dev@openindiana.org
https://openindiana.org/mailman/listinfo/oi-dev

Re: [oi-dev] GCC rebuilds

[Till Wegmüller]

Hi Stephan

Your idea is sound. The pkg file next to a Makefile in the same 
directory indicate a component.


The pkg file contains a json with all required packages so the grep 
trick works, you can also use jq to filter out the json syntax cleanly.


The rebuild is one thing we should do yes, so if you can bump the 
version numbers I can run it through CI.


It's a good idea! keep going.

-Till

On 01.08.23 07:40, Stephan Althaus wrote:

Hello!

We are stumbling over some faults with regard to the GCC Version change.

i had the idea of a half-automation of these steps:
- find packages that need a rebuild
- patch Makefile and pkg5
- git checkout -b , git add, git commit, etc..

How to identify the appropriate packages ?
When i select packages by " (grep gcc-7-runtime" pkg) == true " i get 
some candidates..


-> Do we have to rebuild all other packages that have been build prior 
to the commit "switch default gcc to version 10"

because of the "ABI change" ??

That would be >200 packages..

Or is my idea "stupid" in the end?

Regards,
Stephan



___
oi-dev mailing list
oi-dev@openindiana.org
https://openindiana.org/mailman/listinfo/oi-dev


___
oi-dev mailing list
oi-dev@openindiana.org
https://openindiana.org/mailman/listinfo/oi-dev