[Bug lto/114505] static archives built with gcc --without-zstd cause ICE when read by gcc with zstd support

2024-03-28 Thread rguenth at gcc dot gnu.org via Gcc-bugs
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=114505

--- Comment #14 from Richard Biener  ---
(In reply to Andrew Pinski from comment #4)
> For the opposite direction, this is the fix:
> ```
> diff --git a/gcc/lto-compress.cc b/gcc/lto-compress.cc
> index c167ac967aa..1475674e7ac 100644
> --- a/gcc/lto-compress.cc
> +++ b/gcc/lto-compress.cc
> @@ -408,7 +408,7 @@ lto_end_uncompression (struct lto_compression_stream
> *stream,
>  }
>  #endif
>if (compression == ZSTD)
> -internal_error ("compiler does not support ZSTD LTO compression");
> +fatal_error ("compiler does not support ZSTD LTO compression");
> 
>lto_uncompression_zlib (stream);
>  }
> ```
> 
> But this direction should work.

This looks obvious to push, btw.  And yes, this was supposed to be "supported"

[Bug lto/114505] static archives built with gcc --without-zstd cause ICE when read by gcc with zstd support

2024-03-27 Thread pinskia at gcc dot gnu.org via Gcc-bugs
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=114505

--- Comment #13 from Andrew Pinski  ---
I understand that this might be the reason why others have enabled zstd always
but reading the code this case should be supported.

That is LTO generated from a GCC without ZSTD support should be supported with
one compiled with it.

Plus:
/* Compression algorithm used for compression of LTO bytecode.  */

enum lto_compression
{
  ZLIB,
  ZSTD
};


ZSTD is 1 so it has to be something corrupting the field ...

[Bug lto/114505] static archives built with gcc --without-zstd cause ICE when read by gcc with zstd support

2024-03-27 Thread eschwartz93 at gmail dot com via Gcc-bugs
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=114505

--- Comment #12 from Eli Schwartz  ---
(In reply to Xi Ruoyao from comment #1)
> Confirmed as I've seen it before.  At least we should turn it into a
> "normal" error message instead of ICE.
> 
> But note that even "downgrading the compression algorithm" is supported, it
> won't really help rolling-release distros.  When GCC itself is upgraded with
> a LTO API version change, all previous LTO static archives will be unusable
> anyway.  Generally the distros should not ship LTO static archives at all.

Agreed, figuring out how to solve that is an open ticket too:
https://bugs.gentoo.org/926120

Still would like to solve the zstd case :D especially since I am sure given the
flexibility of a from-source distro there will be users who insist they'd
rather rebuild the entire OS on a GCC upgrade, so reducing the compression
algorithm edge case will come in use.


(In reply to Andrew Pinski from comment #2)
> On the input side:
> Wait this should not happen.
> 
>   lto_end_uncompression (stream,
>  file_data->lto_section_header.get_compression
> ());


I cannot swear this was the case but the error very specifically occurred as
part of a Gentoo profile change that caused zstd to be enabled.

The version of zstd should be the same, at least...

[Bug lto/114505] static archives built with gcc --without-zstd cause ICE when read by gcc with zstd support

2024-03-27 Thread pinskia at gcc dot gnu.org via Gcc-bugs
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=114505

--- Comment #11 from Andrew Pinski  ---
https://wiki.gentoo.org/wiki/QEMU#lto1:_internal_compiler_error:_original_not_compressed_with_zstd

[Bug lto/114505] static archives built with gcc --without-zstd cause ICE when read by gcc with zstd support

2024-03-27 Thread pinskia at gcc dot gnu.org via Gcc-bugs
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=114505

--- Comment #10 from Andrew Pinski  ---
https://www.reddit.com/r/Gentoo/comments/twtxhc/problems_compiling_gjs_1712_on_amd64/

[Bug lto/114505] static archives built with gcc --without-zstd cause ICE when read by gcc with zstd support

2024-03-27 Thread pinskia at gcc dot gnu.org via Gcc-bugs
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=114505

Andrew Pinski  changed:

   What|Removed |Added

   See Also||https://github.com/openwrt/
   ||openwrt/issues/10832

--- Comment #9 from Andrew Pinski  ---
openwrt ran into this too.

[Bug lto/114505] static archives built with gcc --without-zstd cause ICE when read by gcc with zstd support

2024-03-27 Thread pinskia at gcc dot gnu.org via Gcc-bugs
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=114505

--- Comment #8 from Andrew Pinski  ---
OE (Yocto) ran into this:
https://www.mail-archive.com/openembedded-core@lists.openembedded.org/msg158289.html

[Bug lto/114505] static archives built with gcc --without-zstd cause ICE when read by gcc with zstd support

2024-03-27 Thread pinskia at gcc dot gnu.org via Gcc-bugs
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=114505

--- Comment #7 from Andrew Pinski  ---
Something else must be going wrong here. Maybe a mistmatched version of ZSTD
which are incompatible somehow. Maybe ZSTD being miscompiled in one case.

The original ICE listed in this bug report should not happen unless there is
something wrong with ZSTD.

[Bug lto/114505] static archives built with gcc --without-zstd cause ICE when read by gcc with zstd support

2024-03-27 Thread sjames at gcc dot gnu.org via Gcc-bugs
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=114505

--- Comment #6 from Sam James  ---
I'm confident I've seen this happen before but I haven't got a quick testcase
(it should be easy to make one, but I don't have a gcc w/ zstd right this
second.)

[Bug lto/114505] static archives built with gcc --without-zstd cause ICE when read by gcc with zstd support

2024-03-27 Thread xry111 at gcc dot gnu.org via Gcc-bugs
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=114505

Xi Ruoyao  changed:

   What|Removed |Added

   Keywords||needs-reduction

--- Comment #5 from Xi Ruoyao  ---
(In reply to Andrew Pinski from comment #3)
> THis should not case should not being ICE and should be handled correctly. I
> am thinking something else is wrong.
> 
> Now the opposite direction is definitely should be a fatal_error instead of
> an internal_error.

I'm pretty sure I've seen the ICE before (when I forgot to strip the LTO
sections from a fat LTO archive built by a "GCC w/o zstd" to produce a non-LTO
archive).

Just need to find a test case...

[Bug lto/114505] static archives built with gcc --without-zstd cause ICE when read by gcc with zstd support

2024-03-27 Thread pinskia at gcc dot gnu.org via Gcc-bugs
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=114505

--- Comment #4 from Andrew Pinski  ---
For the opposite direction, this is the fix:
```
diff --git a/gcc/lto-compress.cc b/gcc/lto-compress.cc
index c167ac967aa..1475674e7ac 100644
--- a/gcc/lto-compress.cc
+++ b/gcc/lto-compress.cc
@@ -408,7 +408,7 @@ lto_end_uncompression (struct lto_compression_stream
*stream,
 }
 #endif
   if (compression == ZSTD)
-internal_error ("compiler does not support ZSTD LTO compression");
+fatal_error ("compiler does not support ZSTD LTO compression");

   lto_uncompression_zlib (stream);
 }
```

But this direction should work.

[Bug lto/114505] static archives built with gcc --without-zstd cause ICE when read by gcc with zstd support

2024-03-27 Thread pinskia at gcc dot gnu.org via Gcc-bugs
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=114505

Andrew Pinski  changed:

   What|Removed |Added

 Status|UNCONFIRMED |WAITING
   Last reconfirmed||2024-03-28
 Ever confirmed|0   |1

--- Comment #3 from Andrew Pinski  ---
THis should not case should not being ICE and should be handled correctly. I am
thinking something else is wrong.

Now the opposite direction is definitely should be a fatal_error instead of an
internal_error.

[Bug lto/114505] static archives built with gcc --without-zstd cause ICE when read by gcc with zstd support

2024-03-27 Thread pinskia at gcc dot gnu.org via Gcc-bugs
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=114505

Andrew Pinski  changed:

   What|Removed |Added

 Ever confirmed|1   |0
 Status|NEW |UNCONFIRMED
   Last reconfirmed|2024-03-28 00:00:00 |

--- Comment #2 from Andrew Pinski  ---


On the output side:
#ifdef HAVE_ZSTD_H
  lto_compression compression = ZSTD;
#else
  lto_compression compression = ZLIB;
#endif

  bool slim_object = flag_generate_lto && !flag_fat_lto_objects;
  lto_section s
= { LTO_major_version, LTO_minor_version, slim_object, 0, 0 };
  s.set_compression (compression);

(flags of lto_section is just set to compression)


On the input side:
Wait this should not happen.

  lto_end_uncompression (stream,
 file_data->lto_section_header.get_compression ());

[Bug lto/114505] static archives built with gcc --without-zstd cause ICE when read by gcc with zstd support

2024-03-27 Thread xry111 at gcc dot gnu.org via Gcc-bugs
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=114505

Xi Ruoyao  changed:

   What|Removed |Added

 Status|UNCONFIRMED |NEW
   Last reconfirmed||2024-03-28
 CC||xry111 at gcc dot gnu.org
 Ever confirmed|0   |1
   Keywords||lto

--- Comment #1 from Xi Ruoyao  ---
Confirmed as I've seen it before.  At least we should turn it into a "normal"
error message instead of ICE.

But note that even "downgrading the compression algorithm" is supported, it
won't really help rolling-release distros.  When GCC itself is upgraded with a
LTO API version change, all previous LTO static archives will be unusable
anyway.  Generally the distros should not ship LTO static archives at all.