Red Hat & Fedora -- largely stepping out of this ecosystem

[Jeff Law]

I have no illusions that this message is going to change anything at Red 
Hat.  But I feel I need to get this off my chest.  I'm speaking strictly 
for myself, not for my current or any former employer, not for the GCC 
project and not for any other group/organization I might have some 
affiliation with.


--


First a bit of background.  I came to Red Hat via the Cygnus 
acquisition.  My combined service spanned nearly 30 years.  During that 
time I had the opportunity to be involved in the formation of Fedora, 
strategic planning for RHEL while at the same time being able to spend 
much of my time on optimizing compilers.


I left Red Hat in 2021 to refocus on what I really enjoy in a small 
company where I can make a clear difference in the company's direction 
and success.  It was, by far, the most difficult decision in my 
professional career.  I left many friends and colleagues behind.


The point being I had a fantastic career at Cygnus/Red Hat.  I got to do 
and learn things I never could have imagined.  I got to work with many 
amazing people spanning many organizations within the company 
(engineering, sales, marketing, legal, product management, executives, 
etc).   By no means do I consider myself a disgruntled former employee.


--



What Red Hat has done recently is depressing, but not a huge surprise to 
me.  Red Hat struggled repeatedly with how to deal with "the clones". 
The core idea I always came back to in those discussions was that the 
value isn't in the bits, but in the stability, services and ecosystem 
Red Hat enables around the bits.  Arguments for protecting the bits were 
met with something like "if that's what we need to do to be successful, 
then we're failing to provide real value".


At some point in the last 20 years (I forget exactly when) Red Hat was 
looking to codify its values.  Naturally the topic of open source came 
up during those discussions.   When open source didn't make the cut, one 
could say the writing was on the wall -- open source was a means to an 
end.  In my mind that opened the door for numerous changes we've seen in 
subsequent years.


One could see signs of where this was going with the adjustments that 
were made to the exposed RHEL kernel sources some time ago.  Then the 
dissolution of CentOS a couple years back and most recently with the 
lockdown of the RHEL sources.


What Red Hat has done may be technically legal and perhaps good for its 
business.  However, to me it's ethically unconscionable.   Those who 
know me know I'm not an zealot, but I do have a baseline set of ethical 
values and Red Hat crossed that line.



--


Back in 2002 or 2003 when we were trying to figure out how to salvage 
the ill-fated "Red Hat Community Linux Project" resulting in what we now 
 know as Fedora, one of the key concepts that we pushed to the 
executive team at Red Hat was that it was strategically important for 
both Fedora and Red Hat to have a strong association with each other.  I 
think we largely succeeded in building that close association.



I've been a Fedora user since before FC1.  I run Fedora on my laptop. 
When I need a docker image for something, I start with a Fedora image. 
When I need to spin up a server (say to run the GCC CI/CD system) I load 
it with Fedora.   It's an ecosystem I'm technically comfortable in and 
easiest for me to utilize.


That will change across the board this summer.  That's a bit hard for me 
to swallow, but I can't get past that association we built between Red 
Hat and Fedora and Red Hat's recent actions.


I'll still have to deal with the RHEL/CentOS/Fedora ecosystem on a 
professional level.  Obviously, I'll do what I need to do to help make 
my employer successful -- but when a choice exists, Fedora/CentOS/RHEL 
won't be where I land going forward.


I wish it hadn't come to this, but I also can't say I'm terribly surprised.

Thanks for your time,
Jeff





___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: subarchitectures and RISC-V, was Re: F39 proposal: RPM 4.19 (System-Wide Change proposal)

[Jeff Law]

On 4/19/23 00:26, Benson Muite wrote:




Probably each hardware vendor will need to become more involved in
creating an RPM distribution for their use case and providing hardware
for test builds.  A single monolithic Fedora will not work.  Having a
subset of base packages would be very helpful.

The rpm subarches might be a useful path, but then determining what you
build for multiple of them and how needs to be addressed.


The great thing about Linux is easy customisability.  There may be a few
RISC V variants that are widely used, but not clear which at present.
The D1 chip is very affordable, and can have much use in education and
IoT, though most support at present is available for Debian and OpenWRT.
I certainly don't see Ventana taking that direction -- the market we've 
targeted isn't interested in a custom distro.  In fact, they're very 
much of the mindset of using an existing distro.


And seriously, who wants to recreate all the infrastructure necessary to 
build and support a system like Fedora or Ubuntu.  That's a major 
investment with marginal value.


jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: subarchitectures and RISC-V, was Re: F39 proposal: RPM 4.19 (System-Wide Change proposal)

[Jeff Law]

On 4/19/23 07:46, Florian Weimer wrote:

* Jeff Law:


Rather than trying to track all the individual extensions and
combinations thereof, I would suggest focusing on RVI defined
profiles. Essentially they provide a set of mandatory features the
architecture must support (to be compliant with the profile).


Do at least some of these profiles form an ascending chain?
That's the plan as I understand it.  *But* there are some chip vendors 
pushing back against some aspects of the profile plan, say for example 
requiring the vector extension.  At least that's what I'm hearing 
through the grapevine.


If it were up to me, I'd let them go their own way.  If they don't see 
the value in conforming to a profile, then that's a choice they can 
make, but it'll restrict their ability to leverage all the work done at 
the distro level.


Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: subarchitectures and RISC-V, was Re: F39 proposal: RPM 4.19 (System-Wide Change proposal)

[Jeff Law]

On 4/15/23 00:25, David Abdurachmanov wrote:

On Sat, Apr 15, 2023 at 7:08 AM Jeff Law  wrote:




On 4/14/23 20:14, Neal Gompa wrote:





We should not screw up with RISC-V in Fedora like RHEL did with ARM.
Yes, I'm saying RHEL's ARM strategy was a mistake, and still is, to
some degree. We see aspects of this being walked back now as the
ecosystem didn't go the way RHEL ARM folks hoped, and breaking further
into that ecosystem requires reversing some of those choices. We
should learn from that for Fedora RISC-V.

Well, the RHEL direction was essentially mandated by the markets vendors
were chasing.  Plain and simple.  You may say RHEL's ARM strategy was a
mistake, but I'd disagree strongly.


Fedora != CentOS Stream or/and RHEL.
I am *well* aware.  I wouldn't have brought RHEL into the discussion at 
all if Neal hadn't ;-)




I am gonna repeat myself. I doubt there will be a need to support
anything below RVA23 in CentOS Stream. That of course would mean that
existing (or near future, yet to be announced) SBCs wouldn't be able
to run it.
Agreed.  It's hard to see a case where anyone that wants Centos on 
RVA20.  RVA23 is a much more realistic target.





That's already happened.  Stratification was inevitable given the RISC-V
model.  The only questions in my mind are viability in the server space
and whether or not that could one day lead to offerings between server
class and the SBC systems.


Brings high-performance (and yet cheap) SBCs that it's fully compliant
is too expensive today (i.e. you would lose money doing it [personal
opinion]). It's going to happen, it's most likely coming from the
Chinese market [personal opinion].

You're probably right on all those points.




I could say that Ubuntu is the dominating distribution for RISCV SBCs.
Canonical engineering resources and willingness to support pretty much
every SBC (but with vendor kernel or/and firmware) is really hard to
compete with. If you want to get the best out-of-the-box experience on
your RISCV SBCs definitely Ubuntu is #1 suggestion, but that wasn't
the case for many years. Ubuntu came to RISCV land very late (I would
say somewhat recently), but now should be dominating (no way to
confirm). Nice work by the Ubuntu community and Canonical engineers.
They truly want to support every piece of hardware available out
there.
Agreed.  And one can certainly question the sanity and business model of 
supporting all those SBCs.  But if you look beyond the SBC space, Ubuntu 
has positioned themselves well to be the distro of choice for aspiring 
server platforms.  One might even look at the SBC engagement as really 
just positioning themselves for the future (speculation on my part).


Ubuntu may have come late, but they're engaged at a level that the 
Fedora community isn't even close to matching.


Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: subarchitectures and RISC-V, was Re: F39 proposal: RPM 4.19 (System-Wide Change proposal)

[Jeff Law]

On 4/15/23 00:10, David Abdurachmanov wrote:




We have to support SCBs as-is. We even have 64-core OoO (and even
dual-socket 128-core) systems coming that are still RVA20 (what I call
"a large scale SBC trying to be a server").
I think elsewhere you suggested treating the profile as the subarch for 
RPM.  That may be a viable way to handle the SBC space.


I still worry about supporting multiple profiles and would like to see a 
clear path to deprecating profiles that are out of date.  The amount of 
pain I've seen in both the x86 and aarch worlds WRT baselines is 
something I'm keen to avoid repeating.





But Fedora RISCV as-is is not how future Fedora RISCV should be. The
future is RVA23 (or beyond) and standard compliance. There will be
significant performance differences between profiles for some time.
Especially RVA20 -> RVA23. RVA20 (or RVA64GC) is just way too small
with lack of modern ISA. We have had a toy for ~10 years now, and now
we are moving toward high-performance servers, even HPC, Android (see
latest Google announcements), etc. That's not driven by RVA20. That's
RVA23 (and beyond) + vendor extensions.






Yes! Blocked by lack of "hw probe" interface. There will be "hw probe"
syscall for RISCV, in v6.4 and v6.5 kernel (I hope). We don't have
cpuid instruction, HWCAP is way too small for all the extensions
available.
I'm sure folks will get this sorted out.  My only concern with the 
syscall was to make sure the glibc folks were on board with using that 
to drive ifunc selection.  It's a fairly different approach than has 
been taken in the past and I want to make sure it's not DOA for glibc.


Once the kernel & glibc teams reach API agreement I think we'll be able 
to make a reasonable query about the system properties in multiple 
contexts, including the installer, rpm, etc.







Note that toolchains don't accept RISCV Profiles yet, but that has
been under discussion for quite some time already. I would assume
starting GCC 14 timeframe all of that should work.
It'll get sorted out.  We've got some bigger fish to fry WRT landing V 
support (if you've managed to avoid that mess, consider yourself lucky).


Lots of stuff is backed up behind getting gcc-13 out the door so that 
gcc-14 development can open up.   Not sure where profiles will fall into 
the priority list, it's hard to see past the V effort right now.


jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: subarchitectures and RISC-V, was Re: F39 proposal: RPM 4.19 (System-Wide Change proposal)

[Jeff Law]

On 4/14/23 20:14, Neal Gompa wrote:





We should not screw up with RISC-V in Fedora like RHEL did with ARM.
Yes, I'm saying RHEL's ARM strategy was a mistake, and still is, to
some degree. We see aspects of this being walked back now as the
ecosystem didn't go the way RHEL ARM folks hoped, and breaking further
into that ecosystem requires reversing some of those choices. We
should learn from that for Fedora RISC-V.
Well, the RHEL direction was essentially mandated by the markets vendors 
were chasing.  Plain and simple.  You may say RHEL's ARM strategy was a 
mistake, but I'd disagree strongly.




Like ARM, RISC-V risks (pun intended!) total stratification between
low/mid range SBCs and unobtanium servers. Given the current path of the
groups working in RISC-V, this is certain to happen. Thus, if Fedora
RISC-V cannot run well on RISC-V SBCs, then we effectively have no
useful RISC-V port, since nobody can use it.
That's already happened.  Stratification was inevitable given the RISC-V 
model.  The only questions in my mind are viability in the server space 
and whether or not that could one day lead to offerings between server 
class and the SBC systems.





We already have similar problems with POWER (ppc64le) and Z (s390x),
but the former was built in an era when there were computers of
reasonable performance across all price points. The latter is
basically funded by IBM development and nobody can really do anything
with it.
The era where POWER was a viable platform outside the server space has 
been gone for a long time.  x86 just killed the market and it's not 
clear there's space for anyone else in that market.  Apple may prove me 
wrong in that space, but the investment they've made is enormous.





For the first time in a long time, Fedora has the opportunity to have
a first-mover advantage and become the default platform for hobbyist,
professional, and high-end systems of an architecture. Let's not
squander it on myopic views of datacenter class hardware that don't
exist yet leveraging specifications that nobody is implementing in
currently available hardware.
First mover advantage is already lost to Ubuntu.  Sad but true in my 
mind.  Fedora has zero presence in the spaces that matter.





Insofar as subarches go, let's just define them in RPM like we have
for 32-bit x86 and more recently 64-bit x86. If we know what kind of
ISA profiles are out there, we should just incorporate them into RPM
and set up the compatibility detection logic for them to work.
The current and future crop of SBCs just aren't going to have the oomph 
to be a viable platform in my mind.  I can take a 10+ year old xeon run 
qemu on top of it and get dramatically better risc-v performance than 
the SBCs out there


Creating subarch around something like rv20, go ahead.  But it's going 
to be worse than the armv7 situation was.


jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: subarchitectures and RISC-V, was Re: F39 proposal: RPM 4.19 (System-Wide Change proposal)

[Jeff Law]

On 4/12/23 10:08, przemek klosowski via devel wrote:


That may rule out certain processors, but it ultimately provides a 
higher performing baseline architecture for systems that are 
(hopefully) going to be good performing parts rather than embedded 
focused parts.


Yes, good point, but there's already a number of profiles even though 
they are barely out of the gate:


https://github.com/riscv/riscv-profiles/blob/main/profiles.adoc
I know.  While I'm not involved in setting the profiles, I do get 
briefed on them reasonably often.




I of course agree with you that it makes sense to focus support on a 
small number of existing platforms with good reputation and performance 
(for instance both VisionFive and Pine64 are based on StarFive JH7110 SoC).
Those are not particularly interesting in my mind for a distro target. 
If it can't run a performant system I'd put on my desk or in my rack, 
then it's an embedded target in my mind (yes, I'm over-generalizing). 
There's a place for those, but I don't think that should be Fedora's 
focus for RISC-V.


Full disclosure.  I work for Ventana and have a long history with Red 
Hat and Fedora.  My vision for Fedora going back before it was actually 
launched as for a desktop/developer focused distribution similar to RHL, 
but free --  and as a feeder distribution into what was then known as 
Advanced Server, now RHEL.





Still, it would be neat if there was a good technical solution for 
subarchitecture diversity because there will be more of it in the 
future. Jim Keller, the prolific CPU designer who worked on DEC Alpha, 
AMD K7 and Zen, Intel, Tesla, and Apple, has an interesting talk where 
he justifies going to RISC-V architecture because it is so much easier 
to extend it for high performance on specific tasks:
The fragmentation will kill RISC-V from a distro standpoint if it's not 
brought under control -- and I had that position well before I joined 
the RISC-V world.   Profiles are a step, but by no means a complete 
solution.   Distros are terrible at supporting ISA customizations.


heff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: subarchitectures and RISC-V, was Re: F39 proposal: RPM 4.19 (System-Wide Change proposal)

[Jeff Law]

On 4/12/23 10:57, David Abdurachmanov wrote:



We have been focusing and building for RV64GC, which is kinda
represented by the RVA20 profile. RVA20 is considered a major profile,
but it significantly lacks modern ISA extensions. There is also RVA22,
which is considered a "minor" profile. The next "major" one will be
RVA23. RVA23 will be a true start for RISCV (especially entering the
high-performance market).


My sense is RVA23 is likely also going to be considered a minor profile, 
for better or worse.  RVA24 might be a better target.




 Profiles are important for the binaries we

ship, but there is more. The most top-level specification that we will
target is RISCV Platforms (defines also non-ISA stuff), which is
currently blocked by a number of other WIP specifications. Right now
the base for that is RVA22, but I would be surprised that the final
version of it will require RVA23 (just a speculation at this point).

Thus in the future we want to support platforms that properly
implement and comply with RISCV Platforms specifications. There will
probably be a compliance test suite at some point too.

All the SoCs/SBCs available (or soon to be available on the market)
are still RVA20 / RV64GC + not-ratified versions of RVI extension
or/and vendor extensions.
Yea, but I'm not sure we want to focus on the SBCs.  They're woefully 
underpowered and we'll end up leaving a notable amount of performance on 
the table for the beefier systems that are coming online.  RVA22 would 
be a better target in my mind.


WRT compliance tests.  Those are still pretty weak at this point IMHO, 
and while I see signs of improvement, it's agonizingly slow.  I wouldn't 
expect much in that space in the timeframes that matter for the 
decisions we need to be making.






My preference would be to (in short):
- Do not change the baseline. That is "riscv64" is RVA20 / RV64GC.
This means that existing hardware is supported for a good amount of
time.
- Introduce RVA20 / RVA22 / RVA23 arch strings (blocked by "hw probe"
syscall landing in Linux kernel [most likely 6.4 or 6.5 kernel]).
- Skip RVA22 as it's a stop-gap solution and considered as a "minor"
ISA profile.
- Profile a full RVA23 Fedora/RISCV rebuild (i.e. different arch repository).
- Review policies for 3rd party repositories (i.e. vendor
repositories)., DNF, vendor lock, consider enabling vendor lock by
default (already supported by DNF in most commands, but vendor lock is
not on by default).
That's not too bad.  Essentially you're suggesting we stick to RVA20 
until RVA23 is available, then we make the switch.  I could support 
that, though I fear the screaming when we do make the switch to RVA23 
and leave some folks behind.





Here is a thing. You don't want to run RVA20 binaries on top of RVA23
hardware. You most likely will lose tons of performance. It will not
be something like running x86_64 baseline on top of x86_64-{v3,v4}
hardware where applications get 3%, 5%, 10% in some very specific
workloads a bit more. IIRC bit-manip alone can deliver significant
performance improvements. I think there are ~130 extensions right now,
and there are another ~50 in the pipeline right now.
Exactly.  And while there may be ~130 extensions, many just don't 
matter.  Don't tell the relevant folks I said that ;-)





We can use glibc-hwcaps, and have a small predefined list of packages
(probably max a few hundreds), but that doesn't apply to the binaries,
just loadable libraries.
The set should be *very* small.  glibc and perhaps openssl are worth 
implementing dynamic dispatch.  The rest I wouldn't bother.  The QE 
overhead of testing gets out of hand quickly.  We're better off getting 
to a good profile to set the floor at a reasonable place.


RVA20 isn't that reasonable place.  One could even claim that anything 
without V isn't reasonable in the modern world.






jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: subarchitectures and RISC-V, was Re: F39 proposal: RPM 4.19 (System-Wide Change proposal)

[Jeff Law]

On 4/11/23 19:14, przemek klosowski via devel wrote:

On 4/4/23 10:28, Dan Čermák wrote:

Chris Adams  writes:


Yeah, it'd be back to the i386/i586/i686 days... which was a bit of a
PITA sometimes.  But cramming multiple architectures of core libraries
into a single RPM would be bad for disk space, image size, downloads,
etc.

But something that didn't exist in the i386/i686 days is containers -
whether base images like for podman or full things like Flatpaks.
Before going too deep into multi-level architectures, that should be
taken into account.

Afaik at least container runtimes do not support really support x86_64
subarchitectures: https://github.com/containers/podman/discussions/15256


The situation in the RISC-V universe is even more complicated because of 
all the extensions


https://en.wikichip.org/wiki/risc-v/standard_extensions

It'll be challenging to write and package software that is portable 
between all those variants---the fat binaries are just not practical due 
to combinatorial complexity, so it'll have to be either install-time or 
link-time configuration.


Whatever standard scheme Fedora uses for x86 will hopefully be very 
useful for RiSC-V era that is apparently coming, with Linux-capable SBC 
boards like VisionFive ($65)  and Pine64 ($70 and $8 (!) ox64) just 
becoming available, and a lot of activity on the horizon.
Rather than trying to track all the individual extensions and 
combinations thereof, I would suggest focusing on RVI defined profiles. 
Essentially they provide a set of mandatory features the architecture 
must support (to be compliant with the profile).


That may rule out certain processors, but it ultimately provides a 
higher performing baseline architecture for systems that are (hopefully) 
going to be good performing parts rather than embedded focused parts.


jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Risc-V SIG?

[Jeff Law]

On 2/9/23 08:49, David Abdurachmanov wrote:


There are two generic issues in Fedora that keep popping up:
- Wrong assumption about valgrind. Those are easy to fix, but are in a
number of places. Note that valgrind upstreaming will happen
relatively soonish too.
Yea, but I don't have a high degree of confidence in that code.  I was 
looking at it a couple weeks ago and it's just not in that good of 
shape.I certainly hope they're able to fix things and get it working 
on real code rather than just trivial tests as valgrind is the biggest 
missing piece in the tools world as I narrowly define it :-)






There is also an issue in how sub-word atomics are being handled in
riscv64 today. This basically means that packages like to fail
building somewhat sporadically. Basically you must link to libatomic
and GCC SPEC will do that if -pthread is used (this covers most of the
cases, but not all). Also a number of packages don't use -pthread and
just -lpthread. There are patches to improve this, but that's more
likely GCC 14 material now. A quick "fix" (not the best) would be
remove -pthread check and let GCC to attempt linking to libatomic in
all cases (--as-needed -latomic).
subword atomics are a pain point.  RIght or wrong, they got pulled into 
the largeer atomics issues in the risc-v space.   Basically neither GCC 
nor LLVM actually implement the atomics correctly and there's some 
belief that we need to fix that first, then we can add the inline 
subword atomics.


Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Fedora 38 mass rebuild is finished

[Jeff Law]

On 1/24/23 15:52, Gary Buhrmaster wrote:



Had I seen cstdint I like to think that I would have
tried a rebuild with gcc 13 earlier to see what
(if any) upstream(s) needed some encouragement
for support gcc 13.
I actually did stuff like this in the past -- build all of Fedora with 
the new compiler as early as possible and then proceeded to fix as many 
of these trivial things as I could.


It was mostly done to keep the noise down in the tester so that the vast 
majority of the failures were actually compiler issues the team needed 
to address and to find deeper issues earlier in the cycle.


I've since moved on from Red Hat and don't even worry about any of the 
significant Fedora platforms anymore (I'm in risc-v land).  The mass 
rebuild testing I did was looking for something completely different, 
hence I didn't get through analysis of generic build failures as I was 
focused on a specific issue.


jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Fedora 38 mass rebuild is finished

[Jeff Law]

On 1/24/23 00:16, Jakub Jelinek wrote:

On Tue, Jan 24, 2023 at 10:00:47AM +0300, Vascom wrote:

I have some packages failed.
One of them libtins. Problem is that:

error: 'uint32_t' is not a member of 'std';

Is it normal? Is it GCC 13 change?


See
https://gcc.gnu.org/gcc-13/porting_to.html#header-dep-changes
Some libstdc++ headers included  in older versions
as an implementation detail but no longer do.

Including stdint.h will introduce ::uint32_t type among others,
but not std::uint32_t, if you use the latter, you need to
include .
I've got a partial list of packages affected by the ongoing header 
cleanups in libstdc++:


intel-igc
j4-dmenu-desktop
julia
jd
jigdo
mtxclient
openclonk
texlive-base
tcpflow
synergy
R-svglite
rstudio
rssguard
rpi-imager
rocm-opencl
rocksdb
qucs
parzip
root
openmsx
openlierox
openexr
openexr2
openms
mongo-cxx-driver
kakoune


I'm sure there's more, I've probably looked at about 20% of the packaged 
I'd flagged as needing further analysis.  But that might help folks 
chase these things down.



Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: RISC-V -- are we ready for more, and what do we need to do it?

[Jeff Law]

On 1/9/23 07:14, Neal Gompa wrote:



It is very unlikely that CentOS Stream 10 will include RISC-V as a
fully included architecture.  Perhaps via a CentOS Stream SIG.



I believe that was the implication in the first place, hence
mentioning CentOS Stream rather than RHEL.

The Alternative Architectures SIG in CentOS would be where this would
happen. But the work needs to be done in Fedora first.
Hard to see a path for CentOS and certainly not RHEL until after Fedora 
is in better shape.  Even then ISTM we need pull from sites looking to 
deploy this stuff at scale.


jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: RISC-V -- are we ready for more, and what do we need to do it?

[Jeff Law]

On 1/6/23 16:19, Jun Aruga (he / him) wrote:

I posted the same article on Fedora Discussion.[1] However let me
share it again on the devel@ to tell it to many people.

This is interesting news about RISC-V this week. Perhaps, it’s time to
prepare to add the RISC-V CPU to the Koji build system?

I think the big question in this space is access to suitable hardware. 
What's out there right now is scarce and woefully under-powered.


My plan is to set up and maintain a riscv64 shadow builder as soon as 
possible after our Veyron-v1 bring-up.  Ideally that will help forge a 
path to official Fedora builds as hardware availability expands.



jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F40 proposal: Porting Fedora to Modern C (System-Wide Change proposal)

[Jeff Law]

On 10/26/22 02:58, Florian Weimer wrote:

* Richard W. M. Jones:


On Tue, Oct 25, 2022 at 10:05:52AM -0400, Ben Cotton wrote:

Neither change is trivial to implement because introducing errors for
these constructs (as required by C99) alters the result of autoconf
configure checks. Quite a few such checks use an implicitly declared
`exit` function, for instance. These failures are not really related
to the feature under test. If the build system is well written, the
build still succeeds, the relevant features are automatically disabled
in the test suite and removed from reference ABI lists, and it's not
immediately apparent that feature is gone. Therefore, some care is
needed that no such alterations happen, and packages need to be ported
to C99. Various tools for this porting activity are being developed to
support this proposal. Cross-distribution collaboration will help as
well, sharing patches and insights.

Would it help if we made autoreconf (and the equivalent step for other
build systems) mandatory?  Debian has declared this as "good practice"
since forever[0].

It would mean we could fix any problems in current autoconf
concurrently with the changes to GCC and Fedora.

I'm not aware of any recent-ish core autoconf issues that would be
solved by running autoreconf (if that actually worked …).  As far as I
can tell, autoconf never relied on implicit function declarations for
AC_CHECK_FUNC, not since 1993.  The sed hack we already have is for LTO
compatibility.


Right.  I nearly chimed in on this topic.  It helps if we can autoreconf 
as I think that might let us remove some of the LTO insanity, but it's 
independent of implicit functions.



Jeff

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Help needed: build failure trying to upgrade healpix

[Jeff Law]

On 8/14/2022 6:29 AM, Jarryd Lisher via devel wrote:

Hey,
I figured out that this is mainly caused by the configure script 
containing multi-line definitions. So, the hacks only operate on the 
first part and land up leaving random lines of nonsense in the file.
You can fix this by adding the following "unwrap" `sed` script just 
before the `%{configure}` line:

```
%{__sed} -i -z -e 's/\\\n//g' configure
```
It's not perfect, but is a simple fix to remove those errors from the 
configure.
That seems like a pretty big hammer with fair potential for changing 
something we didn't really want to.  Can we key it's behavior somehow on 
lines we care about?


Thanks,
Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Help needed: build failure trying to upgrade healpix

[Jeff Law]

On 8/3/2022 2:00 PM, Kevin Kofler via devel wrote:

Jeff Law wrote:

That would eliminate the need for those crazy macros.  The problem is
many packages have configury bits that are ancient and can't be rebuilt
with modern autotools.

I have a hard time believing that.
Well, having been the one that did the work, I can tell you that trying 
to convert circa 2008 autoconf files to anything modern isn't always 
trivial.    I'll also note that locally I was testing things by 
capturing the generated config.h files before/after all these bits went 
in and comparing them - across all of the Fedora packages.  That's 
actually how we knew what insanity we had to fix in the generated 
configure scripts to ensure the resulting config.h files were 
identical.  BUt if you think there's a good way to get to a place where 
we consistently use a reasonably modern autoconf, then by all means, be 
my guest to make it happen and we can zap the configure sed hacking 
insanity.

Even the gigantic KDE 3 autotools mess last updated by upstream in 2008 can
be regenerated with current autotools with only 6 patches (2 of which only
fix hardcoded maximum version numbers).
"gigantic KDE 3 autotools mess" -- now imagine whatever effort there was 
to fix that and apply it to potentially hundreds of packages. Worse yet, 
ponder how you're going to determine if you got it right,  particularly 
on a package you don't know anything about :-)


I'd *love* to it to be policy that things work after an autoconf -fiv 
(or whatever args we choose), but someone will have to step up and do 
the work.


jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Help needed: build failure trying to upgrade healpix

[Jeff Law]

On 8/3/2022 6:55 AM, Richard W.M. Jones wrote:

On Sun, Jul 31, 2022 at 03:33:51PM +0200, Kevin Kofler via devel wrote:

Jerry James wrote:


On Sat, Jul 30, 2022 at 10:35 AM Kevin Kofler via devel
 wrote:

What I see is that the hacks that you apply to configure are apparently
not working:

checking command to parse /usr/bin/nm -B output from gcc object...
./configure: line 7304:  -e 's/^T .* \(.*\)$/extern int \1();/p' -e
's/^[ABCDGIRSTW][ABCDGIRSTW]* .* \(.*\)$/extern char \1;/p': No such file
or directory

I think that's redhat-rpm-config.  See %_fix_broken_configure_for_lto
in /usr/lib/rpm/redhat/macros.  I see the same output from every
package I maintain that uses autoconf-generated configure scripts.

So somebody needs to fix redhat-rpm-config then. It's funny when something
called "fix_broken_configure" produces… a broken configure. ;-)

That macro looks very hairy.
They are 100% totally insane.  I suspect we've got a quoting issue that 
needs to be fixed.




At some point we should just require autoreconf.  I think Debian have
been advising (not requiring) that for a while:

https://wiki.debian.org/Autoreconf
That would eliminate the need for those crazy macros.  The problem is 
many packages have configury bits that are ancient and can't be rebuilt 
with modern autotools.


Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Making Fedora faster (was Re: F37 proposal: Add -fno-omit-frame-pointer to default compilation flags (System-Wide Change proposal))

[Jeff Law]

On 7/10/2022 11:36 AM, Gordon Messmer wrote:

On 7/10/22 04:38, Vitaly Zaitsev via devel wrote:
Have you rebuilt all system packages with -fno-omit-frame-pointer or 
just tested packages? 



No.  Early in the thread, Tomasz Torcz posted a link to a Phoronix 
article as evidence that Fedora's performance was behind other 
distributions.  At one point, I expressed doubt that intel_pstate in 
performance mode was the reason for some significant differences, so I 
ran some tests to try to either prove or disprove that explanation.  
Having done so, I don't think the results that Phoronix published are 
useful as evidence of anything.
Phoronix is notorious for poor benchmarking methodology and trying to 
draw any conclusions from their data is pointless as a result.


Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F37 proposal: Add -fno-omit-frame-pointer to default compilation flags (System-Wide Change proposal)

[Jeff Law]

On 7/6/2022 1:05 PM, Florian Weimer wrote:

* Michael Catanzaro:


On Wed, Jul 6 2022 at 08:06:45 AM -0600, Jeff Law
 wrote:

If I'm understanding things correctly, the original proposal is trying
to make a very special case of profiling work better -- a case that
99.9% of Fedora users do not need or care about.That seems like a
particularly bad cost/benefit for this proposal.

But all Fedora users benefit from performance improvements implemented
as a result of profiling.

I think we have no evidence that you could not get the same results
using Fedora's current profiling tools.  If the GNOME's sysprof does not
work with Fedora, fix it or use something else.  Do not change how
Fedora is built.  It's not really going to work anyway because typical
workloads spent 5% to 10% in glibc's string functions.  Those functions
won't have frame pointers without some non-trivial development work (and
also an ongoing maintenance cost).  If you change compiler flags only,
you still won't get accurate backtraces in many cases.

Yea, it's not a complete solution.



I had some interactions with Red Hat's performance teams over the years,
and to my knowledge, the lack of frame pointers has never come up.
I had some discussions in this space with Peter Z (IIRC) when he was 
still with Red Hat.  We ran into a brick wall with the kernel insistence 
on no dwarf unwinding and the performance impact of keeping frame 
pointers around.  This was the oprofile era IIRC, but the same 
principles apply.


Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F37 proposal: Add -fno-omit-frame-pointer to default compilation flags (System-Wide Change proposal)

[Jeff Law]

On 7/6/2022 8:20 AM, Michael Catanzaro wrote:
On Wed, Jul 6 2022 at 08:06:45 AM -0600, Jeff Law 
 wrote:

If I'm understanding things correctly, the original proposal is trying
to make a very special case of profiling work better -- a case that
99.9% of Fedora users do not need or care about.    That seems like a
particularly bad cost/benefit for this proposal.


But all Fedora users benefit from performance improvements implemented 
as a result of profiling.
Yes, but, IMHO, you need to find another way to do the profiling you 
need to get those improvements.


jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F37 proposal: Add -fno-omit-frame-pointer to default compilation flags (System-Wide Change proposal)

[Jeff Law]

On 7/6/2022 7:26 AM, Marek Polacek wrote:

On Tue, Jul 05, 2022 at 03:47:26PM -0400, Matthias Clasen wrote:

On Tue, Jul 5, 2022 at 3:40 PM Marek Polacek  wrote:


Maybe not, but even ~1% is still an unacceptable slowdown.  It would take
about a year for the compiler to catch up.



(Un)acceptable for whom?

GCC maintainers in Fedora, at least.


And why would it be unacceptable?

Because it's too much.


You just said compilers will make up for it quickly, not to mention
hardware continuously getting faster too...

Dozens of developers working a whole release (if not more) is not quick.
  

I haven't seen any convincing arguments as to why such a small
drop would be the end of the world.

And likewise, I haven't seen how this proposal would be helpful to the
majority of users, nevermind that it'd likely break programs using
inline assembly that use %rbp.  But others have already raised similar
points in this thread.
  

And I don't think Fedora is or should be used in high-speed trading or
similar
environments where every microsecond matters.

I think you may be underestimating how much even 1% matters.
Amen.  A 1% hit is very significant -- as Marek indicated, that's 
roughly a year of work for the GCC community to recover.


Sometimes we're willing to take a 1% hit, sometimes not.  It's a 
question of balancing the performance hit against the benefits of 
whatever change is being considered.


If I'm understanding things correctly, the original proposal is trying 
to make a very special case of profiling work better -- a case that 
99.9% of Fedora users do not need or care about.    That seems like a 
particularly bad cost/benefit for this proposal.


Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: LTO objects after build: Rebuilding vs erroring out

[Jeff Law]

On 11/15/2021 6:06 AM, Florian Weimer wrote:

In the future, we might want to switch GCC not to generate both object
code and LTO representation during the build process.  For most
packages, dual generation is not necessary because no relocatable object
files for static linking are included in the RPM (neither as separate
ET_REL .o files, nor in static .a archives).  Final (non-relocatable)
links of any kind will generate object code, so only LTO representation
needs to be written by GCC.

Yup.  It's something I wanted to do, but never had the time to complete.



But in case relocatable object files are produced by the package (e.g.,
for a -static subpackage for static linking), it is necessary to
generate object code for relocatable files as well.  The reason is that
only object code (not LTO representation) is a stable format, and it's
the only way to achieve cross-toolchain linking.

The way I envisioned LTO-only building for GCC was to replace
the brp-strip-lto script



with somehting that errors out (fails the build) if any relocatable
object files (.o) or static archives (.a) by default, and stop producing
object code by default, only LTO representation.  If a special
redhat-rpm-config flag is set, brp-strip-lto comes back, and GCC is
configured to produce both object code and LTO representation (basically
what we have today).
Right.  In fact, I had a brp-strip-lto which did precisely this and I 
did a Fedora build with that brp-strip-lto to get a set of packages that 
wanted to install a .o or .a composed from .o files. I did a build with 
that, but I don't have the results anymore.




However, Clang has chosen a different approach: Build object code in the
final stages, via the brp-llvm-compile-lto-elf script:



This does not really work for GCC in downstream because we have multiple
GCCs there, with incompatible LTO representation.  We also cannot
replicate the exact command line options that have been used during the
package-internal build process; we only have the default
redhat-rpm-config flags at this point, and whatever has been serialized
into the LTO representation.

Fedora has multiple LLVMs (e.g., rust in Fedora 35 is at LLVM 12, but
/usr/bin/clang is LLVM 13).  LLVM bitcode is supposed to be more
compatible:

   

But don't know to what extent we test that.

I'd prefer to use a single mechanism for both toolchains.  But it seems
that Clang does not support creating ELF object files that contain LLVM
IR (GCC's default mode we use today).  Given the problems with
post-building object code for GCC, I'm not sure if this is feasible.

Thoughts?
It'd be nice to have the same approach, but it may not be ultimately 
feasible.


Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: final link failed: memory exhausted on armv7l

[Jeff Law]

On 10/13/2021 8:51 AM, Björn 'besser82' Esser wrote:

Am Mittwoch, dem 13.10.2021 um 15:44 +0200 schrieb Iñaki Ucar:

Hi,

RStudio is failing consistently on armv7l on F35 [1, 2] and rawhide
[3, 4] with this message (memory exhausted). The same build on the
same machine (CPU and RAM) succeeds on F34 [5]. Any clue what's going
on? Why rawhide and F35 and not F34? Anything I can do in the SPEC to
fix this?

[1] https://koji.fedoraproject.org/koji/taskinfo?taskID=77159810
[2] https://koji.fedoraproject.org/koji/taskinfo?taskID=77170457
[3] https://koji.fedoraproject.org/koji/taskinfo?taskID=77159795
[4] https://koji.fedoraproject.org/koji/taskinfo?taskID=77170370
[5] https://koji.fedoraproject.org/koji/taskinfo?taskID=77159829


As the package doesn't build any *distributed* static library, you can
try to avoid building the linker object files to contain non-lto code:

%global optflags %(echo '%{optflags}' | sed -e 's!-ffat-lto-objects!-
fno-fat-lto-objects!g')

That should drastically cut the amount of memory the linker needs to
create the final ELF binary.  It doesn't hurt to do that on all arches /
releases, as it will also result in significantly shorter build time.
I would strongly discourage this.  This really needs to be addressed in 
redhat-rpm-config.  See my change proposal for the details.


jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: final link failed: memory exhausted on armv7l

[Jeff Law]

On 10/13/2021 10:06 AM, Björn 'besser82' Esser wrote:

Building with lto disabled is a bad idea, as Fedora intentionally
enabled lto by default.
Yes, but there is nothing inherently wrong with not using LTO.  Many 
packages opt-out for various reasons.




What you describe as lto requires a lot of memory is caused by building
lto along with non-lto in the same object file requires significantly
more memory.  For that reason one can disable building non-lto along
with lto using the `-f-no-fat-lto-objects` compiler flags instead of
`-f-fat-lto-objects`, if and *only IF* the package in question does
*NOT* ship static libraries.
I doubt fat-lto-objects is the issue here.  The "fat" stuff is ignored 
at link time as the LTO bytecodes will take precedence.  It is far more 
likely that the parallel link with all those LTO bytecodes is what's 
sucking up all the memory.


Note that we build with fat-lto-objects for a reason.  If a package 
installs any static objects, then those objects must be compiled down to 
machine code as the LTO bytestreams are not compatible across GCC 
releases.  -ffat-lto-objects ensures this.


To remedy this situation you have to put in bits to 
redhat-rpm-config/brp-whatever to fail builds when they install .o/.a 
files into the buildroot that are purely LTO bytecode streams and 
identify every package that fails that test and fix them to turn on 
fat-lto-objects.  I actually wrote some code to do this and ran a Fedora 
build, but never had the time to act on the resultant data. I've since 
left Red Hat and haven't had time to come back to the issue.


Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: final link failed: memory exhausted on armv7l

[Jeff Law]

On 10/13/2021 10:37 AM, Michael Catanzaro wrote:
On Wed, Oct 13 2021 at 06:06:50 PM +0200, Björn 'besser82' Esser 
 wrote:

What you describe as lto requires a lot of memory is caused by building
lto along with non-lto in the same object file requires significantly
more memory.  For that reason one can disable building non-lto along
with lto using the `-f-no-fat-lto-objects` compiler flags instead of
`-f-fat-lto-objects`, if and *only IF* the package in question does
*NOT* ship static libraries.


More background: this default is, of course, backwards. Fedora 
packages do not generally ship static libraries, so it makes more 
sense for the few packages that do to opt-in instead of opt-out. Jeff 
proposed a change to improve that here:


https://fedoraproject.org/wiki/Changes/LTOBuildImprovements

but he left Red Hat, so it hasn't been implemented.

I'd still like to tackle this but my time is limited.

However, I strongly suspect fat-lto-objects is not the problem here.  If 
the build is running out of memory at link time, that is the LTO phase.  
The best solution for that is to either disable LTO on the arm target, 
or (better) limit the parallelism at link time. There was a change to 
redhat-rpm-config that I think made it into f35 to allow a package to 
throttle the link-time parallelism.


jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Enable LTO build for crash utility

[Jeff Law]

On 8/17/2021 8:45 PM, lijiang wrote:

Hi,

In Fedora crash.spec, currently the LTO is disabled as below:

+# This package has an internal copy of GDB which has broken configure 
code for

+# INTDIV0_RAISES_SIGFPE and MUST_REINSTALL_SIGHANDLERS
+# Updating that code properly seems nontrivial and best left to the 
package

+# maintainer.
+# Disable LTO
+%define _lto_cflags %{nil}

commit: c6fc69dae3a2 ("Disable LTO")
So FWIW, those may be non-issues for crash.  One of the tests I did when 
bringing up LTO was to validate that LTO didn't change the autoconf 
detected state for all packages.  Those were common failures, but it's 
pretty rare that packages really care about those features, especially 
INTDIV0_RAISES_SIGFPE -- that test just gets copied all over the place, 
but very very few packages actually care.


I didn't know enough about crash or how it uses gdb to be able to make 
that decision or how to validate crash's behavior.  So the safest


Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Any recent changes to the arm builders?

[Jeff Law]

On 8/14/2021 10:19 AM, Kevin Fenzi wrote:

On Fri, Aug 13, 2021 at 09:34:11PM -0600, Orion Poplawski wrote:

Have there been any recent changes to the arm (32bit) builders?  It seems
like I'm having much more issues there with builds likely running out of
memory or similar.

Yes. They were mistakenly running the normal kernel (so they had ~3GB
memory available). I moved them back to the lpae kernel (so they see
40GB memory), but this causes

https://bugzilla.redhat.com/show_bug.cgi?id=1920183

basically OOM kills kojid, which restarts kojid, which restarts the
build, which kills kojid, etc...

I've tried all kinds of things here, but haven't been able to find any
way to make it work. Arm folks can't duplicate it on non koji builders.
I suspect the number of people using lpae on 32bit arm is... low.
We could just go back to non lpae, but that breaks building some other
packages (llvm fails to build for example).

It makes me wonder if we should consider letting 32bit arm go...
(insert pitchforks and torches).

Anyhow, if anyone has any ideas, let me know.
Letting 32bit arm go would have my support.  I suspect it's less and 
less interesting as a platform every day and it causes nothing but 
headaches.


Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Fedora 34 Change: LTO Build Improvements (System-Wide Change proposal)

[Jeff Law]

On 8/12/2021 10:09 AM, Vít Ondruch wrote:
Just wonder, have somebody inherited this feature after Jeff? And is 
anybody actually working on the LTO?



~~~

$ grep -R lto | grep -E 'global|define' | grep nil | wc -l
190
I don't think anyone is actively working on reducing that set.   But 
that to me would be quite low priority.


Addressing the -ffat-lto-objects would seem to be more important as it 
would significantly reduce build times.  I did a test run a while back 
and identified all the packages that would need to get fixed, but 
changed jobs before I could start fixing them.


We're doing some light Fedora work here, which I expect to ramp up again 
in several months.  Reducing build times is something I can make a case 
to mgt for as it'll affect some of the work we're doing rather 
significantly.


Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F35 mass rebuild is finished

[Jeff Law]

On 7/27/2021 10:31 AM, Björn 'besser82' Esser wrote:

Am Dienstag, dem 27.07.2021 um 16:23 +0200 schrieb Tomas Hrcka:

Hi all,

Per the Fedora 35 schedule[1] we started a mass rebuild for Fedora 35
on Jul 21st, 2021. We did a mass rebuild for Fedora 35 for:

https://fedoraproject.org/wiki/Changes/LTOBuildImprovements

This change hasn't been implemented as well.
Correct.  I've changed jobs and not looking at this right now.  I likely 
will again in the future though, just got to get over certain higher 
priority hurdles.


jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: i686 build OOMing

[Jeff Law]

On 5/18/2021 4:44 PM, David Airlie wrote:

https://kojipkgs.fedoraproject.org//work/tasks/3814/68223814/build.log

cd 
/builddir/build/BUILD/Vulkan-ValidationLayers-sdk-1.2.176.1/i686-redhat-linux-gnu/layers
&& /usr/bin/g++ -DAPI_NAME=\"Vulkan\" -DVK_ENABLE_BETA_EXTENSIONS
-DVK_USE_PLATFORM_WAYLAND_KHR -DVK_USE_PLATFORM_WAYLAND_KHX
-DVK_USE_PLATFORM_XCB_KHR -DVK_USE_PLATFORM_XCB_KHX
-DVK_USE_PLATFORM_XLIB_KHR -DVK_USE_PLATFORM_XLIB_KHX
-DVK_USE_PLATFORM_XLIB_XRANDR_EXT -DVkLayer_khronos_validation_EXPORTS
-I/builddir/build/BUILD/Vulkan-ValidationLayers-sdk-1.2.176.1/layers
-I/builddir/build/BUILD/Vulkan-ValidationLayers-sdk-1.2.176.1/layers/generated
-I/usr/include/glslang -I/usr/include/spirv/include
-I/builddir/build/BUILD/Vulkan-ValidationLayers-sdk-1.2.176.1/i686-redhat-linux-gnu
-I/builddir/build/BUILD/Vulkan-ValidationLayers-sdk-1.2.176.1/i686-redhat-linux-gnu/layers
-O2 -flto=auto -ffat-lto-objects -fexceptions -g1
-grecord-gcc-switches -pipe -Wall -Werror=format-security
-Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS
-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
-fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1
-m32 -march=i686 -mtune=generic -msse2 -mfpmath=sse -mstackrealign
-fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection
-Wpointer-arith -Wno-unused-function -Wno-sign-compare -DNDEBUG -fPIC
-Wall -Wextra -Wno-unused-parameter -Wno-missing-field-initializers
-fno-strict-aliasing -fno-builtin-memcmp -fvisibility=hidden
-Wimplicit-fallthrough=0 -std=gnu++11 -MD -MT
layers/CMakeFiles/VkLayer_khronos_validation.dir/generated/chassis.cpp.o
-MF CMakeFiles/VkLayer_khronos_validation.dir/generated/chassis.cpp.o.d
-o CMakeFiles/VkLayer_khronos_validation.dir/generated/chassis.cpp.o
-c 
/builddir/build/BUILD/Vulkan-ValidationLayers-sdk-1.2.176.1/layers/generated/chassis.cpp
cc1plus: out of memory allocating 65536 bytes after a total of 3310292992 bytes

We already use -g1 to try and reduce memory usage, and I tried not
doing parallel jobs in case it was too much? any suggestions, is this
LTO?


Not likely LTO since this isn't a link phase AFAICT.

Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F35 Change: CompilerPolicy Change (System-Wide Change proposal)

[Jeff Law]

On 4/26/2021 9:52 AM, Jakub Jelinek wrote:

On Mon, Apr 26, 2021 at 09:43:34AM -0600, Jeff Law wrote:

On 4/26/2021 8:20 AM, Kevin Kofler via devel wrote:

Vít Ondruch wrote:

Kevin, I'd love to support your stance and use GCC. However, there are
reasons why we will need to consider Clang for e.g. Ruby:

https://bugzilla.redhat.com/show_bug.cgi?id=1721553

And the reason is not even that upstream favors Clang. The situation is
unfortunately not just black and white.

Looks like the best solution there is to disable PCH support in the Ruby
JIT.

FWIW Vit, myself and others have already extensively discussed this
situation.  Clang is really the best solution in the case of the Ruby JIT
given the various constraints in play.

Isn't that MIR instead?


Hopefully one day.  But right now Clang is the best choice for Ruby.

jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F35 Change: CompilerPolicy Change (System-Wide Change proposal)

[Jeff Law]

On 4/24/2021 3:10 AM, Kevin Kofler via devel wrote:

Tom Stellard wrote:

This change was never rejected.  It become stalled waiting for the change
owners to address some feedback from FESCo.  The feedback has been
addressed now, which is why it was resubmitted.

Ah, then I hereby urge FESCo to finally reject this for real. The mailing
list feedback was overwhelmingly negative last time this was brought up.


That's not a fair characterization of the discussion.  There were 
concerns raised, but I wouldn't say it was "overwhelmingly negative"




It is a distribution choice which compiler produces the best results for the
distribution (best optimization, highest security, etc.). If that compiler
is GCC, then why would we want to build packages with Clang, unless there is
no other way? (Or if Clang is really better, then why do we not build the
whole distribution with Clang? But as far as I know, GCC is still the better
compiler overall, so there is no point in switching.)


There are cases where Clang is the better choice and other cases where 
GCC is the better choice.  The upstream projects are in the best 
position to make such decisions for their projects and the Fedora 
maintainers are in the best position to bring that decision into Fedora.



Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F35 Change: CompilerPolicy Change (System-Wide Change proposal)

[Jeff Law]

On 4/24/2021 8:26 AM, Michael Catanzaro wrote:
On Fri, Apr 23 2021 at 08:01:03 PM +0200, Jakub Jelinek 
 wrote:

I'll be probably repeating myself, but the two compilers are known to be
ABI incompatible in very important ways, none of the
https://bugs.llvm.org/buglist.cgi?quicksearch=42439%2C19909%2C44228%2C12207 

bugs have been touched since last time this was discussed and I'm not 
aware

of any ABI compatibility testing between the two compilers.
So if some library packages switch compilers and programs using those
libraries don't or vice versa, this proposal has quite high risks of
introducing hard to debug debugs.


Perhaps this proposal should be revisited in the future when there are 
fewer scary ABI bugs.


That's not really a scary ABI bug.   There's disagreement about a fairly 
unusual case involving extension of 8 bit arguments and some issues with 
128 bit types.  I wouldn't consider either a show stopper for this proposal.



jeff

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F35 Change: CompilerPolicy Change (System-Wide Change proposal)

[Jeff Law]

On 4/26/2021 8:00 AM, Zbigniew Jędrzejewski-Szmek wrote:

On Fri, Apr 23, 2021 at 07:24:28PM +0200, Vitaly Zaitsev via devel wrote:

On 23.04.2021 17:18, Ben Cotton wrote:

The goal is to give the packager the ability to use their own
technical judgement to choose the best compiler.

+1 for this change.

Same here. I like the no-nonsense policy of "trust the maintainer" to
make the right technical decisions.


Exactly.


Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F35 Change: CompilerPolicy Change (System-Wide Change proposal)

[Jeff Law]

On 4/26/2021 8:20 AM, Kevin Kofler via devel wrote:

Vít Ondruch wrote:

Kevin, I'd love to support your stance and use GCC. However, there are
reasons why we will need to consider Clang for e.g. Ruby:

https://bugzilla.redhat.com/show_bug.cgi?id=1721553

And the reason is not even that upstream favors Clang. The situation is
unfortunately not just black and white.

Looks like the best solution there is to disable PCH support in the Ruby
JIT.


FWIW Vit, myself and others have already extensively discussed this 
situation.  Clang is really the best solution in the case of the Ruby 
JIT given the various constraints in play.


Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F35 Change: CompilerPolicy Change (System-Wide Change proposal)

[Jeff Law]

On 4/23/2021 6:37 PM, Tom Stellard wrote:

On 4/23/21 4:55 PM, Kevin Kofler via devel wrote:

Ben Cotton wrote:

https://fedoraproject.org/wiki/Changes/CompilerPolicy


Is the change owners' plan here to resubmit this same rejected change 
for
every single release until people get so fed up that they end up 
approving

it just to get it out of the way?



This change was never rejected.  It become stalled waiting for the change
owners to address some feedback from FESCo.  The feedback has been 
addressed now,

which is why it was resubmitted.


Right.  I just never had time to take the FESCo feedback and run the 
change to completion.  I'm not sure why Kevin is trying to characterize 
this as a rejected change.



Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F35 Change: CompilerPolicy Change (System-Wide Change proposal)

[Jeff Law]

On 4/23/2021 9:57 AM, Daniel P. Berrangé wrote:

On Fri, Apr 23, 2021 at 11:18:59AM -0400, Ben Cotton wrote:
https://fedoraproject.org/wiki/Changes/CompilerPolicy == Summary == 
Fedora has historically forced packages to build with GCC unless the 
upstream project for the package only supported Clang/LLVM. This 
change proposal replaces that policy with one where, given a good 
technical reason, a packager may: * Choose to build with their 
package with clang even if the upstream project supports gcc. * 
Choose to build with gcc even if upstream does not support it. The 
goal is to give the packager the ability to use their own technical 
judgement to choose the best compiler. 
Does the choice have to be consistent per package, or is it within 
expectations to use different compilers on different Fedora arches ?
I'd think most packages would be consistent, but I don't think it 
necessarily needs to be policy.  If a package maintainer has a good 
reason to use gcc for part of the package and llvm/clang for another, 
then they should be able to do that.


== Detailed Description == 
It is worth noting that Clang/LLVM's implementation of 
-fstack-clash-protection, which is one of Fedora's default compiler 
flags, is under development and does not yet have an implementation 
for AArch64. 
This would suggest need to use GCC on x86 and CLang on aarch64 within 
a single package.
Note the lack of stack-clash on aarch64 is a transient issue.  It is 
being worked on, but just didn't make it into llvm12, I would expect it 
to land for llvm13 (Tom's a better resource on that going forward, so if 
he's got newer information than mine, believe his :-)



Upstream QEMU has been considering whether there's any loss of 
features implied by replacing GCC with use of CLang in QEMU builds. 
One edge case was discovered for s390x target. GCC has a min code 
generation target of z900 (hw release circa 2000), CLang has a min 
code genreation target of z10 (hw release circa 2008) This doesn't 
matter for compiling QEMU itself, as anyone actually running QEMU on 
an s390x is likely to have z10 or newer. It does cause a problem with 
firmware builds though. If QEMU wants to emulate old hardware 
(qemu-system-s390x -cpu z900 ..args...), then the firmware blob needs 
to have been built to target z900 which is not possible with CLang. So 
if QEMU is switched to use CLang, then we will possibly need to use 
GCC still for QEMU's firmware builds, or only use CLang on certain 
Fedora arches. This is quite a niche problem that's unlikely to cause 
issues for most people, but its a illustration that swapping compilers 
out can have unexpected consequences/complications.
This is a great example where trying to force consistency doesn't make 
sense and the maintainers can/should do what is best for the package and 
Fedora.


jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F35 Change: CompilerPolicy Change (System-Wide Change proposal)

[Jeff Law]

On 4/23/2021 10:32 AM, Tom Stellard wrote:

On 4/23/21 8:27 AM, Ben Cotton wrote:

On Fri, Apr 23, 2021 at 11:18 AM Ben Cotton  wrote:


change proposal replaces that policy with one where, given a good
technical reason, a packager may:
* Choose to build with their package with clang even if the upstream
project supports gcc.
* Choose to build with gcc even if upstream does not support it.


To be clear, does "given a good technical reason" imply that there is
some kind of approval process for this? Or that there's a way to
object to the compiler usage based on an insufficiently-good technical
reason?

Or is it just a way of saying "we trust you to exercise good judgment"?



I did not have in mind any kind of process for either approving the
technical justification or for auditing packages that had decided to
switch compilers to make sure their reasons are valid.
Nor did I when I made the original proposal.  I'm of the opinion that we 
should be
trusting the package maintainers and upstreams to make these kinds of 
decisions.





I think it would be better to not have an approval process, but I would
rather have this proposal + an approval process than no proposal at all.

Agreed.

And just to be clear to the wider community.  Tom asked to take over 
pushing on

this proposal after I left Red Hat.  I fully support that effort.

Thanks,
Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Fedora 35 Change: Autoconf-2.71 (Self-Contained Change proposal)

[Jeff Law]

On 2/23/21 4:54 PM, Jerry James wrote:
> On Tue, Feb 23, 2021 at 4:52 PM Jeff Law  wrote:
>> On 2/23/21 4:39 PM, Brian C. Lane wrote:
>>> On Tue, Feb 23, 2021 at 11:37:42AM +0100, Ondrej Dubaj wrote:
>>>> [2] http://torsion.usersys.redhat.com:8080/job/Fedora-autoconf/
>>> Doesn't work, with or without :8080
>> Not sure what you're referring to, it just worked fine for me.
> As Ondrej said: "Attaching also results from a tester launched by Jeff
> Law [2] (accessible only on Red Hat VPN)".  Those of us without access
> to the Red Hat VPN cannot see those results.
But Brian is (or can be) inside the VPN.  That's what I was referring to.

jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Fedora 35 Change: Autoconf-2.71 (Self-Contained Change proposal)

[Jeff Law]

On 2/23/21 4:39 PM, Brian C. Lane wrote:
> On Tue, Feb 23, 2021 at 11:37:42AM +0100, Ondrej Dubaj wrote:
>> [2] http://torsion.usersys.redhat.com:8080/job/Fedora-autoconf/
> Doesn't work, with or without :8080
Not sure what you're referring to, it just worked fine for me.

>
> parted is failing with a pile of newly obsolete things, and one error
> (as far as I can tell) with a missing build-aux/compile
Yes, partd falls down spectacularly.


http://torsion.usersys.redhat.com:8080/job/Fedora-autoconf/view/Failing/job/parted/216/consoleFull

Search for the first "autoreconf" and see all the diagnostics.

Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Fedora 35 Change: Autoconf-2.71 (Self-Contained Change proposal)

[Jeff Law]

On 2/10/21 11:00 AM, Miro Hrončok wrote:
> On 10. 02. 21 18:47, Ben Cotton wrote:
>> == Upgrade/compatibility impact ==
>> Problems during build can appear in multiple packages what can lead to
>> build failure, as multiple packages require autoconf-2.69 as their
>> upstream dependency. These problems have to be resolved before adding
>> autoconf-2.71 into Fedora. It seems aprox. 20% of dependent packages
>> are having problems during build, which could be caused by a problem
>> with same pattern.
>
> In absolute numbers, what is 20%? I see ~200 failures at
> https://copr.fedorainfracloud.org/coprs/odubaj/autoconf-2.70/monitor/
> (obviously not all of them are necessarily caused by autoconf).
>
> Should this be a system-wide change instead?
Given how many packages use autoconf, I think so.

--

I've already volunteered my tester to help shake out this change.  It's
actually a really good fit because of the autoconf/LTO interactions we
had to sort out for F33/LTO.  The plan is to spin it up next week.

In simplest terms autoconf generated code to test for the existence of
certain capabilities can be optimized away completely by LTO.  This
results in autoconf incorrectly claiming certain capabilities exist. 
This can cause packages to FTBFS or to even mis-behave at runtime.

Thus it was critical to find these cases and deal with them as part of
the LTO effort.  So my tester has the ability to capture generated
config.h files across its control and test builds and will report a
failure if the generated config.h files differ (with an ability to
exclude those where timestamps and such end up in the generated config.h
files).

In the test I'm going to run the only difference between the control and
test build will be the version of autoconf.  So the failures should give
us a highly accurate picture of how autoconf-2.70 will impact the
distribution as a whole.

jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 34 Change: LTO Build Improvements (System-Wide Change proposal)

[Jeff Law]

On 2/2/21 5:03 PM, Ian McInerney wrote:
> On Mon, Jan 4, 2021 at 7:28 PM Jeff Law  <mailto:l...@redhat.com>> wrote:
>
>
> snip...
> >
> > What is this macro going to be called?  I would like to get an early
> > start on updating my packages.
> No idea.  I'm open to suggestions.
>
>
> Any update on the name of this macro/its implementation? I am working
> on revising a spec file that has a static library included and if this
> macro is available to use already I would like to add it as long as I
> am working in the spec.
No movement yet.  I had to push this (and other) stuff down to deal with
a variety of F34 firedrills.  I expect to come back to it during the F35
cycle.

jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Test failures

[Jeff Law]

On 1/28/21 10:16 AM, Boian Bonev wrote:
> Hi,
>
> I just did a build (new upstream release of iotop-c) and saw a couple
> of test errors; they look like segfaults or post errors, so I suppose
> that I didn't do some stupid mistake.
>
> Posting the result here, to keep the relevant team informed; in case
> these are known, sorry for the noise :)
>
> https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/rpminspect-pipeline/job/master/8520/testReport/(root)/tests/
It looks like at least one of the issues is that the package is
triggering a fault in abidiff.  I've forwarded this issue to Dodji who
knows those bits better than anyone.  I'd hazard a guess it's dwarf-5
related.

Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: unrecognized DWARF version in .debug_info at 6

[Jeff Law]

On 1/27/21 5:48 AM, Sandro Mani wrote:
>
> Hi
>
> Apitrace is currently failing to build, with [1]
>
> Test project 
> /builddir/build/BUILD/apitrace-37c36e66b8cfa534797ca565c22e8c30923f35d4/x86_64-redhat-linux-gnu
> Start 1: libbacktrace_btest
> 1/6 Test #1: libbacktrace_btest ...***Exception: SegFault  0.11 
> sec
> unrecognized DWARF version in .debug_info at 6
>
> Some dwz issues were mentioned in the runup to the mass rebuild, is this a 
> related issue?
More likely it doesn't understand dwarf5.  GCC changed from defaulting
from dwarf-v4 to dwarf-v5.

I don't offhand recall if apitrace has a copy of GCC's libbacktrace or
its own complete reimplementation.  If it's the former, then you might
consider resyncing with GCC or picking up these changes from
gcc.gnu.org/git/gcc.git:


commit bfde774667fbce6d7d326c8a36a098138e224a95
Author: Ian Lance Taylor 
Date:   Mon Jan 18 14:45:57 2021 -0800

    libbacktrace: don't fail tests if dwz fails
   
    * Makefile.am (%_dwz): If dwz fails, use uncompressed debug
info.
    * Makefile.in: Regenerate.
    * configure: Regenerate.

commit 325e70b47c6c321710c7b9c792b8fbee95cecd63
Author: Ian Lance Taylor 
Date:   Mon Jan 18 14:38:10 2021 -0800

    libbacktrace: use correct directory/filename for DWARF 5
   
    PR debug/98716
    * dwarf.c (read_v2_paths): Allocate zero entry for dirs and
    filenames.
    (read_line_program): Remove parameter u, change caller.  Don't
    subtract one from dirs and filenames index.
    (read_function_entry): Don't subtract one from filenames index.
Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 34 Mass Rebuild

[Jeff Law]

On 1/26/21 9:13 AM, Jonathan Wakely wrote:
> On 26/01/21 16:52 +0100, Fabio Valentini wrote:
>> On Tue, Jan 26, 2021 at 4:47 PM Jonathan Wakely
>>  wrote:
>>>
>>> On 25/01/21 15:16 +0100, Fabio Valentini wrote:
>>> >On Thu, Jan 21, 2021 at 5:10 PM Jakub Jelinek 
>>> wrote:
>>> >>
>>> >> On Wed, Jan 20, 2021 at 02:17:28PM -0500, Mohan Boddu wrote:
>>> >> > We are delaying the mass rebuild by a day as of now due to bugs
>>> in gcc
>>> >> > and dwz. As of now, we are expecting to start mass rebuild
>>> tomorrow,
>>> >> > Jan 21st 2021. There is a new build of gcc running currently
>>> which has
>>> >> > fixes to gcc bugs, but we are still figuring out the dwz fixes. We
>>> >> > will keep you posted with any further developments.
>>> >>
>>> >> Both gcc-11.0.0-0.16.fc34 and dwz-0.13-6.fc34 with the fixes
>>> >> are now in f34-build (and in eln-build too with s/fc34/eln108/).
>>> >> Those who had their builds fail in the last 2 days because of s390x
>>> >> rpm crashes, errors about strip failures of LTO debug sections or
>>> dwz
>>> >> crashes can retry their builds, sorry for the inconvenience.
>>> >>
>>> >> AFAIK we are waiting now for boost and maybe binutils.
>>> >
>>> >boost 1.75 was merged yesterday, and binutils 2.36 was postponed to
>>> F35.
>>
>> (snip)
>>
>>> >I'm wondering something else: Will automation trigger a "second mass
>>> >rebuild" for ELN once all the builds from the Fedora mass rebuild get
>>> >tagged into f34?
>>> >There's apparently issues with ELN right now, it looks like the base
>>> >buildroot is not installable because boost 1.75 builds were done in
>>> >the wrong order.
>>>
>>> Wrong in what way?
>>
>> Looks like this was a transient issue. koschei is no longer
>> complaining about boost.
>> The problem was some dependency chain involving rpm-build ->
>> source-highlight -> boost 1.73, but it appears to have been a
>> temporary problem, if it was a problem at all.
>
> Possible the ELN buildroot contains gdb-headless which depends on
> source-highlight which depends on boost, which causes a bootstrapping
> problem (you can't update boost, because then you can't install the
> buildroot to rebuild anything else). It might have been solved by
> untagging the new boost. I thought that was a problem for RHEL builds,
> but maybe it's ELN too.
>
> The Fedora buildroot appears to contain gdb-minimal instead of
> gdb-headless, and that doesn't depend on source-highlight.
See:
https://src.fedoraproject.org/rpms/gdb/pull-request/8#request_diff

It's on the agenda for the team call today.

jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: /usr/lib/rpm/redhat/brp-strip-lto fails with symbol `.gnu.debuglto_.debug_line_str' required but not present

[Jeff Law]

On 1/21/21 8:26 AM, Miro Hrončok wrote:
> On 20. 01. 21 17:03, Jeff Law wrote:
>>
>>
>> On 1/20/21 8:47 AM, Vitaly Zaitsev via devel wrote:
>>> On 20.01.2021 11:31, Miro Hrončok wrote:
>>>> Right before the mass rebuild. Is this known?
>>>
>>> Fedora 34 will be built again with broken, full of regressions GCC
>>> compiler (just like Fedora 32).
>> Fix for this is already approved upstream and likely going into a new
>> spin of gcc.
>>
>> FWIW, it's related to flipping to dwarf5.  It doesn't affect code
>> generation.
>
> Appears to be fixed in gcc-...-16.fc34.
>
> https://koji.fedoraproject.org/koji/taskinfo?taskID=60148197
Yes.  We've been waiting for that to land.  The arm32 builder croaked
yesterday and it had to get restarted in the mid/late afternoon US
time.   I wouldn't expect it to land until mid/late afternoon today.

jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Crash, maybe, in packaging scripts on Koji/s390

[Jeff Law]

On 1/20/21 3:53 AM, Richard W.M. Jones wrote:
> On Wed, Jan 20, 2021 at 10:52:35AM +, Richard W.M. Jones wrote:
>> https://kojipkgs.fedoraproject.org//work/tasks/9457/60089457/build.log
>> (from https://koji.fedoraproject.org/koji/taskinfo?taskID=60089433)
>>
>> ends with ...
>>
>> Checking for unpackaged file(s): /usr/lib/rpm/check-files 
>> /builddir/build/BUILDROOT/ocaml-extlib-1.7.8-1.fc34.s390x
>> Child return code was: -11
>> EXCEPTION: [Error()]
>> Traceback (most recent call last):
>>   File "/usr/lib/python3.9/site-packages/mockbuild/trace_decorator.py", line 
>> 93, in trace
>> result = func(*args, **kw)
>>   File "/usr/lib/python3.9/site-packages/mockbuild/util.py", line 600, in 
>> do_with_status
>> raise exception.Error("Command failed: \n # %s\n%s" % (command, output), 
>> child.returncode)
>> mockbuild.exception.Error: Command failed: 
>>  # bash --login -c /usr/bin/rpmbuild -bb --target s390x --nodeps 
>> /builddir/build/SPECS/ocaml-extlib.spec
>>
>> I kicked off another build to see if this is reproducible.
> Oh sorry, I see there is already a thread this morning about this:
>
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/XQWO72AZ7MKVC3CSICKKYF77EHNUVJ6U/
Should be fixed now I think.  Hopefully mjw, jakub and florian are
getting some sleep.    The s390x issue was "beyond unexpected".

jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: /usr/lib/rpm/redhat/brp-strip-lto fails with symbol `.gnu.debuglto_.debug_line_str' required but not present

[Jeff Law]

On 1/20/21 9:29 AM, Miro Hrončok wrote:
> On 20. 01. 21 17:21, Jeff Law wrote:
>>
>>
>> On 1/20/21 9:17 AM, Miro Hrončok wrote:
>>> On 20. 01. 21 17:03, Jeff Law wrote:
>>>>
>>>>
>>>> On 1/20/21 8:47 AM, Vitaly Zaitsev via devel wrote:
>>>>> On 20.01.2021 11:31, Miro Hrončok wrote:
>>>>>> Right before the mass rebuild. Is this known?
>>>>>
>>>>> Fedora 34 will be built again with broken, full of regressions GCC
>>>>> compiler (just like Fedora 32).
>>>> Fix for this is already approved upstream and likely going into a new
>>>> spin of gcc.
>>>>
>>>> FWIW, it's related to flipping to dwarf5.  It doesn't affect code
>>>> generation.
>>>
>>> Can we please check the impact of such changes in the future *before*
>>> shipping them?
>> Generally we do.  If you were to look at the hundreds of fixes we've
>> already made in F33 and F34, those are a direct result of doing our own
>> test builds and fixing things we've found, either in the package or in
>> gcc itself.
>
> Thanks!
NP.  It's good for Fedora and good for GCC as well.

>
>> THe dwarf5 change went in very late upstream and thus wasn't subject to
>> the continuous testing and fixing we've done.   Sorry for that.
>
> I understand that stuff can break unexpectedly.
Yea, the timing was awful.  It didn't help that when I saw this
yesterday in the tester I passed it along to Nick thinking it was
actually a binutils problem.  So we lost a fair amount of time because
of that.



jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: /usr/lib/rpm/redhat/brp-strip-lto fails with symbol `.gnu.debuglto_.debug_line_str' required but not present

[Jeff Law]

On 1/20/21 9:08 AM, Vitaly Zaitsev via devel wrote:
> On 20.01.2021 17:03, Jeff Law wrote:
>> Fix for this is already approved upstream and likely going into a new
>> spin of gcc.
>
> Today I got two new GCC regressions in the same package. Reported to
> RHBZ.
THen it's a safe bet we'll fix them.



Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: /usr/lib/rpm/redhat/brp-strip-lto fails with symbol `.gnu.debuglto_.debug_line_str' required but not present

[Jeff Law]

On 1/20/21 9:17 AM, Miro Hrončok wrote:
> On 20. 01. 21 17:03, Jeff Law wrote:
>>
>>
>> On 1/20/21 8:47 AM, Vitaly Zaitsev via devel wrote:
>>> On 20.01.2021 11:31, Miro Hrončok wrote:
>>>> Right before the mass rebuild. Is this known?
>>>
>>> Fedora 34 will be built again with broken, full of regressions GCC
>>> compiler (just like Fedora 32).
>> Fix for this is already approved upstream and likely going into a new
>> spin of gcc.
>>
>> FWIW, it's related to flipping to dwarf5.  It doesn't affect code
>> generation.
>
> Can we please check the impact of such changes in the future *before*
> shipping them?
Generally we do.  If you were to look at the hundreds of fixes we've
already made in F33 and F34, those are a direct result of doing our own
test builds and fixing things we've found, either in the package or in
gcc itself.

THe dwarf5 change went in very late upstream and thus wasn't subject to
the continuous testing and fixing we've done.   Sorry for that.

Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: /usr/lib/rpm/redhat/brp-strip-lto fails with symbol `.gnu.debuglto_.debug_line_str' required but not present

[Jeff Law]

On 1/20/21 8:47 AM, Vitaly Zaitsev via devel wrote:
> On 20.01.2021 11:31, Miro Hrončok wrote:
>> Right before the mass rebuild. Is this known?
>
> Fedora 34 will be built again with broken, full of regressions GCC
> compiler (just like Fedora 32).
Fix for this is already approved upstream and likely going into a new
spin of gcc.

FWIW, it's related to flipping to dwarf5.  It doesn't affect code
generation.

Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 34 Change: LTO Build Improvements (System-Wide Change proposal)

[Jeff Law]

On 1/2/21 10:39 AM, Ian McInerney wrote:
>
>
> On Sat, Jan 2, 2021 at 5:33 PM Jeff Law  <mailto:l...@redhat.com>> wrote:
>
>
>
> On 12/30/20 3:48 PM, Ian McInerney wrote:
> >
> >
> > On Wed, Dec 30, 2020 at 7:54 PM Ben Cotton  <mailto:bcot...@redhat.com>
> > <mailto:bcot...@redhat.com <mailto:bcot...@redhat.com>>> wrote:
> >
> >     https://fedoraproject.org/wiki/Changes/LTOBuildImprovements
> <https://fedoraproject.org/wiki/Changes/LTOBuildImprovements>
> >     <https://fedoraproject.org/wiki/Changes/LTOBuildImprovements
> <https://fedoraproject.org/wiki/Changes/LTOBuildImprovements>>
> >
> >
> >     == Summary ==
> >     Currently all packages that are not opted out of LTO include
> >     -ffat-lto-objects in their build flags.  This proposal would
> remove
> >     -ffat-lto-objects from the default LTO flags and only use it for
> >     packages that actually need it.
> >
> >     == Owner ==
> >     * Name: [[User:law | Jeff Law]]
> >     * Email: l...@redhat.com <mailto:l...@redhat.com>
> <mailto:l...@redhat.com <mailto:l...@redhat.com>>
> >
> >
> >     == Detailed Description ==
> >     -ffat-lto-objects was added to the default LTO flags to
> ensure that
> >     any installed .o/.a files included actual compiled code
> rather than
> >     just LTO bytecodes (which are stripped after the install phase).
> >     However, that is wasteful from a compile-time standpoint as few
> >     packages actually install any .o/.a files.
> >
> >     This proposal would remove -ffat-lto-objects from the
> default LTO
> >     flags and packages that actually need the option would have
> to opt-in
> >     via an RPM macro in their .spec file.  This should significantly
> >     improve build times for most packages in Fedora.
> >
> >
> > Does this mean that packages that are explicitly shipping a static
> > library to the end user need to enable this macro to allow the
> > installed static library to be usable by an end-user's compiler? If
> > this is the case, then the packaging guidelines should be updated to
> > reflect this.
> Yes and the change request reflects that an update to the packaging
> guidelines is necessary.
>
>
> The proposal only says the macro will be documented, but doesn't go
> into any more detail. I think it would be beneficial if the proposal
> mentioned the macro would become required for static library packages
> and that policy needs to be added - since that is adding a new MUST
> section to the packaging guidelines.
ACK.  I'll add some additional text to the proposal.

jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 34 Change: LTO Build Improvements (System-Wide Change proposal)

[Jeff Law]

On 1/4/21 12:22 PM, Tom Stellard wrote:
> On 12/30/20 11:52 AM, Ben Cotton wrote:
>> https://fedoraproject.org/wiki/Changes/LTOBuildImprovements
>>
>>
>> == Summary ==
>> Currently all packages that are not opted out of LTO include
>> -ffat-lto-objects in their build flags.  This proposal would remove
>> -ffat-lto-objects from the default LTO flags and only use it for
>> packages that actually need it.
>>
>> == Owner ==
>> * Name: [[User:law | Jeff Law]]
>> * Email: l...@redhat.com
>>
>>
>> == Detailed Description ==
>> -ffat-lto-objects was added to the default LTO flags to ensure that
>> any installed .o/.a files included actual compiled code rather than
>> just LTO bytecodes (which are stripped after the install phase).
>> However, that is wasteful from a compile-time standpoint as few
>> packages actually install any .o/.a files.
>>
>> This proposal would remove -ffat-lto-objects from the default LTO
>> flags and packages that actually need the option would have to opt-in
>> via an RPM macro in their .spec file.  This should significantly
>> improve build times for most packages in Fedora.
>>
>
> What is this macro going to be called?  I would like to get an early
> start on updating my packages.
No idea.  I'm open to suggestions.

IIRC we had kicked around the idea of having the clang/llvm path
instantiate the LTO bytecodes in .o/.a files, which would avoid these
issues for packages using clang/llvm.  Is that something we still want
to pursue?  I vaguely recall discussing this with David B. and he came
up with a reason why that needed to happen before brp-strip-lto, but I
can't remember any of the details at this point.

Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 34 Change: LTO Build Improvements (System-Wide Change proposal)

[Jeff Law]

On 1/4/21 2:27 AM, Florian Weimer wrote:
>
>>> A lot of the existing RPM post-processing steps detect, report, and
>>> ignore errors because the generated RPM package might still be partially
>>> useful.
>> True, but ignoring the error in this case runs the very real risk that a
>> package could install a .o/.a file with no code/symbols.  That in turn
>> can cause downstream FTBFS errors in other packages and all kinds of
>> headaches on developer systems.  Failing the build here seems much safer.
>>
>> Contrast to ignoring a dwz error.  The resulting binary RPMs are still
>> very much usable, the debuginfo packages are just larger than is
>> strictly necessary.
> The downside is that toolchain bugs tend to go unnoticed and aren't
> fixed as a result.
Right.  And I think that's been a real problem.  I really dislike
ignoring errors :(

jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: HEADS UP: OpenEXR + ilmbase = (new) openexr

[Jeff Law]

On 1/1/21 7:08 PM, Richard Shaw wrote:
> All builds have completed in a side tag with the exception of gmic
> which looks to be failing for gcc 11 related issues? But only on
> 32-bit arches.
>
> https://koji.fedoraproject.org/koji/taskinfo?taskID=58753095
> 
Looks like a GCC bug to me.  Can you get a bug report filed?  THanks!

jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 34 Change: LTO Build Improvements (System-Wide Change proposal)

[Jeff Law]

On 12/30/20 3:48 PM, Ian McInerney wrote:
>
>
> On Wed, Dec 30, 2020 at 7:54 PM Ben Cotton  <mailto:bcot...@redhat.com>> wrote:
>
> https://fedoraproject.org/wiki/Changes/LTOBuildImprovements
> <https://fedoraproject.org/wiki/Changes/LTOBuildImprovements>
>
>
> == Summary ==
> Currently all packages that are not opted out of LTO include
> -ffat-lto-objects in their build flags.  This proposal would remove
> -ffat-lto-objects from the default LTO flags and only use it for
> packages that actually need it.
>
> == Owner ==
> * Name: [[User:law | Jeff Law]]
> * Email: l...@redhat.com <mailto:l...@redhat.com>
>
>
> == Detailed Description ==
> -ffat-lto-objects was added to the default LTO flags to ensure that
> any installed .o/.a files included actual compiled code rather than
> just LTO bytecodes (which are stripped after the install phase).
> However, that is wasteful from a compile-time standpoint as few
> packages actually install any .o/.a files.
>
> This proposal would remove -ffat-lto-objects from the default LTO
> flags and packages that actually need the option would have to opt-in
> via an RPM macro in their .spec file.  This should significantly
> improve build times for most packages in Fedora.
>
>
> Does this mean that packages that are explicitly shipping a static
> library to the end user need to enable this macro to allow the
> installed static library to be usable by an end-user's compiler? If
> this is the case, then the packaging guidelines should be updated to
> reflect this.
Yes and the change request reflects that an update to the packaging
guidelines is necessary.


Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 34 Change: LTO Build Improvements (System-Wide Change proposal)

[Jeff Law]

On 1/2/21 3:13 AM, Florian Weimer wrote:
> * Ben Cotton:
>
>> To ensure that we can identify packages that need the opt-in now and
>> in the future, the plan is to pass to brp-strip-lto a flag indicating
>> whether or not the package has opted into -ffat-lto-objects.  If
>> brp-strip-lto finds .o/.a files, but the package has not opted into
>> -ffat-lto-objects, then brp-strip-lto would signal an error.
> And presumably fail the build?
Yes.  The point here is to ensure that if the build installs .o/.a files
that it must have opted into -ffat-lto-objects so that we don't end up
with useless .o/.a files in our binary RPM packages.

>
> A lot of the existing RPM post-processing steps detect, report, and
> ignore errors because the generated RPM package might still be partially
> useful.
True, but ignoring the error in this case runs the very real risk that a
package could install a .o/.a file with no code/symbols.  That in turn
can cause downstream FTBFS errors in other packages and all kinds of
headaches on developer systems.  Failing the build here seems much safer.

Contrast to ignoring a dwz error.  The resulting binary RPMs are still
very much usable, the debuginfo packages are just larger than is
strictly necessary.

jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Chromium built in rawhide does not render most strings

[Jeff Law]

On 12/20/20 7:45 PM, Kevin Kofler via devel wrote:
> Neal Gompa wrote:
>> QtWebEngine *is* Chromium, so it would make sense that it exhibits the
>> same problem.
> To be clear (and I know you know this, but your readers might not know), 
> QtWebEngine (qt5-qtwebengine) and Chromium (chromium) are distinct packages 
> (none of them depends on each other), each containing a slightly different 
> copy of essentially the same source code (plus some higher-layer code that 
> is entirely different: Qt wrapper libraries vs. a browser application). But 
> since the source code is largely the same, things such as miscompilations by 
> the compiler are likely to affect both the same way. And the symptoms in the 
> 2 screenshots definitely look identical.
I think this has been fixed with the latest gcc drop into rawhide.

jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: gcc11/kconfig_compiler (was: weird build failure on i686)

[Jeff Law]

On 12/19/20 9:40 PM, Rex Dieter wrote:
> Mattia Verga via devel wrote:
>
>> While trying to build a new kde-partitionmanager release, I get a strange
>> build failure which seems related to character encoding:
>>
>> https://kojipkgs.fedoraproject.org//work/tasks/8985/57428985/build.log
>>
>> /builddir/build/BUILD/partitionmanager-20.12.0/i686-redhat-linux-
> gnu/src/config.cpp:48:1:
>> error: extended character   is not valid in an identifier
>>48 | 挀漀渀猀琀 挀栀愀爀⨀ 挀漀渀猀琀 Config::EnumFileSystem::enumToString[] = {
>>"Unknown", "Extended", "Ext2", "Ext3", "Ext4", "LinuxSwap", "Fat16",
>>"Fat32", "Ntfs", "ReiserFS", "Reiser4", "Xfs", "Jfs", "Hfs", "HfsPlus",
>>"Ufs", "Unformatted", "Btrfs", "Hpfs", "Luks", "Ocfs2", "Zfs", "Exfat",
>>"Nilfs2", "Lvm2_PV", "F2fs", "Udf", "Iso9660", "Luks2", "Fat12",
>>"LinuxRaidMember", "BitLocker", "Apfs", "Minix" };
>>   | ^
>> /builddir/build/BUILD/partitionmanager-20.12.0/i686-redhat-linux-
> gnu/src/config.cpp:48:1:
>> error: extended character ⨀ is not valid in an identifier
>> /builddir/build/BUILD/partitionmanager-20.12.0/i686-redhat-linux-
> gnu/src/config.cpp:48:1:
>> error: extended character   is not valid in an identifier
>> /builddir/build/BUILD/partitionmanager-20.12.0/i686-redhat-linux-
> gnu/src/config.cpp:48:1:
>> error: extended character   is not valid in an identifier
>>
>> This seems to happen only on i686 arch, so I don't think sources are
>> corrupted in some way. Also, there seems to be no relevant changes between
>> the latest built release and this one that could justify the failure:
>>
>> https://github.com/KDE/kpmcore/compare/v20.11.90...v20.12.0
>>
>> Any idea what's going on?
> Wierd issue with kconfig_compiler on i686 when built with gcc-11, see bug
>
> https://bugzilla.redhat.com/show_bug.cgi?id=1907799
I wouldn't be surprised if this turns out to be a gcc bug.  In fact,
I've got my eye on upstream gcc bug 98366.

While I'm on PTO for the next two weeks, I'll probably be checking-in on
a few things.  If there's a good way to test this I expect I'll have a
compiler with the 98366 fix handy today/tomorrow.

jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: GCC 11 related build failure

[Jeff Law]

On 12/14/20 4:41 PM, Robert-André Mauchin wrote:
>
> Hello,
>
> While building Googletest, I've encourtered this failure:
>
>
>> /builddir/build/BUILD/aom-2.0.1/third_party/googletest/src/googletest/include/gtest/gtest-printers.h:
>> In instantiation of 'void
>> testing_internal::DefaultPrintNonContainerTo(const T&, std::ostream*)
>> [with T = {anonymous}::Y4mTestParam; std::ostream =
>> std::basic_ostream]':
>> /builddir/build/BUILD/aom-2.0.1/third_party/googletest/src/googletest/include/gtest/gtest-printers.h:465:49:
>>   
>> required from 'void
>> testing::internal::DefaultPrintTo(testing::internal::WrapPrinterType,
>> const T&, std::ostream*) [with T = {anonymous}::Y4mTestParam;
>> std::ostream = std::basic_ostream]'
>> /builddir/build/BUILD/aom-2.0.1/third_party/googletest/src/googletest/include/gtest/gtest-printers.h:500:17:
>>   
>> required from 'void testing::internal::PrintTo(const T&,
>> std::ostream*) [with T = {anonymous}::Y4mTestParam; std::ostream =
>> std::basic_ostream]'
>> /builddir/build/BUILD/aom-2.0.1/third_party/googletest/src/googletest/include/gtest/gtest-printers.h:676:12:
>>   
>> required from 'static void
>> testing::internal::UniversalPrinter::Print(const T&,
>> std::ostream*) [with T = {anonymous}::Y4mTestParam; std::ostream =
>> std::basic_ostream]'
>> /builddir/build/BUILD/aom-2.0.1/third_party/googletest/src/googletest/include/gtest/gtest-printers.h:866:30:
>>   
>> required from 'void testing::internal::UniversalPrint(const T&,
>> std::ostream*) [with T = {anonymous}::Y4mTestParam; std::ostream =
>> std::basic_ostream]'
>> /builddir/build/BUILD/aom-2.0.1/third_party/googletest/src/googletest/include/gtest/gtest-printers.h:794:19:
>>   
>> required from 'static void
>> testing::internal::UniversalTersePrinter::Print(const T&,
>> std::ostream*) [with T = {anonymous}::Y4mTestParam; std::ostream =
>> std::basic_ostream]'
>> /builddir/build/BUILD/aom-2.0.1/third_party/googletest/src/googletest/include/gtest/gtest-printers.h:914:44:
>>   
>> required from 'std::string testing::PrintToString(const T&) [with T =
>> {anonymous}::Y4mTestParam; std::string =
>> std::__cxx11::basic_string]'
>> /builddir/build/BUILD/aom-2.0.1/third_party/googletest/src/googletest/include/gtest/internal/gtest-param-util.h:585:28:
>>   
>> required from 'void
>> testing::internal::ParameterizedTestSuiteInfo::RegisterTests()
>> [with TestSuite = {anonymous}::Y4mVideoWriteTest]'
>> /builddir/build/BUILD/aom-2.0.1/third_party/googletest/src/googletest/include/gtest/internal/gtest-param-util.h:536:8:
>>   
>> required from here
>> /builddir/build/BUILD/aom-2.0.1/third_party/googletest/src/googletest/include/gtest/gtest-printers.h:283:7:
>> error: no match for 'operator<<' (operand types are 'std::ostream'
>> {aka 'std::basic_ostream'} and 'const {anonymous}::Y4mTestParam')
>>   283 |   *os << value;
>>   |   ^~~~
It's an upstream GCC bug:


https://gcc.gnu.org/PR98231 Marek just got back from some PTO today, I'm
sure he'll be on it relatively soon. Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: dav1d SONAME bump

[Jeff Law]

On 12/13/20 10:04 PM, Robert-André Mauchin wrote:
>
>  - VLC: Fails with error: 'numeric_limits' is not a member of 'std'
> Reported upstream:
> https://trac.videolan.org/vlc/ticket/25325
> I have a patch to try for this.
> I have CC RPMFusion.
Yea, this is a common problem with the introduction of gcc-11.  #include
 is the right solution.


jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: HEADS UP: OpenEXR + ilmbase = (new) openexr

[Jeff Law]

On 12/13/20 9:03 AM, Richard Shaw wrote:
> On Sun, Dec 13, 2020 at 9:20 AM Jeff Law  <mailto:l...@redhat.com>> wrote:
>
>
> So I've committed fixes for kdelibs.  So you shouldn't have to worry
> about that anymore.
>
> Also note I fixed an issue with OpenEXR this weekend that you
> don't want
> to lose.  Essentially it uses the wrong format directive and as a
> result
> can create a bogus compression table on some platforms (ppc64le for
> example).  gcc-11 will detect the bogus values in the compression
> table
> and issue a diagnostic.    See OpenEXR-gcc11.patch in rawhide.
>
>
> Just grabbed it and updated it for 2.5.3 (offset 10 lines).
>
> Trying a new scratch build and see if it addresses an issue with the
> built in testing failing on aarch64 and s390x.
I doubt it'll help with those.

The bogus values in the compress table would cause a compile-time
failure with gcc-11.  I think with gcc-10 the bogus values would get
silently truncated down to the right value anyway.

jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: HEADS UP: OpenEXR + ilmbase = (new) openexr

[Jeff Law]

On 12/12/20 1:38 PM, Richard Shaw wrote:
> On Thu, Dec 10, 2020 at 1:52 PM Richard Shaw  > wrote:
>
> For posterity here's the affected packages as far as I can tell:
>
> lembic
> aqsis
> blender
> calligra
> cinelerra-gg
> CTL
> darktable
> Field3D
> freeimage
> gegl04
> gimp
> gmic
> gstreamer1-plugins-bad-free
> gtatool
> hugin
> ilmbase
> ImageMagick
> kdebase3
> kdelibs
> kdelibs3
> kde-runtime
> kf5-kimageformats
> kio-extras
> krita
> luminance-hdr
> luxcorerender
> opencv
> OpenEXR
> OpenEXR_Viewers
> OpenEXR_Viewers-nonfree
> OpenImageIO
> OpenSceneGraph
> openshadinglanguage
> openvdb
> pfstools
> povray
> prusa-slicer
> synfig
> synfigstudio
> vigra
> vips
> YafaRay
>
>
> All deps built fine in COPR with the following exceptions:
>
> 1. Random strange aarch64 failures on some packages, resubmitting
> usually worked, so I'll ignore those errors.
> 2.  kdelibs is FTBFS probably due to new error reported by GCC 11,
> https://bugzilla.redhat.com/show_bug.cgi?id=1907031
> .
> 3. kdebase3 needs kdelibs
> 4. gstreamer1-plugins-bad-free refused to build in COPR due to missing
> deps but a scratch build completed fine... Not sure what's going on here.
>
> Overall there were not any weird issues or need to port any of them so
> I plan to move forward with the plan.
So I've committed fixes for kdelibs.  So you shouldn't have to worry
about that anymore.

Also note I fixed an issue with OpenEXR this weekend that you don't want
to lose.  Essentially it uses the wrong format directive and as a result
can create a bogus compression table on some platforms (ppc64le for
example).  gcc-11 will detect the bogus values in the compression table
and issue a diagnostic.    See OpenEXR-gcc11.patch in rawhide.

jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: HEADS UP: OpenEXR + ilmbase = (new) openexr

[Jeff Law]

On 12/12/20 1:38 PM, Richard Shaw wrote:
> On Thu, Dec 10, 2020 at 1:52 PM Richard Shaw  > wrote:
>
> For posterity here's the affected packages as far as I can tell:
>
> lembic
> aqsis
> blender
> calligra
> cinelerra-gg
> CTL
> darktable
> Field3D
> freeimage
> gegl04
> gimp
> gmic
> gstreamer1-plugins-bad-free
> gtatool
> hugin
> ilmbase
> ImageMagick
> kdebase3
> kdelibs
> kdelibs3
> kde-runtime
> kf5-kimageformats
> kio-extras
> krita
> luminance-hdr
> luxcorerender
> opencv
> OpenEXR
> OpenEXR_Viewers
> OpenEXR_Viewers-nonfree
> OpenImageIO
> OpenSceneGraph
> openshadinglanguage
> openvdb
> pfstools
> povray
> prusa-slicer
> synfig
> synfigstudio
> vigra
> vips
> YafaRay
>
>
> All deps built fine in COPR with the following exceptions:
>
> 1. Random strange aarch64 failures on some packages, resubmitting
> usually worked, so I'll ignore those errors.
> 2.  kdelibs is FTBFS probably due to new error reported by GCC 11,
> https://bugzilla.redhat.com/show_bug.cgi?id=1907031
> .
Yea, this is a new diagnostic starting with gcc-11.  It's almost
certainly the case that it should be using an eq/neq test rather than an
ordered test.    I've fixed numerous instances of this problem
throughout Fedora rawhide.  kdelibs didn't get on my "to fix" list
because it was failing with gcc-10 elsewhere in the build -- I very much
focus on things that work the old compiler, but fail with the new one.

I'll try to take a looksie at both.  The ordered comparison issue is
usually trivial to fix.

Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Nothing provides libgnat-10.so()(64bit)

[Jeff Law]

On 12/7/20 1:48 PM, Pavel Zhukov wrote:
> Hi Jeff.
> Thank you for rebuilding xmlada. I'll continue from there as I want to
> use this bootstrap to update packages to newest upstream version
> anyway .
ACK.  That'll definitely help.  I'll keep my gprbuild bits here (they
aren't complete/working anyway)

jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Nothing provides libgnat-10.so()(64bit)

[Jeff Law]

On 12/7/20 11:07 AM, Jakub Jelinek wrote:
> On Mon, Dec 07, 2020 at 06:29:51PM +0100, Miro Hrončok wrote:
>> Hello. Apparently, many packages in rawhide require libgnat-10.so()(64bit):
> I'm not a provenpackager, so I'm afraid I can't do that myself.
> Perhaps the maintainers can just rebuild them themselves?
And note that some of these are going to require fixes to the packages
themselves as the Ada front-end changed and some of them no longer
build.  IIRC zlib and templates_parser are the most important to get
rebuilt and I think those "just work".

I'll try to get those rebuilt so that the downstream things have a
better chance of working :-)

jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: LTO running out of memory on ARMv7 builders

[Jeff Law]

On 12/1/20 12:05 PM, Richard W.M. Jones wrote:
> This Rawhide build:
> https://koji.fedoraproject.org/koji/taskinfo?taskID=56536942
> seems to have failed because of LTO running out of memory on armv7.
> The error is:
>
>   lto1: out of memory allocating 104 bytes after a total of 2966994944 bytes
>   lto-wrapper: fatal error: gcc returned 1 exit status
>   compilation terminated.
>   /usr/bin/ld: error: lto-wrapper failed
>   collect2: error: ld returned 1 exit status
>
> In fact I had noticed before that this binary takes quite a long time
> to compile with LTO even on x86-64.  It's not especially large (7 MB
> with debug symbols, 2.2 MB fully stripped).
>
> But I suppose one thing which is special about it is that it's a
> "mixed" C and OCaml binary, but as far as I know the OCaml objects
> should be effectively ignored from an LTO point of view.
>
> I can't see why it should need to allocate 2GB(!) in order to LTO it.
Note that the LTO phase is parallelized, so there may be multiple
compilations/links in flight which can stress a builder with low
physical or virtual memory.

What I've been pondering for this is to have a macro that a package
could set to indicate that throttling is needed.  Probably as a first
cut it would throttle to half of what smp_mflags allows.  But that's
just a guess as to where to start.

In the immediate term go ahead and disable lto via the usual mechanism
and a comment that it's being done because of memory requirements.

%global _lto_cflags %{nil} or %define _lto_cflags %{nil}

I've got a spec file scanner that will flag the package as opting out. 
Knowing that I can use it as a testcase for the throttling mechanism.

jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: gcc-11 rawhide mock config?

[Jeff Law]

On 11/25/20 5:41 AM, Gabriel L. Somlo wrote:
> Hi,
>
> Someone dropped a gcc-11 specific patch into a package I
> (co-)maintain:
>
> https://src.fedoraproject.org/rpms/yosys/c/ff37aa8a48b430546f4831ece89896cccfe7b1a1?branch=master
>
> which really rather belongs upstream. I'd like to test it before
> pushing an upstream PR myself, but a cursory googling for a mock
> config file for gcc-11 rawhide didn't return anything useful.
>
> Any pointers to how I could mock-build a package with gcc-11?
There's an ELN side tag that you could use with gcc-11 in it
(eln-build-side-32479).  The change proposal to make gcc-11 snapshots
available in Rawhide was made earlier this week, but hasn't made it to
FESCO yet.

So the failure that patch fixes is:

passes/sat/freduce.cc: In member function 'void 
{anonymous}::PerformReduction::analyze(std::vector >&, 
std::map&, std::vector&, std::string, std::string)':
passes/sat/freduce.cc:457:55: error: 'numeric_limits' is not a member of
'std'   457 | out_depth[idx] = 
std::numeric_limits::max();
  |   ^~
passes/sat/freduce.cc:457:70: error: expected primary-expression before 'int'
  457 | out_depth[idx] = 
std::numeric_limits::max();
  |  ^~~
passes/sat/freduce.cc: In member function 'void 
{anonymous}::PerformReduction::analyze(std::vector
 >&, int)':
passes/sat/freduce.cc:531:55: error: 'numeric_limits' is not a member of 'std'
  531 | out_depth[idx] = 
std::numeric_limits::max();
  |   ^~
passes/sat/freduce.cc:531:70: error: expected primary-expression before 'int'
  531 | out_depth[idx] = 
std::numeric_limits::max();
  |  ^~~
make: *** [Makefile:599: passes/sat/freduce.o] Error 1




Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: qemu & LTO

[Jeff Law]

On 11/5/20 8:24 AM, Richard W.M. Jones wrote:
> We discovered a few days ago that LTO broke qemu on aarch64.
>
> The original bug reported was:
>
>   https://bugzilla.redhat.com/show_bug.cgi?id=1893892
>
> But actually looking at the build.log[1] we see another assertion
> failure in the test suite.  (Unfortunately although we run the test
> suite, the spec file was ignoring the result so the broken build
> escaped into Rawhide.)
>
> Because qemu is a complicated piece of software we're not clear if the
> bugs found are general bugs in LTO, bugs which are specific to LTO on
> aarch64, or bugs in qemu which are exposed by optimizations made
> possible by LTO.
>
> One thing we do suspect is that this could be the tip of the iceberg
> since the qemu test suite only tests a tiny fraction of the code.
>
> LTO has been disabled across all arches for now.  See the list of
> latest commits that Dan has added:
>
> https://src.fedoraproject.org/rpms/qemu/commits/master

Right.  And it's on my list to investigate (probably sometime in early
2021 given other pressing commitments).  FWIW, there's ~150 LTO opt-outs
on that list to investigate.  While I don't think we need to fix &
enable everything, I do want to thoroughly *understand* every issue and
get appropriate bugs filed.  FWIW, I have a Fedora spec file scanner
which notes all the opted-out packages so I know what needs investigation.


What I've typically seen for execution/testsuite failures have mostly
been package issues, not LTO issues.  Probably the biggest thing is that
LTO can inline across translation units.  So things like poorly written
ASMs that under-specified their dataflow suddenly get inlined and that
under-specification becomes critically important.  The other thing I've
seen a few times is ordering of static constructors for C++ -- LTO
can/will change that and code which relies on specific ordering (which
isn't defined) can fail.   On the LTO side, symbol visibility has been
the biggest headache and they can be insanely hard to track down.

I expect at least some of the opt outs, particularly the target
dependent ones are actually latent codegen issues that LTO happens to
expose, but aren't issues in LTO itself.

There's little doubt we'll find more issues as we work through the
opt-outs.  But that's also why we pushed so hard to get the vast
majority of things up with LTO in F33.  Soak time in Fedora is critical
in my mind.


jeff




.  LTO can change the ordering of static constructors for C++, or
aggressively inline across translation units exposing things like poorly
written asms.  We've seen a small number of (insanely annoying) issues
with symbol visibility in the LTO path itself.



I thought we had LTO disabled for QEMU until I could sit down and dig
into it?!?


jeff

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Rawhide build failure on strange archs

[Jeff Law]

On 11/7/20 1:38 PM, Orion Poplawski wrote:
> On 11/7/20 1:26 PM, Steve Dickson wrote:
>> Hello,
>>
>> I'm getting a build failure on the armv7hl arch
>> and the i686 arch, which do not make much sense.
>>
>> The build is [1] and only those arche are  complaining about an
>> sprintf() statement.
>> The rest of the arches are fine with the statment... %99.9 of the arches
>> that are used today... I didn't even realize i686 was still supported!
>>
>> The is the failure:
>> conffile.c: In function 'conf_init_dir':
>> conffile.c:707:22: error: '%s' directive writing between 6 and
>> 2147483645 bytes into a region of size between 4095 and 4096
>> [-Werror=format-overflow=]
>>    707 |   sprintf(fname, "%s/%s", dname, d->d_name);
>>    |  ^~
>> In file included from /usr/include/stdio.h:866,
>>   from conffile.c:45:
>> /usr/include/bits/stdio2.h:38:10: note: '__sprintf_chk' output 8 or
>> more bytes (assuming 2147483648) into a destination of size 4097
>>     38 |   return __builtin___sprintf_chk (__s, __USE_FORTIFY_LEVEL - 1,
>>    |  ^~
>>     39 |   __bos (__s), __fmt, __va_arg_pack ());
>>    |   ~
>>
>> So I change the sprintf() to an snprintf() [2] guaranteeing no
>> overflow and I got the same failure. So it is something esoteric
>> about those arches... that I'm missing...
>>
>> Anybody have clue as to what is going on??
>>
>> tia,
>>
>> steved.
>
> I think the compiler is trying to keep track of how large string
> pointed to by d->d_name could be.  You'll need to look into how that
> is created.

That is precisely what it is doing.


>
> No idea why snprintf() doesn't help.

Likewise.  I recommend filing an upstream GCC bug report.  Feel free to
cc l...@redhat.com and mse...@redhat.com on that upstream bug.


jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: HEADSUP: libsepol and libsemanage soname bump

[Jeff Law]

On 11/5/20 7:59 AM, Petr Lautrbach wrote:
> On Wed, Nov 04, 2020 at 09:47:50AM +0100, Petr Lautrbach wrote:
>> Hi,
>>
>> in order to prevent backward compatibility libsepol and libsemanage used had 
>> few
>> symbols defined twice and used symbol versioning for them. But when LTO was
>> enabled these symbols were completely dropped during compilation, see
>> https://github.com/SELinuxProject/selinux/issues/245
>>
>> In order to fix it, it was decided to drop these duplicate symbols and also 
>> long
>> time deprecated symbols and therefore sonames of libsepol and libsemanage 
>> were
>> bumped.
>>
>> The following SELinux userspace components are built and prepared to be 
>> merged in
>> "f34-build-side-33413" side tag:
>>
>> selinux-policy-3.14.7-7.fc34
>> setools-4.4.0-0.1.20201102git05e90ee.fc34
>> checkpolicy-3.1-4.fc34
>> policycoreutils-3.1-5.fc34
>> libsemanage-3.1-4.fc34
>> libselinux-3.1-4.fc34
>> libsepol-3.1-4.fc34
>>
>> There are few other components which needs to be rebuild:
>>
>> parted - for some reason it links to libsepol even though I haven't found a 
>> code
>>   which would use it. I've proposed patch upstream
>>   
>> https://alioth-lists.debian.net/pipermail/parted-devel/2020-November/005500.html
>>
>> shadow-utils - https://src.fedoraproject.org/rpms/shadow-utils/pull-request/6
>> sssd - https://src.fedoraproject.org/rpms/sssd/pull-request/7
>>
>> As none of packages which require either libsepol or libsemanage use dropped
>> symbols and in order not to break build root during soname bumps I've added 
>> temporary
>> subpackages with original library versions - libsepol-compat with 
>> libsepol.so.1
>> and libsemanage-compat with libsemanage.so.1. These subpackage will be 
>> dropped
>> as soon as everything is rebuilt in Rawhide.
>>
>> I've sucessfuly built all packages also in my COPR repository
>> https://copr.fedorainfracloud.org/coprs/plautrba/selinux-fedora/builds/
>>
>> If there's no objection I'd like to merge the side tag to rawhide as soon as 
>> possible.
>>
> This is merged now 
> https://bodhi.fedoraproject.org/updates/FEDORA-2020-44f878be7e

Thanks!  Good to have those off my LTO list as well :-)


jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 34 Change proposal: Remove make from BuildRoot (System-Wide Change)

[Jeff Law]

On 11/4/20 2:06 PM, Tom Stellard wrote:
> On 11/4/20 3:57 PM, Jakub Jelinek wrote:
>> On Wed, Nov 04, 2020 at 03:51:40PM -0500, Neal Gompa wrote:
 Well, gcc really should have either weak or strong dependency on
 make too
 given that -flto is now used everywhere.
>>>
>>> The goal of this change seems to include removal of Make as a
>>> dependency for the LTO wrapper used by GCC.
>>
>> That is definitely not something that really happened in GCC, the only
>> change that has been done is to make sure that gcc -flto doesn't fail
>> because of missing make, but missing make will just mean that the
>> compilation will be significantly slower.
>>
>
> My understanding was that the performance of gcc -flto when using
> something other than make (e.g. ninja) to build was the same whether
> or not you have make installed[1].  Is this correct?
>
> I thought you had to be using make to get the parallelization benefits
> of -flto=auto.

It will fall back to #cpu threads available if the jobserver isn't
available.  At least that's how it's supposed to work.


jeff

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: ld segfaults on rawhide

[Jeff Law]

On 11/3/20 9:56 AM, Kevin Fenzi wrote:
> On Tue, Nov 03, 2020 at 09:57:48AM -0600, Justin Forbes wrote:
>> On Mon, Nov 2, 2020 at 10:38 AM Justin Forbes  wrote:
>>> On Sat, Oct 31, 2020 at 10:32 AM Jeff Law  wrote:
>>>>
>>>> On 10/31/20 9:13 AM, Christoph Junghans wrote:
>>>>> Hi,
>>>>>
>>>>> I am getting the following error on all archs on rawhide:
>>>>> collect2: fatal error: ld terminated with signal 11 [Segmentation
>>>>> fault], core dumped
>>>>> in https://koji.fedoraproject.org/koji/taskinfo?taskID=54629411
>>>>>
>>>>> any ideas?
>>>> Given it's happening on multiple architectures, I'd guess its a linker
>>>> bug of some kind.
>>>>
>>> Definitely seems to be. Killed every arch for the kernel builds today as 
>>> well.
>>>
>> Would it be possible/wise to untag the bad binutils build from rawhide
>> until an answer is found here?
> Well, it may already be out in yesterdays rawhide?
>
> I see:
> https://koji.fedoraproject.org/koji/buildinfo?buildID=1637660
> landed today... 
>
> Does that one work any better?

I ping'd Nick on this issue about an hour ago and he indicated it was
now fixed in rawhide.


jeff

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: ld segfaults on rawhide

[Jeff Law]

On 10/31/20 9:13 AM, Christoph Junghans wrote:
> Hi,
>
> I am getting the following error on all archs on rawhide:
> collect2: fatal error: ld terminated with signal 11 [Segmentation
> fault], core dumped
> in https://koji.fedoraproject.org/koji/taskinfo?taskID=54629411
>
> any ideas?

Given it's happening on multiple architectures, I'd guess its a linker
bug of some kind.


Nick (on-cc)?  Thoughts here?


Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: [ELN] gcc is going to be updated to gcc11 in the ELN buildroot ahead of Rawhide

[Jeff Law]

On 10/28/20 7:29 AM, Jonathan Wakely wrote:
> On 28/10/20 13:31 +0100, Florian Weimer wrote:
>> * Jonathan Wakely:
>>
>>> Dropping GCC 11 into rawhide now would mean I can't make certain
>>> ABI-breaking changes to the C++20 library in upstream GCC, because it
>>> would be landing on real users' machines. Which means I lose several
>>> weeks of GCC's stage 1 development. No thanks.
>>
>> This is for C++20 library support only, right?
>
> Right.
>
>> Not much software in Fedora uses the C++ standard library (even at older
>> C++ versions), so impact on Fedora itself should be limited.
>
> On Fedora iteself, yes. But not necessarily on users compiling their
> own code (or other third-party libraries) using the system compiler.
>
>> I can help you with diagnosing required ABI transitions and package
>> rebuilds.
>
> I'd rather just be able to change things freely during stage 1 :-)

The only thing on the table in the immediate term is to start dropping
gcc snapshots into rawhide after stage1 development closes -- ie mid
Nov.  So it shouldn't impact your desire to be able to break things
during gcc stage1 :-)


While I think we should seriously consider even earlier drops, that
would be in the gcc-12/F36 timeframe and would require a distinct change
proposal and I think it would be significantly more controversial.


Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: [ELN] gcc is going to be updated to gcc11 in the ELN buildroot ahead of Rawhide

[Jeff Law]

On 10/23/20 5:55 AM, Clement Verna wrote:
>
>
> On Thu, 22 Oct 2020 at 14:28, Aleksandra Fedorova  > wrote:
>
> Hi, all,
>
> this is the informational message, no action required.
>
> Upon agreement between gcc maintainers and ELN SIG we would like to
> switch ELN buildroot to use GCC11 ahead of Fedora Rawhide.
>
> Though ELN is defined as the buildroot where Fedora Rawhide code is
> rebuilt into EL-like environment, in the ELN proposal we also
> mentioned that ELN can be used to test certain buildroot-related
> features on the side so it doesn't block Fedora Rawhide development.
>
> We think that GCC11 is one such feature, where we can benefit from
> testing it first on a small subset of the Fedora content in a separate
> environment.
>
>
> I think this is one of the great benefits of having ELN and being able
> to use it to start integrating such changes. I am looking forward,
> seeing if that makes it easier for GCC11 to land in rawhide, I would
> be nice if you could share the issues that were caught during that
> work in ELN.

We use the experiences found during Fedora builds to populate some of
the upstream GCC release documentation.  ie, what kinds of issues are we
seeing in real world code.  For example:


https://gcc.gnu.org/gcc-10/porting_to.html


There's a document for gcc-11 which has bullet items for things we've
seen in in our Fedora testing (and looking at it, there's a couple
things that need to be added :-).  But we haven't dropped in examples,
it's just the initial bullet list.

https://gcc.gnu.org/gcc-11/porting_to.html


___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: [ELN] gcc is going to be updated to gcc11 in the ELN buildroot ahead of Rawhide

[Jeff Law]

On 10/23/20 8:01 AM, Dominik 'Rathann' Mierzejewski wrote:
> On Friday, 23 October 2020 at 14:45, Aleksandra Fedorova wrote:
> [...]
>> You can blame me for being not clear enough, if you want, but I'd
>> rather move on and use the current GCC11 work as an example which
>> shows the real power of the ELN proposal, and the real benefit for
>> Fedora which it brings.
>>
>> Despite the accusation of us bypassing the Fedora Change process, we
>> are doing a different thing.
>>
>> GCC11 will definitely go through the Change process as usual.
> That's reassuring.

Absolutely.

And as I've mentioned before on this thread, we're trying to figure out
how to drop that change in earlier than we have in the past.   There's
technical as well as non-technical concerns.  The time period Jakub and
myself were looking at was dropping it in at the end of stage1 upstream
gcc development which would be mid Nov.  But we both feel it's
imperative that the implications be discussed here and that the change
in procedure gets highlighted in the usual change proposal.


>> This activity could have been done internally in RHEL, or externally
>> in some upstream working groups. But ELN now allows us to do this work
>> in public in Fedora, and invite Fedora community to join it, if they
>> _want_.
> How can we join, then? How is this better than doing this, say, in COPR?
> Or a rawhide side-tag.

Right now the build is on a side tag and hasn't been merged into ELN (to
the best of my knowledge).  Once the bits are merged in then I expect
anyone can join in the "fun".


>
>> As we promised by the ELN Change we have provided the platform for GCC
>> upstream, RHEL downstream and Fedora community to collaborate on the
>> work for Fedora, and motivation for Red Hat to sponsor this effort.
> So far I haven't seen any clear instructions on how I can, say, rebuild
> my packages with GCC11 to catch any fall-out early. Or where you're
> going to publish (if at all) the results of any rebuilds you'll perform.
We don't generally publish them -- though we've certainly discussed it
in the past and the only impediment is my time.  The jenkins server
which drives my testing is on Red Hat's internal network, but there's a
publisher that would allow us to push the build info out to a public
facing server.  There's nothing inherently private and/or secret about
the work.  We also use that jenkins system to do wide scale testing of
GCC work before it lands in upstream GCC.


If you have specific packages in mind, I'm happy to let you know their
build status with gcc-11.  It's just some clicking.


jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: [ELN] gcc is going to be updated to gcc11 in the ELN buildroot ahead of Rawhide

[Jeff Law]

On 10/22/20 9:28 AM, Petr Pisar wrote:
> On Thu, Oct 22, 2020 at 09:20:28AM -0600, Jeff Law wrote:
>> On 10/22/20 8:27 AM, Petr Pisar wrote:
>>> On Thu, Oct 22, 2020 at 08:10:02AM -0600, Jeff Law wrote:
>>>> I do think we need to make it easier for a Fedora package maintainer to
>>>> get the gcc-11 bits so that if there's a need to debug a bad interaction
>>>> between gcc-11 and a package they can.
>>>>
>>> gcc-11 was built into a side tag. Each build target has a repository in 
>>> Koji.
>>> This one is
>>> <http://kojipkgs.fedoraproject.org/repos/eln-build-side-32479/latest/$basearch>.
>>>
>>> If you worry about ELN-Fedora compatibility, create a new side tag 
>>> intheriting
>>> from f34 and rebuilt gcc-11 there. Or build a module that anyone interested
>>> can enable on his system.
>> Sorry, I wasn't terribly clear.
>>
>>
>> So last year when I was testing gcc-10 against Fedora one of the
>> recurring issues we had was that if I needed input from a package
>> maintainer on an issue flagged by gcc-10 we had no good way for the
>> package maintainer to get gcc-10 rpms to do any investigation on their own.
>>
>>
>> With the gcc-11 side tag build and eventual landing in ELN that issue
>> should be much better.  So if I need to sync with a package maintainer
>> on an issue, we have a way to make that happen.
>>
> I see. This year it's indeed more accessible. E.g. I've just installed it
> from the side tag repository to my Rawhide and verifired that perl builds fine
> with GCC 11 :)

:-)


I already verified that, repeatedly.  On Sept 7th, Sept 20th, Oct 1st
and Oct 15th.  I figure the next build of perl with gcc-11 in my tester
should land in about a week ;-)


Jeff

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: [ELN] gcc is going to be updated to gcc11 in the ELN buildroot ahead of Rawhide

[Jeff Law]

On 10/22/20 8:27 AM, Petr Pisar wrote:
> On Thu, Oct 22, 2020 at 08:10:02AM -0600, Jeff Law wrote:
>> I do think we need to make it easier for a Fedora package maintainer to
>> get the gcc-11 bits so that if there's a need to debug a bad interaction
>> between gcc-11 and a package they can.
>>
> gcc-11 was built into a side tag. Each build target has a repository in Koji.
> This one is
> <http://kojipkgs.fedoraproject.org/repos/eln-build-side-32479/latest/$basearch>.
>
> If you worry about ELN-Fedora compatibility, create a new side tag intheriting
> from f34 and rebuilt gcc-11 there. Or build a module that anyone interested
> can enable on his system.

Sorry, I wasn't terribly clear.


So last year when I was testing gcc-10 against Fedora one of the
recurring issues we had was that if I needed input from a package
maintainer on an issue flagged by gcc-10 we had no good way for the
package maintainer to get gcc-10 rpms to do any investigation on their own.


With the gcc-11 side tag build and eventual landing in ELN that issue
should be much better.  So if I need to sync with a package maintainer
on an issue, we have a way to make that happen.


Of course, dropping gcc-11 into rawhide earlier helps this issue too :-)

Jeff

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: [ELN] gcc is going to be updated to gcc11 in the ELN buildroot ahead of Rawhide

[Jeff Law]

On 10/22/20 8:09 AM, Daniel P. Berrangé wrote:
> On Thu, Oct 22, 2020 at 04:03:33PM +0200, Aleksandra Fedorova wrote:
>> Hi, Daniel,
>>
>> On Thu, Oct 22, 2020 at 3:01 PM Daniel P. Berrangé  
>> wrote:
>>> On Thu, Oct 22, 2020 at 02:27:13PM +0200, Aleksandra Fedorova wrote:
 Hi, all,

 this is the informational message, no action required.

 Upon agreement between gcc maintainers and ELN SIG we would like to
 switch ELN buildroot to use GCC11 ahead of Fedora Rawhide.

 Though ELN is defined as the buildroot where Fedora Rawhide code is
 rebuilt into EL-like environment, in the ELN proposal we also
 mentioned that ELN can be used to test certain buildroot-related
 features on the side so it doesn't block Fedora Rawhide development.

 We think that GCC11 is one such feature, where we can benefit from
 testing it first on a small subset of the Fedora content in a separate
 environment.
>>> I'm not very enthusiastic about this change.
>>>
>>> Fedora maintainers can largely ignore ELN right now, because if stuff
>>> works in rawhide, it will generally work in ELN, and someone else is
>>> taking care of ELN builds.
>>>
>>>
>>> New GCC releases almost always trigger new compile warnings or bugs
>>> in code. So by pushing GCC 11 into ELN, it feels like we're making
>>> it much more likely that ELN builds will fail, and now Fedora
>>> maintainers have to debug ELN specific problems that won't reproduce
>>> in rawhide branches :-(
>> Expectations for Fedora maintainers does not change.
>> ELN is a development playground. Think about sidetag, with slightly
>> better automation around it.
>>
>> We provide ELN as the opportunity, option to play with early releases
>> of the GCC11 on the side. We are not requiring Fedora maintainers to
>> participate, we are inviting people who may be interested in this
>> work.
> As Fedora maintainer I've been sent details of ELN failures / bugs, and
> asked to deal with fixes for ELN branches, so there's clearly an expectation
> placed on Fedora maintainers to be engaged in ELN.

And that's unfortunate.  I've tried to signal to the ELN/Bakery folks
that they should be contacting me first as any build failure related to
teh compiler change I've probably already seen in one form or another. 
But it doesn't seem to have sunk in.


jeff

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: [ELN] gcc is going to be updated to gcc11 in the ELN buildroot ahead of Rawhide

[Jeff Law]

On 10/22/20 7:00 AM, Daniel P. Berrangé wrote:
> On Thu, Oct 22, 2020 at 02:27:13PM +0200, Aleksandra Fedorova wrote:
>> Hi, all,
>>
>> this is the informational message, no action required.
>>
>> Upon agreement between gcc maintainers and ELN SIG we would like to
>> switch ELN buildroot to use GCC11 ahead of Fedora Rawhide.
>>
>> Though ELN is defined as the buildroot where Fedora Rawhide code is
>> rebuilt into EL-like environment, in the ELN proposal we also
>> mentioned that ELN can be used to test certain buildroot-related
>> features on the side so it doesn't block Fedora Rawhide development.
>>
>> We think that GCC11 is one such feature, where we can benefit from
>> testing it first on a small subset of the Fedora content in a separate
>> environment.
> I'm not very enthusiastic about this change.
>
> Fedora maintainers can largely ignore ELN right now, because if stuff
> works in rawhide, it will generally work in ELN, and someone else is
> taking care of ELN builds.
>
> New GCC releases almost always trigger new compile warnings or bugs
> in code. So by pushing GCC 11 into ELN, it feels like we're making
> it much more likely that ELN builds will fail, and now Fedora
> maintainers have to debug ELN specific problems that won't reproduce
> in rawhide branches :-(

True, but the GCC team (me in particular) have already been building
rawhide with gcc-11 snapshots and fixing these issues.


I do think we need to make it easier for a Fedora package maintainer to
get the gcc-11 bits so that if there's a need to debug a bad interaction
between gcc-11 and a package they can.

jeff

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: [ELN] gcc is going to be updated to gcc11 in the ELN buildroot ahead of Rawhide

[Jeff Law]

On 10/22/20 6:53 AM, Neal Gompa wrote:
> On Thu, Oct 22, 2020 at 8:27 AM Aleksandra Fedorova  
> wrote:
>> Hi, all,
>>
>> this is the informational message, no action required.
>>
>> Upon agreement between gcc maintainers and ELN SIG we would like to
>> switch ELN buildroot to use GCC11 ahead of Fedora Rawhide.
>>
>> Though ELN is defined as the buildroot where Fedora Rawhide code is
>> rebuilt into EL-like environment, in the ELN proposal we also
>> mentioned that ELN can be used to test certain buildroot-related
>> features on the side so it doesn't block Fedora Rawhide development.
>>
>> We think that GCC11 is one such feature, where we can benefit from
>> testing it first on a small subset of the Fedora content in a separate
>> environment.
>>
>> We would like to invite everyone to join this effort.
>>
>> The work is currently tracked on Github:
>> https://github.com/fedora-eln/eln/issues/8
>>
>> Once GCC11 is merged to the eln tag in koji, one would be able to use
>> it via, for example, mock or container environment:
>> quay.io/fedoraci/fedora:eln-x86_64
>>
>> For more info on ELN please refer to ELN Docs (as soon as I update
>> them, which hopefully happens later today):
>>
>> https://docs.fedoraproject.org/en-US/eln/
>>
> Why are you not just doing this in Rawhide? I feel like we've been
> screwed now because the whole point was that ELN branches weren't
> going to exist, and now we have one in the most important package!

From a timing standpoint we're trying to get ELN up and running with
gcc-11 right now.  The earliest Jakub was comfortable dropping gcc-11
snapshots into rawhide was after stage1 development closes for upstream
gcc (mid-Nov) and even that probably needs to be discussed with FESCO
and the wider Fedora dev community as it is potentially disrupting
(regardless of how much testing and fixing I've already done in rawhide
in preparation for gcc-11).  Delaying gcc-11 into ELN would throw
everything downstream of that off from a scheduling standpoint.


But I very much agree that GCC should be moving into rawhide earlier
than has been done in the past.  In an ideal world it would have gone in
just after F33 branched and updated regularly.

Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: [ELN] gcc is going to be updated to gcc11 in the ELN buildroot ahead of Rawhide

[Jeff Law]

On 10/22/20 7:02 AM, Kalev Lember wrote:
> On Thu, Oct 22, 2020 at 2:54 PM Neal Gompa  > wrote:
>
> On Thu, Oct 22, 2020 at 8:27 AM Aleksandra Fedorova
> mailto:al...@bookwar.info>> wrote:
> >
> > Hi, all,
> >
> > this is the informational message, no action required.
> >
> > Upon agreement between gcc maintainers and ELN SIG we would like to
> > switch ELN buildroot to use GCC11 ahead of Fedora Rawhide.
> >
> > Though ELN is defined as the buildroot where Fedora Rawhide code is
> > rebuilt into EL-like environment, in the ELN proposal we also
> > mentioned that ELN can be used to test certain buildroot-related
> > features on the side so it doesn't block Fedora Rawhide development.
> >
> > We think that GCC11 is one such feature, where we can benefit from
> > testing it first on a small subset of the Fedora content in a
> separate
> > environment.
> >
> > We would like to invite everyone to join this effort.
> >
> > The work is currently tracked on Github:
> > https://github.com/fedora-eln/eln/issues/8
> 
> >
> > Once GCC11 is merged to the eln tag in koji, one would be able
> to use
> > it via, for example, mock or container environment:
> > quay.io/fedoraci/fedora:eln-x86_64
> 
> >
> > For more info on ELN please refer to ELN Docs (as soon as I update
> > them, which hopefully happens later today):
> >
> > https://docs.fedoraproject.org/en-US/eln/
> 
> >
>
> Why are you not just doing this in Rawhide? I feel like we've been
> screwed now because the whole point was that ELN branches weren't
> going to exist, and now we have one in the most important package!
>
>
> "We're screwed" is a bit harsh, but beyond that, I second to what
> Neal said.
>
> Historically, new gcc releases have landed in rawhide right before a
> mass rebuild and then there's often been fallout in the mass rebuild
> due to the new compiler. And then afterwards there's a lot of crunch
> to get all of the fallout from the new compiler fixed quickly before
> the Beta release.
>
> I think it would be much smoother to introduce new gcc releases
> earlier in the cycle (as in, now would be a good time) to give
> packagers time to fix things up before the mass rebuild starts.

I'm very much trying to get things moving in this direction.  The
current model of waiting until just before the mass rebuild for even
numbered Fedora releases is far from ideal.  It's hard on the Fedora
maintainer community as a whole and it's hard on the GCC community. 
We'd both be better served with earlier drops of the development
versions of gcc into rawhide, IMHO.


While we may not get to the state that glibc is in (dropping in
development snapshots weekly), I do think we should be looking at a drop
of a gcc-11 snapshot into rawhide roughly at the same time that gcc-11's
stage1 development window closes (mid-Nov) with semi-regular updates
from that point.


Jeff


___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: glibc troubles in rawhide?

[Jeff Law]

On 10/20/20 5:14 PM, Adam Williamson wrote:
> On Tue, 2020-10-20 at 18:47 -0400, Neal Gompa wrote:
>> Can
>> we use our gating system to kick back builds that are obviously broken
>> that even the package manager stops working?
> Theoretically, yes, this is possible. We could add this and any other
> functional tests we liked to Fedora CI or openQA and gate on the
> results.
>
> Practically, it would at least require packagers to be conscientious
> about using side tags when building packages that need to be updated
> together, which currently they are not. It's still common practice to,
> say, build libfoo.so.n+1 first directly into rawhide, then rebuild
> libfoo's deps afterwards, without using a side tag. As long as people
> do that we will have trouble doing Rawhide gating on functional
> testing, because any test that uses a dep of libfoo will likely fail on
> the build of libfoo (because there won't be a rebuilt version of the
> dep to test with).
>
> To the point, this happened just recently with dnf and libdnf.
> An soname bump of libdnf was built directly into rawhide first:
>
> https://koji.fedoraproject.org/koji/buildinfo?buildID=1621613
>
> then dnf was built afterwards:
> https://koji.fedoraproject.org/koji/buildinfo?buildID=1621786
>
> any gating 'functional test of the package manager' would have blocked
> the libdnf build because the older dnf would not have worked with the
> new libdnf.
>
> Of course, I guess if we start doing that, it'll teach people to use
> side tags in a hurry. So maybe we should? :)

Perhaps.  In the immediate term, can someone untag the latest glibc
build (big sigh)?


Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: glibc troubles in rawhide?

[Jeff Law]

On 10/20/20 4:47 PM, Neal Gompa wrote:
> On Tue, Oct 20, 2020 at 6:44 PM Jeff Law  wrote:
>>
>> On 10/20/20 4:41 PM, Jerry James wrote:
>>> On Tue, Oct 20, 2020 at 4:38 PM Fabio Valentini  
>>> wrote:
>>>> Looks like the most recent glibc update in rawhide broke some stuff,
>>>> including running dnf:
>>> That looks like the same problem I wrote about this morning:
>>>
>>> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/Q5EL4U5DXCNAA3HSTRPR2XJGJWAGBZ2Z/
>>>
>>> If I'm reading the change correctly, everything referencing those
>>> symbols just needs a rebuild ... but that might be a significant chunk
>>> of the distribution.
>> Upstream glibc is almost certainly putting them back ;-)  I think ftimes
>> already went back in and the others are under discussion right now.
>>
> This is a serious violation of glibc's stated promise of infinite
> backwards compatibility. How in the world did this slip through? Can
> we use our gating system to kick back builds that are obviously broken
> that even the package manager stops working?
>
> Also, I'm surprised that reinstating symbols needs any discussion at
> all. The removals obviously have to be reverted, per their own
> infinite backwards compatibility promise.

They promise backwards compat for existing binaries.  They don't promise
backwards compat at the source level.  The distinction is often important.

The timeb related failures certainly fit in that category -- existing
binaries would continue to work, but nothing could be built that
referenced the timeb header file or the deprecated functions within it. 
Of course, once that change went live they realized that significant
source code still referenced that stuff so they reverted.

I can't comment on the  other symbols other than knowing upstream was
looking to reinstate them this morning -- I didn't dig into those as
they didn't impact the issue I was working on yesterday.


jeff

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: glibc troubles in rawhide?

[Jeff Law]

On 10/20/20 4:41 PM, Jerry James wrote:
> On Tue, Oct 20, 2020 at 4:38 PM Fabio Valentini  wrote:
>> Looks like the most recent glibc update in rawhide broke some stuff,
>> including running dnf:
> That looks like the same problem I wrote about this morning:
>
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/Q5EL4U5DXCNAA3HSTRPR2XJGJWAGBZ2Z/
>
> If I'm reading the change correctly, everything referencing those
> symbols just needs a rebuild ... but that might be a significant chunk
> of the distribution.

Upstream glibc is almost certainly putting them back ;-)  I think ftimes
already went back in and the others are under discussion right now.

jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)

[Jeff Law]

On 9/30/20 3:50 AM, Jan Kratochvil wrote:
> On Wed, 30 Sep 2020 01:31:29 +0200, Jeff Law wrote:
>> But the GCC community
>> doesn't really test that option and it's known to be broken with LTO.
> I haven't seen any GCC PR for -fdebug-types-section being broken with LTO.

 I'm not aware of one either.  But as Jakub has previously pointed out
debug-types-section is disabled when LTO is enabled.  I don't know the
details of why that is done.


>
> During one abigail diff I did not see any difference. I plan to run a full
> distribution abigail massrebuild+check as stated in the Change to exclude any
> possible incompatibilities. That would discover unfiled GCC problems with
> -fdebug-types-section.
>
> Also I do not see why fixing -fdebug-types-section should be anyhow difficult
> if the compiler can produce -fno-debug-types-section. I can also write
> postprocessor to get -fdebug-types-section if GCC is unable to do that.
> That would sure lose the -fdebug-types-section compilation time performance
> benefits.
>
>
>> And, not surprisingly, our team has had significant input on the options
>> we're using *and* the GCC implementation of those options.   We make
>> recommendations based on our experiences. That same experience would lead us
>> to recommending against -fdebug-types-section at this time.
> I think I have also some DWARF experience. Could you suggest what is wrong on
> -fdebug-types-section?

Your best bet is to discuss with Jakub and perhaps Jason.   They're far
more familiar with the debuginfo generation than I am.


>
>
>> It would certainly be good to improve the on-disk distro size.
> OK, going to file a Change to enable gcc -gz option (zlib section compression)
> as that will reduce on-disk *.debug size by 52.84%! Then we can disable both
> DWZ and -fdebug-types-section as those become pointless then.

Note Mark's reply in the other thread.  Section compression has
significant tradeoffs.


>
>
>> So the only paths forward I see are to either fix -fdebug-types-section or
>> improve dwz.
> And obviously much easier is to fix -fdebug-types-section than DWZ (if there
> are really any bugs in -fdebug-types-section, there are known bugs nobody
> wants to fix in DWZ).

I think you're making an unsubstantiated leap here.  Neither of us know
what's wrong with GCC LTO and debug-types-section and others are working
on dwz.


>
>
>> Putting my Red Hat hat on, I get pushback from PM on *any* size increases in
>> the RHEL space.
> When we start talking about RHEL (and CentOS) DWZ is completely pointless then
> as DWZ there saves only 0.28% of *-debuginfo.rpm (20MB of 7.2GB).
> Therefore approx. 0.14% of the distribution size.

Umm, we're fighting with PM these days over things in the 10M range. 
So, no it's not pointless.


>
>
>> As much as we'd like to be in a world were a 1% increase in distro
>> size doesn't matter, that's not the actual world we live in.
> Unfortunately DWZ cannot decrease RHEL size by that 1%.

I'm not asking it to.  I'm saying that sizes matter, even in cases where
you think they shouldn't. 


>
>
>> And our RHEL customers absolutey do care about the size of debuginfo
>> becuause it affects link times.
> System debuginfo format does not affect link times. Using DWZ during linking
> customer's applications definitely only increases linking time as it is an
> extra step. Not sure what do you talk about.

Most customers don't use dwz.  But they consume its output for the RPMs
that we provide.


Jeff

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: ELF section/file compression

[Jeff Law]

On 10/6/20 3:59 PM, Mark Wielaard wrote:
> Hi,
>
> Changing subject because this has nothing to do with that Change
> Proposal anymore.
>
> On Tue, Oct 06, 2020 at 01:49:13PM -0600, Jeff Law wrote:
>> However, I think it's perfectly valid to discuss zstd if folks wanted to
>> change the compression scheme for debug sections.  In fact, I'd claim
>> sticking with zlib, gzip,  bzip2, xz, 7z, etc is unwise.  The world has
>> moved and zstd seems like the place we should be.  In fact, we use it
>> for various things within GCC already.
> Personally I must admit that I am not really a fan of using ELF
> section/file compression. It makes it impossible to simply mmap the
> data in or to quickly read just a tiny bit because you first have to
> decompress (and allocate new memory) for it. IMHO .debug files are no
> different from other ELF files for which we would also not do this. We
> can just use rpm package compression to reduce the distro distribution
> size, but we should not (re)compress the install/on-disk files. That
> will just mean programs will create an extra cache of uncompressed
> files they need to consult frequently.

I'm not taking a position on whether or not we compress sections.  My
position is that if we're compressing them, then zstd seems like a
better solution than the others mentioned.  I certainly understand the
desire to just mmap in the stuff and move on.


[ ... ]

Secondly you can use ELF section compression. The ELF spec leaves room
> for adding new compression algorithms. The Chdr struct(s) contain a
> ch_type which describes the algorithm. Currently only one is
> specified, but there is a lot of room for expansion:
>
> /* Legal values for ch_type (compression algorithm).  */
> #define ELFCOMPRESS_ZLIB1  /* ZLIB/DEFLATE algorithm.  */
> #define ELFCOMPRESS_LOOS0x6000 /* Start of OS-specific.  */
> #define ELFCOMPRESS_HIOS0x6fff /* End of OS-specific.  */
> #define ELFCOMPRESS_LOPROC  0x7000 /* Start of processor-specific.  */
> #define ELFCOMPRESS_HIPROC  0x7fff /* End of processor-specific.  */
>
> So you could propose something on gnu-g...@sourceware.org for a GNU
> extension or at generic-...@googlegroups.com for a generic ELF one.
> And then get the ELF processing tools to adopt the new compression
> type.

ohhh, I didn't know it was baked in at this level.  Yea, if we're going
to do section compression with zstd, then it's clearly best to get it
officially supported at the ABI level.


jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: readelf seems broken in F33

[Jeff Law]

On 10/6/20 3:05 PM, Florian Weimer wrote:
> * Steve Grubb:
>
>> I was doing some binary analysis of files in F33 and have run across
>> something odd.
>>
>> readelf -s  /usr/sbin/auditd | grep GLIBC
>>
>> produces a lot of output like:
>>
>>182: 0 FUNCGLOBAL DEFAULT  UND [...]@GLIBC_2.2.5 
>> (3)
>>184: 0 FUNCGLOBAL DEFAULT  UND [...]@GLIBC_2.2.5 
>> (3)
>>185: 0 FUNCGLOBAL DEFAULT  UND [...]@GLIBC_2.2.5 
>> (3)
>>186: 0 FUNCGLOBAL DEFAULT  UND [...]@GLIBC_2.2.5 
>> (3)
>>187: 0 FUNCGLOBAL DEFAULT  UND [...]@GLIBC_2.3.2 
>> (2)
>>191: 0 FUNCGLOBAL DEFAULT  UND alarm@GLIBC_2.2.5 
>> (3)
>>194: 0 FUNCGLOBAL DEFAULT  UND [...]@GLIBC_2.2.5 
>> (3)
>>195: 0 FUNCGLOBAL DEFAULT  UND close@GLIBC_2.2.5 
>> (3)
>>197: 0 FUNCGLOBAL DEFAULT  UND [...]@GLIBC_2.2.5 
>> (3)
>>
>> It's missing a lot of symbols. Is this something with readelf or an odd
>> effect of the LTO changes?
> This question looks familiar.
>
> It's a deliberate change to indicate truncation.  Use readelf -sW if you
> want to avoid it.  Truncation has been silent before, so it's necessary
> to educate users about readelf -W.

I think I may have used --wide in my test because I wanted to see more
info...


jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: readelf seems broken in F33

[Jeff Law]

On 9/16/20 3:44 PM, Steve Grubb wrote:
> Hello,
>
> I was doing some binary analysis of files in F33 and have run across 
> something odd.
>
> readelf -s  /usr/sbin/auditd | grep GLIBC
>
> produces a lot of output like:
>
>182: 0 FUNCGLOBAL DEFAULT  UND [...]@GLIBC_2.2.5 
> (3)
>184: 0 FUNCGLOBAL DEFAULT  UND [...]@GLIBC_2.2.5 
> (3)
>185: 0 FUNCGLOBAL DEFAULT  UND [...]@GLIBC_2.2.5 
> (3)
>186: 0 FUNCGLOBAL DEFAULT  UND [...]@GLIBC_2.2.5 
> (3)
>187: 0 FUNCGLOBAL DEFAULT  UND [...]@GLIBC_2.3.2 
> (2)
>191: 0 FUNCGLOBAL DEFAULT  UND alarm@GLIBC_2.2.5 
> (3)
>194: 0 FUNCGLOBAL DEFAULT  UND [...]@GLIBC_2.2.5 
> (3)
>195: 0 FUNCGLOBAL DEFAULT  UND close@GLIBC_2.2.5 
> (3)
>197: 0 FUNCGLOBAL DEFAULT  UND [...]@GLIBC_2.2.5 
> (3)
>
> It's missing a lot of symbols. Is this something with readelf or an odd
> effect of the LTO changes?

Odd, I'm running F33 bits here and I get 159 symbols from running that
(out of 206 total symbols).


jeff




pEpkey.asc
Description: application/pgp-keys
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)

[Jeff Law]

On 10/4/20 2:48 PM, Jan Kratochvil wrote:
> On Tue, 29 Sep 2020 22:29:44 +0200, Mark Wielaard wrote:
>> I was just discussing that recently with the Hotspot Perf GUI
>> maintainer. And we concluded that if .debug files would be compressed
>> then we would need an uncompressed cache somewhere. The issue with
>> having the on-disk debuginfo files compressed is that for
>> debugger/tracing/profiling tools it incurs an significant
>> decompression time delay and extra memory usage. Especially for a
>> profiling tool that only needs to quickly get a little information it
>> is much more convenient to be able to simply mmap the .debug file,
>> check the aranges and directly jump to the correct DIE offset. See
>> e.g. https://github.com/KDAB/hotspot/issues/115
> First is is a marginal use case. For the GDB popular here I tested zlib on
> some IIRC 500MB+ .debug file and the startup time was 11.00s->12.45s
> = +13.20%. Given GDB takes minutes to print something on such .debug files the
> <2s larger startup does not matter.

I'm not sure it's that marginal.  It may not matter for GDB since I
don't think the bottleneck is likely the decompression.  It would
certainly matter for profiling and the like -- I'd probably argue that
dwarf is a terrible choice for those tools, but I don't see CTF as a
viable alternative though, so they're stuck in the dwarf world.


>
> And then all this is about zlib compression. Facebook has developed zstd which
> is much faster. Google says faster than reading the .debug files, on my
> machine both zstd and NVMe disk read are both 2GB/s. I expect btrfs has even
> in-memory cache for decompressed files but I have not checked it now as all
> the numbers I have collected have no effect here anyway.

Please, let's stop talking about btrfs here.  It's not useful.


However, I think it's perfectly valid to discuss zstd if folks wanted to
change the compression scheme for debug sections.  In fact, I'd claim
sticking with zlib, gzip,  bzip2, xz, 7z, etc is unwise.  The world has
moved and zstd seems like the place we should be.  In fact, we use it
for various things within GCC already.


Jeff



pEpkey.asc
Description: application/pgp-keys
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 33 blocker status

[Jeff Law]

On 10/2/20 12:47 PM, Peter Robinson wrote:

On Fri, Oct 2, 2020 at 7:37 PM Jeff Law  wrote:


On 10/2/20 12:32 PM, Ben Cotton wrote:

Beta is out! Time to focus on Final blockers.

Action summary


Accepted blockers
-
1. firefox — Firefox not using langpacks for localization — ASSIGNED
ACTION: firefox maintainers to fix issue

2. sddm — login stuck when changing users repeatedly (log out, log in
a different one) — ASSIGNED
ACTION: kf5-kglobalaccel maintainers to provide systemd service file

3. gnome-control-center — Select Printer Driver hangs — POST
ACTION: none (waiting for coming upstream release)

   [ ... ]

Presumably it's still OK at this stage to fix obviously failing packages
in F33.  In particular I'm still working through the list of QT packages
that are running afoul of the symbol binding issues which in turn cause
those packages to segfault immediately at startup.

Yep, we don't freeze until next week and if they're submitted they may
well go stable before. Once we freeze unless they Blockers/Freeze
exceptions they'll just go our as zero days.


Just wanted to be sure.  I'm not usually involved in Fedora at this stage.


I've got ~50 packages still to investigate -- based on the ones I've 
looked at so only a small percentage will actually need rebuilding.  In 
the end I expect it'll be ~10 rebuilds for F33.



Jeff


___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 33 blocker status

[Jeff Law]

On 10/2/20 12:32 PM, Ben Cotton wrote:

Beta is out! Time to focus on Final blockers.

Action summary


Accepted blockers
-
1. firefox — Firefox not using langpacks for localization — ASSIGNED
ACTION: firefox maintainers to fix issue

2. sddm — login stuck when changing users repeatedly (log out, log in
a different one) — ASSIGNED
ACTION: kf5-kglobalaccel maintainers to provide systemd service file

3. gnome-control-center — Select Printer Driver hangs — POST
ACTION: none (waiting for coming upstream release)


 [ ... ]

Presumably it's still OK at this stage to fix obviously failing packages 
in F33.  In particular I'm still working through the list of QT packages 
that are running afoul of the symbol binding issues which in turn cause 
those packages to segfault immediately at startup.



Jeff

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: LTO and F33

[Jeff Law]

On 10/2/20 6:31 AM, Vitaly Zaitsev via devel wrote:

On 01.10.2020 22:48, Jeff Law wrote:

What you want to do to fix this is force -fPIC into the build flags.
That inhibits local symbol resolution and the copy relocs that are so
problematical for QT.  You can see examples of how to do this in the
clementine package.

Telegram Desktop already uses -fPIC:


I would suggest looking for any uses of -fPIE when compiling the C/C++ 
sources.  PIE allows local binding for some object acceses (and again, 
its local binding of objects that runs afoul of key aspects of the QT 
libraries).



The package I've been looking at (nextcloud) has this gem in a couple of 
its CMakeLists.txt files:



 if(UNIX AND NOT APPLE)
  set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -fPIE")
  set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fPIE")
  set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -pie")
 endif()

Which means we pass -fPIE when building .o files and thus the flag is 
ultimately passed through to LTO compiles which in turn enables the 
problematic symbol binding.  -fPIC is the right thing for QT 
applications given the way the library works.  Note that in the case of 
nextcloud, I don't think that was a global setting across the entire 
package -- it was used in just specific subdirectories and thus didn't 
show up in every invocation of the compiler.



Note that compiling the objects with -fPIC still allows creating a PIE 
executable via the -pie option at link time.  This is critically 
important from a security standpoint.


jeff

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: LTO and F33

[Jeff Law]

On 10/2/20 10:01 AM, Jeff Law wrote:


On 10/2/20 6:31 AM, Vitaly Zaitsev via devel wrote:

On 01.10.2020 22:48, Jeff Law wrote:

What you want to do to fix this is force -fPIC into the build flags.
That inhibits local symbol resolution and the copy relocs that are so
problematical for QT.  You can see examples of how to do this in the
clementine package.

Telegram Desktop already uses -fPIC:


You'll have to look at each and every .o file you generate in your 
build.  I bet you're going to find one or more that aren't compiled 
with PIC.




So I've got a package which is exhibiting similar behavior.  It appears 
the objects are being compiled with the right PIC options, but I'm still 
getting local resolution and COPY relocs.  I'll update you once I know 
what's going on.


jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: LTO and F33

[Jeff Law]

On 10/2/20 6:31 AM, Vitaly Zaitsev via devel wrote:

On 01.10.2020 22:48, Jeff Law wrote:

What you want to do to fix this is force -fPIC into the build flags.
That inhibits local symbol resolution and the copy relocs that are so
problematical for QT.  You can see examples of how to do this in the
clementine package.

Telegram Desktop already uses -fPIC:


You'll have to look at each and every .o file you generate in your 
build.  I bet you're going to find one or more that aren't compiled with 
PIC.



Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: LTO and F33

[Jeff Law]

On 9/30/20 10:20 AM, Vitaly Zaitsev via devel wrote:

On 30.09.2020 15:39, Robert-André Mauchin wrote:

I have an issue with both Clementine and Strawberry (a fork of Clementine)
in F33 and above, users reported that disabling LTO fixes the problem.

I have the same issue with Telegram Desktop:
https://bugzilla.redhat.com/show_bug.cgi?id=1880290


Ah, this isn't a Fedora package.  That's why it didn't show in Florian's 
symbol search or my dynamic relocation search.



What you want to do to fix this is force -fPIC into the build flags.  
That inhibits local symbol resolution and the copy relocs that are so 
problematical for QT.  You can see examples of how to do this in the 
clementine package.



jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: LTO and F33

[Jeff Law]

On 9/30/20 10:20 AM, Vitaly Zaitsev via devel wrote:

On 30.09.2020 15:39, Robert-André Mauchin wrote:

I have an issue with both Clementine and Strawberry (a fork of Clementine)
in F33 and above, users reported that disabling LTO fixes the problem.

I have the same issue with Telegram Desktop:
https://bugzilla.redhat.com/show_bug.cgi?id=1880290


ACK.  I'll verify it's the same issue and that the fix we're using for 
kstars, twinkle, etc works for the telegram desktop.



jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: LTO and F33

[Jeff Law]

On 9/30/20 7:39 AM, Robert-André Mauchin wrote:

On Tuesday, 18 August 2020 17:12:02 CEST Jeff Law wrote:

So we're at a point where the F33 FTBFS issues related to LTO that I'm aware
of

  have been resolved (by opting the package out of LTO).   I still expect

some LTO issues will pop up as packages fix things like missing
dependencies, cmake macros, etc.  I continue to be available to investigate
potential LTO issues, but package maintainers will need to contact me as
I'm not actively looking for new LTO issues.

My focus is now turning to the packages with LTO opt-outs.  I'll be
extracting

  bug reports for upstream (primarily GCC), trying simple

workarounds for old style symbol versioning, identifying backports from
upstream GCC that allow us to remove LTO opt-outs and the like.  So there
should be a trickle of opt-outs removed, but otherwise should largely be
invisible to the F33 release process.
I'd like to thank everyone involved for being patient while we worked
through

  various issues and I look forward to continuing to make

incremental improvements now that the bulk of the LTO work has landed.

jeff


I have an issue with both Clementine and Strawberry (a fork of Clementine)
in F33 and above, users reported that disabling LTO fixes the problem.


Can you run objdump -CR on the executable and send me the result 
privately?   That will be enough to verify its the same problem with the 
same solution as kstars.



Jeff

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org