On Wed Jul 1, 2026 at 2:18 PM CEST, Lewis Hyatt <[email protected]> wrote:
> On Tue, Jun 30, 2026 at 2:38 PM Michal Jireš <[email protected]> wrote:
>>
>>
>> On Tue Jun 30, 2026 at 1:01 PM CEST, Lewis Hyatt <[email protected]> wrote:
>> > On Tue, Jun 30, 2026 at 3:52 AM Richard Biener
>> > <[email protected]> wrote:
>> >>
>> >> On Tue, Jun 30, 2026 at 5:03 AM Lewis Hyatt <[email protected]> wrote:
>> >> >
>> >> > On Wed, Apr 22, 2026 at 04:29:41PM +0200, Richard Biener wrote:
>> >> > > On Thu, Jan 1, 2026 at 6:03 PM Lewis Hyatt <[email protected]> wrote:
>> >> > > >
>> >> > > > After the previous changes in this series, the LTO front end always
>> >> > > > has an
>> >> > > > appropriate linemap structure for interpreting diagnostic pragmas,
>> >> > > > so it is
>> >> > > > straightforward to implement them, as is done here.
>> >> > > >
>> >> > > > The pragmas are streamed out in each linemap section; since all
>> >> > > > locations
>> >> > > > from a given linemap section will be contiguous in the reconstructed
>> >> > > > linemap, they are automatically ordered properly for the existing
>> >> > > > diagnostic
>> >> > > > pragma infrastructure to work as-is.
>> >> > > >
>> >> > > > One wrinkle is that a single function may have been streamed out in
>> >> > > > multiple
>> >> > > > sections. (For example, an inline function will be streamed out in
>> >> > > > all
>> >> > > > partitions that need it.) In this case, when merging them, LTO
>> >> > > > keeps only
>> >> > > > one of the sections, as directed by the linker resolution, so the
>> >> > > > diagnostic
>> >> > > > pragmas that will be in force (in case they were not the same for
>> >> > > > the
>> >> > > > different translation units) will be whichever were applicable to
>> >> > > > the
>> >> > > > section LTO decided to keep.
>> >> > >
>> >> > > LGTM if the rest of the series is approved.
>> >> > >
>> >> > > Thanks,
>> >> > > Richard.
>> >> > >
>> >> >
>> >> > Hi Richard-
>> >> >
>> >> > Firstly, thank you again for your time in reviewing these patches. I
>> >> > thought
>> >> > everything was finally across the finish line, but as I was reviewing
>> >> > the
>> >> > patches one more time before pushing them, I realized there is one small
>> >> > problem with the new approach. Could I please ask you to look at one
>> >> > more
>> >> > patch which addresses that? I attached it here as an incremental change
>> >> > to
>> >> > the rest of the series, but I would propose to squash it into the other
>> >> > patches before pushing.
>> >> >
>> >> > What I missed was that the LTO front end has a mode of operation for
>> >> > incremental linking. I had tested that my new approach works fine with
>> >> > "ld -r" (provided that -frandom-seed is not used to remove uniqueness
>> >> > from
>> >> > the section names), but the LTO front end version of that (which you get
>> >> > when using, say, "gcc -r -flto") does more than just copy the sections;
>> >> > it
>> >> > actually reads all the decls and then re-outputs a new object file with
>> >> > a
>> >> > new identifier, which contains a new decls section plus copies of the
>> >> > function and constructor sections. This means the linemap sections also
>> >> > need
>> >> > to get copied into the output file, and also, it means that an input
>> >> > file to
>> >> > the LTO front end could possibly contain more than one linemap, which
>> >> > was
>> >> > not something I had considered. (I had anticipated that inputs contained
>> >> > just one linemap, except that in LTRANS mode, there would also be one
>> >> > file
>> >> > containing all necessary linemaps copied during WPA).
>> >>
>> >> So I think there's two things now, the older -flto-linker-output=rel and
>> >> the newer -flto-incremental.
>> >>
>> >> That said, I'm not sure about the default behavior of -flto -r and would
>> >> suggest to add an explicit -flto-linker-output=rel here to be unambiguous.
>> >> Did you try that with -ffat-lto-objects as well?
>> >>
>> >
>> > Thanks, what I have understood is:
>> >
>> > -flto-incremental is unrelated to incremental linking per se,
>> > that's about using a cache directory to store inputs + outputs of
>> > WPA+LTRANS, to avoid rerunning the LTRANS step if the partition did
>> > not change. I made sure that this still works the same as before my
>> > patches, that was one motivation for putting all the linemaps into
>> > their own file after WPA, to make sure a change in one partition
>> > doesn't needlessly invalidate the cache for a different one.
>>
>> Do I understand correctly, that LTRANS cache won't notice when location
>> changes while its ID remains the same?
>>
>> In LTRANS, do we use locations purely for diagnostics? = locations
>> cannot influnce the binary output?
>> And if yes, do we have it documented somewhere that locations cannot be
>> used in LTRANS for anything other that diagnostics?
>>
>
> A location can't really change without affecting the streamed object
> file and invalidating the cache. What ends up streamed out (and
> affecting the SHA1) is the map ID and the location_t offset within the
> map, plus any attached tree and discriminator. Any change to the line
> number or column number will either change the location_t or add a new
> map and change all map IDs.
>
Physical line and column cannot change, because they are directly
represented by the location offset.
But what happens with filename, to_line..?
Can't I change #include to different file with identical contents?
Or change #line?
Or if I am missing something, more generally: If the relevant contents
of the linemap-file cannot change without affecting the cached file,
isn't it redundant and we do not need it? If it is needed, it must be
able to contain something that is not fully captured by the cached file,
and must not be used for the binary output.
>> >
>> > -Starting with r6-5271, lto-wrapper provides -flinker-output=rel
>> > by default when using -r.
>> >
>> > -It all works fine with -ffat-lto-objects, yes. The default
>> > testsuite options used by most tests (including the new test of
>> > dg-lto-do incr-link that I added) include this, and it isn't affected
>> > by the linemap changes.
>> >
>> >> > It wasn't too bad to adapt things to work fine in this case too, so the
>> >> > attached patch is not too huge of a change as far as LTO. A lot of the
>> >> > patch
>> >> > consists of changes to testsuite/lib/lto.exp to enable testing the
>> >> > incremental link mode. There was not any test coverage previously for
>> >> > this
>> >> > sort of operation:
>> >> >
>> >> > g++ -flto -c t{1,2,3}.cc
>> >> > g++ -flto -r -o t12.o t{1,2}.o
>> >> > g++ -flto -o t t12.o t3.o
>> >> >
>> >> > (i.e., partial linking of some files and then a final link of all of
>> >> > them
>> >> > together), which is why I didn't see this problem initially. The patch
>> >> > adds
>> >> > a new directive { dg-lto-do incr-link }, which tests a sequence like the
>> >> > above, similar to how { dg-lto-do ar-link } already works. gccint was
>> >> > missing documentation of ar-link, so I added something for that as well
>> >> > as
>> >> > for incr-link.
>> >> >
>> >> > I tested everything again on x86-64, aarch64 (cfarm425), amd64
>> >> > (cfarm420),
>> >> > sparc 32-bit (cfarm216), and ppc64be (cfarm121), including bootstrap
>> >> > with
>> >> > bootstrap-lto, and it looks good. Please let me know if it looks OK, in
>> >> > which case, I am ready to commit the series now. Thanks!
>> >>
>> >> That said, the patch looks reasonable, but I wouldn't call myself an
>> >> expert
>> >> in the area of incremental LTO, of neither mode.
>> >>
>> >> So please give Honza and/or Michal the change to chime in.
>> >>
>> >> Thanks,
>> >> Richard.
>> >
>> > Thanks, sure, I will wait for additional feedback.
>> >
>> > -Lewis
>>
>>
>>
>> >/* Given an input linemap ID, compute how it will be known in the output,
>> >given
>> > that the output could be the combination of multiple input files. */
>> >inline unsigned
>> >lto_linemap_output_id (unsigned linemap_id, const lto_file_decl_data
>> >*file_data)
>> >{
>> > return linemap_id + file_data->order;
>> >}
>>
>> What happens when there are two input files both with multiple linemap
>> sections? Won't the output IDs overlap?
>>
>
> It's arranged that the output ID is unique. file_data->order is
> incremented (in lto/lto-common.cc), after reading each file, by the
> number of linemaps in that file. So if the input IDs are like (file A:
> 0,1,2) (file B: 0,1) then on output they will get assigned to
> (0,1,2,3,4).
Alright, I missed the lto/lto-common.cc change.
>
> Thanks a lot for taking a look at this! Happy to look into anything further.
>
> -Lewis
Michal
