On Mon, Jul 6, 2026 at 3:42 AM Richard Biener <[email protected]> wrote: > > On Fri, Jul 3, 2026 at 4:47 PM Lewis Hyatt <[email protected]> wrote: > > > > On Thu, Jul 02, 2026 at 08:20:19AM -0400, Lewis Hyatt wrote: > > > On Wed, Jul 1, 2026 at 2:54 PM Michal Jireš <[email protected]> wrote: > > > > > > > > > > > > On Wed Jul 1, 2026 at 2:18 PM CEST, Lewis Hyatt <[email protected]> > > > > wrote: > > > > > On Tue, Jun 30, 2026 at 2:38 PM Michal Jireš <[email protected]> wrote: > > > > >> > > > > >> > > > > >> On Tue Jun 30, 2026 at 1:01 PM CEST, Lewis Hyatt <[email protected]> > > > > >> wrote: > > > > >> > On Tue, Jun 30, 2026 at 3:52 AM Richard Biener > > > > >> > <[email protected]> wrote: > > > > >> >> > > > > >> >> On Tue, Jun 30, 2026 at 5:03 AM Lewis Hyatt <[email protected]> > > > > >> >> wrote: > > > > >> >> > > > > > >> >> > On Wed, Apr 22, 2026 at 04:29:41PM +0200, Richard Biener wrote: > > > > >> >> > > On Thu, Jan 1, 2026 at 6:03 PM Lewis Hyatt <[email protected]> > > > > >> >> > > wrote: > > > > >> >> > > > > > > > >> >> > > > After the previous changes in this series, the LTO front > > > > >> >> > > > end always has an > > > > >> >> > > > appropriate linemap structure for interpreting diagnostic > > > > >> >> > > > pragmas, so it is > > > > >> >> > > > straightforward to implement them, as is done here. > > > > >> >> > > > > > > > >> >> > > > The pragmas are streamed out in each linemap section; since > > > > >> >> > > > all locations > > > > >> >> > > > from a given linemap section will be contiguous in the > > > > >> >> > > > reconstructed > > > > >> >> > > > linemap, they are automatically ordered properly for the > > > > >> >> > > > existing diagnostic > > > > >> >> > > > pragma infrastructure to work as-is. > > > > >> >> > > > > > > > >> >> > > > One wrinkle is that a single function may have been > > > > >> >> > > > streamed out in multiple > > > > >> >> > > > sections. (For example, an inline function will be streamed > > > > >> >> > > > out in all > > > > >> >> > > > partitions that need it.) In this case, when merging them, > > > > >> >> > > > LTO keeps only > > > > >> >> > > > one of the sections, as directed by the linker resolution, > > > > >> >> > > > so the diagnostic > > > > >> >> > > > pragmas that will be in force (in case they were not the > > > > >> >> > > > same for the > > > > >> >> > > > different translation units) will be whichever were > > > > >> >> > > > applicable to the > > > > >> >> > > > section LTO decided to keep. > > > > >> >> > > > > > > >> >> > > LGTM if the rest of the series is approved. > > > > >> >> > > > > > > >> >> > > Thanks, > > > > >> >> > > Richard. > > > > >> >> > > > > > > >> >> > > > > > >> >> > Hi Richard- > > > > >> >> > > > > > >> >> > Firstly, thank you again for your time in reviewing these > > > > >> >> > patches. I thought > > > > >> >> > everything was finally across the finish line, but as I was > > > > >> >> > reviewing the > > > > >> >> > patches one more time before pushing them, I realized there is > > > > >> >> > one small > > > > >> >> > problem with the new approach. Could I please ask you to look > > > > >> >> > at one more > > > > >> >> > patch which addresses that? I attached it here as an > > > > >> >> > incremental change to > > > > >> >> > the rest of the series, but I would propose to squash it into > > > > >> >> > the other > > > > >> >> > patches before pushing. > > > > >> >> > > > > > >> >> > What I missed was that the LTO front end has a mode of > > > > >> >> > operation for > > > > >> >> > incremental linking. I had tested that my new approach works > > > > >> >> > fine with > > > > >> >> > "ld -r" (provided that -frandom-seed is not used to remove > > > > >> >> > uniqueness from > > > > >> >> > the section names), but the LTO front end version of that > > > > >> >> > (which you get > > > > >> >> > when using, say, "gcc -r -flto") does more than just copy the > > > > >> >> > sections; it > > > > >> >> > actually reads all the decls and then re-outputs a new object > > > > >> >> > file with a > > > > >> >> > new identifier, which contains a new decls section plus copies > > > > >> >> > of the > > > > >> >> > function and constructor sections. This means the linemap > > > > >> >> > sections also need > > > > >> >> > to get copied into the output file, and also, it means that an > > > > >> >> > input file to > > > > >> >> > the LTO front end could possibly contain more than one linemap, > > > > >> >> > which was > > > > >> >> > not something I had considered. (I had anticipated that inputs > > > > >> >> > contained > > > > >> >> > just one linemap, except that in LTRANS mode, there would also > > > > >> >> > be one file > > > > >> >> > containing all necessary linemaps copied during WPA). > > > > >> >> > > > > >> >> So I think there's two things now, the older > > > > >> >> -flto-linker-output=rel and > > > > >> >> the newer -flto-incremental. > > > > >> >> > > > > >> >> That said, I'm not sure about the default behavior of -flto -r > > > > >> >> and would > > > > >> >> suggest to add an explicit -flto-linker-output=rel here to be > > > > >> >> unambiguous. > > > > >> >> Did you try that with -ffat-lto-objects as well? > > > > >> >> > > > > >> > > > > > >> > Thanks, what I have understood is: > > > > >> > > > > > >> > -flto-incremental is unrelated to incremental linking per se, > > > > >> > that's about using a cache directory to store inputs + outputs of > > > > >> > WPA+LTRANS, to avoid rerunning the LTRANS step if the partition did > > > > >> > not change. I made sure that this still works the same as before my > > > > >> > patches, that was one motivation for putting all the linemaps into > > > > >> > their own file after WPA, to make sure a change in one partition > > > > >> > doesn't needlessly invalidate the cache for a different one. > > > > >> > > > > >> Do I understand correctly, that LTRANS cache won't notice when > > > > >> location > > > > >> changes while its ID remains the same? > > > > >> > > > > >> In LTRANS, do we use locations purely for diagnostics? = locations > > > > >> cannot influnce the binary output? > > > > >> And if yes, do we have it documented somewhere that locations cannot > > > > >> be > > > > >> used in LTRANS for anything other that diagnostics? > > > > >> > > > > > > > > > > A location can't really change without affecting the streamed object > > > > > file and invalidating the cache. What ends up streamed out (and > > > > > affecting the SHA1) is the map ID and the location_t offset within the > > > > > map, plus any attached tree and discriminator. Any change to the line > > > > > number or column number will either change the location_t or add a new > > > > > map and change all map IDs. > > > > > > > > > > > > > Physical line and column cannot change, because they are directly > > > > represented by the location offset. > > > > > > > > But what happens with filename, to_line..? > > > > Can't I change #include to different file with identical contents? > > > > > > Yes. In that specific case it's hard to see how recompilation would be > > > necessary, but the point stands. > > > > > > > Or change #line? > > > > > > The addition of the #line directive would alter the linemap structure > > > and affect the subsequent map IDs most likely, but not always. > > > Thinking about it some more, if for instance you added a blank line > > > with enough columns of spaces at the start of the file, it would go > > > into its own map, which would not be streamed out since nothing refers > > > to it, and that would change line numbering without changing the > > > streamed out location IDs. > > > > > > > Or if I am missing something, more generally: If the relevant contents > > > > of the linemap-file cannot change without affecting the cached file, > > > > isn't it redundant and we do not need it? If it is needed, it must be > > > > able to contain something that is not fully captured by the cached file, > > > > and must not be used for the binary output. > > > > > > As you alluded to originally, it's really there for generating > > > diagnostics, and for implementing #pragma GCC diagnostic. I understand > > > you're asking if there can be certainty that it's impossible to make a > > > change which does affect code generation, but which does not > > > invalidate the cache because it changes the linemap sections only. If > > > that could happen, then it would be a problem for -flto-incremental. > > > It's a good point, and I agree with you that I have implicitly assumed > > > this can't happen. > > > I feel like it really shouldn't be an issue in practice, but I don't > > > have a more convincing answer than that. I'm going to take a look and > > > see if I can either demonstrate that it's fine, or else, adjust the > > > location streaming so that something in the cached file will reflect > > > it if the linemap changes in this way. Thanks! > > > > > > > The below patch (incremental to the others) resolves this issue by > > outputting a hash code identifying each line map along with the location ID > > when streaming out the location. I used a 32-bit hash (as provided by > > inchash::hash) to minimize the space overhead; it seems like this should be > > sufficient but it could be swapped for something with more bits as > > well. This increases the size of the streamed LTO by approximately 1.5%, > > here for instance is the size of the LTRANS inputs when compiling cc1plus: > > > > master: 723 MB > > patch v1 (sent previously): 705 MB > > patch v2 (this one): 716 MB > > > > It's still smaller than the current location streaming approach, seems > > worth it to me... What do you think? Thanks... > > I think it's reasonable but I don't know either if this is an issue in > practice. > I suppose that if the line numbers can shift (with vertical space added) > without affecting the location_t then we could miss emitting the correct > line numbers into the DWARF?
I was able to reproduce such a case. If -g is omitted on the initial compile, but specified at WPA time, then some debug info is generated using the LTO location_t data that was streamed out. If an input file is changed in one of the limited ways that can affect line numbers without invalidating any of the location references, it's possible for the LTRANS cache to fail to recompile with a new line number, without this last patch I sent. I also saw this for Mach-O targets: config/rs6000/rs6000.cc- add_compiler_branch_island (labelname, funname, config/rs6000/rs6000.cc: insn_line ((const rtx_insn*)insn)); which seems to establish some precedent that back end code generation could depend on the streamed location_t (via insn_line()). In this particular case, I don't think that the line number is used anymore (it was removed by r13-2361 "STABS: remove -gstabs and -gxcoff functionality"), and it was also related to debug symbols, but I thought it was further evidence that this issue can't be ruled out. -Lewis -Lewis
