I understand the stock behavior of pilling variables on may happen to improve cache usage.
However, in a multicore setting it is a never-ending source of unintentionally showing up and disappearing false-sharing depending on unrelated variables being added or removed. I'm aware one can manually add padding or explicitly tell the linker to put a specific variable elsewhere, but in large codebases getting to a point where everything is annotated is quite a challenge. To give you one example of a project which did not get there despite literally decades of multicore development: the Linux kernel (or in fact any open-source kernel out there, last I checked). In the meantime a great clutch would contain such problems to a specific .o file. To illustrate, suppose there are 3 .o files with variables spanning 2 cache lines. The first .o file has something frequently read and it lands in the first cache line. At the same time the last .o file has a frequently modified var and it currently lands in the second cache line. Now some vars got removed from the "middle" .o file and as a result the frequently modified var is pulled up to the first cache line, false-sharing with a frequently read var from something unrelated. This results in a slowdown stemming from a change which should have no impact. I once more note ideally this all would be annotated in project-specific ways, but so far even Linux did not get there despite playing whack-a-mole. An option to keep variables from different .o files in separate lines would be a great damage-controlling measure. I'll note even on CPUs with 64 byte cache lines one may want to pick a higher value (e.g., due to the adjacent cache line prefetcher on Intel CPUs). Thus the best option would probably accept an argument specifying how much alignment one expects. For illustrative purposes, I don't care about the name: -align-object-data-section=64 Thoughts? -- Mateusz Guzik <mjguzik gmail.com>
