On 08/10/26 14:38, Andi Kleen wrote:
External email: Use caution opening links or attachments
In both section bodies "int" follows the section's compact flag (varint when
compact, big-endian otherwise), as for other section bodies.
Hi Andi,
It's not clear to me why we need both compact and non compact, or what the
trade offs between the two are and why would the user decide for one or another.
Is varint worth it? Normally the files are not that big anyways.
Or is it significantly slower? If there is no big difference perhaps it
can be always used.
Varint is worth it, it saves a lot of space (table below). This is the
default way LLVM streams out its profiles (via the uleb128 type). Compact
mode was added mostly to have the ability to see how much smaller the
most-compressed version of the profile practically could be (modulo
external compression algorithms). It could certainly be the default mode,
we have not measured the streaming overhead as such.
If the goal is just smaller files could always store them compressed
with a standard library. But I haven't seen a case where the size of the
gcov (as opposed to the perf.data) was a problem.
The basic goals behind the design were to try and have the smallest-possible
format which can be directly consumed by the compiler while being low
overhead to stream in. Using the integrated compression library makes
a lot of sense, we can explore doing that as well.
IIRC one issue that made files bigger with the old autofdo is that it
saved zero counts, but perhaps that is already fixed.
Zero counts are still streamed out, however v4 has a location type that
encodes it in the type itself (LOCATION_ZERO), which saves a huge amount
of space already.
Also why do we bother with endian conversion? It seems like unnecessary
complexity.
It makes the profiles portable across machines and architectures. I am
not sure how relevant it is in practice, but it seemed like a good thing
to have. I think it would be good regardless to have the profile be in
fixed-endian format, so maybe it makes more sense to have it be LE by
default.
Maybe I missed it, but it would be good to have a separate spec
for the file format.
A spec was sent out as an RFC earlier:
https://gcc.gnu.org/pipermail/gcc/2026-April/247992.html.
There have been some modifications to it since then that are included
in the patch, but the basic premise still holds: the compact mode
saves ~70% on the size over v3 compared to ~40% with default v4.
Copying the relevant section from the RFC:
7.1. SPEC CPU 2017
Profile v3 v4 v4c v4 v4c
------- -- -- --- ----- -----
cpugcc_r 7.1M 4.1M 1.9M -43% -74%
cpuxalan_r 1.3M 790K 516K -40% -61%
deepsjeng_r 103K 59K 30K -44% -71%
exchange2_r 27K 16K 7.7K -43% -71%
leela_r 299K 195K 113K -35% -62%
mcf_r 27K 16K 7.8K -43% -71%
omnetpp_r 1.2M 764K 425K -37% -65%
perlbench_r 412K 229K 105K -44% -75%
specrand_ir 3.3K 1.6K 705 -53% -79%
x264_r 434K 247K 113K -43% -74%
xz_r 75K 46K 23K -39% -69%
TOTAL 11M 6.4M 3.2M -42% -71%
7.2. GCC Bootstrap
Profile v3 v4 v4c v4 v4c
------- -- -- --- ----- -----
all.fda 56M 32M 16M -43% -71%
--
Regards,
Dhruv
