Sean Christopherson <[email protected]> writes:
> On Thu, Aug 13, 2026, Ackerley Tng wrote:
>> Sean Christopherson <[email protected]> writes:
>> > That's why I think it's worth analyzing the cost: if it's in the
>> > noise, leave it alone. If it's meaningful, figure out a not-too-gross way
>> > to skip
>> > the entire thing if kvm_arch_gmem_make_shared() is a glorified nop in the
>> > end.
>>
>> Is noise defined relative to the entire conversion process? Would this
>> benchmark look like
>>
>> 1. Convert 4G to shared on TDX with CONFIG_AMD_SEV defined
>> 2. Convert 4G to shared on TDX without CONFIG_AMD_SEV defined
>>
>> and then compare the difference in time taken?
>
> That'd work, though I was envisioning something even simpler: use rdtsc() to
> count the cycles it takes to iterate over various ranges of memory. Do
> whatever
> is easiest for you though.
I made some changes to add rdtsc() for the conversion process as Sean
suggested [1], and exercised conversion like this [2]:
1. Initialize some memory as private
2. Get the guest to fault them into Secure EPTs
3. Converts the memory to shared <<== this is being benchmarked
4. Converts memory back to private
I made it build the VM once and convert 5 times:
./gmem_benchmark_tdx_convert --iterations=5 --size=1g
Here are the raw prints:
[ 396.403818] guest_memfd to_shared: nr_pages=1, make_shared=930
cycles, total=39278 cycles
[ 396.412087] guest_memfd to_shared: nr_pages=1, make_shared=252
cycles, total=28060 cycles
[ 396.420379] guest_memfd to_shared: nr_pages=1, make_shared=176
cycles, total=26952 cycles
[ 396.428643] guest_memfd to_shared: nr_pages=1, make_shared=176
cycles, total=27038 cycles
[ 396.436921] guest_memfd to_shared: nr_pages=1, make_shared=176
cycles, total=26980 cycles
[ 400.338157] guest_memfd to_shared: nr_pages=1, make_shared=1072
cycles, total=37236 cycles
[ 400.346509] guest_memfd to_shared: nr_pages=1, make_shared=316
cycles, total=28338 cycles
[ 400.354805] guest_memfd to_shared: nr_pages=1, make_shared=176
cycles, total=27182 cycles
[ 400.363090] guest_memfd to_shared: nr_pages=1, make_shared=176
cycles, total=26972 cycles
[ 400.371349] guest_memfd to_shared: nr_pages=1, make_shared=176
cycles, total=26886 cycles
[ 448.761474] guest_memfd to_shared: nr_pages=262144,
make_shared=15041018 cycles, total=6616067680 cycles
[ 455.807988] guest_memfd to_shared: nr_pages=262144,
make_shared=14937462 cycles, total=6608542680 cycles
[ 462.848856] guest_memfd to_shared: nr_pages=262144,
make_shared=15138858 cycles, total=6599494898 cycles
[ 469.885694] guest_memfd to_shared: nr_pages=262144,
make_shared=15721972 cycles, total=6610219850 cycles
[ 476.928399] guest_memfd to_shared: nr_pages=262144,
make_shared=15000406 cycles, total=6615114540 cycles
[ 762.980690] guest_memfd to_shared: nr_pages=1048576,
make_shared=61902982 cycles, total=26400884028 cycles
[ 791.022885] guest_memfd to_shared: nr_pages=1048576,
make_shared=61746114 cycles, total=26401170984 cycles
[ 819.054580] guest_memfd to_shared: nr_pages=1048576,
make_shared=61096794 cycles, total=26404409058 cycles
[ 847.120994] guest_memfd to_shared: nr_pages=1048576,
make_shared=61446290 cycles, total=26447461896 cycles
[ 875.219750] guest_memfd to_shared: nr_pages=1048576,
make_shared=61774646 cycles, total=26444608360 cycles
And here's the above, tabulated:
nr_pages make_shared total percentage
---------- --------------- --------------- ------------
1 930 39278 2.3677%
1 252 28060 0.8981%
1 176 26952 0.6530%
1 176 27038 0.6509%
1 176 26980 0.6523%
1 1072 37236 2.8789%
1 316 28338 1.1151%
1 176 27182 0.6475%
1 176 26972 0.6525%
1 176 26886 0.6546%
262144 15041018 6616067680 0.2273%
262144 14937462 6608542680 0.2260%
262144 15138858 6599494898 0.2294%
262144 15721972 6610219850 0.2378%
262144 15000406 6615114540 0.2268%
1048576 61902982 26400884028 0.2345%
1048576 61746114 26401170984 0.2339%
1048576 61096794 26404409058 0.2314%
1048576 61446290 26447461896 0.2323%
1048576 61774646 26444608360 0.2336%
Looks to me it is within noise.
I also actually tried measuring the conversion time from userspace with
CONFIG_AMD_SEV enabled and disabled. Converting a 1G-sized TD was faster
by 0.2%, which is in line with the above table. Interestingly, when
converting a 4G-sized TD, skipping kvm_gmem_make_shared() was _slower_
over 2 runs. I don't have an explanation for that. I think the code was
correct. (If it makes a difference, I skipped kvm_gmem_make_shared()
using a custom guest_memfd creation time flag and skipped make_shared if
the flag was set on the inode.)
I thought adding a kvm_arch_has_gmem_make_shared(), defaulting it to
false for all archs and having x86 override with
!!kvm_x86_ops.gmem_make_shared is not too bad either:
+ doesn't leak anything, since the function being called is
kvm_arch_gmem_make_shared and the accompanying function is
kvm_arch_has_gmem_make_shared. Or maybe just a little, since all the
other ops don't have the accompanying _has_ function
+ it's a kernel-internal thing
+ not too many lines of code, not too complex
Sean, let me know if you prefer to add kvm_arch_has_gmem_make_shared()
and for my learning, what your considerations are regarding adding
kvm_arch_has_gmem_make_shared() or not.
[1]
https://github.com/googleprodkernel/linux-cc/commit/3295072cfeb8920e48d9e80c98fd1695d4bbad99
[2]
https://github.com/googleprodkernel/linux-cc/commit/a644618f270f02c832fb7e8402fcae98bc91f4e5
