xenlockprof with my patch (some zero lines have been removed for brevity):
$ sudo xenlockprof -r ; for i in $(seq 1 5) ; do time sudo /usr/lib/xen-4.14/bin/xl destroy test-vm-${i} & done ; wait $(jobs -p) ; sudo xenlockprof
real 0m1,224s
user 0m0,013s
sys 0m0,710s
real 0m1,262s
user 0m0,006s
sys 0m0,762s
real 0m1,319s
user 0m0,017s
sys 0m0,800s
real 0m1,400s
user 0m0,015s
sys 0m0,889s
real 0m1,424s
user 0m0,003s
sys 0m0,932s
global lock platform_timer_lock : lock: 2( 0.000001599s), block: 0( 0.000000000s)
global lock flush_lock : lock: 42538( 0.367507762s), block: 10875( 0.265902959s)
global lock pci_config_lock : lock: 14( 0.000007060s), block: 0( 0.000000000s)
global lock vpmu_lock : lock: 10( 0.000000314s), block: 0( 0.000000000s)
global lock cpupool_lock : lock: 10( 0.000001061s), block: 0( 0.000000000s)
global lock vm_lock : lock: 35( 0.000002936s), block: 0( 0.000000000s)
global lock tasklet_lock : lock: 30( 0.000003322s), block: 0( 0.000000000s)
global lock sysctl_lock : lock: 82( 0.000309193s), block: 0( 0.000000000s)
global lock call_lock : lock: 34( 0.000088186s), block: 1( 0.000002591s)
global lock heap_lock : lock: 2788704( 0.362311723s), block: 222681( 0.091152276s)
global lock global_virq_handlers_lock : lock: 519( 0.000045863s), block: 0( 0.000000000s)
global lock domlist_update_lock : lock: 5( 0.000000588s), block: 0( 0.000000000s)
global lock domctl_lock : lock: 544( 0.872791187s), block: 0( 0.000000000s)
domain 0 lock event_lock : lock: 51( 0.000036035s), block: 0( 0.000000000s)
domain 0 lock page_alloc_lock : lock: 1( 0.000000135s), block: 0( 0.000000000s)
total profiling time: 0.000000000s
total locked time: 1.603106964s
total blocked time: 0.357057826s
Without my patch:
$ sudo xenlockprof -r ; for i in $(seq 1 5) ; do time sudo /usr/lib/xen-4.14/bin/xl destroy test-vm-${i} & done ; wait $(jobs -p) ; sudo xenlockprof
real 0m3,937s
user 0m0,010s
sys 0m3,433s
real 0m4,321s
user 0m0,006s
sys 0m3,817s
real 0m4,326s
user 0m0,016s
sys 0m3,805s
real 0m4,323s
user 0m0,013s
sys 0m3,803s
real 0m4,330s
user 0m0,011s
sys 0m3,814s
global lock platform_timer_lock : lock: 8( 0.000006219s), block: 0( 0.000000000s)
global lock flush_lock : lock: 29576( 0.245458882s), block: 6415( 0.172432644s)
global lock pci_config_lock : lock: 52( 0.000042975s), block: 0( 0.000000000s)
global lock vector_lock : lock: 4( 0.000012864s), block: 0( 0.000000000s)
global lock vpmu_lock : lock: 10( 0.000000323s), block: 0( 0.000000000s)
global lock cpupool_lock : lock: 10( 0.000000740s), block: 0( 0.000000000s)
global lock vm_lock : lock: 35( 0.000001799s), block: 0( 0.000000000s)
global lock tasklet_lock : lock: 30( 0.000002877s), block: 0( 0.000000000s)
global lock sysctl_lock : lock: 80( 0.000293255s), block: 0( 0.000000000s)
global lock call_lock : lock: 30( 0.000191153s), block: 1( 0.000002899s)
global lock heap_lock : lock: 3342357( 2.268295505s), block: 3263853( 18.556650797s)
global lock global_virq_handlers_lock : lock: 9825( 0.000615622s), block: 504( 0.000532967s)
global lock domlist_update_lock : lock: 5( 0.000000332s), block: 0( 0.000000000s)
global lock domctl_lock : lock: 9850( 0.000424312s), block: 0( 0.000000000s)
domain 0 lock event_lock : lock: 59( 0.000033557s), block: 3( 0.000000723s)
total profiling time: 0.000000000s
total locked time: 2.515380415s
total blocked time: 18.729620030s
From above two listings, it can be seen that the main difference is the lock waiting time ("blocked time") on the "global lock heap_lock" spinlock.
20.09.2021, 11:34, "Dmitry Isaikin" <[email protected]>:
I think it contend on page insertion into free pages list. But I don't confirm or denied this yet. I'm new in hypervisors...
11:19, 20 сентября 2021 г., Jan Beulich <[email protected]>:On 17.09.2021 18:01, Julien Grall wrote:
I will quote what Hongyan wrote back on the first report:
"
The best solution is to make the heap scalable instead of a global
lock, but that is not going to be trivial.
Of course, another solution is to keep the domctl lock dropped in
domain_kill() but have another domain_kill lock so that competing
domain_kill()s will try to take that lock and back off with hypercall
continuation. But this is kind of hacky (we introduce a lock to reduce
spinlock contention elsewhere), which is probably not a solution but a
workaround.
"
Interesting. Is contention on the heap lock merely suspected or
was that lock proven to be the problem one?
Jan
--
Отправлено из мобильного приложения Яндекс.Почты
