Hey, After noticing this commit: https://github.com/torvalds/linux/commit/7ab5e10eda02da1d9562ffde562c51055d368e9c and because of high energy prices here, I did some tests with an energy monitor to see what the power consumption is for my AMD RX480when its not used. The RX480 is normally only used to pass through to a vm, but this vm is mostly switched off so the RX480 is not used.
The result is that when I boot my system with vfio-pci enabled it uses ~90Watt (RX480 driver = vfio-pci), but when I boot my system with vfio-pci disabled (RX480 driver = amdgpu) then my system uses ~80Watt. Both times the RX480 is in D3hot power state. Any idea why using the vfio-pci driver results in ~10Watt more power consumption? Another weirdest thing is that when I change the drivers after boot using bind/unbind, my system will consistently use another 10W more, ie when testing as follows: - system boots with vfio-pci enabled // -> 90 Watt - echo 0000:02:00.0 > /sys/bus/pci/drivers/vfio-pci/unbind - echo 0000:02:00.0 > /sys/bus/pci/drivers/amdgpu/bind // -> 100Watt - echo 0000:02:00.0 > /sys/bus/pci/drivers/amdgpu/unbind - echo 0000:02:00.0 > /sys/bus/pci/drivers/vfio-pci/bind // -> still 100 Watt even though RX480 is in D3hot state Cheers, Pim Relevant outputs: # uname -a Linux 5.19.11-051911-generic #202209231341 SMP PREEMPT_DYNAMIC Fri Sep 23 14:11:03 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux # dmesg | grep 0000\:02\:00 [ 0.395249] pci 0000:02:00.0: [1002:67df] type 00 class 0x030000 [ 0.395305] pci 0000:02:00.0: reg 0x10: [mem 0xa0000000-0xafffffff 64bit pref] [ 0.395362] pci 0000:02:00.0: reg 0x18: [mem 0xb0000000-0xb01fffff 64bit pref] [ 0.395412] pci 0000:02:00.0: reg 0x20: [io 0xd000-0xd0ff] [ 0.395453] pci 0000:02:00.0: reg 0x24: [mem 0xfbd00000-0xfbd3ffff] [ 0.395496] pci 0000:02:00.0: reg 0x30: [mem 0xfbd40000-0xfbd5ffff pref] [ 0.395536] pci 0000:02:00.0: enabling Extended Tags [ 0.395656] pci 0000:02:00.0: supports D1 D2 [ 0.395689] pci 0000:02:00.0: PME# supported from D1 D2 D3hot D3cold [ 0.395860] pci 0000:02:00.1: [1002:aaf0] type 00 class 0x040300 [ 0.395920] pci 0000:02:00.1: reg 0x10: [mem 0xfbd60000-0xfbd63fff 64bit] [ 0.396018] pci 0000:02:00.1: enabling Extended Tags [ 0.396112] pci 0000:02:00.1: supports D1 D2 [ 0.409125] pci 0000:02:00.0: vgaarb: setting as boot VGA device [ 0.409125] pci 0000:02:00.0: vgaarb: bridge control possible [ 0.409125] pci 0000:02:00.0: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none [ 0.470935] pci 0000:02:00.0: CLS mismatch (64 != 32), using 64 bytes [ 0.470996] pci 0000:02:00.1: D0 power state depends on 0000:02:00.0 [ 0.475972] pci 0000:02:00.0: Adding to iommu group 46 [ 0.476027] pci 0000:02:00.1: Adding to iommu group 46 [ 1.074885] vfio-pci 0000:02:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none [ 9.174548] vfio-pci 0000:02:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none # cat /sys/bus/pci/devices/0000\:02\:00.0/d3cold_allowed 1 # cat /sys/bus/pci/devices/0000\:02\:00.0/power_state D3hot # cat /sys/bus/pci/devices/0000\:02\:00.0/uevent DRIVER=vfio-pci PCI_CLASS=30000 PCI_ID=1002:67DF PCI_SUBSYS_ID=1462:3413 PCI_SLOT_NAME=0000:02:00.0 MODALIAS=pci:v00001002d000067DFsv00001462sd00003413bc03sc00i00
_______________________________________________ vfio-users mailing list vfio-users@redhat.com https://listman.redhat.com/mailman/listinfo/vfio-users
