[gem5-users] Re: Fail to bootup with KVM in se.py on X86 arch
Hmm... That's interesting. It could have something to do with the I/O hole for x86 at 3-4GB. This is a huge pain for gem5's config scripts. We hack around this in our config scripts (e.g., for gem5-resources). See https://gem5.googlesource.com/public/gem5-resources/+/refs/heads/stable/src/npb/configs/system/system.py#59 for an example. Those scripts have been tested with 8 cores and more than 4GB of memory (maybe 64G?). We may also have run it with 32 cores, but I'm not sure if that has been tested recently. Cheers, Jason On Tue, May 11, 2021 at 8:03 PM Liyichao wrote: > Hi Jason: > > > > I find that if I set –mem-size < 4GB, there is no error, but when > I set –mem-size=4GB or more than 4GB, the error occurred. > > > > > > > > *发件人:* Liyichao > *发送时间:* 2021年5月12日 0:26 > *收件人:* Jason Lowe-Power > *抄送:* gem5 users mailing list > *主题:* RE: [gem5-users] Fail to bootup with KVM in se.py on X86 arch > > > > Hi Jason： > I make two tests： > > 1. I write a test code follow "open /dev/kvm” "create vm” "create vcpu” > ，it has no error； > > 2. I put allocate pmem and create vcpu in create_vm function in gem5， > not outside create_vm，it has no error. > > > > so I suspect if the allocate pmem function in gem5 source has any wrongs > ？ > > *发件人： *Jason Lowe-Power > > *收件人： *Liyichao > > *抄送： *gem5 users mailing list > > *主题： *Re: [gem5-users] Fail to bootup with KVM in se.py on X86 arch > > *时间： *2021-05-11 22:47:19 > > > > Hello, > > > > I wonder if you have a maximum number of vcpus set on your host system. > Otherwise, I can't think of any specific limitation to creating vcpus. > > > > Cheers, > > Jason > > > > On Tue, May 11, 2021 at 2:36 AM Liyichao wrote: > > Hi Jason: > > > > I use strace to follow the call stack, I find that the ioctl() > with KVM_CREATE_VCPU returned EEXIST errno, this means the vCPU exist. > > > >ioctl(5, KVM_CREATE_VCPU, 2)= -1 EEXIST (File > exists) > > > > > > *发件人:* Liyichao > *发送时间:* 2021年5月11日 12:54 > *收件人:* 'Jason Lowe-Power' ; gem5 users mailing list < > gem5-users@gem5.org> > *主题:* 答复: [gem5-users] Fail to bootup with KVM in se.py on X86 arch > > > > Hi Jason: > > I have add a DPRINTF in line 559 of vm.cc , it showed “ 0: > system.kvm_vm: *debug vcpuID is 0” > > > > Any KVM use limitations in X86? > > > > > > gem5 version 21.0.0.0 > > gem5 compiled May 11 2021 01:04:20 > > gem5 started May 11 2021 01:09:26 > > gem5 executing on ubuntu, pid 53534 > > command line: ./build/X86/gem5.opt --debug-flags=Kvm configs/example/se.py > --cpu-type=X86KvmCPU --cpu-clock=2.6GHz --sys-clock=2.6GHz --caches > --l1d_size=64kB --l1i_size=64kB --l2cache --l2_size=32MB --mem-size=32GB > --num-cpus=1 -I 5000 -c ./test > > > > warn: membus.slave is deprecated. `slave` is now called `cpu_side_ports` > > warn: tol2bus.master is deprecated. `master` is now called `mem_side_ports` > > warn: membus.slave is deprecated. `slave` is now called `cpu_side_ports` > > warn: tol2bus.slave is deprecated. `slave` is now called `cpu_side_ports` > > warn: tol2bus.slave is deprecated. `slave` is now called `cpu_side_ports` > > warn: tol2bus.slave is deprecated. `slave` is now called `cpu_side_ports` > > warn: tol2bus.slave is deprecated. `slave` is now called `cpu_side_ports` > > warn: membus.master is deprecated. `master` is now called `mem_side_ports` > > warn: membus.master is deprecated. `master` is now called `mem_side_ports` > > warn: membus.slave is deprecated. `slave` is now called `cpu_side_ports` > > Global frequency set at 1 ticks per second > > warn: DRAM device capacity (8192 Mbytes) does not match the address range > assigned (32768 Mbytes) > > 0: system.cpu: vcpuID is 0 > > 0: system.remote_gdb: listening for remote gdb on port 7000 > > 0: system.cpu: ActivateContext 0 > > REAL SIMULATION > > 0: system.kvm_vm: Mapping 1 memory region(s) > > 0: system.kvm_vm: Mapping region: 0x0x7f0b007b7000 -> 0x0 [size: > 0x8] > > 0: system.kvm_vm: vmFD is 5, p1 is 140724538190576 > > 0: system.cpu: charlie, vcpuID is 0 > > 0: system.kvm_vm: vmFD is 5, p1 is 0 > > 0: system.kvm_vm: *debug vcpuID is 0 > > panic: KVM: Failed to create virtual CPU > > Memory Usage: 33838804 KBytes > > Program aborted at tick 0 > > --- BEGIN LIBC BACKTRACE --- > > ./build/X86/gem5.opt(_Z15print_backtracev+0x2c)[0x5622a4c15f8c] > > ./build/X86/gem5.opt(_Z12abortHandleri+0x4a)[0x5622a4c30b6a] > > /lib/x86_64-linux-gnu/libpthread.so.0(+0x12980)[0x7f1305ea6980] > > /lib/x86_64-linux-gnu/libc.so.6(gsignal+0xc7)[0x7f1304430fb7] > > /lib/x86_64-linux-gnu/libc.so.6(abort+0x141)[0x7f1304432921] > > ./build/X86/gem5.opt(+0x6e5cef)[0x5622a3f20cef] > > ./build/X86/gem5.opt(_ZN5KvmVM10createVCPUEl+0x9ff)[0x5622a4953b8f] > > ./build/X86/gem5.opt(_ZN10BaseKvmCPU7startupEv+0x9e)[0x5622a494bc8e] > > ./build/X86/gem5.opt(_ZN9X86KvmCPU7startupEv+0x9)[0x5622a4969509] > > > --
[gem5-users] Re: Fail to bootup with KVM in se.py on X86 arch
Hi Jason： I make two tests： 1. I write a test code follow "open /dev/kvm” "create vm” "create vcpu”，it has no error； 2. I put allocate pmem and create vcpu in create_vm function in gem5，not outside create_vm，it has no error. so I suspect if the allocate pmem function in gem5 source has any wrongs？ 发件人： Jason Lowe-Powermailto:ja...@lowepower.com>> 收件人： Liyichaomailto:liyic...@huawei.com>> 抄送： gem5 users mailing listmailto:gem5-users@gem5.org>> 主题： Re: [gem5-users] Fail to bootup with KVM in se.py on X86 arch 时间： 2021-05-11 22:47:19 Hello, I wonder if you have a maximum number of vcpus set on your host system. Otherwise, I can't think of any specific limitation to creating vcpus. Cheers, Jason On Tue, May 11, 2021 at 2:36 AM Liyichao mailto:liyic...@huawei.com>> wrote: Hi Jason: I use strace to follow the call stack, I find that the ioctl() with KVM_CREATE_VCPU returned EEXIST errno, this means the vCPU exist. ioctl(5, KVM_CREATE_VCPU, 2)= -1 EEXIST (File exists) 发件人: Liyichao 发送时间: 2021年5月11日 12:54 收件人: 'Jason Lowe-Power' mailto:ja...@lowepower.com>>; gem5 users mailing list mailto:gem5-users@gem5.org>> 主题: 答复: [gem5-users] Fail to bootup with KVM in se.py on X86 arch Hi Jason: I have add a DPRINTF in line 559 of vm.cc , it showed “ 0: system.kvm_vm: *debug vcpuID is 0” Any KVM use limitations in X86? gem5 version 21.0.0.0 gem5 compiled May 11 2021 01:04:20 gem5 started May 11 2021 01:09:26 gem5 executing on ubuntu, pid 53534 command line: ./build/X86/gem5.opt --debug-flags=Kvm configs/example/se.py --cpu-type=X86KvmCPU --cpu-clock=2.6GHz --sys-clock=2.6GHz --caches --l1d_size=64kB --l1i_size=64kB --l2cache --l2_size=32MB --mem-size=32GB --num-cpus=1 -I 5000 -c ./test warn: membus.slave is deprecated. `slave` is now called `cpu_side_ports` warn: tol2bus.master is deprecated. `master` is now called `mem_side_ports` warn: membus.slave is deprecated. `slave` is now called `cpu_side_ports` warn: tol2bus.slave is deprecated. `slave` is now called `cpu_side_ports` warn: tol2bus.slave is deprecated. `slave` is now called `cpu_side_ports` warn: tol2bus.slave is deprecated. `slave` is now called `cpu_side_ports` warn: tol2bus.slave is deprecated. `slave` is now called `cpu_side_ports` warn: membus.master is deprecated. `master` is now called `mem_side_ports` warn: membus.master is deprecated. `master` is now called `mem_side_ports` warn: membus.slave is deprecated. `slave` is now called `cpu_side_ports` Global frequency set at 1 ticks per second warn: DRAM device capacity (8192 Mbytes) does not match the address range assigned (32768 Mbytes) 0: system.cpu: vcpuID is 0 0: system.remote_gdb: listening for remote gdb on port 7000 0: system.cpu: ActivateContext 0 REAL SIMULATION 0: system.kvm_vm: Mapping 1 memory region(s) 0: system.kvm_vm: Mapping region: 0x0x7f0b007b7000 -> 0x0 [size: 0x8] 0: system.kvm_vm: vmFD is 5, p1 is 140724538190576 0: system.cpu: charlie, vcpuID is 0 0: system.kvm_vm: vmFD is 5, p1 is 0 0: system.kvm_vm: *debug vcpuID is 0 panic: KVM: Failed to create virtual CPU Memory Usage: 33838804 KBytes Program aborted at tick 0 --- BEGIN LIBC BACKTRACE --- ./build/X86/gem5.opt(_Z15print_backtracev+0x2c)[0x5622a4c15f8c] ./build/X86/gem5.opt(_Z12abortHandleri+0x4a)[0x5622a4c30b6a] /lib/x86_64-linux-gnu/libpthread.so.0(+0x12980)[0x7f1305ea6980] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0xc7)[0x7f1304430fb7] /lib/x86_64-linux-gnu/libc.so.6(abort+0x141)[0x7f1304432921] ./build/X86/gem5.opt(+0x6e5cef)[0x5622a3f20cef] ./build/X86/gem5.opt(_ZN5KvmVM10createVCPUEl+0x9ff)[0x5622a4953b8f] ./build/X86/gem5.opt(_ZN10BaseKvmCPU7startupEv+0x9e)[0x5622a494bc8e] ./build/X86/gem5.opt(_ZN9X86KvmCPU7startupEv+0x9)[0x5622a4969509] 发件人: Jason Lowe-Power [mailto:ja...@lowepower.com] 发送时间: 2021年5月10日 23:56 收件人: gem5 users mailing list mailto:gem5-users@gem5.org>> 抄送: Liyichao mailto:liyic...@huawei.com>> 主题: Re: [gem5-users] Fail to bootup with KVM in se.py on X86 arch Hmm, I don't immediately know what's going wrong. I would extend the panic on line 559 of vm.cc to also print the error code number so you can look it up. I believe you can use `errno` like normal after calling `ioctl`. For instance, you could add `strerror(errno)` to the panic. Cheers, Jason On Mon, May 10, 2021 at 12:49 AM Liyichao via gem5-users mailto:gem5-users@gem5.org>> wrote: Hi All: I use KVM CPU in se mode on X86 arch, but it showed a panic “KVM: Failed to create virtual CPU”. My host is X86 server of Intel 6148 and it can support kvm: lsmod |grep kvm kvm_intel 172032 0 kvm 548864 1 kvm_intel irqbypass 16384 1 kvm ll /dev/kvm crwxrwxrwx 1 root root 10, 232 Feb 16 20:40 /dev/kvm My GEM5 version is master(v21.0.0.0), ea7d012c00e857ef999b88a8ec2bde
[gem5-users] Re: Fail to bootup with KVM in se.py on X86 arch
Hello, I wonder if you have a maximum number of vcpus set on your host system. Otherwise, I can't think of any specific limitation to creating vcpus. Cheers, Jason On Tue, May 11, 2021 at 2:36 AM Liyichao wrote: > Hi Jason: > > > > I use strace to follow the call stack, I find that the ioctl() > with KVM_CREATE_VCPU returned EEXIST errno, this means the vCPU exist. > > > >ioctl(5, KVM_CREATE_VCPU, 2)= -1 EEXIST (File > exists) > > > > > > *发件人:* Liyichao > *发送时间:* 2021年5月11日 12:54 > *收件人:* 'Jason Lowe-Power' ; gem5 users mailing list < > gem5-users@gem5.org> > *主题:* 答复: [gem5-users] Fail to bootup with KVM in se.py on X86 arch > > > > Hi Jason: > > I have add a DPRINTF in line 559 of vm.cc , it showed “ 0: > system.kvm_vm: *debug vcpuID is 0” > > > > Any KVM use limitations in X86? > > > > > > gem5 version 21.0.0.0 > > gem5 compiled May 11 2021 01:04:20 > > gem5 started May 11 2021 01:09:26 > > gem5 executing on ubuntu, pid 53534 > > command line: ./build/X86/gem5.opt --debug-flags=Kvm configs/example/se.py > --cpu-type=X86KvmCPU --cpu-clock=2.6GHz --sys-clock=2.6GHz --caches > --l1d_size=64kB --l1i_size=64kB --l2cache --l2_size=32MB --mem-size=32GB > --num-cpus=1 -I 5000 -c ./test > > > > warn: membus.slave is deprecated. `slave` is now called `cpu_side_ports` > > warn: tol2bus.master is deprecated. `master` is now called `mem_side_ports` > > warn: membus.slave is deprecated. `slave` is now called `cpu_side_ports` > > warn: tol2bus.slave is deprecated. `slave` is now called `cpu_side_ports` > > warn: tol2bus.slave is deprecated. `slave` is now called `cpu_side_ports` > > warn: tol2bus.slave is deprecated. `slave` is now called `cpu_side_ports` > > warn: tol2bus.slave is deprecated. `slave` is now called `cpu_side_ports` > > warn: membus.master is deprecated. `master` is now called `mem_side_ports` > > warn: membus.master is deprecated. `master` is now called `mem_side_ports` > > warn: membus.slave is deprecated. `slave` is now called `cpu_side_ports` > > Global frequency set at 1 ticks per second > > warn: DRAM device capacity (8192 Mbytes) does not match the address range > assigned (32768 Mbytes) > > 0: system.cpu: vcpuID is 0 > > 0: system.remote_gdb: listening for remote gdb on port 7000 > > 0: system.cpu: ActivateContext 0 > > REAL SIMULATION > > 0: system.kvm_vm: Mapping 1 memory region(s) > > 0: system.kvm_vm: Mapping region: 0x0x7f0b007b7000 -> 0x0 [size: > 0x8] > > 0: system.kvm_vm: vmFD is 5, p1 is 140724538190576 > > 0: system.cpu: charlie, vcpuID is 0 > > 0: system.kvm_vm: vmFD is 5, p1 is 0 > > 0: system.kvm_vm: *debug vcpuID is 0 > > panic: KVM: Failed to create virtual CPU > > Memory Usage: 33838804 KBytes > > Program aborted at tick 0 > > --- BEGIN LIBC BACKTRACE --- > > ./build/X86/gem5.opt(_Z15print_backtracev+0x2c)[0x5622a4c15f8c] > > ./build/X86/gem5.opt(_Z12abortHandleri+0x4a)[0x5622a4c30b6a] > > /lib/x86_64-linux-gnu/libpthread.so.0(+0x12980)[0x7f1305ea6980] > > /lib/x86_64-linux-gnu/libc.so.6(gsignal+0xc7)[0x7f1304430fb7] > > /lib/x86_64-linux-gnu/libc.so.6(abort+0x141)[0x7f1304432921] > > ./build/X86/gem5.opt(+0x6e5cef)[0x5622a3f20cef] > > ./build/X86/gem5.opt(_ZN5KvmVM10createVCPUEl+0x9ff)[0x5622a4953b8f] > > ./build/X86/gem5.opt(_ZN10BaseKvmCPU7startupEv+0x9e)[0x5622a494bc8e] > > ./build/X86/gem5.opt(_ZN9X86KvmCPU7startupEv+0x9)[0x5622a4969509] > > > -- > > > > *发件人:* Jason Lowe-Power [mailto:ja...@lowepower.com ] > > *发送时间:* 2021年5月10日 23:56 > *收件人:* gem5 users mailing list > *抄送:* Liyichao > *主题:* Re: [gem5-users] Fail to bootup with KVM in se.py on X86 arch > > > > Hmm, I don't immediately know what's going wrong. I would extend the panic > on line 559 of vm.cc to also print the error code number so you can look it > up. I believe you can use `errno` like normal after calling `ioctl`. For > instance, you could add `strerror(errno)` to the panic. > > > > Cheers, > > Jason > > > > > > On Mon, May 10, 2021 at 12:49 AM Liyichao via gem5-users < > gem5-users@gem5.org> wrote: > > Hi All: > > I use KVM CPU in se mode on X86 arch, but it showed a panic “KVM: > Failed to create virtual CPU”. > > > > > > My host is X86 server of Intel 6148 and it can support kvm: > > lsmod |grep kvm > > kvm_intel 172032 0 > > kvm 548864 1 kvm_intel > > irqbypass 16384 1 kvm > > > > ll /dev/kvm > > crwxrwxrwx 1 root root 10, 232 Feb 16 20:40 /dev/kvm > > > >My GEM5 version is master(v21.0.0.0), > ea7d012c00e857ef999b88a8ec2bde801a1f > > > > > > ./build/X86/gem5.opt configs/example/se.py --cpu-type=X86KvmCPU > --cpu-clock=2.6GHz --sys-clock=2.6GHz --caches --l1d_size=64kB > --l1i_size=64kB --l2cache --l2_size=32MB --mem-size=32GB --num-cpus=1 -I > 5000 -c "./test" > > gem5 Simulator System. http://gem5.org > > gem5 is copyrighted software; use the --cop
[gem5-users] Re: Fail to bootup with KVM in se.py on X86 arch
Hmm, I don't immediately know what's going wrong. I would extend the panic on line 559 of vm.cc to also print the error code number so you can look it up. I believe you can use `errno` like normal after calling `ioctl`. For instance, you could add `strerror(errno)` to the panic. Cheers, Jason On Mon, May 10, 2021 at 12:49 AM Liyichao via gem5-users < gem5-users@gem5.org> wrote: > Hi All: > > I use KVM CPU in se mode on X86 arch, but it showed a panic “KVM: > Failed to create virtual CPU”. > > > > > > My host is X86 server of Intel 6148 and it can support kvm: > > lsmod |grep kvm > > kvm_intel 172032 0 > > kvm 548864 1 kvm_intel > > irqbypass 16384 1 kvm > > > > ll /dev/kvm > > crwxrwxrwx 1 root root 10, 232 Feb 16 20:40 /dev/kvm > > > >My GEM5 version is master(v21.0.0.0), > ea7d012c00e857ef999b88a8ec2bde801a1f > > > > > > ./build/X86/gem5.opt configs/example/se.py --cpu-type=X86KvmCPU > --cpu-clock=2.6GHz --sys-clock=2.6GHz --caches --l1d_size=64kB > --l1i_size=64kB --l2cache --l2_size=32MB --mem-size=32GB --num-cpus=1 -I > 5000 -c "./test" > > gem5 Simulator System. http://gem5.org > > gem5 is copyrighted software; use the --copyright option for details. > > > > gem5 version 21.0.0.0 > > gem5 compiled May 10 2021 03:39:51 > > gem5 started May 10 2021 03:57:48 > > gem5 executing on ubuntu, pid 112188 > > command line: ./build/X86/gem5.opt configs/example/se.py > --cpu-type=X86KvmCPU --cpu-clock=2.6GHz --sys-clock=2.6GHz --caches > --l1d_size=64kB --l1i_size=64kB --l2cache --l2_size=32MB --mem-size=32GB > --num-cpus=1 -I 5000 -c ./test > > > > warn: membus.slave is deprecated. `slave` is now called `cpu_side_ports` > > warn: tol2bus.master is deprecated. `master` is now called `mem_side_ports` > > warn: membus.slave is deprecated. `slave` is now called `cpu_side_ports` > > warn: tol2bus.slave is deprecated. `slave` is now called `cpu_side_ports` > > warn: tol2bus.slave is deprecated. `slave` is now called `cpu_side_ports` > > warn: tol2bus.slave is deprecated. `slave` is now called `cpu_side_ports` > > warn: tol2bus.slave is deprecated. `slave` is now called `cpu_side_ports` > > warn: membus.master is deprecated. `master` is now called `mem_side_ports` > > warn: membus.master is deprecated. `master` is now called `mem_side_ports` > > warn: membus.slave is deprecated. `slave` is now called `cpu_side_ports` > > Global frequency set at 1 ticks per second > > warn: DRAM device capacity (8192 Mbytes) does not match the address range > assigned (32768 Mbytes) > > 0: system.remote_gdb: listening for remote gdb on port 7000 > > REAL SIMULATION > > panic: KVM: Failed to create virtual CPU > ___ > gem5-users mailing list -- gem5-users@gem5.org > To unsubscribe send an email to gem5-users-le...@gem5.org > %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s ___ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
