Hi Giacomo,
Thanks for letting me know what the problem is, I am able to verify the
solution on my end as well :-)
Just curious, as I am not familiar with the arm system, do you always
need to provide a separate bootloader to boot arm FS simulation in gem5?
Will the one packed with the kernel work? (I assume when sim x86, it
uses the bootloader from the kernel).
Best,
Wenqi
On 4/9/21 5:09 AM, Giacomo Travaglini wrote:
Hi Wenqi,
I have found the problem; in order to fix it you should recompile the
bootloaders from system/arm/bootloader/arm64 and replace all of them with the
ones in M5_PATH/binaries
That solves the problem on my side.
I am gonna update the guest binaries tarball in gem5.org (I was going to do
that anyway after the 21.0 release)
Kind Regards
Giacomo
-----Original Message-----
From: Giacomo Travaglini
Sent: 07 April 2021 09:13
To: Wenqi Yin <wq...@utexas.edu>; gem5 users mailing list <gem5-
us...@gem5.org>
Subject: RE: [gem5-users] Fail to Boot Multicore Arm System with KVM CPU
Thanks Wenqi,
I was about to suggest the same; please feel free to open a JIRA ticket at:
https://gem5.atlassian.net/jira/software/c/projects/GEM5/issues/
Kind Regards
Giacomo
-----Original Message-----
From: Wenqi Yin <wq...@utexas.edu>
Sent: 07 April 2021 06:40
To: gem5 users mailing list <gem5-users@gem5.org>
Cc: Giacomo Travaglini <giacomo.travagl...@arm.com>
Subject: Re: [gem5-users] Fail to Boot Multicore Arm System with KVM
CPU
Hi All,
The problem turns out to be with the recent release v21.0. I rolled
back to v20.2, it can boot 8+ KVM cpu after applying all the changes
discusses before.
Shall I report this issue anywhere? If so is there any instructions to do so?
Thanks
Best,
Wenqi
On Apr 6, 2021, at 11:54, wq...@utexas.edu wrote:
Hi Giacomo,
I pasted the perhaps most relevant dmesg output of guest kernel and
the
disassembled dtb file snippet (In this case, I use fs_bigLittle.py and
boot 4
vCPU) . Also attached the full dts and dmesg output in case I left anything.
[ 0.000000] CPU features: GIC system register CPU interface present but
disab
led by higher exception level
[ 0.052000] smp: Bringing up secondary CPUs ...
[ 2.004021] random: fast init done
[ 2.132023] CPU1: failed to come online
[ 2.132023] CPU1: failed in unknown state : 0x0
[ 4.244045] CPU2: failed to come online
[ 4.244045] CPU2: failed in unknown state : 0x0
[ 6.356068] CPU3: failed to come online
[ 6.356068] CPU3: failed in unknown state : 0x0
[ 8.468091] CPU4: failed to come online
[ 8.468091] CPU4: failed in unknown state : 0x0
[ 8.468091] smp: Brought up 1 node, 1 CPU
cpus {
#address-cells = <0x1>;
#size-cells = <0x0>;
cpu@0 {
device_type = "cpu";
compatible = "gem5,arm-cpu";
reg = <0x0>;
enable-method = "spin-table";
cpu-release-addr = <0x0 0x87fffff8>;
clock-frequency = <0x773593ff>;
phandle = <0xf>;
};
cpu@1 {
device_type = "cpu";
compatible = "gem5,arm-cpu";
reg = <0x1>;
enable-method = "spin-table";
cpu-release-addr = <0x0 0x87fffff8>;
clock-frequency = <0x773593ff>;
phandle = <0x10>;
};
cpu@2 {
device_type = "cpu";
compatible = "gem5,arm-cpu";
reg = <0x2>;
enable-method = "spin-table";
cpu-release-addr = <0x0 0x87fffff8>;
clock-frequency = <0x773593ff>;
phandle = <0x11>;
};
cpu@3 {
device_type = "cpu";
compatible = "gem5,arm-cpu";
reg = <0x3>;
enable-method = "spin-table";
cpu-release-addr = <0x0 0x87fffff8>;
clock-frequency = <0x773593ff>;
phandle = <0x12>;
};
cpu@104 {
device_type = "cpu";
compatible = "gem5,arm-cpu";
reg = <0x104>;
enable-method = "spin-table";
cpu-release-addr = <0x0 0x87fffff8>;
clock-frequency = <0x3b9ac9ff>;
phandle = <0x13>;
};
};
best,
Wenqi
On 4/6/21 7:35 AM, Giacomo Travaglini wrote:
Hi Wenqi,
Could you provide us with the booting log of the guest kernel?
Kind Regards
Giacomo
-----Original Message-----
From: Wenqi Yin <wq...@utexas.edu>
Sent: 06 April 2021 05:48
To: Giacomo Travaglini <giacomo.travagl...@arm.com>
Cc: gem5 users mailing list <gem5-users@gem5.org>; Wenqi Yin
<wq...@utexas.edu>
Subject: Re: [gem5-users] Fail to Boot Multicore Arm System with
KVM CPU
Hi Giacomo,
Just want to follow up a bit on this, any suggestions on what
could be causing the kernel failing to bring up all those secondary CPUs?
Best,
Wenqi
On Apr 1, 2021, at 16:53, Wenqi Yin <wq...@utexas.edu> wrote:
HI Giacomo,
I add that line of code you pointed me to to the fs_bigLittle.py,
now I can
boot into the terminal, however, the kernel cannot bring all but
one cpu online. It’s also a bit strange in the sense that I
specify ‘-smp 12’, and inside the guest when I check lscpu, it
shows total of 13 CPUs and only cpu0 is online, cpu1-12 are offline.
I tried both the stock kernel as well as compiling kernel myself
following
your suggestions from earlier email, both have the same issue. Do
you have any suggestions on what could be causing the problem?
Best,
Wenqi
On Apr 1, 2021, at 03:28, Giacomo Travaglini
<giacomo.travagl...@arm.com> wrote:
Yes, apologies, I forgot to mention that.
As your host is likely using a GICv3 interrupt controller, you
need to
entirely emulate the GICv2 and Generic Timer in userspace (gem5).
This is done via the simulate_gic (as you have already done) and
by removing the system register interface info from the DTB node
Here's an example:
https://github.com/gem5/gem5/blob/stable/tests/gem5/configs/arm_gene
r
ic.py#L99
Let me know if this works
Kind Regards
Giacomo
-----Original Message-----
From: wq...@utexas.edu <wq...@utexas.edu>
Sent: 31 March 2021 19:18
To: Giacomo Travaglini <giacomo.travagl...@arm.com>; gem5
users
mailing list <gem5-users@gem5.org>; wq...@utexas.edu
Subject: Re: [gem5-users] Fail to Boot Multicore Arm System
with KVM CPU
Hi Giacomo,
Thanks for your reply. I tried the solution you suggested, but
seems there are still problems. Just make sure I understood
correctly, I specified the 'machine-type' as 'VExpress_GEM5_V1'
and in the VExpress_GEM5_V1_Base class's definition
(src/dev/arm/RealView.py),
when instantiating the gic, I use:
gic = kvm_gicv2_class(dist_addr=0x2c001000,
cpu_addr=0x2c002000,
it_lines=512, gem5_extensions=True)
The kvm_gicv2_class will be resolved as MuxingKvmGic, which
inherent from
GicV2 class. To make sure the change applied, I also change the
default value of gem5_extensions from False to True in the
GicV2 class's definition (src/dev/arm/Gic.py). After rebuild,
when starting gem5 there will be a panic says "KVM: failed to
create virtual CPU" (from /src/cpu/kvm/vm.cc, when using ioctl
to create
vcpu)
Another thing I tried is to set the "simulate_gic" of the
MuxingKvmGic class from False to True, this seems help me get
around the vCPU creation failure, however, at some point it
faces an assertion
failure of:
gem5.opt: build/ARM/sim/eventq.hh:763: void
EventQueue::schedule(Event*, Tick, bool): Assertion `when >=
getCurTick()'
failed.
Program aborted at tick 112565277000
Best,
Wenqi
On 3/31/21 7:57 AM, Giacomo Travaglini wrote:
Hi Wenqi,
First of all thanks for the detailed explanation of your problem.
-----Original Message-----
From: wqyin--- via gem5-users <gem5-users@gem5.org>
Sent: 30 March 2021 22:47
To: gem5-users@gem5.org; wq...@utexas.edu
Cc: wq...@utexas.edu
Subject: [gem5-users] Fail to Boot Multicore Arm System with
KVM CPU
Hello all,
I am trying to model a multicore arm64 system with the
example/arm/fs_bigLittle.py using kvm cpu. However, when I
specify
"- machine-type VExpress_Gem5_V2_XXX", the kernel panic
during
booting, giving an error message of:
[ 0.000000] GICv3: Distributor has Range Selector support
[ 0.000000]
GICv3: no VLPI support, direct LPI support [ 0.000000] ITS
[mem
0x2e010000-0x2e02ffff] [ 0.000000] ITS@0x000000002e010000:
allocated
262144 Devices
@fc600000 (flat, esz 8, psz 64K, shr 1)
[ 0.000000] ITS@0x000000002e010000: allocated 8192 Interrupt
Collections
@fc460000 (flat, esz 8, psz 64K, shr 1) [ 0.000000] GIC: using LPI
property
table @0x00000000fc470000 [ 0.000000] ITS: Allocated 1792
chunks
for
LPIs
[ 0.000000] GICv3: CPU0: found redistributor 0 region
0:0x000000002c010000
[ 0.000000] CPU0: using LPI pending table
@0x00000000fc480000
[ 0.000000]
GIC: using cache flushing for LPI property table [ 0.000000]
GICv3:
GIC:
unable to set SRE (disabled at EL2), panic ahead [ 0.000000] ----
---
----
-
[ cut
here ]------------ [ 0.000000] kernel BUG at /work/gem5-
scripts/submodules/linux/arch/arm64/kernel/entry.S:602!
[ 0.000000] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP
[ 0.000000]
Modules linked in:
[ 0.000000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.18.0+
#1
[ 0.000000] Hardware name: V2P-CA15 (DT) [ 0.000000] pstate:
80000085
(Nzcv daIf -PAN -UAO)
Unfortunately KVM with GICv3 is not currently supported.
I saw this error message regardless of how many vCPU I try to
model.
However, when I switch to machine-type of "VExpress-Gem5",
the
guest can boot, but it only allows up to 8 vCPUs. I suppose
this is because in this config it doesn't support GICv3.
For my use case, I do need more than 8 cpus in the modeled
system, can anyone help me here?
We provide a workaround for that, allowing you to use more
than
8 CPUs
with GICv2.
That requires you to set the Gicv2.gem5_extensions parameter
to
True.
A change in the Linux GIC driver is also needed. IIRC the
prebuilt binary
should already contain that.
If not, you would have to rebuild the kernel yourself from the
following
Linux fork:
https://gem5.googlesource.com/arm/linux/+/refs/heads/gem5/v4.15
(Use the gem5_defconfig)
Let me know if it works
Here is the environment I used for the experiment:
Host: aarch64 platform
Gem5 version: v21.0 both stable and develop branch
kernel and disk image: both downloaded from gem5 homepage:
kernel: http://dist.gem5.org/dist/current/arm/aarch-system-
201901106.tar.bz2
disk:
http://dist.gem5.org/dist/current/arm/disks/ubuntu-18.04-
arm64-
docker.img.bz2
launch cmd: build/ARM/gem5.opt
configs/example/arm/fs_bigLITTLE.py
--kernel ../gem5_image/binaries/vmlinux.arm64 --machine-
type
VExpress_GEM5_V2 --disk ../gem5_image/ubuntu-18.04-
arm64-
docker.img
--cpu-type kvm --big-cpus X
Best,
Wenqi
_______________________________________________
gem5-users mailing list -- gem5-users@gem5.org To unsubscribe
send an email to gem5-users- le...@gem5.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
Kind Regards
Giacomo
IMPORTANT NOTICE: The contents of this email and any
attachments
are
confidential and may also be privileged. If you are not the
intended recipient, please notify the sender immediately and do
not disclose the contents to any other person, use it for any
purpose, or store or copy the information in any medium. Thank
you.
IMPORTANT NOTICE: The contents of this email and any
attachments
are
confidential and may also be privileged. If you are not the
intended recipient, please notify the sender immediately and do
not disclose the contents to any other person, use it for any
purpose, or store or copy the information in any medium. Thank you.
IMPORTANT NOTICE: The contents of this email and any attachments
are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose
the contents to any other person, use it for any purpose, or store or
copy the information in any medium. Thank you.
<dmsg.txt><system.dts>
IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended recipient,
please notify the sender immediately and do not disclose the contents to any
other person, use it for any purpose, or store or copy the information in any
medium. Thank you.
_______________________________________________
gem5-users mailing list -- gem5-users@gem5.org
To unsubscribe send an email to gem5-users-le...@gem5.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
