Re: [PATCH v26 01/10] acpi: nvdimm: change NVDIMM_UUID_LE to a common macro

[gengdongjiu]

On 2020/5/12 3:41, Igor Mammedov wrote:
> for future, adding RESEND doesn't make sence here. If you change patches then 
> just bump version.

Igor,
Thanks for the reminder, Just now I submitted a new patchset version to 
avoid this confusion.

Re: [PATCH v26 01/10] acpi: nvdimm: change NVDIMM_UUID_LE to a common macro

[gengdongjiu]

>> +    (node3), (node4), (node5) }
>> +
>>   #define UUID_FMT "%02hhx%02hhx%02hhx%02hhx-" \
>>    "%02hhx%02hhx-%02hhx%02hhx-" \
>>    "%02hhx%02hhx-" \
>> diff --git a/slirp b/slirp
>> index 2faae0f..55ab21c 16
>> --- a/slirp
>> +++ b/slirp
>> @@ -1 +1 @@
>> -Subproject commit 2faae0f778f818fadc873308f983289df697eb93
>> +Subproject commit 55ab21c9a36852915b81f1b41ebaf3b6509dd8ba
> 
> The SLiRP submodule change is certainly unrelated.

Thanks Philippe's review and comments. I submitted another patchset "[PATCH 
RESEND v26 00/10] Add ARMv8 RAS virtualization support in QEMU" to fix it, 
please review that patchset.

> 
> 
> .
> 

Re: [PATCH v26 01/10] acpi: nvdimm: change NVDIMM_UUID_LE to a common macro

[gengdongjiu]

On 2020/5/11 20:29, Philippe Mathieu-Daudé wrote:
>> @@ -1 +1 @@
>> -Subproject commit 2faae0f778f818fadc873308f983289df697eb93
>> +Subproject commit 55ab21c9a36852915b81f1b41ebaf3b6509dd8ba
> 
> The SLiRP submodule change is certainly unrelated.

Thanks Philippe's review and comments. I submitted another patchset "[PATCH 
RESEND v26 00/10] Add ARMv8 RAS virtualization support in QEMU" to fix it, 
please review that patchset.

Re: [PATCH v25 00/10] Add ARMv8 RAS virtualization support in QEMU

[gengdongjiu]

On 2020/5/7 4:25, Michael S. Tsirkin wrote:
> On Wed, May 06, 2020 at 07:42:19PM +0800, gengdongjiu wrote:
>> On 2020/4/17 21:32, Peter Maydell wrote:
>>> On Fri, 10 Apr 2020 at 12:46, Dongjiu Geng  wrote:
>>>>
>>>> In the ARMv8 platform, the CPU error types includes synchronous external 
>>>> abort(SEA)
>>>> and SError Interrupt (SEI). If exception happens in guest, host does not 
>>>> know the detailed
>>>> information of guest, so it is expected that guest can do the recovery. 
>>>> For example, if an
>>>> exception happens in a guest user-space application, host does not know 
>>>> which application
>>>> encounters errors, only guest knows it.
>>>>
>>>> For the ARMv8 SEA/SEI, KVM or host kernel delivers SIGBUS to notify 
>>>> userspace.
>>>> After user space gets the notification, it will record the CPER into guest 
>>>> GHES
>>>> buffer and inject an exception or IRQ to guest.
>>>>
>>>> In the current implementation, if the type of SIGBUS is BUS_MCEERR_AR, we 
>>>> will
>>>> treat it as a synchronous exception, and notify guest with ARMv8 SEA
>>>> notification type after recording CPER into guest.
>>>
>>> Hi. I left a comment on patch 1. The other 3 patches unreviewed
>>> are 5, 6 and 8, which are all ACPI core code, so that's for
>>> MST, Igor or Shannon to review.
>>>
>>> Once those have been reviewed, please ping me if you want this
>>> to go via target-arm.next.
>>
>> Hi Peter,
>>Igor have reviewed all ACPI core code. whether you can apply this series 
>> to target-arm.next I can make another patches to solve your comments on 
>> patch1 and another APCI comment.
>> Thanks very much in advance.
> 
> Given it all starts with patch 1, it's probably easier to address the
> comment and repost.

  Done.

  Hi Peter,
  Please review the patch 1 in the patchset v26. Thanks.
> 
> 
>>>
>>> thanks
>>> -- PMM
>>>
>>> .
>>>
> 
> .
> 

Re: [PATCH v25 00/10] Add ARMv8 RAS virtualization support in QEMU

[gengdongjiu]

On 2020/5/7 4:25, Michael S. Tsirkin wrote:
> On Wed, May 06, 2020 at 07:42:19PM +0800, gengdongjiu wrote:
>> On 2020/4/17 21:32, Peter Maydell wrote:
>>> On Fri, 10 Apr 2020 at 12:46, Dongjiu Geng  wrote:
>>>>
>>>> In the ARMv8 platform, the CPU error types includes synchronous external 
>>>> abort(SEA)
>>>> and SError Interrupt (SEI). If exception happens in guest, host does not 
>>>> know the detailed
>>>> information of guest, so it is expected that guest can do the recovery. 
>>>> For example, if an
>>>> exception happens in a guest user-space application, host does not know 
>>>> which application
>>>> encounters errors, only guest knows it.
>>>>
>>>> For the ARMv8 SEA/SEI, KVM or host kernel delivers SIGBUS to notify 
>>>> userspace.
>>>> After user space gets the notification, it will record the CPER into guest 
>>>> GHES
>>>> buffer and inject an exception or IRQ to guest.
>>>>
>>>> In the current implementation, if the type of SIGBUS is BUS_MCEERR_AR, we 
>>>> will
>>>> treat it as a synchronous exception, and notify guest with ARMv8 SEA
>>>> notification type after recording CPER into guest.
>>>
>>> Hi. I left a comment on patch 1. The other 3 patches unreviewed
>>> are 5, 6 and 8, which are all ACPI core code, so that's for
>>> MST, Igor or Shannon to review.
>>>
>>> Once those have been reviewed, please ping me if you want this
>>> to go via target-arm.next.
>>
>> Hi Peter,
>>Igor have reviewed all ACPI core code. whether you can apply this series 
>> to target-arm.next I can make another patches to solve your comments on 
>> patch1 and another APCI comment.
>> Thanks very much in advance.
> 
> Given it all starts with patch 1, it's probably easier to address the
> comment and repost.

Ok, I will do it. thanks.

> 
> 
>>>
>>> thanks
>>> -- PMM
>>>
>>> .
>>>
> 
> .
> 

Re: [PATCH v25 00/10] Add ARMv8 RAS virtualization support in QEMU

[gengdongjiu]

On 2020/4/17 21:32, Peter Maydell wrote:
> On Fri, 10 Apr 2020 at 12:46, Dongjiu Geng  wrote:
>>
>> In the ARMv8 platform, the CPU error types includes synchronous external 
>> abort(SEA)
>> and SError Interrupt (SEI). If exception happens in guest, host does not 
>> know the detailed
>> information of guest, so it is expected that guest can do the recovery. For 
>> example, if an
>> exception happens in a guest user-space application, host does not know 
>> which application
>> encounters errors, only guest knows it.
>>
>> For the ARMv8 SEA/SEI, KVM or host kernel delivers SIGBUS to notify 
>> userspace.
>> After user space gets the notification, it will record the CPER into guest 
>> GHES
>> buffer and inject an exception or IRQ to guest.
>>
>> In the current implementation, if the type of SIGBUS is BUS_MCEERR_AR, we 
>> will
>> treat it as a synchronous exception, and notify guest with ARMv8 SEA
>> notification type after recording CPER into guest.
> 
> Hi. I left a comment on patch 1. The other 3 patches unreviewed
> are 5, 6 and 8, which are all ACPI core code, so that's for
> MST, Igor or Shannon to review.
> 
> Once those have been reviewed, please ping me if you want this
> to go via target-arm.next.

Hi Peter,
   Igor have reviewed all ACPI core code. whether you can apply this series to 
target-arm.next? I can make another patches to solve your comments on patch1 
and another APCI comment.
Thanks very much in advance.

> 
> thanks
> -- PMM
> 
> .
> 

Re: [PATCH v25 08/10] ACPI: Record Generic Error Status Block(GESB) table

[gengdongjiu]

On 2020/5/5 18:44, Igor Mammedov wrote:
>> Signed-off-by: Dongjiu Geng 
>> Signed-off-by: Xiang Zheng 
> Reviewed-by: Igor Mammedov 
> 
> Also we should ratelimit error messages that could be triggered at runtime
> from acpi_ghes_record_errors() and functions it's calling.
> It could be a patch on top.

Ok, thanks Igor.
I can make another patch for that after this series is applied.

> 
> 

Re: [PATCH v25 00/10] Add ARMv8 RAS virtualization support in QEMU

[gengdongjiu]

> 
> On Thu, 30 Apr 2020 11:56:24 +0800
> gengdongjiu  wrote:
> 
> > On 2020/4/17 21:32, Peter Maydell wrote:
> > > On Fri, 10 Apr 2020 at 12:46, Dongjiu Geng  wrote:
> > >>
> > >> In the ARMv8 platform, the CPU error types includes synchronous
> > >> external abort(SEA) and SError Interrupt (SEI). If exception
> > >> happens in guest, host does not know the detailed information of
> > >> guest, so it is expected that guest can do the recovery. For
> > >> example, if an exception happens in a guest user-space application, host 
> > >> does not know which application encounters errors, only
> guest knows it.
> > >>
> > >> For the ARMv8 SEA/SEI, KVM or host kernel delivers SIGBUS to notify 
> > >> userspace.
> > >> After user space gets the notification, it will record the CPER
> > >> into guest GHES buffer and inject an exception or IRQ to guest.
> > >>
> > >> In the current implementation, if the type of SIGBUS is
> > >> BUS_MCEERR_AR, we will treat it as a synchronous exception, and
> > >> notify guest with ARMv8 SEA notification type after recording CPER into 
> > >> guest.
> > >
> > > Hi. I left a comment on patch 1. The other 3 patches unreviewed are
> > > 5, 6 and 8, which are all ACPI core code, so that's for MST, Igor or
> > > Shannon to review.
> >
> > Ping MST, Igor and Shannon, sorry for the noise.
> 
> I put it on my review queue

Igor, thank you very much in advance.

> 
> >
> > >
> > > Once those have been reviewed, please ping me if you want this to go
> > > via target-arm.next.
> > >
> > > thanks
> > > -- PMM
> > >
> > > .
> > >
> >

Re: [PATCH v25 00/10] Add ARMv8 RAS virtualization support in QEMU

[gengdongjiu]

On 2020/4/17 21:32, Peter Maydell wrote:
> On Fri, 10 Apr 2020 at 12:46, Dongjiu Geng  wrote:
>>
>> In the ARMv8 platform, the CPU error types includes synchronous external 
>> abort(SEA)
>> and SError Interrupt (SEI). If exception happens in guest, host does not 
>> know the detailed
>> information of guest, so it is expected that guest can do the recovery. For 
>> example, if an
>> exception happens in a guest user-space application, host does not know 
>> which application
>> encounters errors, only guest knows it.
>>
>> For the ARMv8 SEA/SEI, KVM or host kernel delivers SIGBUS to notify 
>> userspace.
>> After user space gets the notification, it will record the CPER into guest 
>> GHES
>> buffer and inject an exception or IRQ to guest.
>>
>> In the current implementation, if the type of SIGBUS is BUS_MCEERR_AR, we 
>> will
>> treat it as a synchronous exception, and notify guest with ARMv8 SEA
>> notification type after recording CPER into guest.
> 
> Hi. I left a comment on patch 1. The other 3 patches unreviewed
> are 5, 6 and 8, which are all ACPI core code, so that's for
> MST, Igor or Shannon to review.

Ping MST, Igor and Shannon, sorry for the noise.

> 
> Once those have been reviewed, please ping me if you want this
> to go via target-arm.next.
> 
> thanks
> -- PMM
> 
> .
> 

Re: [PATCH v25 00/10] Add ARMv8 RAS virtualization support in QEMU

[gengdongjiu]

On 2020/4/17 21:32, Peter Maydell wrote:
> On Fri, 10 Apr 2020 at 12:46, Dongjiu Geng  wrote:
>>
>> In the ARMv8 platform, the CPU error types includes synchronous external 
>> abort(SEA)
>> and SError Interrupt (SEI). If exception happens in guest, host does not 
>> know the detailed
>> information of guest, so it is expected that guest can do the recovery. For 
>> example, if an
>> exception happens in a guest user-space application, host does not know 
>> which application
>> encounters errors, only guest knows it.
>>
>> For the ARMv8 SEA/SEI, KVM or host kernel delivers SIGBUS to notify 
>> userspace.
>> After user space gets the notification, it will record the CPER into guest 
>> GHES
>> buffer and inject an exception or IRQ to guest.
>>
>> In the current implementation, if the type of SIGBUS is BUS_MCEERR_AR, we 
>> will
>> treat it as a synchronous exception, and notify guest with ARMv8 SEA
>> notification type after recording CPER into guest.
> 
> Hi. I left a comment on patch 1. The other 3 patches unreviewed
> are 5, 6 and 8, which are all ACPI core code, so that's for
> MST, Igor or Shannon to review.

Hi MST, Igor or Shannon
when you have time, could you review 5, 6 and 8? thanks very much in 
advance.
It seems this series of patches lasted a long time, hope they can be applied 
soon.

> 
> Once those have been reviewed, please ping me if you want this
> to go via target-arm.next>
> thanks
> -- PMM
> 
> .
> 

Re: [PATCH v25 00/10] Add ARMv8 RAS virtualization support in QEMU

[gengdongjiu]

On 2020/4/17 21:32, Peter Maydell wrote:
> On Fri, 10 Apr 2020 at 12:46, Dongjiu Geng  wrote:
>>
>> In the ARMv8 platform, the CPU error types includes synchronous external 
>> abort(SEA)
>> and SError Interrupt (SEI). If exception happens in guest, host does not 
>> know the detailed
>> information of guest, so it is expected that guest can do the recovery. For 
>> example, if an
>> exception happens in a guest user-space application, host does not know 
>> which application
>> encounters errors, only guest knows it.
>>
>> For the ARMv8 SEA/SEI, KVM or host kernel delivers SIGBUS to notify 
>> userspace.
>> After user space gets the notification, it will record the CPER into guest 
>> GHES
>> buffer and inject an exception or IRQ to guest.
>>
>> In the current implementation, if the type of SIGBUS is BUS_MCEERR_AR, we 
>> will
>> treat it as a synchronous exception, and notify guest with ARMv8 SEA
>> notification type after recording CPER into guest.
> 
> Hi. I left a comment on patch 1. The other 3 patches unreviewed
> are 5, 6 and 8, which are all ACPI core code, so that's for
> MST, Igor or Shannon to review.
> 
> Once those have been reviewed, please ping me if you want this
> to go via target-arm.next.

Ok, Thanks very much for the review and comments.

> 
> thanks
> -- PMM
> 
> .
> 

RE: [PATCH v25 00/10] Add ARMv8 RAS virtualization support in QEMU

[gengdongjiu]

ok，thanks very much for peter's time and reply

发件人：Peter Maydell 
收件人：gengdongjiu 
抄 送：imammedo ;mst ;xiaoguangrong.eric 
;shannon.zhaosl ;fam 
;rth ;Eduardo Habkost 
;mtosatti ;qemu-devel 
;kvm ;qemu-arm 
;pbonzini ;zhengxiang (A) 
;Linuxarm ;Jonathan Cameron 

时 间：2020-04-16 22:02:34
主 题：Re: [PATCH v25 00/10] Add ARMv8 RAS virtualization support in QEMU

On Thu, 16 Apr 2020 at 14:54, gengdongjiu  wrote:
>
> ping

Hi; this is on my to-review queue, but so are 25 other patchsets.
(I've built up a bit of a backlog due to concentrating on work
for the 5.0 release while we're in the freeze period.) I will
get to it eventually if nobody else does first...

thanks
-- PMM

RE: [PATCH v25 00/10] Add ARMv8 RAS virtualization support in QEMU

[gengdongjiu]

ping

发件人：gengdongjiu 
收件人：imammedo ;mst ;xiaoguangrong.eric 
;Peter Maydell 
;shannon.zhaosl ;fam 
;rth ;Eduardo Habkost 
;mtosatti ;qemu-devel 
;kvm ;qemu-arm 
;pbonzini 
抄 送：gengdongjiu ;zhengxiang (A) 
;Jonathan Cameron 
;Linuxarm 
时 间：2020-04-10 19:45:28
主 题：[PATCH v25 00/10] Add ARMv8 RAS virtualization support in QEMU

In the ARMv8 platform, the CPU error types includes synchronous external 
abort(SEA)
and SError Interrupt (SEI). If exception happens in guest, host does not know 
the detailed
information of guest, so it is expected that guest can do the recovery. For 
example, if an
exception happens in a guest user-space application, host does not know which 
application
encounters errors, only guest knows it.

For the ARMv8 SEA/SEI, KVM or host kernel delivers SIGBUS to notify userspace.
After user space gets the notification, it will record the CPER into guest GHES
buffer and inject an exception or IRQ to guest.

In the current implementation, if the type of SIGBUS is BUS_MCEERR_AR, we will
treat it as a synchronous exception, and notify guest with ARMv8 SEA
notification type after recording CPER into guest.

A) This series of patches are based on Qemu 4.2, which include two parts:
1. Generate APEI/GHES table.
2. Handle the SIGBUS signal, record the CPER in runtime and fill it into guest
   memory, then notify guest according to the type of SIGBUS.

B) The solution was suggested by James(james.mo...@arm.com); The APEI part 
solution was suggested by Laszlo(ler...@redhat.com). Show some discussions in 
[1].

C) This series of patches have already been tested on ARM64 platform with RAS
feature enabled:
1. Show the APEI part verification result in [2].
2. Show the SIGBUS of BUS_MCEERR_AR handling verification result in [3].

D) Add 'ras' option in command Line to enable guest RAS error recovery feature, 
For example:
KVM model: ./qemu-system-aarch64 --enable-kvm -cpu host --bios QEMU_EFI.fd_new  
-machine virt,gic-version=3,ras,kernel-irqchip=on
-smp 4 -nographic -kernel Image  -append "rdinit=/init console=ttyAMA0 mem=512M 
root=/dev/ram0" -initrd guestfs_new.cpio.gz
TCG model: ./qemu-system-aarch64 -cpu cortex-a57 --bios QEMU_EFI.fd_new  
-machine virt,gic-version=3,ras,kernel-irqchip=on  -smp 4
-nographic -kernel Image  -append "rdinit=/init console=ttyAMA0 mem=512M 
root=/dev/ram0" -initrd guestfs_new.cpio.gz
---
Change since v23:
1. fix a warning for uuid

Change since v22:
1. Using 1 * KiB instead of 0x400 to define max size of one error block
2. Make the alignment to 8 bytes in bios_linker_loader_alloc()
3. Change "Copyright (c) 2019" to "Copyright (c) 2020" in file header
4. Fix some code style warnings/errors and add some comments in code
5. Address Jonathan's comments to easily support CCIX error injection
6. Add vmstate_ghes_state .subsections in vmstate_acpi_ged

Change since v21:
1. Make the user-facing 'ras' option description more clearly to address 
Peter's comments.
2. Update the doc description in "docs/specs/acpi_hest_ghes.rst"
3. Split HEST/GHES patches to more patches to make the review easily
4. Using source_id to index the location to save the CPER.
5. Optimize and simplify the logic to build HEST/GHES table to address 
Igor/Michael/Beata comments.
6. make ghes_addr_le a part of GED device.

Change since v20:
1. Move some implementation details from acpi_ghes.h to acpi_ghes.c
2. Add the reviewers for the ACPI/APEI/GHES part

Change since v19:
1. Fix clang compile error
2. Fix sphinx build error

Change since v18:
1. Fix some code-style and typo/grammar problems.
2. Remove no_ras in the VirtMachineClass struct.
3. Convert documentation to rst format.
4. Simplize the code and add comments for some magic value.
5. Move kvm_inject_arm_sea() function into the patch where it's used.
6. Register the reset handler(kvm_unpoison_all()) in the kvm_init() function.

Change since v17:
1. Improve some commit messages and comments.
2. Fix some code-style problems.
3. Add a *ras* machine option.
4. Move HEST/GHES related structures and macros into "hw/acpi/acpi_ghes.*".
5. Move HWPoison page functions into "include/sysemu/kvm_int.h".
6. Fix some bugs.
7. Improve the design document.

Change since v16:
1. check whether ACPI table is enabled when handling the memory error in the 
SIGBUS handler.

Change since v15:
1. Add a doc-comment in the proper format for 'include/exec/ram_addr.h'
2. Remove write_part_cpustate_to_list() because there is another bug fix patch
   has been merged "arm: Allow system registers for KVM guests to be changed by 
QEMU code"
3. Add some comments for kvm_inject_arm_sea() in 'target/arm/kvm64.c'
4. Compare the arm_current_el() return value to 0,1,2,3, not to PSTATE_MODE_* 
constants.
5. Change the RAS support wasn't introduced before 4.1 QEMU version.
6. Move the no_ras flag  patch to begin in this series

Change since v14:
1. Remove the BUS_MCEERR_AO handling logic because thi

Re: [PATCH v24 00/10] Add ARMv8 RAS virtualization support in QEMU

[gengdongjiu]

On 2020/2/26 0:59, Igor Mammedov wrote:
> On Mon, 24 Feb 2020 16:37:44 +0800
> gengdongjiu  wrote:
> 
>> On 2020/2/21 22:09, Peter Maydell wrote:
>>> On Mon, 17 Feb 2020 at 13:10, Dongjiu Geng  wrote:  
>>>>
>>>> In the ARMv8 platform, the CPU error types includes synchronous external 
>>>> abort(SEA) and SError Interrupt (SEI). If exception happens in guest, host 
>>>> does not know the detailed information of guest, so it is expected that 
>>>> guest can do the recovery.
>>>> For example, if an exception happens in a guest user-space application, 
>>>> host does
>>>> not know which application encounters errors, only guest knows it.
>>>>
>>>> For the ARMv8 SEA/SEI, KVM or host kernel delivers SIGBUS to notify 
>>>> userspace.
>>>> After user space gets the notification, it will record the CPER into guest 
>>>> GHES
>>>> buffer and inject an exception or IRQ to guest.
>>>>
>>>> In the current implementation, if the type of SIGBUS is BUS_MCEERR_AR, we 
>>>> will
>>>> treat it as a synchronous exception, and notify guest with ARMv8 SEA
>>>> notification type after recording CPER into guest.  
>>>
>>> Hi; I have reviewed the remaining arm bit of this series (patch 9),
>>> and made some comments on patch 1. Still to be reviewed are
>>> patches 4, 5, 6, 8: I'm going to assume that Michael or Igor
>>> will look at those.  
>>
>> Thanks very much for Peter's review.
>> Michael/Igor, hope you can review patches 4, 5, 6, 8, thank you very much in 
>> advance.
> 
> done

Thanks a lot, do you think whether it is ready to merge if the comments that 
you mentioned are fixed?
If there are some other places that still needs to modify, hope you can pointed 
it out, so that in next patch series it can be ready to merge. thanks.

> 
>>>
>>> thanks
>>> -- PMM
>>>
>>> .
>>>   
>>
> 
> .
> 

Re: [PATCH v24 02/10] hw/arm/virt: Introduce a RAS machine option

[gengdongjiu]

> 
> On Tue, 25 Feb 2020 08:54:07 +
> Peter Maydell  wrote:
> 
> > On Tue, 25 Feb 2020 at 08:34, Igor Mammedov  wrote:
> > >
> > > On Mon, 17 Feb 2020 21:12:40 +0800
> > > Dongjiu Geng  wrote:
> > >
> > > > RAS Virtualization feature is not supported now, so add a RAS
> > > > machine
> > >
> > > > option and disable it by default.
> > >  
> > >
> > > this doesn't match the patch.
> >
> > Hmm? It seems right to me -- the patch adds a machine option and makes
> > it disabled-by-default, doesn't it?
> 
> Right, I misread 'and' as 'to' somehow.
> My apologies

Thanks Peter and Igor's clarification.

> 
> >
> > thanks
> > -- PMM
> >

Re: [PATCH v24 00/10] Add ARMv8 RAS virtualization support in QEMU

[gengdongjiu]

On 2020/2/21 22:09, Peter Maydell wrote:
> On Mon, 17 Feb 2020 at 13:10, Dongjiu Geng  wrote:
>>
>> In the ARMv8 platform, the CPU error types includes synchronous external 
>> abort(SEA) and SError Interrupt (SEI). If exception happens in guest, host 
>> does not know the detailed information of guest, so it is expected that 
>> guest can do the recovery.
>> For example, if an exception happens in a guest user-space application, host 
>> does
>> not know which application encounters errors, only guest knows it.
>>
>> For the ARMv8 SEA/SEI, KVM or host kernel delivers SIGBUS to notify 
>> userspace.
>> After user space gets the notification, it will record the CPER into guest 
>> GHES
>> buffer and inject an exception or IRQ to guest.
>>
>> In the current implementation, if the type of SIGBUS is BUS_MCEERR_AR, we 
>> will
>> treat it as a synchronous exception, and notify guest with ARMv8 SEA
>> notification type after recording CPER into guest.
> 
> Hi; I have reviewed the remaining arm bit of this series (patch 9),
> and made some comments on patch 1. Still to be reviewed are
> patches 4, 5, 6, 8: I'm going to assume that Michael or Igor
> will look at those.

Thanks very much for Peter's review.
Michael/Igor, hope you can review patches 4, 5, 6, 8, thank you very much in 
advance.


> 
> thanks
> -- PMM
> 
> .
> 

Re: [PATCH v22 5/9] ACPI: Record the Generic Error Status Block address

[gengdongjiu]

On 2020/1/17 0:44, Peter Maydell wrote:
> On Wed, 8 Jan 2020 at 11:33, Dongjiu Geng  wrote:
>>
>> Record the GHEB address via fw_cfg file, when recording
>> a error to CPER, it will use this address to find out
>> Generic Error Data Entries and write the error.
>>
>> Make the HEST GHES to a GED device.
>>
>> Signed-off-by: Dongjiu Geng 
>> Signed-off-by: Xiang Zheng 
>> ---
>>  hw/acpi/generic_event_device.c | 15 ++-
>>  hw/acpi/ghes.c | 16 
>>  hw/arm/virt-acpi-build.c   | 13 -
>>  include/hw/acpi/generic_event_device.h |  2 ++
>>  include/hw/acpi/ghes.h |  6 ++
>>  5 files changed, 50 insertions(+), 2 deletions(-)
>>
>> diff --git a/hw/acpi/generic_event_device.c b/hw/acpi/generic_event_device.c
>> index 9cee90c..9bf37e4 100644
>> --- a/hw/acpi/generic_event_device.c
>> +++ b/hw/acpi/generic_event_device.c
>> @@ -234,12 +234,25 @@ static const VMStateDescription vmstate_ged_state = {
>>  }
>>  };
>>
>> +static const VMStateDescription vmstate_ghes_state = {
>> +.name = "acpi-ghes-state",
>> +.version_id = 1,
>> +.minimum_version_id = 1,
>> +.fields  = (VMStateField[]) {
>> +VMSTATE_UINT64(ghes_addr_le, AcpiGhesState),
>> +VMSTATE_END_OF_LIST()
>> +}
>> +};
>> +
>>  static const VMStateDescription vmstate_acpi_ged = {
>>  .name = "acpi-ged",
>>  .version_id = 1,
>>  .minimum_version_id = 1,
>>  .fields = (VMStateField[]) {
>> -VMSTATE_STRUCT(ged_state, AcpiGedState, 1, vmstate_ged_state, 
>> GEDState),
>> +VMSTATE_STRUCT(ged_state, AcpiGedState, 1,
>> +   vmstate_ged_state, GEDState),
>> +VMSTATE_STRUCT(ghes_state, AcpiGedState, 1,
>> +   vmstate_ghes_state, AcpiGhesState),
>>  VMSTATE_END_OF_LIST(),
>>  },
>>  .subsections = (const VMStateDescription * []) {
> 
> You can't just add fields to an existing VMStateDescription
> like this -- it will break migration compatibility. Instead you
> need to add a new subsection to this vmstate, with a '.needed'
> function which indicates when the subsection should be present.

Hi Peter/Igor
   In order to avoid migration failure, do you think whether below change is Ok 
to make error table address(AcpiGhesState) to a part of GED device? thanks a 
lot in advance.

-
diff --git a/hw/acpi/generic_event_device.c b/hw/acpi/generic_event_device.c
index 021ed2b..264154d 100644
--- a/hw/acpi/generic_event_device.c
+++ b/hw/acpi/generic_event_device.c
@@ -234,16 +234,34 @@ static const VMStateDescription vmstate_ged_state = {
 }
 };

+static bool ghes_needed(void *opaque)
+{
+return object_property_get_bool(qdev_get_machine(), "ras", NULL);
+}
+
+static const VMStateDescription vmstate_ghes_state = {
+.name = "acpi-ged/ghes",
+.version_id = 1,
+.minimum_version_id = 1,
+.needed = ghes_needed,
+.fields  = (VMStateField[]) {
+VMSTATE_STRUCT(ghes_state, AcpiGedState, 1, vmstate_ghes_state, 
AcpiGhesState),
+VMSTATE_END_OF_LIST()
+}
+};
+
 static const VMStateDescription vmstate_acpi_ged = {
 .name = "acpi-ged",
 .version_id = 1,
 .minimum_version_id = 1,
 .fields = (VMStateField[]) {
-VMSTATE_STRUCT(ged_state, AcpiGedState, 1, vmstate_ged_state, 
GEDState),
+VMSTATE_STRUCT(ged_state, AcpiGedState, 1,
+   vmstate_ged_state, GEDState),
 VMSTATE_END_OF_LIST(),
 },
 .subsections = (const VMStateDescription * []) {
 _memhp_state,
+_ghes_state,
 NULL
 }
 };
diff --git a/hw/acpi/ghes.c b/hw/acpi/ghes.c
index a67b1de..3bf32ec 100644
--- a/hw/acpi/ghes.c
+++ b/hw/acpi/ghes.c
@@ -24,6 +24,7 @@
 #include "hw/acpi/acpi.h"
 #include "hw/acpi/ghes.h"
 #include "hw/acpi/aml-build.h"
+#include "hw/acpi/generic_event_device.h"
 #include "hw/nvram/fw_cfg.h"
 #include "sysemu/sysemu.h"
 #include "qemu/error-report.h"
@@ -216,3 +217,18 @@ void acpi_build_hest(GArray *table_data, BIOSLinker 
*linker)
 build_header(linker, table_data, (void *)(table_data->data + hest_start),
 "HEST", table_data->len - hest_start, 1, NULL, "");
 }
+
+void acpi_ghes_add_fw_cfg(AcpiGhesState *ags, FWCfgState *s,
+  GArray *hardware_error)
+{
+size_t size = 2 * sizeof(uint64_t) + ACPI_GHES_MAX_RAW_DATA_LENGTH;
+size_t request_block_size = ACPI_GHES_ERROR_SOURCE_COUNT * size;
+
+/* Create a read-only fw_cfg file for GHES */
+fw_cfg_add_file(s, ACPI_GHES_ERRORS_FW_CFG_FILE, hardware_error->data,
+request_block_size);
+
+/* Create a read-write fw_cfg file for Address */
+fw_cfg_add_file_callback(s, ACPI_GHES_DATA_ADDR_FW_CFG_FILE, NULL, NULL,
+NULL, &(ags->ghes_addr_le), sizeof(ags->ghes_addr_le), false);
+}
diff --git a/hw/arm/virt-acpi-build.c 

Re: [PATCH v22 4/9] ACPI: Build Hardware Error Source Table

[gengdongjiu]

On 2020/2/6 0:43, Jonathan Cameron wrote:
> On Wed, 8 Jan 2020 19:32:18 +0800
> Dongjiu Geng  wrote:
> 
>> This patch builds Hardware Error Source Table(HEST) via fw_cfg blobs.
>> Now it only supports ARMv8 SEA, a type of Generic Hardware Error
>> Source version 2(GHESv2) error source. Afterwards, we can extend
>> the supported types if needed. For the CPER section, currently it
>> is memory section because kernel mainly wants userspace to handle
>> the memory errors.
>>
>> This patch follows the spec ACPI 6.2 to build the Hardware Error
>> Source table. For more detailed information, please refer to
>> document: docs/specs/acpi_hest_ghes.rst
>>
>> build_append_ghes_notify() will help to add Hardware Error Notification
>> to ACPI tables without using packed C structures and avoid endianness
>> issues as API doesn't need explicit conversion.
>>
>> Signed-off-by: Dongjiu Geng 
>> Signed-off-by: Xiang Zheng 
>> Reviewed-by: Michael S. Tsirkin 
>> Acked-by: Xiang Zheng 
> 
> Hi. 
> 
> I was forwards porting my old series adding CCIX error injection support
> and came across a place this could 'possibly' be improved.

Jonathan, It is great that you add CCIX error injection support based on this 
series.
thanks for using it.

> 
> I say possibly because it's really about enabling more flexibility
> in how this code is reused than actually 'fixing' anything here.
> 
> If you don't make the change here, I'll just add a precursor patch to my
> series.  Just seems nice to tidy it up at source.

sure, I make a change to make your patch work well.

> 
> The rest of the partMake your patch very good work.s of this series I am 
> using seems to work great.




> 
> Thanks!
> 
> Jonathan
> 
>> ---
>>  hw/acpi/ghes.c   | 118 
>> ++-
>>  hw/arm/virt-acpi-build.c |   2 +
>>  include/hw/acpi/ghes.h   |  40 
>>  3 files changed, 159 insertions(+), 1 deletion(-)
>>
>> diff --git a/hw/acpi/ghes.c b/hw/acpi/ghes.c
>> index b7fdbbb..9d37798 100644
>> --- a/hw/acpi/ghes.c
>> +++ b/hw/acpi/ghes.c
>> @@ -34,9 +34,42 @@
>> +
> ...
>> +/* Build Generic Hardware Error Source version 2 (GHESv2) */
>> +static void build_ghes_v2(GArray *table_data, int source_id, BIOSLinker 
>> *linker)
> This function takes source ID, which uses the enum of all sources registered.
> However, it doesn't use it to locate the actual physical addresses.
> 
> Currently the code effectively assumes the value is 0.

yes, because there is only one source, so the value is 0.

> 
>> +{
>> +uint64_t address_offset;
>> +/*
>> + * Type:
>> + * Generic Hardware Error Source version 2(GHESv2 - Type 10)
>> + */
>> +build_append_int_noprefix(table_data, 
>> ACPI_GHES_SOURCE_GENERIC_ERROR_V2, 2);
>> +/* Source Id */
>> +build_append_int_noprefix(table_data, source_id, 2);
>> +/* Related Source Id */
>> +build_append_int_noprefix(table_data, 0x, 2);
>> +/* Flags */
>> +build_append_int_noprefix(table_data, 0, 1);
>> +/* Enabled */
>> +build_append_int_noprefix(table_data, 1, 1);
>> +
>> +/* Number of Records To Pre-allocate */
>> +build_append_int_noprefix(table_data, 1, 4);
>> +/* Max Sections Per Record */
>> +build_append_int_noprefix(table_data, 1, 4);
>> +/* Max Raw Data Length */
>> +build_append_int_noprefix(table_data, ACPI_GHES_MAX_RAW_DATA_LENGTH, 4);
>> +
>> +address_offset = table_data->len;
>> +/* Error Status Address */
>> +build_append_gas(table_data, AML_AS_SYSTEM_MEMORY, 0x40, 0,
>> + 4 /* QWord access */, 0);
>> +bios_linker_loader_add_pointer(linker, ACPI_BUILD_TABLE_FILE,
>> +address_offset + GAS_ADDR_OFFSET,
>> +sizeof(uint64_t), ACPI_GHES_ERRORS_FW_CFG_FILE, 0);
> 
> The offset here would need to be source_id * sizeof(uint64_t) I think
> 
>> +
>> +/*
>> + * Notification Structure
>> + * Now only enable ARMv8 SEA notification type
>> + */
>> +build_ghes_hw_error_notification(table_data, ACPI_GHES_NOTIFY_SEA);
> Perhaps a switch for this to allow for other options later.

OK, I will make this change in order to easily support more hardware error 
source.

> 
>   switch (source_id) {
>   case ACPI_HEST_SRC_ID_SEA:
>   ...
>   break;
>   default:
>   //print some error message.
> 
>   }

ok

>> +
>> +/* Error Status Block Length */
>> +build_append_int_noprefix(table_data, ACPI_GHES_MAX_RAW_DATA_LENGTH, 4);
>> +
>> +/*
>> + * Read Ack Register
>> + * ACPI 6.1: 18.3.2.8 Generic Hardware Error Source
>> + * version 2 (GHESv2 - Type 10)
>> + */
>> +address_offset = table_data->len;
>> +build_append_gas(table_data, AML_AS_SYSTEM_MEMORY, 0x40, 0,
>> + 4 /* QWord access */, 0);
>> +bios_linker_loader_add_pointer(linker, ACPI_BUILD_TABLE_FILE,
>> +address_offset + GAS_ADDR_OFFSET,
>> +sizeof(uint64_t), 

Re: [PATCH v22 4/9] ACPI: Build Hardware Error Source Table

[gengdongjiu]

On 2020/1/23 23:48, Igor Mammedov wrote:
> On Wed, 8 Jan 2020 19:32:18 +0800
> Dongjiu Geng  wrote:
> 
>> This patch builds Hardware Error Source Table(HEST) via fw_cfg blobs.
>> Now it only supports ARMv8 SEA, a type of Generic Hardware Error
>> Source version 2(GHESv2) error source. Afterwards, we can extend
>> the supported types if needed. For the CPER section, currently it
>> is memory section because kernel mainly wants userspace to handle
>> the memory errors.
>>
>> This patch follows the spec ACPI 6.2 to build the Hardware Error
>> Source table. For more detailed information, please refer to
>> document: docs/specs/acpi_hest_ghes.rst
>>
>> build_append_ghes_notify() will help to add Hardware Error Notification
>> to ACPI tables without using packed C structures and avoid endianness
>> issues as API doesn't need explicit conversion.
>>
>> Signed-off-by: Dongjiu Geng 
>> Signed-off-by: Xiang Zheng 
>> Reviewed-by: Michael S. Tsirkin 
>> Acked-by: Xiang Zheng 
> 
> 
> Overall it looks fine to me, see couple nits below
> 
> 
>> ---
>>  hw/acpi/ghes.c   | 118 
>> ++-
>>  hw/arm/virt-acpi-build.c |   2 +
>>  include/hw/acpi/ghes.h   |  40 
>>  3 files changed, 159 insertions(+), 1 deletion(-)
>>
>> diff --git a/hw/acpi/ghes.c b/hw/acpi/ghes.c
>> index b7fdbbb..9d37798 100644
>> --- a/hw/acpi/ghes.c
>> +++ b/hw/acpi/ghes.c
>> @@ -34,9 +34,42 @@
>>  
>>  /* The max size in bytes for one error block */
>>  #define ACPI_GHES_MAX_RAW_DATA_LENGTH   0x400
>> -
>>  /* Now only support ARMv8 SEA notification type error source */
>>  #define ACPI_GHES_ERROR_SOURCE_COUNT1
>> +/* Generic Hardware Error Source version 2 */
>> +#define ACPI_GHES_SOURCE_GENERIC_ERROR_V2   10
>> +/* Address offset in Generic Address Structure(GAS) */
>> +#define GAS_ADDR_OFFSET 4
>> +
>> +/*
>> + * Hardware Error Notification
>> + * ACPI 4.0: 17.3.2.7 Hardware Error Notification
>> + * Composes dummy Hardware Error Notification descriptor of specified type
>> + */
>> +static void build_ghes_hw_error_notification(GArray *table, const uint8_t 
>> type)
>> +{
>> +/* Type */
>> +build_append_int_noprefix(table, type, 1);
>> +/*
>> + * Length:
>> + * Total length of the structure in bytes
>> + */
>> +build_append_int_noprefix(table, 28, 1);
>> +/* Configuration Write Enable */
>> +build_append_int_noprefix(table, 0, 2);
>> +/* Poll Interval */
>> +build_append_int_noprefix(table, 0, 4);
>> +/* Vector */
>> +build_append_int_noprefix(table, 0, 4);
>> +/* Switch To Polling Threshold Value */
>> +build_append_int_noprefix(table, 0, 4);
>> +/* Switch To Polling Threshold Window */
>> +build_append_int_noprefix(table, 0, 4);
>> +/* Error Threshold Value */
>> +build_append_int_noprefix(table, 0, 4);
>> +/* Error Threshold Window */
>> +build_append_int_noprefix(table, 0, 4);
>> +}
>>  
>>  /*
>>   * Build table for the hardware error fw_cfg blob.
>> @@ -92,3 +125,86 @@ void build_ghes_error_table(GArray *hardware_errors, 
>> BIOSLinker *linker)
>>  bios_linker_loader_write_pointer(linker, 
>> ACPI_GHES_DATA_ADDR_FW_CFG_FILE,
>>  0, sizeof(uint64_t), ACPI_GHES_ERRORS_FW_CFG_FILE, 0);
>>  }
>> +
>> +/* Build Generic Hardware Error Source version 2 (GHESv2) */
>> +static void build_ghes_v2(GArray *table_data, int source_id, BIOSLinker 
>> *linker)
>> +{
>> +uint64_t address_offset;
>> +/*
>> + * Type:
>> + * Generic Hardware Error Source version 2(GHESv2 - Type 10)
>> + */
>> +build_append_int_noprefix(table_data, 
>> ACPI_GHES_SOURCE_GENERIC_ERROR_V2, 2);
>> +/* Source Id */
>> +build_append_int_noprefix(table_data, source_id, 2);
>> +/* Related Source Id */
>> +build_append_int_noprefix(table_data, 0x, 2);
>> +/* Flags */
>> +build_append_int_noprefix(table_data, 0, 1);
>> +/* Enabled */
>> +build_append_int_noprefix(table_data, 1, 1);
>> +
>> +/* Number of Records To Pre-allocate */
>> +build_append_int_noprefix(table_data, 1, 4);
>> +/* Max Sections Per Record */
>> +build_append_int_noprefix(table_data, 1, 4);
>> +/* Max Raw Data Length */
>> +build_append_int_noprefix(table_data, ACPI_GHES_MAX_RAW_DATA_LENGTH, 4);
>> +
>> +address_offset = table_data->len;
>> +/* Error Status Address */
>> +build_append_gas(table_data, AML_AS_SYSTEM_MEMORY, 0x40, 0,
>> + 4 /* QWord access */, 0);
>> +bios_linker_loader_add_pointer(linker, ACPI_BUILD_TABLE_FILE,
>> +address_offset + GAS_ADDR_OFFSET,
>> +sizeof(uint64_t), ACPI_GHES_ERRORS_FW_CFG_FILE, 0);
>> +
>> +/*
>> + * Notification Structure
>> + * Now only enable ARMv8 SEA notification type
>> + */
>> +build_ghes_hw_error_notification(table_data, ACPI_GHES_NOTIFY_SEA);
>> +
>> +/* Error Status Block Length */
>> +build_append_int_noprefix(table_data, 

Re: [PATCH v22 3/9] ACPI: Build related register address fields via hardware error fw_cfg blob

[gengdongjiu]

On 2020/1/23 23:14, Igor Mammedov wrote:
> On Wed, 8 Jan 2020 19:32:17 +0800
> Dongjiu Geng  wrote:
> 
>> This patch builds error_block_address and read_ack_register fields
>> in hardware errors table , the error_block_address points to Generic
>> Error Status Block(GESB) via bios_linker. The max size for one GESB
>> is 0x1000 in bytes, For more detailed information, please refer to
> 
> s/0x1000/4Kb/

will update it.

> 
>> document: docs/specs/acpi_hest_ghes.rst
>>
>> Now we only support one Error source, if necessary, we can extend to
>> support more.
>>
>> Suggested-by: Laszlo Ersek 
>> Signed-off-by: Dongjiu Geng 
>> Signed-off-by: Xiang Zheng 
>> Reviewed-by: Michael S. Tsirkin 
>> ---
>>  default-configs/arm-softmmu.mak |  1 +
>>  hw/acpi/Kconfig |  4 ++
>>  hw/acpi/Makefile.objs   |  1 +
>>  hw/acpi/aml-build.c |  2 +
>>  hw/acpi/ghes.c  | 94 
>> +
>>  hw/arm/virt-acpi-build.c|  6 +++
>>  include/hw/acpi/aml-build.h |  1 +
>>  include/hw/acpi/ghes.h  | 26 
>>  8 files changed, 135 insertions(+)
>>  create mode 100644 hw/acpi/ghes.c
>>  create mode 100644 include/hw/acpi/ghes.h
>>
>> diff --git a/default-configs/arm-softmmu.mak 
>> b/default-configs/arm-softmmu.mak
>> index 1f2e0e7..5722f31 100644
>> --- a/default-configs/arm-softmmu.mak
>> +++ b/default-configs/arm-softmmu.mak
>> @@ -40,3 +40,4 @@ CONFIG_FSL_IMX25=y
>>  CONFIG_FSL_IMX7=y
>>  CONFIG_FSL_IMX6UL=y
>>  CONFIG_SEMIHOSTING=y
>> +CONFIG_ACPI_APEI=y
>> diff --git a/hw/acpi/Kconfig b/hw/acpi/Kconfig
>> index 12e3f1e..ed8c34d 100644
>> --- a/hw/acpi/Kconfig
>> +++ b/hw/acpi/Kconfig
>> @@ -23,6 +23,10 @@ config ACPI_NVDIMM
>>  bool
>>  depends on ACPI
>>  
>> +config ACPI_APEI
>> +bool
>> +depends on ACPI
>> +
>>  config ACPI_PCI
>>  bool
>>  depends on ACPI && PCI
>> diff --git a/hw/acpi/Makefile.objs b/hw/acpi/Makefile.objs
>> index 9925305..7b5db4b 100644
>> --- a/hw/acpi/Makefile.objs
>> +++ b/hw/acpi/Makefile.objs
>> @@ -5,6 +5,7 @@ common-obj-$(CONFIG_ACPI_CPU_HOTPLUG) += cpu_hotplug.o
>>  common-obj-$(CONFIG_ACPI_MEMORY_HOTPLUG) += memory_hotplug.o
>>  common-obj-$(CONFIG_ACPI_CPU_HOTPLUG) += cpu.o
>>  common-obj-$(CONFIG_ACPI_NVDIMM) += nvdimm.o
>> +common-obj-$(CONFIG_ACPI_APEI) += ghes.o
>>  common-obj-$(CONFIG_ACPI_VMGENID) += vmgenid.o
>>  common-obj-$(CONFIG_ACPI_HW_REDUCED) += generic_event_device.o
>>  common-obj-$(call lnot,$(CONFIG_ACPI_X86)) += acpi-stub.o
>> diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
>> index 2c3702b..3681ec6 100644
>> --- a/hw/acpi/aml-build.c
>> +++ b/hw/acpi/aml-build.c
>> @@ -1578,6 +1578,7 @@ void acpi_build_tables_init(AcpiBuildTables *tables)
>>  tables->table_data = g_array_new(false, true /* clear */, 1);
>>  tables->tcpalog = g_array_new(false, true /* clear */, 1);
>>  tables->vmgenid = g_array_new(false, true /* clear */, 1);
>> +tables->hardware_errors = g_array_new(false, true /* clear */, 1);
>>  tables->linker = bios_linker_loader_init();
>>  }
>>  
>> @@ -1588,6 +1589,7 @@ void acpi_build_tables_cleanup(AcpiBuildTables 
>> *tables, bool mfre)
>>  g_array_free(tables->table_data, true);
>>  g_array_free(tables->tcpalog, mfre);
>>  g_array_free(tables->vmgenid, mfre);
>> +g_array_free(tables->hardware_errors, mfre);
>>  }
>>  
>>  /*
>> diff --git a/hw/acpi/ghes.c b/hw/acpi/ghes.c
>> new file mode 100644
>> index 000..b7fdbbb
>> --- /dev/null
>> +++ b/hw/acpi/ghes.c
>> @@ -0,0 +1,94 @@
>> +/*
>> + * Support for generating APEI tables and recording CPER for Guests
>> + *
>> + * Copyright (c) 2019 HUAWEI TECHNOLOGIES CO., LTD.
> s/2019/2020/
> 
> the same for other places that add this string within this series

sure, thanks for the pointing out.

> 
>> + *
>> + * Author: Dongjiu Geng 
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License as published by
>> + * the Free Software Foundation; either version 2 of the License, or
>> + * (at your option) any later version.
>> +
>> + * This program is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> + * GNU General Public License for more details.
>> +
>> + * You should have received a copy of the GNU General Public License along
>> + * with this program; if not, see .
>> + */
>> +
>> +#include "qemu/osdep.h"
>> +#include "hw/acpi/acpi.h"
>> +#include "hw/acpi/ghes.h"
>> +#include "hw/acpi/aml-build.h"
>> +#include "hw/nvram/fw_cfg.h"
>> +#include "sysemu/sysemu.h"
>> +#include "qemu/error-report.h"
>> +
>> +#include "hw/acpi/bios-linker-loader.h"
>> +
>> +#define ACPI_GHES_ERRORS_FW_CFG_FILE"etc/hardware_errors"
>> +#define ACPI_GHES_DATA_ADDR_FW_CFG_FILE "etc/hardware_errors_addr"
>> 

Re: [PATCH v22 7/9] ACPI: Record Generic Error Status Block(GESB) table

[gengdongjiu]

On 2020/1/28 23:29, Igor Mammedov wrote:
> On Wed, 8 Jan 2020 19:32:21 +0800
> Dongjiu Geng  wrote:
> 
>> kvm_arch_on_sigbus_vcpu() error injection uses source_id as
>> index in etc/hardware_errors to find out Error Status Data
>> Block entry corresponding to error source. So supported source_id
>> values should be assigned here and not be changed afterwards to
>> make sure that guest will write error into expected Error Status
>> Data Block even if guest was migrated to a newer QEMU.
>>
>> Before QEMU writes a new error to ACPI table, it will check whether
>> previous error has been acknowledged. Otherwise it will ignore the new
>> error. For the errors section type, QEMU simulate it to memory section
>> error.
>>
>> Signed-off-by: Dongjiu Geng 
>> Signed-off-by: Xiang Zheng 
>> Reviewed-by: Michael S. Tsirkin 
>> ---
>>  hw/acpi/ghes.c | 224 
>> -
>>  include/hw/acpi/ghes.h |   3 +
>>  include/qemu/uuid.h|   5 ++
>>  3 files changed, 230 insertions(+), 2 deletions(-)
>>
>> diff --git a/hw/acpi/ghes.c b/hw/acpi/ghes.c
>> index 68f4abf..f2ecffe 100644
>> --- a/hw/acpi/ghes.c
>> +++ b/hw/acpi/ghes.c
>> @@ -28,21 +28,56 @@
>>  #include "sysemu/sysemu.h"
>>  #include "qemu/error-report.h"
>>  
>> -#include "hw/acpi/bios-linker-loader.h"
> why it's moved to header?

Because the function declaration "void acpi_build_hest(GArray *table_data, 
GArray *hardware_error,
  BIOSLinker *linker);" needs it in the header file, 
Otherwise The compilation will be failed.


Anyway I will move this change to patch "[v22,4/9] ACPI: Build Hardware Error 
Source Table", because that patch needs it .

> 
>> -
>>  #define ACPI_GHES_ERRORS_FW_CFG_FILE"etc/hardware_errors"
>>  #define ACPI_GHES_DATA_ADDR_FW_CFG_FILE "etc/hardware_errors_addr"
>>  
>>  /* The max size in bytes for one error block */
>>  #define ACPI_GHES_MAX_RAW_DATA_LENGTH   0x400
>> +
>>  /* Now only support ARMv8 SEA notification type error source */
>>  #define ACPI_GHES_ERROR_SOURCE_COUNT1
>> +
>>  /* Generic Hardware Error Source version 2 */
>>  #define ACPI_GHES_SOURCE_GENERIC_ERROR_V2   10
>> +
>>  /* Address offset in Generic Address Structure(GAS) */
>>  #define GAS_ADDR_OFFSET 4
>>  
>>  /*
>> + * The total size of Generic Error Data Entry
>> + * ACPI 6.1/6.2: 18.3.2.7.1 Generic Error Data,
>> + * Table 18-343 Generic Error Data Entry
>> + */
>> +#define ACPI_GHES_DATA_LENGTH   72
>> +
>> +/* The memory section CPER size, UEFI 2.6: N.2.5 Memory Error Section */
>> +#define ACPI_GHES_MEM_CPER_LENGTH   80
>> +
>> +/* Masks for block_status flags */
>> +#define ACPI_GEBS_UNCORRECTABLE 1
>> +
>> +#define UEFI_CPER_SEC_PLATFORM_MEM  \
>> +UUID_LE(0xA5BC1114, 0x6F64, 0x4EDE, 0xB8, 0x63, 0x3E, 0x83, \
>> +0xED, 0x7C, 0x83, 0xB1)
>> +
>> +/*
>> + * Total size for Generic Error Status Block except Generic Error Data 
>> Entries
>> + * ACPI 6.2: 18.3.2.7.1 Generic Error Data,
>> + * Table 18-380 Generic Error Status Block
>> + */
>> +#define ACPI_GHES_GESB_SIZE 20
>> +
>> +/*
>> + * Values for error_severity field
>> + */
>> +enum AcpiGenericErrorSeverity {
>> +ACPI_CPER_SEV_RECOVERABLE = 0,
>> +ACPI_CPER_SEV_FATAL = 1,
>> +ACPI_CPER_SEV_CORRECTED = 2,
>> +ACPI_CPER_SEV_NONE = 3,
>> +};
>> +
>> +/*
>>   * Hardware Error Notification
>>   * ACPI 4.0: 17.3.2.7 Hardware Error Notification
>>   * Composes dummy Hardware Error Notification descriptor of specified type
>> @@ -73,6 +108,127 @@ static void build_ghes_hw_error_notification(GArray 
>> *table, const uint8_t type)
>>  }
>>  
>>  /*
>> + * Generic Error Data Entry
>> + * ACPI 6.1: 18.3.2.7.1 Generic Error Data
>> + */
>> +static void acpi_ghes_generic_error_data(GArray *table, QemuUUID 
>> section_type,
>> +uint32_t error_severity, uint8_t validation_bits, uint8_t 
>> flags,
>> +uint32_t error_data_length, QemuUUID fru_id,
>> +uint64_t time_stamp)
>> +{
>> +/* Section Type */
>> +g_array_append_vals(table, section_type.data,
>> +ARRAY_SIZE(section_type.data));
>> +
>> +/* Error Severity */
>> +build_append_int_noprefix(table, error_severity, 4);
>> +/* Revision */
>> +build_append_int_noprefix(table, 0x300, 2);
>> +/* Validation Bits */
>> +build_append_int_noprefix(table, validation_bits, 1);
>> +/* Flags */
>> +build_append_int_noprefix(table, flags, 1);
>> +/* Error Data Length */
>> +build_append_int_noprefix(table, error_data_length, 4);
>> +
>> +/* FRU Id */
>> +g_array_append_vals(table, fru_id.data, ARRAY_SIZE(fru_id.data));
>> +
>> +/* FRU Text */
>> +build_append_int_noprefix(table, 0, 20);
>> +/* Timestamp */
>> +build_append_int_noprefix(table, time_stamp, 8);
>> +}
>> +
>> +/*
>> + * Generic Error Status Block
>> + * ACPI 6.1: 18.3.2.7.1 Generic 

Re: [PATCH v22 5/9] ACPI: Record the Generic Error Status Block address

[gengdongjiu]

sorry for the late response due to Chinese new year

On 2020/1/28 22:41, Igor Mammedov wrote:
> On Wed, 8 Jan 2020 19:32:19 +0800
> Dongjiu Geng  wrote:
> 
> in addition to comments of others:
> 
>> Record the GHEB address via fw_cfg file, when recording
>> a error to CPER, it will use this address to find out
>> Generic Error Data Entries and write the error.
>>
>> Make the HEST GHES to a GED device.
> 
> It's hard to parse this even kno
> Pls rephrase/make commit message more verbose,
> so it would describe why and what patch is supposed to do
Ok, thanks for the comments.

> 
> 
>> Signed-off-by: Dongjiu Geng 
>> Signed-off-by: Xiang Zheng 
>> ---
>>  hw/acpi/generic_event_device.c | 15 ++-
>>  hw/acpi/ghes.c | 16 
>>  hw/arm/virt-acpi-build.c   | 13 -
>>  include/hw/acpi/generic_event_device.h |  2 ++
>>  include/hw/acpi/ghes.h |  6 ++
>>  5 files changed, 50 insertions(+), 2 deletions(-)
>>
>> diff --git a/hw/acpi/generic_event_device.c b/hw/acpi/generic_event_device.c
>> index 9cee90c..9bf37e4 100644
>> --- a/hw/acpi/generic_event_device.c
>> +++ b/hw/acpi/generic_event_device.c
>> @@ -234,12 +234,25 @@ static const VMStateDescription vmstate_ged_state = {
>>  }
>>  };
>>  
>> +static const VMStateDescription vmstate_ghes_state = {
>> +.name = "acpi-ghes-state",
>> +.version_id = 1,
>> +.minimum_version_id = 1,
>> +.fields  = (VMStateField[]) {
>> +VMSTATE_UINT64(ghes_addr_le, AcpiGhesState),
>> +VMSTATE_END_OF_LIST()
>> +}
>> +};
>> +
>>  static const VMStateDescription vmstate_acpi_ged = {
>>  .name = "acpi-ged",
>>  .version_id = 1,
>>  .minimum_version_id = 1,
>>  .fields = (VMStateField[]) {
>> -VMSTATE_STRUCT(ged_state, AcpiGedState, 1, vmstate_ged_state, 
>> GEDState),
>> +VMSTATE_STRUCT(ged_state, AcpiGedState, 1,
>> +   vmstate_ged_state, GEDState),
>> +VMSTATE_STRUCT(ghes_state, AcpiGedState, 1,
>> +   vmstate_ghes_state, AcpiGhesState),
>>  VMSTATE_END_OF_LIST(),
>>  },
>>  .subsections = (const VMStateDescription * []) {
>> diff --git a/hw/acpi/ghes.c b/hw/acpi/ghes.c
>> index 9d37798..68f4abf 100644
>> --- a/hw/acpi/ghes.c
>> +++ b/hw/acpi/ghes.c
>> @@ -23,6 +23,7 @@
>>  #include "hw/acpi/acpi.h"
>>  #include "hw/acpi/ghes.h"
>>  #include "hw/acpi/aml-build.h"
>> +#include "hw/acpi/generic_event_device.h"
>>  #include "hw/nvram/fw_cfg.h"
>>  #include "sysemu/sysemu.h"
>>  #include "qemu/error-report.h"
>> @@ -208,3 +209,18 @@ void acpi_build_hest(GArray *table_data, GArray 
>> *hardware_errors,
>>  build_header(linker, table_data, (void *)(table_data->data + 
>> hest_start),
>>  "HEST", table_data->len - hest_start, 1, NULL, "");
>>  }
>> +
>> +void acpi_ghes_add_fw_cfg(AcpiGhesState *ags, FWCfgState *s,
>> +GArray *hardware_error)
> 
> not aligned properly

will modify it.

> 
>> +{
>> +size_t size = 2 * sizeof(uint64_t) + ACPI_GHES_MAX_RAW_DATA_LENGTH;
>> +size_t request_block_size = ACPI_GHES_ERROR_SOURCE_COUNT * size;
>> +
>> +/* Create a read-only fw_cfg file for GHES */
>> +fw_cfg_add_file(s, ACPI_GHES_ERRORS_FW_CFG_FILE, hardware_error->data,
>> +request_block_size);
>> +
>> +/* Create a read-write fw_cfg file for Address */
>> +fw_cfg_add_file_callback(s, ACPI_GHES_DATA_ADDR_FW_CFG_FILE, NULL, NULL,
>> +NULL, &(ags->ghes_addr_le), sizeof(ags->ghes_addr_le), false);
>> +}
>> diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c
>> index 837bbf9..c8aa94d 100644
>> --- a/hw/arm/virt-acpi-build.c
>> +++ b/hw/arm/virt-acpi-build.c
>> @@ -797,6 +797,7 @@ void virt_acpi_build(VirtMachineState *vms, 
>> AcpiBuildTables *tables)
>>  unsigned dsdt, xsdt;
>>  GArray *tables_blob = tables->table_data;
>>  MachineState *ms = MACHINE(vms);
>> +AcpiGedState *acpi_ged_state;
>>  
>>  table_offsets = g_array_new(false, true /* clear */,
>>  sizeof(uint32_t));
>> @@ -831,7 +832,9 @@ void virt_acpi_build(VirtMachineState *vms, 
>> AcpiBuildTables *tables)
>>  acpi_add_table(table_offsets, tables_blob);
>>  build_spcr(tables_blob, tables->linker, vms);
>>  
>> -if (vms->ras) {
>> +acpi_ged_state = ACPI_GED(object_resolve_path_type("", TYPE_ACPI_GED,
>> +   NULL));
>> +if (acpi_ged_state &&  vms->ras) {
> 
> there is vms->acpi_dev which is GED, so you don't need to look it up
> 
> suggest:
   Thanks for the suggestion.

>  if (ras) {
> assert(ged)
  assert(vms->acpi_dev), right?

> do other fun stuff ...
>  }

> 
>>  acpi_add_table(table_offsets, tables_blob);
>>  build_ghes_error_table(tables->hardware_errors, tables->linker);
>>  acpi_build_hest(tables_blob, tables->hardware_errors,
>> @@ 

Re: [PATCH v22 8/9] target-arm: kvm64: handle SIGBUS signal from kernel or KVM

[gengdongjiu]

On 2020/1/20 20:15, Peter Maydell wrote:
> On Fri, 17 Jan 2020 at 10:05, gengdongjiu  wrote:
>>
>> On 2020/1/17 0:28, Peter Maydell wrote:
>>> On Wed, 8 Jan 2020 at 11:33, Dongjiu Geng  wrote:
>>>
>>>> +void kvm_arch_on_sigbus_vcpu(CPUState *c, int code, void *addr)
>>>> +{
>>>> +ram_addr_t ram_addr;
>>>> +hwaddr paddr;
>>>> +
>>>> +assert(code == BUS_MCEERR_AR || code == BUS_MCEERR_AO);
>>>> +
>>>> +if (acpi_enabled && addr &&
>>>> +object_property_get_bool(qdev_get_machine(), "ras", NULL)) {
>>>> +ram_addr = qemu_ram_addr_from_host(addr);
>>>> +if (ram_addr != RAM_ADDR_INVALID &&
>>>> +kvm_physical_memory_addr_from_host(c->kvm_state, addr, 
>>>> )) {
>>>> +kvm_hwpoison_page_add(ram_addr);
>>>> +/*
>>>> + * Asynchronous signal will be masked by main thread, so
>>>> + * only handle synchronous signal.
>>>> + */
>>>
>>> I don't understand this comment. (I think we've had discussions
>>> about it before, but it's still not clear to me.)
>>>
>>> This function (kvm_arch_on_sigbus_vcpu()) will be called in two contexts:
>>>
>>> (1) in the vcpu thread:
>>>   * the real SIGBUS handler sigbus_handler() sets a flag and arranges
>>> for an immediate vcpu exit
>>>   * the vcpu thread reads the flag on exit from KVM_RUN and
>>> calls kvm_arch_on_sigbus_vcpu() directly
>>>   * the error could be MCEERR_AR or MCEERR_AOFor the vcpu thread, the error 
>>> can be MCEERR_AR or MCEERR_AO,
>> but kernel/KVM usually uses MCEERR_AR(action required) instead of MCEERR_AO, 
>> because it needs do action immediately. For MCEERR_AO error, the action is 
>> optional and the error can be ignored.
>> At least I do not find Linux kernel/KVM deliver MCEERR_AO in the vcpu 
>> threads.
>>
>>> (2) MCE errors on other threads:
>>>   * here SIGBUS is blocked, so MCEERR_AR (action-required)
>>> errors will cause the kernel to just kill the QEMU process
>>>   * MCEERR_AO errors will be handled via the iothread's use
>>> of signalfd(), so kvm_on_sigbus() will get called from
>>> the main thread, and it will call kvm_arch_on_sigbus_vcpu()
>>>   * in this case the passed in CPUState will (arbitrarily) be that
>>> for the first vCPU
>>
>> For the MCE errors on other threads, it can only handle MCEERR_AO. If it is 
>> MCEERR_AR, the QEMU will assert and exit[2].
>>
>> Case1: Other APP indeed can send MCEERR_AO to QEMU， QEMU handle it via the 
>> iothread's use of signalfd() through above path.
>> Case2: But if the MCEERR_AO is delivered by kernel, I see QEMU ignore it 
>> because SIGBUS is masked in main thread[3], for this case, I do not see QEMU 
>> handle it via signalfd() for MCEERR_AO errors from my test.
> 
> SIGBUS is blocked in the main thread because we use signalfd().
> The function sigfd_handler() should be called and it will then
> manually invoke the correct function for the signal.
> 
>> For Case1，I think we should not let guest know it, because it is not 
>> triggered by guest. only other APP send SIGBUS to tell QEMU do somethings.
> 
> I don't understand what you mean here by "other app" or
> "guest" triggering of MCEERR. I thought that an MCEERR meant
> "the hardware has detected that there is a problem with the
> RAM". If there's a problem with the RAM and it's the RAM that's
> being used as guest RAM, we need to tell the guest, surely ?

  sure, If the error is guest RAM, we need to test the guest.
  I mean if the RAM that is being used as QEMU RAM(not guest RAM), we should 
not tell the guest.
  OR if another user space manually send SIGBUS to qemu, such as using "kill -s 
SIGBUS xxx" commands, we should not tell the guest.

> 
>> For Case2，it does not call call kvm_arch_on_sigbus_vcpu().
> 
> It should do. The code you quote calls that function
> for that case:
  According to our analysis, I also think it should call the function for that 
case.
  But from my test, I see  kvm_arch_on_sigbus_vcpu() is not called when 
KVM/Kernel delivers SIGBUS to QEMU main thread.
  So I am also confused. I haven't even dig into the reason yet.

 If anyone has done the test or knows the reason, welcome comments.


> 
>> [1]:
>> /* Called synchronously (via signalfd) in main thread.  */
>> int kvm_on_sigbus(int code, void *addr)

Re: [PATCH v22 9/9] MAINTAINERS: Add ACPI/HEST/GHES entries

[gengdongjiu]

On 2020/1/17 19:22, Philippe Mathieu-Daudé wrote:
> On 1/17/20 12:09 PM, Peter Maydell wrote:
>> On Fri, 17 Jan 2020 at 07:22, Philippe Mathieu-Daudé  
>> wrote:
>>>
>>> Hi Peter,
>>>
>>> On 1/16/20 5:46 PM, Peter Maydell wrote:
 On Wed, 8 Jan 2020 at 11:32, Dongjiu Geng  wrote:
>
> I and Xiang are willing to review the APEI-related patches and
> volunteer as the reviewers for the HEST/GHES part.
>
> Signed-off-by: Dongjiu Geng 
> Signed-off-by: Xiang Zheng 
> ---
>    MAINTAINERS | 9 +
>    1 file changed, 9 insertions(+)
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 387879a..5af70a5 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -1423,6 +1423,15 @@ F: tests/bios-tables-test.c
>    F: tests/acpi-utils.[hc]
>    F: tests/data/acpi/
>
> +ACPI/HEST/GHES
> +R: Dongjiu Geng 
> +R: Xiang Zheng 
> +L: qemu-...@nongnu.org
> +S: Maintained
> +F: hw/acpi/ghes.c
> +F: include/hw/acpi/ghes.h
> +F: docs/specs/acpi_hest_ghes.rst
> +
>    ppc4xx
>    M: David Gibson 
>    L: qemu-...@nongnu.org
> -- 

 Michael, Igor: since this new MAINTAINERS section is
 moving files out of the 'ACPI/SMBIOS' section that you're
 currently responsible for, do you want to provide an
 acked-by: that you think this division of files makes sense?
>>>
>>> The files are not 'moved out', Michael and Igor are still the
>>> maintainers of the supported ACPI/SMBIOS subsystem: 

In fact, I am willing to maintain with others to the new added files , I see 
some new added files in "hw/acpi" folders are moved out, such as NVDIMM/vmgenid.

NVDIMM
M: Xiao Guangrong 
S: Maintained
F: hw/acpi/nvdimm.c
F: hw/mem/nvdimm.c
F: include/hw/mem/nvdimm.h
F: docs/nvdimm.txt

VM Generation ID
M: Ben Warren 
S: Maintained
F: hw/acpi/vmgenid.c
F: include/hw/acpi/vmgenid.h
F: docs/specs/vmgenid.txt
F: tests/vmgenid-test.c
F: stubs/vmgenid.c

Re: [PATCH v22 5/9] ACPI: Record the Generic Error Status Block address

[gengdongjiu]

On 2020/1/17 15:39, Philippe Mathieu-Daudé wrote:
>>     table_offsets = g_array_new(false, true /* clear */,
>>   sizeof(uint32_t));
>> @@ -831,7 +832,9 @@ void virt_acpi_build(VirtMachineState *vms, 
>> AcpiBuildTables *tables)
>>   acpi_add_table(table_offsets, tables_blob);
>>   build_spcr(tables_blob, tables->linker, vms);
>>   -    if (vms->ras) {
>> +    acpi_ged_state = ACPI_GED(object_resolve_path_type("", TYPE_ACPI_GED,
>> +   NULL));
> 
> Testing vms->ras first is cheaper than calling object_resolve_path_type(). 
> Since some people are spending lot of time to reduce VM boot time, it might 
> be worth considering.
Thanks Philippe's comments.

Do you think it should be written to below[1]? right?

[1]:
if (vms->ras && acpi_ged_state)


> 
>> +    if (acpi_ged_state &&  vms->ras) {
>>   acpi_add_table(table_offsets, tables_blob);
>>   build_ghes_error_table(tables->hardware_errors, tables->linker);
>>   acpi_build_hest(tables_blob, tables->hardware_errors,
>> @@ -925,6 +928,7 @@ void virt_acpi_setup(VirtMachineState *vms)
>>   { 

Re: [PATCH v22 5/9] ACPI: Record the Generic Error Status Block address

[gengdongjiu]

On 2020/1/17 0:44, Peter Maydell wrote:
>>  .minimum_version_id = 1,
>>  .fields = (VMStateField[]) {
>> -VMSTATE_STRUCT(ged_state, AcpiGedState, 1, vmstate_ged_state, 
>> GEDState),
>> +VMSTATE_STRUCT(ged_state, AcpiGedState, 1,
>> +   vmstate_ged_state, GEDState),
>> +VMSTATE_STRUCT(ghes_state, AcpiGedState, 1,
>> +   vmstate_ghes_state, AcpiGhesState),
>>  VMSTATE_END_OF_LIST(),
>>  },
>>  .subsections = (const VMStateDescription * []) {
> You can't just add fields to an existing VMStateDescription
> like this -- it will break migration compatibility. Instead you
> need to add a new subsection to this vmstate, with a '.needed'
> function which indicates when the subsection should be present.
Thanks Peter's pointing out. I will change it.


> 
> thanks
> -- PMM
> .
> 

Re: [PATCH v22 9/9] MAINTAINERS: Add ACPI/HEST/GHES entries

[gengdongjiu]

On 2020/1/17 15:22, Philippe Mathieu-Daudé wrote:
> Hi Peter,
> 
> On 1/16/20 5:46 PM, Peter Maydell wrote:
>> On Wed, 8 Jan 2020 at 11:32, Dongjiu Geng  wrote:
>>>
>>> I and Xiang are willing to review the APEI-related patches and
>>> volunteer as the reviewers for the HEST/GHES part.
>>>
>>> Signed-off-by: Dongjiu Geng 
>>> Signed-off-by: Xiang Zheng 
>>> ---
>>>   MAINTAINERS | 9 +
>>>   1 file changed, 9 insertions(+)
>>>
>>> diff --git a/MAINTAINERS b/MAINTAINERS
>>> index 387879a..5af70a5 100644
>>> --- a/MAINTAINERS
>>> +++ b/MAINTAINERS
>>> @@ -1423,6 +1423,15 @@ F: tests/bios-tables-test.c
>>>   F: tests/acpi-utils.[hc]
>>>   F: tests/data/acpi/
>>>
>>> +ACPI/HEST/GHES
>>> +R: Dongjiu Geng 
>>> +R: Xiang Zheng 
>>> +L: qemu-...@nongnu.org
>>> +S: Maintained
>>> +F: hw/acpi/ghes.c
>>> +F: include/hw/acpi/ghes.h
>>> +F: docs/specs/acpi_hest_ghes.rst
>>> +
>>>   ppc4xx
>>>   M: David Gibson 
>>>   L: qemu-...@nongnu.org
>>> -- 
>>
>> Michael, Igor: since this new MAINTAINERS section is
>> moving files out of the 'ACPI/SMBIOS' section that you're
>> currently responsible for, do you want to provide an
>> acked-by: that you think this division of files makes sense?
> 
> The files are not 'moved out', Michael and Igor are still the maintainers of 
> the supported ACPI/SMBIOS subsystem:
> 
> ACPI/SMBIOS
> M: Michael S. Tsirkin 
> M: Igor Mammedov 
> S: Supported
> F: include/hw/acpi/*
> F: hw/acpi/*
> 
> Dongjiu and Xiang only add themselves as reviewers to get notified on changes 
> on these specific files. The more eyes the better :)
> 
> The docs/specs/acpi_hest_ghes.rst document has no maintainer, as these others 
> too:
If this file has no maintainer, may be it needs a M tag for this file, 
otherwise when people change this file, and use "./scripts/get_maintainer.pl 
x" to get maintainer, it will be empty.

> 
> - docs/specs/acpi_cpu_hotplug.txt
> - docs/specs/acpi_hw_reduced_hotplug.rst
> - docs/specs/acpi_mem_hotplug.txt
> - docs/specs/acpi_nvdimm.txt
> 
> The only ACPI file reported as maintained in docs/specs/ is 
> acpi_pci_hotplug.txt, from this entry:
> 
> PCI
> M: Michael S. Tsirkin 
> M: Marcel Apfelbaum 
> S: Supported
> F: docs/specs/*pci*
> 
> FWIW:
> Reviewed-by: Philippe Mathieu-Daudé 
> 
> .
> 

Re: [PATCH v22 8/9] target-arm: kvm64: handle SIGBUS signal from kernel or KVM

[gengdongjiu]

On 2020/1/17 0:28, Peter Maydell wrote:
> On Wed, 8 Jan 2020 at 11:33, Dongjiu Geng  wrote:
> 
>> +void kvm_arch_on_sigbus_vcpu(CPUState *c, int code, void *addr)
>> +{
>> +ram_addr_t ram_addr;
>> +hwaddr paddr;
>> +
>> +assert(code == BUS_MCEERR_AR || code == BUS_MCEERR_AO);
>> +
>> +if (acpi_enabled && addr &&
>> +object_property_get_bool(qdev_get_machine(), "ras", NULL)) {
>> +ram_addr = qemu_ram_addr_from_host(addr);
>> +if (ram_addr != RAM_ADDR_INVALID &&
>> +kvm_physical_memory_addr_from_host(c->kvm_state, addr, )) 
>> {
>> +kvm_hwpoison_page_add(ram_addr);
>> +/*
>> + * Asynchronous signal will be masked by main thread, so
>> + * only handle synchronous signal.
>> + */
> 
> I don't understand this comment. (I think we've had discussions
> about it before, but it's still not clear to me.)
> 
> This function (kvm_arch_on_sigbus_vcpu()) will be called in two contexts:
> 
> (1) in the vcpu thread:
>   * the real SIGBUS handler sigbus_handler() sets a flag and arranges
> for an immediate vcpu exit
>   * the vcpu thread reads the flag on exit from KVM_RUN and
> calls kvm_arch_on_sigbus_vcpu() directly
>   * the error could be MCEERR_AR or MCEERR_AOFor the vcpu thread, the error 
> can be MCEERR_AR or MCEERR_AO,
but kernel/KVM usually uses MCEERR_AR(action required) instead of MCEERR_AO, 
because it needs do action immediately. For MCEERR_AO error, the action is 
optional and the error can be ignored.
At least I do not find Linux kernel/KVM deliver MCEERR_AO in the vcpu threads.

> (2) MCE errors on other threads:
>   * here SIGBUS is blocked, so MCEERR_AR (action-required)
> errors will cause the kernel to just kill the QEMU process
>   * MCEERR_AO errors will be handled via the iothread's use
> of signalfd(), so kvm_on_sigbus() will get called from
> the main thread, and it will call kvm_arch_on_sigbus_vcpu()
>   * in this case the passed in CPUState will (arbitrarily) be that
> for the first vCPU

For the MCE errors on other threads, it can only handle MCEERR_AO. If it is 
MCEERR_AR, the QEMU will assert and exit[2].

Case1: Other APP indeed can send MCEERR_AO to QEMU， QEMU handle it via the 
iothread's use of signalfd() through above path.
Case2: But if the MCEERR_AO is delivered by kernel, I see QEMU ignore it 
because SIGBUS is masked in main thread[3], for this case, I do not see QEMU 
handle it via signalfd() for MCEERR_AO errors from my test.

For Case1，I think we should not let guest know it, because it is not triggered 
by guest. only other APP send SIGBUS to tell QEMU do somethings.
For Case2，it does not call call kvm_arch_on_sigbus_vcpu().


[1]:
/* Called synchronously (via signalfd) in main thread.  */
int kvm_on_sigbus(int code, void *addr)
{
#ifdef KVM_HAVE_MCE_INJECTION
/* Action required MCE kills the process if SIGBUS is blocked.  Because
 * that's what happens in the I/O thread, where we handle MCE via signalfd,
 * we can only get action optional here.
 */
[2]: assert(code != BUS_MCEERR_AR);
kvm_arch_on_sigbus_vcpu(first_cpu, code, addr);
return 0;
#else
return 1;
#endif
}

[3]: https://lists.gnu.org/archive/html/qemu-devel/2017-11/msg03575.html


> 
> For MCEERR_AR, the code here looks correct -- we know this is
> only going to happen for the relevant vCPU so we can go ahead
> and do the "record it in the ACPI table and tell the guest" bit.
> 
> But shouldn't we be doing something with the MCEERR_AO too?

Above all, from my test, for MCEERR_AO error which is triggered by guest, it 
not call kvm_arch_on_sigbus_vcpu().
so I think currently we can just report error. If afterwards  MCEERR_AO error 
can call kvm_arch_on_sigbus_vcpu(), we can let the guest know about it.

> That of course will be trickier because we're not necessarily
> in the vcpu thread, but it would be nice to let the guest
> know about it.
> 
> One comment that would work with the current code would be:
> 
> /*
>  * If this is a BUS_MCEERR_AR, we know we have been called
>  * synchronously from the vCPU thread, so we can easily
>  * synchronize the state and inject an error.
>  *
>  * TODO: we currently don't tell the guest at all about BUS_MCEERR_AO.
>  * In that case we might either be being called synchronously from
>  * the vCPU thread, or a bit later from the main thread, so doing
At least I do not find Linux kernel/KVM deliver MCEERR_AO in the vcpu threads.
In the main thread, signalfd() is not called when it is BUS_MCEERR_AO which is 
triggered by guest.

>  * the injection of the error would be more complicated.
>  */
> 
> but I don't know if that's what you meant to say/implement...
we can implement MCEERR_AO logic when QEMU can receive MCEERR_AO error which is 
triggered by guest.

> 
>> +if (code == BUS_MCEERR_AR) {
>> +kvm_cpu_synchronize_state(c);
>> +if 

RE: [PATCH v22 0/9] Add ARMv8 RAS virtualization support in QEMU

[gengdongjiu]

Ping.



发件人：gengdongjiu 
收件人：pbonzini ;gengdongjiu ;mst 
;imammedo ;shannon.zhaosl 
;peter.maydell ;fam 
;rth ;ehabkost 
;mtosatti ;xuwei (O) 
;Jonathan Cameron ;james.morse 
;qemu-devel ;kvm 
;qemu-arm 
抄 送：zhengxiang (A) ;Linuxarm 
时 间：2020-01-08 19:32:33
主题[PATCH v22 0/9] Add ARMv8 RAS virtualization support in QEMU

In the ARMv8 platform, the CPU error types are synchronous external abort(SEA)
and SError Interrupt (SEI). If exception happens in guest, sometimes it's better
for guest to perform the recovery, because host does not know the detailed
information of guest. For example, if an exception happens in a user-space
application within guest, host does not know which application encounters
errors.

For the ARMv8 SEA/SEI, KVM or host kernel delivers SIGBUS to notify userspace.
After user space gets the notification, it will record the CPER into guest GHES
buffer and inject an exception or IRQ into guest.

In the current implementation, if the type of SIGBUS is BUS_MCEERR_AR, we will
treat it as a synchronous exception, and notify guest with ARMv8 SEA
notification type after recording CPER into guest.

This series of patches are based on Qemu 4.2, which include two parts:
1. Generate APEI/GHES table.
2. Handle the SIGBUS signal, record the CPER in runtime and fill it into guest
   memory, then notify guest according to the type of SIGBUS.

The whole solution was suggested by James(james.mo...@arm.com); The solution of
APEI section was suggested by Laszlo(ler...@redhat.com).
Show some discussions in [1].

This series of patches have already been tested on ARM64 platform with RAS
feature enabled:
Show the APEI part verification result in [2].
Show the BUS_MCEERR_AR SIGBUS handling verification result in [3].

---
Change since v21:
1. Make the user-facing 'ras' option description more clearly to address 
Peter's comments.
2. Update the doc description in "docs/specs/acpi_hest_ghes.rst"
3. Split HEST/GHES patches to more patches to make the review easily
4. Using source_id to index the location to save the CPER.
5. Optimize and simplify the logic to build HEST/GHES table to address 
Igor/Michael/Beata comments.
6. make ghes_addr_le a part of GED device.

Change since v20:
1. Move some implementation details from acpi_ghes.h to acpi_ghes.c
2. Add the reviewers for the ACPI/APEI/GHES part

Change since v19:
1. Fix clang compile error
2. Fix sphinx build error

Change since v18:
1. Fix some code-style and typo/grammar problems.
2. Remove no_ras in the VirtMachineClass struct.
3. Convert documentation to rst format.
4. Simplize the code and add comments for some magic value.
5. Move kvm_inject_arm_sea() function into the patch where it's used.
6. Register the reset handler(kvm_unpoison_all()) in the kvm_init() function.

Change since v17:
1. Improve some commit messages and comments.
2. Fix some code-style problems.
3. Add a *ras* machine option.
4. Move HEST/GHES related structures and macros into "hw/acpi/acpi_ghes.*".
5. Move HWPoison page functions into "include/sysemu/kvm_int.h".
6. Fix some bugs.
7. Improve the design document.

Change since v16:
1. check whether ACPI table is enabled when handling the memory error in the 
SIGBUS handler.

Change since v15:
1. Add a doc-comment in the proper format for 'include/exec/ram_addr.h'
2. Remove write_part_cpustate_to_list() because there is another bug fix patch
   has been merged "arm: Allow system registers for KVM guests to be changed by 
QEMU code"
3. Add some comments for kvm_inject_arm_sea() in 'target/arm/kvm64.c'
4. Compare the arm_current_el() return value to 0,1,2,3, not to PSTATE_MODE_* 
constants.
5. Change the RAS support wasn't introduced before 4.1 QEMU version.
6. Move the no_ras flag  patch to begin in this series

Change since v14:
1. Remove the BUS_MCEERR_AO handling logic because this asynchronous signal was 
masked by main thread
2. Address some Igor Mammedov's comments(ACPI part)
   1) change the comments for the enum AcpiHestNotifyType definition and remove 
ditto in patch 1
   2) change some patch commit messages and separate "APEI GHES table 
generation" patch to more patches.
3. Address some peter's comments(arm64 Synchronous External Abort injection)
   1) change some code notes
   2) using arm_current_el() for current EL
   2) use the helper functions for those (syn_data_abort_*).

Change since v13:
1. Move the patches that set guest ESR and inject virtual SError out of this 
series
2. Clean and optimize the APEI part patches
3. Update the commit messages and add some comments for the code

Change since v12:
1. Address Paolo's comments to move HWPoisonPage definition to 
accel/kvm/kvm-all.c
2. Only call kvm_cpu_synchronize_state() when get the BUS_MCEERR_AR signal
3. Only add and enable GPIO-Signal and ARMv8 SEA two hardware error sources
4. Address Michael's comments to not sync SPDX from Linux kernel header file

Change since v11:
Address James's comments(james.mo...@arm.com)

Re: [PATCH v22 0/9] Add ARMv8 RAS virtualization support in QEMU

[gengdongjiu]

1. How to enable this feature:
(a) In KVM mode: ./qemu-system-aarch64 --enable-kvm -cpu host --bios 
QEMU_EFI.fd_new  -machine virt,gic-version=3,ras,kernel-irqchip=on  -smp 4 
-nographic -kernel Image  -append "rdinit=/init console=ttyAMA0 mem=512M 
root=/dev/ram0" -initrd guestfs_new.cpio.gz
(b）In TCG mode: ./qemu-system-aarch64 -cpu cortex-a57 -bios QEMU_EFI.fd_new  
-machine virt,gic-version=3,ras  -smp 4 -nographic -kernel Image  -append 
"rdinit=/init console=ttyAMA0 mem=512M root=/dev/ram0 earlycon=pl011,0x900 
rw" -initrd guestfs_new.cpio.gz

2. How to quickly test this series patches:
（a）hack some code as shown in [1].
 (b) build and run the qemu: ./qemu-system-aarch64 -cpu cortex-a57 -bios 
QEMU_EFI.fd_new  -machine virt,gic-version=3,ras  -smp 4 -nographic -kernel 
Image  -append "rdinit=/init console=ttyAMA0 mem=512M root=/dev/ram0 
earlycon=pl011,0x900 rw" -initrd guestfs_new.cpio.gz
 (c) find the process id(PID) of "qemu-system-aarch64": $ ps -aux | grep "qemu
 (d) then send SIGBUS to qemu process: kill -s SIGBUS PID


[1]:
diff --git a/cpus.c b/cpus.c
index b472378b70..a0b67f5947 100644
--- a/cpus.c
+++ b/cpus.c
@@ -1087,6 +1087,8 @@ static void sigbus_reraise(void)

 static void sigbus_handler(int n, siginfo_t *siginfo, void *ctx)
 {
+siginfo->si_code = BUS_MCEERR_AO;
+
 if (siginfo->si_code != BUS_MCEERR_AO && siginfo->si_code != 
BUS_MCEERR_AR) {
 sigbus_reraise();
 }
diff --git a/target/arm/kvm64.c b/target/arm/kvm64.c
index f3b05c1977..03e8e20f4a 100644
--- a/target/arm/kvm64.c
+++ b/target/arm/kvm64.c
@@ -1323,17 +1323,23 @@ int kvm_arch_get_registers(CPUState *cs)

 void kvm_arch_on_sigbus_vcpu(CPUState *c, int code, void *addr)
 {
+#if 0
 ram_addr_t ram_addr;
 hwaddr paddr;

 assert(code == BUS_MCEERR_AR || code == BUS_MCEERR_AO);
-
-if (acpi_enabled && addr &&
+#else
+hwaddr paddr = 0x400a1000;
+code = BUS_MCEERR_AR;
+#endif
+if (acpi_enabled && paddr &&
 object_property_get_bool(qdev_get_machine(), "ras", NULL)) {
+#if 0
 ram_addr = qemu_ram_addr_from_host(addr);
 if (ram_addr != RAM_ADDR_INVALID &&
 kvm_physical_memory_addr_from_host(c->kvm_state, addr, )) {
 kvm_hwpoison_page_add(ram_addr);
+#endif
 /*
  * Asynchronous signal will be masked by main thread, so
  * only handle synchronous signal.
@@ -1348,11 +1354,13 @@ void kvm_arch_on_sigbus_vcpu(CPUState *c, int code, 
void *addr)
 }
 }
 return;
+#if 0
 }
 if (code == BUS_MCEERR_AO) {
 error_report("Hardware memory error at addr %p for memory used by "
 "QEMU itself instead of guest system!", addr);
 }
+#endif
 }

 if (code == BUS_MCEERR_AR) {



On 2020/1/8 19:32, Dongjiu Geng wrote:
> In the ARMv8 platform, the CPU error types are synchronous external abort(SEA)
> and SError Interrupt (SEI). If exception happens in guest, sometimes it's 
> better
> for guest to perform the recovery, because host does not know the detailed
> information of guest. For example, if an exception happens in a user-space
> application within guest, host does not know which application encounters
> errors.
> 
> For the ARMv8 SEA/SEI, KVM or host kernel delivers SIGBUS to notify userspace.
> After user space gets the notification, it will record the CPER into guest 
> GHES
> buffer and inject an exception or IRQ into guest.
> 
> In the current implementation, if the type of SIGBUS is BUS_MCEERR_AR, we will
> treat it as a synchronous exception, and notify guest with ARMv8 SEA
> notification type after recording CPER into guest.
> 
> This series of patches are based on Qemu 4.2, which include two parts:
> 1. Generate APEI/GHES table.
> 2. Handle the SIGBUS signal, record the CPER in runtime and fill it into guest
>memory, then notify guest according to the type of SIGBUS.
> 
> The whole solution was suggested by James(james.mo...@arm.com); The solution 
> of
> APEI section was suggested by Laszlo(ler...@redhat.com).
> Show some discussions in [1].
> 
> This series of patches have already been tested on ARM64 platform with RAS
> feature enabled:
> Show the APEI part verification result in [2].
> Show the BUS_MCEERR_AR SIGBUS handling verification result in [3].
> 
> ---
> Change since v21:
> 1. Make the user-facing 'ras' option description more clearly to address 
> Peter's comments.
> 2. Update the doc description in "docs/specs/acpi_hest_ghes.rst"
> 3. Split HEST/GHES patches to more patches to make the review easily
> 4. Using source_id to index the location to save the CPER.
> 5. Optimize and simplify the logic to build HEST/GHES table to address 
> Igor/Michael/Beata comments.
> 6. make ghes_addr_le a part of GED device.
> 
> Change since v20:
> 1. Move some implementation details from acpi_ghes.h to acpi_ghes.c
> 2. Add the reviewers for the ACPI/APEI/GHES part
> 
> Change since v19:
> 1. Fix clang 

Re: [RESEND PATCH v21 5/6] target-arm: kvm64: handle SIGBUS signal from kernel or KVM

[gengdongjiu]

On 2019/11/16 0:37, Igor Mammedov wrote:
>> +
>> +/* zero means OSPM does not acknowledge the error */
>> +if (!read_ack_register) {
>> +if (loop < 3) {
>> +usleep(100 * 1000);
>> +loop++;
>> +goto retry;
> as minimum this loop can stall guest repeatedly for 0.3s if guest triggers 
> BQL,
> until it handles error.

I think reparations for 0.3s is reasonable.
1. 0.3s is the worst case to repeat, if guest acknowledge it in before 0.3s, 
the guest can not stall
2. if the previous error is not acknowledged, the next error will be lost, 
error handling(safety) is more important than others.


>
> (not sure what to suggest here though)
> 
> (not sure what to suggest here though)
> 

Re: [RESEND PATCH v21 5/6] target-arm: kvm64: handle SIGBUS signal from kernel or KVM

[gengdongjiu]

On 2019/12/9 21:05, Beata Michalska wrote:
>> Here we set the FnV to not valid, not to set it to valid.
>> because Guest will use the physical address that recorded in APEI table.
>>
> To be precise : the FnV is  giving the status of FAR - so what you are setting
> here is status of 0b0 which means FAR is valid, not FnV on it's own.
> And my point was that you are changing the prototype for syn_data_abort_no_iss
> just for this case only so I was just thinking that it might not be
> worth that, instead
> you could just set it here ... or to be more flexible , provide a way
> to set specific bits
> on demand.

No, I set the FnV to 0b1, not 0b0, the whole esr_el1's value is 0x96000410, as 
shown below log:
I remember changing the prototype for syn_data_abort_no_iss is suggested by 
Peter Maydell.


[1]:
[   62.851830] Internal error: synchronous external abort: 96000410 [#1] 
PREEMPT SMP
[   62.854465] Modules linked in:



> 

Re: [RESEND PATCH v21 1/6] hw/arm/virt: Introduce a RAS machine option

[gengdongjiu]

> 
> I think we could make the user-facing description of
> the option a little clearer: something like
> "Set on/off to enable/disable reporting host memory errors
> to a KVM guest using ACPI and guest external abort exceptions"
> 
> ?
Peter, sorry for the late response.
sure, we have already updated it, and will send PATCH V22.

> 
> Otherwise
> Reviewed-by: Peter Maydell 
> 
> thanks
> -- PMM
> .
> 

Re: [RESEND PATCH v21 5/6] target-arm: kvm64: handle SIGBUS signal from kernel or KVM

[gengdongjiu]

On 2019/11/22 23:47, Beata Michalska wrote:
> Hi,
> 
> On Mon, 11 Nov 2019 at 01:48, Xiang Zheng  wrote:
>>
>> From: Dongjiu Geng 
>>
>> Add a SIGBUS signal handler. In this handler, it checks the SIGBUS type,
>> translates the host VA delivered by host to guest PA, then fills this PA
>> to guest APEI GHES memory, then notifies guest according to the SIGBUS
>> type.
>>
>> When guest accesses the poisoned memory, it will generate a Synchronous
>> External Abort(SEA). Then host kernel gets an APEI notification and calls
>> memory_failure() to unmapped the affected page in stage 2, finally
>> returns to guest.
>>
>> Guest continues to access the PG_hwpoison page, it will trap to KVM as
>> stage2 fault, then a SIGBUS_MCEERR_AR synchronous signal is delivered to
>> Qemu, Qemu records this error address into guest APEI GHES memory and
>> notifes guest using Synchronous-External-Abort(SEA).
>>
>> In order to inject a vSEA, we introduce the kvm_inject_arm_sea() function
>> in which we can setup the type of exception and the syndrome information.
>> When switching to guest, the target vcpu will jump to the synchronous
>> external abort vector table entry.
>>
>> The ESR_ELx.DFSC is set to synchronous external abort(0x10), and the
>> ESR_ELx.FnV is set to not valid(0x1), which will tell guest that FAR is
>> not valid and hold an UNKNOWN value. These values will be set to KVM
>> register structures through KVM_SET_ONE_REG IOCTL.
>>
>> Signed-off-by: Dongjiu Geng 
>> Signed-off-by: Xiang Zheng 
>> Reviewed-by: Michael S. Tsirkin 
>> ---
>>  hw/acpi/acpi_ghes.c | 297 
>>  include/hw/acpi/acpi_ghes.h |   4 +
>>  include/sysemu/kvm.h|   3 +-
>>  target/arm/cpu.h|   4 +
>>  target/arm/helper.c |   2 +-
>>  target/arm/internals.h  |   5 +-
>>  target/arm/kvm64.c  |  64 
>>  target/arm/tlb_helper.c |   2 +-
>>  target/i386/cpu.h   |   2 +
>>  9 files changed, 377 insertions(+), 6 deletions(-)
>>
>> diff --git a/hw/acpi/acpi_ghes.c b/hw/acpi/acpi_ghes.c
>> index 42c00ff3d3..f5b54990c0 100644
>> --- a/hw/acpi/acpi_ghes.c
>> +++ b/hw/acpi/acpi_ghes.c
>> @@ -39,6 +39,34 @@
>>  /* The max size in bytes for one error block */
>>  #define ACPI_GHES_MAX_RAW_DATA_LENGTH   0x1000
>>
>> +/*
>> + * The total size of Generic Error Data Entry
>> + * ACPI 6.1/6.2: 18.3.2.7.1 Generic Error Data,
>> + * Table 18-343 Generic Error Data Entry
>> + */
>> +#define ACPI_GHES_DATA_LENGTH   72
>> +
>> +/*
>> + * The memory section CPER size,
>> + * UEFI 2.6: N.2.5 Memory Error Section
>> + */
>> +#define ACPI_GHES_MEM_CPER_LENGTH   80
>> +
>> +/*
>> + * Masks for block_status flags
>> + */
>> +#define ACPI_GEBS_UNCORRECTABLE 1
> 
> Why not listing all supported statuses ? Similar to error severity below ?
> 
>> +
>> +/*
>> + * Values for error_severity field
>> + */
>> +enum AcpiGenericErrorSeverity {
>> +ACPI_CPER_SEV_RECOVERABLE,
>> +ACPI_CPER_SEV_FATAL,
>> +ACPI_CPER_SEV_CORRECTED,
>> +ACPI_CPER_SEV_NONE,
>> +};
>> +
>>  /*
>>   * Now only support ARMv8 SEA notification type error source
>>   */
>> @@ -49,6 +77,16 @@
>>   */
>>  #define ACPI_GHES_SOURCE_GENERIC_ERROR_V2   10
>>
>> +#define UUID_BE(a, b, c, d0, d1, d2, d3, d4, d5, d6, d7)\
>> +{{{ ((a) >> 24) & 0xff, ((a) >> 16) & 0xff, ((a) >> 8) & 0xff, (a) & 
>> 0xff, \
>> +((b) >> 8) & 0xff, (b) & 0xff,   \
>> +((c) >> 8) & 0xff, (c) & 0xff,\
>> +(d0), (d1), (d2), (d3), (d4), (d5), (d6), (d7) } } }
>> +
>> +#define UEFI_CPER_SEC_PLATFORM_MEM   \
>> +UUID_BE(0xA5BC1114, 0x6F64, 0x4EDE, 0xB8, 0x63, 0x3E, 0x83, \
>> +0xED, 0x7C, 0x83, 0xB1)
>> +
>>  /*
>>   * | +--+ 0
>>   * | |Header|
>> @@ -77,6 +115,174 @@ typedef struct AcpiGhesState {
>>  uint64_t ghes_addr_le;
>>  } AcpiGhesState;
>>
>> +/*
>> + * Total size for Generic Error Status Block
>> + * ACPI 6.2: 18.3.2.7.1 Generic Error Data,
>> + * Table 18-380 Generic Error Status Block
>> + */
>> +#define ACPI_GHES_GESB_SIZE 20
> 
> Minor: This is not entirely correct: GEDE is part of GESB so the total length
> would be ACPI_GHES_GESB_SIZE + n* sizeof(GEDE)
yes, the comments needs to correct.

> 
>> +/* The offset of Data Length in Generic Error Status Block */
>> +#define ACPI_GHES_GESB_DATA_LENGTH_OFFSET   12
>> +
> 
> If those were nicely represented as structures you get the offsets easily
> without having number of defines. That could simplify the code and make it
> more readable - see comments below
> 
>> +/*
>> + * Record the value of data length for each error status block to avoid 
>> getting
>> + * this value from guest.
>> + */
>> +static uint32_t acpi_ghes_data_length[ACPI_GHES_ERROR_SOURCE_COUNT];
>> +
>> +/*
>> + * Generic Error Data Entry
>> + * ACPI 6.1: 18.3.2.7.1 Generic Error Data
>> + */
>> +static void 

Re: [RESEND PATCH v21 0/6] Add ARMv8 RAS virtualization support in QEMU

[gengdongjiu]

On 2019/12/3 2:27, Peter Maydell wrote:
>> application within guest, host does not know which application encounters
>> errors.
>>
>> For the ARMv8 SEA/SEI, KVM or host kernel delivers SIGBUS to notify 
>> userspace.
>> After user space gets the notification, it will record the CPER into guest 
>> GHES
>> buffer and inject an exception or IRQ into guest.
>>
>> In the current implementation, if the type of SIGBUS is BUS_MCEERR_AR, we 
>> will
>> treat it as a synchronous exception, and notify guest with ARMv8 SEA
>> notification type after recording CPER into guest.
> Hi; I've given you reviewed-by tags on a couple of patches; other
> people have given review comments on some of the other patches,
> so I think you have enough to do a v22 addressing those.
Thanks very much for the reviewed-by tags,  we will upload v22.


> > thanks
> -- PMM
> .
> 

Re: [RESEND PATCH v21 3/6] ACPI: Add APEI GHES table generation support

[gengdongjiu]

On 2019/11/25 17:48, Igor Mammedov wrote:
>>>..
>>> bios_linker_loader_add_pointer(linker, ACPI_BUILD_TABLE_FILE,
>>> ACPI_GHES_ERROR_STATUS_ADDRESS_OFFSET(hest_start, source_id),
>>> sizeof(uint64_t), ACPI_GHES_ERRORS_FW_CFG_FILE,
>>> source_id * sizeof(uint64_t));
>>>   ...
>>> }
>>>
>>> My previous series patch support 2 error sources, but now only enable 'SEA' 
>>> type Error Source  
>> I'd try to merge this, worry about extending things later.
>> This is at v21 and the simpler you can keep things,
>> the faster it'll go in.
> I don't think the series is ready for merging yet.
> It has a number of issues (not stylistic ones) that need to be fixed first.
> 
> As for extending, I think I've suggested to simplify series
> to account for single error source only in some places so it
> would be easier on author and reviewers and worry about extending
> it later.
sure, thanks for the review, we are preparing another series which will fix the 
issues that you mentioned.

> 

Re: [RESEND PATCH v21 3/6] ACPI: Add APEI GHES table generation support

[gengdongjiu]

On 2019/11/18 21:21, Michael S. Tsirkin wrote:
>> If use offset relative to GAS structure, the code does not easily extend to 
>> support more Generic Hardware Error Source.
>> if use offset relative to hest_start, just use a loop, the code can support  
>> more error source, for example:
>> for (source_id = 0; i> {
>>..
>> bios_linker_loader_add_pointer(linker, ACPI_BUILD_TABLE_FILE,
>> ACPI_GHES_ERROR_STATUS_ADDRESS_OFFSET(hest_start, source_id),
>> sizeof(uint64_t), ACPI_GHES_ERRORS_FW_CFG_FILE,
>> source_id * sizeof(uint64_t));
>>   ...
>> }
>>
>> My previous series patch support 2 error sources, but now only enable 'SEA' 
>> type Error Source
> I'd try to merge this, worry about extending things later.
> This is at v21 and the simpler you can keep things,
> the faster it'll go in.
Thanks a lot for the comments. Yes, I think we can merge the v21 series.

Re: [RESEND PATCH v21 3/6] ACPI: Add APEI GHES table generation support

[gengdongjiu]

On 2019/11/18 20:49, gengdongjiu wrote:
>>> + */
>>> +build_append_int_noprefix(table_data, source_id, 2);
>>> +/* Related Source Id */
>>> +build_append_int_noprefix(table_data, 0x, 2);
>>> +/* Flags */
>>> +build_append_int_noprefix(table_data, 0, 1);
>>> +/* Enabled */
>>> +build_append_int_noprefix(table_data, 1, 1);
>>> +
>>> +/* Number of Records To Pre-allocate */
>>> +build_append_int_noprefix(table_data, 1, 4);
>>> +/* Max Sections Per Record */
>>> +build_append_int_noprefix(table_data, 1, 4);
>>> +/* Max Raw Data Length */
>>> +build_append_int_noprefix(table_data, ACPI_GHES_MAX_RAW_DATA_LENGTH, 
>>> 4);
>>> +
>>> +/* Error Status Address */
>>> +build_append_gas(table_data, AML_AS_SYSTEM_MEMORY, 0x40, 0,
>>> + 4 /* QWord access */, 0);
>>> +bios_linker_loader_add_pointer(linker, ACPI_BUILD_TABLE_FILE,
>>> +ACPI_GHES_ERROR_STATUS_ADDRESS_OFFSET(hest_start, source_id),
>> it's fine only if GHESv2 is the only entries in HEST, but once
>> other types are added this macro will silently fall apart and
>> cause table corruption.
   why  silently fall?
   I think the acpi_ghes.c only support GHESv2 type, not support other type.

>>
>> Instead of offset from hest_start, I suggest to use offset relative
>> to GAS structure, here is an idea>>
>> #define GAS_ADDR_OFFSET 4
>>
>> off = table->len
>> build_append_gas()
>> bios_linker_loader_add_pointer(...,
>> off + GAS_ADDR_OFFSET, ...

If use offset relative to GAS structure, the code does not easily extend to 
support more Generic Hardware Error Source.
if use offset relative to hest_start, just use a loop, the code can support  
more error source, for example:
for (source_id = 0; i

Re: [RESEND PATCH v21 3/6] ACPI: Add APEI GHES table generation support

[gengdongjiu]

Hi,Igor,
   Thanks for you review and time.

>
>> +/*
>> + * Type:
>> + * Generic Hardware Error Source version 2(GHESv2 - Type 10)
>> + */
>> +build_append_int_noprefix(table_data, 
>> ACPI_GHES_SOURCE_GENERIC_ERROR_V2, 2);
>> +/*
>> + * Source Id
> 
>> + * Once we support more than one hardware error sources, we need to
>> + * increase the value of this field.
> I'm not sure ^^^ is correct, according to spec it's just unique id per
> distinct error structure, so we just assign arbitrary values to each
> declared source and that never changes once assigned.
The source id is used to distinct the error source, for each source， the 
‘source id’ is unique，
but different source has different source id. for example, the 'source id' of 
the error source 0 is 0,
the 'source id' of the error source 1 is 1.



> 
> For now I'd make source_id an enum with one member
>   enum {
> ACPI_HEST_SRC_ID_SEA = 0,
> /* future ids go here */
> ACPI_HEST_SRC_ID_RESERVED,
>   }
If we only have one error source, we can use enum instead of allocating magic 0.
But if we have more error source , such as 10 error source. using enum  maybe 
not a good idea.

for example, if there are 10 error sources, I can just using below loop

for(i=0; i< 10; i++)
   build_ghes_v2（source_id++）;

> 
> and use that instead of allocating magic 0 at the beginning of the function.
>  build_ghes_v2(ACPI_HEST_GHES_SEA);
> Also add a comment to declaration that already assigned values are not to be 
> changed
> 
>> + */
>> +build_append_int_noprefix(table_data, source_id, 2);
>> +/* Related Source Id */
>> +build_append_int_noprefix(table_data, 0x, 2);
>> +/* Flags */
>> +build_append_int_noprefix(table_data, 0, 1);
>> +/* Enabled */
>> +build_append_int_noprefix(table_data, 1, 1);
>> +
>> +/* Number of Records To Pre-allocate */
>> +build_append_int_noprefix(table_data, 1, 4);
>> +/* Max Sections Per Record */
>> +build_append_int_noprefix(table_data, 1, 4);
>> +/* Max Raw Data Length */
>> +build_append_int_noprefix(table_data, ACPI_GHES_MAX_RAW_DATA_LENGTH, 4);
>> +
>> +/* Error Status Address */
>> +build_append_gas(table_data, AML_AS_SYSTEM_MEMORY, 0x40, 0,
>> + 4 /* QWord access */, 0);
>> +bios_linker_loader_add_pointer(linker, ACPI_BUILD_TABLE_FILE,
>> +ACPI_GHES_ERROR_STATUS_ADDRESS_OFFSET(hest_start, source_id),
> it's fine only if GHESv2 is the only entries in HEST, but once
> other types are added this macro will silently fall apart and
> cause table corruption.
> 
> Instead of offset from hest_start, I suggest to use offset relative
> to GAS structure, here is an idea
> 
> #define GAS_ADDR_OFFSET 4
> 
> off = table->len
> build_append_gas()
> bios_linker_loader_add_pointer(...,
> off + GAS_ADDR_OFFSET, ...
I think your suggestion is good.

> 
>> +ACPI_GHES_ADDRESS_SIZE, ACPI_GHES_ERRORS_FW_CFG_FILE,
>> +source_id * ACPI_GHES_ADDRESS_SIZE);
>> +
>> +/*
>> + * Notification Structure
>> + * Now only enable ARMv8 SEA notification type
>> + */
>> +acpi_ghes_build_notify(table_data, ACPI_GHES_NOTIFY_SEA);
>> +
>> +/* Error Status Block Length */
>> +build_append_int_noprefix(table_data, ACPI_GHES_MAX_RAW_DATA_LENGTH, 4);
>> +
>> +/*
>> + * Read Ack Register
>> + * ACPI 6.1: 18.3.2.8 Generic Hardware Error Source
>> + * version 2 (GHESv2 - Type 10)
>> + */
>> +build_append_gas(table_data, AML_AS_SYSTEM_MEMORY, 0x40, 0,
>> + 4 /* QWord access */, 0);
>> +bios_linker_loader_add_pointer(linker, ACPI_BUILD_TABLE_FILE,
>> +ACPI_GHES_READ_ACK_REGISTER_ADDRESS_OFFSET(hest_start, 0),
> ditto
> 
>> +ACPI_GHES_ADDRESS_SIZE, ACPI_GHES_ERRORS_FW_CFG_FILE,
>> +(ACPI_GHES_ERROR_SOURCE_COUNT + source_id) * 
>> ACPI_GHES_ADDRESS_SIZE);
>> +
>> +/*
>> + * Read Ack Preserve
>> + * We only provide the first bit in Read Ack Register to OSPM to write
>> + * while the other bits are preserved.
>> + */
>> +build_append_int_noprefix(table_data, ~0x1ULL, 8);
>> +/* Read Ack Write */
>> +build_append_int_noprefix(table_data, 0x1, 8);
>> +
>> +build_header(linker, table_data, (void *)(table_data->data + 
>> hest_start),
>> +"HEST", table_data->len - hest_start, 1, NULL, "GHES");
> hest is not GHEST specific so s/GHES/NULL/
>  
>> +}
>> +
>> +static AcpiGhesState ges;
>> +void acpi_ghes_add_fw_cfg(FWCfgState *s, GArray *hardware_error)
>> +{
>> +
>> +size_t size = 2 * ACPI_GHES_ADDRESS_SIZE + 
>> ACPI_GHES_MAX_RAW_DATA_LENGTH;
>> +size_t request_block_size = ACPI_GHES_ERROR_SOURCE_COUNT * size;
>> +
> 
>> +/* Create a read-only fw_cfg file for GHES */
>> +fw_cfg_add_file(s, ACPI_GHES_ERRORS_FW_CFG_FILE, hardware_error->data,
>> +request_block_size);
>> +
>> +   

Re: [RESEND PATCH v21 3/6] ACPI: Add APEI GHES table generation support

[gengdongjiu]

> On Fri, 15 Nov 2019 14:32:47 +
> gengdongjiu  wrote:
> 
> > > > + */
> > > > +static void acpi_ghes_build_notify(GArray *table, const uint8_t
> > > > +type)
> > >
> > > typically format should be build_WHAT(), so
> > >  build_ghes_hw_error_notification()
> > >
> > > And I'd move this out into its own patch.
> > > this applies to other trivial in-depended sub-tables, that take all data 
> > > needed to construct them from supplied arguments.
> >
> > I very used your suggested method in previous series[1], but other
> > maintainer suggested to move this function to this file, because he
> > think only GHES used it
> 
> Using this file is ok, what I've meant for you to split function from this 
> patch into a separate smaller patch.
> 
> With the way code split now I have to review 2 big complex patches at the 
> same which makes reviewing hard and I have a hard time
> convincing myself that code it correct.
> 
> Moving small self-contained chunks of code in to separate smaller patches 
> makes review easier.

Ok, got it. Thanks very much for the explanations

Re: [RESEND PATCH v21 3/6] ACPI: Add APEI GHES table generation support

[gengdongjiu]

> > + */
> > +static void acpi_ghes_build_notify(GArray *table, const uint8_t type)
> 
> typically format should be build_WHAT(), so
>  build_ghes_hw_error_notification()
> 
> And I'd move this out into its own patch.
> this applies to other trivial in-depended sub-tables, that take all data 
> needed to construct them from supplied arguments.

I very used your suggested method in previous series[1], but other maintainer 
suggested to move this function to this file, because he think only GHES used it

[1]:
https://patchwork.ozlabs.org/cover/1099428/

> 
> > +{
> > +/* Type */
> > +build_append_int_noprefix(table, type, 1);
> > +/*
> > + * Length:
> > + * Total length of the structure in bytes
> > + */
> > +build_append_int_noprefix(table, 28, 1);
> > +/* Configuration Write Enable */
> > +build_append_int_noprefix(table, 0, 2);
> > +/* Poll Interval */
> > +build_append_int_noprefix(table, 0, 4);
> > +/* Vector */
> > +build_append_int_noprefix(table, 0, 4);
> > +/* Switch To Polling Threshold Value */
> > +build_append_int_noprefix(table, 0, 4);
> > +/* Switch To Polling Threshold Window */
> > +build_append_int_noprefix(table, 0, 4);
> > +/* Error Threshold Value */
> > +build_append_int_noprefix(table, 0, 4);
> > +/* Error Threshold Window */
> > +build_append_int_noprefix(table, 0, 4); }
> > +
> 
> /*
>   Initialize "etc/hardware_errors" and "etc/hardware_errors_addr" fwcfg blobs.
>   See docs/specs/acpi_hest_ghes.rst for blobs format */
> > +void acpi_ghes_build_error_table(GArray *hardware_errors, BIOSLinker
> > +*linker)
> build_ghes_error_table()
> 
> also I'd move this function into its own patch along with other related code 
> that initializes and wires it into virt board.

I ever use your suggested method[1], but other maintainer, it seems Michael, 
suggested to move these functions to this file that used it, because he think 
only GHES used it.

[1]:
https://patchwork.ozlabs.org/patch/1099424/
https://patchwork.ozlabs.org/patch/1099425/
https://patchwork.ozlabs.org/patch/1099430/

Re: [PATCH v21 3/6] ACPI: Add APEI GHES table generation support

[gengdongjiu]

On 2019/11/4 20:14, Xiang Zheng wrote:
> From: Dongjiu Geng 
> 
> This patch implements APEI GHES Table generation via fw_cfg blobs. Now
> it only supports ARMv8 SEA, a type of GHESv2 error source. Afterwards,
> we can extend the supported types if needed. For the CPER section,
> currently it is memory section because kernel mainly wants userspace to
> handle the memory errors.
> 
> This patch follows the spec ACPI 6.2 to build the Hardware Error Source
> table. For more detailed information, please refer to document:
> docs/specs/acpi_hest_ghes.rst
> 
> Suggested-by: Laszlo Ersek 
> Signed-off-by: Dongjiu Geng 
> Signed-off-by: Xiang Zheng 

Hi Xiang,
   please add "Reviewed-by: Michael S. Tsirkin  " which from 
Michael, thanks.

Re: [PATCH v20 0/5] Add ARMv8 RAS virtualization support in QEMU

[gengdongjiu]

On 2019/10/28 23:16, Peter Maydell wrote:
>> Hi Peter,
>> what do you think about it?
> I suggest you just use R: for the moment. The code will all end up going
> through my tree or perhaps Michael's anyway, so it doesn't make much
> practical difference.

ok, got it, thanks.

> 
> thanks
> -- PMM
> .
> 

Re: [PATCH v20 0/5] Add ARMv8 RAS virtualization support in QEMU

[gengdongjiu]

On 2019/10/28 22:56, Michael S. Tsirkin wrote:
> On Mon, Oct 28, 2019 at 09:50:21PM +0800, gengdongjiu wrote:
>> Hi Michael,
>>
>> On 2019/10/28 16:28, Michael S. Tsirkin wrote:
>>>>> gets some testing.  I'll leave this decision to the ARM maintainer.  For
>>>>> ACPI parts:
>>>>>
>>>>> Reviewed-by: Michael S. Tsirkin 
>>>> Got it, Thanks for the Reviewed-by from Michael.
>>>>
>>>> Hi Michael,
>>>>   According to discussion with QEMU community, I finished and developed 
>>>> the whole ARM RAS virtualization solution, and introduce the ARM APEI 
>>>> table in the first time.
>>>> For the newly created files, which are mainly about ARM APEI/GHES part,I 
>>>> would like to maintain them. If you agree it, whether I can add new 
>>>> maintainers[1]? thanks a lot.
>>>>
>>>>
>>>> [1]:
>>>> +ARM APEI Subsystem
>>>> +M: Dongjiu Geng 
>>>> +M: Xiang zheng 
>>>> +L: qemu-...@nongnu.org
>>>> +S: Maintained
>>>> +F: hw/acpi/acpi_ghes.c
>>>>
>>> I think for now you want to be a designated reviewer.  So I'd use an R:
>>> tag.
>>
>>  Thanks for the reply.
>>  I want to be a maintainer for my newly created files, so whether I can use 
>> M: tag. I would like to contribute some time to maintain that, thanks a lot.
> 
> This will fundamentally be up to Peter.

Thanks.

Hi Peter,
what do you think about it?

> 
> 
> 
>>>
>>>>>
>>>>>> ---
> .
> 

Re: [PATCH v20 0/5] Add ARMv8 RAS virtualization support in QEMU

[gengdongjiu]

Hi Michael,

On 2019/10/28 16:28, Michael S. Tsirkin wrote:
>>> gets some testing.  I'll leave this decision to the ARM maintainer.  For
>>> ACPI parts:
>>>
>>> Reviewed-by: Michael S. Tsirkin 
>> Got it, Thanks for the Reviewed-by from Michael.
>>
>> Hi Michael,
>>   According to discussion with QEMU community, I finished and developed the 
>> whole ARM RAS virtualization solution, and introduce the ARM APEI table in 
>> the first time.
>> For the newly created files, which are mainly about ARM APEI/GHES part,I 
>> would like to maintain them. If you agree it, whether I can add new 
>> maintainers[1]? thanks a lot.
>>
>>
>> [1]:
>> +ARM APEI Subsystem
>> +M: Dongjiu Geng 
>> +M: Xiang zheng 
>> +L: qemu-...@nongnu.org
>> +S: Maintained
>> +F: hw/acpi/acpi_ghes.c
>> +F: include/hw/acpi/acpi_ghes.h
>> +F: docs/specs/acpi_hest_ghes.rst
>>
> I think for now you want to be a designated reviewer.  So I'd use an R:
> tag.

 Thanks for the reply.
 I want to be a maintainer for my newly created files, so whether I can use M: 
tag. I would like to contribute some time to maintain that, thanks a lot.

> 
>>>
 ---

Re: [PATCH v20 0/5] Add ARMv8 RAS virtualization support in QEMU

[gengdongjiu]

Michael,

On 2019/10/28 16:28, Michael S. Tsirkin wrote:
>> Got it, Thanks for the Reviewed-by from Michael.
>>
>> Hi Michael,
>>   According to discussion with QEMU community, I finished and developed the 
>> whole ARM RAS virtualization solution, and introduce the ARM APEI table in 
>> the first time.
>> For the newly created files, which are mainly about ARM APEI/GHES part,I 
>> would like to maintain them. If you agree it, whether I can add new 
>> maintainers[1]? thanks a lot.
>>
>>
>> [1]:
>> +ARM APEI Subsystem
>> +M: Dongjiu Geng 
>> +M: Xiang zheng 
>> +L: qemu-...@nongnu.org
>> +S: Maintained
>> +F: hw/acpi/acpi_ghes.c
>> +F: include/hw/acpi/acpi_ghes.h
>> +F: docs/specs/acpi_hest_ghes.rst
>>
> I think for now you want to be a designated reviewer.  So I'd use an R:
> tag.

 Thanks for the reply.
 I want to be a maintainer for my newly created files, so whether I can use M: 
tag. I would like to contribute some times to maintain that, thanks.

> 
>>>

Re: [PATCH v20 0/5] Add ARMv8 RAS virtualization support in QEMU

[gengdongjiu]

Hi Michael/All

On 2019/10/27 18:17, Michael S. Tsirkin wrote:
> On Sat, Oct 26, 2019 at 11:24:42AM +0800, Xiang Zheng wrote:
>> In the ARMv8 platform, the CPU error types are synchronous external 
>> abort(SEA)
>> and SError Interrupt (SEI). If exception happens in guest, sometimes it's 
>> better
>> for guest to perform the recovery, because host does not know the detailed
>> information of guest. For example, if an exception happens in a user-space
>> application within guest, host does not know which application encounters
>> errors.
>>
>> For the ARMv8 SEA/SEI, KVM or host kernel delivers SIGBUS to notify 
>> userspace.
>> After user space gets the notification, it will record the CPER into guest 
>> GHES
>> buffer and inject an exception or IRQ into guest.
>>
>> In the current implementation, if the type of SIGBUS is BUS_MCEERR_AR, we 
>> will
>> treat it as a synchronous exception, and notify guest with ARMv8 SEA
>> notification type after recording CPER into guest.
>>
>> This series of patches are based on Qemu 4.1, which include two parts:
>> 1. Generate APEI/GHES table.
>> 2. Handle the SIGBUS signal, record the CPER in runtime and fill it into 
>> guest
>>memory, then notify guest according to the type of SIGBUS.
>>
>> The whole solution was suggested by James(james.mo...@arm.com); The solution 
>> of
>> APEI section was suggested by Laszlo(ler...@redhat.com).
>> Show some discussions in [1].
>>
>> This series of patches have already been tested on ARM64 platform with RAS
>> feature enabled:
>> Show the APEI part verification result in [2].
>> Show the BUS_MCEERR_AR SIGBUS handling verification result in [3].
> 
> This looks mostly OK to me.  I sent some minor style comments but they
> can be addressed by follow up patches.
> 
> Maybe it's a good idea to merge this before soft freeze to make sure it
> gets some testing.  I'll leave this decision to the ARM maintainer.  For
> ACPI parts:
> 
> Reviewed-by: Michael S. Tsirkin 

Got it, Thanks for the Reviewed-by from Michael.

Hi Michael,
  According to discussion with QEMU community, I finished and developed the 
whole ARM RAS virtualization solution, and introduce the ARM APEI table in the 
first time.
For the newly created files, which are mainly about ARM APEI/GHES part,I would 
like to maintain them. If you agree it, whether I can add new maintainers[1]? 
thanks a lot.


[1]:
+ARM APEI Subsystem
+M: Dongjiu Geng 
+M: Xiang zheng 
+L: qemu-...@nongnu.org
+S: Maintained
+F: hw/acpi/acpi_ghes.c
+F: include/hw/acpi/acpi_ghes.h
+F: docs/specs/acpi_hest_ghes.rst


> 
> 
>> ---

RE: [PATCH v18 0/6] Add ARMv8 RAS virtualization support in QEMU

[gengdongjiu]

ping.

Hi peter/Igor/all，
  can you review these patches，thanks a lot.




--
耿东久 Geng Dongjiu
Mobile: +86-18221809728
Email: gengdong...@huawei.com<mailto:gengdong...@huawei.com>
发件人：zhengxiang (A) 
收件人：pbonzini ;mst ;imammedo 
;shannon.zhaosl ;peter.maydell 
;lersek ;james.morse 
;gengdongjiu ;mtosatti 
;rth ;ehabkost 
;Jonathan Cameron ;xuwei (O) 
;kvm ;qemu-devel 
;qemu-arm ;Linuxarm 

抄 送：Wanghaibin (D) 
时 间：2019-09-17 20:40:21
主题Re: [PATCH v18 0/6] Add ARMv8 RAS virtualization support in QEMU

Hi all,

This patch series has been tested for both TCG and KVM scenes.

1) Test for TCG:
   - Re-compile qemu after applying the patch refered to 
https://patchwork.kernel.org/cover/10942757/#22640271).
   - Use command line shown below to start qemu:
./qemu-system-aarch64 \
-name guest=ras \
-machine virt,gic-version=3,ras=on \
-cpu cortex-a57 \
-bios /usr/share/edk2/aarch64/QEMU_EFI.fd \
-nodefaults \
-kernel ${GUEST_KERNEL} \
-initrd ${GUEST_FS} \
-append "rdinit=init console=ttyAMA0 earlycon=pl011,0x900" \
-m 8192 \
-smp 4 \
-serial stdio \

   - Send a signal to one of the VCPU threads:
kill -s SIGBUS 71571

   - The result of test is shown below:

[   41.194753] {1}[Hardware Error]: Hardware error from APEI Generic 
Hardware Error Source: 0
[   41.197329] {1}[Hardware Error]: event severity: recoverable
[   41.199078] {1}[Hardware Error]:  Error 0, type: recoverable
[   41.200829] {1}[Hardware Error]:   section_type: memory error
[   41.202603] {1}[Hardware Error]:   physical_address: 0x400a1000
[   41.204649] {1}[Hardware Error]:   error_type: 0, unknown
[   41.206328] EDAC MC0: 1 UE Unknown on unknown label ( page:0x400a1 
offset:0x0 grain:0)
[   41.208788] Internal error: synchronous external abort: 96000410 [#1] SMP
[   41.210879] Modules linked in:
[   41.211823] CPU: 2 PID: 0 Comm: swapper/2 Not tainted 4.19.0+ #8
[   41.213698] Hardware name: QEMU KVM Virtual Machine, BIOS 0.0.0 
02/06/2015
[   41.215812] pstate: 60c00085 (nZCv daIf +PAN +UAO)
[   41.217296] pc : cpu_do_idle+0x8/0xc
[   41.218400] lr : arch_cpu_idle+0x2c/0x1b8
[   41.219629] sp : 09f9bf00
[   41.220649] x29: 09f9bf00 x28: 
[   41.222310] x27:  x26: 8001fe471d80
[   41.223945] x25:  x24: 0937ba38
[   41.225581] x23: 090b3338 x22: 09379000
[   41.227220] x21: 0937b000 x20: 0004
[   41.228871] x19: 090a6000 x18: 
[   41.230517] x17:  x16: 
[   41.232165] x15:  x14: 
[   41.233810] x13: 089f4da8 x12: 000e
[   41.235448] x11: 089f4d80 x10: 0af0
[   41.237101] x9 : 09f9be80 x8 : 8001fe4728d0
[   41.238738] x7 : 0004 x6 : 8001fffbaf30
[   41.240380] x5 : 0c43b940 x4 : 8001f6f0c000
[   41.242030] x3 : 0001 x2 : 09f9bf00
[   41.243666] x1 : 8001fffb82c8 x0 : 090a6018
[   41.245306] Process swapper/2 (pid: 0, stack limit = 0x(ptrval))
[   41.247378] Call trace:
[   41.248117]  cpu_do_idle+0x8/0xc
[   41.249111]  do_idle+0x1dc/0x2a8
[   41.250111]  cpu_startup_entry+0x28/0x30
[   41.251319]  secondary_start_kernel+0x180/0x1c8
[   41.252725] Code: a8c17bfd d65f03c0 d5033f9f d503207f (d65f03c0)
[   41.254606] ---[ end trace 221bc8a614fb5a1d ]---
[   41.256030] Kernel panic - not syncing: Fatal exception
[   41.257644] SMP: stopping secondary CPUs
[   41.258912] Kernel Offset: disabled
[   41.260011] CPU features: 0x0,22a00238
[   41.261178] Memory Limit: none
[   41.262122] ---[ end Kernel panic - not syncing: Fatal exception ]---

2) Test for KVM:
   - Use command line shown below to start qemu:
./qemu-system-aarch64 \
-name guest=ras \
-machine virt,accel=kvm,gic-version=3,ras=on \
-cpu host \
-bios /usr/share/edk2/aarch64/QEMU_EFI.fd \
-nodefaults \
-kernel ${GUEST_KERNEL} \
-initrd ${GUEST_FS} \
-append "rdinit=init console=ttyAMA0 earlycon=pl011,0x900" \
-m 8192 \
-smp 4 \
-serial stdio \

   - Run mca-recover and get the GPA(IPA) of allocated page which would be 
corrupted on the later.
   - Convert the GPA to HPA and corrupt this HPA via APEI/EINJ.
   - Go back to guest and continue to read this page.

   - The result of test is shown below:

root@genericarmv8:~/tools# ./mca-recover
pagesize: 0x1000
before clear cache
 

Re: [PATCH v18 0/6] Add ARMv8 RAS virtualization support in QEMU

[gengdongjiu]

Thanks xiang's continue upstream and test.
Hope maintainer can review it.


On 2019/9/17 20:39, Xiang Zheng wrote:
> Hi all,
> 
> This patch series has been tested for both TCG and KVM scenes.
> 
> 1) Test for TCG:
>- Re-compile qemu after applying the patch refered to 
> https://patchwork.kernel.org/cover/10942757/#22640271).
>- Use command line shown below to start qemu:
> ./qemu-system-aarch64 \
> -name guest=ras \
> -machine virt,gic-version=3,ras=on \
> -cpu cortex-a57 \
> -bios /usr/share/edk2/aarch64/QEMU_EFI.fd \
> -nodefaults \
> -kernel ${GUEST_KERNEL} \
> -initrd ${GUEST_FS} \
> -append "rdinit=init console=ttyAMA0 
> earlycon=pl011,0x900" \
> -m 8192 \
> -smp 4 \
> -serial stdio \
> 
>- Send a signal to one of the VCPU threads:
> kill -s SIGBUS 71571
> 
>- The result of test is shown below:
> 
> [   41.194753] {1}[Hardware Error]: Hardware error from APEI Generic 
> Hardware Error Source: 0
> [   41.197329] {1}[Hardware Error]: event severity: recoverable
> [   41.199078] {1}[Hardware Error]:  Error 0, type: recoverable
> [   41.200829] {1}[Hardware Error]:   section_type: memory error
> [   41.202603] {1}[Hardware Error]:   physical_address: 0x400a1000
> [   41.204649] {1}[Hardware Error]:   error_type: 0, unknown
> [   41.206328] EDAC MC0: 1 UE Unknown on unknown label ( page:0x400a1 
> offset:0x0 grain:0)
> [   41.208788] Internal error: synchronous external abort: 96000410 [#1] 
> SMP
> [   41.210879] Modules linked in:
> [   41.211823] CPU: 2 PID: 0 Comm: swapper/2 Not tainted 4.19.0+ #8
> [   41.213698] Hardware name: QEMU KVM Virtual Machine, BIOS 0.0.0 
> 02/06/2015
> [   41.215812] pstate: 60c00085 (nZCv daIf +PAN +UAO)
> [   41.217296] pc : cpu_do_idle+0x8/0xc
> [   41.218400] lr : arch_cpu_idle+0x2c/0x1b8
> [   41.219629] sp : 09f9bf00
> [   41.220649] x29: 09f9bf00 x28: 
> [   41.222310] x27:  x26: 8001fe471d80
> [   41.223945] x25:  x24: 0937ba38
> [   41.225581] x23: 090b3338 x22: 09379000
> [   41.227220] x21: 0937b000 x20: 0004
> [   41.228871] x19: 090a6000 x18: 
> [   41.230517] x17:  x16: 
> [   41.232165] x15:  x14: 
> [   41.233810] x13: 089f4da8 x12: 000e
> [   41.235448] x11: 089f4d80 x10: 0af0
> [   41.237101] x9 : 09f9be80 x8 : 8001fe4728d0
> [   41.238738] x7 : 0004 x6 : 8001fffbaf30
> [   41.240380] x5 : 0c43b940 x4 : 8001f6f0c000
> [   41.242030] x3 : 0001 x2 : 09f9bf00
> [   41.243666] x1 : 8001fffb82c8 x0 : 090a6018
> [   41.245306] Process swapper/2 (pid: 0, stack limit = 
> 0x(ptrval))
> [   41.247378] Call trace:
> [   41.248117]  cpu_do_idle+0x8/0xc
> [   41.249111]  do_idle+0x1dc/0x2a8
> [   41.250111]  cpu_startup_entry+0x28/0x30
> [   41.251319]  secondary_start_kernel+0x180/0x1c8
> [   41.252725] Code: a8c17bfd d65f03c0 d5033f9f d503207f (d65f03c0)
> [   41.254606] ---[ end trace 221bc8a614fb5a1d ]---
> [   41.256030] Kernel panic - not syncing: Fatal exception
> [   41.257644] SMP: stopping secondary CPUs
> [   41.258912] Kernel Offset: disabled
> [   41.260011] CPU features: 0x0,22a00238
> [   41.261178] Memory Limit: none
> [   41.262122] ---[ end Kernel panic - not syncing: Fatal exception ]---
> 
> 2) Test for KVM:
>- Use command line shown below to start qemu:
> ./qemu-system-aarch64 \
> -name guest=ras \
> -machine virt,accel=kvm,gic-version=3,ras=on \
> -cpu host \
> -bios /usr/share/edk2/aarch64/QEMU_EFI.fd \
> -nodefaults \
> -kernel ${GUEST_KERNEL} \
> -initrd ${GUEST_FS} \
> -append "rdinit=init console=ttyAMA0 earlycon=pl011,0x900" \
> -m 8192 \
> -smp 4 \
> -serial stdio \
> 
>- Run mca-recover and get the GPA(IPA) of allocated page which would be 
> corrupted on the later.
>- Convert the GPA to HPA and corrupt this HPA via APEI/EINJ.
>- Go back to guest and continue to read this page.
> 
>- The result of test is shown below:
> 
> root@genericarmv8:~/tools# ./mca-recover
> pagesize: 0x1000
> before clear cache
> flags for page 0x2317b2: uptodate active mmap anon swapbacked
> vtop(0x9c9e8000) = 0x2317b2000
> Hit any key to access: before read
> 
> after read
> Access at Tue Sep 17 01:41:14 2019
> 
> flags for page 0x2317b2: uptodate active 

Re: [Qemu-devel] [PATCH] hw/arm/boot: Load the Non Linux initrd to the memory

[gengdongjiu]

On 2019/8/27 17:47, Peter Maydell wrote:
> On Tue, 27 Aug 2019 at 10:42, Dongjiu Geng  wrote:
>>
>> Except support linux operation system, qemu also supports other
>> operation system which is non linux, such as microkernel system.
>>
>> But now Qemu only load linux initrd, so change it to load both
>> linux and Non-linux initrd Image.
> 
> We currently support two methods of booting:
>  (1) using the boot protocol defined by the Linux kernel
>  (which includes how to find the DTB, initrd, what the
>  secondary CPUs do, and so on)
>  (2) you're a 'bare-metal' image, in which case you get
>  complete control of all the CPUs at once in the same
>  way the hardware does. Raw hardware doesn't provide
>  initrd files, and nor does QEMU.
> 
> This patch seems to be trying to introduce a third hybrid
> thing. Is there a specification for whatever this boot
> protocol is? How many guest OSes use it? Are they common?
> 
> If you want an initrd, you can always wrap your guest OS in
> a shim which complies with the Linux kernel boot protocol.
> I think that would be a better approach than this patch.

OK， thanks for the suggestion, I will use your suggested method.

> 
> thanks
> -- PMM
> .
> 

Re: [Qemu-devel] [PATCH v17 07/10] ACPI: Add APEI GHES table generation support

[gengdongjiu]

On 2019/6/24 20:27, Igor Mammedov wrote:
> On Tue, 14 May 2019 04:18:20 -0700
> Dongjiu Geng  wrote:
> 
>> This implements APEI GHES Table generation via fw_cfg blobs.
>> Now it only support GPIO-Signal and ARMv8 SEA two types of GHESv2 error
>> source. Afterwards, we can extend the supported types if needed. For the
>> CPER section type, currently it is memory section because kernel
>> mainly wants userspace to handle the memory errors.
>>
>> This patch follows the spec ACPI 6.2 to build the Hardware Error Source
>> table, for the detailed information, please refer to document:
>> docs/specs/acpi_hest_ghes.txt
>>
>> Suggested-by: Laszlo Ersek 
>> Signed-off-by: Dongjiu Geng 
>> ---
>>  default-configs/arm-softmmu.mak |   1 +
>>  hw/acpi/Kconfig |   4 +
>>  hw/acpi/Makefile.objs   |   1 +
>>  hw/acpi/acpi_ghes.c | 171 
>> 
>>  hw/acpi/aml-build.c |   2 +
>>  hw/arm/virt-acpi-build.c|  12 +++
>>  include/hw/acpi/acpi_ghes.h |  79 +++
>>  include/hw/acpi/aml-build.h |   1 +
>>  8 files changed, 271 insertions(+)
>>  create mode 100644 hw/acpi/acpi_ghes.c
>>  create mode 100644 include/hw/acpi/acpi_ghes.h
>>
>> diff --git a/default-configs/arm-softmmu.mak 
>> b/default-configs/arm-softmmu.mak
>> index 613d19a..7b33ae9 100644
>> --- a/default-configs/arm-softmmu.mak
>> +++ b/default-configs/arm-softmmu.mak
>> @@ -160,3 +160,4 @@ CONFIG_MUSICPAL=y
>>  
>>  # for realview and versatilepb
>>  CONFIG_LSI_SCSI_PCI=y
>> +CONFIG_ACPI_APEI=y
>> diff --git a/hw/acpi/Kconfig b/hw/acpi/Kconfig
>> index eca3bee..5228a4b 100644
>> --- a/hw/acpi/Kconfig
>> +++ b/hw/acpi/Kconfig
>> @@ -23,6 +23,10 @@ config ACPI_NVDIMM
>>  bool
>>  depends on ACPI
>>  
>> +config ACPI_APEI
>> +bool
>> +depends on ACPI
>> +
>>  config ACPI_VMGENID
>>  bool
>>  default y
>> diff --git a/hw/acpi/Makefile.objs b/hw/acpi/Makefile.objs
>> index 2d46e37..5099ada 100644
>> --- a/hw/acpi/Makefile.objs
>> +++ b/hw/acpi/Makefile.objs
>> @@ -6,6 +6,7 @@ common-obj-$(CONFIG_ACPI_MEMORY_HOTPLUG) += memory_hotplug.o
>>  common-obj-$(CONFIG_ACPI_CPU_HOTPLUG) += cpu.o
>>  common-obj-$(CONFIG_ACPI_NVDIMM) += nvdimm.o
>>  common-obj-$(CONFIG_ACPI_VMGENID) += vmgenid.o
>> +common-obj-$(CONFIG_ACPI_APEI) += acpi_ghes.o
>>  common-obj-$(call lnot,$(CONFIG_ACPI_X86)) += acpi-stub.o
>>  
>>  common-obj-y += acpi_interface.o
>> diff --git a/hw/acpi/acpi_ghes.c b/hw/acpi/acpi_ghes.c
>> new file mode 100644
>> index 000..d03e797
>> --- /dev/null
>> +++ b/hw/acpi/acpi_ghes.c
>> @@ -0,0 +1,171 @@
>> +/* Support for generating APEI tables and record CPER for Guests
>> + *
>> + * Copyright (C) 2017 HuaWei Corporation.
>> + *
>> + * Author: Dongjiu Geng 
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License as published by
>> + * the Free Software Foundation; either version 2 of the License, or
>> + * (at your option) any later version.
>> +
>> + * This program is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> + * GNU General Public License for more details.
>> +
>> + * You should have received a copy of the GNU General Public License along
>> + * with this program; if not, see .
>> + */
>> +
>> +#include "qemu/osdep.h"
>> +#include "hw/acpi/acpi.h"
>> +#include "hw/acpi/aml-build.h"
>> +#include "hw/acpi/acpi_ghes.h"
>> +#include "hw/nvram/fw_cfg.h"
>> +#include "sysemu/sysemu.h"
>> +#include "qemu/error-report.h"
>> +
>> +/* Build table for the hardware error fw_cfg blob */
>> +void build_hardware_error_table(GArray *hardware_errors, BIOSLinker *linker)
>> +{
>> +int i;
>> +
>> +/*
>> + * | +--+
>> + * | |error_block_address   |
>> + * | |  ..  |
>> + * | +--+
>> + * | |read_ack_register |
>> + * | | ...  |
>> + * | +--+
>> + * | |  Error Status Data Block |
>> + * | |  |
>> + * | +--+
>> + */
>> +
>> +/* Build error_block_address */
>> +build_append_int_noprefix((void *)hardware_errors, 0,
>> +GHES_ADDRESS_SIZE * ACPI_HEST_ERROR_SOURCE_COUNT);
> read CODING_STYLE wrt indentation rules.
 thanks for the reminder.

> 
>> +
>> +/* Build read_ack_register */
>> +for (i = 0; i < ACPI_HEST_ERROR_SOURCE_COUNT; i++)
>> +/* Initialize the value of read_ack_register to 1, so GHES can be
>> + * writeable in the first time
> where does this come from? a pointer to spec?

It is come from "ACPI 6.2, Table 18-382 Generic Hardware Error Source version 2 
(GHESv2) Structure", I will add a comment.
QEMU init the 

Re: [Qemu-devel] [PATCH v17 01/10] hw/arm/virt: Add RAS platform version for migration

[gengdongjiu]

On 2019/6/25 21:16, Igor Mammedov wrote:
>>> In light of the race for leaner QEMU and faster startup times,
>>> it might be better to make RAS optional and make user explicitly
>>> enable it using a machine option.  
>> I will add a machine option to make RAS optional, do you think we should 
>> enable or disable it by default? I think it is better if we enable it by 
>> default.
> I'd start with disabled by default as it's easy to make it
> enabled by default later and we can't do so other way around.
ok

> 
>  
>>>   
  }
  DEFINE_VIRT_MACHINE(4, 0)
  
 diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt

Re: [Qemu-devel] [PATCH v17 08/10] KVM: Move related hwpoison page functions to accel/kvm/ folder

[gengdongjiu]

On 2019/6/24 20:32, Igor Mammedov wrote:
> On Tue, 14 May 2019 04:18:21 -0700
> Dongjiu Geng  wrote:
> 
>> kvm_hwpoison_page_add() and kvm_unpoison_all() will be used both
>> by X86 and ARM platforms, so move these functions to a common
>> accel/kvm/ folder to avoid duplicate code.
>>
>> Signed-off-by: Dongjiu Geng 
>> ---
>>  accel/kvm/kvm-all.c | 33 +
>>  include/exec/ram_addr.h | 24 
>>  target/arm/kvm.c|  3 +++
>>  target/i386/kvm.c   | 34 +-
>>  4 files changed, 61 insertions(+), 33 deletions(-)
>>
>> diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
>> index 524c4dd..b9f9f29 100644
>> --- a/accel/kvm/kvm-all.c
>> +++ b/accel/kvm/kvm-all.c
>> @@ -625,6 +625,39 @@ int kvm_vm_check_extension(KVMState *s, unsigned int 
>> extension)
>>  return ret;
>>  }
>>  
>> +typedef struct HWPoisonPage {
>> +ram_addr_t ram_addr;
>> +QLIST_ENTRY(HWPoisonPage) list;
>> +} HWPoisonPage;
>> +
>> +static QLIST_HEAD(, HWPoisonPage) hwpoison_page_list =
>> +QLIST_HEAD_INITIALIZER(hwpoison_page_list);
>> +
>> +void kvm_unpoison_all(void *param)
>> +{
>> +HWPoisonPage *page, *next_page;
>> +
>> +QLIST_FOREACH_SAFE(page, _page_list, list, next_page) {
>> +QLIST_REMOVE(page, list);
>> +qemu_ram_remap(page->ram_addr, TARGET_PAGE_SIZE);
>> +g_free(page);
>> +}
>> +}
>> +
>> +void kvm_hwpoison_page_add(ram_addr_t ram_addr)
>> +{
>> +HWPoisonPage *page;
>> +
>> +QLIST_FOREACH(page, _page_list, list) {
>> +if (page->ram_addr == ram_addr) {
>> +return;
>> +}
>> +}
>> +page = g_new(HWPoisonPage, 1);
>> +page->ram_addr = ram_addr;
>> +QLIST_INSERT_HEAD(_page_list, page, list);
>> +}
>> +
>>  static uint32_t adjust_ioeventfd_endianness(uint32_t val, uint32_t size)
>>  {
>>  #if defined(HOST_WORDS_BIGENDIAN) != defined(TARGET_WORDS_BIGENDIAN)
>> diff --git a/include/exec/ram_addr.h b/include/exec/ram_addr.h
>> index 139ad79..193b0a7 100644
>> --- a/include/exec/ram_addr.h
>> +++ b/include/exec/ram_addr.h
> 
> it's not file for KVM specific code,
> maybe Paolo could suggest a bettor place ...
I remember some people suggests me to move the code to KVM before, may be he is 
Peter.
Paolo, can you give a better place?

> 
> 
>> @@ -116,6 +116,30 @@ void qemu_ram_free(RAMBlock *block);
>>  
>>  int qemu_ram_resize(RAMBlock *block, ram_addr_t newsize, Error **errp);
>>  
>> +/**
>> + * kvm_hwpoison_page_add:
>> + *
>> + * Parameters:
>> + *  @ram_addr: the address in the RAM for the poisoned page
>> + *
>> + * Add a poisoned page to the list
>> + *
>> + * Return: None.
>> + */
>> +void kvm_hwpoison_page_add(ram_addr_t ram_addr);
>> +
>> +/**
>> + * kvm_unpoison_all:
>> + *
>> + * Parameters:
>> + *  @param: some data may be passed to this function
>> + *
>> + * Free and remove all the poisoned pages in the list
>> + *
>> + * Return: None.
>> + */
>> +void kvm_unpoison_all(void *param);
>> +
>>  #define DIRTY_CLIENTS_ALL ((1 << DIRTY_MEMORY_NUM) - 1)
>>  #define DIRTY_CLIENTS_NOCODE  (DIRTY_CLIENTS_ALL & ~(1 << 
>> DIRTY_MEMORY_CODE))
>>  
>> diff --git a/target/arm/kvm.c b/target/arm/kvm.c
>> index 5995634..6d3b25b 100644
>> --- a/target/arm/kvm.c
>> +++ b/target/arm/kvm.c
>> @@ -29,6 +29,7 @@
>>  #include "exec/address-spaces.h"
>>  #include "hw/boards.h"
>>  #include "qemu/log.h"
>> +#include "exec/ram_addr.h"
>>  
>>  const KVMCapabilityInfo kvm_arch_required_capabilities[] = {
>>  KVM_CAP_LAST_INFO
>> @@ -187,6 +188,8 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
>>  
>>  cap_has_mp_state = kvm_check_extension(s, KVM_CAP_MP_STATE);
>>  
>> +qemu_register_reset(kvm_unpoison_all, NULL);
>> +
>>  return 0;
>>  }
>>  
>> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
>> index 3b29ce5..9bdb879 100644
>> --- a/target/i386/kvm.c
>> +++ b/target/i386/kvm.c
>> @@ -46,6 +46,7 @@
>>  #include "migration/blocker.h"
>>  #include "exec/memattrs.h"
>>  #include "trace.h"
>> +#include "exec/ram_addr.h"
>>  
>>  //#define DEBUG_KVM
>>  
>> @@ -467,39 +468,6 @@ uint32_t kvm_arch_get_supported_msr_feature(KVMState 
>> *s, uint32_t index)
>>  }
>>  
>>  
>> -typedef struct HWPoisonPage {
>> -ram_addr_t ram_addr;
>> -QLIST_ENTRY(HWPoisonPage) list;
>> -} HWPoisonPage;
>> -
>> -static QLIST_HEAD(, HWPoisonPage) hwpoison_page_list =
>> -QLIST_HEAD_INITIALIZER(hwpoison_page_list);
>> -
>> -static void kvm_unpoison_all(void *param)
>> -{
>> -HWPoisonPage *page, *next_page;
>> -
>> -QLIST_FOREACH_SAFE(page, _page_list, list, next_page) {
>> -QLIST_REMOVE(page, list);
>> -qemu_ram_remap(page->ram_addr, TARGET_PAGE_SIZE);
>> -g_free(page);
>> -}
>> -}
>> -
>> -static void kvm_hwpoison_page_add(ram_addr_t ram_addr)
>> -{
>> -HWPoisonPage *page;
>> -
>> -QLIST_FOREACH(page, _page_list, list) {
>> -if (page->ram_addr == ram_addr) {
>> -return;
>> -  

Re: [Qemu-devel] [PATCH v17 10/10] target-arm: kvm64: handle SIGBUS signal from kernel or KVM

[gengdongjiu]

On 2019/6/24 21:08, Igor Mammedov wrote:
> On Tue, 14 May 2019 04:18:23 -0700
> Dongjiu Geng  wrote:
> 
>> Add SIGBUS signal handler. In this handler, it checks the SIGBUS type,
>> translates the host VA delivered by host to guest PA, then fill this PA
>> to guest APEI GHES memory, then notify guest according to the SIGBUS type.
>>
>> If guest accesses the poisoned memory, it generates Synchronous External
>> Abort(SEA). Then host kernel gets an APEI notification and call 
>> memory_failure()
>> to unmapped the affected page for the guest's stage 2, finally return
>> to guest.
>>
>> Guest continues to access PG_hwpoison page, it will trap to KVM as stage2 
>> fault,
>> then a SIGBUS_MCEERR_AR synchronous signal is delivered to Qemu, Qemu record 
>> this
>> error address into guest APEI GHES memory and notify guest using
>> Synchronous-External-Abort(SEA).
>>
>> Suggested-by: James Morse 
>> Signed-off-by: Dongjiu Geng 
>> ---
>>  hw/acpi/acpi_ghes.c | 177 
>> 
>>  include/hw/acpi/acpi_ghes.h |   6 +-
>>  include/sysemu/kvm.h|   2 +-
>>  target/arm/kvm64.c  |  39 ++
>>  4 files changed, 222 insertions(+), 2 deletions(-)
>>
>> diff --git a/hw/acpi/acpi_ghes.c b/hw/acpi/acpi_ghes.c
>> index d03e797..06b7374 100644
>> --- a/hw/acpi/acpi_ghes.c
>> +++ b/hw/acpi/acpi_ghes.c
>> @@ -26,6 +26,101 @@
>>  #include "sysemu/sysemu.h"
>>  #include "qemu/error-report.h"
>>  
>> +/* UEFI 2.6: N.2.5 Memory Error Section */
>> +static void build_append_mem_cper(GArray *table, uint64_t 
>> error_physical_addr)
>> +{
>> +/*
>> + * Memory Error Record
>> + */
>> +build_append_int_noprefix(table,
>> + (1UL << 14) | /* Type Valid */
>> + (1UL << 1) /* Physical Address Valid */,
>> + 8);
> bad indent
I will update it

> 
>> +/* Memory error status information */
>> +build_append_int_noprefix(table, 0, 8);
>> +/* The physical address at which the memory error occurred */
>> +build_append_int_noprefix(table, error_physical_addr, 8);
>> +build_append_int_noprefix(table, 0, 48);
>> +build_append_int_noprefix(table, 0 /* Unknown error */, 1);
>> +build_append_int_noprefix(table, 0, 7);
>> +}
>> +
>> +static int ghes_record_mem_error(uint64_t error_block_address,
>> +uint64_t error_physical_addr)
> bad indent
I will update it

> 
> 
>> +{
>> +GArray *block;
>> +uint64_t current_block_length;
>> +uint32_t data_length;
>> +/* Memory section */
>> +char mem_section_id_le[] = {0x14, 0x11, 0xBC, 0xA5, 0x64, 0x6F, 0xDE,
>> +0x4E, 0xB8, 0x63, 0x3E, 0x83, 0xED, 0x7C,
>> +0x83, 0xB1};
>> +uint8_t fru_id[16] = {0};
>> +uint8_t fru_text[20] = {0};
>> +
>> +/* Generic Error Status Block
>> + * | +-+
>> + * | | block_status|
>> + * | +-+
>> + * | |raw_data_offset  |
>> + * | +-+
>> + * | |raw_data_length  |
>> + * | +-+
>> + * | | data_length |
>> + * | +-+
>> + * | |   error_severity|
>> + * | +-+
>> + */
>> +block = g_array_new(false, true /* clear */, 1);
>> +
>> +/* Get the length of the Generic Error Data Entries */
>> +cpu_physical_memory_read(error_block_address +
>> +offsetof(AcpiGenericErrorStatus, data_length), _length, 4);
>> +/* The current whole length of the generic error status block */
>> +current_block_length = sizeof(AcpiGenericErrorStatus) + 
>> le32_to_cpu(data_length);
> I might be missing something but why do you read length from guest?
> Isn't it something provided by QEMU/host?
The length of the Generic Error Data Entries is not fixed, as the CPER number 
increases, the length will increase.
there is already a member to record the length for the CPER in the table, this 
table is in the guest.
so it is better directly read the length from the table instead of providing by 
QEMU/host.


> 
>> +
>> +/* This is the length if adding a new generic error data entry*/
>> +data_length += GHES_DATA_LENGTH;
>> +data_length += GHES_MEM_CPER_LENGTH;
>> +
>> +/* Check whether it will run out of the preallocated memory if adding a 
>> new
>> + * generic error data entry
>> + */
>> +if ((data_length + sizeof(AcpiGenericErrorStatus)) > 
>> GHES_MAX_RAW_DATA_LENGTH) {
>> +error_report("Record CPER out of boundary!!!");
>> +return GHES_CPER_FAIL;
>> +}
>> +
>> +/* Build the new generic error status block header */
>> +build_append_ghes_generic_status(block, 
>> cpu_to_le32(ACPI_GEBS_UNCORRECTABLE), 0, 0,
>> +cpu_to_le32(data_length), cpu_to_le32(ACPI_CPER_SEV_RECOVERABLE));
>> +
>> +/* Write back above generic error status block header to guest memory */
>> +

Re: [Qemu-devel] [PATCH v17 05/10] acpi: add build_append_ghes_generic_status() helper for Generic Error Status Block

[gengdongjiu]

On 2019/6/20 20:42, Igor Mammedov wrote:
> On Tue, 14 May 2019 04:18:18 -0700
> Dongjiu Geng  wrote:
> 
>> It will help to add Generic Error Status Block to ACPI tables
>> without using packed C structures and avoid endianness
>> issues as API doesn't need explicit conversion.
>>
>> Signed-off-by: Dongjiu Geng 
>> ---
>>  hw/acpi/aml-build.c | 14 ++
>>  include/hw/acpi/aml-build.h |  6 ++
>>  2 files changed, 20 insertions(+)
>>
>> diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
>> index 102a288..ce90970 100644
>> --- a/hw/acpi/aml-build.c
>> +++ b/hw/acpi/aml-build.c
>> @@ -296,6 +296,20 @@ void build_append_ghes_notify(GArray *table, const 
>> uint8_t type,
>>  build_append_int_noprefix(table, error_threshold_window, 4);
>>  }
>>  
>> +/* Generic Error Status Block
>> + * ACPI 4.0: 17.3.2.6.1 Generic Error Data
>> + */
>> +void build_append_ghes_generic_status(GArray *table, uint32_t block_status,
> maybe ..._generic_error_status???
good point, the build_append_ghes_generic_error_status() is better than 
build_append_ghes_generic_status()

> 
>> +  uint32_t raw_data_offset, uint32_t raw_data_length,
>> +  uint32_t data_length, uint32_t error_severity)
> see CODING_STYLE, 1.1 Multiline Indent
> 
>> +{
> when describing filds from spec try to add 'verbatim' copy of the name from 
> spec
> so it would be esy to grep for it. Like:
>/* Block Status */
>> +build_append_int_noprefix(table, block_status, 4);
>/* Raw Data Offset */
> 
> note applies all other places where you compose ACPI tables
ok

> 
>> +build_append_int_noprefix(table, raw_data_offset, 4);
>> +build_append_int_noprefix(table, raw_data_length, 4);
>> +build_append_int_noprefix(table, data_length, 4);
>> +build_append_int_noprefix(table, error_severity, 4);
>> +}
>> +
>>  /* Generic Error Data Entry
>>   * ACPI 4.0: 17.3.2.6.1 Generic Error Data
>>   */
>> diff --git a/include/hw/acpi/aml-build.h b/include/hw/acpi/aml-build.h
>> index a71db2f..1ec7e1b 100644
>> --- a/include/hw/acpi/aml-build.h
>> +++ b/include/hw/acpi/aml-build.h
>> @@ -425,6 +425,12 @@ void build_append_ghes_generic_data(GArray *table, 
>> const char *section_type,
>>  uint32_t error_data_length, uint8_t 
>> *fru_id,
>>  uint8_t *fru_text, uint64_t time_stamp);
>>  
>> +void
>> +build_append_ghes_generic_status(GArray *table, uint32_t block_status,
>> + uint32_t raw_data_offset,
>> + uint32_t raw_data_length,
>> + uint32_t data_length, uint32_t 
>> error_severity);
> this and previous patch, it might be better to to move declaration
> to its own header, for example to include/hw/acpi/acpi_ghes.h
> that you are adding later in the series.
> And maybe move helpers to hw/acpi/acpi_ghes.c
> They are not really independent ACPI primitives that are shared
> with other tables, aren't they?
Some ACPI primitives are shared with other table, such as Notification 
Structure.
we have 10 types of error sources, some error source will share the  
Notification Structure primitives.
Now I only implement Generic Hardware Error Source version 2 (GHESv2 - Type 10)

> .
>> +
>>  void build_srat_memory(AcpiSratMemoryAffinity *numamem, uint64_t base,
>> uint64_t len, int node, MemoryAffinityFlags flags);
>>  
> 
> .
> 

Re: [Qemu-devel] [PATCH v17 02/10] ACPI: add some GHES structures and macros definition

[gengdongjiu]

On 2019/6/24 19:16, Igor Mammedov wrote:
 On 2019/6/20 20:10, Igor Mammedov wrote:  
>> + */
>> +struct AcpiGenericErrorStatus {
>> +/* It is a bitmask composed of ACPI_GEBS_xxx macros */
>> +uint32_t block_status;
>> +uint32_t raw_data_offset;
>> +uint32_t raw_data_length;
>> +uint32_t data_length;
>> +uint32_t error_severity;
>> +} QEMU_PACKED;
>> +typedef struct AcpiGenericErrorStatus AcpiGenericErrorStatus;
> there shouldn't be packed structures,
> is it a leftover from previous version?
 I remember some people suggest to add QEMU_PACKED before, anyway I will 
 remove it in my next version patch.  
>>> Question is why it's  there and where it is used?  
>> sorry, it is my carelessness. it should be packed structures.
>>
>> I used this structures to get its actual total size and member offset in 
>> [PATCH v17 10/10].
>> If it is not packed structures, the total size and member offset may be not 
>> right.
> I'd suggest to drop these typedefs and use a macro with size for that purpose,
> Also it might be good to make it local to the file that would use it.
so you mean we also use macro for the  member offset  in the structures?  such 
as the offset of data_length,
may be there is many hardcode.

> 
>>> BTW:
>>> series doesn't apply to master anymore.

Re: [Qemu-devel] [PATCH v17 04/10] acpi: add build_append_ghes_generic_data() helper for Generic Error Data Entry

[gengdongjiu]

On 2019/6/20 20:28, Igor Mammedov wrote:
> On Tue, 14 May 2019 04:18:17 -0700
> Dongjiu Geng  wrote:
> 
>> It will help to add Generic Error Data Entry to ACPI tables
>> without using packed C structures and avoid endianness
>> issues as API doesn't need explicit conversion.
>>
>> Signed-off-by: Dongjiu Geng 
>> ---
>>  hw/acpi/aml-build.c | 32 
>>  include/hw/acpi/aml-build.h |  6 ++
>>  2 files changed, 38 insertions(+)
>>
>> diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
>> index fb53f21..102a288 100644
>> --- a/hw/acpi/aml-build.c
>> +++ b/hw/acpi/aml-build.c
>> @@ -296,6 +296,38 @@ void build_append_ghes_notify(GArray *table, const 
>> uint8_t type,
>>  build_append_int_noprefix(table, error_threshold_window, 4);
>>  }
>>  
>> +/* Generic Error Data Entry
>> + * ACPI 4.0: 17.3.2.6.1 Generic Error Data
>> + */
>> +void build_append_ghes_generic_data(GArray *table, const char *section_type,
> s/build_append_ghes_generic_data/build_append_ghes_generic_error_data/
> 
>> +uint32_t error_severity, uint16_t 
>> revision,
>> +uint8_t validation_bits, uint8_t flags,
>> +uint32_t error_data_length, uint8_t 
>> *fru_id,
>> +uint8_t *fru_text, uint64_t time_stamp)
> checkpatch probably will complain due to too long lines
> you can use:
> void build_append_ghe...
>  uint32_t error_severity, uint16_t revision,
>  ...
> 
>> +{
>> +int i;
>> +
>> +for (i = 0; i < 16; i++) {
>> +build_append_int_noprefix(table, section_type[i], 1);
> ^^^
> use QemuUUID instead, see vmgenid_build_acpi
ok.

> 
>> +}
>> +
>> +build_append_int_noprefix(table, error_severity, 4);
>> +build_append_int_noprefix(table, revision, 2);
>> +build_append_int_noprefix(table, validation_bits, 1);
>> +build_append_int_noprefix(table, flags, 1);
>> +build_append_int_noprefix(table, error_data_length, 4);
>> +
>> +for (i = 0; i < 16; i++) {
>> +build_append_int_noprefix(table, fru_id[i], 1);
> same as section_type
ok.

> 
>> +}
>> +
>> +for (i = 0; i < 20; i++) {
>> +build_append_int_noprefix(table, fru_text[i], 1);
>> +}
> instead of loop use g_array_insert_vals()
ok

> 
>> +
>> +build_append_int_noprefix(table, time_stamp, 8);
> that's not part of 'Table 17-13'
> where does it come from?


It comes from "ACPI 6.1: Table 18-343 Generic Error Data Entry", I will update 
the comments, thanks for the pointing out.

> 
>> +}
>> +
>>  /*
>>   * Build NAME(, 0x) where 0x is encoded as a dword,
>>   * and return the offset to 0x for runtime patching.
>> diff --git a/include/hw/acpi/aml-build.h b/include/hw/acpi/aml-build.h
>> index 90c8ef8..a71db2f 100644
>> --- a/include/hw/acpi/aml-build.h
>> +++ b/include/hw/acpi/aml-build.h
>> @@ -419,6 +419,12 @@ void build_append_ghes_notify(GArray *table, const 
>> uint8_t type,
>>uint32_t error_threshold_value,
>>uint32_t error_threshold_window);
>>  
>> +void build_append_ghes_generic_data(GArray *table, const char *section_type,
>> +uint32_t error_severity, uint16_t 
>> revision,
>> +uint8_t validation_bits, uint8_t flags,
>> +uint32_t error_data_length, uint8_t 
>> *fru_id,
>> +uint8_t *fru_text, uint64_t time_stamp);
>> +
>>  void build_srat_memory(AcpiSratMemoryAffinity *numamem, uint64_t base,
>> uint64_t len, int node, MemoryAffinityFlags flags);
>>  
> 
> .
> 

Re: [Qemu-devel] [PATCH v17 01/10] hw/arm/virt: Add RAS platform version for migration

[gengdongjiu]

On 2019/6/20 20:04, Igor Mammedov wrote:
> On Tue, 14 May 2019 04:18:14 -0700
> Dongjiu Geng  wrote:
> 
>> Support this feature since version 4.1, disable it by
>> default in the old version.
>>
>> Signed-off-by: Dongjiu Geng 
>> ---
>>  hw/arm/virt.c | 6 ++
>>  include/hw/arm/virt.h | 1 +
>>  2 files changed, 7 insertions(+)
>>
>> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
>> index 5331ab7..7bdd41b 100644
>> --- a/hw/arm/virt.c
>> +++ b/hw/arm/virt.c
>> @@ -2043,8 +2043,14 @@ DEFINE_VIRT_MACHINE_AS_LATEST(4, 1)
>>  
>>  static void virt_machine_4_0_options(MachineClass *mc)
>>  {
>> +VirtMachineClass *vmc = VIRT_MACHINE_CLASS(OBJECT_CLASS(mc));
>> +
>>  virt_machine_4_1_options(mc);
>>  compat_props_add(mc->compat_props, hw_compat_4_0, hw_compat_4_0_len);
>> +/* Disable memory recovery feature for 4.0 as RAS support was
>> + * introduced with 4.1.
>> + */
>> +vmc->no_ras = true;
> 
> So it would mean that the feature is enabled unconditionally for
> new machine types and consumes resources whether user needs it or not.
> 
> In light of the race for leaner QEMU and faster startup times,
> it might be better to make RAS optional and make user explicitly
> enable it using a machine option.

I will add a machine option to make RAS optional, do you think we should enable 
or disable it by default? I think it is better if we enable it by default.

> 
> 
>>  }
>>  DEFINE_VIRT_MACHINE(4, 0)
>>  
>> diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt.h
>> index 4240709..7f1a033 100644
>> --- a/include/hw/arm/virt.h
>> +++ b/include/hw/arm/virt.h
>> @@ -104,6 +104,7 @@ typedef struct {
>>  bool disallow_affinity_adjustment;
>>  bool no_its;
>>  bool no_pmu;
>> +bool no_ras;
>>  bool claim_edge_triggered_timers;
>>  bool smbios_old_sys_ver;
>>  bool no_highmem_ecam;
> 
> .
> 

Re: [Qemu-devel] [PATCH v17 02/10] ACPI: add some GHES structures and macros definition

[gengdongjiu]

On 2019/6/20 23:09, Igor Mammedov wrote:
> On Thu, 20 Jun 2019 22:04:01 +0800
> gengdongjiu  wrote:
> 
>> Hi Igor,
>>Thanks for your review.
>>
>> On 2019/6/20 20:10, Igor Mammedov wrote:
>>>> + */
>>>> +struct AcpiGenericErrorStatus {
>>>> +/* It is a bitmask composed of ACPI_GEBS_xxx macros */
>>>> +uint32_t block_status;
>>>> +uint32_t raw_data_offset;
>>>> +uint32_t raw_data_length;
>>>> +uint32_t data_length;
>>>> +uint32_t error_severity;
>>>> +} QEMU_PACKED;
>>>> +typedef struct AcpiGenericErrorStatus AcpiGenericErrorStatus;  
>>> there shouldn't be packed structures,
>>> is it a leftover from previous version?  
>>
>> I remember some people suggest to add QEMU_PACKED before, anyway I will 
>> remove it in my next version patch.
> 
> Question is why it's  there and where it is used?
sorry, it is my carelessness. it should be packed structures.

I used this structures to get its actual total size and member offset in [PATCH 
v17 10/10].
If it is not packed structures, the total size and member offset may be not 
right.


> 
> BTW:
> series doesn't apply to master anymore.
> Do you have a repo somewhere available for testing?

Thanks, I appreciated that you can have a test.

I still do not upload repo, you can reset to below commit[1] in master and 
apply this series.

BTW：
If test series, you should make an guest memory hardware error, let guest 
access the error address, then it will happen RAS error.
I provide a software hard code method to test this series, you can refer to 
https://www.mail-archive.com/qemu-devel@nongnu.org/msg619771.html


[1]:
commit efb4f3b62c69383a7308d7b739a3193e7c0ccae8
Merge: 5f02262 e841257
Author: Peter Maydell 
Date:   Fri May 10 14:49:36 2019 +0100



> 
>>
>>>   
>>>> +
>>>> +/*
>>>> + * Masks for block_status flags above
>>>> + */
>>>> +#define ACPI_GEBS_UNCORRECTABLE 1
>>>> +
>>>> +/*  
>>
> 
> .
> 

Re: [Qemu-devel] [PATCH v17 02/10] ACPI: add some GHES structures and macros definition

[gengdongjiu]

Hi Igor,
   Thanks for your review.

On 2019/6/20 20:10, Igor Mammedov wrote:
>> + */
>> +struct AcpiGenericErrorStatus {
>> +/* It is a bitmask composed of ACPI_GEBS_xxx macros */
>> +uint32_t block_status;
>> +uint32_t raw_data_offset;
>> +uint32_t raw_data_length;
>> +uint32_t data_length;
>> +uint32_t error_severity;
>> +} QEMU_PACKED;
>> +typedef struct AcpiGenericErrorStatus AcpiGenericErrorStatus;
> there shouldn't be packed structures,
> is it a leftover from previous version?

I remember some people suggest to add QEMU_PACKED before, anyway I will remove 
it in my next version patch.

> 
>> +
>> +/*
>> + * Masks for block_status flags above
>> + */
>> +#define ACPI_GEBS_UNCORRECTABLE 1
>> +
>> +/*

Re: [Qemu-devel] [PATCH v17 07/10] ACPI: Add APEI GHES table generation support

[gengdongjiu]

> 
> On Tue, 14 May 2019 04:18:20 -0700
> Dongjiu Geng  wrote:
> 
> > This implements APEI GHES Table generation via fw_cfg blobs.
> > Now it only support GPIO-Signal and ARMv8 SEA two types of GHESv2
> > error source. Afterwards, we can extend the supported types if needed.
> > For the CPER section type, currently it is memory section because
> > kernel mainly wants userspace to handle the memory errors.
> >
> > This patch follows the spec ACPI 6.2 to build the Hardware Error
> > Source table, for the detailed information, please refer to document:
> > docs/specs/acpi_hest_ghes.txt
> >
> > Suggested-by: Laszlo Ersek 
> > Signed-off-by: Dongjiu Geng 
> 
> Just one general question, why support the GPIO-Signal error source in this 
> patch set?  Unless I'm missing something, it's not actually
> used and would require some additional infrastructure (instantiate the GEDD 
> or similar).

In my Initial patches, for the asynchronous error, I want to use the 
GPIO-Signal to notify guest that there is asynchronous error happen. But later 
I found QEMU will mask the SIGBUS with BUS_MCEERR_AO.
SIGBUS with BUS_MCEERR_AO means the error is asynchronous error. So GPIO-Signal 
notification is not used since I do not handle BUS_MCEERR_AO in the sigbus 
handler.

> 
> Might be better to drop it for now and simplify this set a little bit?

Ok, I can drop the GPIO-Signal support.

> 
> Note I'm using the gpio path for CCIX PER error emulation so have it hooked 
> up.
> The code here is fine, just not used (I think).

By the way, you also use the SIGBUS with BUS_MCEERR_AO to report CCIX error to 
QEMU, I am afraid this signal is also masked by QEMU.

> 
> Thanks,
> 
> Jonathan
> 
> 
> 
> > ---
> >  default-configs/arm-softmmu.mak |   1 +
> >  hw/acpi/Kconfig |   4 +
> >  hw/acpi/Makefile.objs   |   1 +
> >  hw/acpi/acpi_ghes.c | 171 
> > 
> >  hw/acpi/aml-build.c |   2 +
> >  hw/arm/virt-acpi-build.c|  12 +++
> >  include/hw/acpi/acpi_ghes.h |  79 +++
> >  include/hw/acpi/aml-build.h |   1 +
> >  8 files changed, 271 insertions(+)
> >  create mode 100644 hw/acpi/acpi_ghes.c  create mode 100644
> > include/hw/acpi/acpi_ghes.h
> >
> > diff --git a/default-configs/arm-softmmu.mak
> > b/default-configs/arm-softmmu.mak index 613d19a..7b33ae9 100644
> > --- a/default-configs/arm-softmmu.mak
> > +++ b/default-configs/arm-softmmu.mak
> > @@ -160,3 +160,4 @@ CONFIG_MUSICPAL=y
> >
> >  # for realview and versatilepb
> >  CONFIG_LSI_SCSI_PCI=y
> > +CONFIG_ACPI_APEI=y
> > diff --git a/hw/acpi/Kconfig b/hw/acpi/Kconfig index eca3bee..5228a4b
> > 100644
> > --- a/hw/acpi/Kconfig
> > +++ b/hw/acpi/Kconfig
> > @@ -23,6 +23,10 @@ config ACPI_NVDIMM
> >  bool
> >  depends on ACPI
> >
> > +config ACPI_APEI
> > +bool
> > +depends on ACPI
> > +
> >  config ACPI_VMGENID
> >  bool
> >  default y
> > diff --git a/hw/acpi/Makefile.objs b/hw/acpi/Makefile.objs index
> > 2d46e37..5099ada 100644
> > --- a/hw/acpi/Makefile.objs
> > +++ b/hw/acpi/Makefile.objs
> > @@ -6,6 +6,7 @@ common-obj-$(CONFIG_ACPI_MEMORY_HOTPLUG) +=
> > memory_hotplug.o
> >  common-obj-$(CONFIG_ACPI_CPU_HOTPLUG) += cpu.o
> >  common-obj-$(CONFIG_ACPI_NVDIMM) += nvdimm.o
> >  common-obj-$(CONFIG_ACPI_VMGENID) += vmgenid.o
> > +common-obj-$(CONFIG_ACPI_APEI) += acpi_ghes.o
> >  common-obj-$(call lnot,$(CONFIG_ACPI_X86)) += acpi-stub.o
> >
> >  common-obj-y += acpi_interface.o
> > diff --git a/hw/acpi/acpi_ghes.c b/hw/acpi/acpi_ghes.c new file mode
> > 100644 index 000..d03e797
> > --- /dev/null
> > +++ b/hw/acpi/acpi_ghes.c
> > @@ -0,0 +1,171 @@
> > +/* Support for generating APEI tables and record CPER for Guests
> > + *
> > + * Copyright (C) 2017 HuaWei Corporation.
> > + *
> > + * Author: Dongjiu Geng 
> > + *
> > + * This program is free software; you can redistribute it and/or
> > +modify
> > + * it under the terms of the GNU General Public License as published
> > +by
> > + * the Free Software Foundation; either version 2 of the License, or
> > + * (at your option) any later version.
> > +
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > +
> > + * You should have received a copy of the GNU General Public License
> > + along
> > + * with this program; if not, see .
> > + */
> > +
> > +#include "qemu/osdep.h"
> > +#include "hw/acpi/acpi.h"
> > +#include "hw/acpi/aml-build.h"
> > +#include "hw/acpi/acpi_ghes.h"
> > +#include "hw/nvram/fw_cfg.h"
> > +#include "sysemu/sysemu.h"
> > +#include "qemu/error-report.h"
> > +
> > +/* Build table for the hardware error fw_cfg blob */ void
> > +build_hardware_error_table(GArray *hardware_errors, BIOSLinker
> > +*linker) {
> > +int i;
> > +
> > 

Re: [Qemu-devel] [PATCH v17 10/10] target-arm: kvm64: handle SIGBUS signal from kernel or KVM

[gengdongjiu]

> 
> On Tue, 14 May 2019 04:18:23 -0700
> Dongjiu Geng  wrote:
> 
> > Add SIGBUS signal handler. In this handler, it checks the SIGBUS type,
> > translates the host VA delivered by host to guest PA, then fill this
> > PA to guest APEI GHES memory, then notify guest according to the SIGBUS 
> > type.
> >
> > If guest accesses the poisoned memory, it generates Synchronous
> > External Abort(SEA). Then host kernel gets an APEI notification and
> > call memory_failure() to unmapped the affected page for the guest's
> > stage 2, finally return to guest.
> >
> > Guest continues to access PG_hwpoison page, it will trap to KVM as
> > stage2 fault, then a SIGBUS_MCEERR_AR synchronous signal is delivered
> > to Qemu, Qemu record this error address into guest APEI GHES memory
> > and notify guest using Synchronous-External-Abort(SEA).
> >
> > Suggested-by: James Morse 
> > Signed-off-by: Dongjiu Geng 
> Hi Dongjiu,
> 
> Good to see this moving forwards again.
> 
> A few really minor things inline.

Jonathan,
   It is good to see your comments, thanks for the review.

> 
> Thanks,
> 
> Jonathan
> 
> > ---
> >  hw/acpi/acpi_ghes.c | 177 
> > 
> >  include/hw/acpi/acpi_ghes.h |   6 +-
> >  include/sysemu/kvm.h|   2 +-
> >  target/arm/kvm64.c  |  39 ++
> >  4 files changed, 222 insertions(+), 2 deletions(-)
> >
> > diff --git a/hw/acpi/acpi_ghes.c b/hw/acpi/acpi_ghes.c index
> > d03e797..06b7374 100644
> > --- a/hw/acpi/acpi_ghes.c
> > +++ b/hw/acpi/acpi_ghes.c
> > @@ -26,6 +26,101 @@
> >  #include "sysemu/sysemu.h"
> >  #include "qemu/error-report.h"
> >
> > +/* UEFI 2.6: N.2.5 Memory Error Section */ static void
> > +build_append_mem_cper(GArray *table, uint64_t error_physical_addr) {
> > +/*
> > + * Memory Error Record
> > + */
> > +build_append_int_noprefix(table,
> > + (1UL << 14) | /* Type Valid */
> > + (1UL << 1) /* Physical Address Valid */,
> > + 8);
> > +/* Memory error status information */
> > +build_append_int_noprefix(table, 0, 8);
> > +/* The physical address at which the memory error occurred */
> > +build_append_int_noprefix(table, error_physical_addr, 8);
> > +build_append_int_noprefix(table, 0, 48);
> 
> This could do with a comment to say we are basically skipping all the 
> detailed information normally found in such a record.

Ok, it is good to have such a comment, I will add it.

> 
> 
> > +build_append_int_noprefix(table, 0 /* Unknown error */, 1);
> > +build_append_int_noprefix(table, 0, 7);
> A similar comment for this last section would probably use useful as well.

Ok

> 
> > +}
> > +
> > +static int ghes_record_mem_error(uint64_t error_block_address,
> > +uint64_t error_physical_addr) {
> > +GArray *block;
> > +uint64_t current_block_length;
> > +uint32_t data_length;
> > +/* Memory section */
> The variable name is clear I think, so not sure this comment adds any 
> information.

Ok, I will remove this comment since the variable name can tell the meaning.

> 
> > +char mem_section_id_le[] = {0x14, 0x11, 0xBC, 0xA5, 0x64, 0x6F, 0xDE,
> > +0x4E, 0xB8, 0x63, 0x3E, 0x83, 0xED, 0x7C,
> > +0x83, 0xB1};
> > +uint8_t fru_id[16] = {0};
> > +uint8_t fru_text[20] = {0};
> > +
> > +/* Generic Error Status Block
> > + * | +-+
> > + * | | block_status|
> > + * | +-+
> > + * | |raw_data_offset  |
> > + * | +-+
> > + * | |raw_data_length  |
> > + * | +-+
> > + * | | data_length |
> > + * | +-+
> > + * | |   error_severity|
> > + * | +-+
> > + */
> > +block = g_array_new(false, true /* clear */, 1);
> > +
> > +/* Get the length of the Generic Error Data Entries */
> > +cpu_physical_memory_read(error_block_address +
> > +offsetof(AcpiGenericErrorStatus, data_length), _length,
> > + 4);
> > +
> > +/* The current whole length of the generic error status block */
> > +current_block_length = sizeof(AcpiGenericErrorStatus) +
> > + le32_to_cpu(data_length);
> > +
> > +/* This is the length if adding a new generic error data entry*/
> > +data_length += GHES_DATA_LENGTH;
> > +data_length += GHES_MEM_CPER_LENGTH;
> > +
> > +/* Check whether it will run out of the preallocated memory if adding 
> > a new
> > + * generic error data entry
> > + */
> > +if ((data_length + sizeof(AcpiGenericErrorStatus)) > 
> > GHES_MAX_RAW_DATA_LENGTH) {
> > +error_report("Record CPER out of boundary!!!");
> > +return GHES_CPER_FAIL;
> > +}
> > +
> > +/* Build the new generic error status block header */
> > +build_append_ghes_generic_status(block, 
> > 

Re: [Qemu-devel] [PATCH v17 02/10] ACPI: add some GHES structures and macros definition

[gengdongjiu]

On 2019/5/29 11:40, Michael S. Tsirkin wrote:
> On Tue, May 14, 2019 at 04:18:15AM -0700, Dongjiu Geng wrote:
>> Add Generic Error Status Block structures and some macros
>> definitions, which is referred to the ACPI 4.0 or ACPI 6.2. The
>> HEST table generation and CPER record will use them.
>>
>> Signed-off-by: Dongjiu Geng 
> 
> Are these all still used? I'd rather you moved stuff to
> where it's used.
Ok, I will move them, thanks

> 
> 
> 
>> ---
>>  include/hw/acpi/acpi-defs.h | 52 
>> +
>>  1 file changed, 52 insertions(+)
>>
>> diff --git a/include/hw/acpi/acpi-defs.h b/include/hw/acpi/acpi-defs.h
>> index f9aa4bd..d1996fb 100644
>> --- a/include/hw/acpi/acpi-defs.h
>> +++ b/include/hw/acpi/acpi-defs.h
>> @@ -224,6 +224,25 @@ typedef struct AcpiMultipleApicTable 
>> AcpiMultipleApicTable;
>>  #define ACPI_APIC_RESERVED  16   /* 16 and greater are reserved 
>> */
>>  
>>  /*
>> + * Values for Hardware Error Notification Type field
>> + */
>> +enum AcpiHestNotifyType {
>> +ACPI_HEST_NOTIFY_POLLED = 0,
>> +ACPI_HEST_NOTIFY_EXTERNAL = 1,
>> +ACPI_HEST_NOTIFY_LOCAL = 2,
>> +ACPI_HEST_NOTIFY_SCI = 3,
>> +ACPI_HEST_NOTIFY_NMI = 4,
>> +ACPI_HEST_NOTIFY_CMCI = 5,  /* ACPI 5.0: 18.3.2.7, Table 18-290 */
>> +ACPI_HEST_NOTIFY_MCE = 6,   /* ACPI 5.0: 18.3.2.7, Table 18-290 */
>> +ACPI_HEST_NOTIFY_GPIO = 7,  /* ACPI 6.0: 18.3.2.7, Table 18-332 */
>> +ACPI_HEST_NOTIFY_SEA = 8,   /* ACPI 6.1: 18.3.2.9, Table 18-345 */
>> +ACPI_HEST_NOTIFY_SEI = 9,   /* ACPI 6.1: 18.3.2.9, Table 18-345 */
>> +ACPI_HEST_NOTIFY_GSIV = 10, /* ACPI 6.1: 18.3.2.9, Table 18-345 */
>> +ACPI_HEST_NOTIFY_SDEI = 11, /* ACPI 6.2: 18.3.2.9, Table 18-383 */
>> +ACPI_HEST_NOTIFY_RESERVED = 12 /* 12 and greater are reserved */
>> +};
>> +
> 
> If there's a single user, the best thing to do
> is just to open-code with a comment that matches
> spec names. It's hard to find stuff like ACPI_HEST_NOTIFY_GSIV
> in a spec.
ok, I will do it, thanks

> 
>> +/*
>>   * MADT sub-structures (Follow MULTIPLE_APIC_DESCRIPTION_TABLE)
>>   */
>>  #define ACPI_SUB_HEADER_DEF   /* Common ACPI sub-structure header */\
>> @@ -400,6 +419,39 @@ struct AcpiSystemResourceAffinityTable {
>>  } QEMU_PACKED;
>>  typedef struct AcpiSystemResourceAffinityTable 
>> AcpiSystemResourceAffinityTable;
>>  
>> +/*
>> + * Generic Error Status Block
>> + */
>> +struct AcpiGenericErrorStatus {
>> +/* It is a bitmask composed of ACPI_GEBS_xxx macros */
>> +uint32_t block_status;
>> +uint32_t raw_data_offset;
>> +uint32_t raw_data_length;
>> +uint32_t data_length;
>> +uint32_t error_severity;
>> +} QEMU_PACKED;
>> +typedef struct AcpiGenericErrorStatus AcpiGenericErrorStatus;
>> +
>> +/*
>> + * Masks for block_status flags above
>> + */
>> +#define ACPI_GEBS_UNCORRECTABLE 1
>> +
>> +/*
>> + * Values for error_severity field above
>> + */
>> +enum AcpiGenericErrorSeverity {
>> +ACPI_CPER_SEV_RECOVERABLE,
>> +ACPI_CPER_SEV_FATAL,
>> +ACPI_CPER_SEV_CORRECTED,
>> +ACPI_CPER_SEV_NONE,
>> +};
>> +
>> +/*
>> + * Generic Hardware Error Source version 2
>> + */
>> +#define ACPI_HEST_SOURCE_GENERIC_ERROR_V210
>> +
>>  #define ACPI_SRAT_PROCESSOR_APIC 0
>>  #define ACPI_SRAT_MEMORY 1
>>  #define ACPI_SRAT_PROCESSOR_x2APIC   2
>> -- 
>> 1.8.3.1
> .
> 

Re: [Qemu-devel] [PATCH v17 07/10] ACPI: Add APEI GHES table generation support

[gengdongjiu]

Hi Michael,
   Thanks for the review.

On 2019/5/29 11:37, Michael S. Tsirkin wrote:
> On Tue, May 14, 2019 at 04:18:20AM -0700, Dongjiu Geng wrote:
>> This implements APEI GHES Table generation via fw_cfg blobs.
>> Now it only support GPIO-Signal and ARMv8 SEA two types of GHESv2 error
>> source. Afterwards, we can extend the supported types if needed. For the
>> CPER section type, currently it is memory section because kernel
>> mainly wants userspace to handle the memory errors.
>>
>> This patch follows the spec ACPI 6.2 to build the Hardware Error Source
>> table, for the detailed information, please refer to document:
>> docs/specs/acpi_hest_ghes.txt
>>
>> Suggested-by: Laszlo Ersek 
>> Signed-off-by: Dongjiu Geng 
>> ---
>>  default-configs/arm-softmmu.mak |   1 +
>>  hw/acpi/Kconfig |   4 +
>>  hw/acpi/Makefile.objs   |   1 +
>>  hw/acpi/acpi_ghes.c | 171 
>> 
>>  hw/acpi/aml-build.c |   2 +
>>  hw/arm/virt-acpi-build.c|  12 +++
>>  include/hw/acpi/acpi_ghes.h |  79 +++
>>  include/hw/acpi/aml-build.h |   1 +
>>  8 files changed, 271 insertions(+)
>>  create mode 100644 hw/acpi/acpi_ghes.c
>>  create mode 100644 include/hw/acpi/acpi_ghes.h
>>
>> diff --git a/default-configs/arm-softmmu.mak 
>> b/default-configs/arm-softmmu.mak
>> index 613d19a..7b33ae9 100644
>> --- a/default-configs/arm-softmmu.mak
>> +++ b/default-configs/arm-softmmu.mak
>> @@ -160,3 +160,4 @@ CONFIG_MUSICPAL=y
>>  
>>  # for realview and versatilepb
>>  CONFIG_LSI_SCSI_PCI=y
>> +CONFIG_ACPI_APEI=y
>> diff --git a/hw/acpi/Kconfig b/hw/acpi/Kconfig
>> index eca3bee..5228a4b 100644
>> --- a/hw/acpi/Kconfig
>> +++ b/hw/acpi/Kconfig
>> @@ -23,6 +23,10 @@ config ACPI_NVDIMM
>>  bool
>>  depends on ACPI
>>  
>> +config ACPI_APEI
>> +bool
>> +depends on ACPI
>> +
>>  config ACPI_VMGENID
>>  bool
>>  default y
>> diff --git a/hw/acpi/Makefile.objs b/hw/acpi/Makefile.objs
>> index 2d46e37..5099ada 100644
>> --- a/hw/acpi/Makefile.objs
>> +++ b/hw/acpi/Makefile.objs
>> @@ -6,6 +6,7 @@ common-obj-$(CONFIG_ACPI_MEMORY_HOTPLUG) += memory_hotplug.o
>>  common-obj-$(CONFIG_ACPI_CPU_HOTPLUG) += cpu.o
>>  common-obj-$(CONFIG_ACPI_NVDIMM) += nvdimm.o
>>  common-obj-$(CONFIG_ACPI_VMGENID) += vmgenid.o
>> +common-obj-$(CONFIG_ACPI_APEI) += acpi_ghes.o
>>  common-obj-$(call lnot,$(CONFIG_ACPI_X86)) += acpi-stub.o
>>  
>>  common-obj-y += acpi_interface.o
>> diff --git a/hw/acpi/acpi_ghes.c b/hw/acpi/acpi_ghes.c
>> new file mode 100644
>> index 000..d03e797
>> --- /dev/null
>> +++ b/hw/acpi/acpi_ghes.c
>> @@ -0,0 +1,171 @@
>> +/* Support for generating APEI tables and record CPER for Guests
>> + *
>> + * Copyright (C) 2017 HuaWei Corporation.
>> + *
>> + * Author: Dongjiu Geng 
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License as published by
>> + * the Free Software Foundation; either version 2 of the License, or
>> + * (at your option) any later version.
>> +
>> + * This program is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> + * GNU General Public License for more details.
>> +
>> + * You should have received a copy of the GNU General Public License along
>> + * with this program; if not, see .
>> + */
>> +
>> +#include "qemu/osdep.h"
>> +#include "hw/acpi/acpi.h"
>> +#include "hw/acpi/aml-build.h"
>> +#include "hw/acpi/acpi_ghes.h"
>> +#include "hw/nvram/fw_cfg.h"
>> +#include "sysemu/sysemu.h"
>> +#include "qemu/error-report.h"
>> +
>> +/* Build table for the hardware error fw_cfg blob */
>> +void build_hardware_error_table(GArray *hardware_errors, BIOSLinker *linker)
>> +{
>> +int i;
>> +
>> +/*
>> + * | +--+
>> + * | |error_block_address   |
>> + * | |  ..  |
>> + * | +--+
>> + * | |read_ack_register |
>> + * | | ...  |
>> + * | +--+
>> + * | |  Error Status Data Block |
>> + * | |  |
>> + * | +--+
>> + */
>> +
>> +/* Build error_block_address */
>> +build_append_int_noprefix((void *)hardware_errors, 0,
>> +GHES_ADDRESS_SIZE * ACPI_HEST_ERROR_SOURCE_COUNT);
> 
> Why are you casting everything to void *?
> 
> C type safety goes out the window ...
  I will remove the "void *" transform.

> 
>> +
>> +/* Build read_ack_register */
>> +for (i = 0; i < ACPI_HEST_ERROR_SOURCE_COUNT; i++)
>> +/* Initialize the value of read_ack_register to 1, so GHES can be
>> + * writeable in the first time
>> + */
>> +build_append_int_noprefix((void *)hardware_errors, 1, 

Re: [Qemu-devel] [PATCH v17 00/10] Add ARMv8 RAS virtualization support in QEMU

[gengdongjiu]

ping...

发件人：gengdongjiu 
收件人：pbonz...@redhat.com ;m...@redhat.com 
;imamm...@redhat.com 
;shannon.zha...@gmail.com 
;peter.mayd...@linaro.org 
;ler...@redhat.com 
;james.mo...@arm.com 
;mtosa...@redhat.com 
;r...@twiddle.net ;ehabk...@redhat.com 
;zhengxiang (A) ;Jonathan Cameron 
;xuwei (O) 
;k...@vger.kernel.org 
;qemu-devel@nongnu.org 
;qemu-...@nongnu.org ;Linuxarm 

时间：2019-05-15 17:40:49
主 题：Re: [PATCH v17 00/10] Add ARMv8 RAS virtualization support in QEMU

Hi All,
 for this series patch, we can use below simple method to test:

1). Apply below hard code change after applying this series patch:
diff --git a/cpus.c b/cpus.c
index e58e7ab..7149f54 100644
--- a/cpus.c
+++ b/cpus.c
@@ -1131,6 +1131,8 @@ static void sigbus_reraise(void)

 static void sigbus_handler(int n, siginfo_t *siginfo, void *ctx)
 {
+siginfo->si_code = BUS_MCEERR_AO;
+
 if (siginfo->si_code != BUS_MCEERR_AO && siginfo->si_code != 
BUS_MCEERR_AR) {
 sigbus_reraise();
 }
diff --git a/target/arm/kvm64.c b/target/arm/kvm64.c
index d2eac28..6c9956e 100644
--- a/target/arm/kvm64.c
+++ b/target/arm/kvm64.c
@@ -1035,20 +1035,23 @@ int kvm_arch_get_registers(CPUState *cs)

 void kvm_arch_on_sigbus_vcpu(CPUState *c, int code, void *addr)
 {
+#if 0
 ram_addr_t ram_addr;
-hwaddr paddr;
-
+#endif
+hwaddr paddr = 0x400a1000;
 assert(code == BUS_MCEERR_AR || code == BUS_MCEERR_AO);

+#if 0
 if (acpi_enabled && addr) {
 ram_addr = qemu_ram_addr_from_host(addr);
 if (ram_addr != RAM_ADDR_INVALID &&
 kvm_physical_memory_addr_from_host(c->kvm_state, addr, )) {
 kvm_hwpoison_page_add(ram_addr);
+#endif
 /* Asynchronous signal will be masked by main thread, so
  * only handle synchronous signal.
  */
-if (code == BUS_MCEERR_AR) {
+if (code == BUS_MCEERR_AR || BUS_MCEERR_AO == code) {
 kvm_cpu_synchronize_state(c);
 if (GHES_CPER_FAIL != ghes_record_errors(ACPI_HEST_NOTIFY_SEA, 
paddr)) {
 kvm_inject_arm_sea(c);
@@ -1057,11 +1060,13 @@ void kvm_arch_on_sigbus_vcpu(CPUState *c, int code, 
void *addr)
 }
 }
 return;
+#if 0
 }
 fprintf(stderr, "Hardware memory error for memory used by "
 "QEMU itself instead of guest system!\n");
 }

+#endif
 if (code == BUS_MCEERR_AR) {
 fprintf(stderr, "Hardware memory error!\n");
 exit(1);

2) In Arm64 machine run below command to run a virtual machine with kvm 
disabled, and enable bios:
./qemu-system-aarch64 -cpu cortex-a57 -machine virt,gic-version=3  -smp 4 
-nographic -kernel Image -bios QEMU_EFI.fd -append "rdinit=/init 
console=ttyAMA0 mem=512M root=/dev/ram0 earlycon=pl011,0x900 rw" -initrd 
guestfs_new.cpio.gz

3) Deliver SGIBUS signal to QEMU:
   kill -s SIGBUS 11522

4) then you can see QEMU record the error memory address to APEI table, and 
inject a SEA to guest, for example:

[   75.514270] {1}[Hardware Error]: Hardware error from APEI Generic Hardware 
Error Source: 1
[   75.531858] {1}[Hardware Error]: event severity: recoverable
[   75.535095] {1}[Hardware Error]:  Error 0, type: recoverable
[   75.539223] {1}[Hardware Error]:   section_type: memory error
[   75.543878] {1}[Hardware Error]:   physical_address: 0x400a1000
[   75.548696] {1}[Hardware Error]:   error_type: 0, unknown
[   75.577156] Internal error: synchronous external abort: 96000410 [#1] 
PREEMPT SMP

On 2019/5/14 19:18, Dongjiu Geng wrote:
> In the ARMv8 platform, the CPU error type are synchronous external
> abort(SEA) and SError Interrupt (SEI). If exception happens to guest,
> sometimes guest itself do the recovery is better, because host
> does not know guest's detailed information. For example, if a guest
> user-space application happen exception, host does not which application
> encounter errors.
>
> For the ARMv8 SEA/SEI, KVM or host kernel delivers SIGBUS to notify user
> space. After user space gets  the notification, it will record the CPER
> to guest GHES buffer for guest and inject a exception or IRQ to guest.
>
> In the current implement, if the SIGBUS is BUS_MCEERR_AR, we will
> treat it as synchronous exception, and use ARMv8 SEA notification type
> to notify guest after recording CPER for guest;
>
> This series patches are based on Qemu 4.0, which have two parts:
> 1. Generate APEI/GHES table.
> 2. Handle the SIGBUS signal, record the CPER in runtime and fill into guest 
> memory,
>then according to SIGBUS type to notify guest.
>
> Whole solution was suggested by James(james.mo...@arm.com); APEI part 
> solution is suggested by
> Laszlo(ler...@redhat.com). Shown some discussion in [1].
>
>
> This series patches have already tested on ARM64 platform with RAS featu

Re: [Qemu-devel] [PATCH v17 00/10] Add ARMv8 RAS virtualization support in QEMU

[gengdongjiu]

Hi All,
 for this series patch, we can use below simple method to test:

1). Apply below hard code change after applying this series patch:
diff --git a/cpus.c b/cpus.c
index e58e7ab..7149f54 100644
--- a/cpus.c
+++ b/cpus.c
@@ -1131,6 +1131,8 @@ static void sigbus_reraise(void)

 static void sigbus_handler(int n, siginfo_t *siginfo, void *ctx)
 {
+siginfo->si_code = BUS_MCEERR_AO;
+
 if (siginfo->si_code != BUS_MCEERR_AO && siginfo->si_code != 
BUS_MCEERR_AR) {
 sigbus_reraise();
 }
diff --git a/target/arm/kvm64.c b/target/arm/kvm64.c
index d2eac28..6c9956e 100644
--- a/target/arm/kvm64.c
+++ b/target/arm/kvm64.c
@@ -1035,20 +1035,23 @@ int kvm_arch_get_registers(CPUState *cs)

 void kvm_arch_on_sigbus_vcpu(CPUState *c, int code, void *addr)
 {
+#if 0
 ram_addr_t ram_addr;
-hwaddr paddr;
-
+#endif
+hwaddr paddr = 0x400a1000;
 assert(code == BUS_MCEERR_AR || code == BUS_MCEERR_AO);

+#if 0
 if (acpi_enabled && addr) {
 ram_addr = qemu_ram_addr_from_host(addr);
 if (ram_addr != RAM_ADDR_INVALID &&
 kvm_physical_memory_addr_from_host(c->kvm_state, addr, )) {
 kvm_hwpoison_page_add(ram_addr);
+#endif
 /* Asynchronous signal will be masked by main thread, so
  * only handle synchronous signal.
  */
-if (code == BUS_MCEERR_AR) {
+if (code == BUS_MCEERR_AR || BUS_MCEERR_AO == code) {
 kvm_cpu_synchronize_state(c);
 if (GHES_CPER_FAIL != ghes_record_errors(ACPI_HEST_NOTIFY_SEA, 
paddr)) {
 kvm_inject_arm_sea(c);
@@ -1057,11 +1060,13 @@ void kvm_arch_on_sigbus_vcpu(CPUState *c, int code, 
void *addr)
 }
 }
 return;
+#if 0
 }
 fprintf(stderr, "Hardware memory error for memory used by "
 "QEMU itself instead of guest system!\n");
 }

+#endif
 if (code == BUS_MCEERR_AR) {
 fprintf(stderr, "Hardware memory error!\n");
 exit(1);

2) In Arm64 machine run below command to run a virtual machine with kvm 
disabled, and enable bios:
./qemu-system-aarch64 -cpu cortex-a57 -machine virt,gic-version=3  -smp 4 
-nographic -kernel Image -bios QEMU_EFI.fd -append "rdinit=/init 
console=ttyAMA0 mem=512M root=/dev/ram0 earlycon=pl011,0x900 rw" -initrd 
guestfs_new.cpio.gz

3) Deliver SGIBUS signal to QEMU:
   kill -s SIGBUS 11522

4) then you can see QEMU record the error memory address to APEI table, and 
inject a SEA to guest, for example:

[   75.514270] {1}[Hardware Error]: Hardware error from APEI Generic Hardware 
Error Source: 1
[   75.531858] {1}[Hardware Error]: event severity: recoverable
[   75.535095] {1}[Hardware Error]:  Error 0, type: recoverable
[   75.539223] {1}[Hardware Error]:   section_type: memory error
[   75.543878] {1}[Hardware Error]:   physical_address: 0x400a1000
[   75.548696] {1}[Hardware Error]:   error_type: 0, unknown
[   75.577156] Internal error: synchronous external abort: 96000410 [#1] 
PREEMPT SMP

On 2019/5/14 19:18, Dongjiu Geng wrote:
> In the ARMv8 platform, the CPU error type are synchronous external
> abort(SEA) and SError Interrupt (SEI). If exception happens to guest,
> sometimes guest itself do the recovery is better, because host 
> does not know guest's detailed information. For example, if a guest
> user-space application happen exception, host does not which application
> encounter errors.
> 
> For the ARMv8 SEA/SEI, KVM or host kernel delivers SIGBUS to notify user
> space. After user space gets  the notification, it will record the CPER
> to guest GHES buffer for guest and inject a exception or IRQ to guest.
> 
> In the current implement, if the SIGBUS is BUS_MCEERR_AR, we will
> treat it as synchronous exception, and use ARMv8 SEA notification type
> to notify guest after recording CPER for guest;
> 
> This series patches are based on Qemu 4.0, which have two parts:
> 1. Generate APEI/GHES table.
> 2. Handle the SIGBUS signal, record the CPER in runtime and fill into guest 
> memory,
>then according to SIGBUS type to notify guest.
> 
> Whole solution was suggested by James(james.mo...@arm.com); APEI part 
> solution is suggested by
> Laszlo(ler...@redhat.com). Shown some discussion in [1].
> 
> 
> This series patches have already tested on ARM64 platform with RAS feature 
> enabled:
> Show the APEI part verification result in [2]
> Show the BUS_MCEERR_AR SIGBUS handling verification result in [3]
> 
> ---
> change since v16:
> 1. check whether ACPI table is enabled when handling the memory error in the 
> SIGBUS handler.
> 
> Change since v15:
> 1. Add a doc-comment in the proper format for 'include/exec/ram_addr.h'
> 2. Remove write_part_cpustate_to_list() because there is another bug fix patch
>has been merged "arm: Allow system registers for KVM guests to be changed 
> by QEMU code"
> 3. Add some comments for kvm_inject_arm_sea() in 'target/arm/kvm64.c'
> 4. Compare the 

Re: [Qemu-devel] [PATCH v16 10/10] target-arm: kvm64: handle SIGBUS signal from kernel or KVM

[gengdongjiu]

> 
>> +void kvm_arch_on_sigbus_vcpu(CPUState *c, int code, void *addr)
>> +{
>> +ARMCPU *cpu = ARM_CPU(c);
>> +CPUARMState *env = >env;
>> +ram_addr_t ram_addr;
>> +hwaddr paddr;
>> +
>> +assert(code == BUS_MCEERR_AR || code == BUS_MCEERR_AO);
>> +
>> +if (addr) {
>> +ram_addr = qemu_ram_addr_from_host(addr);
>> +if (ram_addr != RAM_ADDR_INVALID &&
>> +kvm_physical_memory_addr_from_host(c->kvm_state, addr, )) 
>> {
>> +kvm_hwpoison_page_add(ram_addr);
>> +/* Asynchronous signal will be masked by main thread, so
>> + * only handle synchronous signal.
>> + */
>> +if (code == BUS_MCEERR_AR) {
>> +kvm_cpu_synchronize_state(c);
>> +if (GHES_CPER_FAIL != 
>> ghes_record_errors(ACPI_HEST_NOTIFY_SEA, paddr)) {
>> +kvm_inject_arm_sea(c);
>> +} else {
>> +fprintf(stderr, "failed to record the error\n");
>> +}
>> +}
>> +return;
>> +}
>> +fprintf(stderr, "Hardware memory error for memory used by "
>> +"QEMU itself instead of guest system!\n");
>> +}
>> +
>> +if (code == BUS_MCEERR_AR) {
>> +fprintf(stderr, "Hardware memory error!\n");
>> +exit(1);
>> +}
>> +}
> 
> This code appears to still be unconditionally trying to
> notify the guest of the error via the ACPI tables without
> checking whether those ACPI tables even exist. I told you
> about this in a previous round of review :-(

Thanks very much for the comments, and sorry for my forgetting
I added the ACPI checking in the new V17 version.

> 
> thanks
> -- PMM
> .
> 

Re: [Qemu-devel] [PATCH for-4.0?] arm: Allow system registers for KVM guests to be changed by QEMU code

[gengdongjiu]

On 2019/3/25 18:25, Peter Maydell wrote:
> Further testing from Alex suggests this is some unrelated
> bug or regression (ie not caused by this patch), but:
> since the only in-tree use for this patch is to get nested
> debugging working and it would be broken for this other
> reason even with this patch, I'm going to postpone applying
> this patch until the start of the 4.1 cycle.
OK, thanks Peter's following.

Re: [Qemu-devel] [PATCH for-4.0?] arm: Allow system registers for KVM guests to be changed by QEMU code

[gengdongjiu]

On 2019/3/18 20:49, Peter Maydell wrote:
> On Mon, 18 Mar 2019 at 12:34, gengdongjiu  wrote:
>>
>>
>>
>> On 2019/3/16 4:11, Philippe Mathieu-Daudé wrote:
>>>> Signed-off-by: Peter Maydell 
>>>> ---
>>>> Should we try to put this in for rc1? Not sure... Testing
>>>> definitely appreciated.
>>> You might include it for rc1 and we still have rc2/rc3 to revert it.
>>
>> why we still have rc2/rc3 to revert it?
> 
> I think Philippe's point is that it's reasonably safe to
> apply this patch in rc1 (ie now), because if we do do that
> and then discover that we have some other bug in it, we
> still have plenty of time to take the patch out again
> before release.
> 
> (If we did discover another bug in this patch in future, I
> would favour reverting it for the 4.0 release rather than
> trying to fix whatever that bug was, because two unexpected
> bugs in the patch means I clearly didn't understand
> the code well enough to produce a reliable patch. The cases
> that this patch fixes are pretty rare -- it does fix a bug
> but only in handling of some cases of debugging a KVM guest;
> but the patch potentially affects the behaviour of any
> KVM guest, even if the user isn't trying to debug. So it's
> riskier than some other kinds of change, and on balance
> that means that when we're making a decision at rc2 or rc3
> I'm more in favour of just reverting it rather than
> applying what we hope is a fix.)

Got it, understand it now, thanks very much for Peter's detailed explanation.

> 
> thanks
> -- PMM
> 
> .
> 

Re: [Qemu-devel] [PATCH for-4.0?] arm: Allow system registers for KVM guests to be changed by QEMU code

[gengdongjiu]

On 2019/3/16 4:11, Philippe Mathieu-Daudé wrote:
>> Signed-off-by: Peter Maydell 
>> ---
>> Should we try to put this in for rc1? Not sure... Testing
>> definitely appreciated.
> You might include it for rc1 and we still have rc2/rc3 to revert it.

why we still have rc2/rc3 to revert it?
If we can find the root cause about the regression and fix it, we can apply it.
so we may need to debug this issue.

> 
>> ---
>>  target/arm/cpu.h |  9 -

Re: [Qemu-devel] About Revert this patch: arm: Allow system registers for KVM guests to be changed by QEMU code

[gengdongjiu]

On 2019/3/14 20:43, Peter Maydell wrote:
>> If you think this patch will introduce some issue, we can add one function 
>> write_part_cpustate_to_list()[2] to change the specified
>> register instead of all the registers[1].
> It definitely does introduce an issue, but we need to address
> it by working out why it breaks something that I wasn't
> expecting to break. Then we can fix it properly. Looking
> it this is on my todo list for this week.
Got it, great, thanks very much to Peter.

Re: [Qemu-devel] About Revert this patch: arm: Allow system registers for KVM guests to be changed by QEMU code

[gengdongjiu]

If you think this patch will introduce some issue, we can add one function 
write_part_cpustate_to_list()[2] to change the specified
register instead of all the registers[1].

Below function that you added will modified all the register if new value is 
different with old value.
[1]:
bool write_cpustate_to_list(ARMCPU *cpu, bool kvm_sync)
{

}

[2]:
+bool write_part_cpustate_to_list(ARMCPU *cpu, ptrdiff_t fieldoffset)
+{
+const ARMCPRegInfo *ri;
+uint32_t regidx, i;
+
+for (i = 0; i < cpu->cpreg_array_len; i++) {
+regidx = kvm_to_cpreg_id(cpu->cpreg_indexes[i]);
+ri = get_arm_cp_reginfo(cpu->cp_regs, regidx);
+if (!ri) {
+continue;
+}
+
+if (ri->type & ARM_CP_NO_RAW) {
+continue;
+}
+if (ri->fieldoffset == fieldoffset) {
+cpu->cpreg_values[i] = read_raw_cp_reg(>env, ri);
+return true;
+}
+}
+return false;
+}
+



On 2019/3/14 19:31, gengdongjiu wrote:
>  Hi   Peter/Eric,
>I think we should fix the regression issue instead of revert this patch,  
> I think the reason of
>this issue is that QEMU modified some unexpected resisters, we should find 
> out.
> 
> 
> Revert "arm: Allow system registers for KVM guests to be changed by QEMU 
> code"
> 
> This reverts commit 823e1b3818f9b10b824ddcd756983b6e2fa68730,
> which introduces a regression running EDK2 guest firmware
> under KVM:
> 
> error: kvm run failed Function not implemented
>  PC=00013f5a6208 X00=404003c4 X01=003a
> X02= X03=404003c4 X04=
> X05=9646 X06=00013d2ef270 X07=00013e3d1710
> X08=09010755ffaf8ba8 X09=ffaf8b9cfeeb5468 X10=feeb546409010756
> X11=09010757ffaf8b90 X12=feeb50680903068b X13=090306a1ffaf8bc0
> X14= X15= X16=00013f872da0
> X17=a6ab X18= X19=00013f5a92d0
> X20=00013f5a7a78 X21=003a X22=00013f5a7ab2
> X23=00013f5a92e8 X24=00013f631090 X25=0010
> X26=0100 X27=00013f89501b X28=00013e3d14e0
> X29=00013e3d12a0 X30=00013f5a2518  SP=00013b7be0b0
> PSTATE=404003c4 -Z-- EL1t
> 
> with
> [ 3507.926571] kvm [35042]: load/store instruction decoding not 
> implemented
> in the host dmesg.
> 
> Revert the change for the moment until we can investigate the
> cause of the regression.
> 
> Reported-by: Eric Auger 
> Signed-off-by: Peter Maydell 
> 

[Qemu-devel] About Revert this patch: arm: Allow system registers for KVM guests to be changed by QEMU code

[gengdongjiu]

 Hi   Peter/Eric,
   I think we should fix the regression issue instead of revert this patch,  I 
think the reason of
   this issue is that QEMU modified some unexpected resisters, we should find 
out.


Revert "arm: Allow system registers for KVM guests to be changed by QEMU 
code"

This reverts commit 823e1b3818f9b10b824ddcd756983b6e2fa68730,
which introduces a regression running EDK2 guest firmware
under KVM:

error: kvm run failed Function not implemented
 PC=00013f5a6208 X00=404003c4 X01=003a
X02= X03=404003c4 X04=
X05=9646 X06=00013d2ef270 X07=00013e3d1710
X08=09010755ffaf8ba8 X09=ffaf8b9cfeeb5468 X10=feeb546409010756
X11=09010757ffaf8b90 X12=feeb50680903068b X13=090306a1ffaf8bc0
X14= X15= X16=00013f872da0
X17=a6ab X18= X19=00013f5a92d0
X20=00013f5a7a78 X21=003a X22=00013f5a7ab2
X23=00013f5a92e8 X24=00013f631090 X25=0010
X26=0100 X27=00013f89501b X28=00013e3d14e0
X29=00013e3d12a0 X30=00013f5a2518  SP=00013b7be0b0
PSTATE=404003c4 -Z-- EL1t

with
[ 3507.926571] kvm [35042]: load/store instruction decoding not implemented
in the host dmesg.

Revert the change for the moment until we can investigate the
cause of the regression.

Reported-by: Eric Auger 
Signed-off-by: Peter Maydell 

Re: [Qemu-devel] [PATCH V3] target/arm: change arch timer registers access permission

[gengdongjiu]

Add "Reviewed-by" from Richard because Richard has reviewed it in patch v2[1], 
thanks.

Reviewed-by: Richard Henderson 

[1]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg604509.html

On 2019/3/12 20:52, Dongjiu Geng wrote:
> Some generic arch timer registers are Config-RW in the EL0,
> which means the EL0 exception level can have write permission
> if it is appropriately configured.
> 
> When VM access registers, QEMU firstly checks whether they have RW
> permission, then check whether it is appropriately configured.
> If they are defined to read only in EL0, even though they have been
> appropriately configured, they still do not have write permission.
> So need to add the write permission according to ARMV8 spec when
> define it.
> 
> Signed-off-by: Dongjiu Geng 
> ---
> Change since V2:
> 1. Change 'Ready only' to 'read only' in the comments
> 
> Change since V1:
> 1. Change 'PL1_RW | RL0_RW' to PL0_RW because PL0_RW implied that the higher 
> PLx all have RW permission
> 
> When VM kernel or Hypervisor configures the timer registers to RW in EL0
> user space, it will still have below panic when EL0 user space access
> the timer registers.
> 
> [INFO ]@(el0_sync:60): UNIMPLEMENTED, esr=200
> [INFO ]@(unimpl_exception:88): KERNEL UNIMPLEMENTED EXCEPTION
> [INFO ]@(unimpl_exception:98): FAR=, ESR=0200 (EC=0x0, 
> IL=0x1, ISS=0x0)
> [INFO ]@(dump_registers:64): KERNEL REGISTERS
> [INFO ]@(dump_registers:68): X0=f52b7d50 X1=040d5040
> [INFO ]@(dump_registers:68): X2=00433e10 X3=
> [INFO ]@(dump_registers:68): X4=7fff X5=0020
> [INFO ]@(dump_registers:68): X6=0020 X7=0c00b030
> ---
>  target/arm/helper.c | 30 +++---
>  1 file changed, 15 insertions(+), 15 deletions(-)
> 
> diff --git a/target/arm/helper.c b/target/arm/helper.c
> index 2607d39..c8d3c21 100644
> --- a/target/arm/helper.c
> +++ b/target/arm/helper.c
> @@ -2665,7 +2665,7 @@ static const ARMCPRegInfo generic_timer_cp_reginfo[] = {
>  /* per-timer control */
>  { .name = "CNTP_CTL", .cp = 15, .crn = 14, .crm = 2, .opc1 = 0, .opc2 = 
> 1,
>.secure = ARM_CP_SECSTATE_NS,
> -  .type = ARM_CP_IO | ARM_CP_ALIAS, .access = PL1_RW | PL0_R,
> +  .type = ARM_CP_IO | ARM_CP_ALIAS, .access = PL0_RW,
>.accessfn = gt_ptimer_access,
>.fieldoffset = offsetoflow32(CPUARMState,
> cp15.c14_timer[GTIMER_PHYS].ctl),
> @@ -2674,7 +2674,7 @@ static const ARMCPRegInfo generic_timer_cp_reginfo[] = {
>  { .name = "CNTP_CTL_S",
>.cp = 15, .crn = 14, .crm = 2, .opc1 = 0, .opc2 = 1,
>.secure = ARM_CP_SECSTATE_S,
> -  .type = ARM_CP_IO | ARM_CP_ALIAS, .access = PL1_RW | PL0_R,
> +  .type = ARM_CP_IO | ARM_CP_ALIAS, .access = PL0_RW,
>.accessfn = gt_ptimer_access,
>.fieldoffset = offsetoflow32(CPUARMState,
> cp15.c14_timer[GTIMER_SEC].ctl),
> @@ -2682,14 +2682,14 @@ static const ARMCPRegInfo generic_timer_cp_reginfo[] 
> = {
>  },
>  { .name = "CNTP_CTL_EL0", .state = ARM_CP_STATE_AA64,
>.opc0 = 3, .opc1 = 3, .crn = 14, .crm = 2, .opc2 = 1,
> -  .type = ARM_CP_IO, .access = PL1_RW | PL0_R,
> +  .type = ARM_CP_IO, .access = PL0_RW,
>.accessfn = gt_ptimer_access,
>.fieldoffset = offsetof(CPUARMState, cp15.c14_timer[GTIMER_PHYS].ctl),
>.resetvalue = 0,
>.writefn = gt_phys_ctl_write, .raw_writefn = raw_write,
>  },
>  { .name = "CNTV_CTL", .cp = 15, .crn = 14, .crm = 3, .opc1 = 0, .opc2 = 
> 1,
> -  .type = ARM_CP_IO | ARM_CP_ALIAS, .access = PL1_RW | PL0_R,
> +  .type = ARM_CP_IO | ARM_CP_ALIAS, .access = PL0_RW,
>.accessfn = gt_vtimer_access,
>.fieldoffset = offsetoflow32(CPUARMState,
> cp15.c14_timer[GTIMER_VIRT].ctl),
> @@ -2697,7 +2697,7 @@ static const ARMCPRegInfo generic_timer_cp_reginfo[] = {
>  },
>  { .name = "CNTV_CTL_EL0", .state = ARM_CP_STATE_AA64,
>.opc0 = 3, .opc1 = 3, .crn = 14, .crm = 3, .opc2 = 1,
> -  .type = ARM_CP_IO, .access = PL1_RW | PL0_R,
> +  .type = ARM_CP_IO, .access = PL0_RW,
>.accessfn = gt_vtimer_access,
>.fieldoffset = offsetof(CPUARMState, cp15.c14_timer[GTIMER_VIRT].ctl),
>.resetvalue = 0,
> @@ -2706,31 +2706,31 @@ static const ARMCPRegInfo generic_timer_cp_reginfo[] 
> = {
>  /* TimerValue views: a 32 bit downcounting view of the underlying state 
> */
>  { .name = "CNTP_TVAL", .cp = 15, .crn = 14, .crm = 2, .opc1 = 0, .opc2 = 
> 0,
>.secure = ARM_CP_SECSTATE_NS,
> -  .type = ARM_CP_NO_RAW | ARM_CP_IO, .access = PL1_RW | PL0_R,
> +  .type = ARM_CP_NO_RAW | ARM_CP_IO, .access = PL0_RW,
>.accessfn = gt_ptimer_access,
>.readfn = gt_phys_tval_read, .writefn = gt_phys_tval_write,
>  },
>  { .name = "CNTP_TVAL_S",
>.cp = 15, .crn = 

Re: [Qemu-devel] [PATCH V2] target/arm: change arch timer registers access permission

[gengdongjiu]

On 2019/3/12 23:02, Richard Henderson wrote:
>> [INFO ]@(dump_registers:68): X6=0020 X7=0c00b030
>> ---
>>  target/arm/helper.c | 30 +++---
>>  1 file changed, 15 insertions(+), 15 deletions(-)
> Reviewed-by: Richard Henderson 

  Thanks for this "Reviewed-by"

> 

[Qemu-devel] [PATCH V2] target/arm: change arch timer registers access permission

[gengdongjiu]

From: Dongjiu Geng 

Some generic arch timer registers are Config-RW in the EL0,
which means the EL0 exception level can have write permission
if it is appropriately configured.

When VM access registers, it firstly checks whether they have RW
permission, then check whether it is appropriately configured.
If they are defined to Ready only in EL0, even though they have been
appropriately configured, they still do not have write permission.
So need to add the write permission according to ARMV8 spec when
define it.

Signed-off-by: Dongjiu Geng 
---
When VM kernel or Hypervisor configures the timer registers to RW in EL0
user space, it will still have below panic when EL0 user space access
the timer registers.

[INFO ]@(el0_sync:60): UNIMPLEMENTED, esr=200
[INFO ]@(unimpl_exception:88): KERNEL UNIMPLEMENTED EXCEPTION
[INFO ]@(unimpl_exception:98): FAR=, ESR=0200 (EC=0x0, 
IL=0x1, ISS=0x0)
[INFO ]@(dump_registers:64): KERNEL REGISTERS
[INFO ]@(dump_registers:68): X0=f52b7d50 X1=040d5040
[INFO ]@(dump_registers:68): X2=00433e10 X3=
[INFO ]@(dump_registers:68): X4=7fff X5=0020
[INFO ]@(dump_registers:68): X6=0020 X7=0c00b030
---
 target/arm/helper.c | 30 +++---
 1 file changed, 15 insertions(+), 15 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 2607d39..c8d3c21 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -2665,7 +2665,7 @@ static const ARMCPRegInfo generic_timer_cp_reginfo[] = {
 /* per-timer control */
 { .name = "CNTP_CTL", .cp = 15, .crn = 14, .crm = 2, .opc1 = 0, .opc2 = 1,
   .secure = ARM_CP_SECSTATE_NS,
-  .type = ARM_CP_IO | ARM_CP_ALIAS, .access = PL1_RW | PL0_R,
+  .type = ARM_CP_IO | ARM_CP_ALIAS, .access = PL0_RW,
   .accessfn = gt_ptimer_access,
   .fieldoffset = offsetoflow32(CPUARMState,
cp15.c14_timer[GTIMER_PHYS].ctl),
@@ -2674,7 +2674,7 @@ static const ARMCPRegInfo generic_timer_cp_reginfo[] = {
 { .name = "CNTP_CTL_S",
   .cp = 15, .crn = 14, .crm = 2, .opc1 = 0, .opc2 = 1,
   .secure = ARM_CP_SECSTATE_S,
-  .type = ARM_CP_IO | ARM_CP_ALIAS, .access = PL1_RW | PL0_R,
+  .type = ARM_CP_IO | ARM_CP_ALIAS, .access = PL0_RW,
   .accessfn = gt_ptimer_access,
   .fieldoffset = offsetoflow32(CPUARMState,
cp15.c14_timer[GTIMER_SEC].ctl),
@@ -2682,14 +2682,14 @@ static const ARMCPRegInfo generic_timer_cp_reginfo[] = {
 },
 { .name = "CNTP_CTL_EL0", .state = ARM_CP_STATE_AA64,
   .opc0 = 3, .opc1 = 3, .crn = 14, .crm = 2, .opc2 = 1,
-  .type = ARM_CP_IO, .access = PL1_RW | PL0_R,
+  .type = ARM_CP_IO, .access = PL0_RW,
   .accessfn = gt_ptimer_access,
   .fieldoffset = offsetof(CPUARMState, cp15.c14_timer[GTIMER_PHYS].ctl),
   .resetvalue = 0,
   .writefn = gt_phys_ctl_write, .raw_writefn = raw_write,
 },
 { .name = "CNTV_CTL", .cp = 15, .crn = 14, .crm = 3, .opc1 = 0, .opc2 = 1,
-  .type = ARM_CP_IO | ARM_CP_ALIAS, .access = PL1_RW | PL0_R,
+  .type = ARM_CP_IO | ARM_CP_ALIAS, .access = PL0_RW,
   .accessfn = gt_vtimer_access,
   .fieldoffset = offsetoflow32(CPUARMState,
cp15.c14_timer[GTIMER_VIRT].ctl),
@@ -2697,7 +2697,7 @@ static const ARMCPRegInfo generic_timer_cp_reginfo[] = {
 },
 { .name = "CNTV_CTL_EL0", .state = ARM_CP_STATE_AA64,
   .opc0 = 3, .opc1 = 3, .crn = 14, .crm = 3, .opc2 = 1,
-  .type = ARM_CP_IO, .access = PL1_RW | PL0_R,
+  .type = ARM_CP_IO, .access = PL0_RW,
   .accessfn = gt_vtimer_access,
   .fieldoffset = offsetof(CPUARMState, cp15.c14_timer[GTIMER_VIRT].ctl),
   .resetvalue = 0,
@@ -2706,31 +2706,31 @@ static const ARMCPRegInfo generic_timer_cp_reginfo[] = {
 /* TimerValue views: a 32 bit downcounting view of the underlying state */
 { .name = "CNTP_TVAL", .cp = 15, .crn = 14, .crm = 2, .opc1 = 0, .opc2 = 0,
   .secure = ARM_CP_SECSTATE_NS,
-  .type = ARM_CP_NO_RAW | ARM_CP_IO, .access = PL1_RW | PL0_R,
+  .type = ARM_CP_NO_RAW | ARM_CP_IO, .access = PL0_RW,
   .accessfn = gt_ptimer_access,
   .readfn = gt_phys_tval_read, .writefn = gt_phys_tval_write,
 },
 { .name = "CNTP_TVAL_S",
   .cp = 15, .crn = 14, .crm = 2, .opc1 = 0, .opc2 = 0,
   .secure = ARM_CP_SECSTATE_S,
-  .type = ARM_CP_NO_RAW | ARM_CP_IO, .access = PL1_RW | PL0_R,
+  .type = ARM_CP_NO_RAW | ARM_CP_IO, .access = PL0_RW,
   .accessfn = gt_ptimer_access,
   .readfn = gt_sec_tval_read, .writefn = gt_sec_tval_write,
 },
 { .name = "CNTP_TVAL_EL0", .state = ARM_CP_STATE_AA64,
   .opc0 = 3, .opc1 = 3, .crn = 14, .crm = 2, .opc2 = 0,
-  .type = ARM_CP_NO_RAW | ARM_CP_IO, .access = PL1_RW | PL0_R,
+  .type = ARM_CP_NO_RAW | ARM_CP_IO, .access = PL0_RW,
   .accessfn = gt_ptimer_access, .resetfn = 

Re: [Qemu-devel] [RESEND PATCH] target/arm: change arch timer registers access permission

[gengdongjiu]

understand now，thanks for the explaination, I will update it.
发件人：Peter Maydell 
收件人：gengdongjiu 
抄 送：qemu-arm ;QEMU Developers 
时间：2019-03-11 23:39:32
主 题：Re: [RESEND PATCH] target/arm: change arch timer registers access permission

On Mon, 11 Mar 2019 at 15:24, gengdongjiu  wrote:
> So If QEMU defined the timer registers read only in PL0, even though it has 
> been configured to have write permission in PL0 by high PLx, we still cannot 
> have
> Write permission, because QEMU will firstly check the defined permission and 
> then check the configured permission by high PLx.

I'm afraid I don't understand what you're trying to say here.

Yes, it was a bug that we marked these registers as read-only for PL0,
and we should fix it.

What I am trying to say is that
 .access = PL1_RW | PL0_RW

is exactly equivalent to
 .access = PL0_RW

and we should use the simpler version of the expression.

(If you look at the definitions of all the PL*_ constants,
you can see that PL0_W implies PL1_W:
#define PL0_W (0x01 | PL1_W)
and similarly for PL0_R. So all the bits in the bitfield that would be
set by PL1_RW are also set by PL0_RW.)

thanks
-- PMM

Re: [Qemu-devel] [RESEND PATCH] target/arm: change arch timer registers access permission

[gengdongjiu]

> Hi Peter,
>Thanks for the review.
> 
> > >
> > > From: Dongjiu Geng 
> > >
> > > Some generic arch timer registers are Config-RW in the EL0, which
> > > means the EL0 exception level can have write permission if it is
> > > appropriately configured.
> > >
> > > When VM access registers, it firstly checks whether they have RW
> > > permission, then check whether it is appropriately configured.
> > > If they are defined to Ready only in EL0, even though they have been
> >
> > "read only" ?
> Yes, read only.
> 
> >
> > > appropriately configured, they still do not have write permission.
> > > So need to add the write permission according to ARMV8 spec when
> > > define it.
> > >
> > > Signed-off-by: Dongjiu Geng 
> >
> > Yes, this seems to be a bug which has been present since I added the 
> > generic timer support in commit 55d284af8e31b in 2013.
> >
> > I'm not sure why the bug is there -- my best guess is that I
> > incorrectly copied the permission flags from the CNTFRQ register (which 
> > really is RO from PL0 but RW from PL1 and above) to these other
> registers which should be fully RW from PL0.
> 
> The current logic is show below[1], handle_sys() will check whether the 
> system register have write permission in PL0, if have, then check
> whether It have been configured in the high PLx by ri->accessfn().

So If QEMU defined the timer registers read only in PL0, even though it has 
been configured to have write permission in PL0 by high PLx, we still cannot 
have
Write permission, because QEMU will firstly check the defined permission and 
then check the configured permission by high PLx.

> 
> 
> [1]:
> handle_sys()
> --> cp_access_ok(s->current_el, ri, isread)
> --> gen_helper_access_check_cp_reg()
>->ri->accessfn()
> 
> [2]:
> ---
> static void handle_sys(DisasContext *s, uint32_t insn, bool isread,
>unsigned int op0, unsigned int op1, unsigned int op2,
>unsigned int crn, unsigned int crm, unsigned int rt) {
> 
> 
> ---
>/* Check access permissions */
> if (!cp_access_ok(s->current_el, ri, isread)) {
> unallocated_encoding(s);
> return;
> }
> 
> if (ri->accessfn) {
> ..
> gen_helper_access_check_cp_reg(cpu_env, tmpptr, tcg_syn, tcg_isread);
> ..
>   }
> ---
> }
> 
> 
> 
> >
> > > diff --git a/target/arm/helper.c b/target/arm/helper.c index
> > > 2607d39..3db94c6 100644
> > > --- a/target/arm/helper.c
> > > +++ b/target/arm/helper.c
> > > @@ -2665,7 +2665,7 @@ static const ARMCPRegInfo 
> > > generic_timer_cp_reginfo[] = {
> > >  /* per-timer control */
> > >  { .name = "CNTP_CTL", .cp = 15, .crn = 14, .crm = 2, .opc1 = 0, 
> > > .opc2 = 1,
> > >.secure = ARM_CP_SECSTATE_NS,
> > > -  .type = ARM_CP_IO | ARM_CP_ALIAS, .access = PL1_RW | PL0_R,
> > > +  .type = ARM_CP_IO | ARM_CP_ALIAS, .access = PL1_RW | PL0_RW,
> >
> > The PLx_{R,W,RW} constants all imply access is also possible from the 
> > higher PLs, so this can just be written ".access = PL0_RW".
> 
> Ok, I will uses your suggested method and to see whether it can be workable.
> 
> >
> > thanks
> > -- PMM

Re: [Qemu-devel] [RESEND PATCH] target/arm: change arch timer registers access permission

[gengdongjiu]

Hi Peter,
   Thanks for the review.

> >
> > From: Dongjiu Geng 
> >
> > Some generic arch timer registers are Config-RW in the EL0, which
> > means the EL0 exception level can have write permission if it is
> > appropriately configured.
> >
> > When VM access registers, it firstly checks whether they have RW
> > permission, then check whether it is appropriately configured.
> > If they are defined to Ready only in EL0, even though they have been
> 
> "read only" ?
Yes, read only.

> 
> > appropriately configured, they still do not have write permission.
> > So need to add the write permission according to ARMV8 spec when
> > define it.
> >
> > Signed-off-by: Dongjiu Geng 
> 
> Yes, this seems to be a bug which has been present since I added the generic 
> timer support in commit 55d284af8e31b in 2013.
> 
> I'm not sure why the bug is there -- my best guess is that I incorrectly 
> copied the permission flags from the CNTFRQ register (which really is
> RO from PL0 but RW from PL1 and above) to these other registers which should 
> be fully RW from PL0.

The current logic is show below[1], handle_sys() will check whether the system 
register have write permission in PL0, if have, then check whether
It have been configured in the high PLx by ri->accessfn()


[1]:
handle_sys()
--> cp_access_ok(s->current_el, ri, isread)
--> gen_helper_access_check_cp_reg()
   ->ri->accessfn()

[2]:
---
static void handle_sys(DisasContext *s, uint32_t insn, bool isread,
   unsigned int op0, unsigned int op1, unsigned int op2,
   unsigned int crn, unsigned int crm, unsigned int rt)
{
   
---
   /* Check access permissions */
if (!cp_access_ok(s->current_el, ri, isread)) {
unallocated_encoding(s);
return;
}

if (ri->accessfn) {
..
gen_helper_access_check_cp_reg(cpu_env, tmpptr, tcg_syn, tcg_isread);
..
}
---
}



> 
> > diff --git a/target/arm/helper.c b/target/arm/helper.c index
> > 2607d39..3db94c6 100644
> > --- a/target/arm/helper.c
> > +++ b/target/arm/helper.c
> > @@ -2665,7 +2665,7 @@ static const ARMCPRegInfo generic_timer_cp_reginfo[] 
> > = {
> >  /* per-timer control */
> >  { .name = "CNTP_CTL", .cp = 15, .crn = 14, .crm = 2, .opc1 = 0, .opc2 
> > = 1,
> >.secure = ARM_CP_SECSTATE_NS,
> > -  .type = ARM_CP_IO | ARM_CP_ALIAS, .access = PL1_RW | PL0_R,
> > +  .type = ARM_CP_IO | ARM_CP_ALIAS, .access = PL1_RW | PL0_RW,
> 
> The PLx_{R,W,RW} constants all imply access is also possible from the higher 
> PLs, so this can just be written ".access = PL0_RW".

Ok, I will uses your suggested method and to see whether it can be workable.

> 
> thanks
> -- PMM

[Qemu-devel] [PATCH] target/arm: change arch timer registers access permission

[gengdongjiu]

Some generic arch timer registers are Config-RW in the EL0,
which means the EL0 exception level can have write permission
if it is appropriately configured.

When VM access registers, it firstly checks whether they have RW
permission, then check whether it is appropriately configured.
If they are defined to Ready only in EL0, even though they have been
appropriately configured, they still do not have write permission.
So need to add the write permission according to ARMV8 spec when
define it.

Signed-off-by: Dongjiu Geng 
---
When kernel or Hypervisor configures the timer registers to RW in EL0
user space, it will still have below panic when EL0 user space access
the timer registers.

[INFO ]@(el0_sync:60): UNIMPLEMENTED, esr=200
[INFO ]@(unimpl_exception:88): KERNEL UNIMPLEMENTED EXCEPTION
[INFO ]@(unimpl_exception:98): FAR=, ESR=0200 (EC=0x0, 
IL=0x1, ISS=0x0)
[INFO ]@(dump_registers:64): KERNEL REGISTERS
[INFO ]@(dump_registers:68): X0=f52b7d50 X1=040d5040
[INFO ]@(dump_registers:68): X2=00433e10 X3=
[INFO ]@(dump_registers:68): X4=7fff X5=0020
[INFO ]@(dump_registers:68): X6=0020 X7=0c00b030
---
 target/arm/helper.c | 30 +++---
 1 file changed, 15 insertions(+), 15 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 2607d39..3db94c6 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -2665,7 +2665,7 @@ static const ARMCPRegInfo generic_timer_cp_reginfo[] = {
 /* per-timer control */
 { .name = "CNTP_CTL", .cp = 15, .crn = 14, .crm = 2, .opc1 = 0, .opc2 = 1,
   .secure = ARM_CP_SECSTATE_NS,
-  .type = ARM_CP_IO | ARM_CP_ALIAS, .access = PL1_RW | PL0_R,
+  .type = ARM_CP_IO | ARM_CP_ALIAS, .access = PL1_RW | PL0_RW,
   .accessfn = gt_ptimer_access,
   .fieldoffset = offsetoflow32(CPUARMState,
cp15.c14_timer[GTIMER_PHYS].ctl),
@@ -2674,7 +2674,7 @@ static const ARMCPRegInfo generic_timer_cp_reginfo[] = {
 { .name = "CNTP_CTL_S",
   .cp = 15, .crn = 14, .crm = 2, .opc1 = 0, .opc2 = 1,
   .secure = ARM_CP_SECSTATE_S,
-  .type = ARM_CP_IO | ARM_CP_ALIAS, .access = PL1_RW | PL0_R,
+  .type = ARM_CP_IO | ARM_CP_ALIAS, .access = PL1_RW | PL0_RW,
   .accessfn = gt_ptimer_access,
   .fieldoffset = offsetoflow32(CPUARMState,
cp15.c14_timer[GTIMER_SEC].ctl),
@@ -2682,14 +2682,14 @@ static const ARMCPRegInfo generic_timer_cp_reginfo[] = {
 },
 { .name = "CNTP_CTL_EL0", .state = ARM_CP_STATE_AA64,
   .opc0 = 3, .opc1 = 3, .crn = 14, .crm = 2, .opc2 = 1,
-  .type = ARM_CP_IO, .access = PL1_RW | PL0_R,
+  .type = ARM_CP_IO, .access = PL1_RW | PL0_RW,
   .accessfn = gt_ptimer_access,
   .fieldoffset = offsetof(CPUARMState, cp15.c14_timer[GTIMER_PHYS].ctl),
   .resetvalue = 0,
   .writefn = gt_phys_ctl_write, .raw_writefn = raw_write,
 },
 { .name = "CNTV_CTL", .cp = 15, .crn = 14, .crm = 3, .opc1 = 0, .opc2 = 1,
-  .type = ARM_CP_IO | ARM_CP_ALIAS, .access = PL1_RW | PL0_R,
+  .type = ARM_CP_IO | ARM_CP_ALIAS, .access = PL1_RW | PL0_RW,
   .accessfn = gt_vtimer_access,
   .fieldoffset = offsetoflow32(CPUARMState,
cp15.c14_timer[GTIMER_VIRT].ctl),
@@ -2697,7 +2697,7 @@ static const ARMCPRegInfo generic_timer_cp_reginfo[] = {
 },
 { .name = "CNTV_CTL_EL0", .state = ARM_CP_STATE_AA64,
   .opc0 = 3, .opc1 = 3, .crn = 14, .crm = 3, .opc2 = 1,
-  .type = ARM_CP_IO, .access = PL1_RW | PL0_R,
+  .type = ARM_CP_IO, .access = PL1_RW | PL0_RW,
   .accessfn = gt_vtimer_access,
   .fieldoffset = offsetof(CPUARMState, cp15.c14_timer[GTIMER_VIRT].ctl),
   .resetvalue = 0,
@@ -2706,31 +2706,31 @@ static const ARMCPRegInfo generic_timer_cp_reginfo[] = {
 /* TimerValue views: a 32 bit downcounting view of the underlying state */
 { .name = "CNTP_TVAL", .cp = 15, .crn = 14, .crm = 2, .opc1 = 0, .opc2 = 0,
   .secure = ARM_CP_SECSTATE_NS,
-  .type = ARM_CP_NO_RAW | ARM_CP_IO, .access = PL1_RW | PL0_R,
+  .type = ARM_CP_NO_RAW | ARM_CP_IO, .access = PL1_RW | PL0_RW,
   .accessfn = gt_ptimer_access,
   .readfn = gt_phys_tval_read, .writefn = gt_phys_tval_write,
 },
 { .name = "CNTP_TVAL_S",
   .cp = 15, .crn = 14, .crm = 2, .opc1 = 0, .opc2 = 0,
   .secure = ARM_CP_SECSTATE_S,
-  .type = ARM_CP_NO_RAW | ARM_CP_IO, .access = PL1_RW | PL0_R,
+  .type = ARM_CP_NO_RAW | ARM_CP_IO, .access = PL1_RW | PL0_RW,
   .accessfn = gt_ptimer_access,
   .readfn = gt_sec_tval_read, .writefn = gt_sec_tval_write,
 },
 { .name = "CNTP_TVAL_EL0", .state = ARM_CP_STATE_AA64,
   .opc0 = 3, .opc1 = 3, .crn = 14, .crm = 2, .opc2 = 0,
-  .type = ARM_CP_NO_RAW | ARM_CP_IO, .access = PL1_RW | PL0_R,
+  .type = ARM_CP_NO_RAW | ARM_CP_IO, .access = PL1_RW | PL0_RW,
   

[Qemu-devel] [PATCH] target/arm: change arch timer registers access permission

[gengdongjiu]

Some generic arch timer registers are Config-RW in the EL0,
which means the EL0 exception level can have write permission
if it is appropriately configured.

When VM access registers, it firstly checks whether they have RW
permission, then check whether it is appropriately configured.
If they are defined to Ready only in EL0, even though they have been
appropriately configured, they still do not have write permission.
So need to add the write permission according to ARMV8 spec when
define it.

Signed-off-by: Dongjiu Geng 
---
When kernel or Hypervisor configures the timer registers to RW in EL0
user space, it will still have below panic when EL0 user space access
the timer registers.

[INFO ]@(el0_sync:60): UNIMPLEMENTED, esr=200
[INFO ]@(unimpl_exception:88): KERNEL UNIMPLEMENTED EXCEPTION
[INFO ]@(unimpl_exception:98): FAR=, ESR=0200 (EC=0x0, 
IL=0x1, ISS=0x0)
[INFO ]@(dump_registers:64): KERNEL REGISTERS
[INFO ]@(dump_registers:68): X0=f52b7d50 X1=040d5040
[INFO ]@(dump_registers:68): X2=00433e10 X3=
[INFO ]@(dump_registers:68): X4=7fff X5=0020
[INFO ]@(dump_registers:68): X6=0020 X7=0c00b030
---
 target/arm/helper.c | 30 +++---
 1 file changed, 15 insertions(+), 15 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 2607d39..3db94c6 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -2665,7 +2665,7 @@ static const ARMCPRegInfo generic_timer_cp_reginfo[] = {
 /* per-timer control */
 { .name = "CNTP_CTL", .cp = 15, .crn = 14, .crm = 2, .opc1 = 0, .opc2 = 1,
   .secure = ARM_CP_SECSTATE_NS,
-  .type = ARM_CP_IO | ARM_CP_ALIAS, .access = PL1_RW | PL0_R,
+  .type = ARM_CP_IO | ARM_CP_ALIAS, .access = PL1_RW | PL0_RW,
   .accessfn = gt_ptimer_access,
   .fieldoffset = offsetoflow32(CPUARMState,
cp15.c14_timer[GTIMER_PHYS].ctl),
@@ -2674,7 +2674,7 @@ static const ARMCPRegInfo generic_timer_cp_reginfo[] = {
 { .name = "CNTP_CTL_S",
   .cp = 15, .crn = 14, .crm = 2, .opc1 = 0, .opc2 = 1,
   .secure = ARM_CP_SECSTATE_S,
-  .type = ARM_CP_IO | ARM_CP_ALIAS, .access = PL1_RW | PL0_R,
+  .type = ARM_CP_IO | ARM_CP_ALIAS, .access = PL1_RW | PL0_RW,
   .accessfn = gt_ptimer_access,
   .fieldoffset = offsetoflow32(CPUARMState,
cp15.c14_timer[GTIMER_SEC].ctl),
@@ -2682,14 +2682,14 @@ static const ARMCPRegInfo generic_timer_cp_reginfo[] = {
 },
 { .name = "CNTP_CTL_EL0", .state = ARM_CP_STATE_AA64,
   .opc0 = 3, .opc1 = 3, .crn = 14, .crm = 2, .opc2 = 1,
-  .type = ARM_CP_IO, .access = PL1_RW | PL0_R,
+  .type = ARM_CP_IO, .access = PL1_RW | PL0_RW,
   .accessfn = gt_ptimer_access,
   .fieldoffset = offsetof(CPUARMState, cp15.c14_timer[GTIMER_PHYS].ctl),
   .resetvalue = 0,
   .writefn = gt_phys_ctl_write, .raw_writefn = raw_write,
 },
 { .name = "CNTV_CTL", .cp = 15, .crn = 14, .crm = 3, .opc1 = 0, .opc2 = 1,
-  .type = ARM_CP_IO | ARM_CP_ALIAS, .access = PL1_RW | PL0_R,
+  .type = ARM_CP_IO | ARM_CP_ALIAS, .access = PL1_RW | PL0_RW,
   .accessfn = gt_vtimer_access,
   .fieldoffset = offsetoflow32(CPUARMState,
cp15.c14_timer[GTIMER_VIRT].ctl),
@@ -2697,7 +2697,7 @@ static const ARMCPRegInfo generic_timer_cp_reginfo[] = {
 },
 { .name = "CNTV_CTL_EL0", .state = ARM_CP_STATE_AA64,
   .opc0 = 3, .opc1 = 3, .crn = 14, .crm = 3, .opc2 = 1,
-  .type = ARM_CP_IO, .access = PL1_RW | PL0_R,
+  .type = ARM_CP_IO, .access = PL1_RW | PL0_RW,
   .accessfn = gt_vtimer_access,
   .fieldoffset = offsetof(CPUARMState, cp15.c14_timer[GTIMER_VIRT].ctl),
   .resetvalue = 0,
@@ -2706,31 +2706,31 @@ static const ARMCPRegInfo generic_timer_cp_reginfo[] = {
 /* TimerValue views: a 32 bit downcounting view of the underlying state */
 { .name = "CNTP_TVAL", .cp = 15, .crn = 14, .crm = 2, .opc1 = 0, .opc2 = 0,
   .secure = ARM_CP_SECSTATE_NS,
-  .type = ARM_CP_NO_RAW | ARM_CP_IO, .access = PL1_RW | PL0_R,
+  .type = ARM_CP_NO_RAW | ARM_CP_IO, .access = PL1_RW | PL0_RW,
   .accessfn = gt_ptimer_access,
   .readfn = gt_phys_tval_read, .writefn = gt_phys_tval_write,
 },
 { .name = "CNTP_TVAL_S",
   .cp = 15, .crn = 14, .crm = 2, .opc1 = 0, .opc2 = 0,
   .secure = ARM_CP_SECSTATE_S,
-  .type = ARM_CP_NO_RAW | ARM_CP_IO, .access = PL1_RW | PL0_R,
+  .type = ARM_CP_NO_RAW | ARM_CP_IO, .access = PL1_RW | PL0_RW,
   .accessfn = gt_ptimer_access,
   .readfn = gt_sec_tval_read, .writefn = gt_sec_tval_write,
 },
 { .name = "CNTP_TVAL_EL0", .state = ARM_CP_STATE_AA64,
   .opc0 = 3, .opc1 = 3, .crn = 14, .crm = 2, .opc2 = 0,
-  .type = ARM_CP_NO_RAW | ARM_CP_IO, .access = PL1_RW | PL0_R,
+  .type = ARM_CP_NO_RAW | ARM_CP_IO, .access = PL1_RW | PL0_RW,
   

[Qemu-devel] [RESEND PATCH] target/arm: change arch timer registers access permission

[gengdongjiu]

From: Dongjiu Geng 

Some generic arch timer registers are Config-RW in the EL0,
which means the EL0 exception level can have write permission
if it is appropriately configured.

When VM access registers, it firstly checks whether they have RW
permission, then check whether it is appropriately configured.
If they are defined to Ready only in EL0, even though they have been
appropriately configured, they still do not have write permission.
So need to add the write permission according to ARMV8 spec when
define it.

Signed-off-by: Dongjiu Geng 
---
When kernel or Hypervisor configures the timer registers to RW in EL0
user space, it will still have below panic when EL0 user space access
the timer registers.

[INFO ]@(el0_sync:60): UNIMPLEMENTED, esr=200
[INFO ]@(unimpl_exception:88): KERNEL UNIMPLEMENTED EXCEPTION
[INFO ]@(unimpl_exception:98): FAR=, ESR=0200 (EC=0x0, 
IL=0x1, ISS=0x0)
[INFO ]@(dump_registers:64): KERNEL REGISTERS
[INFO ]@(dump_registers:68): X0=f52b7d50 X1=040d5040
[INFO ]@(dump_registers:68): X2=00433e10 X3=
[INFO ]@(dump_registers:68): X4=7fff X5=0020
[INFO ]@(dump_registers:68): X6=0020 X7=0c00b030
---
 target/arm/helper.c | 30 +++---
 1 file changed, 15 insertions(+), 15 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 2607d39..3db94c6 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -2665,7 +2665,7 @@ static const ARMCPRegInfo generic_timer_cp_reginfo[] = {
 /* per-timer control */
 { .name = "CNTP_CTL", .cp = 15, .crn = 14, .crm = 2, .opc1 = 0, .opc2 = 1,
   .secure = ARM_CP_SECSTATE_NS,
-  .type = ARM_CP_IO | ARM_CP_ALIAS, .access = PL1_RW | PL0_R,
+  .type = ARM_CP_IO | ARM_CP_ALIAS, .access = PL1_RW | PL0_RW,
   .accessfn = gt_ptimer_access,
   .fieldoffset = offsetoflow32(CPUARMState,
cp15.c14_timer[GTIMER_PHYS].ctl),
@@ -2674,7 +2674,7 @@ static const ARMCPRegInfo generic_timer_cp_reginfo[] = {
 { .name = "CNTP_CTL_S",
   .cp = 15, .crn = 14, .crm = 2, .opc1 = 0, .opc2 = 1,
   .secure = ARM_CP_SECSTATE_S,
-  .type = ARM_CP_IO | ARM_CP_ALIAS, .access = PL1_RW | PL0_R,
+  .type = ARM_CP_IO | ARM_CP_ALIAS, .access = PL1_RW | PL0_RW,
   .accessfn = gt_ptimer_access,
   .fieldoffset = offsetoflow32(CPUARMState,
cp15.c14_timer[GTIMER_SEC].ctl),
@@ -2682,14 +2682,14 @@ static const ARMCPRegInfo generic_timer_cp_reginfo[] = {
 },
 { .name = "CNTP_CTL_EL0", .state = ARM_CP_STATE_AA64,
   .opc0 = 3, .opc1 = 3, .crn = 14, .crm = 2, .opc2 = 1,
-  .type = ARM_CP_IO, .access = PL1_RW | PL0_R,
+  .type = ARM_CP_IO, .access = PL1_RW | PL0_RW,
   .accessfn = gt_ptimer_access,
   .fieldoffset = offsetof(CPUARMState, cp15.c14_timer[GTIMER_PHYS].ctl),
   .resetvalue = 0,
   .writefn = gt_phys_ctl_write, .raw_writefn = raw_write,
 },
 { .name = "CNTV_CTL", .cp = 15, .crn = 14, .crm = 3, .opc1 = 0, .opc2 = 1,
-  .type = ARM_CP_IO | ARM_CP_ALIAS, .access = PL1_RW | PL0_R,
+  .type = ARM_CP_IO | ARM_CP_ALIAS, .access = PL1_RW | PL0_RW,
   .accessfn = gt_vtimer_access,
   .fieldoffset = offsetoflow32(CPUARMState,
cp15.c14_timer[GTIMER_VIRT].ctl),
@@ -2697,7 +2697,7 @@ static const ARMCPRegInfo generic_timer_cp_reginfo[] = {
 },
 { .name = "CNTV_CTL_EL0", .state = ARM_CP_STATE_AA64,
   .opc0 = 3, .opc1 = 3, .crn = 14, .crm = 3, .opc2 = 1,
-  .type = ARM_CP_IO, .access = PL1_RW | PL0_R,
+  .type = ARM_CP_IO, .access = PL1_RW | PL0_RW,
   .accessfn = gt_vtimer_access,
   .fieldoffset = offsetof(CPUARMState, cp15.c14_timer[GTIMER_VIRT].ctl),
   .resetvalue = 0,
@@ -2706,31 +2706,31 @@ static const ARMCPRegInfo generic_timer_cp_reginfo[] = {
 /* TimerValue views: a 32 bit downcounting view of the underlying state */
 { .name = "CNTP_TVAL", .cp = 15, .crn = 14, .crm = 2, .opc1 = 0, .opc2 = 0,
   .secure = ARM_CP_SECSTATE_NS,
-  .type = ARM_CP_NO_RAW | ARM_CP_IO, .access = PL1_RW | PL0_R,
+  .type = ARM_CP_NO_RAW | ARM_CP_IO, .access = PL1_RW | PL0_RW,
   .accessfn = gt_ptimer_access,
   .readfn = gt_phys_tval_read, .writefn = gt_phys_tval_write,
 },
 { .name = "CNTP_TVAL_S",
   .cp = 15, .crn = 14, .crm = 2, .opc1 = 0, .opc2 = 0,
   .secure = ARM_CP_SECSTATE_S,
-  .type = ARM_CP_NO_RAW | ARM_CP_IO, .access = PL1_RW | PL0_R,
+  .type = ARM_CP_NO_RAW | ARM_CP_IO, .access = PL1_RW | PL0_RW,
   .accessfn = gt_ptimer_access,
   .readfn = gt_sec_tval_read, .writefn = gt_sec_tval_write,
 },
 { .name = "CNTP_TVAL_EL0", .state = ARM_CP_STATE_AA64,
   .opc0 = 3, .opc1 = 3, .crn = 14, .crm = 2, .opc2 = 0,
-  .type = ARM_CP_NO_RAW | ARM_CP_IO, .access = PL1_RW | PL0_R,
+  .type = ARM_CP_NO_RAW | ARM_CP_IO, .access = PL1_RW | 

Re: [Qemu-devel] [RFC] arm: Allow system registers for KVM guests to be changed by QEMU code

[gengdongjiu]

I think Peter's patch is good, I will resubmit my series patches based on this.

On 2019/2/14 4:18, Alex Bennée wrote:
> 
> Peter Maydell  writes:
> 
>> On Wed, 13 Feb 2019 at 16:29, Alex Bennée  wrote:
>>>
>>>
>>> Peter Maydell  writes:
>>>
 At the moment the Arm implementations of kvm_arch_{get,put}_registers()
 don't support having QEMU change the values of system registers
 (aka coprocessor registers for AArch32). This is because although
 kvm_arch_get_registers() calls write_list_to_cpustate() to
 update the CPU state struct fields (so QEMU code can read the
 values in the usual way), kvm_arch_put_registers() does not
 call write_cpustate_to_list(), meaning that any changes to
 the CPU state struct fields will not be passed back to KVM.

 The rationale for this design is documented in a comment in the
 AArch32 kvm_arch_put_registers() -- writing the values in the
 cpregs list into the CPU state struct is "lossy" because the
 write of a register might not succeed, and so if we blindly
 copy the CPU state values back again we will incorrectly
 change register values for the guest. The assumption was that
 no QEMU code would need to write to the registers.

 However, when we implemented debug support for KVM guests, we
 broke that assumption: the code to handle "set the guest up
 to take a breakpoint exception" does so by updating various
 guest registers including ESR_EL1.

 Support this by making kvm_arch_put_registers() synchronize
 CPU state back into the list. We sync only those registers
 where the initial write succeeds, which should be sufficient.
>>
>>> [snip a long test setup that I didn't really understand]
> 
> Sorry that was just making sure I'd documented the exact steps of the
> manual test for posterity.
> 
>>
>> I'm confused -- I'm not sure if the things you're saying
>> didn't work:
>>  (a) didn't work before this patch and still don't
>>  (b) used to work and are broken by this patch
>>  (c) used to not work and are fixed by this patch
> 
> option (c) - this patch has made things better hence the t-b and r-b
> tags. I might send you a follow up if we can also have single-step
> working as well but that requires me to test a kernel patch (and remove
> the error leg for guest single step in kvm64).
> 
>>
>> More generally:
>>  * this patch claims to fix the code path where QEMU sets
>>up the guest to take a breakpoint exception (specifically
>>making our change of ESR_EL1 actually take effect) -- so does
>>it do that? Or is it impossible for that code path in QEMU
>>to be used (if so, can we just remove it) ?
> 
> No it's needed - I'm observing exceptions coming through the path and
> then being delivered to the guest while the host is debugging - modulo
> the you can't have active HW breakpoints in both guest and host at the
> same time.
> 
> Even here now we are sure the guest state is correctly reflected we
> could make the debug code smarter and try it's best to preserve guest
> debug regs while using it's own entirely in userspace. But I suspect
> that is a bit niche.
> 
>>
>> thanks
>> -- PMM
> 
> 
> --
> Alex Bennée
> 
> .
> 

Re: [Qemu-devel] [PATCH v15 10/10] target-arm: kvm64: handle SIGBUS signal from kernel or KVM

[gengdongjiu]

> 
> On Thu, 8 Nov 2018 at 10:18, Dongjiu Geng  wrote:
> >
> > Add SIGBUS signal handler. In this handler, it checks the SIGBUS type,
> > translates the host VA delivered by host to guest PA, then fill this
> > PA to guest APEI GHES memory, then notify guest according to the SIGBUS 
> > type.
> > There are two kinds of SIGBUS that QEMU needs to handle, which are
> > BUS_MCEERR_AO and BUS_MCEERR_AR.
> 
> > +bool ghes_record_errors(uint32_t notify, uint64_t physical_address) {
> > +uint64_t error_block_addr, read_ack_register_addr;
> > +int read_ack_register = 0, loop = 0;
> > +uint64_t start_addr = le32_to_cpu(ges.ghes_addr_le);
> > +bool ret = GHES_CPER_FAIL;
> > +const uint8_t error_source_id[] = { 0xff, 0xff, 0xff, 0xff,
> > +0xff, 0xff, 0xff, 0, 1};
> > +
> > +/*
> > + * | +-+ ges.ghes_addr_le
> > + * | |error_block_address0|
> > + * | +-+
> > + * | |error_block_address1|
> > + * | +-+ --+--
> > + * | |.| GHES_ADDRESS_SIZE
> > + * | +-+ --+--
> > + * | |error_block_addressN|
> > + * | +-+
> > + * | | read_ack_register0  |
> > + * | +-+ --+--
> > + * | | read_ack_register1  | GHES_ADDRESS_SIZE
> > + * | +-+ --+--
> > + * | |   . |
> > + * | +-+
> > + * | | read_ack_registerN  |
> > + * | +-+ --+--
> > + * | |  CPER   |   |
> > + * | |     | GHES_MAX_RAW_DATA_LENGT
> > + * | |  CPER   |   |
> > + * | +-+ --+--
> > + * | |..   |
> > + * | +-+
> > + * | |  CPER   |
> > + * | |     |
> > + * | |  CPER   |
> > + * | +-+
> > + */
> > +if (physical_address && notify < ACPI_HEST_NOTIFY_RESERVED) {
> 
> Why is a zero physical address special here? Shouldn't we be able to report 
> "the memory backing physaddr 0 has a fault" ?

Thanks a lot for the reminder, the physical memory address 0 can also have a 
fault. It should not a special address.

> 
> thanks
> -- PMM

Re: [Qemu-devel] [RFC] arm: Allow system registers for KVM guests to be changed by QEMU code

[gengdongjiu]

On 2018/12/6 23:14, Peter Maydell wrote:
> At the moment the Arm implementations of kvm_arch_{get,put}_registers()
> don't support having QEMU change the values of system registers
> (aka coprocessor registers for AArch32). This is because although
> kvm_arch_get_registers() calls write_list_to_cpustate() to
> update the CPU state struct fields (so QEMU code can read the
> values in the usual way), kvm_arch_put_registers() does not
> call write_cpustate_to_list(), meaning that any changes to
> the CPU state struct fields will not be passed back to KVM.
> 
> The rationale for this design is documented in a comment in the
> AArch32 kvm_arch_put_registers() -- writing the values in the
> cpregs list into the CPU state struct is "lossy" because the
> write of a register might not succeed, and so if we blindly
> copy the CPU state values back again we will incorrectly
> change register values for the guest. The assumption was that
> no QEMU code would need to write to the registers.
> 
> However, when we implemented debug support for KVM guests, we
> broke that assumption: the code to handle "set the guest up
> to take a breakpoint exception" does so by updating various
> guest registers including ESR_EL1.
> 
> Support this by making kvm_arch_put_registers() synchronize
> CPU state back into the list. We sync only those registers
> where the initial write succeeds, which should be sufficient.
> 
> Signed-off-by: Peter Maydell 

Tested-by: Dongjiu Geng 

> ---
> I think this patch:
>  * should be necessary for the current KVM debug code to be able
>to actually set the ESR_EL1 for the guest when it wants to
>deliver a breakpoint exception to it
>  * will also be necessary for Dongjiu's patchset to inject
>external aborts on memory errors
> 
> I have tagged it "RFC" because I don't have a setup to test
> that; I've just given it a quick smoke test that it runs a
> VM OK. Please test this and check whether it really does
> fix the bugs I think it does :-)

I have tested this patch in my aarch64 platform,
it works well to inject external aborts on memory errors.
Thanks for this patch.

> 
> Opinions welcome on whether the "try the write-and-read-back"
> approach in write_cpustate_to_list() is too hacky and it
> would be better to actually record whether write_list_to_cpustate()
> succeeded for each register. (That would require another array.)
> ---
>  target/arm/cpu.h |  9 -
>  target/arm/helper.c  | 27 +--
>  target/arm/kvm32.c   | 20 ++--
>  target/arm/kvm64.c   |  2 ++
>  target/arm/machine.c |  2 +-
>  5 files changed, 38 insertions(+), 22 deletions(-)
> 
> diff --git a/target/arm/cpu.h b/target/arm/cpu.h
> index 2a73fed9a01..c0c111378ea 100644
> --- a/target/arm/cpu.h
> +++ b/target/arm/cpu.h
> @@ -2321,18 +2321,25 @@ bool write_list_to_cpustate(ARMCPU *cpu);
>  /**
>   * write_cpustate_to_list:
>   * @cpu: ARMCPU
> + * @kvm_sync: true if this is for syncing back to KVM
>   *
>   * For each register listed in the ARMCPU cpreg_indexes list, write
>   * its value from the ARMCPUState structure into the cpreg_values list.
>   * This is used to copy info from TCG's working data structures into
>   * KVM or for outbound migration.
>   *
> + * @kvm_sync is true if we are doing this in order to sync the
> + * register state back to KVM. In this case we will only update
> + * values in the list if the previous list->cpustate sync actually
> + * successfully wrote the CPU state. Otherwise we will keep the value
> + * that is in the list.
> + *
>   * Returns: true if all register values were read correctly,
>   * false if some register was unknown or could not be read.
>   * Note that we do not stop early on failure -- we will attempt
>   * reading all registers in the list.
>   */
> -bool write_cpustate_to_list(ARMCPU *cpu);
> +bool write_cpustate_to_list(ARMCPU *cpu, bool kvm_sync);
>  
>  #define ARM_CPUID_TI915T  0x54029152
>  #define ARM_CPUID_TI925T  0x54029252
> diff --git a/target/arm/helper.c b/target/arm/helper.c
> index 0da1424f72d..bc2969eb4d8 100644
> --- a/target/arm/helper.c
> +++ b/target/arm/helper.c
> @@ -263,7 +263,7 @@ static bool raw_accessors_invalid(const ARMCPRegInfo *ri)
>  return true;
>  }
>  
> -bool write_cpustate_to_list(ARMCPU *cpu)
> +bool write_cpustate_to_list(ARMCPU *cpu, bool kvm_sync)
>  {
>  /* Write the coprocessor state from cpu->env to the (index,value) list. 
> */
>  int i;
> @@ -272,6 +272,7 @@ bool write_cpustate_to_list(ARMCPU *cpu)
>  for (i = 0; i < cpu->cpreg_array_len; i++) {
>  uint32_t regidx = kvm_to_cpreg_id(cpu->cpreg_indexes[i]);
>  const ARMCPRegInfo *ri;
> +uint64_t newval;
>  
>  ri = get_arm_cp_reginfo(cpu->cp_regs, regidx);
>  if (!ri) {
> @@ -281,7 +282,29 @@ bool write_cpustate_to_list(ARMCPU *cpu)
>  if (ri->type & ARM_CP_NO_RAW) {
>  continue;
>  }
> -cpu->cpreg_values[i] = 

Re: [Qemu-devel] [RFC] arm: Allow system registers for KVM guests to be changed by QEMU code

[gengdongjiu]

On 2018/12/6 23:14, Peter Maydell wrote:
> ---
> I think this patch:
>  * should be necessary for the current KVM debug code to be able
>to actually set the ESR_EL1 for the guest when it wants to
>deliver a breakpoint exception to it
>  * will also be necessary for Dongjiu's patchset to inject
>external aborts on memory errors
> 
> I have tagged it "RFC" because I don't have a setup to test
> that; I've just given it a quick smoke test that it runs a
> VM OK. Please test this and check whether it really does
> fix the bugs I think it does :-)

Great! thanks Peter follow up this issue and patch to fix it.
I will test it and give response.

> 
> Opinions welcome on whether the "try the write-and-read-back"
> approach in write_cpustate_to_list() is too hacky and it
> would be better to actually record whether write_list_to_cpustate()
> succeeded for each register. (That would require another array.)

Re: [Qemu-devel] [PATCH RESEND v15 08/10] target-arm: kvm64: inject synchronous External Abort

[gengdongjiu]

> >>
> >> Hi Peter,
> >>   Thanks for the review and comments.
> >>
> >> >
> >> > On 8 November 2018 at 10:29, Dongjiu Geng  wrote:
> >> > > +bool write_part_cpustate_to_list(ARMCPU *cpu, ptrdiff_t
> >> > > +fieldoffset)
> >> >
> >> > What is this about? Nothing else in QEMU needs to mess with the
> >> > cpustate synchronization. My first assumption is that you should not 
> >> > need to do so either.
> >>
> >> We should change the guest CP15 ESR_EL1's value, the only method is
> >> to change the cpu->cpreg_values[] in QEMU, then QEMU call 
> >> write_list_to_kvmstate() to set the cpu->cpreg_values[] to KVM which
> include the specified ESR_EL1 value, KVM do world switch, and then set the 
> specified ESR_EL1's value to guest kernel.
> >
> > Ah, I see. This is a bug in our current handling of the register
> > state, where we implicitly assume that nothing in QEMU will ever want
> > to change any system register values. This assumption is now false --
> > kvm_arm_handle_debug() broke it -- so we need to fix the code that
> > does kvm_arch_put_registers(). There is a comment in the kvm32.c
> > version of that function about this. (The kvm64.c version has the same
> > assumption but doesn't comment on it.)
> >
> > We should (ideally) fix this bug in the code that does register
> > syncing, without requiring places in QEMU that update system registers
> > to have to manually indicate which registers they have changed. I'll
> > have a think about how best to do this.
> >
> >> About the detailed explanation, as shown in [2].
> >>
> >> kvm_arm_handle_debug() does not need to do this because QEMU does not need 
> >> to change CP15 registers, such as ESR_EL1.
> >
> > kvm_arm_handle_debug does change ESR_EL1: it is injecting an exception
> > and so should set the exception register. This happens when it calls
> > the do_interrupt() hook, because arm_cpu_do_interrupt_aarch64() writes
> > to env->cp15.esr_el[new_el].
> >
> > I'm not entirely sure why this is working today, in fact.
> > Alex, did you test whether our debug-exception-injection reports the
> > correct ESR_EL1 to the guest ?
> 
> 
> I did not - I was mostly focusing in the host-debugging-the-guest test case. 
> I'll get a test rig up and check.

Thanks, please test it in the KVM mode, not in the TCG mode. Waiting for your 
test result.

> 
> --
> Alex Bennée

Re: [Qemu-devel] [Qemu-arm] [PATCH RESEND v15 10/10] target-arm: kvm64: handle SIGBUS signal from kernel or KVM

[gengdongjiu]

Hi Peter,
   Thanks for the comments.

> On Fri, 23 Nov 2018 at 14:31, gengdongjiu  wrote:
> >
> > Hi Peter,
> >   Thanks for the comments and mail.
> >
> > >
> > > On 22 November 2018 at 10:28, Peter Maydell  
> > > wrote:
> > > > On 22 November 2018 at 03:05, gengdongjiu  
> > > > wrote:
> > > >>> >
> > > >>> Shouldn't there be something in here to say "only report this error 
> > > >>> to the guest if we are actually reporting RAS errors to the
> guest" ?
> > > >>
> > > >> Yes, We can say something that such as "report this error to the
> > > >> guest", because this error is indeed triggered by guest, which is
> > > >> guest
> > > error.
> > > >
> > > > I'm afraid I don't really understand what you mean. Could you try
> > > > rephrasing it?
> > > >
> > > > My understanding was:
> > > >  * we get this signal if there is a RAS error in the host memory
> > > >  * if we are exposing RAS errors to the guest (ie we have
> > > >told it that in the ACPI table we passed it at startup)
> > > >then we should pass on this error to the guest
> > > >
> > > > but that these are two different conditions.
> > > >
> > > > If the host hardware detects a RAS error in memory used by the
> > > > guest but the guest is not being told about RAS errors, then we
> > > > cannot report the error: we have no mechanism to do so, and the
> > > > guest is not expecting it.
> > >
> > > If you look at the x86 version of this function you can see that it
> > > tests (env->mcg_cap & MCG_SER_P), which I think is the equivalent x86 "is 
> > > the guest CPU/config one we can report these errors to"
> test.
> >
> > MCG_SER_P (software error recovery support present) flag indicates (when 
> > set) that the processor supports software error recovery.
> > env->mcg_cap 's value should be got from KVM as shown in the QEMU code[1], 
> > it indicates whether the KVM support software error
> recovery.
> >
> > [1]:
> > -
> >   ret = kvm_get_mce_cap_supported(cs->kvm_state, _cap, );
> >   if (ret < 0) {
> >fprintf(stderr, "kvm_get_mce_cap_supported: %s", strerror(-ret));
> >return ret;
> >}
> > --
> > ---
> 
> Yes, but if you look at the code which calls that, it goes on to do:
> env->mcg_cap &= mcg_cap | MCG_CAP_BANKS_MASK;
> 
> which means that if the host kernel does not support this feature then we 
> will clear those bits in the env->mcg_cap field, so we do not
> advertise it to the guest. But we might be not advertising it to the guest at 
> all, if env->mcg_cap was 0 before this code was called. That
> happens if we are presenting the guest with a guest CPU type which does not 
> have the feature.

So you mean "env->mcg_cap" uses a bit to indicate that whether guest CPU 
support error recovery, right? If so, how we know whether guest CPU have this 
feature?
Should we initialized MCG_SER_P bit of env->mcg_cap to 1 or 0 before 
initializing the vcpu?  Please together see the reply in the [2], thanks.


> 
> --
> > void kvm_arch_on_sigbus_vcpu(CPUState *c, int code, void *addr) {
> > ...
> >
> > if (ram_addr != RAM_ADDR_INVALID &&
> > kvm_physical_memory_addr_from_host(c->kvm_state, addr,
> > )) {
> >
> >   If it got to here, it means the host hardware detects a RAS error in 
> > memory used by the guest using above two judgments.
> >   Maybe we can test/check whether KVM supports software error recovery
> > in [3]
> >
> 
> The question is not "does the host CPU / KVM support error reporting". It is 
> "does the *guest* CPU / system support error reporting".
> These are distinct questions which may not have the same answer.

[2]:
But QEMU should not know whether *guest* CPU / system support error reporting. 
From my understanding, in the X86 platform, if host CPU / KVM support error 
reporting,
then it will think guest CPU supports error reporting, and set the MCG_SER_P 
bit of env->mcg_cap to 1 

> 
> 
> thanks
> -- PMM

Re: [Qemu-devel] [PATCH RESEND v15 08/10] target-arm: kvm64: inject synchronous External Abort

[gengdongjiu]

Hi Peter,

On 2018/11/24 2:45, Peter Maydell wrote:
> On Wed, 21 Nov 2018 at 14:34, gengdongjiu  wrote:
>>
>> Hi Peter,
>>   Thanks for the review and comments.
>>
>>>
>>> On 8 November 2018 at 10:29, Dongjiu Geng  wrote:
>>>> +bool write_part_cpustate_to_list(ARMCPU *cpu, ptrdiff_t fieldoffset)
>>>
>>> What is this about? Nothing else in QEMU needs to mess with the cpustate 
>>> synchronization. My first assumption is that you should not
>>> need to do so either.
>>
>> We should change the guest CP15 ESR_EL1's value, the only method is to 
>> change the cpu->cpreg_values[] in QEMU, then QEMU call 
>> write_list_to_kvmstate()
>> to set the cpu->cpreg_values[] to KVM which include the specified ESR_EL1 
>> value, KVM do world switch, and then set the specified ESR_EL1's value to 
>> guest kernel.
> 
> Ah, I see. This is a bug in our current handling of the register
> state, where we implicitly assume that nothing in QEMU will ever
> want to change any system register values. This assumption is
> now false -- kvm_arm_handle_debug() broke it -- so we need to
> fix the code that does kvm_arch_put_registers(). There is a comment
> in the kvm32.c version of that function about this. (The kvm64.c
> version has the same assumption but doesn't comment on it.)
> 
> We should (ideally) fix this bug in the code that does register
> syncing, without requiring places in QEMU that update system
> registers to have to manually indicate which registers they have
> changed. I'll have a think about how best to do this.

Ok, it is great that you will think about it, waiting for your wonderful 
solution

> 
>> About the detailed explanation, as shown in [2].
>>
>> kvm_arm_handle_debug() does not need to do this because QEMU does not need 
>> to change CP15 registers, such as ESR_EL1.
> 
> kvm_arm_handle_debug does change ESR_EL1: it is injecting an exception
> and so should set the exception register. This happens when it
> calls the do_interrupt() hook, because arm_cpu_do_interrupt_aarch64()
> writes to env->cp15.esr_el[new_el].

Yes, I see it, but the env->cp15.esr_el[new_el] shouldn't be successfully set 
to KVM when call kvm_arch_put_registers()

> 
> I'm not entirely sure why this is working today, in fact.
> Alex, did you test whether our debug-exception-injection
> reports the correct ESR_EL1 to the guest ?

Alex?

> 
>>>> +/* Inject synchronous external abort */ static void
>>>> +kvm_inject_arm_sea(CPUState *c) {
>>>> +ARMCPU *cpu = ARM_CPU(c);
>>>> +CPUARMState *env = >env;
>>>> +CPUClass *cc = CPU_GET_CLASS(c);
>>>> +uint32_t esr;
>>>> +int ret;
>>>> +
>>>> +/* This exception is synchronous data abort */
>>>> +c->exception_index = EXCP_DATA_ABORT;
>>>> +/* Inject the exception to guest EL1 */
>>>> +env->exception.target_el = 1;
>>>
>>> These comments don't tell us anything that the code does not.
>>
>>  Thanks, do you mean I need to remove it or add more detailed comments to it?
> 
> As a rule of thumb, comments should provide information to
> the reader which they wouldn't get if they only had the code.
> Comments often answer the "why do we do this" question, or
> provide an overall summary of what the code is going to do,
> or refer to an external source (a datasheet, an algorithm)
> that is necessary to understand the code. It's better to
> avoid comments that say "what the code is doing" at a line-by-line
> level, because the code itself already answers the "what"
> question at that level of detail.

sure, got it, thanks for the explanation and guidelines.

> 
> thanks
> -- PMM
> 
> .
> 

Re: [Qemu-devel] [PATCH RESEND v15 10/10] target-arm: kvm64: handle SIGBUS signal from kernel or KVM

[gengdongjiu]

Hi Peter,
  Thanks for the comments and mail.

> 
> On 22 November 2018 at 10:28, Peter Maydell  wrote:
> > On 22 November 2018 at 03:05, gengdongjiu  wrote:
> >>> >
> >>> Shouldn't there be something in here to say "only report this error to 
> >>> the guest if we are actually reporting RAS errors to the guest" ?
> >>
> >> Yes, We can say something that such as "report this error to the guest", 
> >> because this error is indeed triggered by guest, which is guest
> error.
> >
> > I'm afraid I don't really understand what you mean. Could you try
> > rephrasing it?
> >
> > My understanding was:
> >  * we get this signal if there is a RAS error in the host memory
> >  * if we are exposing RAS errors to the guest (ie we have
> >told it that in the ACPI table we passed it at startup)
> >then we should pass on this error to the guest
> >
> > but that these are two different conditions.
> >
> > If the host hardware detects a RAS error in memory used by the guest
> > but the guest is not being told about RAS errors, then we cannot
> > report the error: we have no mechanism to do so, and the guest is not
> > expecting it.
> 
> If you look at the x86 version of this function you can see that it tests 
> (env->mcg_cap & MCG_SER_P), which I think is the equivalent x86 "is
> the guest CPU/config one we can report these errors to" test.

MCG_SER_P (software error recovery support present) flag indicates (when set) 
that the processor supports software error recovery.
env->mcg_cap 's value should be got from KVM as shown in the QEMU code[1], it 
indicates whether the KVM support software error recovery.

[1]:
-
  ret = kvm_get_mce_cap_supported(cs->kvm_state, _cap, );
  if (ret < 0) {
   fprintf(stderr, "kvm_get_mce_cap_supported: %s", strerror(-ret));
   return ret;
   }
-





[2]:
--
void kvm_arch_on_sigbus_vcpu(CPUState *c, int code, void *addr)
{
...

if (ram_addr != RAM_ADDR_INVALID &&
kvm_physical_memory_addr_from_host(c->kvm_state, addr, )) {

  If it got to here, it means the host hardware detects a RAS error in memory 
used by the guest using above two judgments.
  Maybe we can test/check whether KVM supports software error recovery in [3]
   
}
return;
.
}
}
---

[3]:
If we check it as x86, we can check that whether KVM supports software recovery 
though IOCTL, for example.

int r = kvm_ioctl(s, XX_GET_SER_CAP_SUPPORTED, seg_cap);
--
if (seg_cap && addr) {
   
}

> 
> thanks
> -- PMM

[Qemu-devel] 答复: [PATCH RESEND v15 10/10] target-arm: kvm64: handle SIGBUS signal from kernel or KVM

[gengdongjiu]

> >
> > +void kvm_arch_on_sigbus_vcpu(CPUState *c, int code, void *addr) {
> > +ram_addr_t ram_addr;
> > +hwaddr paddr;
> > +
> > +assert(code == BUS_MCEERR_AR || code == BUS_MCEERR_AO);
> > +if (addr) {
> > +ram_addr = qemu_ram_addr_from_host(addr);
> > +if (ram_addr != RAM_ADDR_INVALID &&
> > +kvm_physical_memory_addr_from_host(c->kvm_state, addr, 
> > )) {
> > +kvm_hwpoison_page_add(ram_addr);
> > +if (code == BUS_MCEERR_AR) {
> > +kvm_cpu_synchronize_state(c);
> > +if (ghes_record_errors(ACPI_HEST_NOTIFY_SEA, paddr)) {
> > +kvm_inject_arm_sea(c);
> > +} else {
> > +fprintf(stderr, "failed to record the error\n");
> > +}
> 
> Shouldn't there be something in here to say "only report this error to the 
> guest if we are actually reporting RAS errors to the guest" ?

Yes, We can say something that such as "report this error to the guest", 
because this error is indeed triggered by guest, which is guest error.

> 
> > +}
> > +return;
> > +}
> > +fprintf(stderr, "Hardware memory error for memory used by "
> > +"QEMU itself instead of guest system!\n");
> > +}
> > +
> > +if (code == BUS_MCEERR_AR) {
> > +fprintf(stderr, "Hardware memory error!\n");
> > +exit(1);
> > +}
> > +}
> > +
> >  /* C6.6.29 BRK instruction */
> >  static const uint32_t brk_insn = 0xd420;
> 
> thanks
> -- PMM

Re: [Qemu-devel] [PATCH RESEND v15 08/10] target-arm: kvm64: inject synchronous External Abort

[gengdongjiu]

Hi Peter,
  Thanks for the review and comments.

> 
> On 8 November 2018 at 10:29, Dongjiu Geng  wrote:
> > Add synchronous external abort injection logic, setup exception type
> > and syndrome value. When switch to guest, guest will jump to the
> > synchronous external abort vector table entry.
> >
> > The ESR_ELx.DFSC is set to synchronous external abort(0x10), and
> > ESR_ELx.FnV is set to not valid(0x1), which will tell guest that FAR
> > is not valid and holds an UNKNOWN value.
> > These value will be set to KVM register structures through
> > KVM_SET_ONE_REG IOCTL.
> >
> > Signed-off-by: Dongjiu Geng 
> > ---
> > Marc is against that KVM inject the synchronous external abort(SEA) in
> > [1], so user space how to inject it. The test result that injection
> > SEA to guest by Qemu is shown in [2].
> >
> > [1]: https://lkml.org/lkml/2017/3/2/110
> > [2]:
> > Taking exception 4 [Data Abort]
> > ...from EL0 to EL1
> > ...with ESR 0x24/0x92000410
> > ...with FAR 0x0
> > ...with ELR 0x40cf04
> > ...to EL1 PC 0xffc84c00 PSTATE 0x3c5 after kvm_inject_arm_sea
> > Unhandled fault: synchronous external abort (0x92000410) at
> > 0x007fa234c12c
> > CPU: 0 PID: 536 Comm: devmem Not tainted 4.1.0+ #20 Hardware name:
> > linux,dummy-virt (DT)
> > task: ffc019ab2b00 ti: ffc008134000 task.ti: ffc008134000
> > PC is at 0x40cf04 LR is at 0x40cdec pc : [<0040cf04>] lr :
> > [<0040cdec>] pstate: 6000 sp : 007ff7b24130
> > x29: 007ff7b24260 x28: 
> > x27: 00ad x26: 0049c000
> > x25: 0048904b x24: 0049c000
> > x23: 4060 x22: 007ff7b243a0
> > x21: 0002 x20: 
> > x19: 0020 x18: 
> > x17: 0049c6d0 x16: 007fa22c85c0
> > x15: 5798 x14: 007fa2205f1c
> > x13: 007fa241ccb0 x12: 0137
> > x11:  x10: 
> > x9 :  x8 : 00de
> > x7 :  x6 : 2000
> > x5 : 4060 x4 : 0003
> > x3 : 0001 x2 : 
> > x1 :  x0 : 007fa2418000
> > ---
> >  target/arm/cpu.h   |  2 ++
> >  target/arm/helper.c| 23 +++
> >  target/arm/internals.h |  5 +++--
> >  target/arm/kvm64.c | 39 +++
> >  target/arm/op_helper.c |  2 +-
> >  5 files changed, 68 insertions(+), 3 deletions(-)
> >
> > diff --git a/target/arm/cpu.h b/target/arm/cpu.h index
> > b5eff79..502507d 100644
> > --- a/target/arm/cpu.h
> > +++ b/target/arm/cpu.h
> > @@ -2331,6 +2331,8 @@ bool write_list_to_cpustate(ARMCPU *cpu);
> >   */
> >  bool write_cpustate_to_list(ARMCPU *cpu);
> >
> > +bool write_part_cpustate_to_list(ARMCPU *cpu, ptrdiff_t fieldoffset);
> > +
> >  #define ARM_CPUID_TI915T  0x54029152
> >  #define ARM_CPUID_TI925T  0x54029252
> >
> > diff --git a/target/arm/helper.c b/target/arm/helper.c index
> > 9630193..df078ff 100644
> > --- a/target/arm/helper.c
> > +++ b/target/arm/helper.c
> > @@ -263,6 +263,29 @@ static bool raw_accessors_invalid(const ARMCPRegInfo 
> > *ri)
> >  return true;
> >  }
> >
> > +bool write_part_cpustate_to_list(ARMCPU *cpu, ptrdiff_t fieldoffset)
> > +{
> > +const ARMCPRegInfo *ri;
> > +uint32_t regidx, i;
> > +
> > +for (i = 0; i < cpu->cpreg_array_len; i++) {
> > +regidx = kvm_to_cpreg_id(cpu->cpreg_indexes[i]);
> > +ri = get_arm_cp_reginfo(cpu->cp_regs, regidx);
> > +if (!ri) {
> > +continue;
> > +}
> > +
> > +if (ri->type & ARM_CP_NO_RAW) {
> > +continue;
> > +}
> > +if (ri->fieldoffset == fieldoffset) {
> > +cpu->cpreg_values[i] = read_raw_cp_reg(>env, ri);
> > +return true;
> > +}
> > +}
> > +return false;
> > +}
> 
> What is this about? Nothing else in QEMU needs to mess with the cpustate 
> synchronization. My first assumption is that you should not
> need to do so either.

We should change the guest CP15 ESR_EL1's value, the only method is to change 
the cpu->cpreg_values[] in QEMU, then QEMU call write_list_to_kvmstate()
to set the cpu->cpreg_values[] to KVM which include the specified ESR_EL1 
value, KVM do world switch, and then set the specified ESR_EL1's value to guest 
kernel.
About the detailed explanation, as shown in [2].

kvm_arm_handle_debug() does not need to do this because QEMU does not need to 
change CP15 registers, such as ESR_EL1.

> 
> > +
> >  bool write_cpustate_to_list(ARMCPU *cpu)  {
> >  /* Write the coprocessor state from cpu->env to the (index,value)
> > list. */ diff --git a/target/arm/internals.h b/target/arm/internals.h
> > index 6c2bb2d..04ea074 100644
> > --- a/target/arm/internals.h
> > +++ b/target/arm/internals.h
> > @@ -415,13 +415,14 @@ static inline uint32_t syn_insn_abort(int same_el, 
> > int ea, int s1ptw, int fsc)
> >  | 

Re: [Qemu-devel] [PATCH RESEND v15 09/10] hw/arm/virt: Add RAS platform version for migration

[gengdongjiu]

> 
> On 8 November 2018 at 10:29, Dongjiu Geng  wrote:
> > Support this feature since version 2.12, disable it by default in the
> > old version.
> >
> > Signed-off-by: Dongjiu Geng 
> > ---
> > Address Shannon's comments to add platform version in [1].
> >
> > [1]: https://lkml.org/lkml/2017/8/25/821
> > ---
> >  hw/arm/virt-acpi-build.c | 14 +-
> >  hw/arm/virt.c|  4 
> >  include/hw/arm/virt.h|  1 +
> >  3 files changed, 14 insertions(+), 5 deletions(-)
> >
> > diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c index
> > 19c1b7e..6f50a29 100644
> > --- a/hw/arm/virt-acpi-build.c
> > +++ b/hw/arm/virt-acpi-build.c
> > @@ -836,10 +836,11 @@ void virt_acpi_build(VirtMachineState *vms, 
> > AcpiBuildTables *tables)
> >  acpi_add_table(table_offsets, tables_blob);
> >  build_spcr(tables_blob, tables->linker, vms);
> >
> > -acpi_add_table(table_offsets, tables_blob);
> > -build_hardware_error_table(tables->hardware_errors, tables->linker);
> > -build_apei_hest(tables_blob, tables->hardware_errors, tables->linker);
> > -
> > +if (!vmc->no_ras) {
> > +acpi_add_table(table_offsets, tables_blob);
> > +build_hardware_error_table(tables->hardware_errors, 
> > tables->linker);
> > +build_apei_hest(tables_blob, tables->hardware_errors, 
> > tables->linker);
> > +}
> 
> This looks odd. If we need to gate the addition of the table on whether the 
> machine type asks for it, then we need the patch which adds the
> no_ras flag to come first. Otherwise there's a point in the commit history 
> where we add the tables even for the older machine types.

Ok, thanks, I will move this patch to the front.

> 
> >  if (nb_numa_nodes > 0) {
> >  acpi_add_table(table_offsets, tables_blob); @@ -926,6 +927,7
> > @@ static const VMStateDescription vmstate_virt_acpi_build = {
> >
> >  void virt_acpi_setup(VirtMachineState *vms)  {
> > +VirtMachineClass *vmc = VIRT_MACHINE_GET_CLASS(vms);
> >  AcpiBuildTables tables;
> >  AcpiBuildState *build_state;
> >
> > @@ -957,7 +959,9 @@ void virt_acpi_setup(VirtMachineState *vms)
> >  fw_cfg_add_file(vms->fw_cfg, ACPI_BUILD_TPMLOG_FILE, 
> > tables.tcpalog->data,
> >  acpi_data_len(tables.tcpalog));
> >
> > -ghes_add_fw_cfg(vms->fw_cfg, tables.hardware_errors);
> > +if (!vmc->no_ras) {
> > +ghes_add_fw_cfg(vms->fw_cfg, tables.hardware_errors);
> > +}
> >
> >  build_state->rsdp_mr = acpi_add_rom_blob(build_state, tables.rsdp,
> >ACPI_BUILD_RSDP_FILE,
> > 0); diff --git a/hw/arm/virt.c b/hw/arm/virt.c index a2b8d8f..367306b
> > 100644
> > --- a/hw/arm/virt.c
> > +++ b/hw/arm/virt.c
> > @@ -1920,6 +1920,10 @@ static void virt_machine_2_11_options(MachineClass 
> > *mc)
> >  virt_machine_2_12_options(mc);
> >  SET_MACHINE_COMPAT(mc, VIRT_COMPAT_2_11);
> >  vmc->smbios_old_sys_ver = true;
> > +/* Disable memory recovery feature for 2.11 as RAS support was
> > + * introduced with 2.12.
> > + */
> > +vmc->no_ras = true;
> 
> RAS support wasn't introduced with QEMU 2.12, or even with 3.0 or 3.1. The 
> earliest this will be added will be 4.0, so we need to update
> this code accordingly.

Got it, I will change it to the latest version instead of 2.12.

> 
> >  }
> >  DEFINE_VIRT_MACHINE(2, 11)
> >
> > diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt.h index
> > 4cc57a7..a4490dd 100644
> > --- a/include/hw/arm/virt.h
> > +++ b/include/hw/arm/virt.h
> > @@ -98,6 +98,7 @@ typedef struct {
> >  bool disallow_affinity_adjustment;
> >  bool no_its;
> >  bool no_pmu;
> > +bool no_ras;
> >  bool claim_edge_triggered_timers;
> >  bool smbios_old_sys_ver;
> >  bool no_highmem_ecam;
> > --
> > 1.8.3.1
> >
> 
> thanks
> -- PMM

[Qemu-devel] 答复: [PATCH RESEND v15 07/10] KVM: Move related hwpoison page functions to accel/kvm/ folder

[gengdongjiu]

Hi Peter,
   Thanks very much for the commands and review.

> On 8 November 2018 at 10:29, Dongjiu Geng  wrote:
> > kvm_hwpoison_page_add() and kvm_unpoison_all() will be used both by
> > X86 and ARM platforms, so move these functions to a common accel/kvm/
> > folder to avoid duplicate code.
> >
> > Signed-off-by: Dongjiu Geng 
> > ---
> > Address Peter's comments to move related hwpoison page function to
> > accel/kvm folder in [1] Address Paolo's comments to move HWPoisonPage
> > definition back to accel/kvm/kvm-all.c
> >
> > [1]:
> > https://lists.gnu.org/archive/html/qemu-arm/2017-09/msg00077.html
> > https://lists.gnu.org/archive/html/qemu-arm/2017-09/msg00152.html
> > ---
> 
> > --- a/include/exec/ram_addr.h
> > +++ b/include/exec/ram_addr.h
> > @@ -115,6 +115,11 @@ void qemu_ram_free(RAMBlock *block);
> >
> >  int qemu_ram_resize(RAMBlock *block, ram_addr_t newsize, Error
> > **errp);
> >
> > +/* Add a poisoned page to the list */ void
> > +kvm_hwpoison_page_add(ram_addr_t ram_addr);
> > +/* Free and remove all the poisoned pages in the list */ void
> > +kvm_unpoison_all(void *param);
> > +
> 
> In my previous review comments I said:
> >>> Any new globally-visible function prototype in a header should have
> >>> a doc-comment formatted documentation comment, please.
> >>
> >>
> >> Ok, thanks for this reminder. Do you mean I need to add comments for
> >> this globally-visible function, such as below:
> >>
> >> /*
> >>  * xx
> >>  */
> >> void kvm_hwpoison_page_add(ram_addr_t ram_addr);
> >
> > It should be in the doc-comment format, which begins "/**" and has
> > some stylization of how you list parameters and so on. Lots of
> > examples in the existing headers.
> 
> Can we have a doc-comment in the proper format and with a little more detail 
> than a single line, please?

Sure, I will add it in the proper format, thanks, May be I forget it. 

> 
> thanks
> -- PMM

[Qemu-devel] QEMU bootup hang in tcg model using mainline QEMU code

[gengdongjiu]

Hi All
  using mainline QEMU code and below boot up commands, the QEMU will be hang 
when do bootup, if I add the "-enable-kvm" parameter, it can boot up 
successfully,  do you know the reason about it?
I also try the "remotes/origin/stable-2.12" branch code using below same boot 
up commands, it does not have this hang issue. it seems the mainline code has 
some issue.


./qemu-system-aarch64  -m 1024 -cpu cortex-a57 -machine virt,gic-version=3 \
  -smp 4 -nographic -kernel Image -append "root=/dev/sda1 console=ttyAMA0 
sched_debug ignore_loglevel" \
  -device virtio-scsi-device,id=scsi -drive 
file=./linaro.img,id=rootimg,cache=unsafe,if=none -device scsi-hd,drive=rootimg

Re: [Qemu-devel] [PATCH RFC v11 0/2] add support for VCPU event states

[gengdongjiu]

Hi peter,

Thanks very much for this comments and very sorry for my late response due 
to business trave.

 Yes, the reason that tag the patchset to RFC is that it just  hasn't been 
tested on 32-bit. if someone else can test it on 32-bit platform, it will be 
great.

I do not have 32-bit environment, so only tested it on 64-bit.


On 27 September 2018 at 17:55, Dongjiu Geng  wrote:
> Support KVM_GET/SET_VCPU_EVENTS to get/set the SError exception state, and
> support the state migration.
>
> Now the VCPU event only includes the SError exception status, it can be
> extended if needed. When do migration, If source machine has serror pending,
> the target machine is also needed to pend this serror regardless of whether
> target machine can support to set the serror syndrome.
>
> Note: Because I do not have arm32 environment, I only this patch in the KVM64,
> not test it in the KVM32. So I need someone else test it in the 32 bit KVM 
> platform.
> Thanks.

Hi; could you clarify why this patchset is tagged RFC, please?
Is it just that it hasn't been tested on 32-bit ?

thanks
-- PMM

Re: [Qemu-devel] [PATCH v10 2/2] target/arm: Add support for VCPU event states

[gengdongjiu]

...
> >  };
> > --
> > 2.7.4
> >
> 
> Why aren't kvm_get_vcpu_events() and kvm_put_vcpu_events() in
> target/arm/kvm.c, as I suggested in my last review? There's
> no need to duplicate them for kvm32.c and kvm64.c, afaict.

I misunderstand your previous mail meanings. Sorry for the mistake,
I will put kvm_get/set_vcpu_events() to target/arm/kvm.c to avoid the 
duplication.
Thanks again for your pointing out and review. 

> 
> Thanks,
> drew

Re: [Qemu-devel] [PATCH v9 2/2] target: arm: Add support for VCPU event states

[gengdongjiu]

Hi Andrew,
   Thanks for the review and comments.

[...]
> > @@ -600,6 +605,60 @@ int kvm_arm_cpreg_level(uint64_t regidx)
> >  #define AARCH64_SIMD_CTRL_REG(x)   (KVM_REG_ARM64 | KVM_REG_SIZE_U32 | \
> >   KVM_REG_ARM_CORE | KVM_REG_ARM_CORE_REG(x))
> >
> > +static int kvm_put_vcpu_events(ARMCPU *cpu) {
> > +CPUARMState *env = >env;
> > +struct kvm_vcpu_events events = {};
> > +int ret;
> > +
> > +if (!kvm_has_vcpu_events()) {
> > +return 0;
> > +}
> > +
> > +memset(, 0, sizeof(events));
> 
> nit: this memset is redundant with the '= {}' above. I'd probably keep it and 
> remove the '= {}' though.
  
Will remove the '= {}', thanks

> 
> > +events.exception.serror_pending = env->serror.pending;
> > +
> > +/* Inject SError to guest with specified syndrome if host kernel
> > + * supports it, otherwise inject SError without syndrome.
> > + */
> > +if (have_inject_serror_esr) {
> > +events.exception.serror_has_esr = env->serror.has_esr;
> > +events.exception.serror_esr = env->serror.esr;
> > +}
> > +
> > +ret = kvm_vcpu_ioctl(CPU(cpu), KVM_SET_VCPU_EVENTS, );
> > +if (ret) {
> > +error_report("failed to put vcpu events");
> > +}
> > +
> > +return ret;
> > +}
> > +
> > +static int kvm_get_vcpu_events(ARMCPU *cpu) {
> > +CPUARMState *env = >env;
> > +struct kvm_vcpu_events events;
> > +int ret;
> > +
> > +if (!kvm_has_vcpu_events()) {
> > +return 0;
> > +}
> > +
> > +memset(, 0, sizeof(events));
> > +ret = kvm_vcpu_ioctl(CPU(cpu), KVM_GET_VCPU_EVENTS, );
> > +
> 
> nit: no need for this blank line

Will remove this blank line

> 
> > +if (ret) {
> > +error_report("failed to get vcpu events");
> > +return ret;
> > +}
[...]
> > +
> >  static bool m_needed(void *opaque)
> >  {
> >  ARMCPU *cpu = opaque;
> > @@ -726,6 +747,7 @@ const VMStateDescription vmstate_arm_cpu = {
> > #ifdef TARGET_AARCH64
> >  _sve,
> >  #endif
> > +_serror,
> >  NULL
> >  }
> >  };
> > --
> > 1.8.3.1
> >
> >
> 
> Hmm. kvm_vcpu_events is defined for both 32-bit and 64-bit and the vmstate 
> isn't in the '#ifdef TARGET_AARCH64', like vmstate_sve, yet
> only the target/arm/kvm64.c file is getting updated. Shouldn't we put 
> kvm_get/put_vcpu_events() in target/arm/kvm.c and then also
> update target/arm/kvm32.c?

I will also update the target/arm/kvm32.c, thanks a lot for the suggestion.

> 
> Thanks,
> drew

Re: [Qemu-devel] [PATCH v9 2/2] target: arm: Add support for VCPU event states

[gengdongjiu]

Hi Peter/shannon,
   when you are free, hope you can have a look at this series of patches.
Thanks a lot for your time in advance.


On 2018/9/16 1:32, Dongjiu Geng wrote:
> This patch extends the qemu-kvm state sync logic with support for
> KVM_GET/SET_VCPU_EVENTS, giving access to yet missing SError exception.
> And also it can support the exception state migration.
> 
> The SError exception states include SError pending state and ESR value,
> the kvm_put/get_vcpu_events() will be called when set or get system
> registers. When do migration, if source machine has SError pending,
> QEMU will do this migration regardless whether the target machine supports
> to specify guest ESR value, because if target machine does not support that,
> it can also inject the SError with zero ESR value.
> 
> Signed-off-by: Dongjiu Geng 
> ---
> Change since v8:
> 1. Update the commit message
> 
> Change since v7:
> 1. Change "pending" and "has_esr" from uint32_t to uint8_t for CPUARMState
> 2. Add error_report() in kvm_get_vcpu_events()
> 
> Change since v6:
> 1. Add cover letter
> 2. Change name "cpu/ras" to "cpu/serror"
> 3. Add some comments and check the ioctl return value for 
> kvm_put_vcpu_events()
> 
> Change since v5:
> address Peter's comments:
> 1. Move the "struct serror" before the "end_reset_fields" in CPUARMState
> 2. Remove ARM_FEATURE_RAS_EXT and add a variable have_inject_serror_esr
> 3. Use the variable have_inject_serror_esr to track whether the kernel has 
> state we need to migrate
> 4. Remove printf() in kvm_arch_put_registers()
> 5. ras_needed/vmstate_ras to serror_needed/vmstate_serror
> 6. Check to use "return env.serror.pending != 0" instead of "arm_feature(env, 
> ARM_FEATURE_RAS_EXT)" in the ras_needed()
> 
> Change since v4:
> 1. Rebase the code to latest
> ---
>  target/arm/cpu.h |  7 ++
>  target/arm/kvm64.c   | 69 
> 
>  target/arm/machine.c | 22 +
>  3 files changed, 98 insertions(+)
> 
> diff --git a/target/arm/cpu.h b/target/arm/cpu.h
> index 62c36b4..034b035 100644
> --- a/target/arm/cpu.h
> +++ b/target/arm/cpu.h
> @@ -530,6 +530,13 @@ typedef struct CPUARMState {
>   */
>  } exception;
>  
> +/* Information associated with an SError */
> +struct {
> +uint8_t pending;
> +uint8_t has_esr;
> +uint64_t esr;
> +} serror;
> +
>  /* Thumb-2 EE state.  */
>  uint32_t teecr;
>  uint32_t teehbr;
> diff --git a/target/arm/kvm64.c b/target/arm/kvm64.c
> index e0b8246..e8705e2 100644
> --- a/target/arm/kvm64.c
> +++ b/target/arm/kvm64.c
> @@ -29,6 +29,7 @@
>  #include "hw/arm/arm.h"
>  
>  static bool have_guest_debug;
> +static bool have_inject_serror_esr;
>  
>  /*
>   * Although the ARM implementation of hardware assisted debugging
> @@ -546,6 +547,10 @@ int kvm_arch_init_vcpu(CPUState *cs)
>  
>  kvm_arm_init_debug(cs);
>  
> +/* Check whether userspace can specify guest syndrome value */
> +have_inject_serror_esr = kvm_check_extension(cs->kvm_state,
> + 
> KVM_CAP_ARM_INJECT_SERROR_ESR);
> +
>  return kvm_arm_init_cpreg_list(cpu);
>  }
>  
> @@ -600,6 +605,60 @@ int kvm_arm_cpreg_level(uint64_t regidx)
>  #define AARCH64_SIMD_CTRL_REG(x)   (KVM_REG_ARM64 | KVM_REG_SIZE_U32 | \
>   KVM_REG_ARM_CORE | KVM_REG_ARM_CORE_REG(x))
>  
> +static int kvm_put_vcpu_events(ARMCPU *cpu)
> +{
> +CPUARMState *env = >env;
> +struct kvm_vcpu_events events = {};
> +int ret;
> +
> +if (!kvm_has_vcpu_events()) {
> +return 0;
> +}
> +
> +memset(, 0, sizeof(events));
> +events.exception.serror_pending = env->serror.pending;
> +
> +/* Inject SError to guest with specified syndrome if host kernel
> + * supports it, otherwise inject SError without syndrome.
> + */
> +if (have_inject_serror_esr) {
> +events.exception.serror_has_esr = env->serror.has_esr;
> +events.exception.serror_esr = env->serror.esr;
> +}
> +
> +ret = kvm_vcpu_ioctl(CPU(cpu), KVM_SET_VCPU_EVENTS, );
> +if (ret) {
> +error_report("failed to put vcpu events");
> +}
> +
> +return ret;
> +}
> +
> +static int kvm_get_vcpu_events(ARMCPU *cpu)
> +{
> +CPUARMState *env = >env;
> +struct kvm_vcpu_events events;
> +int ret;
> +
> +if (!kvm_has_vcpu_events()) {
> +return 0;
> +}
> +
> +memset(, 0, sizeof(events));
> +ret = kvm_vcpu_ioctl(CPU(cpu), KVM_GET_VCPU_EVENTS, );
> +
> +if (ret) {
> +error_report("failed to get vcpu events");
> +return ret;
> +}
> +
> +env->serror.pending = events.exception.serror_pending;
> +env->serror.has_esr = events.exception.serror_has_esr;
> +env->serror.esr = events.exception.serror_esr;
> +
> +return 0;
> +}
> +
>  int kvm_arch_put_registers(CPUState *cs, int level)
>  {
>  struct kvm_one_reg reg;
> @@ -727,6 +786,11 @@ int 

[Qemu-devel] 答复: [Qemu-arm] [PATCH v7 2/2] target: arm: Add support for VCPU event states

[gengdongjiu]

Hi Shannon
   Ｉhave changed it according to your comments and repost the patches, thanks 
for the review.

> 
> Hi,
> 
> On 8/28/2018 4:46 AM, Dongjiu Geng wrote:
> > This patch extends the qemu-kvm state sync logic with support for
> > KVM_GET/SET_VCPU_EVENTS, giving access to yet missing SError exception.
> > And also it can support the exception state migration.
> >
> > Signed-off-by: Dongjiu Geng 
> > ---
> > change since v6:
> > 1. Add cover letter
> > 2. Change name "cpu/ras" to "cpu/serror"
> > 3. Add some comments and check the ioctl return value for
> > kvm_put_vcpu_events()
> >
> > Change since v5:
> > address Peter's comments:
> > 1. Move the "struct serror" before the "end_reset_fields" in
> > CPUARMState 2. Remove ARM_FEATURE_RAS_EXT and add a variable
> > have_inject_serror_esr 3. Use the variable have_inject_serror_esr to
> > track whether the kernel has state we need to migrate 4. Remove
> > printf() in kvm_arch_put_registers() 5. ras_needed/vmstate_ras to
> > serror_needed/vmstate_serror 6. Check to use "return
> > env.serror.pending != 0" instead of "arm_feature(env,
> > ARM_FEATURE_RAS_EXT)" in the ras_needed()
> >
> > Change since v4:
> > 1. Rebase the code to latest
> >
> > Change since v3:
> > 1. Add a new new subsection with a suitable 'ras_needed' function
> > controlling whether it is present
> > 2. Add a ARM_FEATURE_RAS feature bit for CPUARMState
> > ---
> >   target/arm/cpu.h |  7 ++
> >   target/arm/kvm64.c   | 68 
> > 
> >   target/arm/machine.c | 22 +
> >   3 files changed, 97 insertions(+)
> >
> > diff --git a/target/arm/cpu.h b/target/arm/cpu.h index
> > 62c36b4..7030680 100644
> > --- a/target/arm/cpu.h
> > +++ b/target/arm/cpu.h
> > @@ -530,6 +530,13 @@ typedef struct CPUARMState {
> >*/
> >   } exception;
> >
> > +/* Information associated with an SError */
> > +struct {
> > +uint32_t pending;
> > +uint32_t has_esr;
> why use uint32_t for a bool parameter here while it's u8 in the kernel header?
> 
> > +uint64_t esr;
> > +} serror;
> > +
> >   /* Thumb-2 EE state.  */
> >   uint32_t teecr;
> >   uint32_t teehbr;
> > diff --git a/target/arm/kvm64.c b/target/arm/kvm64.c index
> > e0b8246..6e0252c 100644
> > --- a/target/arm/kvm64.c
> > +++ b/target/arm/kvm64.c
> > @@ -29,6 +29,7 @@
> >   #include "hw/arm/arm.h"
> >
> >   static bool have_guest_debug;
> > +static bool have_inject_serror_esr;
> >
> >   /*
> >* Although the ARM implementation of hardware assisted debugging @@
> > -546,6 +547,10 @@ int kvm_arch_init_vcpu(CPUState *cs)
> >
> >   kvm_arm_init_debug(cs);
> >
> > +/* Check whether userspace can specify guest syndrome value */
> > +have_inject_serror_esr = kvm_check_extension(cs->kvm_state,
> > +
> > + KVM_CAP_ARM_INJECT_SERROR_ESR);
> > +
> >   return kvm_arm_init_cpreg_list(cpu);
> >   }
> >
> > @@ -600,6 +605,59 @@ int kvm_arm_cpreg_level(uint64_t regidx)
> >   #define AARCH64_SIMD_CTRL_REG(x)   (KVM_REG_ARM64 | KVM_REG_SIZE_U32 | \
> >KVM_REG_ARM_CORE | KVM_REG_ARM_CORE_REG(x))
> >
> > +static int kvm_put_vcpu_events(ARMCPU *cpu) {
> > +CPUARMState *env = >env;
> > +struct kvm_vcpu_events events = {};
> > +int ret;
> > +
> > +if (!kvm_has_vcpu_events()) {
> > +return 0;
> > +}
> > +
> > +memset(, 0, sizeof(events));
> > +events.exception.serror_pending = env->serror.pending;
> > +
> > +/* Inject SError to guest with specified syndrome if host kernel
> > + * supports it, otherwise inject SError without syndrome.
> > + */
> > +if (have_inject_serror_esr) {
> > +events.exception.serror_has_esr = env->serror.has_esr;
> > +events.exception.serror_esr = env->serror.esr;
> > +}
> > +
> > +ret = kvm_vcpu_ioctl(CPU(cpu), KVM_SET_VCPU_EVENTS, );
> > +if (ret) {
> > +error_report("failed to put vcpu events");
> > +}
> > +
> > +return ret;
> > +}
> > +
> > +static int kvm_get_vcpu_events(ARMCPU *cpu) {
> > +CPUARMState *env = >env;
> > +struct kvm_vcpu_events events;
> > +int ret;
> > +
> > +if (!kvm_has_vcpu_events()) {
> > +return 0;
> > +}
> > +
> > +memset(, 0, sizeof(events));
> > +ret = kvm_vcpu_ioctl(CPU(cpu), KVM_GET_VCPU_EVENTS, );
> > +
> > +if (ret < 0) {
> like the put function, it's better to add a error_report here.
> 
> Thanks,
> Shannon
> > +return ret;
> > +}
> > +
> > +env->serror.pending = events.exception.serror_pending;
> > +env->serror.has_esr = events.exception.serror_has_esr;
> > +env->serror.esr = events.exception.serror_esr;
> > +
> > +return 0;
> > +}
> > +
> >   int kvm_arch_put_registers(CPUState *cs, int level)
> >   {
> >   struct kvm_one_reg reg;
> > @@ -727,6 +785,11 @@ int kvm_arch_put_registers(CPUState *cs, int level)
> >   return ret;
> >   }
> >
> > +ret =