Re: [Qemu-devel] [PATCH v9 00/14] ARM SMMUv3 Emulation Support

[Auger Eric]

Hi Peter,

On 12/03/18 13:58, Peter Maydell wrote:
> On 28 February 2018 at 08:44, Auger Eric  wrote:
>> On 27/02/18 20:02, Peter Maydell wrote:
>>> On 17 February 2018 at 18:46, Eric Auger  wrote:
 This series implements the emulation code for ARM SMMUv3.
> 
>>> What state is this series in now? Is it "seems ready to
>>> go, needs review"? Are you hoping it might get in for 2.12,
>>> or targeting 2.13 ?
> 
>> Yes I think it is in a decent state and I would be happy to get some new
>> reviews, for a tentative pull in 2.12. In any case I will be reactive on
>> any comment in that prospect.
> 
> Hi, Eric. I've now gone through the patchset and reviewed it. Sorry
> I didn't manage to get to this in time to be able to put it into 2.12,
> but there weren't any major or structural issues, so we should be
> able to get this into the tree early in the 2.13 dev cycle.

No worries. I am already really grateful to you for the time you spent
on this comprehensive review.

I have taken into account most of your comments already. I will run some
more tests and post v10 this week then.

Thanks

Eric
> 
> thanks
> -- PMM
> 

Re: [Qemu-devel] [PATCH v9 00/14] ARM SMMUv3 Emulation Support

[Peter Maydell]

On 28 February 2018 at 08:44, Auger Eric  wrote:
> On 27/02/18 20:02, Peter Maydell wrote:
>> On 17 February 2018 at 18:46, Eric Auger  wrote:
>>> This series implements the emulation code for ARM SMMUv3.

>> What state is this series in now? Is it "seems ready to
>> go, needs review"? Are you hoping it might get in for 2.12,
>> or targeting 2.13 ?

> Yes I think it is in a decent state and I would be happy to get some new
> reviews, for a tentative pull in 2.12. In any case I will be reactive on
> any comment in that prospect.

Hi, Eric. I've now gone through the patchset and reviewed it. Sorry
I didn't manage to get to this in time to be able to put it into 2.12,
but there weren't any major or structural issues, so we should be
able to get this into the tree early in the 2.13 dev cycle.

thanks
-- PMM

Re: [Qemu-devel] [PATCH v9 00/14] ARM SMMUv3 Emulation Support

[Auger Eric]

Hi Peter,

On 27/02/18 20:02, Peter Maydell wrote:
> On 17 February 2018 at 18:46, Eric Auger  wrote:
>> This series implements the emulation code for ARM SMMUv3.
>>
>> SMMUv3 gets instantiated by adding ",iommu=smmuv3" to the virt
>> machine option.
>>
>> VHOST integration will be handled in a separate series. VFIO
>> integration is not targeted at the moment. Only stage 1 and
>> AArch64 PTW are supported.
>>
>> Main changes since v8:
>> - fix mingw compilation (qemu/log.h)
>> - put gpl v2 license on all files to respect initial license
>> - change proto of smmu_ptw* to clarify inputs/outputs and
>>   prepare for iotlb emulation
>> - fix hash table lookup
>> - cleanup access type handling during ptw
>> - cleanup reset infra (parent_reset)
>> - replace some inline functions by macros
>> - fix some CMD fields
>> - increment cmdq cons only after cmd execution
>> - replace some remaining error_report by qemu_log_mask
> 
> What state is this series in now? Is it "seems ready to
> go, needs review"? Are you hoping it might get in for 2.12,
> or targeting 2.13 ?
Yes I think it is in a decent state and I would be happy to get some new
reviews, for a tentative pull in 2.12. In any case I will be reactive on
any comment in that prospect.

Thanks

Eric
> 
> thanks
> -- PMM
> 

Re: [Qemu-devel] [PATCH v9 00/14] ARM SMMUv3 Emulation Support

[Peter Maydell]

On 17 February 2018 at 18:46, Eric Auger  wrote:
> This series implements the emulation code for ARM SMMUv3.
>
> SMMUv3 gets instantiated by adding ",iommu=smmuv3" to the virt
> machine option.
>
> VHOST integration will be handled in a separate series. VFIO
> integration is not targeted at the moment. Only stage 1 and
> AArch64 PTW are supported.
>
> Main changes since v8:
> - fix mingw compilation (qemu/log.h)
> - put gpl v2 license on all files to respect initial license
> - change proto of smmu_ptw* to clarify inputs/outputs and
>   prepare for iotlb emulation
> - fix hash table lookup
> - cleanup access type handling during ptw
> - cleanup reset infra (parent_reset)
> - replace some inline functions by macros
> - fix some CMD fields
> - increment cmdq cons only after cmd execution
> - replace some remaining error_report by qemu_log_mask

What state is this series in now? Is it "seems ready to
go, needs review"? Are you hoping it might get in for 2.12,
or targeting 2.13 ?

thanks
-- PMM

[Qemu-devel] [PATCH v9 00/14] ARM SMMUv3 Emulation Support

[Eric Auger]

This series implements the emulation code for ARM SMMUv3.

SMMUv3 gets instantiated by adding ",iommu=smmuv3" to the virt
machine option.

VHOST integration will be handled in a separate series. VFIO
integration is not targeted at the moment. Only stage 1 and
AArch64 PTW are supported.

Main changes since v8:
- fix mingw compilation (qemu/log.h)
- put gpl v2 license on all files to respect initial license
- change proto of smmu_ptw* to clarify inputs/outputs and
  prepare for iotlb emulation
- fix hash table lookup
- cleanup access type handling during ptw
- cleanup reset infra (parent_reset)
- replace some inline functions by macros
- fix some CMD fields
- increment cmdq cons only after cmd execution
- replace some remaining error_report by qemu_log_mask

Best Regards

Eric

This series can be found at:
v9: https://github.com/eauger/qemu/tree/v2.11.0-SMMU-v9
Previous version at:
v8: https://github.com/eauger/qemu/tree/v2.11.0-SMMU-v8

History:

v8 -> v9:
- see above description

v7 -> v8:
Took into account Peter's comments:
- revisit queue data structures
- use registerfields.h and got rid of reg array
- use dma_memory_read for all descriptor fetches
- got rid of page table walk for an iova range and
  implemented standard page table walk for single IOVA
- revisit event data structure
- report events in many more situations and pass the event
  handle all along the decode and ptw phases
- fix gerror/gerron computations
- completely got rid of stage2 decoding
- use a machine option for instantiation
- get rid of VFIO integration
- get rid of VHOST integration (this will be added in a
- abort in case vhost/vfio notifiers get detected
  second step together with TLB emulation)
- Tested migration
- fixed TTBR index computation (issue reported by Tomasz)

v6 -> v7:
- DPDK testpmd now running on guest with 2 assigned VFs
- Changed the instantiation method: add the following option to
  the QEMU command line
  -device smmu # for virtio/vhost use cases
  -device smmu,caching-mode # for vfio use cases (based on [1])
- splitted the series into smaller patches to allow the review
- the VFIO integration based on "ltlbi-on-map" smmuv3 driver
  is isolated from the rest: last 2 patches, not for upstream.
  This is shipped for testing/bench until a better solution is found.
- Reworked permission flag checks and event generation

v5 -> v6:
- Rebase on 2.10 and IOMMUMemoryRegion
- add ACPI TLBI_ON_MAP support (VFIO integration also works in
  ACPI mode)
- fix block replay
- handle implementation defined SMMU_CMD_TLBI_NH_VA_AM cmd
  (goes along with TLBI_ON_MAP FW quirk)
- replay systematically unmap the whole range first
- smmuv3_map_hook does not unmap anymore and the unmap is done
  before the replay
- add and use smmuv3_context_device_invalidate instead of
  blindly replaying everything

v4 -> v5:
- initial_level now part of SMMUTransCfg
- smmu_page_walk_64 takes into account the max input size
- implement sys->iommu_ops.replay and sys->iommu_ops.notify_flag_changed
- smmuv3_translate: bug fix: don't walk on bypass
- smmu_update_qreg: fix PROD index update
- I did not yet address Peter's comments as the code is not mature enough
  to be split into sub patches.

v3 -> v4 [Eric]:
- page table walk rewritten to allow scan of the page table within a
  range of IOVA. This prepares for VFIO integration and replay.
- configuration parsing partially reworked.
- do not advertise unsupported/untested features: S2, S1 + S2, HYP,
  PRI, ATS, ..
- added ACPI table generation
- migrated to dynamic traces
- mingw compilation fix

v2 -> v3 [Eric]:
- rebased on 2.9
- mostly code and patch reorganization to ease the review process
- optional patches removed. They may be handled separately. I am currently
  working on ACPI enablement.
- optional instantiation of the smmu in mach-virt
- removed [2/9] (fdt functions) since not mandated
- start splitting main patch into base and derived object
- no new function feature added

v1 -> v2 [Prem]:
- Adopted review comments from Eric Auger
- Make SMMU_DPRINTF to internally call qemu_log
(since translation requests are too many, we need control
 on the type of log we want)
- SMMUTransCfg modified to suite simplicity
- Change RegInfo to uint64 register array
- Code cleanup
- Test cleanups
- Reshuffled patches

v0 -> v1 [Prem]:
- As per SMMUv3 spec 16.0 (only is_ste_consistant() is noticeable)
- Reworked register access/update logic
- Factored out translation code for
- single point bug fix
- sharing/removal in future
- (optional) Unit tests added, with PCI test device
- S1 with 4k/64k, S1+S2 with 4k/64k
- (S1 or S2) only can be verified by Linux 4.7 driver
- (optional) Priliminary ACPI support

v0 [Prem]:
- Implements SMMUv3 spec 11.0
- Supported for PCIe devices,
- Command Queue and Event Queue supported
- LPAE only, S1 is supported and Tested, S2 not tested
- BE mode Translation not supported