Re: [EXT] Re: [PATCH v7 3/5] virtio-iommu: Call iommu notifier for attach/detach
On Wed, Mar 18, 2020 at 12:42:25PM +0100, Auger Eric wrote: > Hi Jean, > > On 3/18/20 12:20 PM, Bharat Bhushan wrote: > > > > > >> -Original Message- > >> From: Jean-Philippe Brucker > >> Sent: Wednesday, March 18, 2020 4:48 PM > >> To: Bharat Bhushan > >> Cc: Auger Eric ; Peter Maydell > >> ; kevin.t...@intel.com; Tomasz Nowicki [C] > >> ; m...@redhat.com; drjo...@redhat.com; > >> pet...@redhat.com; qemu-devel@nongnu.org; alex.william...@redhat.com; > >> qemu-...@nongnu.org; Bharat Bhushan ; > >> linuc.dec...@gmail.com; eric.auger@gmail.com > >> Subject: [EXT] Re: [PATCH v7 3/5] virtio-iommu: Call iommu notifier for > >> attach/detach > >> > >> External Email > >> > >> -- > >> On Wed, Mar 18, 2020 at 03:47:44PM +0530, Bharat Bhushan wrote: > >>> Hi Jean, > >>> > >>> On Tue, Mar 17, 2020 at 9:29 PM Jean-Philippe Brucker > >>> wrote: > >>>> > >>>> On Tue, Mar 17, 2020 at 02:46:55PM +0530, Bharat Bhushan wrote: > >>>>> Hi Jean, > >>>>> > >>>>> On Tue, Mar 17, 2020 at 2:23 PM Jean-Philippe Brucker > >>>>> wrote: > >>>>>> > >>>>>> On Tue, Mar 17, 2020 at 12:40:39PM +0530, Bharat Bhushan wrote: > >>>>>>> Hi Jean, > >>>>>>> > >>>>>>> On Mon, Mar 16, 2020 at 3:41 PM Jean-Philippe Brucker > >>>>>>> wrote: > >>>>>>>> > >>>>>>>> Hi Bharat, > >>>>>>>> > >>>>>>>> Could you Cc me on your next posting? Unfortunately I don't > >>>>>>>> have much hardware for testing this at the moment, but I > >>>>>>>> might be able to help a little on the review. > >>>>>>>> > >>>>>>>> On Mon, Mar 16, 2020 at 02:40:00PM +0530, Bharat Bhushan wrote: > >>>>>>>>>>>>> First issue is: your guest can use 4K page and your > >>>>>>>>>>>>> host can use 64KB pages. In that case VFIO_DMA_MAP > >>>>>>>>>>>>> will fail with -EINVAL. We must devise a way to pass the host > >> settings to the VIRTIO-IOMMU device. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Even with 64KB pages, it did not work for me. I have > >>>>>>>>>>>>> obviously not the storm of VFIO_DMA_MAP failures but > >>>>>>>>>>>>> I have some, most probably due to some wrong notifications > >> somewhere. I will try to investigate on my side. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Did you test with VFIO on your side? > >>>>>>>>>>>> > >>>>>>>>>>>> I did not tried with different page sizes, only tested with 4K > >>>>>>>>>>>> page > >> size. > >>>>>>>>>>>> > >>>>>>>>>>>> Yes it works, I tested with two n/w device assigned > >>>>>>>>>>>> to VM, both interfaces works > >>>>>>>>>>>> > >>>>>>>>>>>> First I will try with 64k page size. > >>>>>>>>>>> > >>>>>>>>>>> 64K page size does not work for me as well, > >>>>>>>>>>> > >>>>>>>>>>> I think we are not passing correct page_size_mask here > >>>>>>>>>>> (config.page_size_mask is set to TARGET_PAGE_MASK ( > >>>>>>>>>>> which is > >>>>>>>>>>> 0xf000)) > >>>>>>>>>> I guess you mean with guest using 4K and host using 64K. > >>>>>>>>>>> > >>>>>>>>>>> We need to set this correctly as per host page size, correct? > >>>>>>>>>> Yes that's correct. We need to put in place a control > >>>>>>>>>> path to retrieve the page settings on host through VFIO to inform > >>>>>>&
Re: [EXT] Re: [PATCH v7 3/5] virtio-iommu: Call iommu notifier for attach/detach
Hi Jean, On 3/18/20 12:20 PM, Bharat Bhushan wrote: > > >> -Original Message- >> From: Jean-Philippe Brucker >> Sent: Wednesday, March 18, 2020 4:48 PM >> To: Bharat Bhushan >> Cc: Auger Eric ; Peter Maydell >> ; kevin.t...@intel.com; Tomasz Nowicki [C] >> ; m...@redhat.com; drjo...@redhat.com; >> pet...@redhat.com; qemu-devel@nongnu.org; alex.william...@redhat.com; >> qemu-...@nongnu.org; Bharat Bhushan ; >> linuc.dec...@gmail.com; eric.auger@gmail.com >> Subject: [EXT] Re: [PATCH v7 3/5] virtio-iommu: Call iommu notifier for >> attach/detach >> >> External Email >> >> -- >> On Wed, Mar 18, 2020 at 03:47:44PM +0530, Bharat Bhushan wrote: >>> Hi Jean, >>> >>> On Tue, Mar 17, 2020 at 9:29 PM Jean-Philippe Brucker >>> wrote: >>>> >>>> On Tue, Mar 17, 2020 at 02:46:55PM +0530, Bharat Bhushan wrote: >>>>> Hi Jean, >>>>> >>>>> On Tue, Mar 17, 2020 at 2:23 PM Jean-Philippe Brucker >>>>> wrote: >>>>>> >>>>>> On Tue, Mar 17, 2020 at 12:40:39PM +0530, Bharat Bhushan wrote: >>>>>>> Hi Jean, >>>>>>> >>>>>>> On Mon, Mar 16, 2020 at 3:41 PM Jean-Philippe Brucker >>>>>>> wrote: >>>>>>>> >>>>>>>> Hi Bharat, >>>>>>>> >>>>>>>> Could you Cc me on your next posting? Unfortunately I don't >>>>>>>> have much hardware for testing this at the moment, but I >>>>>>>> might be able to help a little on the review. >>>>>>>> >>>>>>>> On Mon, Mar 16, 2020 at 02:40:00PM +0530, Bharat Bhushan wrote: >>>>>>>>>>>>> First issue is: your guest can use 4K page and your >>>>>>>>>>>>> host can use 64KB pages. In that case VFIO_DMA_MAP >>>>>>>>>>>>> will fail with -EINVAL. We must devise a way to pass the host >> settings to the VIRTIO-IOMMU device. >>>>>>>>>>>>> >>>>>>>>>>>>> Even with 64KB pages, it did not work for me. I have >>>>>>>>>>>>> obviously not the storm of VFIO_DMA_MAP failures but >>>>>>>>>>>>> I have some, most probably due to some wrong notifications >> somewhere. I will try to investigate on my side. >>>>>>>>>>>>> >>>>>>>>>>>>> Did you test with VFIO on your side? >>>>>>>>>>>> >>>>>>>>>>>> I did not tried with different page sizes, only tested with 4K page >> size. >>>>>>>>>>>> >>>>>>>>>>>> Yes it works, I tested with two n/w device assigned >>>>>>>>>>>> to VM, both interfaces works >>>>>>>>>>>> >>>>>>>>>>>> First I will try with 64k page size. >>>>>>>>>>> >>>>>>>>>>> 64K page size does not work for me as well, >>>>>>>>>>> >>>>>>>>>>> I think we are not passing correct page_size_mask here >>>>>>>>>>> (config.page_size_mask is set to TARGET_PAGE_MASK ( >>>>>>>>>>> which is >>>>>>>>>>> 0xf000)) >>>>>>>>>> I guess you mean with guest using 4K and host using 64K. >>>>>>>>>>> >>>>>>>>>>> We need to set this correctly as per host page size, correct? >>>>>>>>>> Yes that's correct. We need to put in place a control >>>>>>>>>> path to retrieve the page settings on host through VFIO to inform the >> virtio-iommu device. >>>>>>>>>> >>>>>>>>>> Besides this issue, did you try with 64kB on host and guest? >>>>>>>>> >>>>>>>>> I tried Followings >>>>>>>>> - 4k host and 4k guest - it works with v7 version >>>>>>>>> - 64k host and 64k guest - it does not work with v7 >>>>>>>>> hard-coded config.page_size_mask to 0xff
RE: [EXT] Re: [PATCH v7 3/5] virtio-iommu: Call iommu notifier for attach/detach
> -Original Message- > From: Jean-Philippe Brucker > Sent: Wednesday, March 18, 2020 4:48 PM > To: Bharat Bhushan > Cc: Auger Eric ; Peter Maydell > ; kevin.t...@intel.com; Tomasz Nowicki [C] > ; m...@redhat.com; drjo...@redhat.com; > pet...@redhat.com; qemu-devel@nongnu.org; alex.william...@redhat.com; > qemu-...@nongnu.org; Bharat Bhushan ; > linuc.dec...@gmail.com; eric.auger....@gmail.com > Subject: [EXT] Re: [PATCH v7 3/5] virtio-iommu: Call iommu notifier for > attach/detach > > External Email > > -- > On Wed, Mar 18, 2020 at 03:47:44PM +0530, Bharat Bhushan wrote: > > Hi Jean, > > > > On Tue, Mar 17, 2020 at 9:29 PM Jean-Philippe Brucker > > wrote: > > > > > > On Tue, Mar 17, 2020 at 02:46:55PM +0530, Bharat Bhushan wrote: > > > > Hi Jean, > > > > > > > > On Tue, Mar 17, 2020 at 2:23 PM Jean-Philippe Brucker > > > > wrote: > > > > > > > > > > On Tue, Mar 17, 2020 at 12:40:39PM +0530, Bharat Bhushan wrote: > > > > > > Hi Jean, > > > > > > > > > > > > On Mon, Mar 16, 2020 at 3:41 PM Jean-Philippe Brucker > > > > > > wrote: > > > > > > > > > > > > > > Hi Bharat, > > > > > > > > > > > > > > Could you Cc me on your next posting? Unfortunately I don't > > > > > > > have much hardware for testing this at the moment, but I > > > > > > > might be able to help a little on the review. > > > > > > > > > > > > > > On Mon, Mar 16, 2020 at 02:40:00PM +0530, Bharat Bhushan wrote: > > > > > > > > > >>> First issue is: your guest can use 4K page and your > > > > > > > > > >>> host can use 64KB pages. In that case VFIO_DMA_MAP > > > > > > > > > >>> will fail with -EINVAL. We must devise a way to pass the > > > > > > > > > >>> host > settings to the VIRTIO-IOMMU device. > > > > > > > > > >>> > > > > > > > > > >>> Even with 64KB pages, it did not work for me. I have > > > > > > > > > >>> obviously not the storm of VFIO_DMA_MAP failures but > > > > > > > > > >>> I have some, most probably due to some wrong notifications > somewhere. I will try to investigate on my side. > > > > > > > > > >>> > > > > > > > > > >>> Did you test with VFIO on your side? > > > > > > > > > >> > > > > > > > > > >> I did not tried with different page sizes, only tested > > > > > > > > > >> with 4K page > size. > > > > > > > > > >> > > > > > > > > > >> Yes it works, I tested with two n/w device assigned > > > > > > > > > >> to VM, both interfaces works > > > > > > > > > >> > > > > > > > > > >> First I will try with 64k page size. > > > > > > > > > > > > > > > > > > > > 64K page size does not work for me as well, > > > > > > > > > > > > > > > > > > > > I think we are not passing correct page_size_mask here > > > > > > > > > > (config.page_size_mask is set to TARGET_PAGE_MASK ( > > > > > > > > > > which is > > > > > > > > > > 0xf000)) > > > > > > > > > I guess you mean with guest using 4K and host using 64K. > > > > > > > > > > > > > > > > > > > > We need to set this correctly as per host page size, > > > > > > > > > > correct? > > > > > > > > > Yes that's correct. We need to put in place a control > > > > > > > > > path to retrieve the page settings on host through VFIO to > > > > > > > > > inform the > virtio-iommu device. > > > > > > > > > > > > > > > > > > Besides this issue, did you try with 64kB on host and guest? > > > > > > > > > > > > > > > > I tried Followings > > > > > > > > -
