RE: [PATCH 3/4] iommu: Add support to change default domain of an iommu_group
> >>> +free_prev_domain: > >>> + /* > >>> + * Free the existing default domain and replace it with the newly > >>> + * created default domain. No need to set group->domain because > >>> + * __iommu_attach_group() already does it on success. > >>> + */ > >>> + iommu_domain_free(prev_domain); > >>> + group->default_domain = new_domain; > >>> + return 0; > >> > >> It isn't obvious to me from this patch, how to are the dma_ops > >> updated when the default domain changes between identity and dma? > > > > Thanks for raising this. > > For intel_iommu, dma_map_ops is defined at drivers/iommu/intel-iommu.c > > and all the callbacks like alloc(), map_sg() and map_page(), check if > > the device needs DMA mapping or not by calling iommu_need_mapping(). The > callbacks inherently do the right thing based on the outcome. > > So, essentially the dma_ops are same for dma/identity domain. > > This isn't always true as we are considering per-device dma ops. Ahh.. I see. I wasn't aware that per-device dma ops might change this. Thanks for letting me know. I will take this into consideration as well for V2. Regards, Sai ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH 3/4] iommu: Add support to change default domain of an iommu_group
Hi,
On 9/4/19 11:09 AM, Prakhya, Sai Praneeth wrote:
Hi Joerg,
Thanks a lot! for the review. I highly appreciate for sparing your time to
review the patch :)
On Tue, Aug 20, 2019 at 07:42:25PM -0700, Sai Praneeth Prakhya wrote:
+ /*
+* iommu_domain_alloc() takes "struct bus_type" as an argument which
is
+* a member in "struct device". Changing a group's default domain type
+* deals at iommu_group level rather than device level and hence there
+* is no straight forward way to get "bus_type" of an iommu_group that
+* could be passed to iommu_domain_alloc(). So, instead of directly
+* calling iommu_domain_alloc(), use iommu_ops from previous default
+* domain.
+*/
+ if (!prev_domain || !prev_domain->ops ||
+ !prev_domain->ops->domain_alloc || !type)
+ return -EINVAL;
Hmm, this isn't really nice and clean, but I understand why you need it.
I agree.. It didn't look good for me either :(
But, I didn't find any other better solution.
I will think about a better way to get iommu_ops here.
Sure! That will be great!
+free_prev_domain:
+ /*
+* Free the existing default domain and replace it with the newly
+* created default domain. No need to set group->domain because
+* __iommu_attach_group() already does it on success.
+*/
+ iommu_domain_free(prev_domain);
+ group->default_domain = new_domain;
+ return 0;
It isn't obvious to me from this patch, how to are the dma_ops updated when
the default domain changes between identity and dma?
Thanks for raising this.
For intel_iommu, dma_map_ops is defined at drivers/iommu/intel-iommu.c and
all the callbacks like alloc(), map_sg() and map_page(), check if the device
needs DMA mapping or not
by calling iommu_need_mapping(). The callbacks inherently do the right thing
based on the outcome.
So, essentially the dma_ops are same for dma/identity domain.
This isn't always true as we are considering per-device dma ops.
Best regards,
Baolu
I just realized (sorry!) that other iommu drivers (Eg: AMD) doesn't do it the
same way i.e. looks like the callbacks
aren't checking if the device needs a dma mapping or identity mapping.
I will take a look at other iommu drivers and will handle this in V2.
Please let me know if I missed something.
+ /* Check if any device in the group still has a driver binded to it */
+ if (iommu_group_for_each_dev(group, NULL, is_driver_binded)) {
+ pr_err("Active drivers exist for devices in the group\n");
+ return -EBUSY;
+ }
This is racy with device driver probing code. Unfortunatly there is no clean way
out of that either, locking all devices in the group and then do the re-attach
will
introduce a lock-inversion with group->mutex. But please put a comment here
saying that this might race with device driver probing.
Sure! Makes sense. Will add it in V2.
Regards,
Sai
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
RE: [PATCH 3/4] iommu: Add support to change default domain of an iommu_group
Hi Joerg,
Thanks a lot! for the review. I highly appreciate for sparing your time to
review the patch :)
> On Tue, Aug 20, 2019 at 07:42:25PM -0700, Sai Praneeth Prakhya wrote:
> > + /*
> > +* iommu_domain_alloc() takes "struct bus_type" as an argument which
> is
> > +* a member in "struct device". Changing a group's default domain type
> > +* deals at iommu_group level rather than device level and hence there
> > +* is no straight forward way to get "bus_type" of an iommu_group that
> > +* could be passed to iommu_domain_alloc(). So, instead of directly
> > +* calling iommu_domain_alloc(), use iommu_ops from previous default
> > +* domain.
> > +*/
> > + if (!prev_domain || !prev_domain->ops ||
> > + !prev_domain->ops->domain_alloc || !type)
> > + return -EINVAL;
>
> Hmm, this isn't really nice and clean, but I understand why you need it.
I agree.. It didn't look good for me either :(
But, I didn't find any other better solution.
> I will think about a better way to get iommu_ops here.
Sure! That will be great!
> > +free_prev_domain:
> > + /*
> > +* Free the existing default domain and replace it with the newly
> > +* created default domain. No need to set group->domain because
> > +* __iommu_attach_group() already does it on success.
> > +*/
> > + iommu_domain_free(prev_domain);
> > + group->default_domain = new_domain;
> > + return 0;
>
> It isn't obvious to me from this patch, how to are the dma_ops updated when
> the default domain changes between identity and dma?
Thanks for raising this.
For intel_iommu, dma_map_ops is defined at drivers/iommu/intel-iommu.c and
all the callbacks like alloc(), map_sg() and map_page(), check if the device
needs DMA mapping or not
by calling iommu_need_mapping(). The callbacks inherently do the right thing
based on the outcome.
So, essentially the dma_ops are same for dma/identity domain.
I just realized (sorry!) that other iommu drivers (Eg: AMD) doesn't do it the
same way i.e. looks like the callbacks
aren't checking if the device needs a dma mapping or identity mapping.
I will take a look at other iommu drivers and will handle this in V2.
Please let me know if I missed something.
> > + /* Check if any device in the group still has a driver binded to it */
> > + if (iommu_group_for_each_dev(group, NULL, is_driver_binded)) {
> > + pr_err("Active drivers exist for devices in the group\n");
> > + return -EBUSY;
> > + }
>
> This is racy with device driver probing code. Unfortunatly there is no clean
> way
> out of that either, locking all devices in the group and then do the
> re-attach will
> introduce a lock-inversion with group->mutex. But please put a comment here
> saying that this might race with device driver probing.
Sure! Makes sense. Will add it in V2.
Regards,
Sai
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH 3/4] iommu: Add support to change default domain of an iommu_group
On Tue, Aug 20, 2019 at 07:42:25PM -0700, Sai Praneeth Prakhya wrote:
> + /*
> + * iommu_domain_alloc() takes "struct bus_type" as an argument which is
> + * a member in "struct device". Changing a group's default domain type
> + * deals at iommu_group level rather than device level and hence there
> + * is no straight forward way to get "bus_type" of an iommu_group that
> + * could be passed to iommu_domain_alloc(). So, instead of directly
> + * calling iommu_domain_alloc(), use iommu_ops from previous default
> + * domain.
> + */
> + if (!prev_domain || !prev_domain->ops ||
> + !prev_domain->ops->domain_alloc || !type)
> + return -EINVAL;
Hmm, this isn't really nice and clean, but I understand why you need it.
I will think about a better way to get iommu_ops here.
> +free_prev_domain:
> + /*
> + * Free the existing default domain and replace it with the newly
> + * created default domain. No need to set group->domain because
> + * __iommu_attach_group() already does it on success.
> + */
> + iommu_domain_free(prev_domain);
> + group->default_domain = new_domain;
> + return 0;
It isn't obvious to me from this patch, how to are the dma_ops updated
when the default domain changes between identity and dma?
> + /* Check if any device in the group still has a driver binded to it */
> + if (iommu_group_for_each_dev(group, NULL, is_driver_binded)) {
> + pr_err("Active drivers exist for devices in the group\n");
> + return -EBUSY;
> + }
This is racy with device driver probing code. Unfortunatly there is no
clean way out of that either, locking all devices in the group and then
do the re-attach will introduce a lock-inversion with group->mutex. But
please put a comment here saying that this might race with device driver
probing.
Regards,
Joerg
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
