Re: Question about domain_init (v5.3-v5.7)
Hi Jerry, On 12/1/20 1:50 AM, Jerry Snitselaar wrote: Lu Baolu @ 2020-11-26 19:12 MST: Hi Jerry, On 11/27/20 5:35 AM, Jerry Snitselaar wrote: Lu Baolu @ 2020-11-26 04:01 MST: Hi Jerry, On 2020/11/26 4:27, Jerry Snitselaar wrote: Is there a reason we check the requested guest address width against the iommu's mgaw, instead of the agaw that we already know for the iommu? I've run into a case with a new system where the mgaw reported is 57, but if they set PAE to 46 instead of 52 in the bios, then sagaw reports the highest supported agaw is 48 and the domain_init code fails here. In Isn't this a platform bug? If it's too late to fix it in the BIOS, you maybe have to add a platform specific quirk to set mgaw to the highest supported agaw? Best regards, baolu Is there somewhere you can point me to that discusses how they should be setting the mgaw? I misunderstood when I previously asked you about whether the mgaw could be a value that was greater than any of sagaw. If it is a bios issue, then they should fix it there. MGAW indicates the max gpa width supported by 2nd translation. The VT-d spec requires that this value must be at least equal to the host physical addressibility. According to this, BIOS is good, right? For this failure case, domain_init() just wants to find a suitable agaw for the private domain. I think it makes sense to check against iommu->agaw instead of cap_mgaw. Best regards, baolu From this bit in the spec about MGAW: Guest addressability for a given DMA request is limited to the minimum of the value reported through this field and the adjusted guest address width of the corresponding page-table structure. (Adjusted guest address widths supported by hardware are reported through the SAGAW field). That does suggest it should be adjusted down to the sagaw value in this case, yes? Just want to make sure I'm understanding it correctly. Yes. I think so. Best regards, baolu other places like prepare_domain_attach_device, the dmar domain agaw gets adjusted down to the iommu agaw. The agaw of the iommu gets determined based off what is reported for sagaw. I'm wondering if it can't instead do: --- drivers/iommu/intel-iommu.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c index 6ca5c92ef2e5..a8e41ec36d9e 100644 --- a/drivers/iommu/intel-iommu.c +++ b/drivers/iommu/intel-iommu.c @@ -1862,8 +1862,8 @@ static int domain_init(struct dmar_domain *domain, struct intel_iommu *iommu, domain_reserve_special_ranges(domain); /* calculate AGAW */ - if (guest_width > cap_mgaw(iommu->cap)) - guest_width = cap_mgaw(iommu->cap); + if (guest_width > agaw_to_width(iommu->agaw)) + guest_width = agaw_to_width(iommu->agaw); domain->gaw = guest_width; adjust_width = guestwidth_to_adjustwidth(guest_width); agaw = width_to_agaw(adjust_width); -- 2.27.0 Thoughts? With the former code the ehci device for the ilo fails when trying to get a private domain. Thanks, Jerry ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: Question about domain_init (v5.3-v5.7)
Jerry Snitselaar @ 2020-11-30 10:50 MST: > Lu Baolu @ 2020-11-26 19:12 MST: > >> Hi Jerry, >> >> On 11/27/20 5:35 AM, Jerry Snitselaar wrote: >>> Lu Baolu @ 2020-11-26 04:01 MST: >>> Hi Jerry, On 2020/11/26 4:27, Jerry Snitselaar wrote: > Is there a reason we check the requested guest address width against > the > iommu's mgaw, instead of the agaw that we already know for the iommu? > I've run into a case with a new system where the mgaw reported is 57, > but if they set PAE to 46 instead of 52 in the bios, then sagaw reports > the highest supported agaw is 48 and the domain_init code fails here. In Isn't this a platform bug? If it's too late to fix it in the BIOS, you maybe have to add a platform specific quirk to set mgaw to the highest supported agaw? Best regards, baolu >>> Is there somewhere you can point me to that discusses how they >>> should be >>> setting the mgaw? I misunderstood when I previously asked you about >>> whether the mgaw could be a value that was greater than any of sagaw. >>> If it is a bios issue, then they should fix it there. >> >> MGAW indicates the max gpa width supported by 2nd translation. The VT-d >> spec requires that this value must be at least equal to the host >> physical addressibility. According to this, BIOS is good, right? >> >> For this failure case, domain_init() just wants to find a suitable agaw >> for the private domain. I think it makes sense to check against >> iommu->agaw instead of cap_mgaw. >> >> Best regards, >> baolu >> > > From this bit in the spec about MGAW: > > Guest addressability for a given DMA request is limited to the > minimum of the value reported through this field and the adjusted > guest address width of the corresponding page-table structure. > (Adjusted guest address widths supported by hardware are reported > through the SAGAW field). > > That does suggest it should be adjusted down to the sagaw value in this case, > yes? > Just want to make sure I'm understanding it correctly. Or I guess that is really talking about if you had an mgaw lower than the the sagaw, the dma request would be limited to that lower mgaw value. > >>> > other places like prepare_domain_attach_device, the dmar domain agaw > gets adjusted down to the iommu agaw. The agaw of the iommu gets > determined based off what is reported for sagaw. I'm wondering if it > can't instead do: > --- >drivers/iommu/intel-iommu.c | 4 ++-- >1 file changed, 2 insertions(+), 2 deletions(-) > diff --git a/drivers/iommu/intel-iommu.c > b/drivers/iommu/intel-iommu.c > index 6ca5c92ef2e5..a8e41ec36d9e 100644 > --- a/drivers/iommu/intel-iommu.c > +++ b/drivers/iommu/intel-iommu.c > @@ -1862,8 +1862,8 @@ static int domain_init(struct dmar_domain *domain, > struct intel_iommu *iommu, > domain_reserve_special_ranges(domain); > /* calculate AGAW */ > - if (guest_width > cap_mgaw(iommu->cap)) > - guest_width = cap_mgaw(iommu->cap); > + if (guest_width > agaw_to_width(iommu->agaw)) > + guest_width = agaw_to_width(iommu->agaw); > domain->gaw = guest_width; > adjust_width = guestwidth_to_adjustwidth(guest_width); > agaw = width_to_agaw(adjust_width); > -- > 2.27.0 > > Thoughts? With the former code the ehci device for the ilo fails when > trying to get a private domain. > Thanks, > Jerry > >>> ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: Question about domain_init (v5.3-v5.7)
Lu Baolu @ 2020-11-26 19:12 MST: > Hi Jerry, > > On 11/27/20 5:35 AM, Jerry Snitselaar wrote: >> Lu Baolu @ 2020-11-26 04:01 MST: >> >>> Hi Jerry, >>> >>> On 2020/11/26 4:27, Jerry Snitselaar wrote: Is there a reason we check the requested guest address width against the iommu's mgaw, instead of the agaw that we already know for the iommu? I've run into a case with a new system where the mgaw reported is 57, but if they set PAE to 46 instead of 52 in the bios, then sagaw reports the highest supported agaw is 48 and the domain_init code fails here. In >>> >>> Isn't this a platform bug? If it's too late to fix it in the BIOS, you >>> maybe have to add a platform specific quirk to set mgaw to the highest >>> supported agaw? >>> >>> Best regards, >>> baolu >> Is there somewhere you can point me to that discusses how they >> should be >> setting the mgaw? I misunderstood when I previously asked you about >> whether the mgaw could be a value that was greater than any of sagaw. >> If it is a bios issue, then they should fix it there. > > MGAW indicates the max gpa width supported by 2nd translation. The VT-d > spec requires that this value must be at least equal to the host > physical addressibility. According to this, BIOS is good, right? > > For this failure case, domain_init() just wants to find a suitable agaw > for the private domain. I think it makes sense to check against > iommu->agaw instead of cap_mgaw. > > Best regards, > baolu > >From this bit in the spec about MGAW: Guest addressability for a given DMA request is limited to the minimum of the value reported through this field and the adjusted guest address width of the corresponding page-table structure. (Adjusted guest address widths supported by hardware are reported through the SAGAW field). That does suggest it should be adjusted down to the sagaw value in this case, yes? Just want to make sure I'm understanding it correctly. >> >>> other places like prepare_domain_attach_device, the dmar domain agaw gets adjusted down to the iommu agaw. The agaw of the iommu gets determined based off what is reported for sagaw. I'm wondering if it can't instead do: --- drivers/iommu/intel-iommu.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c index 6ca5c92ef2e5..a8e41ec36d9e 100644 --- a/drivers/iommu/intel-iommu.c +++ b/drivers/iommu/intel-iommu.c @@ -1862,8 +1862,8 @@ static int domain_init(struct dmar_domain *domain, struct intel_iommu *iommu, domain_reserve_special_ranges(domain); /* calculate AGAW */ - if (guest_width > cap_mgaw(iommu->cap)) - guest_width = cap_mgaw(iommu->cap); + if (guest_width > agaw_to_width(iommu->agaw)) + guest_width = agaw_to_width(iommu->agaw); domain->gaw = guest_width; adjust_width = guestwidth_to_adjustwidth(guest_width); agaw = width_to_agaw(adjust_width); -- 2.27.0 Thoughts? With the former code the ehci device for the ilo fails when trying to get a private domain. Thanks, Jerry >> ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: Question about domain_init (v5.3-v5.7)
Lu Baolu @ 2020-11-26 19:12 MST: > Hi Jerry, > > On 11/27/20 5:35 AM, Jerry Snitselaar wrote: >> Lu Baolu @ 2020-11-26 04:01 MST: >> >>> Hi Jerry, >>> >>> On 2020/11/26 4:27, Jerry Snitselaar wrote: Is there a reason we check the requested guest address width against the iommu's mgaw, instead of the agaw that we already know for the iommu? I've run into a case with a new system where the mgaw reported is 57, but if they set PAE to 46 instead of 52 in the bios, then sagaw reports the highest supported agaw is 48 and the domain_init code fails here. In >>> >>> Isn't this a platform bug? If it's too late to fix it in the BIOS, you >>> maybe have to add a platform specific quirk to set mgaw to the highest >>> supported agaw? >>> >>> Best regards, >>> baolu >> Is there somewhere you can point me to that discusses how they >> should be >> setting the mgaw? I misunderstood when I previously asked you about >> whether the mgaw could be a value that was greater than any of sagaw. >> If it is a bios issue, then they should fix it there. > > MGAW indicates the max gpa width supported by 2nd translation. The VT-d > spec requires that this value must be at least equal to the host > physical addressibility. According to this, BIOS is good, right? > Yes, the host address width is 46. MGAW reports 57 (56+1), and highest sagaw bit is for 48. > For this failure case, domain_init() just wants to find a suitable agaw > for the private domain. I think it makes sense to check against > iommu->agaw instead of cap_mgaw. > > Best regards, > baolu > >> >>> other places like prepare_domain_attach_device, the dmar domain agaw gets adjusted down to the iommu agaw. The agaw of the iommu gets determined based off what is reported for sagaw. I'm wondering if it can't instead do: --- drivers/iommu/intel-iommu.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c index 6ca5c92ef2e5..a8e41ec36d9e 100644 --- a/drivers/iommu/intel-iommu.c +++ b/drivers/iommu/intel-iommu.c @@ -1862,8 +1862,8 @@ static int domain_init(struct dmar_domain *domain, struct intel_iommu *iommu, domain_reserve_special_ranges(domain); /* calculate AGAW */ - if (guest_width > cap_mgaw(iommu->cap)) - guest_width = cap_mgaw(iommu->cap); + if (guest_width > agaw_to_width(iommu->agaw)) + guest_width = agaw_to_width(iommu->agaw); domain->gaw = guest_width; adjust_width = guestwidth_to_adjustwidth(guest_width); agaw = width_to_agaw(adjust_width); -- 2.27.0 Thoughts? With the former code the ehci device for the ilo fails when trying to get a private domain. Thanks, Jerry >> ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: Question about domain_init (v5.3-v5.7)
Hi Jerry, On 11/27/20 5:35 AM, Jerry Snitselaar wrote: Lu Baolu @ 2020-11-26 04:01 MST: Hi Jerry, On 2020/11/26 4:27, Jerry Snitselaar wrote: Is there a reason we check the requested guest address width against the iommu's mgaw, instead of the agaw that we already know for the iommu? I've run into a case with a new system where the mgaw reported is 57, but if they set PAE to 46 instead of 52 in the bios, then sagaw reports the highest supported agaw is 48 and the domain_init code fails here. In Isn't this a platform bug? If it's too late to fix it in the BIOS, you maybe have to add a platform specific quirk to set mgaw to the highest supported agaw? Best regards, baolu Is there somewhere you can point me to that discusses how they should be setting the mgaw? I misunderstood when I previously asked you about whether the mgaw could be a value that was greater than any of sagaw. If it is a bios issue, then they should fix it there. MGAW indicates the max gpa width supported by 2nd translation. The VT-d spec requires that this value must be at least equal to the host physical addressibility. According to this, BIOS is good, right? For this failure case, domain_init() just wants to find a suitable agaw for the private domain. I think it makes sense to check against iommu->agaw instead of cap_mgaw. Best regards, baolu other places like prepare_domain_attach_device, the dmar domain agaw gets adjusted down to the iommu agaw. The agaw of the iommu gets determined based off what is reported for sagaw. I'm wondering if it can't instead do: --- drivers/iommu/intel-iommu.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c index 6ca5c92ef2e5..a8e41ec36d9e 100644 --- a/drivers/iommu/intel-iommu.c +++ b/drivers/iommu/intel-iommu.c @@ -1862,8 +1862,8 @@ static int domain_init(struct dmar_domain *domain, struct intel_iommu *iommu, domain_reserve_special_ranges(domain); /* calculate AGAW */ - if (guest_width > cap_mgaw(iommu->cap)) - guest_width = cap_mgaw(iommu->cap); + if (guest_width > agaw_to_width(iommu->agaw)) + guest_width = agaw_to_width(iommu->agaw); domain->gaw = guest_width; adjust_width = guestwidth_to_adjustwidth(guest_width); agaw = width_to_agaw(adjust_width); -- 2.27.0 Thoughts? With the former code the ehci device for the ilo fails when trying to get a private domain. Thanks, Jerry ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: Question about domain_init (v5.3-v5.7)
Lu Baolu @ 2020-11-26 04:01 MST: > Hi Jerry, > > On 2020/11/26 4:27, Jerry Snitselaar wrote: >> Is there a reason we check the requested guest address width against >> the >> iommu's mgaw, instead of the agaw that we already know for the iommu? >> I've run into a case with a new system where the mgaw reported is 57, >> but if they set PAE to 46 instead of 52 in the bios, then sagaw reports >> the highest supported agaw is 48 and the domain_init code fails here. In > > Isn't this a platform bug? If it's too late to fix it in the BIOS, you > maybe have to add a platform specific quirk to set mgaw to the highest > supported agaw? > > Best regards, > baolu Is there somewhere you can point me to that discusses how they should be setting the mgaw? I misunderstood when I previously asked you about whether the mgaw could be a value that was greater than any of sagaw. If it is a bios issue, then they should fix it there. > >> other places like prepare_domain_attach_device, the dmar domain agaw >> gets adjusted down to the iommu agaw. The agaw of the iommu gets >> determined based off what is reported for sagaw. I'm wondering if it >> can't instead do: >> --- >> drivers/iommu/intel-iommu.c | 4 ++-- >> 1 file changed, 2 insertions(+), 2 deletions(-) >> diff --git a/drivers/iommu/intel-iommu.c >> b/drivers/iommu/intel-iommu.c >> index 6ca5c92ef2e5..a8e41ec36d9e 100644 >> --- a/drivers/iommu/intel-iommu.c >> +++ b/drivers/iommu/intel-iommu.c >> @@ -1862,8 +1862,8 @@ static int domain_init(struct dmar_domain *domain, >> struct intel_iommu *iommu, >> domain_reserve_special_ranges(domain); >> /* calculate AGAW */ >> -if (guest_width > cap_mgaw(iommu->cap)) >> -guest_width = cap_mgaw(iommu->cap); >> +if (guest_width > agaw_to_width(iommu->agaw)) >> +guest_width = agaw_to_width(iommu->agaw); >> domain->gaw = guest_width; >> adjust_width = guestwidth_to_adjustwidth(guest_width); >> agaw = width_to_agaw(adjust_width); >> -- >> 2.27.0 >> >> Thoughts? With the former code the ehci device for the ilo fails when >> trying to get a private domain. >> Thanks, >> Jerry >> ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: Question about domain_init (v5.3-v5.7)
Hi Jerry, On 2020/11/26 4:27, Jerry Snitselaar wrote: Is there a reason we check the requested guest address width against the iommu's mgaw, instead of the agaw that we already know for the iommu? I've run into a case with a new system where the mgaw reported is 57, but if they set PAE to 46 instead of 52 in the bios, then sagaw reports the highest supported agaw is 48 and the domain_init code fails here. In Isn't this a platform bug? If it's too late to fix it in the BIOS, you maybe have to add a platform specific quirk to set mgaw to the highest supported agaw? Best regards, baolu other places like prepare_domain_attach_device, the dmar domain agaw gets adjusted down to the iommu agaw. The agaw of the iommu gets determined based off what is reported for sagaw. I'm wondering if it can't instead do: --- drivers/iommu/intel-iommu.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c index 6ca5c92ef2e5..a8e41ec36d9e 100644 --- a/drivers/iommu/intel-iommu.c +++ b/drivers/iommu/intel-iommu.c @@ -1862,8 +1862,8 @@ static int domain_init(struct dmar_domain *domain, struct intel_iommu *iommu, domain_reserve_special_ranges(domain); /* calculate AGAW */ - if (guest_width > cap_mgaw(iommu->cap)) - guest_width = cap_mgaw(iommu->cap); + if (guest_width > agaw_to_width(iommu->agaw)) + guest_width = agaw_to_width(iommu->agaw); domain->gaw = guest_width; adjust_width = guestwidth_to_adjustwidth(guest_width); agaw = width_to_agaw(adjust_width); -- 2.27.0 Thoughts? With the former code the ehci device for the ilo fails when trying to get a private domain. Thanks, Jerry ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Question about domain_init (v5.3-v5.7)
Is there a reason we check the requested guest address width against the iommu's mgaw, instead of the agaw that we already know for the iommu? I've run into a case with a new system where the mgaw reported is 57, but if they set PAE to 46 instead of 52 in the bios, then sagaw reports the highest supported agaw is 48 and the domain_init code fails here. In other places like prepare_domain_attach_device, the dmar domain agaw gets adjusted down to the iommu agaw. The agaw of the iommu gets determined based off what is reported for sagaw. I'm wondering if it can't instead do: --- drivers/iommu/intel-iommu.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c index 6ca5c92ef2e5..a8e41ec36d9e 100644 --- a/drivers/iommu/intel-iommu.c +++ b/drivers/iommu/intel-iommu.c @@ -1862,8 +1862,8 @@ static int domain_init(struct dmar_domain *domain, struct intel_iommu *iommu, domain_reserve_special_ranges(domain); /* calculate AGAW */ - if (guest_width > cap_mgaw(iommu->cap)) - guest_width = cap_mgaw(iommu->cap); + if (guest_width > agaw_to_width(iommu->agaw)) + guest_width = agaw_to_width(iommu->agaw); domain->gaw = guest_width; adjust_width = guestwidth_to_adjustwidth(guest_width); agaw = width_to_agaw(adjust_width); -- 2.27.0 Thoughts? With the former code the ehci device for the ilo fails when trying to get a private domain. Thanks, Jerry ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
