Re: [PATCH 8/9] xen: introduce flag when a domain requires cache control
On Fri, May 16, 2025 at 09:16:10AM +0200, Jan Beulich wrote: > On 15.05.2025 12:44, Roger Pau Monné wrote: > > On Mon, May 12, 2025 at 05:24:01PM +0200, Jan Beulich wrote: > >> On 06.05.2025 10:31, Roger Pau Monne wrote: > >>> Such flag is added to the domain create hypercall, and a matching option > >>> is > >>> added to xl and libxl to set the flag: `cache_control`. When the flag is > >>> set, the domain is allowed the usage of cache control operations. If the > >>> flag is not explicitly set, libxl will set it if the domain has any > >>> `iomem` > >>> or `ioports` ranges assigned. > >>> > >>> Modify cache_flush_permitted() helper so that it's return value is no > >>> longer based on the IO resources assigned to the domain, but rather > >>> whether > >>> the domain has been explicitly allowed control over the cache, or if it > >>> has > >>> IOMMU support and there's a single IOMMU in the system that doesn't allow > >>> forcing snooping behind the guest back. As a result of this, the return > >>> of > >>> cache_flush_permitted() is constant for the lifetime of a domain, based on > >>> the domain creation flags and the capabilities of the IOMMU if enabled > >>> for the domain. > >> > >> This then further emphasizes the remark made for patch 7. > > > > Hm, I think you are referring to the remark about how PCI device > > without IO resources would be handled then, and what would > > cache_flush_permitted() return then? > > Or more generally the relationship between that and has_arch_io_resources(). > > > IMO the benefit of the approach presented here is that it aims to know > > whether a domain needs cache control to be set at creation time, and > > not altered during it's runtime. > > I agree that having this not vary over a domain's lifetime makes things easier > for us. At the same time it may negatively affect performance of domains where > hardware devices are added / removed on the fly. I'm planing to leave this change for a further iteration of the series. Initially I just want to attempt to limit flushing when the domain has IOMMU support enabled, and the host globally supports snooping on all IOMMUs, as that's likely less controversial and would unblock a customer reported issue. Thanks, Roger.
Re: [PATCH 8/9] xen: introduce flag when a domain requires cache control
On 15.05.2025 12:44, Roger Pau Monné wrote: > On Mon, May 12, 2025 at 05:24:01PM +0200, Jan Beulich wrote: >> On 06.05.2025 10:31, Roger Pau Monne wrote: >>> Such flag is added to the domain create hypercall, and a matching option is >>> added to xl and libxl to set the flag: `cache_control`. When the flag is >>> set, the domain is allowed the usage of cache control operations. If the >>> flag is not explicitly set, libxl will set it if the domain has any `iomem` >>> or `ioports` ranges assigned. >>> >>> Modify cache_flush_permitted() helper so that it's return value is no >>> longer based on the IO resources assigned to the domain, but rather whether >>> the domain has been explicitly allowed control over the cache, or if it has >>> IOMMU support and there's a single IOMMU in the system that doesn't allow >>> forcing snooping behind the guest back. As a result of this, the return of >>> cache_flush_permitted() is constant for the lifetime of a domain, based on >>> the domain creation flags and the capabilities of the IOMMU if enabled >>> for the domain. >> >> This then further emphasizes the remark made for patch 7. > > Hm, I think you are referring to the remark about how PCI device > without IO resources would be handled then, and what would > cache_flush_permitted() return then? Or more generally the relationship between that and has_arch_io_resources(). > IMO the benefit of the approach presented here is that it aims to know > whether a domain needs cache control to be set at creation time, and > not altered during it's runtime. I agree that having this not vary over a domain's lifetime makes things easier for us. At the same time it may negatively affect performance of domains where hardware devices are added / removed on the fly. Jan
Re: [PATCH 8/9] xen: introduce flag when a domain requires cache control
On Mon, May 12, 2025 at 05:24:01PM +0200, Jan Beulich wrote: > On 06.05.2025 10:31, Roger Pau Monne wrote: > > Such flag is added to the domain create hypercall, and a matching option is > > added to xl and libxl to set the flag: `cache_control`. When the flag is > > set, the domain is allowed the usage of cache control operations. If the > > flag is not explicitly set, libxl will set it if the domain has any `iomem` > > or `ioports` ranges assigned. > > > > Modify cache_flush_permitted() helper so that it's return value is no > > longer based on the IO resources assigned to the domain, but rather whether > > the domain has been explicitly allowed control over the cache, or if it has > > IOMMU support and there's a single IOMMU in the system that doesn't allow > > forcing snooping behind the guest back. As a result of this, the return of > > cache_flush_permitted() is constant for the lifetime of a domain, based on > > the domain creation flags and the capabilities of the IOMMU if enabled > > for the domain. > > This then further emphasizes the remark made for patch 7. Hm, I think you are referring to the remark about how PCI device without IO resources would be handled then, and what would cache_flush_permitted() return then? IMO the benefit of the approach presented here is that it aims to know whether a domain needs cache control to be set at creation time, and not altered during it's runtime. Thanks, Roger.
Re: [PATCH 8/9] xen: introduce flag when a domain requires cache control
On 06.05.2025 10:31, Roger Pau Monne wrote: > Such flag is added to the domain create hypercall, and a matching option is > added to xl and libxl to set the flag: `cache_control`. When the flag is > set, the domain is allowed the usage of cache control operations. If the > flag is not explicitly set, libxl will set it if the domain has any `iomem` > or `ioports` ranges assigned. > > Modify cache_flush_permitted() helper so that it's return value is no > longer based on the IO resources assigned to the domain, but rather whether > the domain has been explicitly allowed control over the cache, or if it has > IOMMU support and there's a single IOMMU in the system that doesn't allow > forcing snooping behind the guest back. As a result of this, the return of > cache_flush_permitted() is constant for the lifetime of a domain, based on > the domain creation flags and the capabilities of the IOMMU if enabled > for the domain. This then further emphasizes the remark made for patch 7. Jan
Re: [PATCH 8/9] xen: introduce flag when a domain requires cache control
On Tue, May 06, 2025 at 11:20:41AM +0100, Julien Grall wrote: > Hi Roger, > > On 06/05/2025 09:31, Roger Pau Monne wrote: > > Such flag is added to the domain create hypercall, and a matching option is > > added to xl and libxl to set the flag: `cache_control`. When the flag is > > set, the domain is allowed the usage of cache control operations. > > Can you clarify whether you are talking about using the hypercall to flush > the cache or the instructions? On x86 at least: both hypercall and instructions if possible, since Xen traps cache flush instructions. Maybe that's different on ARM. As said on a previous reply I've assumed that cache_flush_permitted() was also relevant on ARM, but maybe it's not and domains are unconditionally allowed execution of cache control instructions and hypercalls. Thanks, Roger.
Re: [PATCH 8/9] xen: introduce flag when a domain requires cache control
Hi Roger, On 06/05/2025 09:31, Roger Pau Monne wrote: Such flag is added to the domain create hypercall, and a matching option is added to xl and libxl to set the flag: `cache_control`. When the flag is set, the domain is allowed the usage of cache control operations. Can you clarify whether you are talking about using the hypercall to flush the cache or the instructions? Cheers, -- Julien Grall
