On 30.10.2023 17:10, Roger Pau Monné wrote: > On Mon, Oct 30, 2023 at 03:32:56PM +0100, Jan Beulich wrote: >> On 24.10.2023 15:51, Roger Pau Monne wrote: >>> The current implementation of x2APIC requires to either use Cluster Logical >>> or >>> Physical mode for all interrupts. However the selection of Physical vs >>> Logical >>> is not done at APIC setup, an APIC can be addressed both in Physical or >>> Logical >>> destination modes concurrently. >>> >>> Introduce a new x2APIC mode called mixed, which uses Logical Cluster mode >>> for >>> IPIs, and Physical mode for external interrupts, thus attempting to use the >>> best method for each interrupt type. >>> >>> Using Physical mode for external interrupts allows more vectors to be used, >>> and >>> interrupt balancing to be more accurate. >>> >>> Using Logical Cluster mode for IPIs allows less accesses to the ICR register >>> when sending those, as multiple CPUs can be targeted with a single ICR >>> register >>> write. >>> >>> A simple test calling flush_tlb_all() 10000 times in a tight loop on a 96 >>> CPU >>> box gives the following average figures: >>> >>> Physical mode: 26617931ns >>> Mixed mode: 23865337ns >>> >>> So ~10% improvement versus plain Physical mode. >> >> Nice. >> >>> Note that Xen uses Cluster >>> mode by default, and hence is already using the fastest way for IPI >>> delivery at >>> the cost of reducing the amount of vectors available system-wide. >>> >>> Make the newly introduced mode the default one. >>> >>> Suggested-by: Andrew Cooper <andrew.coop...@citrix.com> >>> Signed-off-by: Roger Pau Monné <roger....@citrix.com> >>> --- >>> Do we want to keep a full Logical Cluster mode available? I don't see a >>> reason >>> to target external interrupts in Logical Cluster mode, but maybe there's >>> something I'm missing. >>> >>> It's not clear to me whether the ACPI FADT flags are meant to apply only to >>> external interrupt delivery mode, or also to IPI delivery. If >>> ACPI_FADT_APIC_PHYSICAL is only meant to apply to external interrupts and >>> not >>> IPIs then we could possibly get rid of physical mode IPI delivery. >>> >>> Still need to put this under XenServer extensive testing, but wanted to get >>> some feedback before in case approach greatly changes. >> >> Looks quite okay, just a couple of minor remarks: > > Thanks. > > Do we still want to keep the pure cluster mode?
I think we should keep it for a full release cycle or two. Jan
