Re: [RFC PATCH 0/3] kvm/arm: New VMID allocator based on asid(2nd approach)
On Fri, Jun 04, 2021 at 04:27:39PM +0100, Marc Zyngier wrote: > > > Plus, I've found this nugget: > > > > > > > > max_pinned_vmids = NUM_USER_VMIDS - num_possible_cpus() - 2; > > > > > > > > > What is this "- 2"? My hunch is that it should really be "- 1" as VMID > > > 0 is reserved, and we have no equivalent of KPTI for S2. > > > > I think this is more related to the "pinned vmid" stuff and was borrowed > > from > > the asid_update_limit() fn in arch/arm64/mm/context.c. But I missed the > > comment that explains the reason behind it. It says, > > > > ---x--- > > /* > > * There must always be an ASID available after rollover. Ensure that, > > * even if all CPUs have a reserved ASID and the maximum number of ASIDs > > * are pinned, there still is at least one empty slot in the ASID map. > > */ > > max_pinned_asids = num_available_asids - num_possible_cpus() - 2; > > ---x--- > > > > So this is to make sure we will have at least one VMID available > > after rollover in case we have pinned the max number of VMIDs. I > > will include that comment to make it clear. > > That doesn't really explain the -2. Or is that that we have one for > the extra empty slot, and one for the reserved? > > Jean-Philippe? Yes, -2 is for ASID#0 and the extra empty slot. A comment higher in asids_update_limit() hints at that, but it could definitely be clearer /* * Expect allocation after rollover to fail if we don't have at least * one more ASID than CPUs. ASID #0 is reserved for init_mm. */ Thanks, Jean ___ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [RFC PATCH 0/3] kvm/arm: New VMID allocator based on asid(2nd approach)
On Fri, 04 Jun 2021 15:51:29 +0100, Shameerali Kolothum Thodi wrote: > > Hi Marc, > > > -Original Message- > > From: Marc Zyngier [mailto:m...@kernel.org] > > Sent: 04 June 2021 14:55 > > To: Shameerali Kolothum Thodi > > Cc: linux-arm-ker...@lists.infradead.org; kvmarm@lists.cs.columbia.edu; > > linux-ker...@vger.kernel.org; w...@kernel.org; catalin.mari...@arm.com; > > james.mo...@arm.com; julien.thierry.k...@gmail.com; > > suzuki.poul...@arm.com; jean-phili...@linaro.org; Alexandru Elisei > > ; Linuxarm > > Subject: Re: [RFC PATCH 0/3] kvm/arm: New VMID allocator based on asid(2nd > > approach) > > > > On Fri, 04 Jun 2021 09:13:10 +0100, > > Shameerali Kolothum Thodi > > wrote: > > > > > > Hi, > > > > > > > -Original Message- > > > > From: Shameerali Kolothum Thodi > > > > Sent: 06 May 2021 17:52 > > > > To: linux-arm-ker...@lists.infradead.org; kvmarm@lists.cs.columbia.edu; > > > > linux-ker...@vger.kernel.org > > > > Cc: m...@kernel.org; w...@kernel.org; catalin.mari...@arm.com; > > > > james.mo...@arm.com; julien.thierry.k...@gmail.com; > > > > suzuki.poul...@arm.com; jean-phili...@linaro.org; Linuxarm > > > > > > > > Subject: [RFC PATCH 0/3] kvm/arm: New VMID allocator based on asid(2nd > > > > approach) > > > > > > > > This is based on a suggestion from Will [0] to try out the asid > > > > based kvm vmid solution as a separate VMID allocator instead of > > > > the shared lib approach attempted in v4[1]. > > > > > > > > The idea is to compare both the approaches and see whether the > > > > shared lib solution with callbacks make sense or not. > > > > > > A gentle ping on this. Please take a look and let me know. > > > > I had a look and I don't overly dislike it. I'd like to see the code > > without the pinned stuff though, at least to ease the reviewing. I > > haven't tested it in anger, but I have pushed the rebased code at [1] > > as it really didn't apply to 5.13-rc4. > > Thanks for taking a look and the rebase. I will remove the pinned stuff > in the next revision as that was added just to compare against the previous > version. > > > > > One thing I'm a bit worried about is that we so far relied on VMID 0 > > never being allocated to a guest, which is now crucial for protected > > KVM. I can't really convince myself that this can never happen with > > this. > > Hmm..not sure I quite follow that. As per the current logic vmid 0 is > reserved and is not allocated to Guest. And that's the bit I'm struggling to validate here. I guess it works because cur_idx is set to 1 in new_vmid(). > > > Plus, I've found this nugget: > > > > > max_pinned_vmids = NUM_USER_VMIDS - num_possible_cpus() - 2; > > > > > > What is this "- 2"? My hunch is that it should really be "- 1" as VMID > > 0 is reserved, and we have no equivalent of KPTI for S2. > > I think this is more related to the "pinned vmid" stuff and was borrowed from > the asid_update_limit() fn in arch/arm64/mm/context.c. But I missed the > comment that explains the reason behind it. It says, > > ---x--- > /* >* There must always be an ASID available after rollover. Ensure that, >* even if all CPUs have a reserved ASID and the maximum number of ASIDs >* are pinned, there still is at least one empty slot in the ASID map. >*/ > max_pinned_asids = num_available_asids - num_possible_cpus() - 2; > ---x--- > > So this is to make sure we will have at least one VMID available > after rollover in case we have pinned the max number of VMIDs. I > will include that comment to make it clear. That doesn't really explain the -2. Or is that that we have one for the extra empty slot, and one for the reserved? Jean-Philippe? Thanks, M. -- Without deviation from the norm, progress is not possible. ___ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
RE: [RFC PATCH 0/3] kvm/arm: New VMID allocator based on asid(2nd approach)
Hi Marc, > -Original Message- > From: Marc Zyngier [mailto:m...@kernel.org] > Sent: 04 June 2021 14:55 > To: Shameerali Kolothum Thodi > Cc: linux-arm-ker...@lists.infradead.org; kvmarm@lists.cs.columbia.edu; > linux-ker...@vger.kernel.org; w...@kernel.org; catalin.mari...@arm.com; > james.mo...@arm.com; julien.thierry.k...@gmail.com; > suzuki.poul...@arm.com; jean-phili...@linaro.org; Alexandru Elisei > ; Linuxarm > Subject: Re: [RFC PATCH 0/3] kvm/arm: New VMID allocator based on asid(2nd > approach) > > On Fri, 04 Jun 2021 09:13:10 +0100, > Shameerali Kolothum Thodi > wrote: > > > > Hi, > > > > > -Original Message- > > > From: Shameerali Kolothum Thodi > > > Sent: 06 May 2021 17:52 > > > To: linux-arm-ker...@lists.infradead.org; kvmarm@lists.cs.columbia.edu; > > > linux-ker...@vger.kernel.org > > > Cc: m...@kernel.org; w...@kernel.org; catalin.mari...@arm.com; > > > james.mo...@arm.com; julien.thierry.k...@gmail.com; > > > suzuki.poul...@arm.com; jean-phili...@linaro.org; Linuxarm > > > > > > Subject: [RFC PATCH 0/3] kvm/arm: New VMID allocator based on asid(2nd > > > approach) > > > > > > This is based on a suggestion from Will [0] to try out the asid > > > based kvm vmid solution as a separate VMID allocator instead of > > > the shared lib approach attempted in v4[1]. > > > > > > The idea is to compare both the approaches and see whether the > > > shared lib solution with callbacks make sense or not. > > > > A gentle ping on this. Please take a look and let me know. > > I had a look and I don't overly dislike it. I'd like to see the code > without the pinned stuff though, at least to ease the reviewing. I > haven't tested it in anger, but I have pushed the rebased code at [1] > as it really didn't apply to 5.13-rc4. Thanks for taking a look and the rebase. I will remove the pinned stuff in the next revision as that was added just to compare against the previous version. > > One thing I'm a bit worried about is that we so far relied on VMID 0 > never being allocated to a guest, which is now crucial for protected > KVM. I can't really convince myself that this can never happen with > this. Hmm..not sure I quite follow that. As per the current logic vmid 0 is reserved and is not allocated to Guest. > Plus, I've found this nugget: > >max_pinned_vmids = NUM_USER_VMIDS - num_possible_cpus() - 2; > > > What is this "- 2"? My hunch is that it should really be "- 1" as VMID > 0 is reserved, and we have no equivalent of KPTI for S2. I think this is more related to the "pinned vmid" stuff and was borrowed from the asid_update_limit() fn in arch/arm64/mm/context.c. But I missed the comment that explains the reason behind it. It says, ---x--- /* * There must always be an ASID available after rollover. Ensure that, * even if all CPUs have a reserved ASID and the maximum number of ASIDs * are pinned, there still is at least one empty slot in the ASID map. */ max_pinned_asids = num_available_asids - num_possible_cpus() - 2; ---x--- So this is to make sure we will have at least one VMID available after rollover in case we have pinned the max number of VMIDs. I will include that comment to make it clear. Thanks, Shameer > > M. > > [1] > https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git/log/?h > =kvm-arm64/mmu/vmid > > -- > Without deviation from the norm, progress is not possible. ___ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [RFC PATCH 0/3] kvm/arm: New VMID allocator based on asid(2nd approach)
On Fri, 04 Jun 2021 09:13:10 +0100, Shameerali Kolothum Thodi wrote: > > Hi, > > > -Original Message- > > From: Shameerali Kolothum Thodi > > Sent: 06 May 2021 17:52 > > To: linux-arm-ker...@lists.infradead.org; kvmarm@lists.cs.columbia.edu; > > linux-ker...@vger.kernel.org > > Cc: m...@kernel.org; w...@kernel.org; catalin.mari...@arm.com; > > james.mo...@arm.com; julien.thierry.k...@gmail.com; > > suzuki.poul...@arm.com; jean-phili...@linaro.org; Linuxarm > > > > Subject: [RFC PATCH 0/3] kvm/arm: New VMID allocator based on asid(2nd > > approach) > > > > This is based on a suggestion from Will [0] to try out the asid > > based kvm vmid solution as a separate VMID allocator instead of > > the shared lib approach attempted in v4[1]. > > > > The idea is to compare both the approaches and see whether the > > shared lib solution with callbacks make sense or not. > > A gentle ping on this. Please take a look and let me know. I had a look and I don't overly dislike it. I'd like to see the code without the pinned stuff though, at least to ease the reviewing. I haven't tested it in anger, but I have pushed the rebased code at [1] as it really didn't apply to 5.13-rc4. One thing I'm a bit worried about is that we so far relied on VMID 0 never being allocated to a guest, which is now crucial for protected KVM. I can't really convince myself that this can never happen with this. Plus, I've found this nugget: What is this "- 2"? My hunch is that it should really be "- 1" as VMID 0 is reserved, and we have no equivalent of KPTI for S2. M. [1] https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git/log/?h=kvm-arm64/mmu/vmid -- Without deviation from the norm, progress is not possible. ___ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
RE: [RFC PATCH 0/3] kvm/arm: New VMID allocator based on asid(2nd approach)
Hi, > -Original Message- > From: Shameerali Kolothum Thodi > Sent: 06 May 2021 17:52 > To: linux-arm-ker...@lists.infradead.org; kvmarm@lists.cs.columbia.edu; > linux-ker...@vger.kernel.org > Cc: m...@kernel.org; w...@kernel.org; catalin.mari...@arm.com; > james.mo...@arm.com; julien.thierry.k...@gmail.com; > suzuki.poul...@arm.com; jean-phili...@linaro.org; Linuxarm > > Subject: [RFC PATCH 0/3] kvm/arm: New VMID allocator based on asid(2nd > approach) > > This is based on a suggestion from Will [0] to try out the asid > based kvm vmid solution as a separate VMID allocator instead of > the shared lib approach attempted in v4[1]. > > The idea is to compare both the approaches and see whether the > shared lib solution with callbacks make sense or not. A gentle ping on this. Please take a look and let me know. Thanks, Shameer > Though we are not yet using the pinned vmids yet, patch #2 has > code for pinned vmid support. This is just to help the comparison. > > Test Setup/Results > > The measurement was made with maxcpus set to 8 and with the > number of VMID limited to 4-bit. The test involves running > concurrently 40 guests with 2 vCPUs. Each guest will then > execute hackbench 5 times before exiting. > > The performance difference between the current algo and the > new one are(avg. of 10 runs): > - 1.9% less entry/exit from the guest > - 0.5% faster > This is more or less comparable to v4 numbers. > > For the complete series, please see, > https://github.com/hisilicon/kernel-dev/tree/private-v5.12-rc7-vmid-2nd-rfc > > and for the shared asid lib v4 solution, > https://github.com/hisilicon/kernel-dev/tree/private-v5.12-rc7-asid-v4 > > As you can see there are of course code duplication with this > approach but may be this one is more easy to maintain considering > the complexity involved. > > Please take a look and let me know your feedback. > > Thanks, > Shameer > > > [0] https://lore.kernel.org/lkml/20210422160846.GB2214@willie-the-truck/ > [1] > https://lore.kernel.org/lkml/20210414112312.13704-1-shameerali.kolothum.t > h...@huawei.com/ > > Julien Grall (2): > arch/arm64: Introduce a capability to tell whether 16-bit VMID is > available > kvm/arm: Align the VMID allocation with the arm64 ASID one > > Shameer Kolothum (1): > kvm/arm: Introduce a new vmid allocator for KVM > > arch/arm64/include/asm/cpucaps.h | 3 +- > arch/arm64/include/asm/kvm_asm.h | 4 +- > arch/arm64/include/asm/kvm_host.h | 11 +- > arch/arm64/include/asm/kvm_mmu.h | 7 +- > arch/arm64/kernel/cpufeature.c | 9 + > arch/arm64/kvm/Makefile| 2 +- > arch/arm64/kvm/arm.c | 115 > arch/arm64/kvm/hyp/nvhe/hyp-main.c | 6 +- > arch/arm64/kvm/hyp/nvhe/tlb.c | 10 +- > arch/arm64/kvm/hyp/vhe/tlb.c | 10 +- > arch/arm64/kvm/mmu.c | 1 - > arch/arm64/kvm/vmid.c | 285 > + > 12 files changed, 354 insertions(+), 109 deletions(-) > create mode 100644 arch/arm64/kvm/vmid.c > > -- > 2.17.1 ___ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
