Re: [PATCH] kvm: make mmu_shrink() fit shrinker's requirement
On Sun, Aug 15, 2010 at 4:14 AM, Avi Kivity a...@redhat.com wrote: On 08/13/2010 11:10 PM, Dave Hansen wrote: On Thu, 2010-08-05 at 12:28 +0300, Avi Kivity wrote: On 08/04/2010 10:13 AM, Lai Jiangshan wrote: mmu_shrink() should attempt to free @nr_to_scan entries. This conflicts with Dave's patchset. Dave, what's going on with those patches? They're starting to smell. These seem to fix the original problem reporter's issue. They were run with 64 guests on a 32GB machine. No stability problems popped up in this testing, or since I last sent the patches to you. The results from both the test with only the first four patches and with the entire set of nine looked pretty identical. That tells me that we should only push the first four for now: abstract kvm x86 mmu-n_free_mmu_pages rename x86 kvm-arch.n_alloc_mmu_pages replace x86 kvm n_free_mmu_pages with n_used_mmu_pages create aggregate kvm_total_used_mmu_pages value Well, patches 3 and 4 have unaddressed review comments. Please fix them up. If you don't have the time, let me know and I'll do it instead. Dave's out on vacation now so it's probably best to assume he wont get those fixups done very quickly. Marcelo's comment on patch 3 is simple. The conversation regarding patch 4 back in June though doesn't read like it clearly concluded...I'd be happy to work something up, but if you've got strong preferences on which route to use for protecting the count maybe I should leave it to you? Tim -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] kvm: make mmu_shrink() fit shrinker's requirement
On 08/13/2010 11:10 PM, Dave Hansen wrote: On Thu, 2010-08-05 at 12:28 +0300, Avi Kivity wrote: On 08/04/2010 10:13 AM, Lai Jiangshan wrote: mmu_shrink() should attempt to free @nr_to_scan entries. This conflicts with Dave's patchset. Dave, what's going on with those patches? They're starting to smell. These seem to fix the original problem reporter's issue. They were run with 64 guests on a 32GB machine. No stability problems popped up in this testing, or since I last sent the patches to you. The results from both the test with only the first four patches and with the entire set of nine looked pretty identical. That tells me that we should only push the first four for now: abstract kvm x86 mmu-n_free_mmu_pages rename x86 kvm-arch.n_alloc_mmu_pages replace x86 kvm n_free_mmu_pages with n_used_mmu_pages create aggregate kvm_total_used_mmu_pages value Well, patches 3 and 4 have unaddressed review comments. Please fix them up. If you don't have the time, let me know and I'll do it instead. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] kvm: make mmu_shrink() fit shrinker's requirement
On Thu, 2010-08-05 at 12:28 +0300, Avi Kivity wrote: On 08/04/2010 10:13 AM, Lai Jiangshan wrote: mmu_shrink() should attempt to free @nr_to_scan entries. This conflicts with Dave's patchset. Dave, what's going on with those patches? They're starting to smell. These seem to fix the original problem reporter's issue. They were run with 64 guests on a 32GB machine. No stability problems popped up in this testing, or since I last sent the patches to you. The results from both the test with only the first four patches and with the entire set of nine looked pretty identical. That tells me that we should only push the first four for now: abstract kvm x86 mmu-n_free_mmu_pages rename x86 kvm-arch.n_alloc_mmu_pages replace x86 kvm n_free_mmu_pages with n_used_mmu_pages create aggregate kvm_total_used_mmu_pages value -- Dave -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] kvm: make mmu_shrink() fit shrinker's requirement
On Thu, 2010-08-05 at 12:28 +0300, Avi Kivity wrote: On 08/04/2010 10:13 AM, Lai Jiangshan wrote: mmu_shrink() should attempt to free @nr_to_scan entries. This conflicts with Dave's patchset. Dave, what's going on with those patches? They're starting to smell. The hardware and test rig that actually found the original scalability problem is a bit contended with issues at the moment, so I've been unable to get the original issue reproduced. But, I think the patches should be able to stand on their own. Even in my testing, it's obvious that the shrinker code gets queries (via nr_to_shrink=0) *way* more often than it's actually called upon to shrink things. Even if we have to come back with some more patches than this in the future for the original problem that was found, we should feel confident that this patch set is worthwhile and does some real, quantifiable good on its own. I'll go and immediately test that the first four patches in the series are still behaving themselves in the way that I expect. -- Dave -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] kvm: make mmu_shrink() fit shrinker's requirement
On 08/04/2010 10:13 AM, Lai Jiangshan wrote:
mmu_shrink() should attempt to free @nr_to_scan entries.
This conflicts with Dave's patchset.
Dave, what's going on with those patches? They're starting to smell.
@@ -3138,37 +3138,51 @@ static int mmu_shrink(struct shrinker *shrink, int
nr_to_scan, gfp_t gfp_mask)
{
struct kvm *kvm;
struct kvm *kvm_freed = NULL;
+ struct kvm *kvm_last;
int cache_count = 0;
spin_lock(kvm_lock);
- list_for_each_entry(kvm,vm_list, vm_list) {
+ if (list_empty(vm_list))
+ goto out;
+
+ kvm_last = list_entry(vm_list.prev, struct kvm, vm_list);
+
+ for (;;) {
int npages, idx, freed_pages;
LIST_HEAD(invalid_list);
+ kvm = list_first_entry(vm_list, struct kvm, vm_list);
idx = srcu_read_lock(kvm-srcu);
spin_lock(kvm-mmu_lock);
npages = kvm-arch.n_alloc_mmu_pages -
kvm-arch.n_free_mmu_pages;
- cache_count += npages;
- if (!kvm_freed nr_to_scan 0 npages 0) {
+ if (kvm_last)
+ cache_count += npages;
+ if (nr_to_scan 0 npages 0) {
freed_pages = kvm_mmu_remove_some_alloc_mmu_pages(kvm,
invalid_list);
+ kvm_mmu_commit_zap_page(kvm,invalid_list);
cache_count -= freed_pages;
kvm_freed = kvm;
- }
- nr_to_scan--;
+ nr_to_scan -= freed_pages;
+ } else if (kvm == kvm_freed)
+ nr_to_scan = 0; /* no more page to be freed, break */
- kvm_mmu_commit_zap_page(kvm,invalid_list);
spin_unlock(kvm-mmu_lock);
srcu_read_unlock(kvm-srcu, idx);
- }
- if (kvm_freed)
list_move_tail(kvm_freed-vm_list,vm_list);
+ if (kvm == kvm_last) /* just scaned all vms */
+ kvm_last = NULL;
+ if (!kvm_last (nr_to_scan= 0 || !kvm_freed))
+ break;
+ }
+
+out:
spin_unlock(kvm_lock);
- return cache_count;
+ return cache_count 0 ? 0 : cache_count;
}
static struct shrinker mmu_shrinker = {
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
error compiling committee.c: too many arguments to function
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] kvm: make mmu_shrink() fit shrinker's requirement
mmu_shrink() should attempt to free @nr_to_scan entries.
Signed-off-by: Lai Jiangshan la...@cn.fujitsu.com
---
diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c
index 9c69725..1034373 100644
--- a/arch/x86/kvm/mmu.c
+++ b/arch/x86/kvm/mmu.c
@@ -3138,37 +3138,51 @@ static int mmu_shrink(struct shrinker *shrink, int
nr_to_scan, gfp_t gfp_mask)
{
struct kvm *kvm;
struct kvm *kvm_freed = NULL;
+ struct kvm *kvm_last;
int cache_count = 0;
spin_lock(kvm_lock);
- list_for_each_entry(kvm, vm_list, vm_list) {
+ if (list_empty(vm_list))
+ goto out;
+
+ kvm_last = list_entry(vm_list.prev, struct kvm, vm_list);
+
+ for (;;) {
int npages, idx, freed_pages;
LIST_HEAD(invalid_list);
+ kvm = list_first_entry(vm_list, struct kvm, vm_list);
idx = srcu_read_lock(kvm-srcu);
spin_lock(kvm-mmu_lock);
npages = kvm-arch.n_alloc_mmu_pages -
kvm-arch.n_free_mmu_pages;
- cache_count += npages;
- if (!kvm_freed nr_to_scan 0 npages 0) {
+ if (kvm_last)
+ cache_count += npages;
+ if (nr_to_scan 0 npages 0) {
freed_pages = kvm_mmu_remove_some_alloc_mmu_pages(kvm,
invalid_list);
+ kvm_mmu_commit_zap_page(kvm, invalid_list);
cache_count -= freed_pages;
kvm_freed = kvm;
- }
- nr_to_scan--;
+ nr_to_scan -= freed_pages;
+ } else if (kvm == kvm_freed)
+ nr_to_scan = 0; /* no more page to be freed, break */
- kvm_mmu_commit_zap_page(kvm, invalid_list);
spin_unlock(kvm-mmu_lock);
srcu_read_unlock(kvm-srcu, idx);
- }
- if (kvm_freed)
list_move_tail(kvm_freed-vm_list, vm_list);
+ if (kvm == kvm_last) /* just scaned all vms */
+ kvm_last = NULL;
+ if (!kvm_last (nr_to_scan = 0 || !kvm_freed))
+ break;
+ }
+
+out:
spin_unlock(kvm_lock);
- return cache_count;
+ return cache_count 0 ? 0 : cache_count;
}
static struct shrinker mmu_shrinker = {
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
