On 09/05/2014 05:35 AM, Johannes Weiner wrote: > On Thu, Sep 04, 2014 at 01:27:26PM -0700, Dave Hansen wrote: >> On 09/04/2014 07:27 AM, Michal Hocko wrote: >>> Ouch. free_pages_and_swap_cache completely kills the uncharge batching >>> because it reduces it to PAGEVEC_SIZE batches. >>> >>> I think we really do not need PAGEVEC_SIZE batching anymore. We are >>> already batching on tlb_gather layer. That one is limited so I think >>> the below should be safe but I have to think about this some more. There >>> is a risk of prolonged lru_lock wait times but the number of pages is >>> limited to 10k and the heavy work is done outside of the lock. If this >>> is really a problem then we can tear LRU part and the actual >>> freeing/uncharging into a separate functions in this path. >>> >>> Could you test with this half baked patch, please? I didn't get to test >>> it myself unfortunately. >> >> 3.16 settled out at about 11.5M faults/sec before the regression. This >> patch gets it back up to about 10.5M, which is good. The top spinlock >> contention in the kernel is still from the resource counter code via >> mem_cgroup_commit_charge(), though. > > Thanks for testing, that looks a lot better. > > But commit doesn't touch resource counters - did you mean try_charge() > or uncharge() by any chance?
I don't have the perf output that I was looking at when I said this, but here's the path that I think I was referring to. The inlining makes this non-obvious, but this memcg_check_events() calls mem_cgroup_update_tree() which is contending on mctz->lock. So, you were right, it's not the resource counters code, it's a lock in 'struct mem_cgroup_tree_per_zone'. But, the contention isn't _that_ high (2% of CPU) in this case. But, that is 2% that we didn't see before. > 1.87% 1.87% [kernel] [k] _raw_spin_lock_irqsave > | > --- _raw_spin_lock_irqsave > | > |--107.09%-- memcg_check_events > | | > | |--79.98%-- > mem_cgroup_commit_charge > | | | > | | |--99.81%-- > do_cow_fault > | | | > handle_mm_fault > | | | > __do_page_fault > | | | > do_page_fault > | | | page_fault > | | | testcase > | | --0.19%-- [...] -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
