On Tue, Jul 14, 2020 at 6:32 AM Flavio Leitner <f...@sysclose.org> wrote: > > On Mon, Jul 13, 2020 at 02:49:50PM -0700, William Tu wrote: > > On Fri, Jul 10, 2020 at 12:12:57PM -0300, Flavio Leitner wrote: > > > On Fri, Jul 10, 2020 at 12:53:13PM +0100, Ross Lagerwall wrote: > > > > mlockall locks thread stack pages into memory, even pages which have not > > > > yet been demand-paged. As vswitchd has a lot of threads and the default > > > > stack size on x86_64 is 8 MiB, this consumes a lot of memory. On two > > > > systems I looked at, vswitchd used ~150 MiB of RSS when idle after > > > > startup. > > > > > > > > Use the new MCL_ONFAULT flag to only lock pages into memory once they > > > > have been demand-paged in. This still satisfies the requirement that > > > > vswitchd is not swapped out but frees up ~144 MiB of unswappable memory > > > > (18 threads x 8 MiB). After this, vswitchd uses ~6 MiB when idle after > > > > startup. > > > > > > The problem with this approach is that when using userspace datapath > > > those page faults can introduce scheduling points which introduces > > > jitter and impacts performance. > > > > > I like this idea of using MCL_ONFAULT. > > IIUC, the performance impact only happens first time when page fault > > happens, and afterward, it should have the same performance. > > Yes, there is a performance impact only when the page fault happens. > > The issue is that we can't tell when it is going to happen. Some > page faults will happen at the initialization, which is fine, but > some might happen when there is production traffic. > > The thread is preempted and all the ongoing traffic is impacted > when that page fault happens. Therefore, this has a potential > of affecting "zero" packet loss, for instance, which is important > to Telco deployments.
I see, that makes sense, thanks for the detail. > > It seems we have two relevant use cases here with conflicting > requirements. I don't think we can just change current behavior > without announcing first and still the current behavior is desired > for those who don't want surprises with page faults. > > Perhaps we can leave --mlockall as is to avoid possible regressions, > and add another way to express the on demand use case? > Agree Thanks, William _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
