On Mon, Jul 13, 2020 at 02:49:50PM -0700, William Tu wrote: > On Fri, Jul 10, 2020 at 12:12:57PM -0300, Flavio Leitner wrote: > > On Fri, Jul 10, 2020 at 12:53:13PM +0100, Ross Lagerwall wrote: > > > mlockall locks thread stack pages into memory, even pages which have not > > > yet been demand-paged. As vswitchd has a lot of threads and the default > > > stack size on x86_64 is 8 MiB, this consumes a lot of memory. On two > > > systems I looked at, vswitchd used ~150 MiB of RSS when idle after > > > startup. > > > > > > Use the new MCL_ONFAULT flag to only lock pages into memory once they > > > have been demand-paged in. This still satisfies the requirement that > > > vswitchd is not swapped out but frees up ~144 MiB of unswappable memory > > > (18 threads x 8 MiB). After this, vswitchd uses ~6 MiB when idle after > > > startup. > > > > The problem with this approach is that when using userspace datapath > > those page faults can introduce scheduling points which introduces > > jitter and impacts performance. > > > I like this idea of using MCL_ONFAULT. > IIUC, the performance impact only happens first time when page fault > happens, and afterward, it should have the same performance.
Yes, there is a performance impact only when the page fault happens. The issue is that we can't tell when it is going to happen. Some page faults will happen at the initialization, which is fine, but some might happen when there is production traffic. The thread is preempted and all the ongoing traffic is impacted when that page fault happens. Therefore, this has a potential of affecting "zero" packet loss, for instance, which is important to Telco deployments. It seems we have two relevant use cases here with conflicting requirements. I don't think we can just change current behavior without announcing first and still the current behavior is desired for those who don't want surprises with page faults. Perhaps we can leave --mlockall as is to avoid possible regressions, and add another way to express the on demand use case? Thanks, -- fbl _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
