Enji, On Thu, Feb 14, 2019 at 05:12:21PM -0800, Enji Cooper wrote: E> > On Wed, Feb 13, 2019 at 07:24:50PM -0600, Justin Hibbits wrote: E> > J> This seems to break 32-bit platforms, or at least 32-bit book-e E> > J> powerpc, which has a limited KVA space (~500MB). It preallocates I've E> > J> seen over 2500 pbufs, at 128kB each, eating up over 300MB KVA, E> > J> leaving very little left for the rest of runtime. E> > J> E> > J> I spent a couple hours earlier today debugging with Mark Johnston, and E> > J> his consensus is that the vnode_pbuf_zone is too big on 32-bit E> > J> platforms. Unfortunately I know very little about this area, so can't E> > J> provide much extra insight, but can readily reproduce the issues I see E> > J> triggered by this change, so am willing to help where I can. E> > E> > Ok, let's roll back to old default on 32-bit platforms and somewhat E> > reduce the default on 64-bits. E> > E> > Can you please confirm that the patch attached works for you? E> E> Quick question: why was the value reduced by a factor of 4 on 64-bit platforms?
Fair question. Replying to you and Bruce. This pool of pbufs is used for sendfile(2) and default value of nswbuf / 2 wasn't enough for modern several Gbit/s speeds. At Netflix we run with nswbuf * 8, since we run up to 100 Gbit/s of sendfile traffic. Together with new pbuf allocator I bumped the value up to what we use. Apparently that was overkill for 32-bit machines, so for them I fully switched it back to old value. Nobody is expected to use these machine as high performance web servers. Also, I decided to reduce the default for 64-bit machines as well. Not everybody runs 100 Gbit/s, but I'd like to see default FreeBSD (no tunables in loader.conf) to be able to run 10 Gbit/s and more. So I've chosen a middle ground between old value of nswbuf / 2 and the value we use at Netflix. P.S. Ideally, this needs to be autotuned. The problem is that now we need to pre-allocate pbufs. -- Gleb Smirnoff _______________________________________________ svn-src-all@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"