On 2015-05-28 02:18 PM, Theodore Ts'o wrote: > On Wed, May 27, 2015 at 02:18:54PM -0700, Jaegeuk Kim wrote: >> The problem that I'd like to address here is to reduce the call counts of >> allocating and freeing a number of pages in pairs. >> >> When I conduct xfstests/224 under 1GB DRAM, I've seen triggering several oom >> killers, and in that moment, a huge number of inactive anonymous pages are >> registered in the page cache. Not sure why those pages are not reclaimed >> seamlessly though. > > If the system is running 8 fio processes, each one writing 1 meg > (BIO_MAX pages) at a time, one of the things that is going on is that > we need to grab 256 4k paegs before the submitting the bio, and then > if there are a large number of bio's queued, we can have potentially a > very large number of pages allocated until the I/O has been completed. > > So the problem is it's extremely difficult to determine ahead of time > how many pages that need to be reserved in a mempool. Simply > increasing the number of in the mempool from 32 to 256 is no guarantee > that it will be enough. We originally only reserved 32 pages so that > in the case of an extreme memory crunch, we could make at least some > amount of forward progress. > > I can imagine a number of different solutions (and these are not > mutally exclusive): > > 1) Try to dynamically adjust the number of pages we keep in the > mempool so that we ramp up under I/O load and then gradually ramp down > when the I/O pressure decreases. > > 2) Keep track of how many temporary encryption outstanding bounce > pages are outstanding, if we exceed some number, push back in > writepages for encrypted inode. That way we can make it be a tunable > so that we don't end up using a huge number of pages, and can start > throttling encrypted writeback even before we start getting allocation > failures. > > I'm currently leaning towards #2, myself. I haven't tried doing some > kernel performance measurements to see how much CPU time we're > spending in alloc_page() and free_page() when under a very heavy > memory load. I assume you've done some measurements and this has been > very heavy. Can you give more details about how much CPU time is > getting burned by alloc_page() and free_page()? I had been assuming > that if we're I/O bound, the extra CPU time to allocate and free the > pages wouldn't be really onerous. If you're seeing something > different, I'd love to see some data (perf traces, etc.) to correct my > impressions. > > Cheers, > > - Ted > > ------------------------------------------------------------------------------ > _______________________________________________ > Linux-f2fs-devel mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel > Ted, After reading this and thinking over the last few days you seem to have a correct idea at least from my understanding of cache write back in the Linux kernel. However this really needs to be bench marked first. I would be glad to benchmark this with perf on my system if you can give me the run down of what needs to be bench marked, I have am a idea but would like a little more clarity. Cheers, Nick
------------------------------------------------------------------------------ _______________________________________________ Linux-f2fs-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
