* Greg Stark (st...@mit.edu) wrote: > On Thu, Apr 17, 2014 at 12:21 PM, Stephen Frost <sfr...@snowman.net> wrote: > > Ehhh. No. If it's a hot page that we've been holding in *our* cache > > long enough, the kernel will happily evict it as 'cold' from *its* > > cache, leading to... > > This is a whole nother problem. > > It is worrisome that we could be benchmarking the page replacement > algorithm in Postgres and choose a page replacement algorithm that > chooses pages that performs well because it tends to evict pages that > are in the OS cache. And then one day (hopefully not too far off) > we'll fix the double buffering problem and end up with a strange > choice of page replacement algorithm.
That's certainly possible but I don't see the double buffering problem going away any time particularly soon and, even if it does, it's likely to either a) mean we're just using the kernel's cache (eg: something w/ mmap, etc), or b) will involve so many other changes that this will end up getting changed anyway. In any case, while I think we should document any such cache management system we employ as having this risk, I don't think we should worry about it terribly much. > It also means that every benchmark is super sensitive to the how large > a fraction of system memory Postgres is managing. If A benchmark of a > page replacement algorithm with 3GB shared buffers might perform well > compared to others on a system with 8GB or 32GB total RAM but actually > be choosing pages very poorly in normal terms and perform terribly on > a system with 4GB total ram. I'm not following you here- benchmarks are already sensitive to how much of the system's memory PG is managing (and how much ends up being *dedicated* to PG's cache and therefore unavailable for other work). > Ideally what I would like to see is instrumentation of Postgres's > buffer pinning so we can generate various test loads and then just run > the different algorithms on them and measure precisely how many page > evictions it's causing and when how often it's choosing pages that > need to be read in soon after and so on. We shouldn't have to run > Postgres to get these counts at all, just run the algorithm as we read > through a text file (or database table) listing the pages being > accessed. Go for it. I'd love to see that also. Thanks, Stephen
signature.asc
Description: Digital signature