On Tue, Dec 19, 2017 at 3:31 AM, Kyotaro HORIGUCHI <horiguchi.kyot...@lab.ntt.co.jp> wrote: > I see three kinds of syscache entries. > > A. An entry for an actually existing object. > B. An entry for an object which once existed but no longer. > C. An entry for a just non-existent objects.
I'm not convinced that it's useful to divide things up this way. Regardless of whether the syscache entries is a positive entry, a negative entry for a dropped object, or a negative energy for an object that never existed in the first place, it's valuable if it's likely to get used again and worthless if not. Positive entries may get used repeatedly, or not; negative entries may get used repeatedly, or not. >> Generally, the way I'm viewing this is that a syscache entry means >> paying memory to save CPU time. Each 8kB of memory we use to store >> system cache entries is one less block we have for the OS page cache >> to hold onto our data blocks. If we had an oracle (the kind from > > Sure > >> Delphi, not Redwood City) that told us with perfect accuracy when to >> discard syscache entries, it would throw away syscache entries > > Except for the B in the aboves. The logic seems somewhat alien to > the time-based cleanup but this can be the measure for quick > bloat of some syscahces. I guess I still don't see why B is different. If somebody sits there and runs queries against non-existent table names at top speed, maybe they'll query the same non-existent table entries more than once, in which case keeping the negative entries for the non-existent table names around until they stop doing it may improve performance. If they are sitting there and running queries against randomly-generated non-existent table names at top speed, then they'll generate a lot of catcache bloat, but that's not really any different from a database with a large number of tables that DO exist which are queried at random. Workloads that access a lot of objects, whether those objects exist or not, are going to use up a lot of cache entries, and I guess that just seems OK to me. > Agreed. The following is the whole image of the measure for > syscache bloat considering "quick bloat". (I still think it is > wanted under some situations.) > > 1. If a removal of any objects that make some syscache entries > stale (this cannot be checked without scanning whole a hash so > just queue it into, for exameple, recently_removed_relations > OID hash.) If we just let some sort of cleanup process that generally blows away rarely-used entries get rid of those entries too, then it should handle this case, too, because the cache entries pertaining to removed relations (or schemas) probably won't get used after that (and if they do, we should keep them). So I don't see that there is a need for this, and it drew objections upthread because of the cost of scanning the whole hash table. Batching relations together might help, but it doesn't really seem worth trying to sort out the problems with this idea when we can do something better and more general. > 2. If the number of the oid-hash entries reasches 1000 or 10000 > (mmm. quite arbitrary..), Immediately clean up syscaches that > accepts/needs removed-reloid cleanup. (The oid hash might be > needed separately for each target cache to avoid readandunt > scan, or to get rid a kind of generation management in the oid > hash.) That is bound to draw a strong negative response from Tom, and for good reason. If the number of relations in the working set is 1001 and your cleanup threshold is 1000, cleanups will happen constantly and performance will be poor. This is exactly why, as I said in the second email on this thread, the limit of on the size of the relcache was removed. >> 1. The first time we are due to expand the hash table, we check >> whether we can forestall that expansion by doing a cleanup; if so, we >> do that instead. > > And if there's any entry in the removed-reloid hash it is > considered while cleanup. As I say, I don't think there's any need for a removed-reloid hash. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
