>Kenneth Marshall would like me to post this: > I agree that in order to manage today's large memory machines, we > need to have less contention in our buffer management strategies. > The two main main choke points are in the buffer hash table routines > and in the buffer management linked lists. Unfortunately most of the > code depends on holding the bufmgr lock on entry which eliminates > many chances for parallelism. > > The number of buffer pools should at the very minimum be equal to > the number of processors in the system. This can allow us to greatly > reduce the number of cache-sync cycles if each processor has its > own lock structures for T1-cpuN, T2-cpuN. Now when we allocate a new > buffer, preferentially grab a buffer from the cpu specific queue before > looking in the other queues. Now we have already decreased the amount > of contention by approximately (1/numberCPUs). > > The next item to address is the buf_table concurrency. It appears that > the same code that was used in the hash index update by Tom Lane could > be used to split the buf_table accesses into a per-bucket access using > a per-bucket lock and not a global lock. Modifying the current dyn_hash > search and update code would make it look effectively like Mr. Lane's > new hash index code. > > The final issue is the churn in the MRU/LRU positions on the buffer > management lists. Currently, we always remove a buffer from the list > (T1, T2,...) and then add it to the new list in the MRU position. On > a busy system, for a given query mix a subset of the buffers will be > very busy and compete for the MRU position. What we want to do is > avoid moving a buffer near the top of the list for some definition > of top. One idea, is to have a "per-CPU per-T* counter" which is > incremented as buffers are added to the MRU position. The key is to > store the counter value in the header. Now when we access the buffer > in the list, if the counter is within a value (settable by a GUC) > the buffer is not moved. This would reduce the MRU churn for the > busy buffers near the top of the lists. > > These ideas are very similar to your own speculations. I hope that > their slightly different slant can contribute to this discussion. > Thank you for your time. > > Yours truly, > Kenneth Marshall >
---------------------------(end of broadcast)--------------------------- TIP 7: don't forget to increase your free space map settings
