On 8/26/14, 6:52 PM, Andres Freund wrote:
On 2014-03-21 19:22:31 +0100, Andres Freund wrote:
>Hi,
>
>I've been annoyed at the amount of memory used by the backend local
>PrivateRefCount array for a couple of reasons:
>
>a) The performance impact of AtEOXact_Buffers() on Assert() enabled
> builds is really, really annoying.
>b) On larger nodes, the L1/2/3 cache impact of randomly accessing
> several megabyte big array at a high frequency is noticeable. I've
> seen the access to that to be the primary (yes, really) source of
> pipeline stalls.
>c) On nodes with significant shared_memory the sum of the per-backend
> arrays is a significant amount of memory, that could very well be
> used more beneficially.
>
>So what I have done in the attached proof of concept is to have a small
>(8 currently) array of (buffer, pincount) that's searched linearly when
>the refcount of a buffer is needed. When more than 8 buffers are pinned
>a hashtable is used to lookup the values.
>
>That seems to work fairly well. On the few tests I could run on my
>laptop - I've done this during a flight - it's a small performance win
>in all cases I could test. While saving a fair amount of memory.
Here's the next version of this patch. The major change is that newly
<snip>
The memory savings are clearly visible. During a pgbench scale 350, -cj
128 readonly run the following awk
for pid in $(pgrep -U andres postgres); do
grep VmData/proc/$pid/status;
done | \
awk 'BEGIN { sum = 0 } {sum += $2;} END { if (NR > 0) print sum/NR; else
print 0;print sum;print NR}'
shows:
before:
AVG: 4626.06
TOT: 619892
NR: 134
after:
AVG: 1610.37
TOT: 217400
NR: 135
These results look very encouraging, especially thinking about the cache
impact. It occurs to me that it'd also be nice to have some stats available on
how this is performing; perhaps a dtrace probe for whenever we overflow to the
hash table, and one that shows maximum usage for a statement? (Presumably
that's not much extra code or overhead...)
--
Jim C. Nasby, Data Architect j...@nasby.net
512.569.9461 (cell) http://jim.nasby.net
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
