"Tom Lane" <[EMAIL PROTECTED]> wrote
>
> The thing that's nagging at me at the moment is the realization that a
> partitioned hashtable will eat more shared memory than a single
> hashtable. It wasn't that long ago that we had to do some hacking to
> ensure that the buffer hashtable couldn't run out of memory after
> startup, and I'm afraid of re-introducing that failure mode. The lock
> manager can run out of memory without crashing the system, but the
> bufmgr can't (or at least could not in the recent past...)
>
IHMO overflow is not avoidable no matter we use hash or range. Theoretically
seems we could have a data structure like this: (1) a set of k partition
tables, each is with a LWLock and size NBuffers/k; (2) a set of k overflow
tables (actually we only need k-1) plus a LWLock protecting them, each is
with size NBuffers/k. If any partition table overflows, we can assign a
overflow table for it to contain extra hash elements. At run time, the hash
tables for buffer pool may look like this:
[partition 0]
[partition 1][overflow 2]
[partition 2][overflow 0]
[partition 3]
But I am not sure how difficult to implement it in current hash code -
another handwaiving ...
Regards,
Qingqing
---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match
