On 01/04/2014 11:11 PM, Tom Lane wrote:
knizhnik <knizh...@garret.ru> writes:
On 01/04/2014 12:05 PM, David Fetter wrote:
Is there some way not to use shared memory for it?
No, IMCS ("In-Memory Columnar Store") is storing data in shared memory.
It would probably be better if it made use of the dynamic shared memory
features that exist in HEAD.

                        regards, tom lane

Thank you, I will try it.
But I have some concerns:

1. I want IMCS to work with PostgreSQL versions not supporting DSM (dynamic shared memory), like 9.2, 9.3.1,...

2. IMCS is using PostgreSQL hash table implementation (ShmemInitHash, hash_search,...) May be I missed something - I just noticed DSM and have no chance to investigate it, but looks like hash table can not be allocated in DSM...

3. IMCS is allocating memory using ShmemAlloc. In case of using DSM I have to provide own allocator (although creation of non-releasing memory allocator should not be a big issue).

4. Current implementation of DSM still suffers from 256Gb problem. Certainly I can create multiple segments and so provide workaround without using huge pages, but it complicates allocator.

5. I wonder if I dynamically add new DSM segment - will it be available for other PostgreSQL processes? For example I run query which loads data in IMCS and so needs more space and allocates new DSM segment. Then another query is executed by other PostgreSQL process which tries to access this data. This process is not forked from the process created this new DSM segment, so I do not understand how this segment will be mapped to the address space of this process, preserving address... Certainly I can prohibit dynamic extension of IMCS storage (hoping that in this case there will be no such problem with DSM). But in this case we will loose the main advantage of using DSM instead of old schema of plugin's private shared memory.

6. IMCS has some configuration parameters which has to be set through postgresql.conf. So in any case user has to edit postgresql.conf file. In case of using DSM it will be not necessary to add IMCS to shared_preload_libraries list. But I do not think that it is so restrictive and critical requirement, is it?

Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:

Reply via email to