Re: [HACKERS] Allow substitute allocators for PGresult.

2011-12-22 Thread Kyotaro HORIGUCHI
Hello, thank you for taking the time for comment. At Wed, 21 Dec 2011 11:09:59 -0500, Robert Haas robertmh...@gmail.com wrote I find the names of the functions added here to be quite confusing and would suggest renaming them. I expected PQgetAsCstring to do something similar to PQgetvalue,

Re: [HACKERS] Allow substitute allocators for PGresult.

2011-12-21 Thread Robert Haas
On Thu, Dec 8, 2011 at 5:41 AM, Kyotaro HORIGUCHI horiguchi.kyot...@oss.ntt.co.jp wrote:  This is the patch to add the documentation of PGresult custom  storage. It shows in section '31.19. Alternative result  storage'. It would be good to consolidate this into the main patch. I find the

Re: [HACKERS] Allow substitute allocators for PGresult.

2011-12-17 Thread Tom Lane
Greg Smith g...@2ndquadrant.com writes: On 12/01/2011 05:48 AM, Kyotaro HORIGUCHI wrote: xfer timePeak RSS Original : 6.02s 850MB libpq patch + Original dblink: 6.11s 850MB full patch : 4.44s 643MB

Re: [HACKERS] Allow substitute allocators for PGresult.

2011-12-16 Thread Greg Smith
On 12/01/2011 05:48 AM, Kyotaro HORIGUCHI wrote: xfer time Peak RSS Original: 6.02s 850MB libpq patch + Original dblink : 6.11s 850MB full patch : 4.44s 643MB These look like

Re: [HACKERS] Allow substitute allocators for PGresult.

2011-12-08 Thread Kyotaro HORIGUCHI
Hello, The documentation had slipped my mind. This is the patch to add the documentation of PGresult custom storage. It shows in section '31.19. Alternative result storage'. regards, -- Kyotaro Horiguchi NTT Open Source Software Center diff --git a/doc/src/sgml/libpq.sgml

Re: [HACKERS] Allow substitute allocators for PGresult.

2011-12-01 Thread Kyotaro HORIGUCHI
Hello, me I'll put further thought into dblink-plus taking it into me account. .. me Please let me have a little more time. I've inquired the developer of dblink-plus about RegisterResourceReleaseCallback(). He said that the function is in bad compatibility with current implementation. In

Re: [HACKERS] Allow substitute allocators for PGresult.

2011-12-01 Thread Kyotaro HORIGUCHI
Hello, This is the next version of Allow substitute allocators for PGresult. Totally chaning the concept from the previous one, this patch allows libpq to handle alternative tuple store for received tuples. Design guidelines are shown below. - No need to modify existing client code of libpq.

Re: [HACKERS] Allow substitute allocators for PGresult.

2011-12-01 Thread Kyotaro HORIGUCHI
Ouch! I'm sorry for making a reverse patch for the first modification. This is an amendment of the message below. The body text is copied into this message. http://archives.postgresql.org/message-id/20111201.192419.103527179.horiguchi.kyot...@oss.ntt.co.jp === Hello, This is the next

Re: [HACKERS] Allow substitute allocators for PGresult.

2011-11-22 Thread Kyotaro HORIGUCHI
Hello, At Fri, 11 Nov 2011 11:29:30 +0200, Heikki Linnakangas wrote You could use the resource owner mechanism to track them. Register a callback function with RegisterResourceReleaseCallback(). Thank you for letting me know about it. I have dug up a message in pg-hackers refering to the

Re: [HACKERS] Allow substitute allocators for PGresult.

2011-11-12 Thread Tom Lane
Kyotaro HORIGUCHI horiguchi.kyot...@oss.ntt.co.jp writes: Hello. This message is a proposal of a pair of patches that enables the memory allocator for PGresult in libpq to be replaced. Since there seems to be rough consensus that something like this would be a good idea, I looked more closely

Re: [HACKERS] Allow substitute allocators for PGresult.

2011-11-12 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: On 11.11.2011 11:18, Kyotaro HORIGUCHI wrote: For these reasons, I propose to make allocators for PGresult replaceable. You could use the resource owner mechanism to track them. BTW, I just thought of a potentially fatal

Re: [HACKERS] Allow substitute allocators for PGresult.

2011-11-12 Thread Matteo Beccati
On 12/11/2011 07:36, Robert Haas wrote: On Sat, Nov 12, 2011 at 12:48 AM, Tom Lanet...@sss.pgh.pa.us wrote: AIUI Kyotaro-san is just suggesting that the app should be able to provide a substitute malloc function for use in allocating PGresult space (and not, I think, anything else that libpq

Re: [HACKERS] Allow substitute allocators for PGresult.

2011-11-12 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: Well, loading data in a form whereby the application can access it without going through the PGresult accessor functions would be an entirely different (and vastly larger) project. Looking through the thread, I agree that it's a different thing than

[HACKERS] Allow substitute allocators for PGresult.

2011-11-11 Thread Kyotaro HORIGUCHI
Hello. This message is a proposal of a pair of patches that enables the memory allocator for PGresult in libpq to be replaced. The comment at the the begging of pqexpbuffer.c says that libpq should not rely on palloc(). Besides, Tom Lane said that palloc should not be visible outside the

Re: [HACKERS] Allow substitute allocators for PGresult.

2011-11-11 Thread Heikki Linnakangas
On 11.11.2011 11:18, Kyotaro HORIGUCHI wrote: The comment at the the begging of pqexpbuffer.c says that libpq should not rely on palloc(). Besides, Tom Lane said that palloc should not be visible outside the backend(*1) and I agree with it. *1:

Re: [HACKERS] Allow substitute allocators for PGresult.

2011-11-11 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: On 11.11.2011 11:18, Kyotaro HORIGUCHI wrote: The comment at the the begging of pqexpbuffer.c says that libpq should not rely on palloc(). Besides, Tom Lane said that palloc should not be visible outside the backend(*1) and I agree

Re: [HACKERS] Allow substitute allocators for PGresult.

2011-11-11 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: Heikki's idea is probably superior so far as PG backend usage is concerned in isolation, but I wonder if there are scenarios where a client application would like to be able to manage libpq's allocations. The answer to that is certainly 'yes'. It was one

Re: [HACKERS] Allow substitute allocators for PGresult.

2011-11-11 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: * Tom Lane (t...@sss.pgh.pa.us) wrote: Heikki's idea is probably superior so far as PG backend usage is concerned in isolation, but I wonder if there are scenarios where a client application would like to be able to manage libpq's allocations. The

Re: [HACKERS] Allow substitute allocators for PGresult.

2011-11-11 Thread Robert Haas
On Sat, Nov 12, 2011 at 12:48 AM, Tom Lane t...@sss.pgh.pa.us wrote: AIUI Kyotaro-san is just suggesting that the app should be able to provide a substitute malloc function for use in allocating PGresult space (and not, I think, anything else that libpq allocates internally). Basically this