Re: [PATCHES] Space management for PGresult

2005-11-24 Thread Simon Riggs
On Thu, 2005-11-24 at 12:32 -0500, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > We really ought to be streaming the result back to the user, not > > downloading it all into a massive client side chunk of memory. > > Have you been paying any attention to the multiple previous discu

Re: [PATCHES] Space management for PGresult

2005-11-24 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > We really ought to be streaming the result back to the user, not > downloading it all into a massive client side chunk of memory. Have you been paying any attention to the multiple previous discussions of that point? (Latest was on pgsql-interfaces within

Re: [PATCHES] Space management for PGresult

2005-11-24 Thread Simon Riggs
On Wed, 2005-11-23 at 16:21 +0900, Atsushi Ogawa wrote: > In space management for PGresult of libpq, the block size of PGresult > is always PGRESULT_DATA_BLOCKSIZE(2048bytes). Therefore, when a large > result of query is received, malloc is executed many times. > > My proposal is to enlarge the si

Re: [PATCHES] Space management for PGresult

2005-11-24 Thread Tom Lane
Atsushi Ogawa <[EMAIL PROTECTED]> writes: > (1)accounts table (4 columns, 1,000,000 tuples) > malloc calls allocated(bytes) unused(bytes) execution time > -- > 8.1.0 80,542164,950,016 2,94

Re: [PATCHES] Space management for PGresult

2005-11-24 Thread Atsushi Ogawa
Alvaro Herrera wrote: > Tom Lane wrote: > > Atsushi Ogawa <[EMAIL PROTECTED]> writes: > > > The number of malloc calls at pqResultAlloc: > > > 8.1.0 : 80542 > > > patched:86 > > > > > Execution time: > > > 8.1.0 : 6.80 sec > > > patched: 6.73 sec > > > > This hardly seems worth adding any

Re: [PATCHES] Space management for PGresult

2005-11-23 Thread Alvaro Herrera
Tom Lane wrote: > Atsushi Ogawa <[EMAIL PROTECTED]> writes: > > The number of malloc calls at pqResultAlloc: > > 8.1.0 : 80542 > > patched:86 > > > Execution time: > > 8.1.0 : 6.80 sec > > patched: 6.73 sec > > This hardly seems worth adding any complexity for ... What about memory usa

Re: [PATCHES] Space management for PGresult

2005-11-23 Thread Tom Lane
Atsushi Ogawa <[EMAIL PROTECTED]> writes: > The number of malloc calls at pqResultAlloc: > 8.1.0 : 80542 > patched:86 > Execution time: > 8.1.0 : 6.80 sec > patched: 6.73 sec This hardly seems worth adding any complexity for ... regards, tom lane --

[PATCHES] Space management for PGresult

2005-11-22 Thread Atsushi Ogawa
In space management for PGresult of libpq, the block size of PGresult is always PGRESULT_DATA_BLOCKSIZE(2048bytes). Therefore, when a large result of query is received, malloc is executed many times. My proposal is to enlarge the size of the block whenever the block is allocated. The size of first