Re: [HACKERS] libpq Alternate Row Processor

2017-02-14 Thread Jim Nasby
On 2/13/17 8:46 AM, Kyle Gearhart wrote: profile_filler.txt 61,410,901 ???:_int_malloc [/usr/lib64/libc-2.17.so] 38,321,887 ???:_int_free [/usr/lib64/libc-2.17.so] 31,400,139 ???:pqResultAlloc [/usr/local/pgsql/lib/libpq.so.5.10] 22,839,505 ???:pqParseInput3

Re: [HACKERS] libpq Alternate Row Processor

2017-02-13 Thread Kyle Gearhart
On Mon, Feb 13, 2017 Merlin Moncure wrote: >A barebones callback mode ISTM is a complete departure from the classic >PGresult interface. This code is pretty unpleasant IMO: acct->abalance = *((int*)PQgetvalue(res, 0, i)); abalance = acct->__bswap_32(acct->abalance); > Your code is faster but

Re: [HACKERS] libpq Alternate Row Processor

2017-02-13 Thread Merlin Moncure
On Mon, Feb 13, 2017 at 8:46 AM, Kyle Gearhart wrote: > On 2/9/17 7:15 PM, Jim Nasby wrote: >> Can you run a trace to see where all the time is going in the single row >> case? I don't see an obvious time-suck with a quick look through the code. >> It'd be

Re: [HACKERS] libpq Alternate Row Processor

2017-02-13 Thread Kyle Gearhart
On 2/9/17 7:15 PM, Jim Nasby wrote: > Can you run a trace to see where all the time is going in the single row > case? I don't see an obvious time-suck with a quick look through the code. > It'd be interesting to see how things change if you eliminate the filler > column from the SELECT.

Re: [HACKERS] libpq Alternate Row Processor

2017-02-09 Thread Jim Nasby
On 2/8/17 5:11 PM, Kyle Gearhart wrote: Overall, wall clock improves 24%. User time elapsed is a 430% improvement. About half the time is spent waiting on the IO with the callback. With the regular pqRowProcessor only about 16% of the time is spent waiting on IO. To wit...

Re: [HACKERS] libpq Alternate Row Processor

2017-02-05 Thread Kyle Gearhart
From: Tom Lane [mailto:t...@sss.pgh.pa.us]: > Kyle Gearhart writes: >> The guts of pqRowProcessor in libpq does a good bit of work to maintain the >> internal data structure of a PGresult. There are a few use cases where the >> caller doesn't need the ability to

Re: [HACKERS] libpq Alternate Row Processor

2017-02-03 Thread Tom Lane
Kyle Gearhart writes: > The guts of pqRowProcessor in libpq does a good bit of work to maintain the > internal data structure of a PGresult. There are a few use cases where the > caller doesn't need the ability to access the result set row by row, column > by

Re: [HACKERS] libpq Alternate Row Processor

2017-02-03 Thread Jim Nasby
On 2/3/17 3:53 PM, Kyle Gearhart wrote: The guts of pqRowProcessor in libpq does a good bit of work to maintain the internal data structure of a PGresult. There are a few use cases where the caller doesn't need the ability to access the result set row by row, column by column using

[HACKERS] libpq Alternate Row Processor

2017-02-03 Thread Kyle Gearhart
The guts of pqRowProcessor in libpq does a good bit of work to maintain the internal data structure of a PGresult. There are a few use cases where the caller doesn't need the ability to access the result set row by row, column by column using PQgetvalue. Think of an ORM that is just going to