On 2/3/17 3:53 PM, Kyle Gearhart wrote:
The guts of pqRowProcessor in libpq does a good bit of work to maintain the 
internal data structure of a PGresult.  There are a few use cases where the 
caller doesn't need the ability to access the result set row by row, column by 
column using PQgetvalue.  Think of an ORM that is just going to copy the data 
from PGresult for each row into its own structures.

I've got a working proof of concept that allows the caller to attach a callback 
that pqRowProcessor will call instead of going thru its own routine.  This 
eliminates all the copying of data from the PGconn buffer to a PGresult buffer 
and then ultimately a series of PQgetvalue calls by the client.  The callback 
allows the caller to receive each row's data directly from the PGconn buffer.

It would require exposing struct pgDataValue in libpq-fe.h.  The prototype for 
the callback pointer would be:
int (*PQrowProcessorCB)(PGresult*, const PGdataValue*, int col_count, void 

My initial testing shows a significant performance improvement.  I'd like some 
opinions on this before wiring up a performance proof and updating the 
documentation for a formal patch submission.

I just did essentially the same thing for SPI (use a callback to allow the caller to handle the tuple instead of shoving it into a tuplestore). A simple test in plpython showed a 460% improvement.
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com
855-TREBLE2 (855-873-2532)

Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:

Reply via email to