On Mon, Jul 30, 2012 at 10:26 PM, Jan Wieck <janwi...@yahoo.com> wrote: > On 7/30/2012 10:31 PM, Leon Smith wrote: >> >> This is not necessarily true, on multiple levels. I mean, some of >> the programs I write are highly concurrent, and this form of batching >> would have almost no risk of stalling the network buffer. And >> the possible use case would be when you are dealing with very small >> rows, when there would typically be several rows inside a single >> network packet or network buffer. > > > With "highly concurrent" you mean multi-threaded? Like one thread reads the > rows in batches and pushes them into a queue while another thread processes > them from that queue? > > If that is the case, then you just added a useless layer of buffering and > the need for thread/thread context switches to PQsetSingleRowMode. Libpq's > "receiver thread" is the kernel itself. Libpq tries to never read partial > kernel buffers already. It always makes sure that there are at least 8K of > free space in the inBuffer. In the case you describe above, where several > rows fit into a single packet, libpq will receive them with a single system > call in one read(2), then the application can get them as fast as possible, > without causing any further context switches because they are already in the > inBuffer.
Yeah: with asynchronous query processing the query gets sent and control returns immediately to your code: that's the whole point. Even if some data races to the network buffer, libpq doesn't 'see' any data until you tell it to by asking for a result (which can block) or draining the buffers with PQconsumeInput. So there is no race in the traditional sense and I'm ok with the PQsetSingleRowMode as such. Removing malloc/free on row iteration seems only to be possible via one of two methods: either a) you introduce a non-PGresult based method of data extraction or b) you preserve the PGresult across row iterations. merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
