On 01/02/2014 07:52 PM, Claudio Freire wrote:

No, because this doesn't scale automatically with the bandwidth-delay
product.  It also requires that the client buffers queries and their
parameters even though the network has to do that anyway.

Why not? I'm talking about transport-level packets, btw, not libpq

Yes, the network stack will sometimes do that. But the it doesn't have
to do it. It does it sometimes, which is not the same.

The network inevitably buffers because the speed of light is not infinite.

Here's a concrete example. Suppose the server is 100ms away, and you want to send data at a constant rate of 10 Mbps. The server needs to acknowledge the data you sent, but this acknowledgment arrives after 200 ms. As a result, you've sent 2 Mbits before the acknowledgment arrives, so the network appears to have buffered 250 KB. This effect can actually be used for data storage, called "delay line memory", but it is somewhat out of fashion now.

And buffering algorithms are quite platform-dependent anyway, so it's
not the best idea to make libpq highly reliant on them.

That is why I think libpq needs to keep sending until the first response from the server arrives. Batching a fixed number of INSERTs together in a single conceptual query does not achieve auto-tuning to the buffering characteristics of the path.

So, trusting the network start to do the quick start won't work. For
steady streams of queries, it will work. But not for short bursts,
which will be the most heavily used case I believe (most apps create
short bursts of inserts and not continuous streams at full bandwidth).

Loading data into the database isn't such an uncommon task. Not everything is OLTP.

Florian Weimer / Red Hat Product Security Team

Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:

Reply via email to