I am trying to create version of COPY command which can scatter/replicate data
to different nodes based on some distribution method.
There is some master process, having information about data distribution, to
which all clients are connected.
This master process should receive copied data from client and scatters tuples
May be somebody can recommend me the best way of implementing such COPY agent?
The obvious plan is the following:
1. Register utility callback
2. Handle T_CopyStmt in this callback
3. Use BeginCopyFrom/NextCopyFrom to receive tuples from client
4. Calculate distribution function for the received tuple
5. Establish connection with correspondent node (if not yet established) and
start the same COPY command to this node (if not started yet).
6. Send data to this node using PQputCopyData.
The problem is with step 6: I do not see any way to copy received data to the
NextCopyFrom returns array of values (Dutums) of tuple columns. But there are
no public methods to send tuple to the copy stream.
All this logic is implemented in src/backend/commands/copy.c and is not
available outside this module.
It is more or less clear how to do it using text or CSV mode: I can use
NextCopyFromRawFields and then construct a line with comma separated list of
But how to handle binary mode? Also, I suspect that copy in text mode is
significantly slower than in binary mode, isn't it?
The dirty solution is just to cut&paste copy.c code. But may be there is some
more elegant way?
Thanks in advance,
Sent via pgsql-hackers mailing list (email@example.com)
To make changes to your subscription: