Hi, On 03/02/2016 03:18 PM, Daniel Verite wrote:
I wrote:postgres=# \copy big2 to /dev/null lost synchronization with server: got message type "d", length -1568669676 The backend aborts with the following backtrace: Program terminated with signal 6, Aborted. #0 0x00007f82ee68e165 in raise () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x00007f82ee6913e0 in abort () from /lib/x86_64-linux-gnu/libc.so.6 #2 0x00007f82ee6c839b in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #3 0x00007f82ee6d1be6 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #4 0x00007f82ee6d698c in free () from /lib/x86_64-linux-gnu/libc.so.6 #5 0x00000000007b5a89 in AllocSetDelete (context=0xffffffffffffffff) at aset.c:643 #6 0x00000000007b63e3 in MemoryContextDelete (context=0x1fa58c8) at mcxt.c:229 #7 0x000000000055fa25 in CopyTo (cstate=0x1fb1050) at copy.c:1967The cause of the crash turns out to be, in enlargeStringInfo(): needed += str->len + 1; /* total space required now */ needed is an int and str->len is an int64, so it overflows when the size has to grow beyond 2^31 bytes, fails to enlarge the buffer and overwrites memory after it. When fixing it with a local int64 copy of the variable, the backend no longer crashes and COPY big2 TO 'file' appears to work. However, getting it to the client with \copy big2 to 'file' still produces the error in psql: lost synchronization with server: got message type "d" and leaves an empty file, so there are more problems to solve to go beyond 2GB text per row.
My guess is this is a problem at the protocol level - the 'd' message is CopyData, and all the messages use int32 to define length. So if there's a 2GB row, it's likely to overflow.
regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
