Re: [PERFORM] Troubles dumping a very large table.

2008-12-29 Thread Dimitri Fontaine
Hi, Le vendredi 26 décembre 2008, Tom Lane a écrit : > Yeah, if he's willing to use COPY BINARY directly. AFAIR there is not > an option to get pg_dump to use it. Would it be possible to consider such an additional switch to pg_dump? Of course the DBA has to know when to use it safely, but if

Re: [PERFORM] Troubles dumping a very large table.

2008-12-28 Thread Ted Allen
I was hoping use pg_dump and not to have to do a manual dump but if that latest solution (moving rows >300mb elsewhere and dealing with them later) does not work I'll try that. Thanks everyone. Merlin Moncure wrote: On Fri, Dec 26, 2008 at 12:38 PM, Tom Lane wrote: Ted Allen writes:

Re: [PERFORM] Troubles dumping a very large table.

2008-12-26 Thread Tom Lane
"Merlin Moncure" writes: > On Fri, Dec 26, 2008 at 12:38 PM, Tom Lane wrote: >> Yeah, the average expansion of bytea data in COPY format is about 3X :-( >> So you need to get the max row length down to around 300mb. I'm curious >> how you got the data in to start with --- were the values assembl

Re: [PERFORM] Troubles dumping a very large table.

2008-12-26 Thread Merlin Moncure
On Fri, Dec 26, 2008 at 12:38 PM, Tom Lane wrote: > Ted Allen writes: >> 600mb measured by get_octet_length on data. If there is a better way to >> measure the row/cell size, please let me know because we thought it was the >> >1Gb problem too. We thought we were being conservative by getting

Re: [PERFORM] Troubles dumping a very large table.

2008-12-26 Thread Tom Lane
Ted Allen writes: > 600mb measured by get_octet_length on data. If there is a better way to > measure the row/cell size, please let me know because we thought it was the > >1Gb problem too. We thought we were being conservative by getting rid of > the larger rows but I guess we need to get ri

Re: [PERFORM] Troubles dumping a very large table.

2008-12-26 Thread Ted Allen
nks, Ted From: Tom Lane [...@sss.pgh.pa.us] Sent: Wednesday, December 24, 2008 12:49 PM To: Ted Allen Cc: pgsql-performance@postgresql.org Subject: Re: [PERFORM] Troubles dumping a very large table. Ted Allen writes: > during the upgrade. The trouble is,

Re: [PERFORM] Troubles dumping a very large table.

2008-12-24 Thread Tom Lane
Ted Allen writes: > during the upgrade. The trouble is, when I dump the largest table, > which is 1.1 Tb with indexes, I keep getting the following error at the > same point in the dump. > pg_dump: SQL command failed > pg_dump: Error message from server: ERROR: invalid string enlargement > r

[PERFORM] Troubles dumping a very large table.

2008-12-24 Thread Ted Allen
(NOTE: I tried sending this email from my excite account and it appears to have been blocked for whatever reason. But if the message does get double posted, sorry for the inconvenience.) Hey all, Merry Christmas Eve, Happy Holidays, and all that good stuff. At my work, I'm trying to upgrade