Re: Assorted improvements in pg_dump

2021-10-25 Thread Andres Freund
Hi, On 2021-10-25 16:02:34 -0400, Tom Lane wrote: > So I'd like a better idea, but I'm not sure that that one is better. I guess we could move the prepared-statement handling into a query execution helper. That could then use a hashtable or something similar to check if a certain prepared stateme

Re: Assorted improvements in pg_dump

2021-10-25 Thread Alvaro Herrera
On 2021-Oct-25, Tom Lane wrote: > Yeah, I wasn't too happy with the static bools either. However, each > function would need its own field in the struct, which seems like a > maintenance annoyance, plus a big hazard for future copy-and-paste > changes (ie, copy and paste the wrong flag name -> tr

Re: Assorted improvements in pg_dump

2021-10-25 Thread Tom Lane
Andres Freund writes: > On 2021-10-24 17:10:55 -0400, Tom Lane wrote: >> +static bool query_prepared = false; > I wonder if it'd be better to store this in Archive or such. The approach with > static variables might run into problems with parallel pg_dump at some > point. These objects aren't

Re: Assorted improvements in pg_dump

2021-10-25 Thread Andres Freund
Hi, On 2021-10-24 17:10:55 -0400, Tom Lane wrote: > 0004 is not committable as-is, because it assumes that the source > server has single-array unnest(), which is not true before 8.4. > We could fix that by using "oid = ANY(array-constant)" conditions > instead, but I'm unsure about the performanc

Re: Assorted improvements in pg_dump

2021-10-24 Thread Tom Lane
Justin Pryzby writes: > On Sun, Oct 24, 2021 at 05:10:55PM -0400, Tom Lane wrote: >> + if (tbloids->len > 1) > I think this should say > + if (tbloids->len > 0) No, >1 is the correct test, because it's checking the string length and we started by stuffing a '{' into

Re: Assorted improvements in pg_dump

2021-10-24 Thread Justin Pryzby
On Sun, Oct 24, 2021 at 05:10:55PM -0400, Tom Lane wrote: > 0003 is the same except I added a missing free(). > > 0004 is a new patch based on an idea from Andres Freund [1]: > in the functions that repetitively issue the same query against > different tables, issue just one query and use a WHERE

Re: Assorted improvements in pg_dump

2021-10-22 Thread Hans Buschmann
Hello Tom! I noticed you are improving pg_dump just now. Some time ago I experimented with a customer database dump in parallel directory mode -F directory -j (2-4) I noticed it took quite long to complete. Further investigation showed that in this mode with multiple jobs the tables are proce

Re: Assorted improvements in pg_dump

2021-10-21 Thread Tom Lane
Justin Pryzby writes: > On Wed, Oct 20, 2021 at 05:14:45PM -0400, Tom Lane wrote: >> Arguably, 0003 is a bug fix that we should consider back-patching. >> However, I've not heard field reports of the problems it fixes, >> so maybe there's no need to bother. > FYI, I see this issue happen in produ

Re: Assorted improvements in pg_dump

2021-10-21 Thread Justin Pryzby
On Wed, Oct 20, 2021 at 05:14:45PM -0400, Tom Lane wrote: > Lastly, patch 0003 addresses the concern I raised at [3] that it's > unsafe to call pg_get_partkeydef() and pg_get_expr(relpartbound) > in getTables(). Looking closer I realized that we can't cast > pg_class.reloftype to regtype at that p