Re: [PATCHES] pg_dump additional options for performance

2008-07-26 Thread Simon Riggs
On Fri, 2008-07-25 at 19:16 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > [ pg_dump_beforeafter.v6.patch ] > Unfortunately there's still a lot of work to do, and I don't feel like > doing it so I'm bouncing this patch back for further work. Fair enough. Thanks for the revi

Re: [PATCHES] pg_dump additional options for performance

2008-07-26 Thread Stephen Frost
* Simon Riggs ([EMAIL PROTECTED]) wrote: > The key capability here is being able to split the dump into multiple > pieces. The equivalent capability on restore is *not* required, because > once the dump has been split the restore never needs to be. It might > seem that the patch should be symmetric

Re: [PATCHES] pg_dump additional options for performance

2008-07-26 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > On Fri, 2008-07-25 at 19:16 -0400, Tom Lane wrote: >> The key problem is that pg_restore is broken: > The key capability here is being able to split the dump into multiple > pieces. The equivalent capability on restore is *not* required, because > once the

Re: [PATCHES] pg_dump additional options for performance

2008-07-26 Thread Tom Lane
Stephen Frost <[EMAIL PROTECTED]> writes: >>> Another issue is that the rules for deciding which objects are "before >>> data" and which are "after data" are wrong. In particular ACLs are after >>> data not before data, which is relatively easy to fix. >> >> OK > This was partially why I was c

Re: [PATCHES] pg_dump additional options for performance

2008-07-26 Thread Stephen Frost
* Tom Lane ([EMAIL PROTECTED]) wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > On Fri, 2008-07-25 at 19:16 -0400, Tom Lane wrote: > >> The key problem is that pg_restore is broken: > > > The key capability here is being able to split the dump into multiple > > pieces. The equivalent capabilit

Re: [PATCHES] pg_dump additional options for performance

2008-07-26 Thread Simon Riggs
On Sat, 2008-07-26 at 12:20 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > On Fri, 2008-07-25 at 19:16 -0400, Tom Lane wrote: > >> The key problem is that pg_restore is broken: > > > The key capability here is being able to split the dump into multiple > > pieces. The equiva

Re: [PATCHES] pg_dump additional options for performance

2008-07-26 Thread Tom Lane
Stephen Frost <[EMAIL PROTECTED]> writes: > * Tom Lane ([EMAIL PROTECTED]) wrote: >> This argument is nonsense. The typical usage of this capability, IMHO, >> will be >> >> pg_dump -Fc >whole.dump >> pg_restore --schema-before-data whole.dump >before.sql >> pg_restore --data-only whole.dump >data

Re: [PATCHES] pg_dump additional options for performance

2008-07-26 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > I want to dump tables separately for performance reasons. There are > documented tests showing 100% gains using this method. There is no gain > adding this to pg_restore. There is a gain to be had - parallelising > index creation, but this patch doesn't pro

Re: [PATCHES] pg_dump additional options for performance

2008-07-26 Thread Joshua D. Drake
On Sat, 2008-07-26 at 13:43 -0400, Tom Lane wrote: > Stephen Frost <[EMAIL PROTECTED]> writes: > > I dislike, and doubt that I'd use, this approach. At the end of the > > day, it ends up processing the same (very large amount of data) multiple > > times. > > Well, that's easily avoided: just rep

Re: [PATCHES] pg_dump additional options for performance

2008-07-26 Thread daveg
On Sat, Jul 26, 2008 at 01:56:14PM -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > I want to dump tables separately for performance reasons. There are > > documented tests showing 100% gains using this method. There is no gain > > adding this to pg_restore. There is a gain to b