Re: heap_sync seems rather oblivious to partitioned tables (wal_level=minimal)

2018-11-20 Thread David Rowley
(Returning from leave) On Tue, 20 Nov 2018 at 03:19, Alvaro Herrera wrote: > Pushed now, thanks. Many thanks for pushing. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: heap_sync seems rather oblivious to partitioned tables (wal_level=minimal)

2018-11-20 Thread David Rowley
On Tue, 20 Nov 2018 at 01:47, Tomas Vondra wrote: > The patch seems fine to me. Thanks for looking at it. > It's a bit unfortunate that we simply disable the optimization on > partitioned tables instead of fixing it somehow, but I guess it's better > than nothing and no one has a very good idea

Re: heap_sync seems rather oblivious to partitioned tables (wal_level=minimal)

2018-11-19 Thread Alvaro Herrera
On 2018-Nov-19, Tomas Vondra wrote: > On 11/17/18 4:53 PM, Alvaro Herrera wrote: > > Here are versions for branches 10 and 11. The main change is that the > > COPY test didn't have the partitioned table, because it was recently > > introduced (0d5f05cde011) so I backpatched that part also. It's

Re: heap_sync seems rather oblivious to partitioned tables (wal_level=minimal)

2018-11-19 Thread Tomas Vondra
On 11/17/18 4:53 PM, Alvaro Herrera wrote: Here are versions for branches 10 and 11. The main change is that the COPY test didn't have the partitioned table, because it was recently introduced (0d5f05cde011) so I backpatched that part also. It's a bit useless, but I'd rather backpatch the same

Re: heap_sync seems rather oblivious to partitioned tables (wal_level=minimal)

2018-11-17 Thread Alvaro Herrera
FWIW I didn't like the error message: cannot perform FREEZE on a partitioned table but then I noticed other messages also say "cannot perform FREEZE". I think they should all say "cannot perform COPY FREEZE" instead. Not something for this patch to fix, though, I think. -- Álvaro Herrera

Re: heap_sync seems rather oblivious to partitioned tables (wal_level=minimal)

2018-11-17 Thread Alvaro Herrera
Here are versions for branches 10 and 11. The main change is that the COPY test didn't have the partitioned table, because it was recently introduced (0d5f05cde011) so I backpatched that part also. It's a bit useless, but I'd rather backpatch the same thing rather than have different lines there

Re: heap_sync seems rather oblivious to partitioned tables (wal_level=minimal)

2018-11-03 Thread David Rowley
On 2 November 2018 at 15:43, Steve Singer wrote: > I am happy with the changes. > I think the patch is ready for a committer. Many thanks for your review. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: heap_sync seems rather oblivious to partitioned tables (wal_level=minimal)

2018-11-01 Thread Steve Singer
On Thu, 1 Nov 2018, David Rowley wrote: I ended up going with: However, this consideration only applies when is minimal for non-partitioned tables as all commands must write WAL otherwise. I've changed this to: Rows will be frozen only if the table being loaded has been

Re: heap_sync seems rather oblivious to partitioned tables (wal_level=minimal)

2018-10-31 Thread David Rowley
Thanks for looking at this. On 1 November 2018 at 16:07, Steve Singer wrote: > --- a/doc/src/sgml/perform.sgml > +++ b/doc/src/sgml/perform.sgml > @@ -1534,9 +1534,10 @@ SELECT * FROM x, y, a, b, c WHERE something AND > somethingelse; > TRUNCATE command. In such cases no WAL > needs

Re: heap_sync seems rather oblivious to partitioned tables (wal_level=minimal)

2018-10-31 Thread Steve Singer
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: not tested Documentation:tested, failed --- a/doc/src/sgml/perform.sgml +++ b/doc/src/sgml/perform.sgml

Re: heap_sync seems rather oblivious to partitioned tables (wal_level=minimal)

2018-09-30 Thread David Rowley
On 28 September 2018 at 19:22, David Rowley wrote: > I've edited that in the attached patch. Also reworded a comment that > Amit mentioned and made a small change to the COPY FREEZE docs to > mention no support for partitioned tables. Added to the November 'fest.

Re: heap_sync seems rather oblivious to partitioned tables (wal_level=minimal)

2018-09-28 Thread Tomas Vondra
On 09/28/2018 06:02 AM, Amit Langote wrote: > On 2018/09/28 12:12, Michael Paquier wrote: >> On Fri, Sep 28, 2018 at 02:46:30PM +1200, David Rowley wrote: >>> I don't agree that we can skip explaining why one of the optimisations >>> can't be applied just because we've explained why a similar

Re: heap_sync seems rather oblivious to partitioned tables (wal_level=minimal)

2018-09-28 Thread David Rowley
On 28 September 2018 at 15:12, Michael Paquier wrote: > On Fri, Sep 28, 2018 at 02:46:30PM +1200, David Rowley wrote: >> I don't agree that we can skip explaining why one of the optimisations >> can't be applied just because we've explained why a similar >> optimisation cannot be applied

Re: heap_sync seems rather oblivious to partitioned tables (wal_level=minimal)

2018-09-27 Thread Amit Langote
On 2018/09/28 12:12, Michael Paquier wrote: > On Fri, Sep 28, 2018 at 02:46:30PM +1200, David Rowley wrote: >> I don't agree that we can skip explaining why one of the optimisations >> can't be applied just because we've explained why a similar >> optimisation cannot be applied somewhere close by.

Re: heap_sync seems rather oblivious to partitioned tables (wal_level=minimal)

2018-09-27 Thread Amit Langote
On 2018/09/28 11:46, David Rowley wrote: > On 28 September 2018 at 14:25, Amit Langote > wrote: >> Looking at the patch itself, does it seem like both the newly added >> comments repeat the same point (that we'll need per-partition hi_options >> to enable these optimizations) and are pretty close

Re: heap_sync seems rather oblivious to partitioned tables (wal_level=minimal)

2018-09-27 Thread Michael Paquier
On Fri, Sep 28, 2018 at 02:46:30PM +1200, David Rowley wrote: > I don't agree that we can skip explaining why one of the optimisations > can't be applied just because we've explained why a similar > optimisation cannot be applied somewhere close by. I think that the > WAL/FSM optimisation can

Re: heap_sync seems rather oblivious to partitioned tables (wal_level=minimal)

2018-09-27 Thread David Rowley
On 28 September 2018 at 14:25, Amit Langote wrote: > Looking at the patch itself, does it seem like both the newly added > comments repeat the same point (that we'll need per-partition hi_options > to enable these optimizations) and are pretty close to each other? Thanks for looking at this. I

Re: heap_sync seems rather oblivious to partitioned tables (wal_level=minimal)

2018-09-27 Thread Amit Langote
Hi, On 2018/09/28 10:23, David Rowley wrote: > On Thu, 20 Sep 2018 at 11:31, Andres Freund wrote: >> >> On 2018-09-19 12:06:47 +0900, Michael Paquier wrote: >>> On Wed, Sep 19, 2018 at 01:14:10PM +1200, David Rowley wrote: Wouldn't it be better to modify copy.c to just perform the heap_sync

Re: heap_sync seems rather oblivious to partitioned tables (wal_level=minimal)

2018-09-27 Thread David Rowley
On Thu, 20 Sep 2018 at 11:31, Andres Freund wrote: > > On 2018-09-19 12:06:47 +0900, Michael Paquier wrote: > > On Wed, Sep 19, 2018 at 01:14:10PM +1200, David Rowley wrote: > > > Wouldn't it be better to modify copy.c to just perform the heap_sync > > > on just the partitions it touches? > > > >

Re: heap_sync seems rather oblivious to partitioned tables (wal_level=minimal)

2018-09-19 Thread Michael Paquier
On Wed, Sep 19, 2018 at 02:48:58PM -0700, Andres Freund wrote: > On 2018-09-19 12:06:47 +0900, Michael Paquier wrote: >> Yeah, my gut is telling me that this would be the best approach for now, >> still I am not sure that this is the best move in the long term. > > ISTM heap_sync() would be the

Re: heap_sync seems rather oblivious to partitioned tables (wal_level=minimal)

2018-09-19 Thread Andres Freund
On 2018-09-19 12:06:47 +0900, Michael Paquier wrote: > On Wed, Sep 19, 2018 at 01:14:10PM +1200, David Rowley wrote: > > Wouldn't it be better to modify copy.c to just perform the heap_sync > > on just the partitions it touches? > > Yeah, my gut is telling me that this would be the best approach

Re: heap_sync seems rather oblivious to partitioned tables (wal_level=minimal)

2018-09-18 Thread Michael Paquier
On Wed, Sep 19, 2018 at 01:14:10PM +1200, David Rowley wrote: > On 19 September 2018 at 12:21, Tomas Vondra > wrote: >> So apparently CopyFrom() invokes heap_sync() on the partitioned >> relation, which attempts to do mdimmedsync() only on the root. That >> seems like a bug to me. And the root

Re: heap_sync seems rather oblivious to partitioned tables (wal_level=minimal)

2018-09-18 Thread David Rowley
On 19 September 2018 at 12:21, Tomas Vondra wrote: > So apparently CopyFrom() invokes heap_sync() on the partitioned > relation, which attempts to do mdimmedsync() only on the root. That > seems like a bug to me. > > Obviously this only applies to wal_level=minimal. There are multiple > callers

heap_sync seems rather oblivious to partitioned tables (wal_level=minimal)

2018-09-18 Thread Tomas Vondra
Hi, I've been doing some testing today, and it seems heap_sync is somewhat confused by partitioned tables. I'm doing a COPY into a partitioned table (lineitem from TPC-H partitioned per month) like this: - BEGIN; CREATE TABLE