Re: [HACKERS] Reduce WAL logging of INSERT SELECT

2011-08-11 Thread Bruce Momjian
Simon Riggs wrote: > On Sat, Aug 6, 2011 at 4:16 AM, Bruce Momjian wrote: > > > Well, if the table is created in the same transaction (which is the only > > case under consideration), no other sessions can write to the table so > > you are just writing the entire table on commit, rather than to t

Re: [HACKERS] Reduce WAL logging of INSERT SELECT

2011-08-06 Thread Jeff Davis
On Fri, 2011-08-05 at 23:16 -0400, Bruce Momjian wrote: > Well, if the table is created in the same transaction (which is the only > case under consideration), no other sessions can write to the table so > you are just writing the entire table on commit, rather than to the WAL. The transaction can

Re: [HACKERS] Reduce WAL logging of INSERT SELECT

2011-08-06 Thread Tom Lane
Jaime Casanova writes: > On Sat, Aug 6, 2011 at 11:05 AM, Heikki Linnakangas > wrote: >> It can be very helpful when loading a lot of data, so I'm not in favor of >> removing it altogether. Maybe WAL-log the first 1 rows or such normally, >> and skip WAL after that. Of course, loading 10001 r

Re: [HACKERS] Reduce WAL logging of INSERT SELECT

2011-08-06 Thread Kevin Grittner
Heikki Linnakangas wrote: > On 06.08.2011 13:13, Simon Riggs wrote: >> I think we should remove the COPY optimisation because of this and >> definitely not extend INSERT SELECT to perform it automatically. > > It can be very helpful when loading a lot of data, so I'm not in > favor of removing it

Re: [HACKERS] Reduce WAL logging of INSERT SELECT

2011-08-06 Thread Jaime Casanova
On Sat, Aug 6, 2011 at 11:05 AM, Heikki Linnakangas wrote: > On 06.08.2011 13:13, Simon Riggs wrote: >> >> I think we should remove the COPY optimisation because of this and >> definitely not extend INSERT SELECT to perform it automatically. > > It can be very helpful when loading a lot of data, s

Re: [HACKERS] Reduce WAL logging of INSERT SELECT

2011-08-06 Thread Heikki Linnakangas
On 06.08.2011 13:13, Simon Riggs wrote: I think we should remove the COPY optimisation because of this and definitely not extend INSERT SELECT to perform it automatically. It can be very helpful when loading a lot of data, so I'm not in favor of removing it altogether. Maybe WAL-log the first

Re: [HACKERS] Reduce WAL logging of INSERT SELECT

2011-08-06 Thread Simon Riggs
On Sat, Aug 6, 2011 at 4:16 AM, Bruce Momjian wrote: > Well, if the table is created in the same transaction (which is the only > case under consideration), no other sessions can write to the table so > you are just writing the entire table on commit, rather than to the WAL. Below a certain poin

Re: [HACKERS] Reduce WAL logging of INSERT SELECT

2011-08-06 Thread Heikki Linnakangas
On 06.08.2011 06:32, Gokulakannan Somasundaram wrote: However, for small operations it's a net loss - you avoid writing a WAL record, but you have to fsync() the heap instead. If you only modify a few rows, the extra fsync (or fsyncs if there are indexes too) is more expensive than writing the W

Re: [HACKERS] Reduce WAL logging of INSERT SELECT

2011-08-05 Thread Gokulakannan Somasundaram
> > However, for small operations it's a net loss - you avoid writing a WAL > record, but you have to fsync() the heap instead. If you only modify a few > rows, the extra fsync (or fsyncs if there are indexes too) is more expensive > than writing the WAL. > > We'd need a heuristic to decide whether

Re: [HACKERS] Reduce WAL logging of INSERT SELECT

2011-08-05 Thread Bruce Momjian
Jeff Davis wrote: > On Thu, 2011-08-04 at 18:07 -0400, Bruce Momjian wrote: > > I am confused how generating WAL traffic that is larger than the heap > > file we are fsync'ing can possibly be slower. Are you just throwing out > > an idea to try to make me prove it? > > That's worded in a slightly

Re: [HACKERS] Reduce WAL logging of INSERT SELECT

2011-08-05 Thread Jeff Davis
On Thu, 2011-08-04 at 18:07 -0400, Bruce Momjian wrote: > I am confused how generating WAL traffic that is larger than the heap > file we are fsync'ing can possibly be slower. Are you just throwing out > an idea to try to make me prove it? That's worded in a slightly confusing way, but here is th

Re: [HACKERS] Reduce WAL logging of INSERT SELECT

2011-08-05 Thread Simon Riggs
On Thu, Aug 4, 2011 at 11:07 PM, Bruce Momjian wrote: > Simon Riggs wrote: >> On Thu, Aug 4, 2011 at 10:46 PM, Bruce Momjian wrote: >> >> > Right. ?I brought up SELECT INTO because you could make the argument >> > that INSERT ... SELECT is not a utility command like the other ones and >> > theref

Re: [HACKERS] Reduce WAL logging of INSERT SELECT

2011-08-04 Thread Heikki Linnakangas
On 05.08.2011 04:23, Jeff Davis wrote: On Thu, 2011-08-04 at 20:55 -0400, Bruce Momjian wrote: It would act like COPY, meaning the table would have to be truncated or created in the same transaction. Well, in that case it could work for any INSERT. No need for a SELECT to be involved. For that

Re: [HACKERS] Reduce WAL logging of INSERT SELECT

2011-08-04 Thread Jeff Davis
On Thu, 2011-08-04 at 20:55 -0400, Bruce Momjian wrote: > It would act like COPY, meaning the table would have to be truncated or > created in the same transaction. Well, in that case it could work for any INSERT. No need for a SELECT to be involved. For that matter, why not make it work for DELET

Re: [HACKERS] Reduce WAL logging of INSERT SELECT

2011-08-04 Thread Robert Haas
On Thu, Aug 4, 2011 at 8:55 PM, Bruce Momjian wrote: > Jeff Davis wrote: >> On Thu, 2011-08-04 at 17:46 -0400, Bruce Momjian wrote: >> > Right.  I brought up SELECT INTO because you could make the argument >> > that INSERT ... SELECT is not a utility command like the other ones and >> > therefore

Re: [HACKERS] Reduce WAL logging of INSERT SELECT

2011-08-04 Thread Bruce Momjian
Jeff Davis wrote: > On Thu, 2011-08-04 at 17:46 -0400, Bruce Momjian wrote: > > Right. I brought up SELECT INTO because you could make the argument > > that INSERT ... SELECT is not a utility command like the other ones and > > therefore can't be done easily, but CREATE TABLE AS is internal SELECT

Re: [HACKERS] Reduce WAL logging of INSERT SELECT

2011-08-04 Thread Jeff Davis
On Thu, 2011-08-04 at 17:46 -0400, Bruce Momjian wrote: > Right. I brought up SELECT INTO because you could make the argument > that INSERT ... SELECT is not a utility command like the other ones and > therefore can't be done easily, but CREATE TABLE AS is internal SELECT > INTO and implemented in

Re: [HACKERS] Reduce WAL logging of INSERT SELECT

2011-08-04 Thread Bruce Momjian
Simon Riggs wrote: > On Thu, Aug 4, 2011 at 10:46 PM, Bruce Momjian wrote: > > > Right. ?I brought up SELECT INTO because you could make the argument > > that INSERT ... SELECT is not a utility command like the other ones and > > therefore can't be done easily, but CREATE TABLE AS is internal SEL

Re: [HACKERS] Reduce WAL logging of INSERT SELECT

2011-08-04 Thread Simon Riggs
On Thu, Aug 4, 2011 at 10:46 PM, Bruce Momjian wrote: > Right.  I brought up SELECT INTO because you could make the argument > that INSERT ... SELECT is not a utility command like the other ones and > therefore can't be done easily, but CREATE TABLE AS is internal SELECT > INTO and implemented in

Re: [HACKERS] Reduce WAL logging of INSERT SELECT

2011-08-04 Thread Bruce Momjian
Andrew Dunstan wrote: > > > On 08/04/2011 04:55 PM, Tom Lane wrote: > > Bruce Momjian writes: > >> One thing we don't optimize is INSERT ... SELECT when the table is > >> created or truncated in the same transaction. Seems we could. > >> We optimize CREATE TABLE AS which is effectively SELECT .

Re: [HACKERS] Reduce WAL logging of INSERT SELECT

2011-08-04 Thread Andrew Dunstan
On 08/04/2011 04:55 PM, Tom Lane wrote: Bruce Momjian writes: One thing we don't optimize is INSERT ... SELECT when the table is created or truncated in the same transaction. Seems we could. We optimize CREATE TABLE AS which is effectively SELECT ... INTO using a different syntax. Is this a

Re: [HACKERS] Reduce WAL logging of INSERT SELECT

2011-08-04 Thread Tom Lane
Bruce Momjian writes: > One thing we don't optimize is INSERT ... SELECT when the table is > created or truncated in the same transaction. Seems we could. > We optimize CREATE TABLE AS which is effectively SELECT ... INTO using a > different syntax. Is this a TODO? Considering that SELECT INTO