Re: [HACKERS] xlogdump fixups and WAL log question.

2006-10-24 Thread Markus Schaber
Hi, Simon, Simon Riggs wrote: 1. Provide a filter that can be easily used by archive_command to remove full page writes from WAL files. This would require us to disable the file size test when we begin recovery on a new WAL files, plus would need to redesign initial location of the

Re: [HACKERS] xlogdump fixups and WAL log question.

2006-10-22 Thread Simon Riggs
On Sat, 2006-10-21 at 19:24 -0400, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: But they can be re-created anew with the same name each time? Or I guess not, but you redefine a view every 30 minutes to point to the latest one? If so, then I have a patch that will speed up COPY

Re: [HACKERS] xlogdump fixups and WAL log question.

2006-10-22 Thread Alvaro Herrera
Simon Riggs wrote: On Sat, 2006-10-21 at 19:24 -0400, Tom Lane wrote: Can you make the patch cover the case of begin; truncate foo; copy foo from ... commit; It might be infeasible to detect this case, but if it's not ... I think it is possible to detect this

Re: [HACKERS] xlogdump fixups and WAL log question.

2006-10-22 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: Simon Riggs wrote: I think it is possible to detect this case without making catalog entries, so I'll give this a try. Methinks that the truncate *must* be the immediately preceding command, otherwise we might have a trigger executing to put rows back

Re: [HACKERS] xlogdump fixups and WAL log question.

2006-10-22 Thread Simon Riggs
On Sun, 2006-10-22 at 12:12 -0400, Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: Simon Riggs wrote: I think it is possible to detect this case without making catalog entries, so I'll give this a try. Methinks that the truncate *must* be the immediately preceding command,

Re: [HACKERS] xlogdump fixups and WAL log question.

2006-10-21 Thread Simon Riggs
On Fri, 2006-10-20 at 17:04 -0400, Theo Schlossnagle wrote: On Oct 20, 2006, at 4:24 PM, Simon Riggs wrote: Is it possible to create tables in fashion that will not write info to the WAL log -- knowingly and intentionally making them unrecoverable? This is very desirable for us. We

Re: [HACKERS] xlogdump fixups and WAL log question.

2006-10-21 Thread Martijn van Oosterhout
On Sat, Oct 21, 2006 at 10:37:51AM +0100, Simon Riggs wrote: Turning off WAL is a difficult topic. Without it you have no crash recovery, which IMHO everybody says they don't care about until they crash, then they realise. It's hard to be selective about writing WAL for specific operations

Re: [HACKERS] xlogdump fixups and WAL log question.

2006-10-21 Thread Theo Schlossnagle
On Oct 21, 2006, at 6:08 AM, Martijn van Oosterhout wrote: On Sat, Oct 21, 2006 at 10:37:51AM +0100, Simon Riggs wrote: Turning off WAL is a difficult topic. Without it you have no crash recovery, which IMHO everybody says they don't care about until they crash, then they realise. It's hard

Re: [HACKERS] xlogdump fixups and WAL log question.

2006-10-21 Thread Simon Riggs
On Sat, 2006-10-21 at 09:00 -0400, Theo Schlossnagle wrote: On Oct 21, 2006, at 6:08 AM, Martijn van Oosterhout wrote: On Sat, Oct 21, 2006 at 10:37:51AM +0100, Simon Riggs wrote: Turning off WAL is a difficult topic. Without it you have no crash recovery, which IMHO everybody says they

Re: [HACKERS] xlogdump fixups and WAL log question.

2006-10-21 Thread Theo Schlossnagle
On Oct 21, 2006, at 3:12 PM, Simon Riggs wrote: On Sat, 2006-10-21 at 09:00 -0400, Theo Schlossnagle wrote: On Oct 21, 2006, at 6:08 AM, Martijn van Oosterhout wrote: On Sat, Oct 21, 2006 at 10:37:51AM +0100, Simon Riggs wrote: Turning off WAL is a difficult topic. Without it you have no

Re: [HACKERS] xlogdump fixups and WAL log question.

2006-10-21 Thread Simon Riggs
On Sat, 2006-10-21 at 15:17 -0400, Theo Schlossnagle wrote: On Oct 21, 2006, at 3:12 PM, Simon Riggs wrote: On Sat, 2006-10-21 at 09:00 -0400, Theo Schlossnagle wrote: On Oct 21, 2006, at 6:08 AM, Martijn van Oosterhout wrote: On Sat, Oct 21, 2006 at 10:37:51AM +0100, Simon Riggs wrote:

Re: [HACKERS] xlogdump fixups and WAL log question.

2006-10-21 Thread Theo Schlossnagle
On Oct 21, 2006, at 4:40 PM, Simon Riggs wrote: On Sat, 2006-10-21 at 15:17 -0400, Theo Schlossnagle wrote: On Oct 21, 2006, at 3:12 PM, Simon Riggs wrote: On Sat, 2006-10-21 at 09:00 -0400, Theo Schlossnagle wrote: On Oct 21, 2006, at 6:08 AM, Martijn van Oosterhout wrote: On Sat, Oct

Re: [HACKERS] xlogdump fixups and WAL log question.

2006-10-21 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: But they can be re-created anew with the same name each time? Or I guess not, but you redefine a view every 30 minutes to point to the latest one? If so, then I have a patch that will speed up COPY when in the same transaction as the table that created

[HACKERS] xlogdump fixups and WAL log question.

2006-10-20 Thread Theo Schlossnagle
Not sure who cares, so xzilla indicated I should drop a note here. I just made the xlogdump stuff work for 8.1 (trivial) and fixed a few other small issues that caused it to not work right both generally and in our environment. http://pgfoundry.org/tracker/index.php?

Re: [HACKERS] xlogdump fixups and WAL log question.

2006-10-20 Thread Tom Lane
Theo Schlossnagle [EMAIL PROTECTED] writes: Is it possible to create tables in fashion that will not write info to the WAL log -- knowingly and intentionally making them unrecoverable? Use temp tables? Also, it's likely that much of the WAL volume is full-page images. While you can't

Re: [HACKERS] xlogdump fixups and WAL log question.

2006-10-20 Thread Theo Schlossnagle
On Oct 20, 2006, at 1:58 PM, Tom Lane wrote: Theo Schlossnagle [EMAIL PROTECTED] writes: Is it possible to create tables in fashion that will not write info to the WAL log -- knowingly and intentionally making them unrecoverable? Use temp tables? temp tables won't work too well -- unless

Re: [HACKERS] xlogdump fixups and WAL log question.

2006-10-20 Thread Simon Riggs
On Fri, 2006-10-20 at 13:18 -0400, Theo Schlossnagle wrote: Not sure who cares, so xzilla indicated I should drop a note here. I just made the xlogdump stuff work for 8.1 (trivial) and fixed a few other small issues that caused it to not work right both generally and in our environment.

Re: [HACKERS] xlogdump fixups and WAL log question.

2006-10-20 Thread Theo Schlossnagle
On Oct 20, 2006, at 4:24 PM, Simon Riggs wrote: On Fri, 2006-10-20 at 13:18 -0400, Theo Schlossnagle wrote: Not sure who cares, so xzilla indicated I should drop a note here. I just made the xlogdump stuff work for 8.1 (trivial) and fixed a few other small issues that caused it to not work