Re: [HACKERS] PANIC caused by open_sync on Linux

2008-03-24 Thread Bruce Momjian
Added to TODO: * Be more aggressive about creating WAL files http://archives.postgresql.org/pgsql-hackers/2007-10/msg01325.php --- Tom Lane wrote: > Greg Smith <[EMAIL PROTECTED]> writes: > > On Fri, 26 Oct 2007, ITAGAK

Re: [HACKERS] PANIC caused by open_sync on Linux

2007-10-29 Thread Andrew Sullivan
On Fri, Oct 26, 2007 at 10:39:12PM -0400, Greg Smith wrote: > There's a couple of potential to-do list ideas that build on the changes > in this area in 8.3: I think that's the right way to go. It's too bad that this may still happen in 8.3, but we're way past the point that this is a bug fix, I

Re: [HACKERS] PANIC caused by open_sync on Linux

2007-10-28 Thread ITAGAKI Takahiro
Tom Lane <[EMAIL PROTECTED]> wrote: > ITAGAKI Takahiro <[EMAIL PROTECTED]> writes: > > I'd like to add: > > - Remove "filling with zero" before we recycle WAL segments. > > Huh? We have never done that. Oh, sorry. I misread the codes. I would avoid PANIC if I have enough segements at start up

Re: [HACKERS] PANIC caused by open_sync on Linux

2007-10-28 Thread Tom Lane
ITAGAKI Takahiro <[EMAIL PROTECTED]> writes: > I'd like to add: > - Remove "filling with zero" before we recycle WAL segments. Huh? We have never done that. regards, tom lane ---(end of broadcast)--- TIP 4: Have you searche

Re: [HACKERS] PANIC caused by open_sync on Linux

2007-10-28 Thread ITAGAKI Takahiro
Greg Smith <[EMAIL PROTECTED]> wrote: > There's a couple of potential to-do list ideas that build on the changes > in this area in 8.3: > > -Aggressively pre-allocate WAL segments > -Space out checkpoint fsync requests in addition to disk writes > -Consider re-inserting a smarter bgwriter all-

Re: [HACKERS] PANIC caused by open_sync on Linux

2007-10-26 Thread Greg Smith
On Fri, 26 Oct 2007, Tom Lane wrote: The 8.3 bgwriter keeps running even during checkpoints, so it's feasible to add such a feature now. I wonder though whether the walwriter wouldn't be a better place for it. I do, too, but that wasn't available until too late in the 8.3 cycle to consider a

Re: [HACKERS] PANIC caused by open_sync on Linux

2007-10-26 Thread Tom Lane
Greg Smith <[EMAIL PROTECTED]> writes: > The 8.3 bgwriter keeps running even during checkpoints, so it's feasible > to add such a feature now. I wonder though whether the walwriter wouldn't be a better place for it. regards, tom lane ---(end of br

Re: [HACKERS] PANIC caused by open_sync on Linux

2007-10-26 Thread Greg Smith
On Fri, 26 Oct 2007, Andrew Sullivan wrote: Sort of a filthy hack, but what about always having an _extra_ segment around? The bgwriter could do that, no? Now it could. The bgwriter in <=8.2 stops executing when there's a checkpoint going on, and needing more WAL segments because a checkpoi

Re: [HACKERS] PANIC caused by open_sync on Linux

2007-10-26 Thread Andrew Sullivan
On Fri, Oct 26, 2007 at 08:34:49AM -0400, Tom Lane wrote: > we only check for the case once per checkpoint and we don't create a > segment unless there's very little space left. Sort of a filthy hack, but what about always having an _extra_ segment around? The bgwriter could do that, no? A --

Re: [HACKERS] PANIC caused by open_sync on Linux

2007-10-26 Thread Jonah H. Harris
On 10/26/07, Tom Lane <[EMAIL PROTECTED]> wrote: > This ties into a loose end we didn't get to yet: being more aggressive > about creating future WAL segments. ISTM there is no good reason for > clients ever to have to wait for WAL segment creation --- the bgwriter, > or possibly the walwriter, ou

Re: [HACKERS] PANIC caused by open_sync on Linux

2007-10-26 Thread Tom Lane
Greg Smith <[EMAIL PROTECTED]> writes: > On Fri, 26 Oct 2007, ITAGAKI Takahiro wrote: >> Mixed usage of buffered and direct i/o is legal, but enforces complexity >> to kernels. If we simplify it, things would be more relaxed. For >> example, dropping zero-filling and only use direct i/o. Is it po

Re: [HACKERS] PANIC caused by open_sync on Linux

2007-10-25 Thread Greg Smith
On Fri, 26 Oct 2007, ITAGAKI Takahiro wrote: My nearby Linux guy says mixed usage of buffered I/O and direct I/O could cause errors (EIO) on many version of Linux kernels. I'd be curious to get some more information about this--specifically which versions have the problems. I'd heard about s

[HACKERS] PANIC caused by open_sync on Linux

2007-10-25 Thread ITAGAKI Takahiro
I encountered PANICs on CentOS 5.0 when I ran write-mostly workload. It occurs only if wal_sync_method is set to open_sync; there were no problem in fdatasync. It occurred on both Postgres 8.2.5 and 8.3dev. PANIC: could not write to log file 0, segment 212 at offset 3399680, length 73