Re: [PATCHES] improve overcommit docs

2003-11-16 Thread Andrew Dunstan
At the time I wrote the original 2.6 was not out even in prerelease, which is why I was deliberately somewhat vague about it. It is still in prerelease, and it will in fact work slightly differently from what was in some 2.4 kernels - there are 2 settings that govern this instead of 1. Here is

[PATCHES] SIGPIPE handling

2003-11-16 Thread Manfred Spraul
Hi, attached is an update of my automatic sigaction patch: I've moved the actual sigaction calls into pqsignal.c and added a helper function (pgsignalinquire(signo)). I couldn't remove the include from fe-connect.c: it's required for the SIGPIPE definition. Additionally I've added a -a flag fo

Re: [PATCHES] ALTER TABLE modifications

2003-11-16 Thread Hannu Krosing
Rod Taylor kirjutas L, 08.11.2003 kell 18:55: > A general re-organization of Alter Table. Node wise, it is a > AlterTableStmt with a list of AlterTableCmds. The Cmds are the > individual actions to be completed (Add constraint, drop constraint, add > column, etc.) > > Processing is done in 2 phas

Re: [PATCHES] SIGPIPE handling

2003-11-16 Thread Bruce Momjian
Better. However, I am confused over when we do sigaction. I thought we were going to do it only if they had a signal handler defined, meaning if (pipehandler != SIG_DFL && pipehandler != SIG_IGN && pipehandler != SIG_ERR) conn->do_sigaction = true

Re: [PATCHES] SRA Win32 sync() code

2003-11-16 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > Not sure how we are going to do this in Win32, but somehow we will have > to record all open files between checkpoints in an area that the > checkpoint process can read during a checkpoint. One reason I like the idea of adopting a sync-when-you-write pol

Re: [PATCHES] SRA Win32 sync() code

2003-11-16 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > Not sure how we are going to do this in Win32, but somehow we will have > > to record all open files between checkpoints in an area that the > > checkpoint process can read during a checkpoint. > > One reason I like the idea of adopti

Re: [PATCHES] SRA Win32 sync() code

2003-11-16 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> One reason I like the idea of adopting a sync-when-you-write policy is >> that it eliminates the need for anything as messy as that. > Yes, but can we do it without causing a performance degredation, and I > would hate to change someth

Re: [PATCHES] SIGPIPE handling

2003-11-16 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > Is running the rest of the > application with SIGPIPE <= SIG_IGN a problem? That is NOT an acceptable thing for a library to do. regards, tom lane ---(end of broadcast)--- TIP 1: s

Re: [PATCHES] SIGPIPE handling

2003-11-16 Thread Manfred Spraul
Bruce Momjian wrote: Better. However, I am confused over when we do sigaction. I thought we were going to do it only if they had a signal handler defined, meaning if (pipehandler != SIG_DFL && pipehandler != SIG_IGN && pipehandler != SIG_ERR) conn-

Re: [PATCHES] improve overcommit docs

2003-11-16 Thread Tom Lane
Andrew Dunstan <[EMAIL PROTECTED]> writes: > At the time I wrote the original 2.6 was not out even in prerelease, > which is why I was deliberately somewhat vague about it. It is still in > prerelease, and it will in fact work slightly differently from what was > in some 2.4 kernels - there are

Re: [PATCHES] SRA Win32 sync() code

2003-11-16 Thread Manfred Spraul
Tom Lane wrote: Seriously though, if we can move the bulk of the writing work into background processes then I don't believe that there will be any significant penalty for regular backends. And I believe that it would be a huge advantage from a correctness point of view if we could stop depending

Re: [PATCHES] improve overcommit docs

2003-11-16 Thread Andrew Dunstan
That covers it extremely well. cheers andrew Tom Lane wrote: Andrew Dunstan <[EMAIL PROTECTED]> writes: At the time I wrote the original 2.6 was not out even in prerelease, which is why I was deliberately somewhat vague about it. It is still in prerelease, and it will in fact work slightly

Re: [PATCHES] SIGPIPE handling

2003-11-16 Thread Kurt Roeckx
On Sun, Nov 16, 2003 at 12:56:10PM +0100, Manfred Spraul wrote: > Hi, > > attached is an update of my automatic sigaction patch: I've moved the > actual sigaction calls into pqsignal.c and added a helper function > (pgsignalinquire(signo)). I couldn't remove the include from > fe-connect.c: it

Re: [PATCHES] SRA Win32 sync() code

2003-11-16 Thread Tom Lane
Manfred Spraul <[EMAIL PROTECTED]> writes: > Which function guarantees that renames of WAL files arrived on the disk? The OS itself is supposed to guarantee that; that's what a journaling file system is for. In any case, I don't think we care. Renaming would apply only to WAL segments that are

Re: [PATCHES] SIGPIPE handling

2003-11-16 Thread Kurt Roeckx
On Sun, Nov 16, 2003 at 06:28:06PM +0100, Kurt Roeckx wrote: > On Sun, Nov 16, 2003 at 12:56:10PM +0100, Manfred Spraul wrote: > > Hi, > > > > attached is an update of my automatic sigaction patch: I've moved the > > actual sigaction calls into pqsignal.c and added a helper function > > (pgsigna

Re: [PATCHES] SIGPIPE handling

2003-11-16 Thread Tom Lane
Manfred Spraul <[EMAIL PROTECTED]> writes: > But how should libpq notice that the caller handles sigpipe signals? > a) autodetection - if the sigpipe handler is not the default, then the > caller knows what he's doing. > b) a new PGsetsignalhandler() function. > c) an additional flag passed to PGc

Re: [PATCHES] SRA Win32 sync() code

2003-11-16 Thread Andrew Dunstan
Manfred Spraul wrote: Tom Lane wrote: Seriously though, if we can move the bulk of the writing work into background processes then I don't believe that there will be any significant penalty for regular backends. And I believe that it would be a huge advantage from a correctness point of view i

Re: [PATCHES] SIGPIPE handling

2003-11-16 Thread Tom Lane
Kurt Roeckx <[EMAIL PROTECTED]> writes: > On Sun, Nov 16, 2003 at 06:28:06PM +0100, Kurt Roeckx wrote: >> Is there a reason we don't make use of the MSG_NOSIGNAL flag to >> send()? Or is the problem in case of SSL? > Oh, seems to be a Linux only thing? That and the SSL problem. I wouldn't objec

Re: [PATCHES] SIGPIPE handling

2003-11-16 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > Is running the rest of the > > application with SIGPIPE <= SIG_IGN a problem? > > That is NOT an acceptable thing for a library to do. Yes, I was afraid of that. Here's another idea. If the signal handler is SIG_DFL, we install our

Re: [PATCHES] SIGPIPE handling

2003-11-16 Thread Bruce Momjian
Tom Lane wrote: > Manfred Spraul <[EMAIL PROTECTED]> writes: > > But how should libpq notice that the caller handles sigpipe signals? > > a) autodetection - if the sigpipe handler is not the default, then the > > caller knows what he's doing. > > b) a new PGsetsignalhandler() function. > > c) an a

Re: [PATCHES] SIGPIPE handling

2003-11-16 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > Yes, I was afraid of that. Here's another idea. If the signal handler > is SIG_DFL, we install our own signal handler for SIGPIPE, and set/clear a > global variable before/after we send(). That would address the speed issue but not the multithread corr

Re: [PATCHES] SRA Win32 sync() code

2003-11-16 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > Tom Lane wrote: > >> One reason I like the idea of adopting a sync-when-you-write policy is > >> that it eliminates the need for anything as messy as that. > > > Yes, but can we do it without causing a performance degredation, and I >

Re: [PATCHES] SIGPIPE handling

2003-11-16 Thread Manfred Spraul
Bruce Momjian wrote: I thought it should be global too, basically testing on the first connection request. What if two PQconnect calls happen at the same time? I would really prefer the manual approach with a new PQsetsighandler function - the autodetection is fragile, it's trivial to find a spec

Re: [PATCHES] SRA Win32 sync() code

2003-11-16 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> Seriously though, if we can move the bulk of the writing work into >> background processes then I don't believe that there will be any >> significant penalty for regular backends. > If the background writer starts using fsync(), we can

Re: [PATCHES] SIGPIPE handling

2003-11-16 Thread Tom Lane
Manfred Spraul <[EMAIL PROTECTED]> writes: > + extern void PQsetsighandling(int internal_sigign); These sorts of things are commonly designed so that the set() operation incidentally returns the previous setting. I'm not sure if anyone would care, but it's only a couple more lines of code to make

Re: [PATCHES] SRA Win32 sync() code

2003-11-16 Thread Jan Wieck
Tom Lane wrote: Bruce Momjian <[EMAIL PROTECTED]> writes: Tom Lane wrote: Seriously though, if we can move the bulk of the writing work into background processes then I don't believe that there will be any significant penalty for regular backends. If the background writer starts using fsync(), we

Re: [PATCHES] SRA Win32 sync() code

2003-11-16 Thread Tom Lane
Jan Wieck <[EMAIL PROTECTED]> writes: > Removing sync() entirely requires very accurate fsync()'ing in the > background writer, the checkpointer and the backends. Basically none of > them can mark a block "clean" if he fails to fsync() the relation later! > This will be a mess to code. Not real

Re: [PATCHES] SRA Win32 sync() code

2003-11-16 Thread Jan Wieck
Tom Lane wrote: Jan Wieck <[EMAIL PROTECTED]> writes: Removing sync() entirely requires very accurate fsync()'ing in the background writer, the checkpointer and the backends. Basically none of them can mark a block "clean" if he fails to fsync() the relation later! This will be a mess to code.

[PATCHES] Alter Table phase 1 -- Please apply to 7.5

2003-11-16 Thread Rod Taylor
Completes: ALTER TABLE ADD COLUMN does not honour DEFAULT and non-CHECK CONSTRAINT ALTER TABLE ADD COLUMN column DEFAULT should fill existing rows with DEFAULT value ALTER TABLE ADD COLUMN column SERIAL doesn't create sequence because of the item ab

Re: [PATCHES] SRA Win32 sync() code

2003-11-16 Thread Tom Lane
Jan Wieck <[EMAIL PROTECTED]> writes: > Well, the bgwriter has basically the same chance the checkpointer has > ... mdblindwrt() in the end, because he doesn't have the relcache handy. We could easily get rid of mdblindwrt --- there is no very good reason that we use the relcache for I/O. There

Re: [PATCHES] SIGPIPE handling

2003-11-16 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > Yes, I was afraid of that. Here's another idea. If the signal handler > > is SIG_DFL, we install our own signal handler for SIGPIPE, and set/clear a > > global variable before/after we send(). > > That would address the speed issue

Re: [PATCHES] SIGPIPE handling

2003-11-16 Thread Bruce Momjian
Manfred Spraul wrote: > Bruce Momjian wrote: > > >I thought it should be global too, basically testing on the first > >connection request. > > > What if two PQconnect calls happen at the same time? > I would really prefer the manual approach with a new PQsetsighandler > function - the autodetecti

Re: [pgsql-hackers-win32] [PATCHES] SRA Win32 sync() code

2003-11-16 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > Tom Lane wrote: > >> Seriously though, if we can move the bulk of the writing work into > >> background processes then I don't believe that there will be any > >> significant penalty for regular backends. > > > If the background write

Re: [pgsql-hackers-win32] [PATCHES] SRA Win32 sync() code

2003-11-16 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > Where am I wrong? I don't think any of this is relevant. There are a certain number of blocks we have to get down to disk before we can declare a transaction committed, and there are a certain number that we have to get down to disk before we can declar

Re: [pgsql-hackers-win32] [PATCHES] SRA Win32 sync() code

2003-11-16 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > Where am I wrong? > > I don't think any of this is relevant. There are a certain number of > blocks we have to get down to disk before we can declare a transaction > committed, and there are a certain number that we have to get down

Re: [pgsql-hackers-win32] [PATCHES] SRA Win32 sync() code

2003-11-16 Thread Bruce Momjian
Tom Lane wrote: > > Do we know that having the background writer fsync a file that was > > written by a backend cause all the data to fsync? I think I could write > > a program to test this by timing each of these tests: > > That might prove something about the particular platform you tested it >

Re: [PATCHES] SIGPIPE handling

2003-11-16 Thread Manfred Spraul
Bruce Momjian wrote: Here is my logic --- 99% of apps don't install a SIGPIPE signal handler, and 90% will not add a SIGPIPE/SIG_IGN call to their applications. I guess I am looking for something that would allow the performance benefit of not doing a pgsignal() call around very send() for the ma