Re: [HACKERS] Postgres-R: internal messaging

2008-07-30 Thread Markus Wanner
Hi, That's now changed in today's snapshot of Postgres-R: the postmaster no longer uses imessages (and thus shared memory) to communicate with the replication manager. Instead the manager signals the postmaster using a newish PMSIGNAL for requesting new helper backends. It now only requests o

Re: [HACKERS] Postgres-R: internal messaging

2008-07-24 Thread Markus Wanner
Hi, Tom Lane wrote: I hope you're not expecting the contents of shared memory to still be trustworthy after a backend crash. Hm.. that's a good point. So I either need to bullet-proof the imessages with checksums or some such. I'm not sure that's doable reliably. Not to speak about performan

Re: [HACKERS] Postgres-R: internal messaging

2008-07-23 Thread Tom Lane
Markus Wanner <[EMAIL PROTECTED]> writes: > ... crashes are more difficult. IMO the replication > manager needs to stay alive during this reinitialization, to keep the > GCS connection. However, it can easily detach from shared memory > temporarily (the imessages stuff is the only shmem place it

Re: [HACKERS] Postgres-R: internal messaging

2008-07-23 Thread Markus Wanner
Hi, what follows are some comments after trying to understand how the autovacuum launcher works and thoughts on how to apply this to the replication manager in Postgres-R. The initial comments in autovacuum.c say: If the fork() call fails in the postmaster, it sets a flag in the shared memo

Re: [HACKERS] Postgres-R: internal messaging

2008-07-23 Thread Markus Wanner
Hi, Tom Lane wrote: You should also look at the current code for communication between autovac launcher and autovac workers. That seems to be largely a similar problem, and it's been solved in a way that seems to be safe enough with respect to the postmaster vs shared memory issue. Oh yeah, t

Re: [HACKERS] Postgres-R: internal messaging

2008-07-23 Thread Tom Lane
Alexey Klyukin <[EMAIL PROTECTED]> writes: > Markus Wanner wrote: >> I'm currently doing this with imessages as well, >> which violates the rule that the postmaster may not to touch shared >> memory. I didn't look into ripping that out, yet. I'm not sure it can be >> done with the existing si

Re: [HACKERS] Postgres-R: internal messaging

2008-07-23 Thread Markus Wanner
Hi Alexey, thanks for your feedback, these are interesting points. Alexey Klyukin wrote: In Replicator we avoided the need for postmaster to read/write backend's shmem data by using it as a signal forwarder. When a backend wants to inform a special process (i.e. queue monitor) about replication

Re: [HACKERS] Postgres-R: internal messaging

2008-07-23 Thread Alexey Klyukin
Markus Wanner wrote: > Besides the communication between the replication manager and the > backends, which is currently done by using these imessages, the > replication manager also needs to communicate with the postmaster: it > needs to be able to request new helper backends and it wants to be > n

[HACKERS] Postgres-R: internal messaging

2008-07-23 Thread Markus Wanner
Hi, As you certainly know by now, Postgres-R introduces an additional manager process. That one is forked from the postmaster, so are all backends, no matter if they are processing local or remote transactions. That led to a communication problem, which has originally (i.e. around Postgres-R for