[PATCHES] fork/exec patch: CreateProcess calls for Win32
For application to HEAD, pending community review. Drops in the CreateProcess calls for Win32 (essentially wrapping up the fork/exec portion of the port), and fixes a handful of whitespace issues (that no doubt I'm to blame for; my apologies) in the source base. Cheers, Claudio --- Certain disclaimers and policies apply to all email sent from Memetrics. For the full text of these disclaimers and policies see a href=http://www.memetrics.com/emailpolicy.html;http://www.memetrics.com/em ailpolicy.html/a diff6c.out Description: Binary data ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [PATCHES] SIGPIPE handling
Manfred Spraul wrote: Bruce Momjian wrote: + /* +* We could lose a signal during this test. +* In a multi-threaded application, this might +* be a problem. Do any non-threaded platforms Threaded or non-threaded? OK, yea, I will use threaded. +* lack sigaction()? +*/ Additionally, the problem is not restricted to multithreaded apps: signal(,SIG_IGN) clears all pending signals. Oh, yuck. Would SIG_DFL be better here? I am thinking of adding sigblock into that code on the assumption that if they have signal(), they have sigblock(). Should we disable threaded builds unless they have sigaction()? I suppose the sigblock() would take care of the pending signal problem too. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [PATCHES] SIGPIPE handling
Manfred Spraul wrote:
Bruce Momjian wrote:
+ /*
+* We could lose a signal during this test.
+* In a multi-threaded application, this might
+* be a problem. Do any non-threaded platforms
Threaded or non-threaded?
+* lack sigaction()?
+*/
Additionally, the problem is not restricted to multithreaded apps:
signal(,SIG_IGN) clears all pending signals.
OK, new function using sigblock():
pqsigfunc
pqsignalinquire(int signo)
{
#if !defined(HAVE_POSIX_SIGNALS)
pqsigfunc old_sigfunc;
int old_sigmask;
/* Prevent signal handler calls during test */
old_sigmask = sigblock(sigmask(signo));
old_sigfunc = signal(signo, SIG_DFL);
signal(signo, old_sigfunc);
sigblock(old_sigmask);
return old_sigfunc;
#else
struct sigaction oact;
if (sigaction(signo, NULL, oact) 0)
return SIG_ERR;
return oact.sa_handler;
#endif /* !HAVE_POSIX_SIGNALS */
}
--
Bruce Momjian| http://candle.pha.pa.us
[EMAIL PROTECTED] | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup.| Newtown Square, Pennsylvania 19073
---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match
Re: [PATCHES] psql \i handling ~ in specified file name
Zach Irmen wrote: Andrew Dunstan [EMAIL PROTECTED] writes: Zach Irmen said: Can I just ifndef WIN32 and not think about it? I'm not sure how that would work either. If we are going to have a Windows port I don't think we should treat it as a poor cousin. I guess I was thinking more about if it should be done as opposed to how it would be done. On the one hand, I think '~' by itself has no meaning in a normal Windows environment, so why should psql on Windows give it one? The readline library on unix, which can be used by psql, interprets the tilde and is the big reason why psql on unix should interpret the tilde as well. On the other hand however, I can see consistency being important in that giving '~' a meaning in psql should give it the same meaning regardless of platform. As I remember, MSDOS uses the ~ to specify short versions of long file names. I think that is enough for us to say that we are best leaving '~' expansion only for Unix. We already dump COPY in a native Win32 format, so it seems we should handle special characters in a similar native way. I will add a comment to this affect in the source code. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PATCHES] psql \i handling ~ in specified file name
Joshua D. Drake wrote: As I remember, MSDOS uses the ~ to specify short versions of long file names. I is not just msdos, it is cmd.exe which exists on (to my knowledge) all versions of windows. For example: Program Files == progra~1 Yes, I meant it is a hold-over from dealing with MS-DOS style 8.3 file names. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [PATCHES] psql \i handling ~ in specified file name
As I remember, MSDOS uses the "~" to specify short versions of long file names. I is not just msdos, it is cmd.exe which exists on (to my knowledge) all versions of windows. For example: Program Files == progra~1 I think that is enough for us to say that we are best leaving '~' expansion only for Unix. We already dump COPY in a native Win32 format, so it seems we should handle special characters in a similar native way. I will add a comment to this affect in the source code. -- Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC Postgresql support, programming shared hosting and dedicated hosting. +1-503-222-2783 - [EMAIL PROTECTED] - http://www.commandprompt.com
[PATCHES] Win32 signal code - first try
Hi! Here is a first sketch at Win32 signal handling. First a couple of comments: * This is just two files. It is not integrated with postgresql yet. * Uses named pipes. Shared mem was slightly faster, named pipes a lot cleaner. And the signal handlers themselves should not be performance critical, AFAICS. * Does *not* use user APCs. Why? Well, we had to use polling. And with user APCs we'd have to go into kernel mode (however briefly) on every poll. I replaced that with a simple counter that is checked. Thast way we don't slow down the main path of the program much. * It is then checked again while protected with a critical section, to make sure we don't setp on our own toes from two threads. * A user APC is queued on every signal anyway. This is only to break out of SleepEx/WaitForMultipleObjectsEx etc functions. Normally, we pick the signals up with polling. A couple of questions that need answering: * Where to put the polling. We need to find a reasonable limited number of places. They just need PG_POLL_SIGNALS() there to go. Also, do we want this one #ifdef:ed in every place, or just #define it to nothing at all on non-windows platforms? * Where to put the functions. Right now, it's just a separate file. Do we want to keep it in separate files, or put it in existing files with #ifdef:s? Or combination? * Function names - I haven't looked at naming conventions at all :-) * We need to replace any blocking waits (such as select() and sleep()) with Ex functions if we want signals to fire while waiting. Probably a bunch more things I forgot to write right now. Oh, and I've included two very small testfiles - pgsignal_srvtest.c which is a sample server and pgkill.c which is a sample client. Very ugly code, but it should show how it's intended to be used :-) Anyway. Comments on these points in particular, and in the whole thing in general? Workable path? //Magnus pgsignal_w32.h Description: pgsignal_w32.h pgsignal_w32.c Description: pgsignal_w32.c pgsignal_srvtest.c Description: pgsignal_srvtest.c pgkill.c Description: pgkill.c ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PATCHES] fork/exec patch: pre-CreateProcess finalization
Claudio Natoli wrote: Something after 2003/11/20 enhanced the query cancel handling. Namely, CVS tip now responds to a query cancel with a postmaster restart canceling all queries. Could the fork/exec stuff be responsible for this? Jan Hi Bruce, Claudio, where are we on this patch? I see an even newer version in the archives: http://archives.postgresql.org/pgsql-patches/2003-12/msg00361.php [Weird you and Google groups missed it!] Anyway, Tom has looked at your newest patch and it looks good to him. I'm happy with the patch from the link above being committed if Tom has no more comments. Was only waiting for a final nod from him. Once it is in the source tree, give me a couple days and I'll fire off a patch with the actual CreateProcess calls... and then we are off into Win32 signal land [shudder]. Cheers, Claudio --- Certain disclaimers and policies apply to all email sent from Memetrics. For the full text of these disclaimers and policies see a href=http://www.memetrics.com/emailpolicy.html;http://www.memetrics.com/em ailpolicy.html/a ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] # ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [PATCHES] psql \i handling ~ in specified file name
Andrew Dunstan wrote: As I remember, MSDOS uses the ~ to specify short versions of long file names. I is not just msdos, it is cmd.exe which exists on (to my knowledge) all versions of windows. For example: Program Files == progra~1 Yes, I meant it is a hold-over from dealing with MS-DOS style 8.3 file names. This pattern should not cause tilde expansion on any platform, I believe. Only a *leading* ~/ or ~username/ should be expanded. The normal home directory on a (modern) Windows machine is usually C:\Documents and Settings\username, IIRC. Yes, I understand, but on an OS that uses tilde so much inside the file name, do we want special meaning when it is leading the file name? -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [PATCHES] Patch of intarray module in v7.4.1
Korea PostgreSQL Users' Group [EMAIL PROTECTED] writes:
In 7.4.1, intarray module have a problme about equal operator (=).
select * from table where intarray_column = '{}';
above query make error.
Good catch. Patch applied --- thanks!
regards, tom lane
---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match
Re: [PATCHES] [pgsql-hackers-win32] Win32 signal code - first try
Also, do we want this one #ifdef:ed in every place, or just #define it to nothing at all on non-windows platforms? I imagine the latter will be cleaner. Anyway. Comments on these points in particular, and in the whole thing in general? Workable path? I think it looks great. Well done! Cheers, Claudio --- Certain disclaimers and policies apply to all email sent from Memetrics. For the full text of these disclaimers and policies see a href=http://www.memetrics.com/emailpolicy.html;http://www.memetrics.com/em ailpolicy.html/a ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] fork/exec patch: pre-CreateProcess finalization
Jan Wieck [EMAIL PROTECTED] writes:
Something after 2003/11/20 enhanced the query cancel handling. Namely,
CVS tip now responds to a query cancel with a postmaster restart
canceling all queries. Could the fork/exec stuff be responsible for this?
Whoever changed this:
status = ProcessStartupPacket(port, false);
if (status != STATUS_OK)
return 0;/* cancel request processed, or error */
to this:
status = ProcessStartupPacket(port, false);
if (status != STATUS_OK)
{
ereport(LOG,
(errmsg(connection startup failed)));
proc_exit(status);
}
is responsible. May we have an explanation of the thought process,
if any?
regards, tom lane
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly
Re: [PATCHES] fork/exec patch: pre-CreateProcess finalization
Tom Lane wrote:
Jan Wieck [EMAIL PROTECTED] writes:
Something after 2003/11/20 enhanced the query cancel handling. Namely,
CVS tip now responds to a query cancel with a postmaster restart
canceling all queries. Could the fork/exec stuff be responsible for this?
Whoever changed this:
status = ProcessStartupPacket(port, false);
if (status != STATUS_OK)
return 0;/* cancel request processed, or error */
to this:
status = ProcessStartupPacket(port, false);
if (status != STATUS_OK)
{
ereport(LOG,
(errmsg(connection startup failed)));
proc_exit(status);
}
is responsible. May we have an explanation of the thought process,
if any?
My guess is that this used to proc_exit the postgres backend, but now
proc_exits the postmaster, but that is only a guess.
--
Bruce Momjian| http://candle.pha.pa.us
[EMAIL PROTECTED] | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup.| Newtown Square, Pennsylvania 19073
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster
Re: [PATCHES] fork/exec patch: pre-CreateProcess finalization
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: My guess is that this used to proc_exit the postgres backend, but now proc_exits the postmaster, but that is only a guess. No. This is post-fork (and had better remain so). The change causes the sub-postmaster that has just finished handling a cancel request to exit with nonzero status, which the postmaster then interprets (correctly) as a child process crash. BTW, how are we going to do cancels in Windows-land? The sub-postmaster isn't gonna have access to the postmaster's list of child PIDs and cancel keys ... I think the postmaster will have access to the cancel key and pid. It should work similar to Unix. However, I do have a win32 TODO item on test cancel key, so we will not forget about it. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [PATCHES] fork/exec patch: pre-CreateProcess finalization
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: My guess is that this used to proc_exit the postgres backend, but now proc_exits the postmaster, but that is only a guess. No. This is post-fork (and had better remain so). The change causes the sub-postmaster that has just finished handling a cancel request to exit with nonzero status, which the postmaster then interprets (correctly) as a child process crash. BTW, how are we going to do cancels in Windows-land? The sub-postmaster isn't gonna have access to the postmaster's list of child PIDs and cancel keys ... When you say sub-postmaster, you mean we fork, then process the cancel request? Seems we might need special handling in there, yea. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PATCHES] fork/exec patch: pre-CreateProcess finalization
Whoever changed this: My fault. Thanks for the catch Jan. proc_exit(status); Should be proc_exit(0). My apologies, Claudio PS. I take it there is no regression test for this. Would this be a difficult one to knock up? --- Certain disclaimers and policies apply to all email sent from Memetrics. For the full text of these disclaimers and policies see a href=http://www.memetrics.com/emailpolicy.html;http://www.memetrics.com/em ailpolicy.html/a ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] fork/exec patch: pre-CreateProcess finalization
Bruce Momjian [EMAIL PROTECTED] writes: When you say sub-postmaster, you mean we fork, then process the cancel request? Seems we might need special handling in there, yea. We fork before we even read the request. Otherwise an attacker can trivially lock up the postmaster by sending a partial startup packet. I've just noticed another serious bit of breakage in CVS tip (though this might be fixed by Claudio's pending patch that reverts a lot of the code rearrangements). There is a reason why the 7.4 code does on_exit_reset *immediately* after fork(), and it's not acceptable to rearrange the code so that that happens at some random later time. Otherwise, any problem in between leads to the child process executing the postmaster's on_exit routines, with such pleasant side effects as destroying the shmem segment. For similar reasons, you don't get to postpone setting IsUnderPostmaster true --- elog pays attention to the value of that flag, and will do the wrong thing if called in a child process that doesn't yet have it set. regards, tom lane ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [PATCHES] fork/exec patch: pre-CreateProcess finalization
Tom Lane wrote: I've just noticed another serious bit of breakage in CVS tip (though this might be fixed by Claudio's pending patch that reverts a lot of the code rearrangements). It's already been applied. Your push in the SubPostmaster direction was really useful; a lot cleaner, and fixed things that I wasn't even familiar enough with the code to know were broken. Cheers, Claudio --- Certain disclaimers and policies apply to all email sent from Memetrics. For the full text of these disclaimers and policies see a href=http://www.memetrics.com/emailpolicy.html;http://www.memetrics.com/em ailpolicy.html/a ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] fork/exec patch: pre-CreateProcess finalization
Claudio Natoli wrote: The only things I've thought of so far are: a) sticking the PID/cancel key list in shared mem [yeech] b) rearranging the entire cancel handling to occur in the postmaster [double yeech] (a) seems like the lesser of the available evils (unless someone has a bright idea for a plan C). Bruce Momjian [EMAIL PROTECTED] writes: I think we need to move in the direction of a separate fork/exec-only shared memory segment that holds the pids and cancel keys for all the backends. That doesn't seem worth the trouble. I'd be inclined to just stick the cancel keys in the PGPROC structures (I believe the PIDs are already in there). The original motivation for keeping them only in postmaster local memory was to keep backend A's cancel key away from the prying eyes of backend B, but is there really any security added? Anyone who can inspect PGPROC hardly needs to know the cancel key --- he can just issue a SIGINT (or worse) directly to the target backend. regards, tom lane ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] fork/exec patch: pre-CreateProcess finalization
Tom Lane wrote: Claudio Natoli wrote: The only things I've thought of so far are: a) sticking the PID/cancel key list in shared mem [yeech] b) rearranging the entire cancel handling to occur in the postmaster [double yeech] (a) seems like the lesser of the available evils (unless someone has a bright idea for a plan C). Bruce Momjian [EMAIL PROTECTED] writes: I think we need to move in the direction of a separate fork/exec-only shared memory segment that holds the pids and cancel keys for all the backends. That doesn't seem worth the trouble. I'd be inclined to just stick the cancel keys in the PGPROC structures (I believe the PIDs are already in there). The original motivation for keeping them only in postmaster local memory was to keep backend A's cancel key away from the prying eyes of backend B, but is there really any security added? Anyone who can inspect PGPROC hardly needs to know the cancel key --- he can just issue a SIGINT (or worse) directly to the target backend. Agreed. I was going for a separate one just to be paranoid. This will only be done for exec(), so I don't see a problem for normal Unix use anyway. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] fork/exec patch: pre-CreateProcess finalization
Bruce Momjian wrote: Tom Lane wrote: Claudio Natoli wrote: The only things I've thought of so far are: a) sticking the PID/cancel key list in shared mem [yeech] b) rearranging the entire cancel handling to occur in the postmaster [double yeech] (a) seems like the lesser of the available evils (unless someone has a bright idea for a plan C). Bruce Momjian [EMAIL PROTECTED] writes: I think we need to move in the direction of a separate fork/exec-only shared memory segment that holds the pids and cancel keys for all the backends. That doesn't seem worth the trouble. I'd be inclined to just stick the cancel keys in the PGPROC structures (I believe the PIDs are already in there). The original motivation for keeping them only in postmaster local memory was to keep backend A's cancel key away from the prying eyes of backend B, but is there really any security added? Anyone who can inspect PGPROC hardly needs to know the cancel key --- he can just issue a SIGINT (or worse) directly to the target backend. Agreed. I was going for a separate one just to be paranoid. This will only be done for exec(), so I don't see a problem for normal Unix use anyway. It doesn't hurt to keep the locations and code as much in sync as possible. I think Tom's idea to move the information into the PGPROC entry is the winner and does not need any OS specific handling. Jan -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] # ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PATCHES] fork/exec patch: pre-CreateProcess finalization
Jan Wieck [EMAIL PROTECTED] writes: It doesn't hurt to keep the locations and code as much in sync as possible. I think Tom's idea to move the information into the PGPROC entry is the winner and does not need any OS specific handling. Actually, on further reflection a separate array to store PIDs and cancel keys is probably a better idea. If we put this stuff in PGPROC then the postmaster will need to be able to obtain the ProcStructLock (or whatever it's called this week) in order to examine/modify that data structure. That gets us into the usual concerns about backend bugs locking up the postmaster, etc. But if it's a separate array then we can just have the rule that no one but the postmaster gets to write in it. I still think it's unnecessary to make a separate shmem segment for it, though. regards, tom lane ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [PATCHES] fork/exec patch: pre-CreateProcess finalization
Tom Lane writes: Actually, on further reflection a separate array to store PIDs and cancel keys is probably a better idea. [snip] I still think it's unnecessary to make a separate shmem segment for it, though. Why is that? Don't we need the backends to have access to it to do a cancel request? I think I've missed something here... Claudio --- Certain disclaimers and policies apply to all email sent from Memetrics. For the full text of these disclaimers and policies see a href=http://www.memetrics.com/emailpolicy.html;http://www.memetrics.com/em ailpolicy.html/a ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] fork/exec patch: pre-CreateProcess finalization
Claudio Natoli wrote: Tom Lane writes: Actually, on further reflection a separate array to store PIDs and cancel keys is probably a better idea. [snip] I still think it's unnecessary to make a separate shmem segment for it, though. Why is that? Don't we need the backends to have access to it to do a cancel request? I think I've missed something here... I think they are saying put the cancel key inside the existing shared memory segment. I don't know when we actually attach to the main shared memory sigment in the child, but it would have to be sooner than when we need the cancel key. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [PATCHES] fork/exec patch: pre-CreateProcess finalization
I think they are saying put the cancel key inside the existing shared memory segment. Ok. Thanks. I don't know when we actually attach to the main shared memory sigment in the child, but it would have to be sooner than when we need the cancel key. Yes. Currently, it happens too late. I'll put my hand up to have a go at this fixing this (and getting processCancelRequest to work under Win32/EXEC_BACKEND at the same time), if no one else particularly cares to. Just to be clear, this would involve turning the BackendList dlllist into an array in shared memory, right? If so, a couple of questions: - what is a suitably large size for this array (2 * MaxBackends, ala canAcceptConnections?) - the postmaster makes all calls referencing this list, with the exception of processCancelRequest, correct? So, as Tom was alluding to, no locking is required (or desired!), and we'll just need to be careful not to introduce a race condition into processCancelRequest. Cheers, Claudio --- Certain disclaimers and policies apply to all email sent from Memetrics. For the full text of these disclaimers and policies see a href=http://www.memetrics.com/emailpolicy.html;http://www.memetrics.com/em ailpolicy.html/a ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [PATCHES] fork/exec patch: pre-CreateProcess finalization
Bruce Momjian wrote: Claudio Natoli wrote: Tom Lane writes: Actually, on further reflection a separate array to store PIDs and cancel keys is probably a better idea. [snip] I still think it's unnecessary to make a separate shmem segment for it, though. Why is that? Don't we need the backends to have access to it to do a cancel request? I think I've missed something here... I think they are saying put the cancel key inside the existing shared memory segment. I don't know when we actually attach to the main shared memory sigment in the child, but it would have to be sooner than when we need the cancel key. I said move it into the PGPROC structure. And keep the pid in both, the PGPROC structure and postmaster local memory. The backend attaches to the shared memory during AttachSharedMemoryAndSemaphores() ... where else? Jan -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] # ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] fork/exec patch: pre-CreateProcess finalization
Claudio Natoli wrote:
I think they are saying put the cancel key inside the existing shared
memory segment.
Ok. Thanks.
I don't know when we actually attach to the main shared
memory sigment in the child, but it would have to be sooner than when we
need the cancel key.
Yes. Currently, it happens too late.
I'll put my hand up to have a go at this fixing this (and getting
processCancelRequest to work under Win32/EXEC_BACKEND at the same time), if
no one else particularly cares to.
Just to be clear, this would involve turning the BackendList dlllist into an
array in shared memory, right? If so, a couple of questions:
- what is a suitably large size for this array (2 * MaxBackends, ala
canAcceptConnections?)
- the postmaster makes all calls referencing this list, with the exception
of processCancelRequest, correct? So, as Tom was alluding to, no locking is
required (or desired!), and we'll just need to be careful not to introduce a
race condition into processCancelRequest.
I assumed a much simpler solution. I thought we would just have:
struct {
pid_t pid;
int cancel_key;
} PidCancel[maxbackend];
in shared memory and we would just sequentially scan looking for a pid
match? Is that wrong?
--
Bruce Momjian| http://candle.pha.pa.us
[EMAIL PROTECTED] | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup.| Newtown Square, Pennsylvania 19073
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster
Re: [PATCHES] fork/exec patch: pre-CreateProcess finalization
Claudio Natoli [EMAIL PROTECTED] writes: Just to be clear, this would involve turning the BackendList dlllist into an array in shared memory, right? If so, a couple of questions: Per Jan's comment, there is no need to mess with the existing datastructure. I'd think more of *copying* the dllist into some array in shared memory. This is code that would only need to exist in the Windows port. - the postmaster makes all calls referencing this list, with the exception of processCancelRequest, correct? The postmaster writes the array (and really would have no need to read it). Sub-postmasters would need to read the array, either to check an incoming cancel request or to get the current-session key value to pass back to the client. I don't think that PostgresMain or any subsidiary routine would ever need to touch it. regards, tom lane ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] fork/exec patch: pre-CreateProcess finalization
Bruce Momjian [EMAIL PROTECTED] writes: Uh, why would you need more than maxbackends? Can't it be indexed by slot number --- each backend has a slot? Maybe I am missing something. The postmaster has no way to know what slot number each backend will get. For that matter, a sub-postmaster doesn't know yet either. I think the simplest way to make this work is to use an array that's 2*MaxBackend items long (corresponding to the max number of children the postmaster will fork). Establish the convention that unused entries are zero. Then: 1. On forking a child, the postmaster scans the array for a free (zero) slot, and stashes the cancel key and PID there (in that order). 2. On receiving a child-termination report, the postmaster scans the array for the corresponding entry, and zeroes it out (PID first). (Obviously these algorithms could be improved if they turn out to be bottlenecks, but for the first cut KISS is applicable.) 3. To find or check a cencel key, a sub-postmaster scans the array looking for the desired PID (either its own, or the one it got from an incoming cancel request message). There is a potential race condition if a sub-postmaster scans the array before the postmaster has been able to store its PID there. I think it is sufficient for the sub-postmaster to sleep a few milliseconds and try again if it can't find its own PID in the array. There is no race condition possible for the ProcessCancelRequest case --- the sub-postmaster that spawned an active backend must have found its entry before it could have sent the cancel key to the client, so any valid cancel request from a client must reference an already-existing entry in the array. regards, tom lane ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] fork/exec patch: pre-CreateProcess finalization
Tom Lane: Per Jan's comment, there is no need to mess with the existing datastructure. I'd think more of *copying* the dllist into some array in shared memory. This is code that would only need to exist in the Windows port. (I thought Jan was referring to the PGPROC struct) This just seems a little odd to me. I mean, they are going to be basically identical (they'll even use the same struct!). Also, let's get back to why we want this: to handle processCancelRequest in the Win32 case. If this array is in Windows only, then we'll obviously need two implementations of the processCancelRequest logic. Or I'm missing something... Cheers, Claudio --- Certain disclaimers and policies apply to all email sent from Memetrics. For the full text of these disclaimers and policies see a href=http://www.memetrics.com/emailpolicy.html;http://www.memetrics.com/em ailpolicy.html/a ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [PATCHES] fork/exec patch: pre-CreateProcess finalization
Tom Lane wrote: I think the simplest way to make this work is to use an array that's 2*MaxBackend items long (corresponding to the max number of children the postmaster will fork). Actually, now that I think about it, is even that big enough. There is a reason BackendList is a list. In pathological situations, the postmaster could be made to fork a much larger number than 2*MaxBackend simultaneous children, although only this many will be allowed to become backends. (I guess we could check the port-canAcceptConnections value, and not add the backend to the array when == CAC_TOOMANY). There is no race condition possible for the ProcessCancelRequest case --- the sub-postmaster that spawned an active backend must have found its entry before it could have sent the cancel key to the client, so any valid cancel request from a client must reference an already-existing entry in the array. Make sense. Great. Cheers, Claudio --- Certain disclaimers and policies apply to all email sent from Memetrics. For the full text of these disclaimers and policies see a href=http://www.memetrics.com/emailpolicy.html;http://www.memetrics.com/em ailpolicy.html/a ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] fork/exec patch: pre-CreateProcess finalization
Claudio Natoli wrote: Tom Lane: Per Jan's comment, there is no need to mess with the existing datastructure. I'd think more of *copying* the dllist into some array in shared memory. This is code that would only need to exist in the Windows port. (I thought Jan was referring to the PGPROC struct) This just seems a little odd to me. I mean, they are going to be basically identical (they'll even use the same struct!). Seems you should use an array of struct bkend or Backend as storage in shared memory. I think the problem with Dllist is that it is dynamically allocated and I don't see how that could be done cleanly in shared memory. I think that's why the array idea seems best. Also, let's get back to why we want this: to handle processCancelRequest in the Win32 case. If this array is in Windows only, then we'll obviously need two implementations of the processCancelRequest logic. Or I'm missing something... Looking up they key would have to be different for fork and fork/exec. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] fork/exec patch: pre-CreateProcess finalization
Claudio Natoli [EMAIL PROTECTED] writes: Tom Lane wrote: I think the simplest way to make this work is to use an array that's 2*MaxBackend items long (corresponding to the max number of children the postmaster will fork). Actually, now that I think about it, is even that big enough. There is a reason BackendList is a list. In pathological situations, the postmaster could be made to fork a much larger number than 2*MaxBackend simultaneous children, although only this many will be allowed to become backends. (I guess we could check the port-canAcceptConnections value, and not add the backend to the array when == CAC_TOOMANY). Probably we could rearrange the logic to test for too-many-children before we even do the fork. It'd be a little uglier that way, but not out of the question. regards, tom lane ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
