Re: [HACKERS] stack depth limit exceeded problem.
Martijn van Oosterhout wrote: I rely on the signal handler that the JVM uses for page-faults (which a stack overflow generally amounts to) and fpe exeptions so I know that they will generate java exceptions in a controlled way (which I in turn translate to elog(ERROR) on the main thread). Well, actually, what I was thinking is if someone sends a -INT or -TERM to the backend, which thread will catch it? You have to block it in every thread except the one you want to catch it in if you want to control it. This means that for any signal handler that PostgreSQL installs, you need to intercept it with a wrapper function to make sure it runs in the right stack. Actually, while running backend code, you're probably fine since the elog stuff will handle it. But if a signal is received while the JVM is running, the signal handler will get the stack of the JVM. Now, PostgreSQLs signal handlers tend not to do much so you may be safe. They tend not to throws errors, but who knows... Still, this is all solvable I think... Yes, the signal handling in PL/Java needs a bit more work. Interrupts doesn't work well when using PL/Java at present. This is what I plan to do (I think this is what you mean too, right?). Many threads are spawned by the JVM and will never enter the backend. I can't control this and I can't add thread initialization code. Hence, I have no way of blocking signals on a per-thread basis the normal way. Instead, all PostgreSQL handlers that might break when called from another thread must be replaced by a wrapper that checks the interrupted thread. Since an arbitrary thread will receive the signal, the wrapper must sometimes dispatch the signal to another thread. The final receiver of the signal must be either the thread that currently executes a backend request, or if no such thread exists, the main thread. PL/Java will be limited to platforms that support that signals are dispatched to specific threads. I don't consider that a limitation. I think many JVM's have the same restriction. Regards, Thomas Hallgren ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] stack depth limit exceeded problem.
Oliver Jowett wrote: Thomas Hallgren wrote: PL/Java runs a JVM. Since a JVM is multi threaded, PL/Java goes to fairly extreme measures to ensure that only one thread at a time can access the backend. So far, this have worked well but there is one small problem. [...] I assume this means you have a single lock serializing requests to the backend? Yes, of course. I also make sure that the main thread cannot return until another thread that is servicing a backend request has completed. There's absolutely no way two threads can execute backend code simultaniously. If you can't solve the depth checking problem (Tom doesn't seem to like the idea of multiple threads calling into the backend..), what about turning the original thread (i.e. the main backend thread) into a backend interface thread that does nothing but feed callbacks into the backend on request? Then run all the user code in a separate thread that passes backend requests to the interface thread rather than directly executing them. If it starts extra threads which makes DB requests, the mechanism stays the same.. I though about that. The drawback is that each and every call must spawn a new thread, no matter how trivial that call might be. If you do a select from a table with 10,000 records and execute a function for each record, you get 20,000 context switches. Avoiding that kind of overhead is one of the motivating factors for keeping the VM in-process. I don't rule out such a solution but I'd like to have a discussion with Tom and iron out what the problems are when one thread at a time is allowed to execute. Perhaps I can solve them. Regards, Thomas Hallgren ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] stack depth limit exceeded problem.
On Sat, Sep 24, 2005 at 10:34:42AM +0200, Thomas Hallgren wrote: Oliver Jowett wrote: I assume this means you have a single lock serializing requests to the backend? Yes, of course. I also make sure that the main thread cannot return until another thread that is servicing a backend request has completed. There's absolutely no way two threads can execute backend code simultaniously. Ok, I have a question. PostgreSQL uses sigsetjmp/siglongjmp to handle errors in the backend. If you're changing the stack, how do you avoid the siglongjmp jumping back to a different stack? Or do you somehow avoid this problem altogether? I though about that. The drawback is that each and every call must spawn a new thread, no matter how trivial that call might be. If you do a select from a table with 10,000 records and execute a function for each record, you get 20,000 context switches. Avoiding that kind of overhead is one of the motivating factors for keeping the VM in-process. Well, on linux at least context switches are quite cheap. However, how does Java handle the possibility that functions never return. Do you wrap each call in a PG_TRY/PG_CATCH to propegate errors? Tricky issues... -- Martijn van Oosterhout kleptog@svana.org http://svana.org/kleptog/ Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a tool for doing 5% of the work and then sitting around waiting for someone else to do the other 95% so you can sue them. pgpWDeiFxvZex.pgp Description: PGP signature
Re: [HACKERS] stack depth limit exceeded problem.
Martijn van Oosterhout wrote: On Sat, Sep 24, 2005 at 10:34:42AM +0200, Thomas Hallgren wrote: Oliver Jowett wrote: I assume this means you have a single lock serializing requests to the backend? Yes, of course. I also make sure that the main thread cannot return until another thread that is servicing a backend request has completed. There's absolutely no way two threads can execute backend code simultaniously. Ok, I have a question. PostgreSQL uses sigsetjmp/siglongjmp to handle errors in the backend. If you're changing the stack, how do you avoid the siglongjmp jumping back to a different stack? Or do you somehow avoid this problem altogether? All calls use a PG_TRY/PG_CATCH. So yes, I think I avoid that problem altogether. I though about that. The drawback is that each and every call must spawn a new thread, no matter how trivial that call might be. If you do a select from a table with 10,000 records and execute a function for each record, you get 20,000 context switches. Avoiding that kind of overhead is one of the motivating factors for keeping the VM in-process. Well, on linux at least context switches are quite cheap. I know. And as I said, I don't rule out such a solution. But however cheap, there's still a performance penalty and added complexity. I rather avoid both if I can. At least until I know what the real problem is with the solution that I propose. However, how does Java handle the possibility that functions never return. Do you wrap each call in a PG_TRY/PG_CATCH to propegate errors? Yes. All backend exceptions are cought in a PG_CATCH and then propagated to Java as a ServerException. If there's no catch in the Java code, they are rethrown by the java_call_handler. This time with jump buffer that was setup by the backend when it invoked the call_handler. There's also a barrier that will prevent any further calls from the Java code once an exception has been thrown by the backend unless that call was wrapped in a savepoint construct. A savepoint rollback will unlock the barrier (this is not related to the thread issue of course). Regards, Thomas Hallgren ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] stack depth limit exceeded problem.
On Sat, Sep 24, 2005 at 12:26:58PM +0200, Thomas Hallgren wrote: Yes. All backend exceptions are cought in a PG_CATCH and then propagated to Java as a ServerException. If there's no catch in the Java code, they are rethrown by the java_call_handler. This time with jump buffer that was setup by the backend when it invoked the call_handler. There's also a barrier that will prevent any further calls from the Java code once an exception has been thrown by the backend unless that call was wrapped in a savepoint construct. A savepoint rollback will unlock the barrier (this is not related to the thread issue of course). Well, you seem to have dealt with the obvious issues I can see. I imagine you need also to worry about things like signal handling. Is there no way to reserve a stack just for PostgreSQL and switch to that stack, rather than switch threads (although, the stack is really the only thing that differentiates threads anyway...). Linux has sigaltstack so you can catch the stack overflow signal (and other signals obviously, but that's its main use), but it's not terribly portable. What you really need to do is set the stack_base_ptr every time you execute postgres with a new stack; that preserves existing semantics. Signals are the only way the kernel can pass control unexpectedly so if you handle those, postgres would never know it's threaded. I do wonder if there are any other assumptions made... Have a nice day, -- Martijn van Oosterhout kleptog@svana.org http://svana.org/kleptog/ Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a tool for doing 5% of the work and then sitting around waiting for someone else to do the other 95% so you can sue them. pgpAoT6QNSsBT.pgp Description: PGP signature
Re: [HACKERS] stack depth limit exceeded problem.
Martijn van Oosterhout wrote:
Linux has sigaltstack so you can catch the stack overflow signal (and
other signals obviously, but that's its main use), but it's not terribly
portable.
I rely on the signal handler that the JVM uses for page-faults (which a
stack overflow generally amounts to) and fpe exeptions so I know that
they will generate java exceptions in a controlled way (which I in turn
translate to elog(ERROR) on the main thread).
What you really need to do is set the stack_base_ptr every
time you execute postgres with a new stack; that preserves existing
semantics.
Exactly!. What I'd really like to do in threads other than main is:
void* currentBase = switchStackBase(stackBaseOfMyThread);
PG_TRY
{
/* service the call here */
switchStackBase(currentBase);
}
PG_CATCH
{
switchStackBase(currentBase);
/* generate Java exception as usual */
}
Signals are the only way the kernel can pass control unexpectedly so if
you handle those, postgres would never know it's threaded. I do wonder
if there are any other assumptions made...
Have a nice day,
You too. And thanks for all your input.
Regards,
Thomas Hallgren
---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] stack depth limit exceeded problem.
On Sat, Sep 24, 2005 at 02:38:35PM +0200, Thomas Hallgren wrote: Martijn van Oosterhout wrote: Linux has sigaltstack so you can catch the stack overflow signal (and other signals obviously, but that's its main use), but it's not terribly portable. I rely on the signal handler that the JVM uses for page-faults (which a stack overflow generally amounts to) and fpe exeptions so I know that they will generate java exceptions in a controlled way (which I in turn translate to elog(ERROR) on the main thread). Well, actually, what I was thinking is if someone sends a -INT or -TERM to the backend, which thread will catch it? You have to block it in every thread except the one you want to catch it in if you want to control it. This means that for any signal handler that PostgreSQL installs, you need to intercept it with a wrapper function to make sure it runs in the right stack. Actually, while running backend code, you're probably fine since the elog stuff will handle it. But if a signal is received while the JVM is running, the signal handler will get the stack of the JVM. Now, PostgreSQLs signal handlers tend not to do much so you may be safe. They tend not to throws errors, but who knows... Still, this is all solvable I think... -- Martijn van Oosterhout kleptog@svana.org http://svana.org/kleptog/ Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a tool for doing 5% of the work and then sitting around waiting for someone else to do the other 95% so you can sue them. pgpLeqyX5gIeM.pgp Description: PGP signature
[HACKERS] stack depth limit exceeded problem.
Hi, I have a problem with PL/Java that, if it's going to have a good solution, requires your help. PL/Java runs a JVM. Since a JVM is multi threaded, PL/Java goes to fairly extreme measures to ensure that only one thread at a time can access the backend. So far, this have worked well but there is one small problem. Here's a use-case: Someone loads a library that contains a method that spawns a new thread. That thread is the first to access some class. The class loader will now make an attempt to load it. PL/Java uses SPI to load classes so a call is made to SPI. This call is not made from the main thread that originally called the PL/Java function. That thread is suspended at this point. Now, the check_stack_depth() in postgres.c is called. The new thread has a stack of it's own of course, so it fails. I know that multi threading is very controversial and I'm in no way asking that the backend should support it. What I would like is a workaround for my problem. The easiest way would be if I could change the stack_base_ptr temporarily when this happens, a try/catch that kicks in when I detect a call from a thread other then main. The only other solution is to set the max_stack_depth to a ridiculously high value and effectively turn stack checking off. I don't want to do that. Any opinions on this? Kind regards, Thomas Hallgren ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] stack depth limit exceeded problem.
Thomas Hallgren [EMAIL PROTECTED] writes: Someone loads a library that contains a method that spawns a new thread. They already broke the backend when they did that. max_stack_depth is just the tip of the iceberg. regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] stack depth limit exceeded problem.
Tom Lane wrote: Thomas Hallgren [EMAIL PROTECTED] writes: Someone loads a library that contains a method that spawns a new thread. They already broke the backend when they did that. max_stack_depth is just the tip of the iceberg. I knew I'd get a response like that from you :-) Why is the backend broken? There's no concurrency issue. Only one thread is executing. Regards, Thomas Hallgren ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] stack depth limit exceeded problem.
Thomas Hallgren wrote: PL/Java runs a JVM. Since a JVM is multi threaded, PL/Java goes to fairly extreme measures to ensure that only one thread at a time can access the backend. So far, this have worked well but there is one small problem. [...] I assume this means you have a single lock serializing requests to the backend? If you can't solve the depth checking problem (Tom doesn't seem to like the idea of multiple threads calling into the backend..), what about turning the original thread (i.e. the main backend thread) into a backend interface thread that does nothing but feed callbacks into the backend on request? Then run all the user code in a separate thread that passes backend requests to the interface thread rather than directly executing them. If it starts extra threads which makes DB requests, the mechanism stays the same.. -O ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
