Re: [HACKERS] Ideas for easier debugging of backend problems
Added to TODO: * Add GUC variable to run a command on database panic or smart/fast/immediate shutdown --- Peter Eisentraut wrote: Martijn van Oosterhout wrote: 3. Add either a GUC or a command line switch or PGOPTION switch to automatically invoke and attach gdb on certain types of error. Obviously you can only do this where stdin, stdout and stderr have not been redirected. Samba has a configuration parameter that allows you to set an arbitrary command as a panic action script. This can then be used to gather debugging information or prepare a bug report (see other thread). This also has the added flexibility that binary packagers can add extra information specific to their environment. It may be worthwhile to research whether we could do something similar. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Ideas for easier debugging of backend problems
Martijn van Oosterhout wrote: 3. Add either a GUC or a command line switch or PGOPTION switch to automatically invoke and attach gdb on certain types of error. Obviously you can only do this where stdin, stdout and stderr have not been redirected. Samba has a configuration parameter that allows you to set an arbitrary command as a panic action script. This can then be used to gather debugging information or prepare a bug report (see other thread). This also has the added flexibility that binary packagers can add extra information specific to their environment. It may be worthwhile to research whether we could do something similar. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Ideas for easier debugging of backend problems
On Tue, Nov 01, 2005 at 12:33:39PM +0100, Peter Eisentraut wrote: Martijn van Oosterhout wrote: 3. Add either a GUC or a command line switch or PGOPTION switch to automatically invoke and attach gdb on certain types of error. Obviously you can only do this where stdin, stdout and stderr have not been redirected. Samba has a configuration parameter that allows you to set an arbitrary command as a panic action script. This can then be used to gather debugging information or prepare a bug report (see other thread). This also has the added flexibility that binary packagers can add extra information specific to their environment. It may be worthwhile to research whether we could do something similar. TODO? -- Jim C. Nasby, Sr. Engineering Consultant [EMAIL PROTECTED] Pervasive Software http://pervasive.comwork: 512-231-6117 vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461 ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Ideas for easier debugging of backend problems
What about the Google Core Dumper? :) http://sourceforge.net/projects/goog-coredumper/ Chris Peter Eisentraut wrote: Martijn van Oosterhout wrote: 3. Add either a GUC or a command line switch or PGOPTION switch to automatically invoke and attach gdb on certain types of error. Obviously you can only do this where stdin, stdout and stderr have not been redirected. Samba has a configuration parameter that allows you to set an arbitrary command as a panic action script. This can then be used to gather debugging information or prepare a bug report (see other thread). This also has the added flexibility that binary packagers can add extra information specific to their environment. It may be worthwhile to research whether we could do something similar. ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Ideas for easier debugging of backend problems
On Thu, Oct 27, 2005 at 11:44:24AM -0500, Jim C. Nasby wrote: The second option would help us where users are stymied by the system trying to change the core size ulimit, why not make it easier? It would also be very good if there was a way to verify that the backend should be able to generate a core, such as being able to see what ulimits the backend is running under. This would have saved me some pain snip Well, I've sent something to -patches that allows you to set an option so you get one of the following messages: NOTICE: Core dumps hard disabled by admin NOTICE: Core dumps already enabled by admin (size) NOTICE: Core limit successfully changed to (size) You use it like: $ PGOPTIONS=-C ./psql test NOTICE: Core limit successfully changed to (unlimited) Welcome to psql 8.1beta2, the PostgreSQL interactive terminal. snip I think a GUC would be a waste of space. It's not like you want to skip the first three segfaults and dump on the fourth. It shouldn't be a global option. It shouldn't be easy to enable, but the option should be there. This way doesn't require any changes to clients, as it can be controlled by the environment. Bloat, I don't know, maybe. I think the gain outweighs the costs, but I'll leave it to TPTB to decide that. Have a nice day, -- Martijn van Oosterhout kleptog@svana.org http://svana.org/kleptog/ Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a tool for doing 5% of the work and then sitting around waiting for someone else to do the other 95% so you can sue them. pgp4Lj9vuqLtQ.pgp Description: PGP signature
Re: [HACKERS] Ideas for easier debugging of backend problems
Martijn van Oosterhout kleptog@svana.org writes: 1. Move the test for strange memory alloc sizes to the palloc macros so that on error, it points at the palloc call rather than mcxt.c. What would that accomplish other than bloating the backend? We can't do it anyway, because of double-evaluation risk. 2. Add either a GUC or a command line switch or PGOPTION switch to call setrlimit to set the core size to something bigger. Most places only soft limit the core size, not hard limit. 3. Add either a GUC or a command line switch or PGOPTION switch to automatically invoke and attach gdb on certain types of error. Obviously you can only do this where stdin, stdout and stderr have not been redirected. Both of these presume you have a programmer running the database, or at least someone who's not scared of gdb. regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Ideas for easier debugging of backend problems
On Thu, Oct 27, 2005 at 10:41:08AM -0400, Tom Lane wrote: Martijn van Oosterhout kleptog@svana.org writes: 1. Move the test for strange memory alloc sizes to the palloc macros so that on error, it points at the palloc call rather than mcxt.c. What would that accomplish other than bloating the backend? We can't do it anyway, because of double-evaluation risk. The double-evaluation is avoidable (on GCC at least). I was thinking about when you --enable-cassert or something. But you're right, the test in palloc is always on, so you don't save much. 2. Add either a GUC or a command line switch or PGOPTION switch to call setrlimit to set the core size to something bigger. Most places only soft limit the core size, not hard limit. 3. Add either a GUC or a command line switch or PGOPTION switch to automatically invoke and attach gdb on certain types of error. Obviously you can only do this where stdin, stdout and stderr have not been redirected. Both of these presume you have a programmer running the database, or at least someone who's not scared of gdb. I've been thinking more along the lines that gdb is scriptable. We could invoke gdb in a way that immediatly dumps a backtrace, the local variables and some of the globals to a file and display a message to the user: please attach this to your bug report... Like today, having all this info upfront would save some time, because then we wouldn't have to teach people how to use gdb or what variables are important. Ofcourse, we'd really need to distinguish between run-of-the-mill errors and actual serious problems. The second option would help us where users are stymied by the system trying to change the core size ulimit, why not make it easier? Have a nice day, -- Martijn van Oosterhout kleptog@svana.org http://svana.org/kleptog/ Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a tool for doing 5% of the work and then sitting around waiting for someone else to do the other 95% so you can sue them. pgpPbCnmBVbfx.pgp Description: PGP signature
Re: [HACKERS] Ideas for easier debugging of backend problems
Tom Lane wrote: Both of these presume you have a programmer running the database, or at least someone who's not scared of gdb. I think you have the set relationship the wrong way around ;-) Personally, I only use gdb in extremis, and I am sure the average DBA would be less familiar with it than I am. cheers andrew ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Ideas for easier debugging of backend problems
On Thu, Oct 27, 2005 at 05:11:41PM +0200, Martijn van Oosterhout wrote: On Thu, Oct 27, 2005 at 10:41:08AM -0400, Tom Lane wrote: Martijn van Oosterhout kleptog@svana.org writes: 1. Move the test for strange memory alloc sizes to the palloc macros so that on error, it points at the palloc call rather than mcxt.c. What would that accomplish other than bloating the backend? We can't do it anyway, because of double-evaluation risk. The double-evaluation is avoidable (on GCC at least). I was thinking about when you --enable-cassert or something. But you're right, the test in palloc is always on, so you don't save much. 2. Add either a GUC or a command line switch or PGOPTION switch to call setrlimit to set the core size to something bigger. Most places only soft limit the core size, not hard limit. 3. Add either a GUC or a command line switch or PGOPTION switch to automatically invoke and attach gdb on certain types of error. Obviously you can only do this where stdin, stdout and stderr have not been redirected. Both of these presume you have a programmer running the database, or at least someone who's not scared of gdb. I've been thinking more along the lines that gdb is scriptable. We could invoke gdb in a way that immediatly dumps a backtrace, the local variables and some of the globals to a file and display a message to the user: please attach this to your bug report... Like today, having all this info upfront would save some time, because then we wouldn't have to teach people how to use gdb or what variables are important. Ofcourse, we'd really need to distinguish between run-of-the-mill errors and actual serious problems. Perhaps have a flag that people can turn on when they're having issues? Still get a good amount of bloat, but at least not all the time... The second option would help us where users are stymied by the system trying to change the core size ulimit, why not make it easier? It would also be very good if there was a way to verify that the backend should be able to generate a core, such as being able to see what ulimits the backend is running under. This would have saved me some pain yesterday. It would also be useful to be able to force the backend to dump core so you can see if it's actually working (granted, I know you can end up hitting the ulimit depending on how much memory is being consumed). Maybe there is a way to do this already and I just don't know it... -- Jim C. Nasby, Sr. Engineering Consultant [EMAIL PROTECTED] Pervasive Software http://pervasive.comwork: 512-231-6117 vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461 ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Ideas for easier debugging of backend problems
Jim C. Nasby [EMAIL PROTECTED] writes: It would also be useful to be able to force the backend to dump core so you can see if it's actually working kill -ABRT backend-PID regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Ideas for easier debugging of backend problems
Jim C. Nasby [EMAIL PROTECTED] writes: It would also be useful to be able to force the backend to dump core so you can see if it's actually working (granted, I know you can end up hitting the ulimit depending on how much memory is being consumed). Maybe there is a way to do this already and I just don't know it... SIGABRT? -Doug ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Ideas for easier debugging of backend problems
On Thu, Oct 27, 2005 at 01:13:37PM -0400, Douglas McNaught wrote: Jim C. Nasby [EMAIL PROTECTED] writes: It would also be useful to be able to force the backend to dump core so you can see if it's actually working (granted, I know you can end up hitting the ulimit depending on how much memory is being consumed). Maybe there is a way to do this already and I just don't know it... SIGABRT? Still doesn't do anything if core dumps are disabled. The point is to enable the user to enable core dumps from the frontend, rather than having to fiddle the backend environment directly... Have a nice day, -- Martijn van Oosterhout kleptog@svana.org http://svana.org/kleptog/ Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a tool for doing 5% of the work and then sitting around waiting for someone else to do the other 95% so you can sue them. pgp1sSpn88HlP.pgp Description: PGP signature
Re: [HACKERS] Ideas for easier debugging of backend problems
On Thu, Oct 27, 2005 at 07:20:57PM +0200, Martijn van Oosterhout wrote: On Thu, Oct 27, 2005 at 01:13:37PM -0400, Douglas McNaught wrote: Jim C. Nasby [EMAIL PROTECTED] writes: It would also be useful to be able to force the backend to dump core so you can see if it's actually working (granted, I know you can end up hitting the ulimit depending on how much memory is being consumed). Maybe there is a way to do this already and I just don't know it... SIGABRT? Still doesn't do anything if core dumps are disabled. The point is to enable the user to enable core dumps from the frontend, rather than having to fiddle the backend environment directly... Actually, SIGABRT is what I was looking for... I wanted a way to force a backend to dump core so I could see if a corefile would actually be generated. -- Jim C. Nasby, Sr. Engineering Consultant [EMAIL PROTECTED] Pervasive Software http://pervasive.comwork: 512-231-6117 vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461 ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
