Re: [HACKERS] thread safety on clients

2009-12-14 Thread Alvaro Herrera
Tom Lane wrote: > But my recollection of the parallel psql patch discussion is that it was > rejected because nobody felt comfortable with the API design. Do we > have any better ideas in that department yet? It wasn't rejected AFAICT. A finalized API with which there was (almost?) no dissent w

Re: [HACKERS] thread safety on clients

2009-12-14 Thread Tom Lane
Greg Stark writes: > On Fri, Dec 11, 2009 at 6:08 PM, Tom Lane wrote: >> I'll commit a fix for that shortly, but this reminds me once again that >> the EvalPlanQual logic is desperately under-tested in our normal >> regression testing.  We really need some kind of testing infrastructure >> that w

Re: [HACKERS] thread safety on clients

2009-12-13 Thread Greg Stark
On Fri, Dec 11, 2009 at 6:08 PM, Tom Lane wrote: > I wrote: > I'll commit a fix for that shortly, but this reminds me once again that > the EvalPlanQual logic is desperately under-tested in our normal > regression testing.  We really need some kind of testing infrastructure > that would let us exe

Re: [HACKERS] thread safety on clients

2009-12-11 Thread Tom Lane
I wrote: > ... What I would have expected is crashes on the very > similar updates to pgbench_branches, which is designed to be > high-contention. It might be that there is some other effect going on > here that explains why that wasn't happening. Need to go back and look > more closely. ... and

Re: [HACKERS] thread safety on clients

2009-12-11 Thread Tom Lane
Greg Smith writes: > It sounds like random pgbench run for a while would certainly expose the > same thing you're concerned about eventually. Yeah. Actually the odd thing about it is that the crash seemed to invariably be on conflicting pgbench_accounts updates, which is a fairly low-contention

Re: [HACKERS] thread safety on clients

2009-12-11 Thread Greg Smith
Tom Lane wrote: Also, the reason why Bruce's mistake exposed this is interesting. Omitting the #define for ENABLE_THREAD_SAFETY does not actually break "pgbench -j" at all -- it has a fallback strategy that uses multiple subprocesses instead of multiple threads. However, it has only one srandom(

Re: [HACKERS] thread safety on clients

2009-12-11 Thread Tom Lane
I wrote: > The crash is real --- I've replicated it here. Still trying to figure > out what is the real cause. Okay, I've sussed it. The crash itself is due to a memory management mistake in the recently-rewritten EvalPlanQual code (pfree'ing a tuple that we still have live Datum references to).

Re: [HACKERS] thread safety on clients

2009-12-11 Thread Marko Kreen
On 12/11/09, Tom Lane wrote: > Alvaro Herrera writes: > > Yes, but what if you test with the broken pgbench? As Tom says, it > > should not be able to crash the backend no matter what it does. > > > The crash is real --- I've replicated it here. Still trying to figure > out what is the real

Re: [HACKERS] thread safety on clients

2009-12-11 Thread Tom Lane
Alvaro Herrera writes: > Yes, but what if you test with the broken pgbench? As Tom says, it > should not be able to crash the backend no matter what it does. The crash is real --- I've replicated it here. Still trying to figure out what is the real cause. regards, tom l

Re: [HACKERS] thread safety on clients

2009-12-11 Thread Alvaro Herrera
Jaime Casanova wrote: > On Thu, Dec 10, 2009 at 11:33 PM, Jaime Casanova > wrote: > > On Thu, Dec 10, 2009 at 11:22 PM, Tom Lane wrote: > >> > >> My bet is that the real problem was a build inconsistency in > >> the backend.  Does "make distclean" and rebuild make it go away? > >> > > > > actuall

Re: [HACKERS] thread safety on clients

2009-12-10 Thread Jaime Casanova
On Thu, Dec 10, 2009 at 11:33 PM, Jaime Casanova wrote: > On Thu, Dec 10, 2009 at 11:22 PM, Tom Lane wrote: >> >> My bet is that the real problem was a build inconsistency in >> the backend.  Does "make distclean" and rebuild make it go away? >> > > actually it was a clean build just after a cvs

Re: [HACKERS] thread safety on clients

2009-12-10 Thread Jaime Casanova
On Thu, Dec 10, 2009 at 11:22 PM, Tom Lane wrote: > > My bet is that the real problem was a build inconsistency in > the backend.  Does "make distclean" and rebuild make it go away? > actually it was a clean build just after a cvs co (not an updated tree), i build the binaries and installed it in

Re: [HACKERS] thread safety on clients

2009-12-10 Thread Tom Lane
Greg Smith writes: > The "-j" option is the recent addition to pgbench that causes it to > launch multiple client threads when enabled, each handling a subset of > the transactions. There's blocks of codes in pgbench.c now that depend > on having sane values for thread safety in libpq. That i

Re: [HACKERS] thread safety on clients

2009-12-10 Thread Bruce Momjian
Bruce Momjian wrote: > Greg Smith wrote: > > Bruce Momjian wrote: > > > Peter Eisentraut wrote: > > > > > >>> if i remove the -j option then it runs without a problem > > >>> > > >> Possibly related to the incomplete removal of the enable-thread-safety > > >> option that I just posted abo

Re: [HACKERS] thread safety on clients

2009-12-10 Thread Bruce Momjian
Greg Smith wrote: > Bruce Momjian wrote: > > Peter Eisentraut wrote: > > > >>> if i remove the -j option then it runs without a problem > >>> > >> Possibly related to the incomplete removal of the enable-thread-safety > >> option that I just posted about. > >> > > > > I thought about

Re: [HACKERS] thread safety on clients

2009-12-10 Thread Greg Smith
Bruce Momjian wrote: Peter Eisentraut wrote: if i remove the -j option then it runs without a problem Possibly related to the incomplete removal of the enable-thread-safety option that I just posted about. I thought about that but I can't figure out how that would affect pgbenc

Re: [HACKERS] thread safety on clients

2009-12-10 Thread Bruce Momjian
Peter Eisentraut wrote: > > starting vacuum...end. > > TRAP: FailedAssertion("!((data - start) == data_size)", File: > > "heaptuple.c", Line: 255) > > Client 0 aborted in state 8. Probably the backend died while processing. > > LOG: server process (PID 30713) was terminated by signal 6: Aborted >

Re: [HACKERS] thread safety on clients

2009-12-10 Thread Peter Eisentraut
On ons, 2009-12-09 at 14:02 -0500, Jaime Casanova wrote: > Hi, > > I compiled current HEAD and trying to use pgbench, i initialized a > test database this way: > bin/pgbench -i -F80 -s100 test > > and then run with this options: > bin/pgbench -c 50 -j 5 -l -t 20 test > > and get this crash: > ""

[HACKERS] thread safety on clients

2009-12-09 Thread Jaime Casanova
Hi, I compiled current HEAD and trying to use pgbench, i initialized a test database this way: bin/pgbench -i -F80 -s100 test and then run with this options: bin/pgbench -c 50 -j 5 -l -t 20 test and get this crash: """ starting vacuum...end. TRAP: FailedAssertion("!((data - start) == data_size)"