Re: [HACKERS] Thread safety and libxml2

2010-04-22 Thread Robert Haas
On Mon, Apr 19, 2010 at 10:52 AM, Robert Haas robertmh...@gmail.com wrote: On Thu, Feb 18, 2010 at 8:41 PM, Bruce Momjian br...@momjian.us wrote: Peter Eisentraut wrote: On ons, 2009-12-30 at 12:55 -0500, Greg Smith wrote: Basically, configure failed on their OpenBSD system because thread

Re: [HACKERS] Thread safety and libxml2

2010-04-19 Thread Robert Haas
On Thu, Feb 18, 2010 at 8:41 PM, Bruce Momjian br...@momjian.us wrote: Peter Eisentraut wrote: On ons, 2009-12-30 at 12:55 -0500, Greg Smith wrote: Basically, configure failed on their OpenBSD system because thread safety is on but the libxml2 wasn't compiled with threaded support:

Re: [HACKERS] Thread safety and libxml2

2010-02-18 Thread Bruce Momjian
Peter Eisentraut wrote: On ons, 2009-12-30 at 12:55 -0500, Greg Smith wrote: Basically, configure failed on their OpenBSD system because thread safety is on but the libxml2 wasn't compiled with threaded support: http://xmlsoft.org/threads.html Disabling either feature (no

Re: [HACKERS] Thread safety and libxml2

2009-12-31 Thread Peter Eisentraut
On ons, 2009-12-30 at 12:55 -0500, Greg Smith wrote: Basically, configure failed on their OpenBSD system because thread safety is on but the libxml2 wasn't compiled with threaded support: http://xmlsoft.org/threads.html Disabling either feature (no --with-libxml or

[HACKERS] Thread safety and libxml2

2009-12-30 Thread Greg Smith
We got an interesting report on the testers list today: http://archives.postgresql.org/pgsql-testers/2009-12/msg0.php Basically, configure failed on their OpenBSD system because thread safety is on but the libxml2 wasn't compiled with threaded support: http://xmlsoft.org/threads.html

Re: [HACKERS] thread safety on clients

2009-12-14 Thread Tom Lane
Greg Stark gsst...@mit.edu writes: On Fri, Dec 11, 2009 at 6:08 PM, Tom Lane t...@sss.pgh.pa.us wrote: I'll commit a fix for that shortly, but this reminds me once again that the EvalPlanQual logic is desperately under-tested in our normal regression testing.  We really need some kind of

Re: [HACKERS] thread safety on clients

2009-12-14 Thread Alvaro Herrera
Tom Lane wrote: But my recollection of the parallel psql patch discussion is that it was rejected because nobody felt comfortable with the API design. Do we have any better ideas in that department yet? It wasn't rejected AFAICT. A finalized API with which there was (almost?) no dissent was

Re: [HACKERS] thread safety on clients

2009-12-13 Thread Greg Stark
On Fri, Dec 11, 2009 at 6:08 PM, Tom Lane t...@sss.pgh.pa.us wrote: I wrote: I'll commit a fix for that shortly, but this reminds me once again that the EvalPlanQual logic is desperately under-tested in our normal regression testing.  We really need some kind of testing infrastructure that

Re: [HACKERS] thread safety on clients

2009-12-11 Thread Alvaro Herrera
Jaime Casanova wrote: On Thu, Dec 10, 2009 at 11:33 PM, Jaime Casanova jcasa...@systemguards.com.ec wrote: On Thu, Dec 10, 2009 at 11:22 PM, Tom Lane t...@sss.pgh.pa.us wrote: My bet is that the real problem was a build inconsistency in the backend.  Does make distclean and rebuild make

Re: [HACKERS] thread safety on clients

2009-12-11 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes: Yes, but what if you test with the broken pgbench? As Tom says, it should not be able to crash the backend no matter what it does. The crash is real --- I've replicated it here. Still trying to figure out what is the real cause.

Re: [HACKERS] thread safety on clients

2009-12-11 Thread Marko Kreen
On 12/11/09, Tom Lane t...@sss.pgh.pa.us wrote: Alvaro Herrera alvhe...@commandprompt.com writes: Yes, but what if you test with the broken pgbench? As Tom says, it should not be able to crash the backend no matter what it does. The crash is real --- I've replicated it here. Still

Re: [HACKERS] thread safety on clients

2009-12-11 Thread Tom Lane
I wrote: The crash is real --- I've replicated it here. Still trying to figure out what is the real cause. Okay, I've sussed it. The crash itself is due to a memory management mistake in the recently-rewritten EvalPlanQual code (pfree'ing a tuple that we still have live Datum references to).

Re: [HACKERS] thread safety on clients

2009-12-11 Thread Greg Smith
Tom Lane wrote: Also, the reason why Bruce's mistake exposed this is interesting. Omitting the #define for ENABLE_THREAD_SAFETY does not actually break pgbench -j at all -- it has a fallback strategy that uses multiple subprocesses instead of multiple threads. However, it has only one srandom()

Re: [HACKERS] thread safety on clients

2009-12-11 Thread Tom Lane
Greg Smith g...@2ndquadrant.com writes: It sounds like random pgbench run for a while would certainly expose the same thing you're concerned about eventually. Yeah. Actually the odd thing about it is that the crash seemed to invariably be on conflicting pgbench_accounts updates, which is a

Re: [HACKERS] thread safety on clients

2009-12-11 Thread Tom Lane
I wrote: ... What I would have expected is crashes on the very similar updates to pgbench_branches, which is designed to be high-contention. It might be that there is some other effect going on here that explains why that wasn't happening. Need to go back and look more closely. ... and the

Re: [HACKERS] thread safety on clients

2009-12-10 Thread Peter Eisentraut
On ons, 2009-12-09 at 14:02 -0500, Jaime Casanova wrote: Hi, I compiled current HEAD and trying to use pgbench, i initialized a test database this way: bin/pgbench -i -F80 -s100 test and then run with this options: bin/pgbench -c 50 -j 5 -l -t 20 test and get this crash: starting

Re: [HACKERS] thread safety on clients

2009-12-10 Thread Bruce Momjian
Peter Eisentraut wrote: starting vacuum...end. TRAP: FailedAssertion(!((data - start) == data_size), File: heaptuple.c, Line: 255) Client 0 aborted in state 8. Probably the backend died while processing. LOG: server process (PID 30713) was terminated by signal 6: Aborted TRAP:

Re: [HACKERS] thread safety on clients

2009-12-10 Thread Greg Smith
Bruce Momjian wrote: Peter Eisentraut wrote: if i remove the -j option then it runs without a problem Possibly related to the incomplete removal of the enable-thread-safety option that I just posted about. I thought about that but I can't figure out how that would affect

Re: [HACKERS] thread safety on clients

2009-12-10 Thread Bruce Momjian
Greg Smith wrote: Bruce Momjian wrote: Peter Eisentraut wrote: if i remove the -j option then it runs without a problem Possibly related to the incomplete removal of the enable-thread-safety option that I just posted about. I thought about that but I can't figure

Re: [HACKERS] thread safety on clients

2009-12-10 Thread Bruce Momjian
Bruce Momjian wrote: Greg Smith wrote: Bruce Momjian wrote: Peter Eisentraut wrote: if i remove the -j option then it runs without a problem Possibly related to the incomplete removal of the enable-thread-safety option that I just posted about. I

Re: [HACKERS] thread safety on clients

2009-12-10 Thread Tom Lane
Greg Smith g...@2ndquadrant.com writes: The -j option is the recent addition to pgbench that causes it to launch multiple client threads when enabled, each handling a subset of the transactions. There's blocks of codes in pgbench.c now that depend on having sane values for thread safety in

Re: [HACKERS] thread safety on clients

2009-12-10 Thread Jaime Casanova
On Thu, Dec 10, 2009 at 11:22 PM, Tom Lane t...@sss.pgh.pa.us wrote: My bet is that the real problem was a build inconsistency in the backend.  Does make distclean and rebuild make it go away? actually it was a clean build just after a cvs co (not an updated tree), i build the binaries and

Re: [HACKERS] thread safety on clients

2009-12-10 Thread Jaime Casanova
On Thu, Dec 10, 2009 at 11:33 PM, Jaime Casanova jcasa...@systemguards.com.ec wrote: On Thu, Dec 10, 2009 at 11:22 PM, Tom Lane t...@sss.pgh.pa.us wrote: My bet is that the real problem was a build inconsistency in the backend.  Does make distclean and rebuild make it go away? actually it

[HACKERS] thread safety on clients

2009-12-09 Thread Jaime Casanova
Hi, I compiled current HEAD and trying to use pgbench, i initialized a test database this way: bin/pgbench -i -F80 -s100 test and then run with this options: bin/pgbench -c 50 -j 5 -l -t 20 test and get this crash: starting vacuum...end. TRAP: FailedAssertion(!((data - start) == data_size),

Re: [HACKERS] Thread safety

2008-11-27 Thread Peter Eisentraut
Magnus Hagander wrote: Can someone remind me why we have --enable-thread-safety? As opposed to it being default and having --disable-thread-safety. I don't have any numbers or a roster to support this, but I suppose that thread-safety is not supported on some platforms. So either we'd have

Re: [HACKERS] Thread safety

2008-11-27 Thread Alvaro Herrera
Peter Eisentraut wrote: Magnus Hagander wrote: Can someone remind me why we have --enable-thread-safety? As opposed to it being default and having --disable-thread-safety. I don't have any numbers or a roster to support this, but I suppose that thread-safety is not supported on some

Re: [HACKERS] Thread safety

2008-11-27 Thread Magnus Hagander
On 27 nov 2008, at 13.00, Alvaro Herrera [EMAIL PROTECTED] wrote: Peter Eisentraut wrote: Magnus Hagander wrote: Can someone remind me why we have --enable-thread-safety? As opposed to it being default and having --disable-thread-safety. I don't have any numbers or a roster to support

Re: [HACKERS] Thread safety

2008-11-27 Thread Andrew Chernow
Magnus Hagander wrote: On 27 nov 2008, at 13.00, Alvaro Herrera [EMAIL PROTECTED] wrote: Peter Eisentraut wrote: Magnus Hagander wrote: Can someone remind me why we have --enable-thread-safety? As opposed to it being default and having --disable-thread-safety. I don't have any numbers or

Re: [HACKERS] Thread safety

2008-11-27 Thread Zdenek Kotala
Andrew Chernow napsal(a): It would probably be useful to nail down the supported platforms, have a list somewhere of the oldest ones: oldest solaris, hpux, irix, aix, sco, etc... I ran into a few --enable-thread-safety problems, Magnus fixed the cygwin build already. hpux 10.20 and

[HACKERS] Thread safety

2008-11-26 Thread Magnus Hagander
Can someone remind me why we have --enable-thread-safety? As opposed to it being default and having --disable-thread-safety. /Magnus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] thread safety tests

2004-07-09 Thread Bruce Momjian
Jan Wieck wrote: I looked over the code and the only place getpwuid_r (through pqGetpwuid) is used is in libpq to look up the default username based on the euid for the connection to the backend. Unfortunately, I can't find any other way to do such a lookup in a thread-safe manner unless

Re: [HACKERS] thread safety tests

2004-07-09 Thread Bruce Momjian
Jan Wieck wrote: On 6/10/2004 2:11 AM, Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Are people OK with requiring PGUSER, $USER, $LOGNAME, or the username to be supplied by the connection string in libpq on platforms that want threads and don't have getpwuid_r() (Solaris,

Re: [HACKERS] thread safety tests

2004-06-10 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes: Are people OK with requiring PGUSER, $USER, $LOGNAME, or the username to be supplied by the connection string in libpq on platforms that want threads and don't have getpwuid_r() (Solaris, FreeBSD, etc.)? AFAICS that was not what Jan was suggesting at

Re: [HACKERS] thread safety tests

2004-06-10 Thread Jan Wieck
On 6/10/2004 2:11 AM, Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Are people OK with requiring PGUSER, $USER, $LOGNAME, or the username to be supplied by the connection string in libpq on platforms that want threads and don't have getpwuid_r() (Solaris, FreeBSD, etc.)? AFAICS that was

Re: [HACKERS] thread safety tests

2004-06-10 Thread Bruce Momjian
Jan Wieck wrote: On 6/10/2004 2:11 AM, Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Are people OK with requiring PGUSER, $USER, $LOGNAME, or the username to be supplied by the connection string in libpq on platforms that want threads and don't have getpwuid_r() (Solaris,

Re: [HACKERS] thread safety tests

2004-06-10 Thread Jan Wieck
On 6/10/2004 8:49 AM, Bruce Momjian wrote: Jan Wieck wrote: Make it so that --enable-thread-safety bombs out, but make another --enable-thread-safey-anyway work the way Tom descibed it. Sure, we can do that by just not running the thread_test program. In fact a cross-compile already skips

[HACKERS] thread safety tests

2004-06-09 Thread Jan Wieck
I am wondering why thread_test.c is checking for mktemp()? That function is nowhere used in the libpq. Also, I would suggest that we allow to build the libpq with thread specific compiler options, even if it is not entirely thread safe. It would require that a really multithreaded application

Re: [HACKERS] thread safety tests

2004-06-09 Thread Bruce Momjian
Jan Wieck wrote: I am wondering why thread_test.c is checking for mktemp()? That function is nowhere used in the libpq. Uh, it isn't checking for mktemp, it is using it, and it is using it because someone didn't like hard-coded paths I was using in the past. Is there something wrong with

Re: [HACKERS] thread safety tests

2004-06-09 Thread Jan Wieck
On 6/9/2004 9:36 AM, Bruce Momjian wrote: Also, I would suggest that we allow to build the libpq with thread specific compiler options, even if it is not entirely thread safe. It would require that a really multithreaded application has to have mutexes around certain operations, but being

Re: [HACKERS] thread safety tests

2004-06-09 Thread Bruce Momjian
Jan Wieck wrote: On 6/9/2004 9:36 AM, Bruce Momjian wrote: Jan Wieck wrote: I am wondering why thread_test.c is checking for mktemp()? That function is nowhere used in the libpq. Uh, it isn't checking for mktemp, it is using it, and it is using it because someone didn't like

Re: [HACKERS] thread safety tests

2004-06-09 Thread Jan Wieck
On 6/9/2004 9:36 AM, Bruce Momjian wrote: Jan Wieck wrote: I am wondering why thread_test.c is checking for mktemp()? That function is nowhere used in the libpq. Uh, it isn't checking for mktemp, it is using it, and it is using it because someone didn't like hard-coded paths I was using in the

Re: [HACKERS] thread safety tests

2004-06-09 Thread Jan Wieck
On 6/9/2004 11:45 AM, Bruce Momjian wrote: Jan Wieck wrote: The problem is that if your thread-safety tests fail, there is no way to build libpq with -pthread and -DREENTRANT or whatever is required on that platform. On Solaris this results in errno being defined as extern int errno; as

Re: [HACKERS] thread safety tests

2004-06-09 Thread Jan Wieck
On 6/9/2004 11:16 AM, Bruce Momjian wrote: Jan Wieck wrote: On 6/9/2004 9:36 AM, Bruce Momjian wrote: Jan Wieck wrote: I am wondering why thread_test.c is checking for mktemp()? That function is nowhere used in the libpq. Uh, it isn't checking for mktemp, it is using it, and it is using it

Re: [HACKERS] thread safety tests

2004-06-09 Thread Bruce Momjian
Jan Wieck wrote: On 6/9/2004 11:45 AM, Bruce Momjian wrote: Jan Wieck wrote: The problem is that if your thread-safety tests fail, there is no way to build libpq with -pthread and -DREENTRANT or whatever is required on that platform. On Solaris this results in errno being defined as

Re: [HACKERS] thread safety tests

2004-06-09 Thread Bruce Momjian
Well, looks like you are missing getpwuid_r(): Your system uses getpwuid() which is not thread-safe. ** and we don't have any workaround for this. --- Jan Wieck wrote: On 6/9/2004 11:16 AM, Bruce Momjian wrote:

Re: [HACKERS] thread safety tests

2004-06-09 Thread Jan Wieck
On 6/9/2004 1:04 PM, Bruce Momjian wrote: What we really need is a way to do the uid-username mapping in a thread-safe way. Could we check the environment for $USER or $LOGNAME? Could we require them to be set for thread builds on OS's without getpwuid_r and in cases where the username is not

Re: [HACKERS] thread safety tests

2004-06-09 Thread Bruce Momjian
Jan Wieck wrote: On 6/9/2004 1:04 PM, Bruce Momjian wrote: What we really need is a way to do the uid-username mapping in a thread-safe way. Could we check the environment for $USER or $LOGNAME? Could we require them to be set for thread builds on OS's without getpwuid_r and in cases

Re: [HACKERS] thread safety tests

2004-06-09 Thread Jan Wieck
On 6/9/2004 1:44 PM, Bruce Momjian wrote: Jan Wieck wrote: On 6/9/2004 1:04 PM, Bruce Momjian wrote: What we really need is a way to do the uid-username mapping in a thread-safe way. Could we check the environment for $USER or $LOGNAME? Could we require them to be set for thread builds on OS's

Re: [HACKERS] thread safety tests

2004-06-09 Thread Bruce Momjian
Jan Wieck wrote: On 6/9/2004 1:44 PM, Bruce Momjian wrote: Jan Wieck wrote: On 6/9/2004 1:04 PM, Bruce Momjian wrote: What we really need is a way to do the uid-username mapping in a thread-safe way. Could we check the environment for $USER or $LOGNAME? Could we require them to

Re: [HACKERS] thread safety tests

2004-06-09 Thread Bruce Momjian
Are people OK with requiring PGUSER, $USER, $LOGNAME, or the username to be supplied by the connection string in libpq on platforms that want threads and don't have getpwuid_r() (Solaris, FreeBSD, etc.)? If so, I can easily create a patch and apply it.

Re: [HACKERS] thread safety

2003-09-05 Thread Mendola Gaetano
Bruce Momjian [EMAIL PROTECTED] wrote: The thing that slows me down the most --- trips like FOSDEM. I am doing one every month or every other month. That takes 1/4 of each month. The threading discussion took 1/1000 of a month, but I do several hundred of those, so it fills up a month

Re: [HACKERS] thread safety

2003-09-05 Thread Bruce Momjian
[ CC to advocacy.] Mendola Gaetano wrote: Bruce Momjian [EMAIL PROTECTED] wrote: The thing that slows me down the most --- trips like FOSDEM. I am doing one every month or every other month. That takes 1/4 of each month. The threading discussion took 1/1000 of a month, but I do several

Re: [HACKERS] thread safety

2003-09-02 Thread Bruce Momjian
Mendola Gaetano wrote: I seen on this list a lot of energy ( also little flames involving SCO Co. ) spent on thread safety; was really necessary spent so much energy in this direction? I was at Fosdem in Bruxelles ( I spoke there about the use of postgres in my project ) and I seen al

Re: [HACKERS] thread safety

2003-09-02 Thread Philip Yarra
On Tue, 2 Sep 2003 03:06 am, Mendola Gaetano wrote: May be I'm wrong but I'd like know why thread safety was so necessary. It bugged Lee enough that he went and wrote a patch for it. It bugged me enough that I couldn't use it on our choice of platform (older Linux) that I did some little

[HACKERS] thread safety

2003-09-01 Thread Mendola Gaetano
I seen on this list a lot of energy ( also little flames involving SCO Co. ) spent on thread safety; was really necessary spent so much energy in this direction? I was at Fosdem in Bruxelles ( I spoke there about the use of postgres in my project ) and I seen al people there was really exicited

[HACKERS] thread safety

2003-09-01 Thread Lee Kindness
Probably because I worked on thread safety and produced a patch. If someone done the same for PITR and produced a patch i'm sure it would have generated much more interest. I couldn't have done PITR, so no loss of resource there. Was Bruce planning to do the PITR work? If so I guess a lot of his

Re: [HACKERS] thread safety

2003-09-01 Thread Bruno Wolff III
On Mon, Sep 01, 2003 at 18:16:25 +0100, Lee Kindness [EMAIL PROTECTED] wrote: Was Bruce planning to do the PITR work? If so I guess a lot of his time's been spent on integrating patches and the like - leaving less time for other developments. Bruce was working on the WIN32 port. That will