Re: [PATCHES] Thread-safe PREPARE in ecpg
On Wed, Sep 26, 2007 at 01:43:34PM +0900, ITAGAKI Takahiro wrote: Here is a revised patch against CVS HEAD. I fixed a bug in ECPGdeallocate_all(). Applied to CVS HEAD. I also added your example to the regression tests. Michael -- Michael Meskes Email: Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) ICQ: 179140304, AIM/Yahoo: michaelmeskes, Jabber: [EMAIL PROTECTED] Go SF 49ers! Go Rhein Fire! Use Debian GNU/Linux! Use PostgreSQL! ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
[PATCHES] Thread-safe PREPARE in ecpg
Here is a WIP patch to make prepared statements thread-safe in ecpg. The variable prep_stmts was global but not protected by any locks. I divided it into per-connection field so that we can access prepared statements separately in each thread. I needed to change the following exported functions, but it will introduce an incompatibility issue. It might be ok for CVS HEAD, but I'd like to port the fix to back versions. Do you have any thoughts on how we can accomplish this better? From: - bool ECPGdeallocate(int, int, const char *name); - bool ECPGdeallocate_all(int, int); - char *ECPGprepared_statement(const char *name, int); To: - bool ECPGdeallocate(int, int, const char *connection_name, const char *name); - bool ECPGdeallocate_all(int, int, const char *connection_name); - char *ECPGprepared_statement(const char *connection_name, const char *name, int); (connection_name argument is added.) Regards, --- ITAGAKI Takahiro NTT Open Source Software Center ecpg_prepare.patch Description: Binary data ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [PATCHES] Thread-safe PREPARE in ecpg
On Tue, Sep 25, 2007 at 03:22:13PM +0900, ITAGAKI Takahiro wrote: Here is a WIP patch to make prepared statements thread-safe in ecpg. The variable prep_stmts was global but not protected by any locks. I divided it into per-connection field so that we can access prepared statements separately in each thread. Thanks a lot. This is exactly how I was planning to implement it, but I haven't even found the time to start coding yet. :-) Could you please create a small example that we could add to the regression suite? I needed to change the following exported functions, but it will introduce an incompatibility issue. It might be ok for CVS HEAD, but I'd like to port the fix to back versions. Do you have any thoughts on how we can accomplish this better? No idea at the moment, sorry. Michael -- Michael Meskes Email: Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) ICQ: 179140304, AIM/Yahoo: michaelmeskes, Jabber: [EMAIL PROTECTED] Go SF 49ers! Go Rhein Fire! Use Debian GNU/Linux! Use PostgreSQL! ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [PATCHES] Thread-safe PREPARE in ecpg
Here is a revised patch against CVS HEAD. I fixed a bug in ECPGdeallocate_all(). Michael Meskes [EMAIL PROTECTED] wrote: Could you please create a small example that we could add to the regression suite? The attached prep.pgc is an example for this fix, that repeats EXEC SQL PREPARE and EXECUTE in loops. It works with pthread and Win32 thread. I needed to change the following exported functions, but it will introduce an incompatibility issue. It might be ok for CVS HEAD, but I'd like to port the fix to back versions. Do you have any thoughts on how we can accomplish this better? No idea at the moment, sorry. For porting back to 8.2, I'm planning to keep ECPGdeallocate, ECPGdeallocate_all and ECPGprepared_statement as their original names, and add new versions with different names. If users use only thread default connection, the simplified version would work enough. Regards, --- ITAGAKI Takahiro NTT Open Source Software Center ecpg_prepare-8.3.patch Description: Binary data prep.pgc Description: Binary data ---(end of broadcast)--- TIP 6: explain analyze is your friend
[PATCHES] Thread-safe snprintf() vsnprintf() and printf()
And while we are on it, I would like to submit minor
changes to make snprintf() vsnprintf() and printf()
functions in src/port/snprintf.c thread-safe.
Best regards,
Nicolai Tufar
Index: src/port/snprintf.c
===
RCS file: /projects/cvsroot/pgsql/src/port/snprintf.c,v
retrieving revision 1.7
diff -c -r1.7 snprintf.c
*** src/port/snprintf.c 28 Feb 2005 14:16:16 - 1.7
--- src/port/snprintf.c 28 Feb 2005 23:11:22 -
***
*** 82,105
* for string length. This covers a nasty loophole.
*
* The other functions are there to prevent NULL pointers from
! * causing nast effects.
**/
! /*static char _id[] = $PostgreSQL: pgsql/src/port/snprintf.c,v 1.7 2005/02/28 14:16:16 momjian Exp $;*/
! static char *end;
! static int SnprfOverflow;
int snprintf(char *str, size_t count, const char *fmt,...);
int vsnprintf(char *str, size_t count, const char *fmt, va_list args);
int printf(const char *format, ...);
! static void dopr(char *buffer, const char *format, va_list args);
int
printf(const char *fmt,...)
{
int len;
va_list args;
! static char* buffer[4096];
char* p;
va_start(args, fmt);
--- 82,103
* for string length. This covers a nasty loophole.
*
* The other functions are there to prevent NULL pointers from
! * causing nasty effects.
**/
! /*static char _id[] = $PostgreSQL: pgsql/src/port/snprintf.c,v 1.6 2005/02/22 04:57:24 momjian Exp $;*/
int snprintf(char *str, size_t count, const char *fmt,...);
int vsnprintf(char *str, size_t count, const char *fmt, va_list args);
int printf(const char *format, ...);
! static void dopr(char *buffer, const char *format, va_list args, char *end);
int
printf(const char *fmt,...)
{
int len;
va_list args;
! char* buffer[4096];
char* p;
va_start(args, fmt);
***
*** 127,136
int
vsnprintf(char *str, size_t count, const char *fmt, va_list args)
{
str[0] = '\0';
end = str + count - 1;
! SnprfOverflow = 0;
! dopr(str, fmt, args);
if (count 0)
end[0] = '\0';
return strlen(str);
--- 125,134
int
vsnprintf(char *str, size_t count, const char *fmt, va_list args)
{
+ char *end;
str[0] = '\0';
end = str + count - 1;
! dopr(str, fmt, args, end);
if (count 0)
end[0] = '\0';
return strlen(str);
***
*** 140,150
* dopr(): poor man's version of doprintf
*/
! static void fmtstr(char *value, int ljust, int len, int zpad, int maxwidth);
! static void fmtnum(long_long value, int base, int dosign, int ljust, int len, int zpad);
! static void fmtfloat(double value, char type, int ljust, int len, int precision, int pointflag);
! static void dostr(char *str, int cut);
! static void dopr_outch(int c);
static char *output;
--- 138,148
* dopr(): poor man's version of doprintf
*/
! static void fmtstr(char *value, int ljust, int len, int zpad, int maxwidth, char *end);
! static void fmtnum(long_long value, int base, int dosign, int ljust, int len, int zpad, char *end);
! static void fmtfloat(double value, char type, int ljust, int len, int precision, int pointflag, char *end);
! static void dostr(char *str, int cut, char *end);
! static void dopr_outch(int c, char *end);
static char *output;
***
*** 154,160
#define FMTCHAR 4
static void
! dopr(char *buffer, const char *format, va_list args)
{
int ch;
long_long value;
--- 152,158
#define FMTCHAR 4
static void
! dopr(char *buffer, const char *format, va_list args, char *end)
{
int ch;
long_long value;
***
*** 417,427
case '%':
break;
default:
! dostr(???, 0);
}
break;
default:
! dopr_outch(ch);
break;
}
}
--- 415,425
case '%':
break;
default:
! dostr(???, 0, end);
}
break;
default:
! dopr_outch(ch, end);
break;
}
}
***
*** 448,474
case FMTSTR:
fmtstr(fmtparptr[i]-value, fmtparptr[i]-ljust,
fmtparptr[i]-len, fmtparptr[i]-zpad,
! fmtparptr[i]-maxwidth);
break;
case FMTNUM:
fmtnum(fmtparptr[i]-numvalue, fmtparptr[i]-base,
fmtparptr[i]-dosign, fmtparptr[i]-ljust,
! fmtparptr[i]-len, fmtparptr[i]-zpad);
break;
case FMTFLOAT:
fmtfloat(fmtparptr[i]-fvalue, fmtparptr[i]-type,
fmtparptr[i]-ljust, fmtparptr[i]-len,
! fmtparptr[i]-precision, fmtparptr[i]-pointflag);
break;
case FMTCHAR:
! dopr_outch(fmtparptr[i]-charvalue);
break;
}
format = fmtpar[i].fmtend;
goto nochar;
}
}
! dopr_outch(ch);
nochar:
/* nothing */
; /* semicolon required
[PATCHES] thread safety testing fix
When checking for thread safety with src/tools/thread/thread_test.c, the mktemp function wants an argument that contains 6 X, while the current version only supplies 5 X which will fail on my SuSE 8.1. Patch attached. Regards, Andreas Index: thread_test.c === RCS file: /projects/cvsroot/pgsql-server/src/tools/thread/thread_test.c,v retrieving revision 1.29 diff -u -r1.29 thread_test.c --- thread_test.c 27 Apr 2004 19:51:12 - 1.29 +++ thread_test.c 28 May 2004 11:39:00 - @@ -64,8 +64,8 @@ void func_call_1(void); void func_call_2(void); -#defineTEMP_FILENAME_1 /tmp/thread_test.1.X -#defineTEMP_FILENAME_2 /tmp/thread_test.2.X +#defineTEMP_FILENAME_1 /tmp/thread_test.1.XX +#defineTEMP_FILENAME_2 /tmp/thread_test.2.XX char *temp_filename_1; char *temp_filename_2; ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [PATCHES] thread safety testing fix
Andreas Pflug [EMAIL PROTECTED] writes: When checking for thread safety with src/tools/thread/thread_test.c, the mktemp function wants an argument that contains 6 X, while the current version only supplies 5 X which will fail on my SuSE 8.1. Patch attached. Isn't this going to break platforms that only want 5 X's ? regards, tom lane ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] thread safety testing fix
Tom Lane wrote: Andreas Pflug [EMAIL PROTECTED] writes: When checking for thread safety with src/tools/thread/thread_test.c, the mktemp function wants an argument that contains 6 X, while the current version only supplies 5 X which will fail on my SuSE 8.1. Patch attached. Isn't this going to break platforms that only want 5 X's ? Probably not., any additional X will just be ignored. AFAICS all platforms want at least 6 X anyway. Regards, Andreas ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [PATCHES] thread safety testing fix
Tom Lane wrote: Andreas Pflug [EMAIL PROTECTED] writes: When checking for thread safety with src/tools/thread/thread_test.c, the mktemp function wants an argument that contains 6 X, while the current version only supplies 5 X which will fail on my SuSE 8.1. Patch attached. Isn't this going to break platforms that only want 5 X's ? My BSD/OS doesn't specify a specific number of X's, but it does use an example with six, so six should be fine. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 8: explain analyze is your friend
[PATCHES] thread
Here is the email I have. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 From [EMAIL PROTECTED] Mon May 3 17:08:51 2004 Return-path: [EMAIL PROTECTED] Received: from postgresql.org (svr1.postgresql.org [200.46.204.71]) by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id i43L8nu29158 for [EMAIL PROTECTED]; Mon, 3 May 2004 17:08:50 -0400 (EDT) X-Original-To: [EMAIL PROTECTED] Received: from localhost (unknown [200.46.204.2]) by svr1.postgresql.org (Postfix) with ESMTP id CB710D1EF8F for [EMAIL PROTECTED]; Mon, 3 May 2004 18:05:28 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (neptune.hub.org [200.46.204.2]) (amavisd-new, port 10024) with ESMTP id 42915-10 for [EMAIL PROTECTED]; Mon, 3 May 2004 18:05:14 -0300 (ADT) Received: from mx-2.sollentuna.net (mx-2.sollentuna.net [195.84.163.199]) by svr1.postgresql.org (Postfix) with ESMTP id 4F9B6D1EF4C for [EMAIL PROTECTED]; Mon, 3 May 2004 18:05:07 -0300 (ADT) Received: from ALGOL.sollentuna.se (janus-en.sollentuna.se [195.84.163.194]) by mx-2.sollentuna.net (Postfix) with ESMTP id 23B618F2F4 for [EMAIL PROTECTED]; Mon, 3 May 2004 20:59:38 +0200 (CEST) Content-Class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/mixed; boundary=_=_NextPart_001_01C43140.C81FB140 X-MimeOLE: Produced By Microsoft Exchange V6.0.6249.0 Subject: [PATCHES] Run-as-admin warning for win32 Date: Mon, 3 May 2004 20:59:37 +0200 Message-ID: [EMAIL PROTECTED] X-MS-Has-Attach: yes Thread-Topic: Run-as-admin warning for win32 Thread-Index: AcQxQMgHMvYqdZrlRw+gl7lC+Xi5CQ== From: Magnus Hagander [EMAIL PROTECTED] To: [EMAIL PROTECTED] X-Virus-Scanned: by amavisd-new at postgresql.org X-Mailing-List: pgsql-patches Precedence: bulk Sender: [EMAIL PROTECTED] X-Spam-Checker-Version: SpamAssassin 2.61 (1.212.2.1-2003-12-09-exp) on candle.pha.pa.us X-Spam-Status: No, hits=-4.9 required=5.0 tests=BAYES_00 autolearn=ham version=2.61 Status: OR This is a multi-part message in MIME format. --_=_NextPart_001_01C43140.C81FB140 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable For review, comments and possible application to HEAD. This code implements a warning when the postmaster is started as a high-privilege account on win32 (administrator or power users). Previously, postgresql has exited out on Unix when running as root - this is a similar check, with the following differences: * We do a ereport(WARNING) instead of exitting out. The reason for this is that we can expect there are win32 admins that will want to run the server with a high privilege account. Just sending a warning will permit this (say, when debugging etc, or if people are just too lazy to care), while clearly stating it's not a recommended way to do it. * The Unix check is directly in main.c. We cannot do this on win32, because at this stage we can only printf and exit. Win32 needs ereport. Consider when runinng as a service - before we have loaded up postgresql.conf and noticed we should write to the eventlog, we cannot inform the user in any way (stderr =3D /dev/null from a service by default). Therefor, the win32 check is in PostmasterMain. There might be a slightly better place to put it, not 100% sure about that.. The win32 specific code is mainly in the file security.c to go in src/backend/port/win32. //Magnus =20 security.c admin_warning.patch=20 --_=_NextPart_001_01C43140.C81FB140 Content-Type: application/octet-stream; name=security.c Content-Transfer-Encoding: base64 Content-Description: security.c Content-Disposition: attachment; filename=security.c LyotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tCiAqCiAqIHNlY3VyaXR5 LmMKICogICAgTWljcm9zb2Z0IFdpbmRvd3MgV2luMzIgU2VjdXJpdHkgU3Vw cG9ydCBGdW5jdGlvbnMKICoKICogUG9ydGlvbnMgQ29weXJpZ2h0IChjKSAx OTk2LTIwMDMsIFBvc3RncmVTUUwgR2xvYmFsIERldmVsb3BtZW50IEdyb3Vw CiAqCiAqIElERU5USUZJQ0FUSU9OCiAqCSAgJFBvc3RncmVTUUwkCiAqCiAq LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQogKi8KCiNpbmNsdWRlICJw b3N0Z3Jlcy5oIgoKLyoKICogUmV0dXJucyBub256ZXJvIGlmIHRoZSBjdXJy ZW50IHVzZXIgaGFzIGFkbWluaXN0cmF0aXZlIHByaXZpbGVnZXMsCiAqIG9y IHplcm8gaWYgbm90LgogKi8KaW50IHBnd2luMzJfaXNfYWRtaW4odm9pZCkg ewoJSEFORExFIEFjY2Vzc1Rva2VuOwoJVUNIQVIgSW5mb0J1ZmZlclsxMDI0 XTsKCVBUT0tFTl9HUk9VUFMgR3JvdXBzID0gKFBUT0tFTl9HUk9VUFMpSW5m b0J1ZmZlcjsgCglEV09SRCBJbmZvQnVmZmVyU2l6ZTsKCVBTSUQgQWRtaW5p c3RyYXRvcnNTaWQ7CglQU0lEIFBvd2VyVXNlcnNTaWQ7CglTSURfSURFTlRJ RklFUl9BVVRIT1JJVFkgTnRBdXRob3JpdHkgPSB7IFNFQ1VSSVRZX05UX0FV
[PATCHES] thread patch
Here is the thread patch. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 Index: configure.in === RCS file: /cvsroot/pgsql-server/configure.in,v retrieving revision 1.287 diff -c -c -r1.287 configure.in *** configure.in12 Sep 2003 16:10:26 - 1.287 --- configure.in13 Sep 2003 14:47:43 - *** *** 1031,1047 # One trick here is that if we don't call AC_CHECK_FUNCS, the # functions are marked not found, which is perfect. # ! if test $enable_thread_safety = yes -a $NEED_REENTRANT_FUNC_NAMES = yes ; then _CFLAGS=$CFLAGS _LIBS=$LIBS CFLAGS=$CFLAGS $THREAD_CFLAGS LIBS=$LIBS $THREAD_LIBS ! AC_CHECK_FUNC(strerror_r, ! [], [AC_MSG_ERROR([strerror_r not found, required on this platform for --enable-thread-safety])]) ! AC_CHECK_FUNC(getpwuid_r, ! [], [AC_MSG_ERROR([getpwuid_r not found, required on this platform for --enable-thread-safety])]) ! AC_CHECK_FUNC(gethostbyname_r, ! [], [AC_MSG_ERROR([gethostbyname_r not found, required on this platform for --enable-thread-safety])]) CFLAGS=$_CFLAGS LIBS=$_LIBS fi --- 1031,1042 # One trick here is that if we don't call AC_CHECK_FUNCS, the # functions are marked not found, which is perfect. # ! if test $enable_thread_safety = yes -a $NEED_REENTRANT_FUNCS = yes ; then _CFLAGS=$CFLAGS _LIBS=$LIBS CFLAGS=$CFLAGS $THREAD_CFLAGS LIBS=$LIBS $THREAD_LIBS ! AC_CHECK_FUNCS([strerror_r getpwuid_r gethostbyname_r]) CFLAGS=$_CFLAGS LIBS=$_LIBS fi Index: src/include/port.h === RCS file: /cvsroot/pgsql-server/src/include/port.h,v retrieving revision 1.13 diff -c -c -r1.13 port.h *** src/include/port.h 5 Sep 2003 17:43:39 - 1.13 --- src/include/port.h 13 Sep 2003 14:47:46 - *** *** 114,124 #ifndef WIN32 extern int pqGetpwuid(uid_t uid, struct passwd * resultbuf, char *buffer, ! size_t buflen, struct passwd ** result); #endif extern int pqGethostbyname(const char *name, ! struct hostent * resbuf, ! char *buf, size_t buflen, ! struct hostent ** result, int *herrno); --- 114,124 #ifndef WIN32 extern int pqGetpwuid(uid_t uid, struct passwd * resultbuf, char *buffer, ! size_t buflen, struct passwd **result); #endif extern int pqGethostbyname(const char *name, ! struct hostent *resultbuf, ! char *buffer, size_t buflen, ! struct hostent **result, int *herrno); Index: src/port/thread.c === RCS file: /cvsroot/pgsql-server/src/port/thread.c,v retrieving revision 1.6 diff -c -c -r1.6 thread.c *** src/port/thread.c 5 Sep 2003 17:43:40 - 1.6 --- src/port/thread.c 13 Sep 2003 14:47:47 - *** *** 14,25 #include postgres.h /* *Threading sometimes requires specially-named versions of functions *that return data in static buffers, like strerror_r() instead of *strerror(). Other operating systems use pthread_setspecific() *and pthread_getspecific() internally to allow standard library ! *functions to return static data to threaded applications. * *Additional confusion exists because many operating systems that *use pthread_setspecific/pthread_getspecific() also have *_r versions --- 14,30 #include postgres.h + #include pthread.h + #include sys/types.h + #include pwd.h + /* *Threading sometimes requires specially-named versions of functions *that return data in static buffers, like strerror_r() instead of *strerror(). Other operating systems use pthread_setspecific() *and pthread_getspecific() internally to allow standard library ! *functions to return static data to threaded applications. And some ! *operating systems have neither, meaning we have to do our own locking. * *Additional confusion exists because many operating systems that *use pthread_setspecific/pthread_getspecific() also have *_r versions *** *** 36,46 *doesn't have strerror_r(), so we can't fall back to only using *_r *functions for threaded programs. * ! *The current setup is to assume either all standard functions are ! *thread-safe (NEED_REENTRANT_FUNC_NAMES=no), or the operating system ! *requires reentrant function names (NEED_REENTRANT_FUNC_NAMES=yes). *Compile and run
