Re: [PATCHES] Change to -f in examples with input files
Uh, how is it more expressive? The only difference I see is the line numbers. Is that it? That could be a very big deal in case of error on a large file, so yes. In IRC, I always recommend that ppl use -f, since it's so much more useful :) Chris ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [PATCHES] WIP: bufmgr rewrite per recent discussions
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Tom Lane Sent: 17 February 2005 15:46 To: Mark Cave-Ayland Cc: pgsql-patches@postgresql.org Subject: Re: [PATCHES] WIP: bufmgr rewrite per recent discussions (cut) 3. Pad the LWLock struct (in src/backend/storage/lmgr/lwlock.c) to some power of 2 up to 128 bytes --- same issue of space wasted versus cross-lock contention. Having seen the results above, is it still worth looking at this? Yeah, probably, because there are other possible contention sources besides buffers that might be alleviated by padding LWLocks. For instance the buffer manager global locks and the LockMgrLock are probably all in the same cache line at the moment. Hi Tom, Here are the results from the LWLock test. Firstly here are the results with your second patch with no modifications as a refresher: PATCH #2 No modifications 10001 10 204.909702 205.01051 345.098727 345.411606 375.812059 376.37741 195.100496 195.197463 348.791481 349.111363 314.718619 315.139878 199.637965 199.735195 313.561366 313.803225 365.061177 365.666103 195.935529 196.029082 325.893744 326.171754 370.040623 370.625072 196.661374 196.756481 314.468751 314.711517 319.643145 320.099164 Mean: 198.4490132 198.5457462 329.5628138 329.841893 349.0551246 349.5815254 Here are the results with ALIGNOF_BUFFER=128 and padding LWLock to 64 bytes: PATCH #2 with ALIGNOF_BUFFER = 128 and LWLock padded to 64 bytes 10001 10 199.672932 199.768756 307.051571 307.299088 367.394745 368.016266 196.443585 196.532912 344.898219 345.204228 375.300921 375.979186 191.098411 191.185807 329.485633 329.77679 305.413304 305.841889 201.110132 201.210049 314.219784 314.476356 314.03306 314.477869 196.615748 196.706032 337.315295 337.62437 370.537538 371.16593 Mean: 196.9881616 197.0807112 326.5941004 326.8761664 346.5359136 347.096228 And finally here are the results with ALIGNOF_BUFFER = 128 and LWLock padded to 128 bytes: PATCH #2 with ALIGNOF_BUFFER = 128 and LWLock padded to 128 bytes 10001 10 195.357405 195.449704 346.916069 347.235895 373.354775 373.934842 190.428061 190.515077 323.932436 324.211975 361.908206 362.476886 206.059573 206.159472 338.288825 338.590642 306.22198 306.618689 195.336711 195.427757 309.316534 309.56603 305.295391 305.695336 188.896205 188.983969 322.889651 323.245394 377.673313 378.269907 Mean: 195.215591 195.3071958 328.268703 328.5699872 344.890733 345.399132 So again I don't see any performance improvement. However, I did manage to find out what was causing the 'freezing' I mentioned in my earlier email. By temporarily turning fsync=false in postgresql.conf, the freezing goes away, so I'm guessing it's something to do with disk/kernel caches and buffering. Since the drives are software RAID 1 with ext3 I guess that the server is running I/O bound under load which is perhaps why padding the data structures doesn't seem to make much difference. I'm not sure whether this makes the test results particularly useful though :( Kind regards, Mark. WebBased Ltd South West Technology Centre Tamar Science Park Plymouth PL6 8BT T: +44 (0)1752 791021 F: +44 (0)1752 791023 W: http://www.webbased.co.uk ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [PATCHES] WIP: pl/pgsql cleanup
Neil Conway wrote: Attached is a revised patch. Applied to HEAD. -Neil ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [PATCHES] [COMMITTERS] pgsql: Use _() macro consistently rather than gettext().
Thanks. Fixed. I wasn't aware the the macro namespace didn't distinguish X(a) and X(a,b) as different macros, so I was undefining _(x). --- Neil Conway wrote: Bruce Momjian wrote: Use _() macro consistently rather than gettext(). Add translation macros around strings that were missing them. This commit induces the following warnings: In file included from /home/neilc/pgsql/src/timezone/localtime.c:20: /home/neilc/pgsql/src/timezone/private.h:107:9: warning: extra tokens at end of #undef directive In file included from /home/neilc/pgsql/src/timezone/strftime.c:27: /home/neilc/pgsql/src/timezone/private.h:107:9: warning: extra tokens at end of #undef directive In file included from /home/neilc/pgsql/src/timezone/zic.c:18: /home/neilc/pgsql/src/timezone/private.h:107:9: warning: extra tokens at end of #undef directive In file included from /home/neilc/pgsql/src/timezone/ialloc.c:11: /home/neilc/pgsql/src/timezone/private.h:107:9: warning: extra tokens at end of #undef directive In file included from /home/neilc/pgsql/src/timezone/scheck.c:11: /home/neilc/pgsql/src/timezone/private.h:107:9: warning: extra tokens at end of #undef directive -Neil -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 Index: src/pl/plperl/SPI.xs === RCS file: /cvsroot/pgsql/src/pl/plperl/SPI.xs,v retrieving revision 1.12 diff -c -c -r1.12 SPI.xs *** src/pl/plperl/SPI.xs22 Feb 2005 04:42:44 - 1.12 --- src/pl/plperl/SPI.xs22 Feb 2005 18:05:37 - *** *** 1,7 /* this must be first: */ #include postgres.h /* Defined by Perl */ ! #undef _(x) /* perl stuff */ #include EXTERN.h --- 1,7 /* this must be first: */ #include postgres.h /* Defined by Perl */ ! #undef _ /* perl stuff */ #include EXTERN.h Index: src/pl/plperl/plperl.c === RCS file: /cvsroot/pgsql/src/pl/plperl/plperl.c,v retrieving revision 1.68 diff -c -c -r1.68 plperl.c *** src/pl/plperl/plperl.c 22 Feb 2005 04:42:44 - 1.68 --- src/pl/plperl/plperl.c 22 Feb 2005 18:05:37 - *** *** 39,45 #include postgres.h /* Defined by Perl */ ! #undef _(x) /* system stuff */ #include ctype.h --- 39,45 #include postgres.h /* Defined by Perl */ ! #undef _ /* system stuff */ #include ctype.h Index: src/pl/plperl/spi_internal.c === RCS file: /cvsroot/pgsql/src/pl/plperl/spi_internal.c,v retrieving revision 1.6 diff -c -c -r1.6 spi_internal.c *** src/pl/plperl/spi_internal.c22 Feb 2005 04:42:44 - 1.6 --- src/pl/plperl/spi_internal.c22 Feb 2005 18:05:37 - *** *** 6,12 #include postgres.h /* Defined by Perl */ ! #undef _(x) #include spi_internal.h --- 6,12 #include postgres.h /* Defined by Perl */ ! #undef _ #include spi_internal.h Index: src/timezone/private.h === RCS file: /cvsroot/pgsql/src/timezone/private.h,v retrieving revision 1.10 diff -c -c -r1.10 private.h *** src/timezone/private.h 22 Feb 2005 04:43:23 - 1.10 --- src/timezone/private.h 22 Feb 2005 18:05:38 - *** *** 104,110 ((TYPE_BIT(type) - TYPE_SIGNED(type)) * 302 / 1000 + 1 + TYPE_SIGNED(type)) #endif /* !defined INT_STRLEN_MAXIMUM */ ! #undef _(x) #define _(msgid) (msgid) /* --- 104,110 ((TYPE_BIT(type) - TYPE_SIGNED(type)) * 302 / 1000 + 1 + TYPE_SIGNED(type)) #endif /* !defined INT_STRLEN_MAXIMUM */ ! #undef _ #define _(msgid) (msgid) /* ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PATCHES] Repleacement for src/port/snprintf.c
On Mon, Feb 21, 2005 at 10:53:08PM -0500, Bruce Momjian wrote:
Applied.
Thanks a lot. The patch attached solves the tread
safety problem. Please review it before applying,
I am not sure I am doing the right thing
On Tue, 22 Feb 2005 19:57:15 +0100, Kurt Roeckx [EMAIL PROTECTED] wrote:
The configure test is a little broken. It needs to quote the
$'s.
I've rewritten the test a little.
This one needs applying too. $'s do get scrambled.
Best regards,
Nicolai.
*** ./src/port/snprintf.c.orig sali Şub 22 20:02:03 2005
--- ./src/port/snprintf.c sali Şub 22 21:59:48 2005
***
*** 80,96
* for string length. This covers a nasty loophole.
*
* The other functions are there to prevent NULL pointers from
! * causing nast effects.
**/
/*static char _id[] = $PostgreSQL: pgsql/src/port/snprintf.c,v 1.6 2005/02/22 04:57:24 momjian Exp $;*/
- static char *end;
- static int SnprfOverflow;
int snprintf(char *str, size_t count, const char *fmt,...);
int vsnprintf(char *str, size_t count, const char *fmt, va_list args);
int printf(const char *format, ...);
! static void dopr(char *buffer, const char *format, va_list args);
int
printf(const char *fmt,...)
--- 80,94
* for string length. This covers a nasty loophole.
*
* The other functions are there to prevent NULL pointers from
! * causing nasty effects.
**/
/*static char _id[] = $PostgreSQL: pgsql/src/port/snprintf.c,v 1.6 2005/02/22 04:57:24 momjian Exp $;*/
int snprintf(char *str, size_t count, const char *fmt,...);
int vsnprintf(char *str, size_t count, const char *fmt, va_list args);
int printf(const char *format, ...);
! static void dopr(char *buffer, const char *format, va_list args, char *end);
int
printf(const char *fmt,...)
***
*** 97,103
{
int len;
va_list args;
! static char* buffer[4096];
char* p;
va_start(args, fmt);
--- 95,101
{
int len;
va_list args;
! char* buffer[4096];
char* p;
va_start(args, fmt);
***
*** 125,134
int
vsnprintf(char *str, size_t count, const char *fmt, va_list args)
{
str[0] = '\0';
end = str + count - 1;
! SnprfOverflow = 0;
! dopr(str, fmt, args);
if (count 0)
end[0] = '\0';
return strlen(str);
--- 123,132
int
vsnprintf(char *str, size_t count, const char *fmt, va_list args)
{
+ char *end;
str[0] = '\0';
end = str + count - 1;
! dopr(str, fmt, args, end);
if (count 0)
end[0] = '\0';
return strlen(str);
***
*** 138,148
* dopr(): poor man's version of doprintf
*/
! static void fmtstr(char *value, int ljust, int len, int zpad, int maxwidth);
! static void fmtnum(long_long value, int base, int dosign, int ljust, int len, int zpad);
! static void fmtfloat(double value, char type, int ljust, int len, int precision, int pointflag);
! static void dostr(char *str, int cut);
! static void dopr_outch(int c);
static char *output;
--- 136,146
* dopr(): poor man's version of doprintf
*/
! static void fmtstr(char *value, int ljust, int len, int zpad, int maxwidth, char *end);
! static void fmtnum(long_long value, int base, int dosign, int ljust, int len, int zpad, char *end);
! static void fmtfloat(double value, char type, int ljust, int len, int precision, int pointflag, char *end);
! static void dostr(char *str, int cut, char *end);
! static void dopr_outch(int c, char *end);
static char *output;
***
*** 152,158
#define FMTCHAR 4
static void
! dopr(char *buffer, const char *format, va_list args)
{
int ch;
long_long value;
--- 150,156
#define FMTCHAR 4
static void
! dopr(char *buffer, const char *format, va_list args, char *end)
{
int ch;
long_long value;
***
*** 415,425
case '%':
break;
default:
! dostr(???, 0);
}
break;
default:
! dopr_outch(ch);
break;
}
}
--- 413,423
case '%':
break;
default:
! dostr(???, 0, end);
}
break;
default:
! dopr_outch(ch, end);
break;
}
}
***
*** 446,465
case FMTSTR:
fmtstr(fmtparptr[i]-value, fmtparptr[i]-ljust,
fmtparptr[i]-len, fmtparptr[i]-zpad,
! fmtparptr[i]-maxwidth);
break;
case FMTNUM:
fmtnum(fmtparptr[i]-numvalue, fmtparptr[i]-base,
fmtparptr[i]-dosign, fmtparptr[i]-ljust,
! fmtparptr[i]-len, fmtparptr[i]-zpad);
break;
case FMTFLOAT:
fmtfloat(fmtparptr[i]-fvalue, fmtparptr[i]-type,
fmtparptr[i]-ljust, fmtparptr[i]-len,
! fmtparptr[i]-precision, fmtparptr[i]-pointflag);
break;
case FMTCHAR:
! dopr_outch(fmtparptr[i]-charvalue);
break;
}
Re: [PATCHES] [pgsql-hackers-win32] Repleacement for src/port/snprintf.c
On Mon, Feb 21, 2005 at 10:53:08PM -0500, Bruce Momjian wrote:
Applied.
The configure test is a little broken. It needs to quote the
$'s.
I've rewritten the test a little.
Kurt
Index: config/c-library.m4
===
RCS file: /projects/cvsroot/pgsql/config/c-library.m4,v
retrieving revision 1.30
diff -u -r1.30 c-library.m4
--- config/c-library.m4 22 Feb 2005 03:55:50 - 1.30
+++ config/c-library.m4 22 Feb 2005 18:53:23 -
@@ -279,19 +279,17 @@
[AC_MSG_CHECKING([printf supports argument control])
AC_CACHE_VAL(pgac_cv_printf_arg_control,
[AC_TRY_RUN([#include stdio.h
+#include string.h
-int does_printf_have_arg_control()
+int main()
{
char buf[100];
/* can it swap arguments? */
- snprintf(buf, 100, %2$d|%1$d, 3, 4);
- if (strcmp(buf, 4|3) != 0)
-return 0;
- return 1;
-}
-main() {
- exit(! does_printf_have_arg_control());
+ snprintf(buf, 100, %2\$d %1\$d, 3, 4);
+ if (strcmp(buf, 4 3) != 0)
+return 1;
+ return 0;
}],
[pgac_cv_printf_arg_control=yes],
[pgac_cv_printf_arg_control=no],
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly
[PATCHES] optimize md5_text
This patch optimizes the md5_text() function (which is used to implement
the md5() SQL-level function). The old code did the following:
1. de-toast the datum
2. convert it to a cstring via textout()
3. get the length of the cstring via strlen()
Since we are treating the datum context as a blob of binary data, the
latter two steps are unnecessary. Once the data has been detoasted, we
can just use it as-is, and derive its length from the varlena metadata.
This patch improves some run-of-the-mill md5() computations by just
under 10% in my limited tests, and passes the regression tests.
I also noticed that md5_text() wasn't checking the return value of
md5_hash(); encountering OOM at precisely the right moment could result
in returning a random md5 hash. This patch corrects that. A better fix
would be to make md5_hash() only return on success (and/or allocate via
palloc()), but since it's used in the frontend as well I don't see an
easy way to do that.
Barring any objections, I'll apply this to HEAD tomorrow.
-Neil
Index: src/backend/libpq/md5.c
===
RCS file: /var/lib/cvs/pgsql/src/backend/libpq/md5.c,v
retrieving revision 1.27
diff -c -r1.27 md5.c
*** src/backend/libpq/md5.c 31 Dec 2004 21:59:50 - 1.27
--- src/backend/libpq/md5.c 23 Feb 2005 07:11:43 -
***
*** 289,296
* characters. you thus need to provide an array
* of 33 characters, including the trailing '\0'.
*
! * RETURNS 0 on failure (out of memory for internal buffers) or
! * non-zero on success.
*
* STANDARDS MD5 is described in RFC 1321.
*
--- 289,296
* characters. you thus need to provide an array
* of 33 characters, including the trailing '\0'.
*
! * RETURNS false on failure (out of memory for internal buffers) or
! * true on success.
*
* STANDARDS MD5 is described in RFC 1321.
*
Index: src/backend/utils/adt/varlena.c
===
RCS file: /var/lib/cvs/pgsql/src/backend/utils/adt/varlena.c,v
retrieving revision 1.118
diff -c -r1.118 varlena.c
*** src/backend/utils/adt/varlena.c 31 Dec 2004 22:01:22 - 1.118
--- src/backend/utils/adt/varlena.c 23 Feb 2005 07:06:44 -
***
*** 2310,2325
Datum
md5_text(PG_FUNCTION_ARGS)
{
! char *buff = PG_TEXT_GET_STR(PG_GETARG_TEXT_P(0));
! size_t len = strlen(buff);
char *hexsum;
text *result_text;
/* leave room for the terminating '\0' */
hexsum = (char *) palloc(MD5_HASH_LEN + 1);
/* get the hash result */
! md5_hash((void *) buff, len, hexsum);
/* convert to text and return it */
result_text = PG_STR_GET_TEXT(hexsum);
--- 2310,2331
Datum
md5_text(PG_FUNCTION_ARGS)
{
! text *in_text = PG_GETARG_TEXT_P(0);
! size_t len;
char *hexsum;
text *result_text;
+ /* Calculate the length of the buffer using varlena metadata */
+ len = VARSIZE(in_text) - VARHDRSZ;
+
/* leave room for the terminating '\0' */
hexsum = (char *) palloc(MD5_HASH_LEN + 1);
/* get the hash result */
! if (md5_hash(VARDATA(in_text), len, hexsum) == false)
! ereport(ERROR,
! (errcode(ERRCODE_OUT_OF_MEMORY),
! errmsg(out of memory)));
/* convert to text and return it */
result_text = PG_STR_GET_TEXT(hexsum);
---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send unregister YourEmailAddressHere to [EMAIL PROTECTED])
