[HACKERS] Intermittent buildfarm failures due to timestamptz test
Multiple systems are occasionally failing the timestamptz test at the check stage. From what I can tell, this is due to the newly added ISO week checks referring to TIMESTAMP_TBL used by the timestamp test instead of TIMESTAMPTZ_TBL. Both checks run at the same segment, so the test fails if the timing between the timestamp and timestamptz checks doesn't match. http://pgbuildfarm.org/cgi-bin/show_log.pl?nm=jackaldt=2007-02-16%2012:21:01 -- Seneca [EMAIL PROTECTED] ---(end of broadcast)--- TIP 6: explain analyze is your friend
[HACKERS] --with-libxml build failures on OS X
jackal's builds have been failing since xml.c started using xmlSaveToBuffer. It turns out that xmlSaveToBuffer is not yet implemented in the version of libxml2 that comes with OS X (10.4.8), but it is being implicitly defined during the build. The version of libxml2 on jackal is 2.6.21, but the requisite function was implemented in 2.6.23. The version of libxml used can be found in libxml/xmlversion.h in numerous forms. -- Seneca [EMAIL PROTECTED] ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Recent SIGSEGV failures in buildfarm HEAD
On Sun, Dec 31, 2006 at 05:43:45PM +0100, Stefan Kaltenbrunner wrote: Tom Lane wrote: What you seem to have here is infinite recursion during relcache initialization. That's surely not hard to believe, considering I just whacked that code around, and indeed changed some of the tests that are intended to prevent such recursion. But what I don't understand is why it'd be platform-specific, much less not perfectly repeatable on the platforms where it does manifest. Anyone have a clue? fwiw - I can trigger that issue now pretty reliably on a fast Opteron box (running Debian Sarge/AMD64) with make regress in a loop - I seem to be able to trigger it in about 20-25% of the runs. the resulting core however looks totally stack corrupted and not really usable :-( By reducing the stack size on jackal from the default of 8MB to 3MB, I can get this to trigger in roughly 30% of the runs while preserving the passed tests in the other parallel groups. -- Seneca [EMAIL PROTECTED] ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Suggestion: Which Binary?
David Wheeler wrote: But that's a PITA. I'd much rather have been able to tell configure *which* perl to use: ./configure --with-perl=/usr/bin/perl5.8.6 Would it be possible to add support for an optional argument to the PL/* options (--with-perl,--with-python, --with-tcl) so that we can get it to use the correct binary without having to resort to any shenanigans? Like passing PERL=/usr/bin/perl5.8.6 to configure? -- Seneca Cunningham [EMAIL PROTECTED] ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] [PORTS] Failed install - libgen.so doesn't exist
Martijn van Oosterhout wrote: On Mon, Feb 06, 2006 at 04:45:11PM -0500, Chris Browne wrote: Further, it appears to be AIX pre-4.3 only, when using it for dlopen() replacement... It would be an attractive idea to have configure detect not whether it's open, but rather whether it is needed, and leave it out for AIX 4.3 and better... That's kinda the point of these discussions, to answer the question: what is in those libraries we need? Which symbol did we want? Rather than trying to detect versions, is there some change in the library (added or removed symbol) that we can base our decision on? Does that library (ld) actually provide dlopen() or something else? libld.a does not provide dlopen(), libdl.a does. libld.a provides functions like ldopen() needed by src/backend/port/dynloader/aix.c for the dl*() substitutes built on AIX versions that lack dlopen and libdl.a, which appears to be versions prior to 4.3. -- Seneca Cunningham [EMAIL PROTECTED] ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Some platform-specific MemSet research
Martijn van Oosterhout wrote: On Tue, Jan 24, 2006 at 05:24:28PM -0500, Seneca Cunningham wrote: After reading the post on -patches proposing that MemSet be changed to use long instead of int32 on the grounds that a pair of x86-64 linux boxes took less time to execute the long code 64*10^6 times[1], I took a look at how the testcode performed on AIX with gcc. While the switch to long did result in a minor performance improvement, dropping the MemSetLoop in favour of the native memset resulted in the tests taking ~25% the time as the MemSetLoop-like int loop. The 32-bit linux system I ran the expanded tests on showed that for the buffer size range that postgres can use the looping MemSet instead of memset (size = 1024 bytes), MemSet generally had better performance. Could you please check the asm output to see what's going on. We've had tests like these produce odd results in the past because the compiler optimised away stuff that didn't have any effect. Since every memset after the first is a no-op, you want to make sure it's still actually doing the work... Well, on both linux and AIX, all 30 of the 6400 iterations loops from the source exist (10 int, 10 long, 10 memset). According to my understanding of the assembler, memset itself is only called for values = 64 bytes on both platforms and the memset is called in each iteration. The assembler for the 64 byte loops, with prepended line number, first loop MemSetLoop int-variant, second loop memset, third loop MemSetLoop long-variant: 64-bit AIX: 419 addi 3,1,112 420 li 4,0 421 bl .gettimeofday 422 nop 423 lis 10,0x3d0 424 cmpld 6,26,16 425 li 11,0 426 ori 10,10,36864 427 L..41: 428 bge 6,L..42 429 mr 9,26 430 li 0,0 431 L..44: 432 stw 0,0(9) 433 addi 9,9,4 434 cmpld 7,16,9 435 bgt 7,L..44 436 L..42: 437 addi 0,11,1 438 extsw 11,0 439 cmpw 7,11,10 440 bne+ 7,L..41 441 li 4,0 442 mr 3,22 443 lis 25,0x3d0 444 li 28,0 445 bl .gettimeofday 446 nop 447 li 4,64 448 addi 5,1,112 449 ld 3,LC..9(2) 450 mr 6,22 451 ori 25,25,36864 452 bl .print_time 453 addi 3,1,112 454 li 4,0 455 bl .gettimeofday 456 nop 457 L..46: 458 mr 3,26 459 li 4,0 460 li 5,64 461 bl .memset 462 nop 463 addi 0,28,1 464 extsw 28,0 465 cmpw 7,28,25 466 bne+ 7,L..46 467 li 4,0 468 mr 3,22 469 bl .gettimeofday 470 nop 471 li 4,64 472 addi 5,1,112 473 ld 3,LC..11(2) 474 mr 6,22 475 bl .print_time 476 addi 3,1,112 477 li 4,0 478 bl .gettimeofday 479 nop 480 lis 10,0x3d0 481 cmpld 6,26,16 482 li 11,0 483 ori 10,10,36864 484 L..48: 485 bge 6,L..49 486 mr 9,26 487 li 0,0 488 L..51: 489 std 0,0(9) 490 addi 9,9,8 491 cmpld 7,9,16 492 blt 7,L..51 493 L..49: 494 addi 0,11,1 495 extsw 11,0 496 cmpw 7,11,10 497 bne+ 7,L..48 498 li 4,0 499 mr 3,22 500 bl .gettimeofday 501 nop 502 li 4,64 503 addi 5,1,112 504 ld 3,LC..13(2) 505 mr 6,22 506 bl .print_time 32-bit Linux: 387 popl%ecx 388 popl%edi 389 pushl $0 390 leal-20(%ebp), %edx 391 pushl %edx 392 callgettimeofday 393 xorl%edx, %edx 394 addl$16, %esp 395 .L41: 396 movl-4160(%ebp), %eax 397 cmpl%eax, -4144(%ebp) 398 jae .L42 399 movl-4144(%ebp), %eax 400 .L44: 401 movl$0, (%eax) 402 addl$4, %eax 403 cmpl%eax, -4160(%ebp) 404 ja .L44 405 .L42: 406 incl%edx 407 cmpl$6400, %edx 408 jne .L41 409 subl$8, %esp 410 pushl $0 411 leal-28(%ebp), %edx 412 pushl %edx 413 callgettimeofday 414 leal-28(%ebp), %eax 415 movl%eax, (%esp) 416 leal-20(%ebp), %ecx 417 movl$64, %edx 418 movl$.LC5, %eax 419 callprint_time 420 popl%eax 421 popl%edx 422 pushl $0 423 leal-20(%ebp), %edx 424 pushl %edx 425 callgettimeofday 426 xorl%edi, %edi 427 addl$16, %esp 428 .L46: 429 pushl %eax 430 pushl $64 431 pushl $0 432 movl-4144(%ebp), %ecx 433 pushl %ecx 434 callmemset 435 incl%edi 436 addl$16, %esp 437 cmpl$6400, %edi 438 jne .L46 439 subl
[HACKERS] Some platform-specific MemSet research
After reading the post on -patches proposing that MemSet be changed to
use long instead of int32 on the grounds that a pair of x86-64 linux
boxes took less time to execute the long code 64*10^6 times[1], I took a
look at how the testcode performed on AIX with gcc. While the switch to
long did result in a minor performance improvement, dropping the
MemSetLoop in favour of the native memset resulted in the tests taking
~25% the time as the MemSetLoop-like int loop. The 32-bit linux system I
ran the expanded tests on showed that for the buffer size range that
postgres can use the looping MemSet instead of memset (size = 1024
bytes), MemSet generally had better performance.
Test results, reformatted for space:
* AIX5.3 ML3
gcc version 4.0.1
OBJECT_MODE=64 gcc -maix64 -O2
sizeof(int) = 4
sizeof(long) = 8
intsize=8 1.876096 1.875817 1.875998
long size=8 0.215347 0.215389 0.215367
memset size=8 0.127711 0.127726 0.127706
intsize=16 0.617316 0.617346 0.617300
long size=16 0.408607 0.408294 0.408263
memset size=16 0.212843 0.176918 0.212854
intsize=32 2.983032 2.982887 2.982724
long size=32 2.172499 2.172440 2.172549
memset size=32 0.255465 0.255449 0.255422
intsize=64 3.560825 3.559743 3.559785
long size=64 2.974126 2.999054 2.942597
memset size=64 1.021843 1.021709 1.021704
intsize=1284.983803 4.983515 4.983236
long size=1283.515213 3.514761 3.514733
memset size=1281.319846 1.319699 1.319671
intsize=2569.071160 9.070497 9.070350
long size=2567.428318 7.001997 6.990831
memset size=2561.830684 1.830558 1.830533
intsize=512 17.330519 17.329175 17.328520
long size=512 14.903931 14.902345 14.902329
memset size=5123.512420 3.512139 3.512111
intsize=1024 34.593734 34.592775 34.591700
long size=1024 23.804386 23.652192 24.043249
memset size=1024 6.010309 6.049034 6.052664
intsize=2048 66.380036 66.374455 66.375010
long size=2048 45.094202 45.087909 45.087128
memset size=2048 11.638963 11.662794 11.664649
intsize=4096 131.777427 131.764230 131.764542
long size=4096 88.906880 88.840758 88.887926
memset size=4096 22.882468 22.921160 22.920992
* Pentium 4 2.80GHz
Ubuntu 5.10 2.6.12-10-686 #1
gcc version 4.0.2 20050808 (prerelease) (Ubuntu 4.0.1-4ubuntu9)
gcc -O2
sizeof(int) = 4
sizeof(long) = 4
intsize=8 0.319620 0.270326 0.288407
long size=8 0.279157 0.278571 0.339791
memset size=8 0.186439 0.192561 0.194865
intsize=16 0.455448 0.459051 0.519848
long size=16 0.455193 0.451253 0.565159
memset size=16 0.257428 0.256752 0.356195
intsize=32 0.732009 0.730730 0.750304
long size=32 0.731353 0.734311 0.743041
memset size=32 1.386004 1.404297 1.378161
intsize=64 1.289708 1.397941 1.288536
long size=64 1.302256 1.380754 1.294904
memset size=64 2.965440 3.197489 2.958864
intsize=1283.162121 3.548065 3.158412
long size=1283.150525 3.161121 3.153037
memset size=1283.705133 3.739082 3.704949
intsize=2565.393701 5.415562 5.583510
long size=2565.420254 5.367381 5.362041
memset size=2569.246601 8.983931 9.040215
intsize=512 10.219667 9.854537 9.851564
long size=5129.906317 9.878196 10.202070
memset size=512 11.290588 11.050312 11.789231
intsize=1024 19.06 20.752631 19.846717
long size=1024 18.934663 18.870325 19.854066
memset size=1024 15.349694 15.487714 15.999638
intsize=2048 28.783087 28.214086 26.228851
long size=2048 26.628890 30.611856 26.245331
memset size=2048 24.434751 24.095879 23.435490
intsize=4096 53.817698 57.266583 51.547177
long size=4096 55.868670 53.012144 51.564656
memset size=4096 45.772710 40.651142 39.702063
[1] http://archives.postgresql.org/pgsql-patches/2006-01/msg00211.php
--
Seneca Cunningham
[EMAIL PROTECTED]
#include stdio.h
#include sys/time.h
#include string.h
#define TYPEALIGN(ALIGNVAL,LEN) \
(((long) (LEN) + ((ALIGNVAL) - 1)) ~((long) ((ALIGNVAL) - 1)))
#define MemSetLoop(type, start, val, len) \
do \
{ \
type * _start = (type *) (start); \
type * _stop = (type *) ((char *) _start + (size_t) (len)); \
\
while (_start _stop) \
*_start++ = 0; \
} while (0)
#define MAXALIGN 8
#define MAXSIZE 4096
#define LOOP (1000*1000*64)
static void print_time(const char* msg, int size, const struct timeval *start, const struct timeval *end)
{
double t;
t = (end-tv_sec - start-tv_sec) + (end-tv_usec - start-tv_usec) / 100.0;
printf(%s (size=%d) : %f\n, msg, size, t);
}
#define TEST(type, size) \
do { \
int i; \
gettimeofday(start, NULL); \
for(i = 0; i LOOP; i++) \
{ \
MemSetLoop(type, buffer, 0, size); \
} \
gettimeofday(end
[HACKERS] Should libedit be preferred to libreadline?
It would certainly seem so on AIX.
In tracking down why postgres 8.x would segfault on AIX 5.3, it became
apparent to me that libreadline.a is a problematic library to link
against and that libedit.a is preferable (and for reasons other than
that readline is GPL while postgres is BSD-licensed).
With AIX 5, the easiest way to get a shared object is to pass -bexpall
to the linker. This results in all symbols being exported. The problem
with this is that the linker will export all of libreadline's and
libhistory's symbols. In the case of libreadline.so.4 (and .5) on AIX 5
this includes symbols like strncpy and memmove, but on .4, not memcpy.
This is likely because libc.a does not export them.
What results from this is that when postgres is linked against readline
on AIX, it gets these memory functions through readline instead of its
own code. When readline 4.3 is used (what IBM provides in their AIX
Toolbox for Linux), postgres is known to crash. These segfaults (if
postgres was compiled with gcc) have occurred on AIX 5.3ML3, AIX 5.3ML1,
and AIX 5.2ML7. With readline 5.0, postgres merely gets these functions
through the shared library memory segments instead of the user memory
segments[6].
While it is possible to build libreadline in a manner that doesn't
export strncpy, neither of the prebuilt readlines for AIX 5 that I
checked were both shared and did not export strncpy. IBM's readline[5]
exports strncpy, UCLA's readline[4] is static. Building a shared
readline that doesn't export strncpy requires creating export files for
libreadline and libhistory that only list the symbols that they are
supposed to export and editing the shared library Makefile to add the
exports flags to the appropriate linker calls.
Whatever strategy we might take, using readline on AIX requires
considerable trickery and hacking around with the build environments.
Simply put, it's ghastly.
On the other hand, the port of NetBSD's editline that I tried[1] works
without build-hackery to the library and has reasonable exports. The
only changes to postgres that I needed to make were confined to telling
the configure script to check for libedit before libreadline and adding
a test for histedit.h. The attached patch contains my modifications.
It is also possible to use a wrapper like rlwrap[2] instead of linking
postgres against libreadline or libedit.
[1] port of NetBSD's editline
http://www.thrysoee.dk/editline/
[2] rlwrap
http://utopia.knoware.nl/~hlub/uck/software/
[3] IBM Redbook AIX 5L Porting Guide, section 9.2
http://www.redbooks.ibm.com/abstracts/sg246034.html?Open
http://www.redbooks.ibm.com/redbooks/pdfs/sg246034.pdf
[4] UCLA's readline package
http://aixpdslib.seas.ucla.edu/packages/readline.html
[5] IBM's readline package
http://www-03.ibm.com/servers/aix/products/aixos/linux/download.html
[6] IBM Redbook Developing and Porting C and C++ Applications on AIX,
page 110
http://www.redbooks.ibm.com/abstracts/sg245674.html?Open
http://www.redbooks.ibm.com/redbooks/pdfs/sg245674.pdf
--
Seneca Cunningham
[EMAIL PROTECTED]
diff -wu postgresql-8.1.0.orig/configure postgresql-8.1.0/configure
--- postgresql-8.1.0.orig/configure 2005-11-04 23:01:38.0 -0500
+++ postgresql-8.1.0/configure 2005-11-21 12:47:28.0 -0500
@@ -998,7 +998,7 @@
else
echo $as_me: WARNING: no configuration information is in $ac_dir 2
fi
-cd $ac_popdir
+cd $ac_popdir
done
fi
@@ -6498,7 +6498,7 @@
else
pgac_cv_check_readline=no
pgac_save_LIBS=$LIBS
-for pgac_rllib in -lreadline -ledit ; do
+for pgac_rllib in -ledit -lreadline ; do
for pgac_lib in -ltermcap -lncurses -lcurses ; do
LIBS=${pgac_rllib}${pgac_lib} $pgac_save_LIBS
cat conftest.$ac_ext _ACEOF
@@ -9646,6 +9646,152 @@
else
+for ac_header in histedit.h
+do
+as_ac_Header=`echo ac_cv_header_$ac_header | $as_tr_sh`
+if eval test \\${$as_ac_Header+set}\ = set; then
+ echo $as_me:$LINENO: checking for $ac_header 5
+echo $ECHO_N checking for $ac_header... $ECHO_C 6
+if eval test \\${$as_ac_Header+set}\ = set; then
+ echo $ECHO_N (cached) $ECHO_C 6
+fi
+echo $as_me:$LINENO: result: `eval echo '${'$as_ac_Header'}'` 5
+echo ${ECHO_T}`eval echo '${'$as_ac_Header'}'` 6
+else
+ # Is the header compilable?
+echo $as_me:$LINENO: checking $ac_header usability 5
+echo $ECHO_N checking $ac_header usability... $ECHO_C 6
+cat conftest.$ac_ext _ACEOF
+/* confdefs.h. */
+_ACEOF
+cat confdefs.h conftest.$ac_ext
+cat conftest.$ac_ext _ACEOF
+/* end confdefs.h. */
+$ac_includes_default
+#include $ac_header
+_ACEOF
+rm -f conftest.$ac_objext
+if { (eval echo $as_me:$LINENO: \$ac_compile\) 5
+ (eval $ac_compile) 2conftest.er1
+ ac_status=$?
+ grep -v '^ *+' conftest.er1 conftest.err
+ rm -f conftest.er1
+ cat conftest.err 5
+ echo $as_me:$LINENO: \$? = $ac_status 5
+ (exit $ac_status); }
+ { ac_try='test -z $ac_c_werror_flag || test ! -s conftest.err'
+ { (eval echo $as_me:$LINENO: \$ac_try
[HACKERS] memcpy SEGV on AIX 5.3
On an powerPC AIX 5.3 box, initdb from 8.1beta4 segfaults at
src/backend/utils/hash/dynahash.c:673. No segfaults occur and all 98
regression tests pass if a test is added to see if keycopy is memcpy and
if it is, go through a loop memcpying one byte at a time instead of
memcpying everything at once.
Configuration details:
-bash-2.05b$ gcc -v
Using built-in specs.
Target: powerpc-ibm-aix5.3.0.0
Configured with: /opt/src/gcc4/gcc-4.0.1/configure
--prefix=/opt/dbs/gcc4 --enable-languages=c,c++
Thread model: aix
gcc version 4.0.1
-bash-2.05b$ oslevel -r
5300-03
./configure --prefix=/opt/dbs/pgsql81b4-afilias-AIX53-2005-10-24
--with-includes=/opt/freeware/include --enable-debug
--enable-thread-safety --with-libraries=/opt/freeware/lib --enable-casert
InitPostgres
`- RelationCacheInitialize
`- formrdesc
`- RelationCacheInsert
`- hash_search
`- keycopy (keycopy == memcpy)
--- dynahash.c.orig 2005-10-14 22:49:33.0 -0400
+++ dynahash.c 2005-10-24 18:00:51.785106864 -0400
@@ -670,7 +670,14 @@
/* copy key into record */
currBucket-hashvalue = hashvalue;
- hashp-keycopy(ELEMENTKEY(currBucket), keyPtr, keysize);
+ if (hashp-keycopy == memcpy)
+ {
+ int tempfoo;
+ for (tempfoo = 0; tempfoo keysize; tempfoo++)
+ memcpy(ELEMENTKEY(currBucket)+tempfoo,
keyPtr+tempfoo, 1);
+ }
+ else
+ hashp-keycopy(ELEMENTKEY(currBucket), keyPtr, keysize);
/* caller is expected to fill the data field on return */
--
Seneca Cunningham
[EMAIL PROTECTED]
---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly
