Re: [HACKERS] is_absolute_path incorrect on Windows
Tom Lane t...@sss.pgh.pa.us wrote: Yeah. If we were to go with Greg's suggestion of inventing a separate is_relative_to_cwd test function, I'd expect that to insist on no .. while it was at it. So it's now two problems, and I think this is my final comment: 1. is_relative_to_cwd() I continue to think is a bad name for something concerned about .. (plus on Windows not having a drive letter other than the current one); the normal meaning of relative path is merely not absolute 2. if this proposed new function is to replace some uses of is_absolute_path() then I'm afraid I'd not picked up on that (as Bruce did) and have no opinion on whether it's a good idea or not, and am not qualified to be the one doing the code investigation (not enough knowledge of the code, it's beta time, and I'm frantically short of time just now as well, sorry) Giles -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] is_absolute_path incorrect on Windows
Greg Stark gsst...@mit.edu wrote: /* NOTE: these two functions aren't complementary under windows, * be sure to use the right one */ /* Check path always means the same thing regardless of cwd */ is_absolute_path() /* Check that path is under cwd */ is_relative_path() Um ... isn't that second function name pretty misleading, if what you want is what the comment above it says? Assuming the comment is what you want (presumably, else you'd just negate a test of is_absolute_path()) then my suggestions for (IMHO :-) clearer names would be is_subdir_path() if you still want path in the name, or just is_subdir() if the meaning will be clear enough from context. Giles -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] is_absolute_path incorrect on Windows
Bruce Momjian br...@momjian.us wrote: is_relative_to_cwd()? ../../../../some/other/place/not/under/cwd Names are hard, but if I understood the original post, the revised function is intended to check that the directory is below the current working directory. If my understanding is wrong (always possible!) and it only has to be on the same drive, then your name is probably better although it doesn't mention 'drive' ... hrm. is_on_current_drive()? (Yuck.) is_on_current_filesystem()? (Yuck, but at least more general.) I think we (or at least I) need some clarification from the original poster about what the code is checking for in detail. Cheers, Giles -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] libpq, PQexecPrepared, data size sent to FE vs. FETCH_COUNT
Abhijit Menon-Sen a...@toroid.org wrote: Unless you explicitly declare and fetch from an SQL-level cursor, your many GBs of data are going to be transmitted to libpq, which will eat lots of memory. (The wire protocol does have something like cursors, but libpq does not use them, it retrieves the entire result set.) Sounds like a project. Anyone got any suggestions about semantics and function names? (Assuming that this can be done without causing more problems on the backend; I'd rather one frontend client get messed up than mess up the server if someone makes a query like that.) I'm not exactly volunteering to work on something like this (my TODO list is a trifle long) but I'm working on a native Go language interface for PostgreSQL presently (influced by but not an exact clone of libpq) so it's perhaps something I could do if I get free time in future. Giles -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] predefined macros for various BSD-based systems?
Tom Lane t...@sss.pgh.pa.us wrote: I suppose that at least some of the *BSD herd really do predefine some of the symbols being attributed to them here, but I would like to see something authoritative about which and what. Documentation follows, but first the summary: FreeBSD: __FreeBSD__ NetBSD: __NetBSD__ OpenBSD: __OpenBSD__ I believe those #defines also tell you what the release is. I didn't look into their encoding schemes just now, but can if you want. (OS X aka Darwin is harder: they seem to like __APPLE__, but to determine the OS version the best I can see is __ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__, which is quite horrid.) Re BSDi, I have no idea really but based on Google searching I'd bet on __bsdi__. Per Wikipedia BSDi was discontinued in 2003 and support ended in 2004. I submit that anyone still using it is not likely to be updating their PostgreSQL installation, so +1 from me for dropping support for it unless a volunteer using it comes forward. FYI (and you may know this, but I didn't learn until recently) GCC will tell you quite easily what #defines are predefined, and all those platforms use gcc: $ cc -E -dM - /dev/null | grep FreeBSD #define __FreeBSD_cc_version 73 #define __VERSION__ 4.2.1 20070719 [FreeBSD] #define __FreeBSD__ 7 But you wanted something authoritative, so here's what I found: FreeBSD === http://www.freebsd.org/doc/en/books/porters-handbook/porting-versions.html __FreeBSD__ is defined in all versions of FreeBSD. NetBSD == From the NetBSD-1.1 release notes (November, 1995): * implement new cpp predefine strategy define __NetBSD__, ... This is still the current behaviour, although the current release is 5.0.2 from February 2010. OpenBSD === http://www.openbsd.org/porting.html Generic Porting Hints * __OpenBSD__ should be used sparingly, if at all. Constructs that look like #if defined(__NetBSD__) || defined(__FreeBSD__) are often inappropriate. Don't add blindly __OpenBSD__ to it. Instead, try to figure out what's going on, and what actual feature is needed. Regards, Giles -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] make check hangs in alpha5
Hitoshi Harada umi.tan...@gmail.com wrote: Just note, I rebooted the guest VM today and retried but things are as before. The host reboot doesn't affect either. Bad luck. :-( I also tried another CentOS5.4 VM on the same VirtualBox and succeeded to build. Another RHEL Server 5.2 (Tikanga) x86_64 real machine can also build alpha5. Well, that sounds like a workaround: at least you can continue work, although it doesn't leave a lot of confidence that you won't have to reinstall _another_ virtual machine from time to time. :-( Seems quite hopeless but it's surely reproducible. They always are, but it can take a while. Back when I used to work in kernel support we had one problem take a year. We were Not Pleased, to put it mildly. (Nor was the main customer seeing it.) We got to the answer in the end, but oh boy, it was ugly on the way, and not our finest moment. (Answer was that someone left 'volatile' off a variable declaration, so it was a race condition. A race condition that came and went as code around the incorrect line of code was changed, so only some patch levels were affected. Plus, like any race condition, customers either tended never to see it or tended to hit it semi-regularly. It was ... difficult. It was't me who solved it; fortunately qsomeone else owned it and eventually thought to disassemble all the different patch levels. (He was looking for a compiler bug, unsurprisingly; when the problem first occurred he had only one patch version to deal with as well, so that approach didn't suggest itself first off, and everyone (where everyone was quite a lot of people) overlooked the missing 'volatile' during code reviews.) Back to VirtualBox: it blew up on me _again_ yesterday, refusing to boot FreeBSD as anything but 32 bit, no matter how I coaxed it and no matter that it had been running a 64bit installation for weeks before that. As booting a FreeBSD 64 bit kernel in 32 bit mode doesn't work (surprise!) I have exorcised VirtualBox from my system (like Parallels before it, which was even worse), and am deciding today qwhat real hardware I shall buy to replace VirtualBox. (Before anyone suggests VMware Fusion: they're honest enough to admit to not supporting one of the operating systems I care about, so that's not a solution, unfortunately.) So I can't test with VirtualBox anymore to help out, sorry. Giles -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] make check hangs in alpha5
Josh Berkus j...@agliodbs.com wrote: Could this be an issue with VirtualBox? Have you used this VM for testing before? As I've hit a few bugs in VirtualBox, this is a definite possibility. (So is Tom's suggestion of inconsistent sources.) Because I could, I just installed a new CentOS 5.4 (no updates, 64 bit) VM on VirtualBox 3.1.6 hosted on OX X 10.5.8 (Leopard) with bridged networking. I installed postgresql-9.0alpha5-revised.tar.bz2, compiled it with the default options, ran 'make check' and everything passed: === All 122 tests passed. === I'm not sure that this helps much as it doesn't rule out VirtualBox, but my experience with it running Linux has been positive. I do reboot the host at the first hint of trouble and that fixes many ills; VirtualBox is not what I would consider production ready, but it's competitive and I switched to it from Parallels which I'd paid for. I'd trial VMware but it doesn't claim to support the *BSDs which is a feature I need, and I've seen at least one report where it didn't work well with FreeBSD. That's hearsay, of course, but it makes me cautious, particularly when I've got something that works well enough. Regards, Giles -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] On file locking
Tom Lane wrote: On HPUX 10.20, flock doesn't seem to exist (hasn't got a man page nor any mention in /usr/include). Correct. Still isn't there in later releases. lockf says All locks for a process are released upon the first close of the file, even if the process still has the file opened, and all locks held by a process are released when the process terminates. and When a file descriptor is closed, all locks on the file from the calling process are deleted, even if other file descriptors for that file (obtained through dup() or open(), for example) still exist. which seems to imply (but doesn't actually say) that HPUX keeps track of exactly which process took out the lock, even if the file is held open by multiple processes. Having done some testing today, I now understand what the standards are trying to say when they talk about locks being inherited. Or at least I think I understand: standards are tricky, locking is subtle, and I'm prepared to be corrected if I'm wrong! All of these lock functions succeed when the same process asks for a lock that it already has. That is: fcntl(fd, ...); fcntl(fd, ...); /* success -- no error returned */ For flock() only, the lock is inherited by a child process along with the file descriptor so the child can re-issue the flock() call and that will pass, too: flock(fd, ...); pid = fork(); if (pid == 0) flock(fd, ...); /* success -- no error returned */ For fcntl() and lockf() the locks are not inherited, and the call in a child fails: fcntl(fd, ...); pid = fork(); if (pid == 0) fcntl(fd, ...); /* will fail and return -1 */ In no case does just closing the file descriptor in the child lose the parent's lock. I rationalise this as follows: 1. flock() is using a last close semantic, so closing the file descriptor is documented not to lose the lock 2. lockf() and fcntl() use a first close, but because the locks are not inherited by the child process the child can't unlock them This all doesn't look good for using file locks in the way I had in mind :-( ... but considering that all these man pages seem pretty vague, maybe some direct experimentation is called for. I conjecture that Tom was looking for a facility to lock a file and have it stay locked if the postmaster or any child process was still running. flock() fits the bill, but it's not portable everywhere. One additional warning: this stuff *is* potentially filesystem dependent, per the source code I looked at, which would call filesystem specific routines. I tested with HP-UX 11.00 (VxFS), NetBSD (FFS) and Linux (ext3). I've put the rough and ready test code up for FTP, if anyone wants to check my working: ftp://ftp.nemeton.com.au/pub/pgsql/ Limitations in the testing: I only used whole file locking (no byte ranges) and didn't prove that a lock taken by flock() is still held after a child calls close() as it is documented to be. Regards, Giles ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] On file locking
Curt Sampson [EMAIL PROTECTED] wrote: On Sun, 2 Feb 2003, Tom Lane wrote: This all doesn't look good for using file locks in the way I had in mind :-( ... but considering that all these man pages seem pretty vague, maybe some direct experimentation is called for. Definitely. I wonder about the NetBSD manpage quotes in the post you followed up to, given that last time I checked flock() was implmented, in the kernel, using fcntl(). Either that's changed, or the manpages are unclear or lying. Using the same kernel code != same semantics. I think the NetBSD manual pages are trying to say that it's safe to have lockf(), fcntl(), and flock() locking playing together. That needn't be the case on all operating systems and the standards don't require it. This has been my experience in the past; locking semantics are subtle and unclear enough that you really need to test for exactly what you want at build time on every system, and you've got to do this testing on the filesystem you intend to put the locks on. What he said ... Giles ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] On file locking
Curt Sampson [EMAIL PROTECTED] wrote: At any rate, it seems to me highly unlikely that, since the child has the *same* descriptor as the parent had, that the lock would disappear. It depends on the lock function. After fork(): o with flock() the lock continues to be held, but will be unlocked if any child process explicitly unlocks it o with fcntl() the lock is not inherited in the child o with lockf() the standards and manual pages don't say Boring reference material follows. flock = From the NetBSD manual page: NOTES Locks are on files, not file descriptors. That is, file descriptors du- plicated through dup(2) or fork(2) do not result in multiple instances of a lock, but rather multiple references to a single lock. If a process holding a lock on a file forks and the child explicitly unlocks the file, the parent will lose its lock. The Red Hat Linux 8.0 manual page has similar wording. (No standards to check here -- flock() is not standardised in POSIX, X/Open, Single Unix Standard, ...) fcntl = The NetBSD manual page notes that these locks are not inherited by child processes: Another minor semantic problem with this interface is that locks are not inherited by a child process created using the fork(2) function. Ditto the Single Unix Standard versions 2 and 3. lockf() === The standards and manual pages that I've checked don't discuss fork() in relation to lockf(), which seems a peculiar ommission and makes me suspect that behaviour has varied historically. In practice I would expect lockf() semantics to be the same as fcntl(). Regards, Giles ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] On file locking
This could be cleaned up rather dramatically if we were to use one of the file locking primitives supplied by the OS to grab an exclusive lock on the file, ... ... The first question for the group is: is it worth doing that? In the past it has been proposed and declined -- there is some stuff in the archives. While it would be beneficial to installations using local data it would introduce new failure modes for installations using NFS. Regards, Giles ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] sync()
Tom Lane writes: Right. Portably was the key word in my comment (sorry for not emphasizing this more clearly). The real problem here is how to know what is the actual behavior of each platform? I'm certainly not prepared to trust reading-between-the-lines-of-some-man-pages. And I can't think of a simple yet reliable direct test. Is the Single Unix Standard, version 2 (aka UNIX98) any better? It says for fsync(): The fsync() function forces all currently queued I/O operations associated with the file indicated by file descriptor fildes to the synchronised I/O completion state. All I/O operations are completed as defined for synchronised I/O file integrity completion. This to me clearly says that changes to the file must be written, not just changes made via this file descriptor. I did have to test this behaviour once (for a customer, strange situation) but I couldn't find a portable way to do it, either. What I did was read the appropriate disk block from the raw device to bypass the buffer cache. As this required low level knowledge of the on-disk filesystem layout it was not very portable. For anyone interested Tom Christiansen's icat program can be ported to UFS derived filesystems fairly easily: http://www.rosat.mpe-garching.mpg.de/mailing-lists/perl5-porters/1997-04/msg00487.html AFAIK, all Unix implementations are paranoid about consistency of filesystem metadata, including directory contents. So fsync'ing directories from a user process strikes me as a waste of time, ... There is one variant where this is not the case: Linux using ext2fs and possibly other filesystems. There was a flame fest of great entertainment value a few years ago between Linus Torvalds and Dan Bernstein. Of course, neither was able to influence the opinion of the other to any noticible degree, but it made fun reading. I think this might be a starting point: http://www.ornl.gov/cts/archives/mailing-lists/qmail/1998/05/msg00667.html A more recent posting from Linus where he continues to recommend fsync() is this: http://www.cs.helsinki.fi/linux/linux-kernel/2001-29/0659.html I've not heard that any other Unix-like OS has abandoned the traditional and POSIX semantic. assuming that it were portable, which I doubt. What we need to worry about is whether fsync'ing a bunch of our own data files is a practical substitute for a global sync() call. I wish that it were. There are situations (serveral GB buffer caches, for example) where I mistrust the current use of sync() to have all writes completed before the sleep() returns. My concern is theoretical at the moment -- I never get to play with machines that large! Regards, Giles ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] [GENERAL] v7.3.1 Bundled and Released ...
Tom Lane wrote: Any other arguments out there? Per-release tags make it easier to see quickly if some code has changed in -current or not. As the CVS tree is available via anoymous CVS (I think?), CVSup, and via the web so there are many potential users who are not active developers and who probably run releases rather than -current. Regards, Giles ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] HP-UX shared library installation is incorrect
I wrote: o so far I've only tested with the HP ANSI C compiler on 11.11, not with gcc or on other HP-UX releases. I can test other releases and probably with gcc, but if someone gets to it before me that would be just fine. I probably won't until the new year. Today is quiet so I did some checking on other HP-UX releases including a build with gcc and the fix seems OK, and I sent it to -patches. Giles ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
[HACKERS] HP-UX shared library installation is incorrect
Hi all, I'm planning an upgrade from 7.2.3 to 7.3.1 on some HP-UX systems. With the major number bump in 7.3.1 I was expecting to be able to do this upgrade without a flag day to re-compile clients, however due to misnaming of the PostgreSQL shared libraries on HP-UX I can't. A summary of the problem is that libraries are installed in such a way that clients link to 'libpq.sl' et al and not to the versioned library names such as 'libpq.2'. I doubt that this is solely a problem with my installation, but in case it is here is the configure command line I use: CC=cc CFLAGS=-O ./configure --prefix=/opt/pgsql-7.3.1 \ --with-includes=$HOME/include --with-libraries=$HOME/lib The extra includes and libraries are to pick up libz and libreadline. I am building on HP-UX 11.11 with the HP ANSI C compiler. The problem is similar with both 7.2.3 and 7.3.1. template1= select version(); version --- PostgreSQL 7.2.3 on hppa2.0w-hp-hpux11.11, compiled by cc -Ae (1 row) Below are the details as I understand them currently. There appear to be two problems. I've not yet investigated to find out if only one of them is the root cause. 1. the libraries are installed with their base names such as 'libpq.sl' being regular files, with versioned names being symbolic links to the base names: $ ls -l libpq* -rw-r--r-- 1 root users 103544 Oct 6 14:54 libpq.a -r-xr-xr-x 1 root users 102400 Oct 6 14:54 libpq.sl lrwxr-xr-x 1 root users8 Oct 6 14:54 libpq.sl.2 - libpq.sl This should be the other way around: $ ls -l libc.2 libc.sl -r-xr-xr-x 1 bin bin1843200 Jul 30 2001 libc.2 lrwxr-xr-x 1 root root15 Apr 18 2002 libc.sl - /usr/lib/libc.2 2. the versioned names are possibly incorrect: 'libpq.sl.2' should be 'libpq.2' by analogy to the libc example above. The effect of these problems is that client programs linked to these libraries link to the non-versioned names: $ ldd psql /usr/lib/libm.2 = /usr/lib/libm.2 /usr/lib/libnsl.1 =/usr/lib/libnsl.1 /usr/lib/libxti.2 =/usr/lib/libxti.2 /usr/lib/libdld.2 =/usr/lib/libdld.2 /usr/lib/libc.2 = /usr/lib/libc.2 /usr/lib/libdld.2 =/usr/lib/libdld.2 /usr/lib/libgen.2 =/usr/lib/libgen.2 /usr/lib/libc.2 = /usr/lib/libc.2 ../../../src/interfaces/libpq/libpq.sl = /opt/pgsql-7.2.3/lib/libpq.sl /usr/lib/libnsl.1 =/usr/lib/libnsl.1 (Aside: the ../../../src/... stuff is ugly, but harmless as far as I know.) Looking at the build process I see that Makefile.shlib can install either the versioned names or the base names: I don't yet understand why it chooses to install the base names on HP-UX. On NetBSD it does it the other way around. Someone familiar with the build process might know what the cause is here? Dropping the .sl from the name might be trickier, since it is expected to be present as a suffix on the base name but not included in the versioned names. Wrong: $ gmake -n install /bin/sh ../../../config/mkinstalldirs /opt/pgsql-7.3.1/lib/pa20_64 /opt/pgsql-7.3.1/include /opt/pgsql-7.3.1/include/internal /bin/sh ../../../config/install-sh -c -m 644 libpq.a /opt/pgsql-7.3.1/lib/pa20_64/libpq.a /bin/sh ../../../config/install-sh -c -m 555 libpq.sl /opt/pgsql-7.3.1/lib/pa20_64/libpq.sl cd /opt/pgsql-7.3.1/lib/pa20_64 \ rm -f libpq.sl.3 \ ln -s libpq.sl libpq.sl.3 /bin/sh ../../../config/install-sh -c -m 644 ./libpq-fe.h /opt/pgsql-7.3.1/include /bin/sh ../../../config/install-sh -c -m 644 ./libpq-int.h /opt/pgsql-7.3.1/include/internal /bin/sh ../../../config/install-sh -c -m 644 ./pqexpbuffer.h /opt/pgsql-7.3.1/include/internal Using NetBSD as an example, it's seen that the symbolic links are generated the other way around: $ gmake -n install /bin/sh ../../../config/mkinstalldirs /usr/local/pgsql-7.3/lib /usr/local/pgsql-7.3/include /usr/local/pgsql-7.3/include/internal /bin/sh ../../../config/install-sh -c -m 644 libpq.a /usr/local/pgsql-7.3/lib/libpq.a /bin/sh ../../../config/install-sh -c -m 755 libpq.so.2.2 /usr/local/pgsql-7.3/lib/libpq.so.2.2 cd /usr/local/pgsql-7.3/lib \ rm -f libpq.so.2 \ ln -s libpq.so.2.2 libpq.so.2 cd /usr/local/pgsql-7.3/lib \ rm -f libpq.so \ ln -s libpq.so.2.2 libpq.so /bin/sh ../../../config/install-sh -c -m 644 ./libpq-fe.h /usr/local/pgsql-7.3/include /bin/sh ../../../config/install-sh -c -m 644 ./libpq-int.h /usr/local/pgsql-7.3/include/internal /bin/sh ../../../config/install-sh -c -m 644 ./pqexpbuffer.h /usr/local/pgsql-7.3/include/internal If fixing this isn't obvious to someone, I'll investigate in detail in the new year. I can see no workaround for existing installations. New installations will want to adjust the libraries before linking clients to them. Regards, Giles
Re: [HACKERS] HP-UX shared library installation is incorrect
Tom Lane wrote: Probably so. I had not realized that HP's linker is affected by which way the symlinks run, but it appears that it is. I've just spoken to someone who knows more about the HP-UX toolchain than I do. The situation is that the library can have an internal name if the +h option was provided to ld. Since no name is specified in the link command in Makefile.shlib for HP-UX, the following (from the ld(1) manual page) applies instead: +h internal_name That is, if +h is not used, the shared library does not have an internal name and when an executable is built with the shared library, the linker records the library name that it looks at. ... That doesn't specifically say what happens when the library that it finds without an internal name is a symbolic link, but doing some testing shows me that the name used is the name that ld found the library as, i.e. libpq.sl rather than the desired libpq.sl.3. ld(1) doesn't read the link when it finds a symbolic link. My recommendation (with a pinch of salt, since I'm still not a HP-UX toolchain guru) is to add +h lib$(NAME)$(DLSUFFIX).$SO_MAJOR_VERSION) to the HP-UX LINK.shared line in src/Makefile.shlib, and to change the way the symlinks run as well, so that libpq.sl is a link to the latest version of libpq that is installed. I'm not sure where/how the symlink change can be made in the build environment, but clearly it can be. If I can figure out the answer, I'll try a test build and see how it goes. 2. the versioned names are possibly incorrect: 'libpq.sl.2' should be 'libpq.2' by analogy to the libc example above. This I disagree with: I consider HP's naming convention ugly and misleading. .sl should be in the name *somewhere*. I'll leave this to your taste and good judgement. The examples above follow your preference. :-) Regards, Giles ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] HP-UX shared library installation is incorrect
I wrote: My recommendation (with a pinch of salt, since I'm still not a HP-UX toolchain guru) is to add +h lib$(NAME)$(DLSUFFIX).$SO_MAJOR_VERSION) to the HP-UX LINK.shared line in src/Makefile.shlib, and to change the way the symlinks run as well, so that libpq.sl is a link to the latest version of libpq that is installed. OK, I've done this. A patch is below, against 7.3.1 source. I've sent it here instead of to the -patches list, since: o here is where the discussion was o so far I've only tested with the HP ANSI C compiler on 11.11, not with gcc or on other HP-UX releases. I can test other releases and probably with gcc, but if someone gets to it before me that would be just fine. I probably won't until the new year. Checking the 32 bit install: $ ls -l libpq* -rw-r--r-- 1 root users 158556 Dec 22 21:22 libpq.a lrwxr-xr-x 1 root users 10 Dec 22 21:22 libpq.sl@ - libpq.sl.3 -r-xr-xr-x 1 root users 143360 Dec 22 21:22 libpq.sl.3* $ /usr/ccs/bin/ldd psql /usr/lib/libm.2 = /usr/lib/libm.2 /usr/lib/libnsl.1 =/usr/lib/libnsl.1 /usr/lib/libxti.2 =/usr/lib/libxti.2 /usr/lib/libdld.2 =/usr/lib/libdld.2 /usr/lib/libc.2 = /usr/lib/libc.2 /usr/lib/libdld.2 =/usr/lib/libdld.2 /usr/lib/libgen.2 =/usr/lib/libgen.2 /usr/lib/libc.2 = /usr/lib/libc.2 ../../../src/interfaces/libpq/libpq.sl.3 = /opt/pgsql-7.3.1/lib/libpq.sl.3 /usr/lib/libnsl.1 =/usr/lib/libnsl.1 There's still that ../../../src/.. in the output, but the library has been found correctly with the version number, which is what is most important. I ran configure with a --prefix=/opt/pgsql-7.3.1 argument, but most clients will link against /opt/pgsql/lib so their paths won't have the version number. The 64 bit libraries are similar: $ ls -l libpq* -rw-r--r-- 1 root users 219202 Dec 22 21:38 libpq.a lrwxr-xr-x 1 root users 10 Dec 22 21:38 libpq.sl@ - libpq.sl.3 -r-xr-xr-x 1 root users 131024 Dec 22 21:38 libpq.sl.3* $ /usr/ccs/bin/ldd psql libpq.sl.3 = /opt/pgsql-7.3.1/lib/pa20_64/libpq.sl.3 libc.2 = /lib/pa20_64/libc.2 libxcurses.1 = /lib/pa20_64/libxcurses.1 libgen.2 = /lib/pa20_64/libgen.2 libdl.1 = /lib/pa20_64/libdl.1 libnsl.1 = /lib/pa20_64/libnsl.1 libm.2 = /lib/pa20_64/libm.2 libnsl.1 = /lib/pa20_64/libnsl.1 libdl.1 = /usr/lib/pa20_64/libdl.1 libdl.1 = /usr/lib/pa20_64/libdl.1 libxti.2 = /usr/lib/pa20_64/libxti.2 Again, the 64 bit binaries have linked to the versioned library. As a bonus the ldd output looks better too. :-) Regards, Giles *** src/Makefile.shlib-orig Wed Oct 9 09:21:54 2002 --- src/Makefile.shlib Sun Dec 22 21:07:43 2002 *** *** 126,134 endif ifeq ($(PORTNAME), hpux) ! # HPUX doesn't believe in version numbers for shlibs ! shlib := lib$(NAME)$(DLSUFFIX) ! LINK.shared = $(LD) -b +b $(libdir) endif ifeq ($(PORTNAME), irix5) --- 126,133 endif ifeq ($(PORTNAME), hpux) ! shlib := lib$(NAME)$(DLSUFFIX).$(SO_MAJOR_VERSION) ! LINK.shared = $(LD) +h $(shlib) -b +b $(libdir) endif ifeq ($(PORTNAME), irix5) ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Geometry regression tests (was Re: Float output formatting options)
I'd be interested to hear results of testing CVS tip (now 7.4devel) on other platforms. Does geometry pass cleanly for you? Yes for NetBSD-1.5.1/i386, where it previously didn't due to processor specific math libraries on this platform. Giles ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] pg_dump and large files - is this a problem?
Bruce Momjian [EMAIL PROTECTED] writes: OK, well BSD/OS now works, but I wonder if there are any other quad off_t OS's out there without fseeko. NetBSD prior to 1.6, released September 14, 2002. (Source: CVS logs.) OpenBSD prior to 2.7, released June 15, 2000. (Source: release notes.) FreeBSD has had fseeko() for some time, but I'm not sure which release introduced it -- perhaps 3.2.0, released May, 1999. (Source: CVS logs.) Regards, Giles ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] pg_dump and large files - is this a problem?
OK, does pre-1.6 NetBSD have fgetpos/fsetpos that is off_t/quad? Yes: int fgetpos(FILE *stream, fpos_t *pos); int fsetpos(FILE *stream, const fpos_t *pos); Per comments in stdio.h fpos_t is the same format as off_t, and off_t and fpos_t have been 64 bit since 1994. http://cvsweb.netbsd.org/bsdweb.cgi/basesrc/include/stdio.h Regards, Giles ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] contrib/fixchar (Was: Large databases, performance)
Well, this is not related to postgresql exactly but to summerise the problem, with libc patch PHCO_19090 or compatible upwards, on HP-UX11, isspace does not work correctly if input value is 127. o isspace() and such are defined in the standards to operate on characters o for historic C reasons, 'char' is widened to 'int' in function calls o it is platform dependent whether 'char' is a signed or unsigned type If your platform has signed 'char' (as HP-UX does on PA-RISC) and you pass a value that is negative it will be sign extended when converted to 'int', and may be outside the range of values for which isspace() is defined. Portable code uses 'unsigned char' when using ctype.h features, even though for many platforms where 'char' is an unsigned type it's not necessary for correct functioning. I don't see any isspace() or similar in the code though, so I'm not sure why this issue is being raised? Regards, Giles ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Improving backend startup interlock
Tom Lane writes: $ man flock No manual entry for flock. $ HPUX has generally taken the position of adopting both BSD and SysV features, so if it doesn't exist here, it's not portable to older Unixen ... If only local locking is at issue then finding any one of fcntl() locking, flock(), or lockf() would do. All Unixen will have one or more of these and autoconf machinery exists to find them. The issue Tom raised about NFS support remains: locking over NFS introduces new failure modes. It also only works for NFS clients that support NFS locking, which not all do. Mind you NFS users are currently entirely unprotected from someone starting a postmaster on a different NFS client using the same data directory right now, which file locking would prevent. So there is some win for NFS users as well as local filesystem users. (Anyone using NFS care to put their hand up? Maybe nobody does?) Is the benefit of better local filesystem behaviour plus multiple client protection for NFS users who have file locking enough to outweigh the drawbacks? My two cents says it is, but my two cents are worth approximately USD$0.01, which is to say not very much ... Regards, Giles ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Potential Large Performance Gain in WAL synching
Curtis Faith writes: I'm no Unix filesystem expert but I don't see how the OS can handle multiple writes and fsyncs to the same file descriptors without blocking other processes from writing at the same time. Why not? Other than the necessary synchronisation for attributes such as file size and modification times, multiple processes can readily write to different areas of the same file at the same time. fsync() may not return until after the buffers it schedules are written, but it doesn't have to block subsequent writes to different buffers in the file either. (Note too Tom Lane's responses about when fsync() is used and not used.) I'll have to write a test and see if there really is a problem. Please do. I expect you'll find things aren't as bad as you fear. In another posting, you write: Hmm, I keep hearing that buffer block writes are asynchronous but I don't read that in the code at all. There are simple write calls with files that are not opened with O_NOBLOCK, so they'll be done synchronously. The code for this is relatively straighforward (once you get past the storage manager abstraction) so I don't see what I might be missing. There is a confusion of terminology here: the write() is synchronous from the point of the application only in that the data is copied into kernel buffers (or pages remapped, or whatever) before the system call returns. For files opened with O_DSYNC the write() would wait for the data to be written to disk. Thus O_DSYNC is synchronous I/O, but there is no equivalently easy name for the regular flush to disk after write() returns that the Unix kernel has done ~forever. The asynchronous I/O that you mention (aio) is a third thing, different from both regular write() and write() with O_DSYNC. I understand that with aio the data is not even transferred to the kernel before the aio_write() call returns, but I've never programmed with aio and am not 100% sure how it works. Regards, Giles ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] pg_dump and large files - is this a problem?
Philip Warner writes: Yes, but there is no guarantee that off_t is implemented as such, nor would we be wise to assume so (most docs say explicitly not to do so). I suspect you're reading old documents, which is why I asked what you were referring to. In the '80s what you are saying would have been best practice, no question: 64 bit type support was not common. When talking of near-current systems with 64 bit off_t you are not going to find one without support for 64 bit integral types. Again yes, but the problem is the same: we need a way of making the *value* of an off_t portable (not just assuming it's a int64). In general that involves knowing how to turn it into a more universal data type (eg. int64, or even a string). So you need to know the size of off_t, which will be 32 bit or 64 bit, and then you need routines to convert that to a portable representation. The canonical solution is XDR, but I'm not sure that you want to bother with it or if it has been extended universally to support 64 bit types. If you limit the file sizes to 1GB (your less preferred option, I know;-) then like the rest of the PostgreSQL code you can safely assume that off_t fits into 32 bits and have a choice of functions (XDR or ntohl() etc) to deal with them and ignore 64 bit off_t issues altogether. If you intend pg_dump files to be portable avoiding the use of large files will be best. It also avoids issues on platforms such as HP-UX where large file support is available, but it has to be enabled on a per-filesystem basis. :-( Does the large file API have functions for representing the off_t values that is portable across architectures? And is the API also portable? The large files API is a way to access large files from 32 bit processes. It is reasonably portable, but is a red herring for what you are wanting to do. (I'm not convinced I am understanding what you're trying to do, but I have 'flu which is not helping. :-) Regards, Giles ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] pg_dump and large files - is this a problem?
Philip Warner writes: My limited reading of off_t stuff now suggests that it would be brave to assume it is even a simple 64 bit number (or even 3 32 bit numbers). What are you reading?? If you find a platform with 64 bit file offsets that doesn't support 64 bit integral types I will not just be surprised but amazed. One alternative, which I am not terribly fond of, is to have pg_dump write multiple files - when we get to 1 or 2GB, we just open another file, and record our file positions as a (file number, file position) pair. Low tech, but at least we know it would work. That does avoid the issue completely, of course, and also avoids problems where a platform might have large file support but a particular filesystem might or might not. Unless anyone knows of a documented way to get 64 bit uint/int file offsets, I don't see we have mush choice. If you're on a platform that supports large files it will either have a straightforward 64 bit off_t or else will support the large files API that is common on Unix-like operating systems. What are you trying to do, exactly? Regards, Giles ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Improving backend startup interlock
Tom Lane wrote: [ discussion of new startup interlock ] This is not quite ready for prime time yet, because it's not very bulletproof against the scenario where two would-be postmasters are starting concurrently. A solution to this is to require would-be postmasters to obtain an exclusive lock on a lock file before touching the pid file. (The lock file perhaps could be the pid file, but it doesn't have to be.) Is there some reason that file locking is not acceptable? Is there any platform or filesystem supported for use with PostgreSQL which doesn't have working exclusive file locking? A possible answer is to create a second lockfile that only exists for the duration of the startup sequence, and use that to ensure that only one process is trying this sequence at a time. ... This reintroduces the same problem we're trying to get away from (must rely on kill(PID, 0) to determine validity of the lock file), but at least the window of vulnerability is much smaller than before. A lock file locked for the whole time the postmaster is running can be responsible for preventing multiple postmasters running without relying on pids. All that is needed is that the OS drop exclusive file locks on process exit and that locks not survive across reboots. The checks of the shared memory segment (number of attachements etc) look after orphaned back end processes, per the proposal. Regards, Giles ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Upgrade process (was Re: 7.2.3?)
Tom lane wrote: True, but I think we'll have to deal with that anyway. Even if the physical database upgrade were trivial, people are going to find application compatibility problems due to schemas and other 7.3 changes. More reasons: a) learning curve -- I want to use 7.3 and gain some experience with 7.2.x - 7.3 migration before rolling out 7.3 to my users. b) change control and configuration freezes sometimes dictate when upgrades may be done. A 7.2.2 - 7.2.3 upgrade for bug fixes is much less intrusive than an upgrade to 7.3. So we're going to have to expend at least some work on fixing critical 7.2.* problems. (I just want to keep a tight rein on how much.) No argument here. Supporting multiple versions eats resources and eventually destabilises the earlier releases, so critial fixes only, please. New features and non-critical fixes however minor are actually unhelpful. Since PostgreSQL is open source, anyone who just has to have some minor new feature back ported can do it, or pay for it to be done. But this doesn't have to effect all users. Regards, Giles ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Improving backend startup interlock
Tom Lane wrote: Giles Lean [EMAIL PROTECTED] writes: Is there some reason that file locking is not acceptable? Is there any platform or filesystem supported for use with PostgreSQL which doesn't have working exclusive file locking? How would we know? We have never tried to use such a feature. I asked because I've not been following this project long enough to know if it had been tried and rejected previously. Newcomers being prone to making silly suggestions and all that. :-) For sure I would not trust it on an NFS filesystem. (Although we disparage running an NFS-mounted database, people do it anyway.) scratches head I can't work out if that's an objection or not. I'm certainly no fan of NFS locking, but if someone trusts their NFS client and server implementations enough to put their data on, they might as well trust it to get a single lock file for startup right too. IMHO. Your mileage may vary. Regards, Giles ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] beta1 packaged
Tom Lane writes: Rod Taylor [EMAIL PROTECTED] writes: SunOS control.shared2 5.7 Generic_106541-20 sun4u sparc SUNW,Ultra-5_10 shows an error in ALTER TABLE tests: ALTER TABLE FKTABLE ADD FOREIGN KEY(ftest1) references pktable(ptest1); NOTICE: ALTER TABLE will create implicit trigger(s) for FOREIGN KEY check(s) + ERROR: Relation pg_temp_5. does not exist That's pretty bizarre. Is it reproducible? Can you get in there with a debugger and try to figure out what's going wrong? I saw a similar error on a NetBSD-1.5.1/i386 box, but have not been able to reproduce it. Subsequent runs of 'gmake check' have all passed. Until I saw Rod's message I was thinking it was more evidence of hardware flakiness with this particular machine, but perhaps not. *** ./expected/alter_table.out Sat Aug 31 05:23:20 2002 --- ./results/alter_table.out Fri Sep 6 16:54:35 2002 *** *** 332,337 --- 332,338 -- Try (and succeed) ALTER TABLE tmp3 add constraint tmpconstr foreign key (a) references tmp2 matc h full; NOTICE: ALTER TABLE will create implicit trigger(s) for FOREIGN KEY check(s) + ERROR: Relation public.^B^UW88F00} does not exist -- Try (and fail) to create constraint from tmp5(a) to tmp4(a) - unique constr aint on -- tmp4 is a,b ALTER TABLE tmp5 add constraint tmpconstr foreign key(a) references tmp4(a) ma tch full; Regards, Giles ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Spinlock performance improvement proposal
Bruce Momjian [EMAIL PROTECTED] wrote: From postmaster startup, by default, could we try larger amounts of buffer memory until it fails then back off and allocate that? Seems like a nice default to me. So performance would vary depending on the amount of shared memory that could be allocated at startup? Not a good idea IMHO. Regards, Giles ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] syslog by default?
I know it can have an adverse effect on a mail server, is syslog going to give us any performance hits? Yes. On some platforms (HP-UX at least) applications can stall ~2s retrying if syslogd is not reading the messages written to its pipe. syslogd also has a reputation for using too much CPU under load, but this is annecdotal only -- I've not seen such a situation myself. Regards, Giles ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Log rotation?
Hi Peter, I've been playing with a little program I wrote whose sole purpose is to write its stdin to a file and close and reopen that file when it receives a signal. I figured this could work well when integrated transparently into pg_ctl. So, is log rotation a concern? Is this a reasonable solution? Other ideas? There was a discussion of this over a year ago. After I contributed to the discussion Tom Lane suggested I write something, and in the tradition of software development I did so but never quite finished it ... I got to the point that it works for me, then got distracted before completing a test suite. You may well prefer your own code, but this one supports rotation on size and/or time basis as well as on receipt of SIGHUP, and places a timestamp on each line written. It's also pretty careful about errors, which is one of the things that was disliked about the Apache program last time it was discussed. I am happy to contribute the code using the standard PostgreSQL license if it's wanted. (If anyone wants it under a BSD license or GPL for another purpose that's fine too.) I use the code on HP-UX, and developed it on NetBSD. There shouldn't be too many portability problems lurking, other than the usual hassles of what % escape to use in printf() for off_t. I doubt anyone wants log files larger than a couple of GB anyway? :-) ftp://ftp.nemeton.com.au/pub/src/logwrite-1.0alpha.tar.gz (No Grand Unified Logging Solutions please. And no, use syslog doesn't count.) grin One improvement I suggest is that the postmaster be taught to start (and restart if necessary) the log program. This avoids fragile startup scripts and also avoids taking down PostgreSQL if someone sends the wrong signal to the log program. Cheers, Giles ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://www.postgresql.org/search.mpl
[HACKERS] Re: [BUGS] pg_regress fails at point test
On 30 Jul 2001 Tom Lane wrote: [EMAIL PROTECTED] writes: gcc2.95; hppa2.0-hp-hpux10.20, postgresql 7.1.2 make check checks the first group all right (tests check out ok and a process postgres: shows up in the process list. When make check displays parallel group (18 tests): point lseg the postgres: process disappears from the process list and the test does not complete. Or, rather, not until I lose patience (15min). Incidentally the shell process that make started keeps running and eats up all the CPU time. Known bug in HPUX's Bourne shell --- evidently it can't cope with so many children in parallel. It works if you do gmake SHELL=/bin/ksh check or if you run the non-parallel installcheck. See FAQ_HPUX. If I still had a support contract in force with HP, I'd file a bug report... I reduced the pg_regress script to a test case and submitted a CR: JAGad84609 /usr/bin/sh hang on PostgreSQL regression test The defect is filed against HP-UX 11i, but I noted that the defect is present on 11.00 and 10.20 as well. Regards, Giles ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://www.postgresql.org/search.mpl
Re: [HACKERS] Re: Call for platforms
At 1:50 AM -0400 4/6/01, Tom Lane wrote:
"Henry B. Hotz" [EMAIL PROTECTED] writes:
Bottom line: 7.1RC1 passes most of the regression tests on
NetBSD/macppc.
The only thing that surprised me here was all of the warnings from
libreadline calls:
tab-complete.c: In function `initialize_readline':
tab-complete.c:103: warning: assignment from incompatible pointer type
tab-complete.c: In function `psql_completion':
tab-complete.c:292: warning: passing arg 2 of `completion_matches'
from incompatible pointer type
tab-complete.c:296: warning: passing arg 2 of `completion_matches'
from incompatible pointer type
What version of libreadline do you have installed, and how does it
declare completion_matches()?
$ uname -srm
NetBSD 1.5 i386
$ grep CPFunction /usr/include/readline.h
typedef char *CPFunction __P((const char *, int));
extern CPFunction *rl_completion_entry_function;
char **completion_matches __P((const char *, CPFunction *));
Putting the 'const' in the relevant PostgreSQL functions (diff against
7.1RC3 below) removes these warnings. I don't know what that does on
a machine using GNU readline ... I can check that in a day or two if
anyone's interested.
The NetBSD libedit-emulating-readline works just fine with psql even
without the warnings fixed -- they're harmless in this case.
Regards,
Giles
*** src/bin/psql/tab-complete.c-origMon Apr 2 05:17:32 2001
--- src/bin/psql/tab-complete.c Tue Apr 10 19:51:21 2001
***
*** 70,80
/* Forward declaration of functions */
! static char **psql_completion(char *text, int start, int end);
! static char *create_command_generator(char *text, int state);
! static char *complete_from_query(char *text, int state);
! static char *complete_from_const(char *text, int state);
! static char *complete_from_list(char *text, int state);
static PGresult *exec_query(char *query);
char *quote_file_name(char *text, int match_type, char *quote_pointer);
--- 70,80
/* Forward declaration of functions */
! static char **psql_completion(const char *text, int start, int end);
! static char *create_command_generator(const char *text, int state);
! static char *complete_from_query(const char *text, int state);
! static char *complete_from_const(const char *text, int state);
! static char *complete_from_list(char const *text, int state);
static PGresult *exec_query(char *query);
char *quote_file_name(char *text, int match_type, char *quote_pointer);
***
*** 177,183
libraries completion_matches() function, so we don't have to worry about it.
*/
static char **
! psql_completion(char *text, int start, int end)
{
/* This is the variable we'll return. */
char **matches = NULL;
--- 177,183
libraries completion_matches() function, so we don't have to worry about it.
*/
static char **
! psql_completion(const char *text, int start, int end)
{
/* This is the variable we'll return. */
char **matches = NULL;
***
*** 796,802
as defined above.
*/
static char *
! create_command_generator(char *text, int state)
{
static int list_index,
string_length;
--- 796,802
as defined above.
*/
static char *
! create_command_generator(const char *text, int state)
{
static int list_index,
string_length;
***
*** 829,835
etc.
*/
static char *
! complete_from_query(char *text, int state)
{
static int list_index,
string_length;
--- 829,835
etc.
*/
static char *
! complete_from_query(const char *text, int state)
{
static int list_index,
string_length;
***
*** 877,883
SQL words that can appear at certain spot.
*/
static char *
! complete_from_list(char *text, int state)
{
static int string_length,
list_index;
--- 877,883
SQL words that can appear at certain spot.
*/
static char *
! complete_from_list(const char *text, int state)
{
static int string_length,
list_index;
***
*** 911,917
The string to be passed must be in completion_charp.
*/
static char *
! complete_from_const(char *text, int state)
{
(void) text;/* We don't care about what was entered
* already. */
--- 911,917
The string to be passed must be in completion_charp.
*/
static char *
! complete_from_const(const char *text, int state)
{
(void) text;/* We don't care about what was entered
* already. */
---(end of
[HACKERS] 7.1RC3 passes as 64 bit application on HP-UX 11.00
As the subject says, PostgreSQL 7.1RC3 passes 'make check' when built as a 64 bit application on HP-UX 11.00. Yes Vince, I've added it to your results page too: http://www.postgresql.org/~vev/regress/report.php?50 Regards, Giles ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
[HACKERS] Re: Call for platforms
Thanks! I'm not too worried about 1.4.2, but be sure to let us know what the problem was; it may help out someone else... NetBSD-1.4.2/i386 passes all tests with 7.1RC3. My previous test failure on this platform was due to the timezone information on the test system not being standard; once that was corrected all tests pass. It is still necessary to add -ltermcap after -ledit in src/Makefile.global to have functional history editing in psql. Regards, Giles ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Call for platforms (HP-UX)
Okay, here are my results: Box 1: C180 (2.0 PA8000), HPUX 10.20 Compile with gcc: all tests pass Compile with cc: two lines of diffs in geometry (attached) Box 2: 715/75 (1.1 PA7100LC), HPUX 10.20 Compile with gcc: all tests pass Compile with cc: all tests pass I haven't had time to look at this further yet, except to build 7.1RC3 a couple of times with the HP ANSI C compiler today: PA-RISC 1.1 code (-Ae +O2 +DAportable): all tests pass PA-RISC 2.0 code (-Ae +O2 +DA2.0 +DS2.0): geometry failures I'm not sure how interesting these differences are anymore -- is there anyone familiar enough with floating point to determine if the results are acceptable (although currently unexpected :-) or not? Regards, Giles ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Escaping strings
what's the postgresql equivalent of mysql_real_escape_string() to escape strings that are going to be passed to queries? There doesn't seem to be a function to do this in libpq, which I find slightly odd. DBD::Pg has quote() function as per usual for perl's DBI, but that's not a lot of help for C. For reference it only doubles single quote characters ' to '' and backslash characters \ to \\. What I do -- and this may not be correct, so I encourage the more knowledgeable to speak up! -- is this: 1. single quotes ' become '' (typical SQL) 2. PostgreSQL supports backslash escape sequences, so unless your input uses these protect \ as \\. 3. I translate nul, formfeed, newline, and carriage return characters to \0, \f, \n, and \r respectively. In comparison mysql_real_escape_string() omits \f but also escapes ^Z and ". For binary data probably other control characters need to be escaped as well. I'm not clear on this yet, but with TOAST in 7.1 I'm sure there'll be more interest in storing arbitary binary data. Regards, Giles ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
[HACKERS] Call for platforms (HP-UX)
Hi all, I've built 7.1beta6 on a number of different HP-UX platforms (11.00 32 bit, 11.00 64 bit, 11i 32 bit). 1. On all these platforms 'make check' hung. Since that's not critical to whether PostgreSQL works or not I worked around it by using a different shell: gmake SHELL=$HOME/bin/pdksh check I'll look at this next week. If someone can confirm that /usr/bin/sh works for make check on HP-UX 10.20 that would be useful. 2. I saw two different sets of output for geometry.out. These seem to relate to the processor level: (a) on PA-RISC 1.1 some of the zero values are negative (b) on PA-RISC 2.0 the negative zeros were produced as on PA-RISC 1.1, plus about three results varied in the least significant digit. The PA-RISC 2.0 values were identical on two platforms: (i) PA8000 running 32 bit 11i (ii) PA8500 running 64 bit 11.00 If these results are OK (I assumed they were for the purposes of Vince's database, so I hope they are :-) then perhaps the attached outputs can be added to the expected results and resultmap updated for HP-UX 11? Regards, Giles -- -- GEOMETRY -- -- -- Points -- SELECT '' AS four, center(f1) AS center FROM BOX_TBL; four | center --+- | (1,1) | (2,2) | (2.5,3) | (3,3) (4 rows) SELECT '' AS four, (@@ f1) AS center FROM BOX_TBL; four | center --+- | (1,1) | (2,2) | (2.5,3) | (3,3) (4 rows) SELECT '' AS six, point(f1) AS center FROM CIRCLE_TBL; six | center -+--- | (0,0) | (1,2) | (1,3) | (1,2) | (100,200) | (100,0) (6 rows) SELECT '' AS six, (@@ f1) AS center FROM CIRCLE_TBL; six | center -+--- | (0,0) | (1,2) | (1,3) | (1,2) | (100,200) | (100,0) (6 rows) SELECT '' AS two, (@@ f1) AS center FROM POLYGON_TBL WHERE (# f1) 2; two | center -+- | (1.33,1.33) | (2.33,1.33) (2 rows) -- "is horizontal" function SELECT '' AS two, p1.f1 FROM POINT_TBL p1 WHERE ishorizontal(p1.f1, point '(0,0)'); two | f1 -+- | (0,0) | (-10,0) (2 rows) -- "is horizontal" operator SELECT '' AS two, p1.f1 FROM POINT_TBL p1 WHERE p1.f1 ?- point '(0,0)'; two | f1 -+- | (0,0) | (-10,0) (2 rows) -- "is vertical" function SELECT '' AS one, p1.f1 FROM POINT_TBL p1 WHERE isvertical(p1.f1, point '(5.1,34.5)'); one | f1 -+ | (5.1,34.5) (1 row) -- "is vertical" operator SELECT '' AS one, p1.f1 FROM POINT_TBL p1 WHERE p1.f1 ?| point '(5.1,34.5)'; one | f1 -+ | (5.1,34.5) (1 row) -- -- Line segments -- -- intersection SELECT '' AS count, p.f1, l.s, l.s # p.f1 AS intersection FROM LSEG_TBL l, POINT_TBL p; ERROR: Unable to identify an operator '#' for types 'lseg' and 'point' You will have to retype this query using an explicit cast -- closest point SELECT '' AS thirty, p.f1, l.s, p.f1 ## l.s AS closest FROM LSEG_TBL l, POINT_TBL p; thirty | f1 | s |closest ++---+--- | (0,0) | [(1,2),(3,4)] | (1,2) | (-10,0)| [(1,2),(3,4)] | (1,2) | (-3,4) | [(1,2),(3,4)] | (1,2) | (5.1,34.5) | [(1,2),(3,4)] | (3,4) | (-5,-12) | [(1,2),(3,4)] | (1,2) | (10,10)| [(1,2),(3,4)] | (3,4) | (0,0) | [(0,0),(6,6)] | (-0,0) | (-10,0)| [(0,0),(6,6)] | (0,0) | (-3,4) | [(0,0),(6,6)] | (0.5,0.5) | (5.1,34.5) | [(0,0),(6,6)] | (6,6) | (-5,-12) | [(0,0),(6,6)] | (0,0) | (10,10)| [(0,0),(6,6)] | (6,6) | (0,0) | [(10,-10),(-3,-4)]| (-2.04878048780488,-4.4390243902439) | (-10,0)| [(10,-10),(-3,-4)]| (-3,-4) | (-3,4) | [(10,-10),(-3,-4)]| (-3,-4) | (5.1,34.5) | [(10,-10),(-3,-4)]| (-3,-4) | (-5,-12) | [(10,-10),(-3,-4)]| (-1.60487804878049,-4.64390243902439) | (10,10)| [(10,-10),(-3,-4)]| (2.39024390243902,-6.48780487804878) | (0,0) | [(-100,200),(30,-40)] | (0.0028402365895872,15.384614860264) | (-10,0)| [(-100,200),(30,-40)] | (-9.99715942258202,15.3864610140472) | (-3,4) | [(-100,200),(30,-40)] | (-2.99789812267519,15.3851688427303) | (5.1,34.5) | [(-100,200),(30,-40)] |
Re: [HACKERS] Call for platforms (HP-UX)
I'll look at this next week. If someone can confirm that /usr/bin/sh works for make check on HP-UX 10.20 that would be useful. It does not work. See FAQ_HPUX. I'm confused: I don't see anything about shells or make check hanging in doc/FAQ_HPUX. There is clear instruction to use GNU make, which I am doing. I'll look into the problem anyway. (a) on PA-RISC 1.1 some of the zero values are negative Hmm, so does it match any of the existing geometry files? No ... I was hoping for that, but not. Regards, Giles ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[HACKERS] Call for platforms
Results of 'make check': NetBSD-1.5/i386 one spurious floating point test failure (mail sent to postgresql-bugs with details) NetBSD_1.5/alphaall tests passed NetBSD-1.4.2/i386 four tests fail timestamp... FAILED abstime ... FAILED tinterval... FAILED test horology ... FAILED I'll look into the 1.4.2 failures when/if I get time. If anyone wants the test output to examine please ask. Regards, Giles ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Call for platforms
PS: AFAIK geometry-positive-zeros-bsd works for all NetBSD platforms - the above difference is only for i386 + fpu. It doesn't on NetBSD-1.5/alpha -- there geometry-positive-zeros is correct. Regards, Giles ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://www.postgresql.org/search.mpl
Re: [HACKERS] Re: Call for platforms
NetBSD 2.8 alpha 7.1 2001-03-22, Giles Lean Correction: NetBSD-1.5/alpha. Ciao, Giles ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://www.postgresql.org/search.mpl
Re: [HACKERS] Call for platforms
Seems that following patch is needed. Now It Works For Me (tm). Giles, does the regress test now succed for you? Yes, but I don't like that it is 1.5 specific. I expect that later NetBSD/i386 releases will also have the "new" floating point behaviour by default, subject to /etc/ld.so.conf setting as Patrick Welche discovered. BTW NetBSD just uses "i386" for any x86. It's not necessary to allow for i486, i586 etc. Perhaps the resultmap format could be enhanced to allow wildcarding of the result files, and just accept either match? geometry/.*-netbsd=geometry-positive-zeros* Regards, Giles Index: src/test/regress/resultmap === RCS file: /home/projects/pgsql/cvsroot/pgsql/src/test/regress/resultmap,v retrieving revision 1.45 diff -u -r1.45 resultmap --- src/test/regress/resultmap2001/03/22 15:13:18 1.45 +++ src/test/regress/resultmap2001/03/22 17:29:49 @@ -17,6 +17,7 @@ geometry/.*-openbsd=geometry-positive-zeros-bsd geometry/.*-irix6=geometry-irix geometry/.*-netbsd=geometry-positive-zeros +geometry/i.86-.*-netbsdelf1.5=geometry-positive-zeros-bsd geometry/.*-sysv5uw7.*:cc=geometry-uw7-cc geometry/.*-sysv5uw7.*:gcc=geometry-uw7-gcc geometry/alpha.*-dec-osf=geometry-alpha-precision ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Re: Final Call: RC1 about to go out the door ...
Thomas Lockhart [EMAIL PROTECTED] writes: HPUX 10.20 (HP-PA architecture) Time to drop 9.2 from the list? I don't have it running here anymore. Is there anyone on the list who can test on HPUX 9? HP haven't supported 9.X since the end of 1999 on servers, and since earlier than that on workstations. I doubt anyone will expect to see it listed on the PostgreSQL list of supported platforms for 7.1. Regards, Giles ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] [Stephen C. Tweedie sct@redhat.com] Re: O_DSYNC flag for open
[ Drifting off topic ... ] Well, that guy might know all about Linux, but he doesn't know anything about HPUX (at least not any version I've ever run). O_SYNC is distinctly different from O_DSYNC around here. There is a HP_UX kernel flag 'o_sync_is_o_dsync' which will cause O_DSYNC to be treated as O_SYNC. It defaults to being off -- it is/was a backward compatibility "feature" since HP-UX 9.X (which is history now) had implemented O_SYNC as O_DSYNC. http://docs.hp.com/cgi-bin/otsearch/getfile?id=/hpux/onlinedocs/os/KCparam.OsyncIsOdsync.html Regards, Giles ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] [Stephen C. Tweedie sct@redhat.com] Re: O_DSYNC flag for open
There is a HP_UX kernel flag 'o_sync_is_o_dsync' which will cause O_DSYNC to be treated as O_SYNC. It defaults to being off -- it ... other way around there, of course. Trying to clarify and adding confusion instead. :-( is/was a backward compatibility "feature" since HP-UX 9.X (which is history now) had implemented O_SYNC as O_DSYNC. Muttering, Giles ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Performance monitor signal handler
Just to get some evidence at hand - could some owners of different platforms compile and run the attached little C source please? $ uname -srm FreeBSD 4.1.1-STABLE $ ./jan Pipe buffer is 16384 bytes Sys-V message queue buffer is 2048 bytes $ uname -srm NetBSD 1.5 alpha $ ./jan Pipe buffer is 4096 bytes Sys-V message queue buffer is 2048 bytes $ uname -srm NetBSD 1.5_BETA2 i386 $ ./jan Pipe buffer is 4096 bytes Sys-V message queue buffer is 2048 bytes $ uname -srm NetBSD 1.4.2 i386 $ ./jan Pipe buffer is 4096 bytes Sys-V message queue buffer is 2048 bytes $ uname -srm NetBSD 1.4.1 sparc $ ./jan Pipe buffer is 4096 bytes Bad system call (core dumped) # no SysV IPC in running kernel $ uname -srm HP-UX B.11.11 9000/800 $ ./jan Pipe buffer is 8192 bytes Sys-V message queue buffer is 16384 bytes $ uname -srm HP-UX B.11.00 9000/813 $ ./jan Pipe buffer is 8192 bytes Sys-V message queue buffer is 16384 bytes $ uname -srm HP-UX B.10.20 9000/871 $ ./jan Pipe buffer is 8192 bytes Sys-V message queue buffer is 16384 bytes HP-UX can also use STREAMS based pipes if the kernel parameter streampipes is set. Using STREAMS based pipes increases the pipe buffer size by a lot: # uname -srm HP-UX B.11.11 9000/800 # ./jan Pipe buffer is 131072 bytes Sys-V message queue buffer is 16384 bytes # uname -srm HP-UX B.11.00 9000/800 # ./jan Pipe buffer is 131072 bytes Sys-V message queue buffer is 16384 bytes Regards, Giles ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://www.postgresql.org/search.mpl
Re: [HACKERS] WAL SHM principles
When you mmap, you don't use write() ! mlock actualy locks page in memory and as long as the page is locked the OS doesn't attempt to store the dirty page. It is intended also for security app to ensure that sensitive data are not written to unsecure storage (hdd). It is definition of mlock so that you can be probably sure with it. News to me ... can you please point to such a definition? I see no reference to such semantics in the mlock() manual page in UNIX98 (Single Unix Standard, version 2). mlock() guarantees that the locked address space is in memory. This doesn't imply that updates are not written to the backing file. I would expect an OS that doesn't have a unified buffer cache but tries to keep a consistent view for mmap() and read()/write() to update the file. Regards, Giles ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://www.postgresql.org/search.mpl
Re: [HACKERS] RE: xlog checkpoint depends on sync() ... seems uns afe
Sounds quite unreliable to me. Unless there's some interlock ... like, say, the second sync not being able to advance past a buffer page that's as yet unwritten by the first sync. But would all Unixen share such a strange detail of implementation? I heard Kirk McKusick tell this story in a 4.4BSD internals class. His explanation was that having an *operator* type 'sync' three times provided enough time for the first sync to do the work before the operator powered the system down or reset it or whatever. I've not heard of any filesystem implementation where the number of sync() system calls issued makes a difference, and imagine that any programmer who has written code to call sync three times has only heard part of the story. :-) Regards, Giles ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
