[HACKERS] PostgreSQL and Windows

2003-06-12 Thread P.M
Hi, I would like to know if in the Windows installer version, you still plan to use cygwin to setup PostgreSQL ? If no, how do you setup files and parameters ? Moreover, What are the traps to pay attention when we install PostgreSQL on Windows ? tia, X04001 __

[HACKERS] Alter strings that don't belong to the application

2003-06-12 Thread Dennis Björklund
I've been looking into the code of psql to fix a problem with charsets and noticed that psql changes the strings it gets back from functions like PQfname() and PQgetvalue(). I can't find in the docs anything that says that it is okay to alter the returned string. It works since it's allocated and

[HACKERS] Printing groupClause List

2003-06-12 Thread Srikanth M
Hi! Can someone tell me where the Attribute name in the group clause of a given query are stored in the 'Query' structure. I have seen a member in 'Query' structure It is List *groupClause, if the Attribute names are stored in this list, then how can we access the names present in

Re: [HACKERS] Pre-allocation of shared memory ...

2003-06-12 Thread Hans-Jürgen Schönig
Yeah, I see it in the Mandrake kernel. But it's not in stock 2.4.19, so you can't assume everybody has it. We had this problem on a recent version of good old Slackware. I think we also had it on RedHat 8 or so. Doing this kind of killing is definitely a bad habit. I thought it had it had to

Re: [HACKERS] Alter strings that don't belong to the application

2003-06-12 Thread Tom Lane
=?ISO-8859-1?Q?Dennis_Bj=F6rklund?= [EMAIL PROTECTED] writes: I've been looking into the code of psql to fix a problem with charsets and noticed that psql changes the strings it gets back from functions like PQfname() and PQgetvalue(). I'd call that a bug in psql. Where do you see it

[HACKERS] CVS -Tip compile issue -- FreeBSD 4.8

2003-06-12 Thread Rod Taylor
gcc -O2 -Wall -Wmissing-prototypes -Wmissing-declarations -g -Wall -Wmissing-prototypes -Wmissin g-declarations -I../../../../src/include -c -o printtup.o printtup.c -MMD In file included from ../../../../src/include/libpq/libpq-be.h:22, from

Re: [HACKERS] Pre-allocation of shared memory ...

2003-06-12 Thread Andrew Dunstan
On this machine (RH9, kernel 2.4.20-18.9) the docs say (in /usr/src/linux-2.4/Documentation/vm/overcommit-accounting ): - The Linux kernel supports four overcommit handling modes 0 - Heuristic overcommit handling. Obvious overcommits of address space

Re: [HACKERS] CVS -Tip compile issue -- FreeBSD 4.8

2003-06-12 Thread Rod Taylor
It would seem the configure test isn't picking up on the structure. checking sys/socket.h usability... yes checking sys/socket.h presence... yes checking for sys/socket.h... yes --snip-- checking for struct sockaddr_storage... no FreeBSD sys/socket.h is attached. Failing configure test below

Re: [HACKERS] MARKED_FOR_UPDATE XMAX_COMMITTED == XMAX_INVALID ?

2003-06-12 Thread Manfred Koizar
On Wed, 11 Jun 2003 09:05:33 -0400, Tom Lane [EMAIL PROTECTED] wrote: If a transaction marks a tuple for update and later commits without actually having updated the tuple, [...] can we simply set the HEAP_XMAX_INVALID hint bit of the tuple? AFAICS this is a reasonable thing to do. Thanks for

Re: [HACKERS] CVS -Tip compile issue -- FreeBSD 4.8

2003-06-12 Thread Tom Lane
Rod Taylor [EMAIL PROTECTED] writes: It would seem the configure test isn't picking up on the structure. It works here (where works is defined as finds the struct on Linux and doesn't find it on HPUX --- both correct according to a search of /usr/include). You'll need to dig into why it fails

Re: [HACKERS] Pre-allocation of shared memory ...

2003-06-12 Thread Jon Lapham
Tom Lane wrote: Is this a Linux machine? If so, the true explanation is probably (c): the kernel is kill 9'ing randomly-chosen database processes whenever it starts to feel low on memory. I would suggest checking the postmaster log to determine the signal number the failed backends are dying

Re: [HACKERS] Alter strings that don't belong to the application

2003-06-12 Thread Dennis Björklund
On Thu, 12 Jun 2003, Tom Lane wrote: I'd call that a bug in psql. Where do you see it happening exactly? It's the utf-8 validation function (mbvalidate) that removes characters that it does not understand. No, I think the return value ought to be treated as const char *. We're hesitant to

Re: [HACKERS] Pre-allocation of shared memory ...

2003-06-12 Thread Tom Lane
Jon Lapham [EMAIL PROTECTED] writes: Just curious. What would a rationally designed OS do in an out of memory situation? Fail malloc() requests. The sysctl docs that Andrew Dunstan just provided give some insight into the problem: the default behavior of Linux is to promise more virtual

Re: [HACKERS] Pre-allocation of shared memory ...

2003-06-12 Thread Jon Lapham
Tom Lane wrote: [snip] The setting now called paranoid overcommit is IMHO the *only* acceptable one for any sort of server system. With anything else, you risk having critical userspace daemons killed through no fault of their own. Wow. Thanks for the info. I found the documentation you are

Re: [HACKERS] Pre-allocation of shared memory ...

2003-06-12 Thread Bruce Momjian
What really kills [:-)] me is that they allocate memory assuming I will not be using it all, then terminate the executable in an unrecoverable way when I go to use the memory. And, they make a judgement on users who don't want this by calling them paranoid. I will add something to the docs

Re: [HACKERS] Pre-allocation of shared memory ...

2003-06-12 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes: What really kills [:-)] me is that they allocate memory assuming I will not be using it all, then terminate the executable in an unrecoverable way when I go to use the memory. To be fair, I'm probably misstating things by referring to malloc(). The big

Re: [HACKERS] Pre-allocation of shared memory ...

2003-06-12 Thread Bruce Momjian
OK, doc patch attached and applied. Improvements? --- Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: What really kills [:-)] me is that they allocate memory assuming I will not be using it all, then terminate

Re: [HACKERS] Pre-allocation of shared memory ...

2003-06-12 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes: OK, doc patch attached and applied. Improvements? I think it would be worth spending another sentence to tell people exactly what the symptom looks like, ie, backends dying with signal 9. regards, tom lane

Re: [HACKERS] Pre-allocation of shared memory ...

2003-06-12 Thread Bruce Momjian
I have added the following sentence to the docs too: Note, you will need enough swap space to cover all your memory needs. I still wish Linux would just fail the fork/malloc when memory is low, rather than requiring swap for everything _or_ overcommitting. I wonder if making a

Re: [HACKERS] Pre-allocation of shared memory ...

2003-06-12 Thread Bruce Momjian
OK, new text is: para Linux has poor default memory overcommit behavior. Rather than failing if it can not reserve enough memory, it returns success, but later fails when the memory can't be mapped and terminates the application with literalkill -9/. To

Re: [HACKERS] Alter strings that don't belong to the application

2003-06-12 Thread Tom Lane
=?ISO-8859-1?Q?Dennis_Bj=F6rklund?= [EMAIL PROTECTED] writes: On Thu, 12 Jun 2003, Tom Lane wrote: I'd call that a bug in psql. Where do you see it happening exactly? It's the utf-8 validation function (mbvalidate) that removes characters that it does not understand. I think the string

Re: [HACKERS] Alter strings that don't belong to the application

2003-06-12 Thread Dennis Björklund
On Thu, 12 Jun 2003, Tom Lane wrote: I think the string ought to be copied first ... although that might create memory-leak problems. Could you take a look at fixing this, while you're in the area? Sure. I can whitelist you if you have a stable IP address --- is that a static or dynamic

Re: [HACKERS] Pre-allocation of shared memory ...

2003-06-12 Thread Andrew Dunstan
A couple of points: . It is probably a good idea to put do this via /etc/sysctl.conf, which will be called earlyish by init scripts (on RH9 it is in the network startup file, for some reason). . The setting is not available on all kernel versions AFAIK. The admin needs to check the docs. I have

Re: [HACKERS] Pre-allocation of shared memory ...

2003-06-12 Thread Bruce Momjian
Well, let's see what feedback we get. --- Andrew Dunstan wrote: A couple of points: . It is probably a good idea to put do this via /etc/sysctl.conf, which will be called earlyish by init scripts (on RH9 it is in the

[HACKERS] pg_conf idea (was Re: [GENERAL] Postgres performance comments from a MySQL user)

2003-06-12 Thread Steve Crawford
On Wednesday 11 June 2003 2:37 pm, Matthew Nuzum wrote: The problem with this is that in troubleshooting there's no frame of reference. Having a stock config file, or stock config file options allows a person to write to the list and say, hey, I'm using medium.conf and I have x ram... The

Re: [HACKERS] CVS -Tip compile issue -- FreeBSD 4.8

2003-06-12 Thread Sean Chittenden
It would seem the configure test isn't picking up on the structure. checking sys/socket.h usability... yes checking sys/socket.h presence... yes checking for sys/socket.h... yes --snip-- checking for struct sockaddr_storage... no Hrm on 5.1-CURRENT (~3 days old) it works: checking

Re: [HACKERS] CVS -Tip compile issue -- FreeBSD 4.8

2003-06-12 Thread Tom Lane
Sean Chittenden [EMAIL PROTECTED] writes: checking for struct sockaddr_storage... no Hrm on 5.1-CURRENT (~3 days old) it works: When did you last update from our CVS? I corrected the configure test a couple hours ago ... regards, tom lane

Re: [HACKERS] CVS -Tip compile issue -- FreeBSD 4.8

2003-06-12 Thread Sean Chittenden
checking for struct sockaddr_storage... no Hrm on 5.1-CURRENT (~3 days old) it works: When did you last update from our CVS? I corrected the configure test a couple hours ago ... Oh err, umm 'bout 10minutes ago I Sup'ed and checked. *wanders off to go read -committers*

[HACKERS] sql_ascii

2003-06-12 Thread Dennis Björklund
Why do we have SQL_ASCII? I could understand it if we only could store 7-bit strings there. But SQL_ASCII lets you store 8-bit values. Should I understand SQL_ASCII simply as 8-bit strings of unknown charset? In the first database I created I used it to store latin1 strings, and that was a

Re: [HACKERS] Pre-allocation of shared memory ...

2003-06-12 Thread Greg Stark
Tom Lane [EMAIL PROTECTED] writes: The policy they're calling paranoid overcommit (don't allocate more virtual memory than you have swap) is as far as I know the standard on all Unixen other than Linux; certainly it's the traditional behavior. Uhm, it's traditional for Unixen without

Re: [HACKERS] CVS -Tip compile issue -- FreeBSD 4.8

2003-06-12 Thread Sean Chittenden
checking for struct sockaddr_storage... no Hrm on 5.1-CURRENT (~3 days old) it works: When did you last update from our CVS? I corrected the configure test a couple hours ago ... Oh err, umm 'bout 10minutes ago I Sup'ed and checked. *wanders off to go read

Re: [HACKERS] SELECT blocking on ALTER TABLE ADD FOREIGN KEY

2003-06-12 Thread Jim C. Nasby
On Wed, Jun 11, 2003 at 03:19:14PM -0400, Tom Lane wrote: Jim C. Nasby [EMAIL PROTECTED] writes: Is it really necessary to block reads on a table that is affected by adding a foreign key constraint? It's trickier than you seem to think. The command is adding an index, which at some point

Re: [HACKERS] CVS -Tip compile issue -- FreeBSD 4.8

2003-06-12 Thread Tom Lane
Sean Chittenden [EMAIL PROTECTED] writes: Tom, you said you needed a shell way of detecting this, does the following work? No, I need something that will work in the regression test resultmap, which basically only knows about the platform identifier string computed by config.guess. We could

Re: [HACKERS] CVS -Tip compile issue -- FreeBSD 4.8

2003-06-12 Thread Sean Chittenden
Tom, you said you needed a shell way of detecting this, does the following work? No, I need something that will work in the regression test resultmap, which basically only knows about the platform identifier string computed by config.guess. We could change

Re: [HACKERS] Pre-allocation of shared memory ...

2003-06-12 Thread Tom Lane
Greg Stark [EMAIL PROTECTED] writes: I think you'll find this overcommit issue affects many if not most Unixen. I'm unconvinced, because I've only ever heard of the problem affecting Postgres on Linux. regards, tom lane ---(end of

Re: [HACKERS] SELECT blocking on ALTER TABLE ADD FOREIGN KEY

2003-06-12 Thread Tom Lane
Jim C. Nasby [EMAIL PROTECTED] writes: Is there any ALTER that would require blocking selects? DROP INDEX, for certain. Even stuff like drop and rename should be protected by versioning, no? No. System-catalog changes are always READ COMMITTED mode. regards, tom

Re: [HACKERS] Pre-allocation of shared memory ...

2003-06-12 Thread Bruce Momjian
Tom Lane wrote: Greg Stark [EMAIL PROTECTED] writes: I think you'll find this overcommit issue affects many if not most Unixen. I'm unconvinced, because I've only ever heard of the problem affecting Postgres on Linux. What I don't understand is why they just don't start failing on

Re: [HACKERS] Pre-allocation of shared memory ...

2003-06-12 Thread Jeroen T. Vermeulen
On Thu, Jun 12, 2003 at 08:08:28PM -0400, Bruce Momjian wrote: I'm unconvinced, because I've only ever heard of the problem affecting Postgres on Linux. What I don't understand is why they just don't start failing on fork/malloc rather than killing things. I may be way off the mark

Re: [HACKERS] [BUGS] PostgreSQL client has problems when libbind is installed

2003-06-12 Thread Tom Lane
Yves R. Crevecoeur [EMAIL PROTECTED] writes: Don't break BeOS support. A new version of BeOS will be released very soon. Well, the BeOS port is already broken, and has been for awhile, because no one's bothered to step up and maintain it. Are you volunteering? It needs work on semaphore

Re: [HACKERS] Pre-allocation of shared memory ...

2003-06-12 Thread Andrew Dunstan
I'm not saying you're wrong, but I also think it's true that typical Linux usage patterns are rather different from those of other *nixen. Linux started out being able to do a lot with a little, and is still often used that way - with more functions crammed into boxes with less resources. When I

Re: [HACKERS] Pre-allocation of shared memory ...

2003-06-12 Thread Tom Lane
Jeroen T. Vermeulen [EMAIL PROTECTED] writes: Given the right allocation proportions, this may mean that in the end the kernel has no way to handle a shortage gracefully by causing fork() or allocations to fail. Sure it does. All you need is a conservative allocation policy: fork() fails if

Re: [HACKERS] Pre-allocation of shared memory ...

2003-06-12 Thread Alvaro Herrera
On Thu, Jun 12, 2003 at 09:18:33PM -0400, Tom Lane wrote: Given that swap space is cheap, and that killing random processes is obviously bad, it's not apparent to me why people think this is not a good approach --- at least for high-reliability servers. And Linux would definitely like to

Re: [HACKERS] CVS -Tip compile issue -- FreeBSD 4.8

2003-06-12 Thread Rod Taylor
We could change float8/i.86-.*-freebsd=float8-small-is-zero to float8/i.86-.*-freebsd4=float8-small-is-zero This change compiles / regresses fine for me, but I didn't read the whole thread to try to see what to look for. -- Rod Taylor [EMAIL PROTECTED] PGP Key:

Re: [HACKERS] Pre-allocation of shared memory ...

2003-06-12 Thread Bruce Momjian
Tom Lane wrote: Jeroen T. Vermeulen [EMAIL PROTECTED] writes: Given the right allocation proportions, this may mean that in the end the kernel has no way to handle a shortage gracefully by causing fork() or allocations to fail. Sure it does. All you need is a conservative allocation

Re: [HACKERS] CVS -Tip compile issue -- FreeBSD 4.8

2003-06-12 Thread Sean Chittenden
We could change float8/i.86-.*-freebsd=float8-small-is-zero to float8/i.86-.*-freebsd4=float8-small-is-zero This change compiles / regresses fine for me, but I didn't read the whole thread to try to see what to look for. FreeBSD 5.1 imported gdtoa which fixed the handling of

Re: [HACKERS] CVS -Tip compile issue -- FreeBSD 4.8

2003-06-12 Thread Tom Lane
Rod Taylor [EMAIL PROTECTED] writes: We could change float8/i.86-.*-freebsd=3Dfloat8-small-is-zero to float8/i.86-.*-freebsd4=3Dfloat8-small-is-zero This change compiles / regresses fine for me, but I didn't read the whole thread to try to see what to look for. Good enough --- change

Re: [HACKERS] Pre-allocation of shared memory ...

2003-06-12 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes: You have to love that swap + 1/2 ram option --- when you need four possible options, there is something wrong with your approach. :-) I'm still wondering what the no overcommit handling option does, exactly. regards, tom lane

Re: [HACKERS] CVS -Tip compile issue -- FreeBSD 4.8

2003-06-12 Thread Sean Chittenden
We could change float8/i.86-.*-freebsd=3Dfloat8-small-is-zero to float8/i.86-.*-freebsd4=3Dfloat8-small-is-zero This change compiles / regresses fine for me, but I didn't read the whole thread to try to see what to look for. Good enough --- change committed. Does anyone know if

Re: [HACKERS] Pre-allocation of shared memory ...

2003-06-12 Thread Greg Stark
Alvaro Herrera [EMAIL PROTECTED] writes: On Thu, Jun 12, 2003 at 09:18:33PM -0400, Tom Lane wrote: Given that swap space is cheap, and that killing random processes is obviously bad, it's not apparent to me why people think this is not a good approach --- at least for high-reliability

Re: [HACKERS] Pre-allocation of shared memory ...

2003-06-12 Thread Bruce Momjian
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: You have to love that swap + 1/2 ram option --- when you need four possible options, there is something wrong with your approach. :-) I'm still wondering what the no overcommit handling option does, exactly. I assume it does no

Re: [HACKERS] Pre-allocation of shared memory ...

2003-06-12 Thread Bruce Momjian
Greg Stark wrote: I suspect this was less of an issue in the days before copy on write because vfork was more widely used/implemented. I'm not sure linux even implements vfork other than just as a wrapper around fork. Even BSD ditched it a while back though I think I saw that NetBSD

Re: [HACKERS] CVS -Tip compile issue -- FreeBSD 4.8

2003-06-12 Thread Tom Lane
Sean Chittenden [EMAIL PROTECTED] writes: Does anyone know if FreeBSD 3.* or before still exist in the wild? We might have to tweak the pattern to match those too. Ehh it probably does. I get emails once every 2-3 mo from someone running it on a 2.x box and that code's probably 6-8

Re: [HACKERS] Pre-allocation of shared memory ...

2003-06-12 Thread Greg Stark
Bruce Momjian [EMAIL PROTECTED] writes: I see no reason RAM can't be used as backing store for possible copy-on-write use. Depends on the scenario. For a database like postgres it would work fairly well since that RAM is still available for filesystem buffers. For Oracle it would suck because

Re: [HACKERS] Pre-allocation of shared memory ...

2003-06-12 Thread Ron Mayer
Jeroen T. Vermeulen wrote: After that, where do you go? Try to find a reasonable process to shoot in the head. From what I heard, although I haven't kept current, a lot of work went into selecting a reasonable process, so there will be some determinism. FWIW, you can browse the logic linux

Re: [HACKERS] Pre-allocation of shared memory ...

2003-06-12 Thread Alvaro Herrera
On Thu, Jun 12, 2003 at 07:22:14PM -0700, Ron Mayer wrote: FWIW, you can browse the logic linux uses to choose which process to kill here: http://lxr.linux.no/source/mm/oom_kill.c Hey, this LXR thing is cool. It'd be nice to have one of those for Postgres. -- Alvaro Herrera

Re: [HACKERS] [BUGS] PostgreSQL client has problems when libbind is installed

2003-06-12 Thread Tom Lane
Palle Girgensohn [EMAIL PROTECTED] writes: [ linking libbind causes some obscure problems ] http://www.freebsd.org/cgi/query-pr.cgi?pr=ports/47218 Given that we're not supporting BeOS at the moment anyway, I wonder whether we need libbind on any platform. I know linking it causes some minor