[HACKERS] questionable item in HISTORY

2005-09-24 Thread Tatsuo Ishii
Following item in HISTORY: * Add support for 3 and 4-byte UTF8 characters (John Hansen) Previously only one and two-byte UTF8 characters were supported. This is particularly important for support for some Chinese characters. is wrong since 3-byte UTF-8 characters are

Re: [HACKERS] stack depth limit exceeded problem.

2005-09-24 Thread Thomas Hallgren
Oliver Jowett wrote: Thomas Hallgren wrote: PL/Java runs a JVM. Since a JVM is multi threaded, PL/Java goes to fairly extreme measures to ensure that only one thread at a time can access the backend. So far, this have worked well but there is one small problem. [...] I assume this

Re: [HACKERS] stack depth limit exceeded problem.

2005-09-24 Thread Martijn van Oosterhout
On Sat, Sep 24, 2005 at 10:34:42AM +0200, Thomas Hallgren wrote: Oliver Jowett wrote: I assume this means you have a single lock serializing requests to the backend? Yes, of course. I also make sure that the main thread cannot return until another thread that is servicing a backend

Re: [HACKERS] stack depth limit exceeded problem.

2005-09-24 Thread Thomas Hallgren
Martijn van Oosterhout wrote: On Sat, Sep 24, 2005 at 10:34:42AM +0200, Thomas Hallgren wrote: Oliver Jowett wrote: I assume this means you have a single lock serializing requests to the backend? Yes, of course. I also make sure that the main thread cannot return until

Re: [HACKERS] stack depth limit exceeded problem.

2005-09-24 Thread Martijn van Oosterhout
On Sat, Sep 24, 2005 at 12:26:58PM +0200, Thomas Hallgren wrote: Yes. All backend exceptions are cought in a PG_CATCH and then propagated to Java as a ServerException. If there's no catch in the Java code, they are rethrown by the java_call_handler. This time with jump buffer that was setup

Re: [HACKERS] stack depth limit exceeded problem.

2005-09-24 Thread Thomas Hallgren
Martijn van Oosterhout wrote: Linux has sigaltstack so you can catch the stack overflow signal (and other signals obviously, but that's its main use), but it's not terribly portable. I rely on the signal handler that the JVM uses for page-faults (which a stack overflow generally amounts to)

Re: [HACKERS] stack depth limit exceeded problem.

2005-09-24 Thread Martijn van Oosterhout
On Sat, Sep 24, 2005 at 02:38:35PM +0200, Thomas Hallgren wrote: Martijn van Oosterhout wrote: Linux has sigaltstack so you can catch the stack overflow signal (and other signals obviously, but that's its main use), but it's not terribly portable. I rely on the signal handler that the

Re: [HACKERS] 64-bit API for large objects

2005-09-24 Thread Tom Lane
Jeremy Drake [EMAIL PROTECTED] writes: On Fri, 23 Sep 2005, Tom Lane wrote: postgresql-fe.h defines a ton of stuff that has no business being visible to libpq's client applications. It's designed to be used by our *own* client-side code (psql and the like), but we have not made any attempt

Re: [HACKERS] 64-bit API for large objects

2005-09-24 Thread Tom Lane
Jeremy Drake [EMAIL PROTECTED] writes: In any case, are there any comments on the changes below libpq (the functions visible to queries on down)? Within the backend, I don't see the point in maintaining a distinction between 32- and 64-bit APIs for inv_api.c: you should just switch 'em to use

Re: [HACKERS] 64-bit API for large objects

2005-09-24 Thread Jeremy Drake
On Fri, 23 Sep 2005, Tom Lane wrote: Jeremy Drake [EMAIL PROTECTED] writes: 2) The lo_*64, in order to be convenient from the client end, have functions added to libpq as the existing lo_* functions. The client side of libpq did not previously know anything about int64 or how to

Re: [HACKERS] [PERFORM] Releasing memory during External sorting?

2005-09-24 Thread Ron Peacetree
From: Dann Corbit [EMAIL PROTECTED] Sent: Sep 23, 2005 5:38 PM Subject: RE: [HACKERS] [PERFORM] Releasing memory during External sorting? _C Unleashed_ also explains how to use a callback function to perform arbitrary radix sorts (you simply need a method that returns the [bucketsize] most

Re: [HACKERS] 64-bit API for large objects

2005-09-24 Thread Tom Lane
David Fetter [EMAIL PROTECTED] writes: On Fri, Sep 23, 2005 at 05:40:09PM -0400, Tom Lane wrote: For that matter, we can't even guarantee that they work at all: not all platforms even *have* int64 types. What platforms that PG supports don't have int64 arithmetic? We claim to build with any

Re: [HACKERS] 64-bit API for large objects

2005-09-24 Thread Alvaro Herrera
Hey, While you guys are hacking at the LO code, it would be nice to consider the suggestions outlined here: http://archives.postgresql.org/pgsql-bugs/2004-07/msg00143.php -- Alvaro Herrera http://www.amazon.com/gp/registry/CTMLCN8V17R4 Y una voz del caos me habló y me dijo

Re: [HACKERS] Proposed patch to clean up signed-ness warnings

2005-09-24 Thread Tom Lane
Tatsuo Ishii [EMAIL PROTECTED] writes: Are you proposing that we change all the char * to unsigned char *? No, I suggest we change all char * to unsigned char * only where it points a string which could hold non ASCII character strings. Which is pretty nearly all of them... To support

Re: [HACKERS] 2 forks for md5?

2005-09-24 Thread Andrew Dunstan
Bruce Momjian wrote: Tom Lane wrote: I'm coming to agree with Andrew that a documentation patch might be the best answer. But where to put it ... under the description of the log_connections GUC var? I am thinking we should wait for someone else to notice the double log entries

Re: [HACKERS] 64-bit API for large objects

2005-09-24 Thread Jeremy Drake
On Sat, 24 Sep 2005, Tom Lane wrote: Jeremy Drake [EMAIL PROTECTED] writes: In any case, are there any comments on the changes below libpq (the functions visible to queries on down)? Within the backend, I don't see the point in maintaining a distinction between 32- and 64-bit APIs for

Re: [HACKERS] 64-bit API for large objects

2005-09-24 Thread Jeremy Drake
On Sat, 24 Sep 2005, Alvaro Herrera wrote: Hey, While you guys are hacking at the LO code, it would be nice to consider the suggestions outlined here: http://archives.postgresql.org/pgsql-bugs/2004-07/msg00143.php Included from that message for easier reference: 0) In Oid lo_creat(PGconn

[HACKERS] gcc4's uninitialized-variable warnings

2005-09-24 Thread Tom Lane
I asked some gcc experts at Red Hat about the new variable-may-be-used- uninitialized warnings that gcc 4.x reports. These occur in cases like int i, j; ... foo(i, j); // use i and j I had thought that gcc was being stricter about the possibility that the

Re: [HACKERS] 64-bit API for large objects

2005-09-24 Thread Tom Lane
Jeremy Drake [EMAIL PROTECTED] writes: 0) In Oid lo_creat(PGconn *conn, int mode), why is there a mode on lo_create? The mode is determined when the object is lo_open()ed, right? I think the docs basically said it is a vestigial feature, it used to be useful but the code evolved in such a way

[HACKERS] \x output blowing up

2005-09-24 Thread Martijn van Oosterhout
Hi, On CVS tip, whenever I do \x output, it dies with an internal error in glibc on free. If you run it under valgrind, it complains about these lines of code: 700{ 701char *my_cell = pg_local_malloc(cell_w[i] + 1); 702 703 [Inv write 1 byte]

Re: [HACKERS] Start translating

2005-09-24 Thread Peter Eisentraut
Tom Lane wrote: One thing I had wanted to do before issuing the call for translations was to make a pass over the recent GiST changes, looking at elog-vs-ereport decisions and message wording. OK, can you tell me where those are and I can look at it. -- Peter Eisentraut

Re: [HACKERS] \x output blowing up

2005-09-24 Thread Martijn van Oosterhout
On Sat, Sep 24, 2005 at 11:45:08PM +0200, Martijn van Oosterhout wrote: Hi, On CVS tip, whenever I do \x output, it dies with an internal error in glibc on free. If you run it under valgrind, it complains about these lines of code: snip Ok, I worked out the direct cause, pg_wcswidth only

Re: [HACKERS] Start translating

2005-09-24 Thread Tom Lane
Peter Eisentraut [EMAIL PROTECTED] writes: Tom Lane wrote: One thing I had wanted to do before issuing the call for translations was to make a pass over the recent GiST changes, looking at elog-vs-ereport decisions and message wording. OK, can you tell me where those are and I can look at

Re: [HACKERS] questionable item in HISTORY

2005-09-24 Thread Bruce Momjian
Tatsuo Ishii wrote: Following item in HISTORY: * Add support for 3 and 4-byte UTF8 characters (John Hansen) Previously only one and two-byte UTF8 characters were supported. This is particularly important for support for some Chinese characters. is wrong since

[HACKERS] Discarding relations from FSM

2005-09-24 Thread Jim C. Nasby
Per http://lnk.nu/developer.postgresql.org/43b.c: * The number of distinct relations tracked is limited by a configuration * variable (MaxFSMRelations). When this would be exceeded, we discard the * least recently used relation. A doubly-linked list with move-to-front * behavior keeps track

Re: [HACKERS] 2 forks for md5?

2005-09-24 Thread Bruce Momjian
Andrew Dunstan wrote: Bruce Momjian wrote: Tom Lane wrote: I'm coming to agree with Andrew that a documentation patch might be the best answer. But where to put it ... under the description of the log_connections GUC var? I am thinking we should wait for someone else

Re: [HACKERS] gcc4's uninitialized-variable warnings

2005-09-24 Thread Bruce Momjian
Tom Lane wrote: I asked some gcc experts at Red Hat about the new variable-may-be-used- uninitialized warnings that gcc 4.x reports. These occur in cases like int i, j; ... foo(i, j); // use i and j I had thought that gcc was being stricter about the

Re: [HACKERS] Discarding relations from FSM

2005-09-24 Thread Bruce Momjian
Jim C. Nasby wrote: Per http://lnk.nu/developer.postgresql.org/43b.c: * The number of distinct relations tracked is limited by a configuration * variable (MaxFSMRelations). When this would be exceeded, we discard the * least recently used relation. A doubly-linked list with

Re: [HACKERS] \x output blowing up

2005-09-24 Thread Bruce Momjian
Well, it seems we are going to have to fix it somehow for 8.1. It is not crashing here so I can't work up a patch. Can you submit a minimal fix for 8.1? Thanks. --- Martijn van Oosterhout wrote: -- Start of PGP signed

Re: [HACKERS] Discarding relations from FSM

2005-09-24 Thread Tom Lane
Jim C. Nasby [EMAIL PROTECTED] writes: Rather than keeping track of which relation was last used (presumably everytime something hits the FSM), wouldn't it make more sense to just drop the relation with the smallest amount of free space in the map? Why? That certainly wouldn't be cheaper to

[HACKERS] \d on database with a lot of tables is slow

2005-09-24 Thread Jim C. Nasby
I have a client with a database that contains 4000 relations according to vacuum verbose, and \d in psql is painfully slow. In particular... - Seq Scan on pg_class c (cost=0.00..2343.09 rows=6124 width=73) (actual time=0.325..22100.840 rows=16856 loops=1) Filter: (((relkind =

[HACKERS] Vacuum questions...

2005-09-24 Thread Jim C. Nasby
Would it be difficult to vacuum as part of a dump? The reasoning behind this is that you have to read the table to do the dump anyway, so it would be a good time to be able to piggy-back other operations that need to read the entire table on top. I know vacuuming of indexes complicates this, so

Re: [HACKERS] \d on database with a lot of tables is slow

2005-09-24 Thread Rod Taylor
On Sat, 2005-09-24 at 18:59 -0500, Jim C. Nasby wrote: I have a client with a database that contains 4000 relations according to vacuum verbose, and \d in psql is painfully slow. In particular... - Seq Scan on pg_class c (cost=0.00..2343.09 rows=6124 width=73) (actual

Re: [HACKERS] Discarding relations from FSM

2005-09-24 Thread Jim C. Nasby
On Sat, Sep 24, 2005 at 07:21:19PM -0400, Tom Lane wrote: Of course maybe a better question is why we even limit based on the number of relations... Shared memory is fixed-size. True, but can't the fixed memory required per-relation just be shared with the fixed memory used to store free

Re: [HACKERS] \d on database with a lot of tables is slow

2005-09-24 Thread Jim C. Nasby
On Sat, Sep 24, 2005 at 08:20:47PM -0400, Rod Taylor wrote: On Sat, 2005-09-24 at 18:59 -0500, Jim C. Nasby wrote: I have a client with a database that contains 4000 relations according to vacuum verbose, and \d in psql is painfully slow. In particular... - Seq Scan on pg_class c

Re: [HACKERS] Discarding relations from FSM

2005-09-24 Thread Bruce Momjian
Jim C. Nasby wrote: On Sat, Sep 24, 2005 at 07:21:19PM -0400, Tom Lane wrote: Of course maybe a better question is why we even limit based on the number of relations... Shared memory is fixed-size. True, but can't the fixed memory required per-relation just be shared with the fixed

Re: [HACKERS] Vacuum questions...

2005-09-24 Thread Joshua D. Drake
Jim C. Nasby wrote: Would it be difficult to vacuum as part of a dump? The reasoning behind this is that you have to read the table to do the dump anyway, so it would be a good time to be able to piggy-back other operations that need to read the entire table on top. I know vacuuming of indexes

Re: [HACKERS] Vacuum questions...

2005-09-24 Thread Alvaro Herrera
On Sat, Sep 24, 2005 at 07:17:38PM -0500, Jim C. Nasby wrote: Finally, if vacuum_delay is enabled, does vacuum_cost_page_miss consider a miss as not in the database buffer, or not in the kernel buffer? The database buffer. I remember discussions about trying to track IO request times to try

Re: [HACKERS] Releasing memory during External sorting?

2005-09-24 Thread Dann Corbit
Generally, when you read from a set of subfiles, the OS will cache the reads to some degree, so the disk-seek jitter is not always that bad. On a highly fragmented disk drive, you might also jump all over the place reading serially from a single subfile. Of course, every situation is