Re: [HACKERS] UTF8 or Unicode

2005-02-18 Thread Karel Zak
On Tue, 2005-02-15 at 14:33 +0100, Peter Eisentraut wrote: Am Dienstag, 15. Februar 2005 10:22 schrieb Karel Zak: in PG: unicode = utf8 = utf-8 Our internal routines in src/backend/utils/mb/encnames.c accept all synonyms. The official internal PG name for UTF-8 is UNICODE :-( I think

Re: [HACKERS] Help me recovering data

2005-02-18 Thread Jürgen Cappel
Just wondering after this discussion: Is transaction wraparound limited to a database or to an installation ? i.e. can heavy traffic in one db affect another db in the same installation ? ---(end of broadcast)--- TIP 8: explain analyze is your friend

Re: [HACKERS] win32 performance - fsync question

2005-02-18 Thread Neil Conway
Tom Lane wrote: Portability, or rather the complete lack of it. Stuff that isn't in the Single Unix Spec is a hard sell. O_DIRECT is reasonably common among modern Unixen (it is supported by Linux, FreeBSD, and probably a couple of the commercial variants like AIX or IRIX); it should also be

Re: [HACKERS] Help me recovering data

2005-02-18 Thread Russell Smith
On Fri, 18 Feb 2005 04:38 pm, Kevin Brown wrote: Tom Lane wrote: Gaetano Mendola [EMAIL PROTECTED] writes: BTW, why not do an automatic vacuum instead of shutdown ? At least the DB do not stop working untill someone study what the problem is and how solve it. No, the entire point

Re: [HACKERS] Help me recovering data

2005-02-18 Thread Russell Smith
On Fri, 18 Feb 2005 08:53 pm, Jürgen Cappel wrote: Just wondering after this discussion: Is transaction wraparound limited to a database or to an installation ? i.e. can heavy traffic in one db affect another db in the same installation ? XID's are global to the pg cluster, or installation.

Re: [HACKERS] UTF8 or Unicode

2005-02-18 Thread Oliver Jowett
Karel Zak wrote: Yes, I think we should fix it and remove UNICODE and WIN encoding names from PG code. The JDBC driver asks for a UNICODE client encoding before it knows the server version it is talking to. How do you avoid breaking this? -O ---(end of

Re: [HACKERS] UTF8 or Unicode

2005-02-18 Thread Karel Zak
On Sat, 2005-02-19 at 00:27 +1300, Oliver Jowett wrote: Karel Zak wrote: Yes, I think we should fix it and remove UNICODE and WIN encoding names from PG code. The JDBC driver asks for a UNICODE client encoding before it knows the server version it is talking to. How do you avoid

Re: [HACKERS] UTF8 or Unicode

2005-02-18 Thread Christopher Kings-Lynne
Add to 8.1 release notes: encoding names 'UNICODE' and 'WIN' are deprecated and it will removed in next release. Please, use correct names UTF-8 and WIN1215. 8.2: remove it. OK? Why on earth remove it? Just leave it in as an alias to UTF8 Chris ---(end of

Re: [HACKERS] win32 performance - fsync question

2005-02-18 Thread Zeugswetter Andreas DAZ SD
One point that I no longer recall the reasoning behind is that xlog.c doesn't think O_SYNC is a preferable default over fsync. For larger (8k) transactions O_SYNC|O_DIRECT is only good with the recent pending patch to group WAL writes together. The fsync method gives the OS a chance to do

Re: [HACKERS] UTF8 or Unicode

2005-02-18 Thread Oliver Jowett
Karel Zak wrote: On Sat, 2005-02-19 at 00:27 +1300, Oliver Jowett wrote: Karel Zak wrote: Yes, I think we should fix it and remove UNICODE and WIN encoding names from PG code. The JDBC driver asks for a UNICODE client encoding before it knows the server version it is talking to. How do you avoid

Re: [HACKERS] UTF8 or Unicode

2005-02-18 Thread Dave Page
-Original Message- From: [EMAIL PROTECTED] on behalf of Oliver Jowett Sent: Fri 2/18/2005 11:27 AM To: Karel Zak Cc: List pgsql-hackers Subject: Re: [HACKERS] UTF8 or Unicode Karel Zak wrote: Yes, I think we should fix it and remove UNICODE and WIN encoding names from PG code.

[HACKERS] sigint psql

2005-02-18 Thread John DeSoi
I'm communicating with psql via a pipe stream. This works pretty well, but one problem I have is trying to cancel an operation. If I send a sigint, psql dies. In looking at the source I gather this is because it assumes I'm in non-interactive mode (pset.notty is true). I was wondering if there

[HACKERS] SPI_finish and RegisterExprContextCallback

2005-02-18 Thread Thomas Hallgren
I'd like to write a C-function that returns a SETOF a complex type. This set is obtained from a query performed using an SPI cursor. I don't want to build the complete set in memory so I tried the following. 1. During the SRF_IS_FIRSTCALL phase, I do an SPI_connect and I create the cursor

Re: [HACKERS] SPI_finish and RegisterExprContextCallback

2005-02-18 Thread Tom Lane
Thomas Hallgren [EMAIL PROTECTED] writes: The callback is called OK but when it calls SPI_finish I get an illegal memory access signal. From where? Minimum respect for the time of your fellow hackers would suggest including a gdb traceback in questions like this.

Re: [HACKERS] Help me recovering data

2005-02-18 Thread Robert Treat
On Thursday 17 February 2005 07:47, [EMAIL PROTECTED] wrote: Gaetano Mendola [EMAIL PROTECTED] writes: We do ~4000 txn/minute so in 6 month you are screewd up... Sure, but if you ran without vacuuming for 6 months, wouldn't you notice the huge slowdowns from all those dead tuples

Re: [HACKERS] win32 performance - fsync question

2005-02-18 Thread Merlin Moncure
Magnus prepared a trivial patch which added the O_SYNC flag for windows and mapped it to FILE_FLAG_WRITE_THROUGH in win32_open.c. We pg_benched it and here are the results of our test on my WinXP workstation on a 10k raptor: Settings were pgbench -t 100 -c 10. fsync = off: ~ 280 tps fsync on,

Re: [HACKERS] Help me recovering data

2005-02-18 Thread Matthew T. O'Connor
Russell Smith wrote: On Fri, 18 Feb 2005 04:38 pm, Kevin Brown wrote: Tom Lane wrote: No, the entire point of this discussion is to whup the DBA upside the head with a big enough cluestick to get him to install autovacuum. Once autovacuum is default, it won't matter anymore. I have

[HACKERS] Data loss, vacuum, transaction wrap-around

2005-02-18 Thread pgsql
I want to see if there is a concensus of opinion out there. We've all known that data loss could happen if vacuum is not run and you perform more than 2b transactions. These days with faster and bigger computers and disks, it more likely that this problem can be hit in months -- not years. To

Re: [HACKERS] SPI_finish and RegisterExprContextCallback

2005-02-18 Thread Thomas Hallgren
Tom Lane wrote: From where? Minimum respect for the time of your fellow hackers would suggest including a gdb traceback in questions like this. My apologies. I'll do that next time. I was on a win32 system and the gdb that comes with the MinGW environment just doesn't do it for me (if anyone

Re: [HACKERS] win32 performance - fsync question

2005-02-18 Thread Magnus Hagander
Magnus prepared a trivial patch which added the O_SYNC flag for windows and mapped it to FILE_FLAG_WRITE_THROUGH in win32_open.c. Attached is this trivial patch. As Merlin says, it needs some more reliability testing. But the numbers are at least reasonable - it *seems* like it's doing the

Re: [HACKERS] sigint psql

2005-02-18 Thread Doug McNaught
John DeSoi [EMAIL PROTECTED] writes: I'm communicating with psql via a pipe stream. This works pretty well, but one problem I have is trying to cancel an operation. If I send a sigint, psql dies. In looking at the source I gather this is because it assumes I'm in non-interactive mode

Re: [HACKERS] SPI_finish and RegisterExprContextCallback

2005-02-18 Thread Tom Lane
Thomas Hallgren [EMAIL PROTECTED] writes: 1. I call a function that does an SPI_connect, SPI_prepare, SPI_cursor_open, and finally attempts to do an SPI_cursor_fetch. 2. Since the SQL statement I'm executing contains a call to function returning SETOF, and since that function in turn

Re: [HACKERS] SPI_finish and RegisterExprContextCallback

2005-02-18 Thread Thomas Hallgren
Tom, You're right, you can't just return from that inner function while leaving its SPI connection open. It might be interesting to redesign SPI around the notion of independent connection objects rather than necessarily having a stack of 'em. I made the same reflection looking at the SPI code.

Re: [HACKERS] Data loss, vacuum, transaction wrap-around

2005-02-18 Thread Tom Lane
[EMAIL PROTECTED] writes: In fact, I think it is so bad, that I think we need to back-port a fix to previous versions and issue a notice of some kind. They already do issue notices --- see VACUUM. A real fix (eg the forcible stop we were talking about earlier) will not be reasonable to

Re: [HACKERS] Data loss, vacuum, transaction wrap-around

2005-02-18 Thread Matthew T. O'Connor
Tom Lane wrote: [EMAIL PROTECTED] writes: In fact, I think it is so bad, that I think we need to back-port a fix to previous versions and issue a notice of some kind. They already do issue notices --- see VACUUM. A real fix (eg the forcible stop we were talking about earlier) will not be

Re: [HACKERS] Data loss, vacuum, transaction wrap-around

2005-02-18 Thread Tom Lane
Matthew T. O'Connor matthew@zeut.net writes: I hope this question isn't too stupid Is it be possible to create a vacuum wraparound or vacuum xidreset command which would do the work required to fix the wraparound problem, without being as expensive as a normal vacuum of an entire

Re: [HACKERS] Data loss, vacuum, transaction wrap-around

2005-02-18 Thread pgsql
More suggestions: (1) At startup, postmaster checks for an XID, if it is close to a problem, force a vacuum. (2) At sig term shutdown, can the postmaster start a vacuum? (3) When the XID count goes past the trip wire can it spontaneously issue a vacuum? NOTE: Suggestions 1 and 2 are for 8.0

Re: [HACKERS] Data loss, vacuum, transaction wrap-around

2005-02-18 Thread Tom Lane
Joshua D. Drake [EMAIL PROTECTED] writes: Tom Lane wrote: A real fix (eg the forcible stop we were talking about earlier) will not be reasonable to back-port. Would at least a automated warning mechanism be a reasonable backport? No, because the hard part of the problem actually is detecting

Re: [HACKERS] Data loss, vacuum, transaction wrap-around

2005-02-18 Thread Tom Lane
[EMAIL PROTECTED] writes: More suggestions: (1) At startup, postmaster checks for an XID, if it is close to a problem, force a vacuum. Useless to a system that's run 24x7; also presumes the existence of a complete solution anyway (since getting the postmaster to find that out is the hard

Re: [HACKERS] Data loss, vacuum, transaction wrap-around

2005-02-18 Thread Joshua D. Drake
There is another issue here, which is that I have no faith that the people who actually need this are going to be clueful enough to update to 7.4.8 or 7.3.10 or whatever they'd need... Well I can't argue with that one ;) regards, tom lane -- Command Prompt, Inc., your source for PostgreSQL

Re: [HACKERS] Data loss, vacuum, transaction wrap-around

2005-02-18 Thread Greg Stark
Tom Lane [EMAIL PROTECTED] writes: (3) When the XID count goes past the trip wire can it spontaneously issue a vacuum? Only in the database you're connected to, which very likely isn't where the problem is. Moreover, having N backends all decide they need to do this at once doesn't

[HACKERS] Get rid of system attributes in pg_attribute?

2005-02-18 Thread Tom Lane
I'm wondering how useful it is to store explicit representations of the system attributes in pg_attribute. We could very easily hard-wire those things instead, which would make for a large reduction in the number of entries in pg_attribute. (In the current regression database nearly half of the

Re: [HACKERS] Data loss, vacuum, transaction wrap-around

2005-02-18 Thread Tom Lane
Greg Stark [EMAIL PROTECTED] writes: There could be a per-database oldest xid that any vacuum on any table updates (by skimming all the oldest xids for the current database). If that's stored in the shared pg_database table then it's accessible regardless of what database you connect to, no?

Re: [HACKERS] Get rid of system attributes in pg_attribute?

2005-02-18 Thread Dave Page
-Original Message- From: [EMAIL PROTECTED] on behalf of Tom Lane Sent: Fri 2/18/2005 8:48 PM To: pgsql-hackers@postgresql.org Subject: [HACKERS] Get rid of system attributes in pg_attribute? Does anyone know of client code that actually pays attention to pg_attribute rows with

Re: [HACKERS] SPI_finish and RegisterExprContextCallback

2005-02-18 Thread Thomas Hallgren
I found another piece of information that might be of interest. This is related to nested calls and the ExprContextCallback but not related to nested active cursors. AtCommitPortals (portalmem.c) iterates over the entries in the PortalHashTable. This causes a chain of calls that sometimes

Re: [HACKERS] Data loss, vacuum, transaction wrap-around

2005-02-18 Thread Russell Smith
On Sat, 19 Feb 2005 04:10 am, Tom Lane wrote: [EMAIL PROTECTED] writes: In fact, I think it is so bad, that I think we need to back-port a fix to previous versions and issue a notice of some kind. They already do issue notices --- see VACUUM. A real fix (eg the forcible stop we were

Re: [HACKERS] Data loss, vacuum, transaction wrap-around

2005-02-18 Thread pgsql
On Sat, 19 Feb 2005 04:10 am, Tom Lane wrote: [EMAIL PROTECTED] writes: In fact, I think it is so bad, that I think we need to back-port a fix to previous versions and issue a notice of some kind. They already do issue notices --- see VACUUM. A real fix (eg the forcible stop we were

Re: [HACKERS] Data loss, vacuum, transaction wrap-around

2005-02-18 Thread Andrew Dunstan
Russell Smith wrote: On Sat, 19 Feb 2005 04:10 am, Tom Lane wrote: [EMAIL PROTECTED] writes: In fact, I think it is so bad, that I think we need to back-port a fix to previous versions and issue a notice of some kind. They already do issue notices --- see VACUUM. A real fix (eg the

Re: [HACKERS] Data loss, vacuum, transaction wrap-around

2005-02-18 Thread lsunley
In [EMAIL PROTECTED], on 02/18/05 at 09:48 PM, Andrew Dunstan [EMAIL PROTECTED] said: Russell Smith wrote: On Sat, 19 Feb 2005 04:10 am, Tom Lane wrote: [EMAIL PROTECTED] writes: In fact, I think it is so bad, that I think we need to back-port a fix to previous versions and issue

Re: [HACKERS] UTF8 or Unicode

2005-02-18 Thread Bruce Momjian
Dave Page wrote: Karel Zak wrote: Yes, I think we should fix it and remove UNICODE and WIN encoding names from PG code. The JDBC driver asks for a UNICODE client encoding before it knows the server version it is talking to. How do you avoid breaking this? So does pgAdmin. I think

Re: [HACKERS] Data loss, vacuum, transaction wrap-around

2005-02-18 Thread Tom Lane
[EMAIL PROTECTED] writes: I think there should be a 100% no data loss fail safe. Possibly we need to recalibrate our expectations here. The current situation is that PostgreSQL will not lose data if: 1. Your disk drive doesn't screw up (eg, lie about write complete, or just

Re: [HACKERS] SPI_finish and RegisterExprContextCallback

2005-02-18 Thread Tom Lane
Thomas Hallgren [EMAIL PROTECTED] writes: AtCommitPortals (portalmem.c) iterates over the entries in the PortalHashTable. This causes a chain of calls that sometimes reach an ExprContextCallback. If that callback issues a succesfull SPI_cursor_close some problems might lay ahead. As the

Re: [HACKERS] UTF8 or Unicode

2005-02-18 Thread Tom Lane
Bruce Momjian pgman@candle.pha.pa.us writes: I think we just need to _favor_ UTF8. I agree. The question is where are we favoring Unicode rather than UTF8? It's the canonical name of the encoding, both in the code and the docs. regression=# create database e encoding 'utf-8'; CREATE

Re: [HACKERS] 8.0.X and the ARC patent

2005-02-18 Thread Bruce Momjian
Tom Lane wrote: Still to be determined: what we lose in extra I/O from the presumably less efficient cache management; also what sort of slowdown occurs on a single-CPU machine that isn't going to get any benefit from the increased amount of lock management. But it looks promising. Yea, that

Re: [HACKERS] Help me recovering data

2005-02-18 Thread Bruce Momjian
Matthew T. O'Connor wrote: Tom Lane wrote: [EMAIL PROTECTED] writes: Maybe I'm missing something, but shouldn't the prospect of data loss (even in the presense of admin ignorance) be something that should be unacceptable? Certainly within the realm normal PostgreSQL operation.

Re: [HACKERS] 8.0.X and the ARC patent

2005-02-18 Thread Tom Lane
Bruce Momjian pgman@candle.pha.pa.us writes: Tom Lane wrote: Still to be determined: what we lose in extra I/O from the presumably less efficient cache management; also what sort of slowdown occurs on a single-CPU machine that isn't going to get any benefit from the increased amount of lock

[HACKERS] problems with locale settings

2005-02-18 Thread Manuel Sugawara
Hi, Starting postgres as part as of the normal boot process of my db server (up2date FC3) leads to an ``unusable'' database since it does not set the locale properly. For instnace: test=# select 'í' ~ '[[:lower:]]'; ?column? -- f (1 fila) However, starting postgres as $