Re: [HACKERS] Crash bug in 8.2.3 on Solaris 10/Sparc

2007-03-30 Thread Zdenek Kotala
Zoltan Boszormenyi wrote: We compiled GCC-4.1.2 on this machine, recompiled PostgreSQL with the new GCC without --enable-integer-datetimes and it fixed the problem we experienced. It seems that my suspicion was right: GCC-3.4.3 on Solaris 10/Sparc is buggy. I tried original S10 gcc (3.4.3)

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-03-30 Thread Zeugswetter Andreas ADI SD
Without a switch, because both full page writes and corresponding logical log is included in WAL, this will increase WAL size slightly (maybe about five percent or so). If everybody is happy with this, we don't need a switch. Sorry, I still don't understand that. What is the

Re: [HACKERS] Autovacuum vs statement_timeout

2007-03-30 Thread Heikki Linnakangas
Tom Lane wrote: I seem to remember that we'd agreed that autovacuum should ignore any globally set statement_timeout, on the grounds that a poorly chosen setting could indefinitely prevent large tables from being vacuumed. But I do not see anything in autovacuum.c that resets the variable. Am I

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-03-30 Thread Simon Riggs
On Fri, 2007-03-30 at 10:22 +0200, Zeugswetter Andreas ADI SD wrote: Without a switch, because both full page writes and corresponding logical log is included in WAL, this will increase WAL size slightly (maybe about five percent or so). If everybody is happy with this, we don't

[HACKERS] Minor changes to Recovery related code

2007-03-30 Thread Simon Riggs
I'd like to make the following changes to recovery related code over the next few days/weeks. If anybody insists I do this by freeze or not at all, then I'll submit patches for 1,3,4,5,10 before Saturday night. I'd rather take a bit more time and do this in one drop and there are some code

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-03-30 Thread Zeugswetter Andreas ADI SD
Archive recovery needs the normal xlog record, which in some cases has been optimised away because the backup block is present, since the full block already contains the changes. Aah, I didn't know that optimization exists. I agree that removing that optimization is good/ok. Andreas

[HACKERS] Proposal: include PL/Proxy into core

2007-03-30 Thread Marko Kreen
PL/Proxy is a small PL whose goal is to allow creating proxy functions that call actual functions in remote database. Basic design is: Function body describes how to deduce final database. Its either CONNECT 'connstr'; -- connect to exactly this db or when partitioning is used: --

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-03-30 Thread Richard Huxton
Simon Riggs wrote: On Fri, 2007-03-30 at 10:22 +0200, Zeugswetter Andreas ADI SD wrote: Without a switch, because both full page writes and corresponding logical log is included in WAL, this will increase WAL size slightly (maybe about five percent or so). If everybody is happy with this,

Re: [HACKERS] Proposal: include PL/Proxy into core

2007-03-30 Thread Cédric Villemain
Le vendredi 30 mars 2007 12:36, Marko Kreen a écrit : Patch: http://plproxy.projects.postgresql.org/plproxy_core.diff.gz Note a perhaps oversight in your makefile : + #REGRESS_OPTS = --dbname=$(PL_TESTDB) --load-language=plpgsql --load-language=plproxy + REGRESS_OPTS = --dbname=regression

Re: [HACKERS] Proposal: include PL/Proxy into core

2007-03-30 Thread Marko Kreen
On 3/30/07, Cédric Villemain [EMAIL PROTECTED] wrote: Le vendredi 30 mars 2007 12:36, Marko Kreen a écrit: Patch: http://plproxy.projects.postgresql.org/plproxy_core.diff.gz Note a perhaps oversight in your makefile : + #REGRESS_OPTS = --dbname=$(PL_TESTDB) --load-language=plpgsql

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-03-30 Thread Simon Riggs
On Fri, 2007-03-30 at 11:27 +0100, Richard Huxton wrote: Is that always true? Could the backup not pick up a partially-written page? Assuming it's being written to as the backup is in progress. (We are talking about when disk blocks are smaller than PG blocks here, so can't guarantee an

Re: [HACKERS] UPDATE using sub selects

2007-03-30 Thread NikhilS
Hi, No. Offhand I think you'd either need to relax EXPR_SUBLINK to allow multiple output columns, or invent a ROW_SUBLINK SubLinkType that is just like EXPR_SUBLINK except for allowing multiple output columns. The latter would probably be less likely to break other things... Yeah, was

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-03-30 Thread Richard Huxton
Simon Riggs wrote: On Fri, 2007-03-30 at 11:27 +0100, Richard Huxton wrote: Is that always true? Could the backup not pick up a partially-written page? Assuming it's being written to as the backup is in progress. (We are talking about when disk blocks are smaller than PG blocks here, so

Re: [HACKERS] Proposal: include PL/Proxy into core

2007-03-30 Thread Hannu Krosing
Ühel kenal päeval, R, 2007-03-30 kell 13:36, kirjutas Marko Kreen: PL/Proxy is a small PL whose goal is to allow creating proxy functions that call actual functions in remote database. Basic design is: Function body describes how to deduce final database. Its either CONNECT 'connstr';

[HACKERS] Proposal: Adding CP51932 encoding

2007-03-30 Thread Tatsuo Ishii
Hi, Here is a proposal to add a client side encoding CP51932. CP51932 is an encoding for Japanese, defined by Microsoft. It is similar to existing EUC_JP encoding but some characters are mapped differently. Unfortunately for most Windows apps EUC-JP means CP51932. As a result, for example, if a

Re: [HACKERS] Autovacuum vs statement_timeout

2007-03-30 Thread Tom Lane
Heikki Linnakangas [EMAIL PROTECTED] writes: statement_timeout interrupts seem to go through the PG_CATCH-block and clean up the entry from the vacuum cycle array as they should. But a SIGINT leading to a terminating connection due to administrator command error does not. Hm, that's an

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Tom Lane
Pavan Deolasee [EMAIL PROTECTED] writes: How about storing the snapshot which we used during planning in CachedPlanSource, if at least one index was seen unusable because its CREATE INDEX transaction was seen as in-progress ? I'm getting tired of repeating this, but: the planner doesn't use a

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Pavan Deolasee
Tom Lane wrote: I'm getting tired of repeating this, but: the planner doesn't use a snapshot. System catalogs run on SnapshotNow. I am really sorry if I sound foolish here. I am NOT suggesting that we use snapshot to read system catalogs. I understand that system catalogs run on SnapshotNow

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Florian G. Pflug
Tom Lane wrote: Pavan Deolasee [EMAIL PROTECTED] writes: How about storing the snapshot which we used during planning in CachedPlanSource, if at least one index was seen unusable because its CREATE INDEX transaction was seen as in-progress ? I'm getting tired of repeating this, but: the

Re: [HACKERS] [pgsql-www] Help: reading the source

2007-03-30 Thread Andrew Hammond
1) Wrong list, you want -hackers. 2) Did you have a specific question based on what you have read or are you asking for suggested reading? On 3/29/07, 李宁 [EMAIL PROTECTED] wrote: Dear everyone, I'm a college student,now I am doing the topic about the postgresql storage management as my

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Tom Lane
Pavan Deolasee [EMAIL PROTECTED] writes: What I am suggesting is to use ActiveSnapshot (actually Florian's idea) to decide whether the transaction that created index was still running when we started. Isn't it the case that some snapshot will be active when we plan ? I do not think you can

[HACKERS] Oracle indemnifies PostgreSQL on its patents

2007-03-30 Thread Bruce Momjian
Here is a surprising article about how Oracle has made open source projects, like PostgreSQL, safe from claims of infringing Oracle patents: http://www.cbronline.com/article_news.asp?guid=A0F5F220-5940-470D-8564-CEA7E2D2B954. Oracle, like IBM, Sony, RedHat, and Novell, is now a member

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Florian G. Pflug
Tom Lane wrote: Pavan Deolasee [EMAIL PROTECTED] writes: What I am suggesting is to use ActiveSnapshot (actually Florian's idea) to decide whether the transaction that created index was still running when we started. Isn't it the case that some snapshot will be active when we plan ? I do not

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Pavan Deolasee
On 3/30/07, Florian G. Pflug [EMAIL PROTECTED] wrote: What about doing PREPARE myplan select ... ; outside of a transaction? Will this be execute inside a transaction? I checked that. PREPARE runs with ActiveSnapshot set. Thanks, Pavan -- EnterpriseDB http://www.enterprisedb.com

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Tom Lane
Florian G. Pflug [EMAIL PROTECTED] writes: Tom Lane wrote: I do not think you can assume that the plan won't be used later with some older snapshot. So maybe we'd need to use the SerializableSnapshot created at the start of each transaction for this check, and not the ActiveSnapshot? Could

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Simon Riggs
On Fri, 2007-03-30 at 11:44 +0530, Pavan Deolasee wrote: ISTM that the run-another-transaction-afterwards idea would have same problem with plan invalidation. When the second transaction commits, the relcache invalidation event is generated. The event may get consumed by other backends, but

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Pavan Deolasee
On 3/30/07, Tom Lane [EMAIL PROTECTED] wrote: That might work, but it doesn't seem to address the core objection: there's no mechanism to cause the query to be replanned once the snapshot is new enough, because no relcache inval will happen. So most likely existing backends will keep using

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Florian G. Pflug
Tom Lane wrote: Florian G. Pflug [EMAIL PROTECTED] writes: Tom Lane wrote: I do not think you can assume that the plan won't be used later with some older snapshot. So maybe we'd need to use the SerializableSnapshot created at the start of each transaction for this check, and not the

Re: [HACKERS] Autovacuum vs statement_timeout

2007-03-30 Thread Tom Lane
I wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: statement_timeout interrupts seem to go through the PG_CATCH-block and clean up the entry from the vacuum cycle array as they should. But a SIGINT leading to a terminating connection due to administrator command error does not. Hm,

Re: [HACKERS] Autovacuum vs statement_timeout

2007-03-30 Thread Alvaro Herrera
Tom Lane wrote: I wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: statement_timeout interrupts seem to go through the PG_CATCH-block and clean up the entry from the vacuum cycle array as they should. But a SIGINT leading to a terminating connection due to administrator command

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Pavan Deolasee
On 3/30/07, Florian G. Pflug [EMAIL PROTECTED] wrote: My idea was to store a list of xid's together with the cached plan that are assumed to be uncommitted accoring to the IndexSnapshot. The query is replanned if upon execution the IndexSnapshot assumes that one of these xid's is committed.

Re: [HACKERS] Autovacuum vs statement_timeout

2007-03-30 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: Hmmm, remember that DatabaseCancelAutovacuumActivity is called on CREATE DATABASE; but what it does is send SIGINT, not SIGTERM. Also, it's not in 8.2. SIGINT does terminate the autovac process however. I haven't read the whole problem report

Re: [HACKERS] Proposal: include PL/Proxy into core

2007-03-30 Thread Tom Lane
Marko Kreen [EMAIL PROTECTED] writes: Now, why put it into core? I don't think you have made a sufficient case for that. I think it should stay as an outside project for awhile and see what sort of userbase it attracts. If it becomes sufficiently popular I'd be willing to consider adding it to

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Florian G. Pflug
Pavan Deolasee wrote: On 3/30/07, Florian G. Pflug [EMAIL PROTECTED] wrote: My idea was to store a list of xid's together with the cached plan that are assumed to be uncommitted accoring to the IndexSnapshot. The query is replanned if upon execution the IndexSnapshot assumes that one of these

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Tom Lane
Pavan Deolasee [EMAIL PROTECTED] writes: Actually, if we are using Serializable Snapshot then there is no chance to replan the query before the transaction completes and the next transaction to start in the session must see the index and hence we must replan. So it would be enough just to

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Simon Riggs
On Fri, 2007-03-30 at 13:54 -0400, Tom Lane wrote: Pavan Deolasee [EMAIL PROTECTED] writes: Actually, if we are using Serializable Snapshot then there is no chance to replan the query before the transaction completes and the next transaction to start in the session must see the index and

Re: [HACKERS] Proposal: include PL/Proxy into core

2007-03-30 Thread Josh Berkus
Hannu, Marko, I, personally, think that it's worth talking about integrating these. However, the old versions were definitely NOT ready for integration, and the new versions went on the internet like a week ago. Heck, I haven't even downloaded them yet. Can we address these on the 8.4

Re: [HACKERS] Arrays of Complex Types

2007-03-30 Thread David Fetter
On Fri, Mar 02, 2007 at 03:40:16PM -0800, David Fetter wrote: Folks, I'd like to take the TODO item that reads, Add support for arrays of complex types, but before I start patching, I'd like to see whether what I'm about to do makes any sense: After several rounds of patches, it appears

Re: [HACKERS] Minor changes to Recovery related code

2007-03-30 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: I'd like to make the following changes to recovery related code over the next few days/weeks. If anybody insists I do this by freeze or not at all, then I'll submit patches for 1,3,4,5,10 before Saturday night. I'd rather take a bit more time and do this

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-03-30 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: Any page written during a backup has a backup block that would not be removable by Koichi's tool, so yes, you'd still be safe. How does it know not to do that? regards, tom lane ---(end of

Re: [HACKERS] UPDATE using sub selects

2007-03-30 Thread Tom Lane
NikhilS [EMAIL PROTECTED] writes: I have invented a ROWEXPR_SUBLINK type that handles multiple output columns. The trouble is that since eventually columns of the parents have to be part of the query's targetList, I am sending the entire subquery as one of the entries in that list and the

Re: [HACKERS] Oracle indemnifies PostgreSQL on its patents

2007-03-30 Thread Josh Berkus
Bruce, Oracle, like IBM, Sony, RedHat, and Novell, is now a member of the Open Invention Network, whose members all offer patent indemnification. Hey! We could go back to using ARC! ;-) -- Josh Berkus PostgreSQL @ Sun San Francisco ---(end of

Re: [HACKERS] Arrays of Complex Types

2007-03-30 Thread Tom Lane
David Fetter [EMAIL PROTECTED] writes: After several rounds of patches, it appears that it might be easier to create a new typtype entry, which I'll tentatively call 'a' because it seems a little fragile and a lot inelegant and hard to maintain to have typtype='c' and typrelid=InvalidOid mean,

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-03-30 Thread Simon Riggs
On Fri, 2007-03-30 at 16:35 -0400, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: Any page written during a backup has a backup block that would not be removable by Koichi's tool, so yes, you'd still be safe. How does it know not to do that? Not sure what you mean, but I'll take a

Re: [HACKERS] Minor changes to Recovery related code

2007-03-30 Thread Simon Riggs
On Fri, 2007-03-30 at 16:34 -0400, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: I'd like to make the following changes to recovery related code over the next few days/weeks. If anybody insists I do this by freeze or not at all, then I'll submit patches for 1,3,4,5,10 before

Re: [HACKERS] Minor changes to Recovery related code

2007-03-30 Thread Florian G. Pflug
Simon Riggs wrote: On Fri, 2007-03-30 at 16:34 -0400, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: 2. pg_stop_backup() should wait until all archive files are safely archived before returning Not sure I agree with that one. If it fails, you can't tell whether the action is done and

[HACKERS] Last minute mini-proposal (I know, I know) for PQexecf()

2007-03-30 Thread korryd
While cleaning up some pg_migrator code (http://pgfoundry.org/projects/pg-migrator/) it occurred to me that a typical libpq client application spends a lot of code constructing SQL commands. The code typically looks like this: a) allocate enough room to hold the command b)

Re: [HACKERS] Last minute mini-proposal (I know, I know) for PQexecf()

2007-03-30 Thread Tom Lane
[EMAIL PROTECTED] writes: I'd like to see a new variant on PQexec(): PGresult * PQexecf(PGconn *conn, const char *fmt, ...); Way too late for 8.3 --- if we were going to do something like this, we should think first and program later. In particular, blindly adopting the sprintf format

Re: [HACKERS] [PATCHES] Full page writes improvement, code update

2007-03-30 Thread Koichi Suzuki
Simon; Tom; Koichi is writing. Your question is how to determine WAL record generated between pg_start_backup and pg_stop_backup and here's an answer. XLogInsert( ) already has a logic to determine if inserting WAL record is between pg_start_backup and pg_stop_backup. Currently it is used to

Re: [HACKERS] Last minute mini-proposal (I know, I know) for PQexecf()

2007-03-30 Thread Gregory Stark
Hm, my first thought was that you should just be using bind parameters instead of interpolating variables directly into the query. But the more I think about it the more I like your idea. It's true that using parameters takes away most of the use cases for the kind of interface you suggest. But

Re: [HACKERS] Modifying TOAST thresholds

2007-03-30 Thread Christopher Browne
[EMAIL PROTECTED] (Tom Lane) wrote: I'm willing to do the code changes to separate TOAST_THRESHOLD from the toast chunk size, but I do not have the time or facilities to do any performance testing for different parameter choices. Anyone want to work on that? What have you got in mind there?

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Pavan Deolasee
On 3/31/07, Simon Riggs [EMAIL PROTECTED] wrote: On Fri, 2007-03-30 at 13:54 -0400, Tom Lane wrote: Hm. So anytime we reject a potentially useful index as being not valid yet, we mark the plan as only good for this top-level transaction? That seems possibly workable --- in particular it

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Pavan Deolasee
On 3/30/07, Tom Lane [EMAIL PROTECTED] wrote: I do not think you can assume that the plan won't be used later with some older snapshot. Consider recursive plpgsql functions for a counterexample: the inner occurrence might be the first to arrive at a given line of the function, hence the first

Re: [HACKERS] UPDATE using sub selects

2007-03-30 Thread NikhilS
Hi, On 3/31/07, Tom Lane [EMAIL PROTECTED] wrote: NikhilS [EMAIL PROTECTED] writes: I have invented a ROWEXPR_SUBLINK type that handles multiple output columns. The trouble is that since eventually columns of the parents have to be part of the query's targetList, I am sending the entire