Re: [HACKERS] Buffer statistics for pg_stat_statements

2009-12-22 Thread Takahiro Itagaki
Cedric Villemain cedric.villem...@dalibo.com wrote: Le vendredi 18 decembre 2009 09:44:40, Takahiro Itagaki a ecrit : I'd like to add per-query buffer usage into contrib/pg_stat_statements. Here is a patch to add buffer usage columns into pg_stat_statements. It also changes initialzation of

Re: [HACKERS] alpha3 release schedule?

2009-12-22 Thread Simon Riggs
On Sun, 2009-12-20 at 19:11 -0500, Robert Haas wrote: On Sun, Dec 20, 2009 at 3:42 PM, Simon Riggs si...@2ndquadrant.com wrote: On Sat, 2009-12-19 at 20:59 +0200, Heikki Linnakangas wrote: I put them on the TODO list at https://wiki.postgresql.org/wiki/Hot_Standby_TODO, under the must-fix

Re: [HACKERS] New VACUUM FULL

2009-12-22 Thread Takahiro Itagaki
Simon Riggs si...@2ndquadrant.com wrote: I think we either need to implement that or document that vacuum will not skip all-visible pages when running VACUUM FULL. All-visible does not always mean filled enough, no? We will need to check both visibility and fillfactor when we optimize

Re: [HACKERS] fdw validation function vs zero catalog id

2009-12-22 Thread Martin Pihlak
I wrote: The validator is run for the generic options specified to CREATE/ALTER FDW, SERVER and USER MAPPING (+ possible future SQL/MED objects). In this case the catalog is always known. Also we can assume that the catalog is known, if a user runs the validator directly. So yes, AFAICS

Re: [HACKERS] New VACUUM FULL

2009-12-22 Thread Simon Riggs
On Tue, 2009-12-22 at 18:11 +0900, Takahiro Itagaki wrote: Old VACUUM FULL was substantially faster than this on tables that had nothing to remove. Yeah, that's why traditional VACUUM FULL is still kept as INPLACE vacuum. Please can you arrange for the cluster operation to skip

Re: [HACKERS] Small typos in Hot Standby docs

2009-12-22 Thread Simon Riggs
On Sat, 2009-12-19 at 18:42 -0800, John Naylor wrote: Here's a patch: Thanks John, much appreciated. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] New VACUUM FULL

2009-12-22 Thread Simon Riggs
On Tue, 2009-12-22 at 18:11 +0900, Takahiro Itagaki wrote: Instead, I'd suggest to have conditional database-wide maintenance commands, something like: VACUUM FULL WHERE the table is fragmented We don't have to support the feature by SQL syntax; it could be done in client tools. How

Re: [HACKERS] New VACUUM FULL

2009-12-22 Thread Takahiro Itagaki
Simon Riggs si...@2ndquadrant.com wrote: Our perception of acceptable change is much higher than most users. If we tell people use VACUUM FULL or vacuumdb -f, then that command should, if possible, continue to work well across many releases. vacuumdb in most people's minds is the command you

Re: [HACKERS] Small Bug in GetConflictingVirtualXIDs

2009-12-22 Thread Simon Riggs
On Tue, 2009-12-22 at 03:19 +0100, Andres Freund wrote: On Monday 21 December 2009 16:48:52 Simon Riggs wrote: Giving the drop database a snapshot is not the answer. I expect Andres to be able to fix this with a simple patch that would not effect the case of normal running. Actually its

Re: [HACKERS] Table size does not include toast size

2009-12-22 Thread Cédric Villemain
2009/12/21 Tom Lane t...@sss.pgh.pa.us: Greg Smith g...@2ndquadrant.com writes: To answer Rafael's concerns directly:  you're right that this is confusing.  pg_relation_size is always going to do what it does right now just because of how that fits into the design of the database. However,

Re: [HACKERS] Buffer statistics for pg_stat_statements

2009-12-22 Thread Cédric Villemain
2009/12/22 Takahiro Itagaki itagaki.takah...@oss.ntt.co.jp: Cedric Villemain cedric.villem...@dalibo.com wrote: Le vendredi 18 decembre 2009 09:44:40, Takahiro Itagaki a ecrit : I'd like to add per-query buffer usage into contrib/pg_stat_statements. Here is a patch to add buffer usage

Re: [HACKERS] alpha3 release schedule?

2009-12-22 Thread Florian Pflug
On 22.12.09 9:34 , Simon Riggs wrote: If you are saying being able to start Hot Standby from a shutdown checkpoint is an important feature for you, then say so, and why. I think it's not so much an important feature but more the removal of a footgun. Image a reporting database where all

Re: [HACKERS] Streaming replication and non-blocking I/O

2009-12-22 Thread Greg Stark
On Tue, Dec 22, 2009 at 6:30 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: I think we can just use load_external_function() to load the library and call WalReceiverMain from AuxiliaryProcessMain(). Ie. hard-code the library name. Walreceiver is quite tightly coupled with the

Re: [HACKERS] alpha3 release schedule?

2009-12-22 Thread Greg Stark
On Tue, Dec 22, 2009 at 8:34 AM, Simon Riggs si...@2ndquadrant.com wrote: If you are saying being able to start Hot Standby from a shutdown checkpoint is an important feature for you, then say so, and why. Can you explain the consequences of missing this? It sounds to me like if I lose my

[HACKERS] Tuplestore should remember the memory context it's created in

2009-12-22 Thread Heikki Linnakangas
With regards to this bug report: http://archives.postgresql.org/pgsql-bugs/2009-12/msg00241.php I think we should change tuplestore code so that callers of tuplestore_put* don't need to switch to the correct memory context (and resource owner, after this patch) before call. Instead,

Re: [HACKERS] Streaming replication and non-blocking I/O

2009-12-22 Thread Heikki Linnakangas
Fujii Masao wrote: On Tue, Dec 22, 2009 at 3:30 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: I think we can just use load_external_function() to load the library and call WalReceiverMain from AuxiliaryProcessMain(). Ie. hard-code the library name. Walreceiver is quite

Re: [HACKERS] Tuplestore should remember the memory context it's created in

2009-12-22 Thread Greg Stark
On Tue, Dec 22, 2009 at 11:45 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: ... AFAICS it is always a bug to be in a different memory context in tuplestore_put* than in tuplestore_begin_heap(), so it would be more robust to not put the burden on the callers. ... Patch

Re: [HACKERS] Tuplestore should remember the memory context it's created in

2009-12-22 Thread Greg Stark
On Tue, Dec 22, 2009 at 11:45 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: With regards to this bug report: http://archives.postgresql.org/pgsql-bugs/2009-12/msg00241.php Hm, do we have any build farm members running with work_mem set to the minimum? -- greg -- Sent via

Re: [HACKERS] alpha3 release schedule?

2009-12-22 Thread Simon Riggs
On Tue, 2009-12-22 at 12:32 +0100, Florian Pflug wrote: On 22.12.09 9:34 , Simon Riggs wrote: If you are saying being able to start Hot Standby from a shutdown checkpoint is an important feature for you, then say so, and why. I think it's not so much an important feature but more the

Re: [HACKERS] alpha3 release schedule?

2009-12-22 Thread Simon Riggs
On Tue, 2009-12-22 at 11:41 +, Greg Stark wrote: On Tue, Dec 22, 2009 at 8:34 AM, Simon Riggs si...@2ndquadrant.com wrote: If you are saying being able to start Hot Standby from a shutdown checkpoint is an important feature for you, then say so, and why. Can you explain the

Re: [HACKERS] New VACUUM FULL

2009-12-22 Thread Simon Riggs
On Tue, 2009-12-22 at 19:45 +0900, Takahiro Itagaki wrote: I used VACUUM FULL because we were discussing to drop VFI completely, but I won't replace the behavior if hot-standby can support VFI. HS can't support VFI now, by definition. We agreed to spend the time getting rid of VFI, which

Re: [HACKERS] Streaming replication and non-blocking I/O

2009-12-22 Thread Heikki Linnakangas
Greg Stark wrote: On Tue, Dec 22, 2009 at 6:30 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: I think we can just use load_external_function() to load the library and call WalReceiverMain from AuxiliaryProcessMain(). Ie. hard-code the library name. Walreceiver is quite

Re: [HACKERS] alpha3 release schedule?

2009-12-22 Thread Heikki Linnakangas
Simon Riggs wrote: If someone does add this, it will require careful thought about how to avoid introducing further subtle ways to break HS, all of which will need testing and re-testing to avoid regression. Well, I *did* add that, but you removed it... -- Heikki Linnakangas

Re: [HACKERS] Table size does not include toast size

2009-12-22 Thread Bernd Helmle
--On 22. Dezember 2009 11:46:32 +0100 Cédric Villemain cedric.villemain.deb...@gmail.com wrote: Did you mean : pg_table_size() = base table + toast table pg_indexes_size() = base indexes + toast indexes ? Since you always have a toast index automatically it makes sense to include them

Re: [HACKERS] Table size does not include toast size

2009-12-22 Thread Bernd Helmle
--On 21. Dezember 2009 12:02:02 -0500 Greg Smith g...@2ndquadrant.com wrote: Tom Lane wrote: Perhaps invent pg_table_size() = base table + toast table + toast index and pg_indexes_size() = all other indexes for table giving us the property pg_table_size + pg_indexes_size =

Re: [HACKERS] Streaming replication and non-blocking I/O

2009-12-22 Thread Fujii Masao
On Tue, Dec 22, 2009 at 8:49 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Ah, I see. The changes were not included in the merge commit after all, but I had simple forgot to git add them. Sorry about that, should be there now. Thanks for doing git push again! But the

Re: [HACKERS] alpha3 release schedule?

2009-12-22 Thread Simon Riggs
On Tue, 2009-12-22 at 16:09 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: If someone does add this, it will require careful thought about how to avoid introducing further subtle ways to break HS, all of which will need testing and re-testing to avoid regression. Well, I *did* add

Re: [HACKERS] Table size does not include toast size

2009-12-22 Thread Tom Lane
=?ISO-8859-1?Q?C=E9dric_Villemain?= cedric.villemain.deb...@gmail.com writes: 2009/12/21 Tom Lane t...@sss.pgh.pa.us: Perhaps invent  pg_table_size() = base table + toast table + toast index and             pg_indexes_size() = all other indexes for table giving us the property pg_table_size +

Re: [HACKERS] Tuplestore should remember the memory context it's created in

2009-12-22 Thread Tom Lane
Greg Stark gsst...@mit.edu writes: On Tue, Dec 22, 2009 at 11:45 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: AFAICS it is always a bug to be in a different memory context in tuplestore_put* than in tuplestore_begin_heap(), so it would be more robust to not put the burden

Re: [HACKERS] alpha3 release schedule?

2009-12-22 Thread Florian Pflug
On 22.12.09 13:21 , Simon Riggs wrote: On Tue, 2009-12-22 at 12:32 +0100, Florian Pflug wrote: Image a reporting database where all transactions but a few daily bulk imports are read-only. To spread the load, you do your bulk loads on the master, but run the reporting queries against a

Re: [HACKERS] Tuplestore should remember the memory context it's created in

2009-12-22 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Patch against CVS HEAD to do that and fix the reported bug attached. Now that the tuplestore_put* switches to the right memory context, we could remove that from all the callers, but this patch only does it for pl_exec.c. BTW, I'm

Re: [HACKERS] alpha3 release schedule?

2009-12-22 Thread Greg Stark
On Tue, Dec 22, 2009 at 3:32 PM, Florian Pflug fgp.phlo@gmail.com wrote: Well, you either wait for master to come up again and restart, or you flip into normal mode and keep running queries from there. You aren't prevented from using the server, except by your own refusal to failover.

Re: [HACKERS] alpha3 release schedule?

2009-12-22 Thread Simon Riggs
On Tue, 2009-12-22 at 16:32 +0100, Florian Pflug wrote: But you are of course free to work on whatever you feel like, and probably need to satisfy your client's needs first. Alluding to me as whimsical or mercenary isn't likely to change my mind. IMHO this isn't one of the more important

Re: [HACKERS] Tuplestore should remember the memory context it's created in

2009-12-22 Thread Heikki Linnakangas
Tom Lane wrote: Greg Stark gsst...@mit.edu writes: On Tue, Dec 22, 2009 at 11:45 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: AFAICS it is always a bug to be in a different memory context in tuplestore_put* than in tuplestore_begin_heap(), so it would be more robust to

Re: [HACKERS] alpha3 release schedule?

2009-12-22 Thread Simon Riggs
On Tue, 2009-12-22 at 15:38 +, Greg Stark wrote: On Tue, Dec 22, 2009 at 3:32 PM, Florian Pflug fgp.phlo@gmail.com wrote: Well, you either wait for master to come up again and restart, or you flip into normal mode and keep running queries from there. You aren't prevented from using

Re: [HACKERS] alpha3 release schedule?

2009-12-22 Thread Heikki Linnakangas
Simon Riggs wrote: By add I meant to write the feature, test and then support it afterwards, not to re-discuss editing the Wiki. That's exactly what I meant too. I *did* write the feature, but you removed it before committing. I can extract the removed parts from the git repository and send

Re: [HACKERS] [PATCH] remove redundant ownership checks

2009-12-22 Thread Robert Haas
2009/12/22 KaiGai Kohei kai...@ak.jp.nec.com: (2009/12/21 12:53), Robert Haas wrote: On Thu, Dec 17, 2009 at 7:19 PM, Tom Lanet...@sss.pgh.pa.us  wrote: KaiGai Koheikai...@ak.jp.nec.com  writes: [ patch to remove EnableDisableRule's permissions check ] I don't particularly like this patch,

Re: [HACKERS] alpha3 release schedule?

2009-12-22 Thread Simon Riggs
On Tue, 2009-12-22 at 18:17 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: By add I meant to write the feature, test and then support it afterwards, not to re-discuss editing the Wiki. That's exactly what I meant too. I *did* write the feature, but you removed it before committing.

Re: [HACKERS] Tuplestore should remember the memory context it's created in

2009-12-22 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Tom Lane wrote: I think it was just a performance optimization. It's probably not measurable though; even in the in-memory case there's at least a palloc inside the put() function, no? Yes. And many of the callers do the memory

Re: [HACKERS] alpha3 release schedule?

2009-12-22 Thread Heikki Linnakangas
Simon Riggs wrote: On Tue, 2009-12-22 at 18:17 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: By add I meant to write the feature, test and then support it afterwards, not to re-discuss editing the Wiki. That's exactly what I meant too. I *did* write the feature, but you removed it

Re: [HACKERS] Largeobject Access Controls (r2460)

2009-12-22 Thread Robert Haas
2009/12/22 KaiGai Kohei kai...@ak.jp.nec.com: (2009/12/21 9:39), KaiGai Kohei wrote: (2009/12/19 12:05), Robert Haas wrote: On Fri, Dec 18, 2009 at 9:48 PM, Tom Lanet...@sss.pgh.pa.us   wrote: Robert Haasrobertmh...@gmail.com   writes: Oh.  This is more complicated than it appeared on the

Re: [HACKERS] alpha3 release schedule?

2009-12-22 Thread Simon Riggs
On Tue, 2009-12-22 at 18:40 +0200, Heikki Linnakangas wrote: The issue I mentioned had nothing to do with starting from a shutdown checkpoint - it's still a problem if you keep the standby running through the restart cycle in the master) - but maybe you thought it was? Or was there something

Re: [HACKERS] Small change of the HS document

2009-12-22 Thread Simon Riggs
On Tue, 2009-12-22 at 11:25 +0900, Fujii Masao wrote: I found there is no primary tag for the HS parameters in config.sgml. Attached patch adds that tag. Thanks -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] alpha3 release schedule?

2009-12-22 Thread Heikki Linnakangas
Simon Riggs wrote: You've been perfectly happy for *years* with the situation that recovery would fail if max_prepared_transactions was not correctly. You're not going to tell me you never noticed? Why is avoidance of obvious misconfiguration of HS such a heavy priority when nothing else ever

Re: [HACKERS] Backup history file should be replicated in Streaming Replication?

2009-12-22 Thread Simon Riggs
On Thu, 2009-11-26 at 17:02 +0900, Fujii Masao wrote: On Thu, Nov 26, 2009 at 4:55 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Fujii Masao wrote: In current SR, since a backup history file is not replicated, the standby always starts an archive recovery without a

Re: [HACKERS] Backup history file should be replicated in Streaming Replication?

2009-12-22 Thread Simon Riggs
On Tue, 2009-12-22 at 14:40 +0900, Fujii Masao wrote: On Sat, Dec 19, 2009 at 1:03 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: I don't think it's worthwhile to modify pg_stop_backup() like that. We should address the general problem. At the moment, you're fine if you

Re: [HACKERS] Tuplestore should remember the memory context it's created in

2009-12-22 Thread Heikki Linnakangas
Tom Lane wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Patch against CVS HEAD to do that and fix the reported bug attached. Now that the tuplestore_put* switches to the right memory context, we could remove that from all the callers, but this patch only does it for

Re: [HACKERS] Segfault from PL/Perl Returning vstring

2009-12-22 Thread David E. Wheeler
On Dec 21, 2009, at 9:04 PM, Andrew Dunstan wrote: I cannot reproduce this. I tested with perl 5.10.1 which is the latest reported stable release at http://www.cpan.org/src/README.html, on an 8.4.2 UTF8 database, and with the same Safe and Encode module versions as above. I've replicated

Re: [HACKERS] alpha3 release schedule?

2009-12-22 Thread Bruce Momjian
Simon Riggs wrote: On Mon, 2009-12-21 at 18:42 +0900, Hiroyuki Yamada wrote: Do you think this problem is must-fix for the final release ? We should be clear that this is a behaviour I told you about, not a shock discovery by yourself. There is no permanent freeze, just a wait, from

Re: [HACKERS] Backup history file should be replicated in Streaming Replication?

2009-12-22 Thread Fujii Masao
On Wed, Dec 23, 2009 at 2:42 AM, Simon Riggs si...@2ndquadrant.com wrote: The backup history file is a slightly bit quirky way of doing things and was designed when the transfer mechanism was file-based. Why don't we just write a new xlog record that contains the information we need? Copy the

Re: [HACKERS] Backup history file should be replicated in Streaming Replication?

2009-12-22 Thread Simon Riggs
On Wed, 2009-12-23 at 03:28 +0900, Fujii Masao wrote: On Wed, Dec 23, 2009 at 2:42 AM, Simon Riggs si...@2ndquadrant.com wrote: The backup history file is a slightly bit quirky way of doing things and was designed when the transfer mechanism was file-based. Why don't we just write a new

Re: [HACKERS] Small Bug in GetConflictingVirtualXIDs

2009-12-22 Thread Andres Freund
On Tuesday 22 December 2009 11:42:30 Simon Riggs wrote: On Tue, 2009-12-22 at 03:19 +0100, Andres Freund wrote: On Monday 21 December 2009 16:48:52 Simon Riggs wrote: Giving the drop database a snapshot is not the answer. I expect Andres to be able to fix this with a simple patch that

Re: [HACKERS] alpha3 release schedule?

2009-12-22 Thread Simon Riggs
On Tue, 2009-12-22 at 19:30 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: You've been perfectly happy for *years* with the situation that recovery would fail if max_prepared_transactions was not correctly. You're not going to tell me you never noticed? Why is avoidance of obvious

Re: [HACKERS] alpha3 release schedule?

2009-12-22 Thread Florian Pflug
On 22.12.09 16:45 , Simon Riggs wrote: On Tue, 2009-12-22 at 16:32 +0100, Florian Pflug wrote: But you are of course free to work on whatever you feel like, and probably need to satisfy your client's needs first. Alluding to me as whimsical or mercenary isn't likely to change my mind.

[HACKERS] Streaming Rep - 2-phase backups and reducing time to full replication

2009-12-22 Thread Simon Riggs
Some ideas to improve current behaviour of SR http://wiki.postgresql.org/wiki/Streaming_Replication The current startup process is copied below. (7) gives some issues if it is a very long step, notably that the master may fill with data and then break off the connection before replication is

Re: [HACKERS] Tuplestore should remember the memory context it's created in

2009-12-22 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Tom Lane wrote: BTW, I'm not convinced that the owner-switchery you added to pl_exec.c is necessary/appropriate. Under what circumstances would that be a good idea? A PL/pgSQL normally runs in the whatever resource owner is

Re: [HACKERS] Backup history file should be replicated in Streaming Replication?

2009-12-22 Thread Fujii Masao
On Wed, Dec 23, 2009 at 2:49 AM, Simon Riggs si...@2ndquadrant.com wrote: (I don't fully understand your words above, sorry.) NM ;-) Possibly an easier way would be to have a size limit, not a number of segments. Something like max_reserved_wal = 240GB. We made that change to shared_buffers

Re: [HACKERS] alpha3 release schedule?

2009-12-22 Thread Simon Riggs
On Tue, 2009-12-22 at 19:53 +0100, Florian Pflug wrote: None of this was meant as an insult of any kind. Then I apologise completely. I've clearly been working too hard and will retire for some rest (even though that is not listed as a task on the Wiki). -- Simon Riggs

Re: [HACKERS] Backup history file should be replicated in Streaming Replication?

2009-12-22 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: The backup history file is a slightly bit quirky way of doing things and was designed when the transfer mechanism was file-based. Why don't we just write a new xlog record that contains the information we need? Certainly not. The history file is, in

Re: [HACKERS] alpha3 release schedule?

2009-12-22 Thread David E. Wheeler
On Dec 22, 2009, at 11:02 AM, Simon Riggs wrote: I've clearly been working too hard and will retire for some rest (even though that is not listed as a task on the Wiki). Someone add it! David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Backup history file should be replicated in Streaming Replication?

2009-12-22 Thread Fujii Masao
On Wed, Dec 23, 2009 at 3:41 AM, Simon Riggs si...@2ndquadrant.com wrote: This means that we can replace a backup history file with the corresponding xlog record. I think that we should simplify the code by making the replacement completely rather than just adding new xlog record. Thought?

Re: [HACKERS] alpha3 release schedule?

2009-12-22 Thread David Fetter
On Tue, Dec 22, 2009 at 11:04:29AM -0800, David Wheeler wrote: On Dec 22, 2009, at 11:02 AM, Simon Riggs wrote: I've clearly been working too hard and will retire for some rest (even though that is not listed as a task on the Wiki). Someone add it! Done! :) Cheers, David. -- David

Re: [HACKERS] Backup history file should be replicated in Streaming Replication?

2009-12-22 Thread Simon Riggs
On Tue, 2009-12-22 at 14:02 -0500, Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: The backup history file is a slightly bit quirky way of doing things and was designed when the transfer mechanism was file-based. Why don't we just write a new xlog record that contains the

Re: [HACKERS] Backup history file should be replicated in Streaming Replication?

2009-12-22 Thread Fujii Masao
On Wed, Dec 23, 2009 at 4:09 AM, Simon Riggs si...@2ndquadrant.com wrote: It's needed to find the place where the backup stopped, so it defines the safe stopping point. We could easily pass that info via WAL, when streaming. It doesn't actually matter until we try to failover. Right. And, it's

Re: [HACKERS] Backup history file should be replicated in Streaming Replication?

2009-12-22 Thread Simon Riggs
On Wed, 2009-12-23 at 04:15 +0900, Fujii Masao wrote: On Wed, Dec 23, 2009 at 4:09 AM, Simon Riggs si...@2ndquadrant.com wrote: It's needed to find the place where the backup stopped, so it defines the safe stopping point. We could easily pass that info via WAL, when streaming. It doesn't

Re: [HACKERS] Backup history file should be replicated in Streaming Replication?

2009-12-22 Thread Tom Lane
Fujii Masao masao.fu...@gmail.com writes: On Wed, Dec 23, 2009 at 4:09 AM, Simon Riggs si...@2ndquadrant.com wrote: It's needed to find the place where the backup stopped, so it defines the safe stopping point. We could easily pass that info via WAL, when streaming. It doesn't actually matter

Re: [HACKERS] Segfault from PL/Perl Returning vstring

2009-12-22 Thread Alvaro Herrera
David E. Wheeler wrote: On Dec 21, 2009, at 9:04 PM, Andrew Dunstan wrote: I cannot reproduce this. I tested with perl 5.10.1 which is the latest reported stable release at http://www.cpan.org/src/README.html, on an 8.4.2 UTF8 database, and with the same Safe and Encode module versions

Re: [HACKERS] LIKE INCLUDING COMMENTS code is a flight of fancy

2009-12-22 Thread Tom Lane
Takahiro Itagaki itagaki.takah...@oss.ntt.co.jp writes: Tom Lane t...@sss.pgh.pa.us wrote: I suggest that we might want to just rip out the support for copying comments on indexes. We have two related ToDo items below. They are a bit inconsintent, but they mean we should forbid COMMENT on

Re: [HACKERS] Tuplestore should remember the memory context it's created in

2009-12-22 Thread Heikki Linnakangas
Tom Lane wrote: Got it. So doesn't tuplesort have the same issue? Tuplesort has the same general problem that the caller of puttuple needs to be in the right resource owner. Which ought to be fixed, especially since tuplesort doesn't require that for the memory context anymore. But we don't

Re: [HACKERS] Backup history file should be replicated in Streaming Replication?

2009-12-22 Thread Heikki Linnakangas
Simon Riggs wrote: On Wed, 2009-12-23 at 04:15 +0900, Fujii Masao wrote: On Wed, Dec 23, 2009 at 4:09 AM, Simon Riggs si...@2ndquadrant.com wrote: It's needed to find the place where the backup stopped, so it defines the safe stopping point. We could easily pass that info via WAL, when

Re: [HACKERS] Backup history file should be replicated in Streaming Replication?

2009-12-22 Thread Heikki Linnakangas
Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: The backup history file is a slightly bit quirky way of doing things and was designed when the transfer mechanism was file-based. Why don't we just write a new xlog record that contains the information we need? Certainly not. The

Re: [HACKERS] LIKE INCLUDING COMMENTS code is a flight of fancy

2009-12-22 Thread Andrew Dunstan
Tom Lane wrote: I'm for forbidding comments on index columns. The amount of work required to support the feature fully seems far out of proportion to its value. In any case, if pg_dump drops such comments (which I had forgotten, but it seems true after a quick look at the code), then we

Re: [HACKERS] Tuplestore should remember the memory context it's created in

2009-12-22 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Tom Lane wrote: Got it. So doesn't tuplesort have the same issue? Tuplesort has the same general problem that the caller of puttuple needs to be in the right resource owner. Which ought to be fixed, especially since tuplesort

Re: [HACKERS] Possible patch for better index name choosing

2009-12-22 Thread Tom Lane
I wrote: Attached is a WIP patch for addressing the problems mentioned in this thread: http://archives.postgresql.org/pgsql-hackers/2009-12/msg01764.php ... There is one thing that is not terribly nice about the behavior, which is that CREATE TABLE LIKE INCLUDING INDEXES is unable to

Re: [HACKERS] Backup history file should be replicated in Streaming Replication?

2009-12-22 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: I think we could get away without the backup history file altogether. Hmmm ... actually I was confusing these with timeline history files, which are definitely not something we can drop. You might be right that the backup history

Re: [HACKERS] Backup history file should be replicated in Streaming Replication?

2009-12-22 Thread Aidan Van Dyk
* Heikki Linnakangas heikki.linnakan...@enterprisedb.com [091222 15:47]: I was just having a chat with Magnus this morning, and he asked if the current patch already provides or if it would be possible to write a stand-alone utility to connect to a master and stream WAL files to an archive

Re: [HACKERS] LIKE INCLUDING COMMENTS code is a flight of fancy

2009-12-22 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: Tom Lane wrote: I'm for forbidding comments on index columns. The amount of work required to support the feature fully seems far out of proportion to its value. In any case, if pg_dump drops such comments (which I had forgotten, but it seems true

Re: [HACKERS] Backup history file should be replicated in Streaming Replication?

2009-12-22 Thread Simon Riggs
On Tue, 2009-12-22 at 22:46 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: On Wed, 2009-12-23 at 04:15 +0900, Fujii Masao wrote: On Wed, Dec 23, 2009 at 4:09 AM, Simon Riggs si...@2ndquadrant.com wrote: It's needed to find the place where the backup stopped, so it defines the safe

[HACKERS] join ordering via Simulated Annealing

2009-12-22 Thread Jan Urbański
Hi, I've been playing with using a Simulated Annealing-type algorithm for determinig join ordering for relations. To get into context see http://archives.postgresql.org/pgsql-hackers/2009-05/msg00098.php (there's also a TODO in the wiki). There's a nice paper on that in

Re: [HACKERS] join ordering via Simulated Annealing

2009-12-22 Thread bin wang
I will follow it. Thank you. 2009/12/23 Jan Urba��ski wulc...@wulczer.org Hi, I've been playing with using a Simulated Annealing-type algorithm for determinig join ordering for relations. To get into context see http://archives.postgresql.org/pgsql-hackers/2009-05/msg00098.php (there's

Re: [HACKERS] creating index names automatically?

2009-12-22 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: Could we create an option to create index names automatically, so you'd only have to write CREATE INDEX ON foo (a); which would pick a name like foo_a_idx. Having done all the groundwork to support that nicely, I find that it doesn't work because of

Re: [HACKERS] creating index names automatically?

2009-12-22 Thread Tom Lane
I wrote: The trouble with changing the index attnames for expressions is that it increases the risk of collisions with attnames for regular index columns. You can hit that case today: regression=# create table foo (f1 int, f2 text); CREATE TABLE regression=# create index fooi on foo(f1,

Re: [HACKERS] creating index names automatically?

2009-12-22 Thread Alvaro Herrera
Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: Could we create an option to create index names automatically, so you'd only have to write CREATE INDEX ON foo (a); which would pick a name like foo_a_idx. Having done all the groundwork to support that nicely, I find that it

Re: [HACKERS] creating index names automatically?

2009-12-22 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes: Tom Lane wrote: ... AFAICT, the only way we could support this syntax would be to make ON a reserved word. Or at least more reserved than it is now. We used up all the wiggle room we had by making CONCURRENTLY non-reserved. And here's Simon

Re: [HACKERS] creating index names automatically?

2009-12-22 Thread Tom Lane
I wrote: Hmm. It would actually work if we made CONCURRENTLY reserved instead; and that would fix Simon's gripe too. That's kind of weird from a standards-compliance POV, but in terms of the risk of breaking applications it might be better than reserving ON. Wait a minute. I must have been

Re: [HACKERS] creating index names automatically?

2009-12-22 Thread David E. Wheeler
On Dec 22, 2009, at 7:31 PM, Tom Lane wrote: Wait a minute. I must have been looking at the wrong keyword list --- ON already is reserved. The problem is exactly that it can't tell whether CREATE INDEX CONCURRENTLY ON ... means to default the index name or to create an index named

Re: [HACKERS] join ordering via Simulated Annealing

2009-12-22 Thread Tom Lane
=?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= wulc...@wulczer.org writes: I've been playing with using a Simulated Annealing-type algorithm for determinig join ordering for relations. Cool. The code I have now creates the initial plan by doing something similar to what gimme_tree does in GEQO, but

Re: [HACKERS] creating index names automatically?

2009-12-22 Thread Alvaro Herrera
David E. Wheeler wrote: On Dec 22, 2009, at 7:31 PM, Tom Lane wrote: Wait a minute. I must have been looking at the wrong keyword list --- ON already is reserved. The problem is exactly that it can't tell whether CREATE INDEX CONCURRENTLY ON ... means to default the index name or to

Re: [HACKERS] creating index names automatically?

2009-12-22 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes: David E. Wheeler wrote: +1 if it prevents indexes from being named CONCURRENTLY. Yeah, if you really want to have an index named like that you can use double quotes. Seems a sensible compromise. Well, this will also break tables and columns