Re: [HACKERS] auto-sizing wal_buffers

2011-01-15 Thread Fujii Masao
On Sat, Jan 15, 2011 at 3:51 PM, Greg Smith g...@2ndquadrant.com wrote: Agreed on both fronts.  Attached patch does the magic.  Also available in branch walbuffers from git://github.com/greg2ndQuadrant/postgres.git +intXLOGbuffersMin = 8; XLOGbuffersMin is a fixed value. I

Re: [HACKERS] We need to log aborted autovacuums

2011-01-15 Thread Greg Smith
Josh Berkus wrote: The lack of vacuum could be occurring for any of 4 reasons: 1) Locking 2) You have a lot of tables and not enough autovac_workers / too much sleep time 3) You need to autovac this particular table more frequently, since it gets dirtied really fast 4) The table has been set

Re: [HACKERS] Spread checkpoint sync

2011-01-15 Thread Greg Smith
Robert Haas wrote: On Tue, Nov 30, 2010 at 3:29 PM, Greg Smith g...@2ndquadrant.com wrote: One of the ideas Simon and I had been considering at one point was adding some better de-duplication logic to the fsync absorb code, which I'm reminded by the pattern here might be helpful

Re: [HACKERS] Recovery control functions

2011-01-15 Thread Fujii Masao
On Fri, Jan 14, 2011 at 8:15 PM, Simon Riggs si...@2ndquadrant.com wrote: On Fri, 2011-01-14 at 11:09 +, Simon Riggs wrote: Functions to control recovery, to aid PITR and Hot Standby. pg_is_xlog_replay_paused() pg_xlog_replay_pause() pg_xlog_replay_resume() recovery.conf parameter:

Re: [HACKERS] Recovery control functions

2011-01-15 Thread Fujii Masao
On Fri, Jan 14, 2011 at 9:00 PM, Simon Riggs si...@2ndquadrant.com wrote: How hard would it be to have a pg_xlog_replay_until(xlog location or timestamp), to have it resume recovery up to that point and then pause again? You can already do that for timestamps. You mean using

Re: [HACKERS] Recovery control functions

2011-01-15 Thread Simon Riggs
On Sat, 2011-01-15 at 20:11 +0900, Fujii Masao wrote: On Fri, Jan 14, 2011 at 9:00 PM, Simon Riggs si...@2ndquadrant.com wrote: How hard would it be to have a pg_xlog_replay_until(xlog location or timestamp), to have it resume recovery up to that point and then pause again? You can

Re: [HACKERS] LOCK for non-tables

2011-01-15 Thread Florian Pflug
On Jan15, 2011, at 02:03 , Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: Me, too. But I don't agree with your particular choice of small syntax adjustment. Maybe we should just let the issue drop for now. Nobody's actually complained about this that I can recall; it's just a

Re: [HACKERS] Recovery control functions

2011-01-15 Thread Simon Riggs
On Sat, 2011-01-15 at 20:00 +0900, Fujii Masao wrote: On Fri, Jan 14, 2011 at 8:15 PM, Simon Riggs si...@2ndquadrant.com wrote: On Fri, 2011-01-14 at 11:09 +, Simon Riggs wrote: Functions to control recovery, to aid PITR and Hot Standby. pg_is_xlog_replay_paused()

Re: [HACKERS] kill -KILL: What happens?

2011-01-15 Thread Florian Pflug
On Jan14, 2011, at 17:45 , Robert Haas wrote: On Fri, Jan 14, 2011 at 11:28 AM, Florian Pflug f...@phlo.org wrote: I gather that the behaviour we want is for normal backends to exit once the postmaster is gone, and for utility processes (bgwriter, ...) to exit once all the backends are gone.

Re: [HACKERS] LOCK for non-tables

2011-01-15 Thread Simon Riggs
On Sat, 2011-01-15 at 12:19 +0100, Florian Pflug wrote: On Jan15, 2011, at 02:03 , Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: Me, too. But I don't agree with your particular choice of small syntax adjustment. Maybe we should just let the issue drop for now. Nobody's

Re: [HACKERS] [RRR] reviewers needed!

2011-01-15 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: lot more people pitch in. I just assigned myself the review of those two patches: https://commitfest.postgresql.org/action/patch_view?id=475 Streaming base backups https://commitfest.postgresql.org/action/patch_view?id=502 FOR KEY LOCK

Re: [HACKERS] LOCK for non-tables

2011-01-15 Thread Magnus Hagander
On Jan 15, 2011 12:30 PM, Simon Riggs si...@2ndquadrant.com wrote: On Sat, 2011-01-15 at 12:19 +0100, Florian Pflug wrote: On Jan15, 2011, at 02:03 , Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: Me, too. But I don't agree with your particular choice of small syntax

Re: [HACKERS] pov 1.0 is released, testers with huge schemas needed

2011-01-15 Thread Joel Jacobson
2011/1/14 Joel Jacobson j...@gluefinance.com: To install: git clone g...@github.com:gluefinance/pov.git Ops, typo, to do this without my ssh key, you need to do: git clone git://github.com/gluefinance/pov.git -- Best regards, Joel Jacobson Glue Finance -- Sent via pgsql-hackers mailing

Re: [HACKERS] Spread checkpoint sync

2011-01-15 Thread Robert Haas
On Sat, Jan 15, 2011 at 5:47 AM, Greg Smith g...@2ndquadrant.com wrote: No toe damage, this is great, I hadn't gotten to coding for this angle yet at all.  Suffering from an overload of ideas and (mostly wasted) test data, so thanks for exploring this concept and proving it works. Yeah -

Re: [HACKERS] LOCK for non-tables

2011-01-15 Thread Robert Haas
On Sat, Jan 15, 2011 at 6:29 AM, Simon Riggs si...@2ndquadrant.com wrote: On Sat, 2011-01-15 at 12:19 +0100, Florian Pflug wrote: On Jan15, 2011, at 02:03 , Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: Me, too.  But I don't agree with your particular choice of small syntax

Re: [HACKERS] kill -KILL: What happens?

2011-01-15 Thread Robert Haas
On Sat, Jan 15, 2011 at 6:27 AM, Florian Pflug f...@phlo.org wrote: On Jan14, 2011, at 17:45 , Robert Haas wrote: On Fri, Jan 14, 2011 at 11:28 AM, Florian Pflug f...@phlo.org wrote: I gather that the behaviour we want is for normal backends to exit once the postmaster is gone, and for utility

Re: [HACKERS] Add support for logging the current role

2011-01-15 Thread Robert Haas
On Fri, Jan 14, 2011 at 9:24 PM, Andrew Dunstan and...@dunslane.net wrote: On 01/14/2011 08:41 PM, Robert Haas wrote: I think we're in the process of designing a manned mission to Mars to solve the problem that our shoelaces are untied. What's your suggestion, then? If there's a practical way

Re: [HACKERS] Spread checkpoint sync

2011-01-15 Thread Simon Riggs
On Sat, 2011-01-15 at 05:47 -0500, Greg Smith wrote: Robert Haas wrote: On Tue, Nov 30, 2010 at 3:29 PM, Greg Smith g...@2ndquadrant.com wrote: One of the ideas Simon and I had been considering at one point was adding some better de-duplication logic to the fsync absorb code, which

Re: [HACKERS] ALTER TYPE 0: Introduction; test cases

2011-01-15 Thread Robert Haas
On Sat, Jan 15, 2011 at 1:30 AM, Noah Misch n...@leadboat.com wrote: Here's v2 based on your feedback. I pruned test coverage down to just the highlights.  By the end of patch series, the net change becomes +67 to alter_table.sql and +111 to alter_table.out.   The alter_table.out delta is

Re: [HACKERS] Spread checkpoint sync

2011-01-15 Thread Robert Haas
On Sat, Jan 15, 2011 at 8:55 AM, Simon Riggs si...@2ndquadrant.com wrote: On Sat, 2011-01-15 at 05:47 -0500, Greg Smith wrote: Robert Haas wrote: On Tue, Nov 30, 2010 at 3:29 PM, Greg Smith g...@2ndquadrant.com wrote: One of the ideas Simon and I had been considering at one point was

Re: [HACKERS] Bug in pg_describe_object, patch v2

2011-01-15 Thread Robert Haas
On Thu, Jan 13, 2011 at 1:24 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Jan 12, 2011 at 7:47 PM, Tom Lane t...@sss.pgh.pa.us wrote: IMO, what this patch needs is to not output the types unless they are actually different from the default (which can

Re: [HACKERS] Spread checkpoint sync

2011-01-15 Thread Greg Smith
Robert Haas wrote: Idea #2: At the beginning of a checkpoint when we scan all the buffers, count the number of buffers that need to be synced for each relation. Use the same hashtable that we use for tracking pending fsync requests. Then, interleave the writes and the fsyncs... Idea #3: Stick

Re: [HACKERS] Spread checkpoint sync

2011-01-15 Thread Robert Haas
On Sat, Jan 15, 2011 at 9:25 AM, Greg Smith g...@2ndquadrant.com wrote: Once upon a time we got a patch from Itagaki Takahiro whose purpose was to sort writes before sending them out: http://archives.postgresql.org/pgsql-hackers/2007-06/msg00541.php Ah, a fine idea! Which has very low odds

Re: [HACKERS] Streaming base backups

2011-01-15 Thread Heikki Linnakangas
On 14.01.2011 13:38, Magnus Hagander wrote: On Fri, Jan 14, 2011 at 11:19, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 14.01.2011 08:45, Fujii Masao wrote: 1. Smart shutdown is requested while walsender is sending a backup. 2. Shutdown causes archiver to end.

[HACKERS] LAST CALL FOR 9.1

2011-01-15 Thread Robert Haas
On Sat, Jan 15, 2011 at 9:25 AM, Greg Smith g...@2ndquadrant.com wrote: I'm going to share it before someone announces a deadline has passed or something.  (whistling) Speaking of that, it is now Saturday, January 15, 2011 and CommitFest 2011-01 is now in progress. If you have any patches that

Re: [HACKERS] limiting hint bit I/O

2011-01-15 Thread Martijn van Oosterhout
On Fri, Jan 14, 2011 at 05:24:31PM -0800, Josh Berkus wrote: On 1/14/11 11:51 AM, Tom Lane wrote: The people whose tables are mostly insert-only complain about it, but that's not the majority of our userbase IMO. We just happen to have a couple of particularly vocal ones, like Berkus.

Re: [HACKERS] ALTER TYPE 0: Introduction; test cases

2011-01-15 Thread Noah Misch
On Sat, Jan 15, 2011 at 08:57:30AM -0500, Robert Haas wrote: On Sat, Jan 15, 2011 at 1:30 AM, Noah Misch n...@leadboat.com wrote: Here's v2 based on your feedback. I pruned test coverage down to just the highlights. ?By the end of patch series, the net change becomes +67 to

Re: [HACKERS] LOCK for non-tables

2011-01-15 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Sat, Jan 15, 2011 at 6:29 AM, Simon Riggs si...@2ndquadrant.com wrote: I think we should have a section in the release notes on Deprecated Features, noting that certain things will be removed later and should be changed now and not relied upon in

Re: [HACKERS] Streaming base backups

2011-01-15 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: BTW, I just spotted a small race condition between creating a new table space and base backup. We take a snapshot of all the tablespaces in pg_tblspc before calling pg_start_backup(). If someone creates a new tablespace and puts

Re: [HACKERS] Spread checkpoint sync

2011-01-15 Thread Greg Smith
Robert Haas wrote: I'll believe it when I see it. How about this: a 1 a 2 sync a b 1 b 2 sync b c 1 c 2 sync c Or maybe some variant, where we become willing to fsync a file a certain number of seconds after writing the last block, or when all the writes are done, whichever comes first.

Re: [HACKERS] Streaming base backups

2011-01-15 Thread Heikki Linnakangas
On 15.01.2011 17:30, Tom Lane wrote: Heikki Linnakangasheikki.linnakan...@enterprisedb.com writes: BTW, I just spotted a small race condition between creating a new table space and base backup. We take a snapshot of all the tablespaces in pg_tblspc before calling pg_start_backup(). If someone

Re: [HACKERS] Bug in pg_describe_object, patch v2

2011-01-15 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Thu, Jan 13, 2011 at 1:24 PM, Tom Lane t...@sss.pgh.pa.us wrote: Read the CREATE OPERATOR CLASS source code.  (It likely would be best to refactor that a bit so it would expose some way to obtain the implied defaults --- I don't think that's done

Re: [HACKERS] kill -KILL: What happens?

2011-01-15 Thread Florian Pflug
On Jan14, 2011, at 17:45 , Robert Haas wrote: On Fri, Jan 14, 2011 at 11:28 AM, Florian Pflug f...@phlo.org wrote: I gather that the behaviour we want is for normal backends to exit once the postmaster is gone, and for utility processes (bgwriter, ...) to exit once all the backends are gone.

Re: [HACKERS] Streaming base backups

2011-01-15 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: On 15.01.2011 17:30, Tom Lane wrote: So what? The needed actions will be covered by WAL replay. No, they won't, if pg_base_backup() is called *after* getting the list of tablespaces. Ah. Then the fix is to change the order in

Re: [HACKERS] Add support for logging the current role

2011-01-15 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Fri, Jan 14, 2011 at 9:24 PM, Andrew Dunstan and...@dunslane.net wrote: What's your suggestion, then? If there's a practical way to add the requested escape, add it to the text format and leave reengineering the CSV format for another day. Yeah, I

Re: [HACKERS] We need to log aborted autovacuums

2011-01-15 Thread Tom Lane
Greg Smith g...@2ndquadrant.com writes: Does try_relation_open need to have a lock acquisition timeout when AV is calling it? Hmm. I think when looking at the AV code, I've always subconsciously assumed that try_relation_open would fail immediately if it couldn't get the lock. That certainly

Re: [HACKERS] log_hostname and pg_stat_activity

2011-01-15 Thread Peter Eisentraut
On tor, 2010-12-23 at 22:21 +0100, Magnus Hagander wrote: On Thu, Dec 23, 2010 at 22:09, Peter Eisentraut pete...@gmx.net wrote: Somehow I fantasized that log_hostname would also turn pg_stat_activity.client_addr into names instead of IP addresses. It doesn't, but perhaps it should? It

Re: [HACKERS] Add support for logging the current role

2011-01-15 Thread Andrew Dunstan
On 01/15/2011 11:08 AM, Tom Lane wrote: Robert Haasrobertmh...@gmail.com writes: On Fri, Jan 14, 2011 at 9:24 PM, Andrew Dunstanand...@dunslane.net wrote: What's your suggestion, then? If there's a practical way to add the requested escape, add it to the text format and leave

Re: [HACKERS] auto-sizing wal_buffers

2011-01-15 Thread Greg Smith
Fujii Masao wrote: +intXLOGbuffersMin = 8; XLOGbuffersMin is a fixed value. I think that defining it as a macro rather than a variable seems better. + if (XLOGbuffers 2048) + XLOGbuffers = 2048; Using XLOG_SEG_SIZE/XLOG_BLCKSZ rather

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2011-01-15 Thread Bruce Momjian
Josh Berkus wrote: On 12/6/10 6:13 PM, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: OK, patch coming then. Right now test_fsync aborts when O_DIRECT fails. What should I have it do instead? Report that it fails, and keep testing the other methods. Patch attached.

Re: [HACKERS] Bug in pg_describe_object, patch v2

2011-01-15 Thread Joel Jacobson
2011/1/15 Tom Lane t...@sss.pgh.pa.us: But I can read the handwriting on the wall: if I want this done right, I'm going to have to do it myself.                        regards, tom lane Excellently put! I will with pride steal that phrase and use it whenever I run into the same situation

[HACKERS] textarray option for file FDW

2011-01-15 Thread Andrew Dunstan
I've been waiting for the latest FDW patches as patiently as I can, and I've been reviewing them this morning, in particular the file_fdw patch and how it interacts with the newly exposed COPY API. Overall it seems to be a whole lot cleaner, and the wholesale duplication of the copy code is

Re: [HACKERS] textarray option for file FDW

2011-01-15 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: ... So now I'd like to add a new option to it: textarray. This option would require that the foreign table have exactly one field, of type text[], and would compose all the field strings read from the file for each record into the array (however

[HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-15 Thread Noah Misch
Hello Pavel, I'm reviewing this patch for CommitFest 2011-01. The patch seems fully desirable. It appropriately contains no documentation updates. It contains no new tests, and that's probably fine, too; I can't think of any corner cases where this would do something other than work correctly

Re: [HACKERS] textarray option for file FDW

2011-01-15 Thread Andrew Dunstan
On 01/15/2011 12:44 PM, Tom Lane wrote: Andrew Dunstanand...@dunslane.net writes: ... So now I'd like to add a new option to it: textarray. This option would require that the foreign table have exactly one field, of type text[], and would compose all the field strings read from the file for

[HACKERS] pg_basebackup for streaming base backups

2011-01-15 Thread Magnus Hagander
This patch creates pg_basebackup in bin/, being a client program for the streaming base backup feature. I think it's more or less done now. I've again split it out of pg_streamrecv, because it had very little shared code with that (basically just the PQconnectdb() wrapper). One thing I'm

Re: [HACKERS] Streaming base backups

2011-01-15 Thread Magnus Hagander
On Sat, Jan 15, 2011 at 16:54, Tom Lane t...@sss.pgh.pa.us wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: On 15.01.2011 17:30, Tom Lane wrote: So what?  The needed actions will be covered by WAL replay. No, they won't, if pg_base_backup() is called *after* getting the

Re: [HACKERS] textarray option for file FDW

2011-01-15 Thread Itagaki Takahiro
On Sun, Jan 16, 2011 at 02:29, Andrew Dunstan and...@dunslane.net wrote: textarray. This option would require that the foreign table have exactly one field, of type text[], and would compose all the field strings read from the file for each record into the array (however many there are).  

Re: [HACKERS] Streaming base backups

2011-01-15 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: Something like this to fix? or is this going to put those warnings by stupid versions of gcc back? Possibly. If so, I'll fix it --- I have an old gcc to test against here. regards, tom lane -- Sent via pgsql-hackers mailing

Re: [HACKERS] Streaming base backups

2011-01-15 Thread Magnus Hagander
On Sat, Jan 15, 2011 at 19:27, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: Something like this to fix? or is this going to put those warnings by stupid versions of gcc back? Possibly.  If so, I'll fix it --- I have an old gcc to test against here. Ok,

Re: [HACKERS] textarray option for file FDW

2011-01-15 Thread Andrew Dunstan
On 01/15/2011 01:24 PM, Itagaki Takahiro wrote: On Sun, Jan 16, 2011 at 02:29, Andrew Dunstanand...@dunslane.net wrote: textarray. This option would require that the foreign table have exactly one field, of type text[], and would compose all the field strings read from the file for each

Re: [HACKERS] Visual Studio 2010/Windows SDK 7.1 support

2011-01-15 Thread Magnus Hagander
On Mon, Jan 3, 2011 at 16:45, Brar Piening b...@gmx.de wrote: On Mon, 3 Jan 2011 10:44:19 +0100, Magnus Hagander mag...@hagander.net wrote: Yeah, it looks that way - it's missing the ordering of the contrib I'll run it once for that now, and then please rebase your patch on top of that -

Re: [HACKERS] LAST CALL FOR 9.1

2011-01-15 Thread David E. Wheeler
On Jan 15, 2011, at 6:50 AM, Robert Haas wrote: It is really already too late for us to be seriously considering integrating sync rep into 9.1. It will lead to another enormous beta period during which the tree will be closed to new patches and everyone will complain, or else we'll open the

Re: [HACKERS] LAST CALL FOR 9.1

2011-01-15 Thread Robert Haas
On Sat, Jan 15, 2011 at 1:59 PM, David E. Wheeler da...@kineticode.com wrote: On Jan 15, 2011, at 6:50 AM, Robert Haas wrote: It is really already too late for us to be seriously considering integrating sync rep into 9.1.  It will lead to another enormous beta period during which the tree will

Re: [HACKERS] estimating # of distinct values

2011-01-15 Thread Tomas Vondra
Hi, a short update regarding the ndistinct stream estimators - I've implemented the estimators described in the papers I've mentioned in my previous posts. If you want try it, the sources are available at github, at http://tvondra.github.com/pg_estimator (ignore the page, I have to update it,

Re: [HACKERS] LOCK for non-tables

2011-01-15 Thread Robert Haas
On Sat, Jan 15, 2011 at 10:25 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Sat, Jan 15, 2011 at 6:29 AM, Simon Riggs si...@2ndquadrant.com wrote: I think we should have a section in the release notes on Deprecated Features, noting that certain things will

Re: [HACKERS] We need to log aborted autovacuums

2011-01-15 Thread Robert Haas
On Sat, Jan 15, 2011 at 11:14 AM, Tom Lane t...@sss.pgh.pa.us wrote: Greg Smith g...@2ndquadrant.com writes: Does try_relation_open need to have a lock acquisition timeout when AV is calling it? Hmm.  I think when looking at the AV code, I've always subconsciously assumed that

Re: [HACKERS] ALTER TYPE 0: Introduction; test cases

2011-01-15 Thread Robert Haas
On Sat, Jan 15, 2011 at 10:25 AM, Noah Misch n...@leadboat.com wrote: Do you value test coverage so little? If you're asking whether I think real-world usability is more important than test coverage, then yes. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL

[HACKERS] test_fsync made modular

2011-01-15 Thread Bruce Momjian
I have modified test_fsync to use modular C so there is less duplicate code and it can be enhanced easier. Applied patch attached, though the diff is larger than the C file so you might want to just look at the C file in git. -- Bruce Momjian br...@momjian.ushttp://momjian.us

Re: [HACKERS] Visual Studio 2010/Windows SDK 7.1 support

2011-01-15 Thread Andrew Chernow
On 1/15/2011 1:42 PM, Magnus Hagander wrote: On Mon, Jan 3, 2011 at 16:45, Brar Pieningb...@gmx.de wrote: On Mon, 3 Jan 2011 10:44:19 +0100, Magnus Hagandermag...@hagander.net wrote: Yeah, it looks that way - it's missing the ordering of the contrib I'll run it once for that now, and then

Re: [HACKERS] textarray option for file FDW

2011-01-15 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: Why is this a good thing? It seems like it would accomplish little except to defeat the SQL type system entirely. It simply allows to have the classical ETL problem solved directly in the database server, in SQL. That's huge. Regards, -- Dimitri Fontaine

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-15 Thread Dimitri Fontaine
Hi, I have an unexpected 5 mins window to do a first reading of the patch, so here goes the quick doc and comments proof reading of it. :) Magnus Hagander mag...@hagander.net writes: This patch creates pg_basebackup in bin/, being a client program for the streaming base backup feature. Great!

Re: [HACKERS] Review: Fix snapshot taking inconsistencies

2011-01-15 Thread Marko Tiikkaja
On 2010-10-21 3:32 PM +0200, Marko Tiikkaja wrote: patch Here's the patch rebased against the master. No code changes since the last patch I sent. Regards, Marko Tiikkaja *** a/src/backend/catalog/pg_proc.c --- b/src/backend/catalog/pg_proc.c *** *** 759,765

Re: [HACKERS] EXPLAIN and nfiltered

2011-01-15 Thread Marko Tiikkaja
On 2010-11-18 5:45 PM +0200, Marko Tiikkaja wrote: Here's a patch for showing in EXPLAIN ANALYZE the number of rows a plan qual filtered from a node's input. Rebased against master. Regards, Marko Tiikkaja *** a/src/backend/commands/explain.c --- b/src/backend/commands/explain.c

Re: [HACKERS] LOCK for non-tables

2011-01-15 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Sat, Jan 15, 2011 at 10:25 AM, Tom Lane t...@sss.pgh.pa.us wrote: I suggest also marking each item with a release in which we intend to do it, so we don't have to try to remember whether a reasonable amount of time has elapsed. You mean like the

Re: [HACKERS] Spread checkpoint sync

2011-01-15 Thread Simon Riggs
On Sat, 2011-01-15 at 09:15 -0500, Robert Haas wrote: On Sat, Jan 15, 2011 at 8:55 AM, Simon Riggs si...@2ndquadrant.com wrote: On Sat, 2011-01-15 at 05:47 -0500, Greg Smith wrote: Robert Haas wrote: On Tue, Nov 30, 2010 at 3:29 PM, Greg Smith g...@2ndquadrant.com wrote: One of the

[HACKERS] Include WAL in base backup

2011-01-15 Thread Magnus Hagander
Here's a cutdown version of the idea about including WAL in the base backup. What I initially wanted was to introduce a way to guarantee that the required WAL (with some sort of cutoff of course) would be available for the backup, but I ran out of time for that. We can always add that later. For

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-15 Thread Magnus Hagander
On Sat, Jan 15, 2011 at 21:16, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Hi, I have an unexpected 5 mins window to do a first reading of the patch, so here goes the quick doc and comments proof reading of it. :) :-) Magnus Hagander mag...@hagander.net writes: This patch creates

Re: [HACKERS] LOCK for non-tables

2011-01-15 Thread Robert Haas
On Sat, Jan 15, 2011 at 3:37 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Sat, Jan 15, 2011 at 10:25 AM, Tom Lane t...@sss.pgh.pa.us wrote: I suggest also marking each item with a release in which we intend to do it, so we don't have to try to remember

Re: [HACKERS] Transaction-scope advisory locks

2011-01-15 Thread Marko Tiikkaja
On 2010-12-14 12:52 AM +0200, Marko Tiikkaja wrote: patch Here's the latest version of the patch. It now uses the API proposed by Simon, but still lacks documentation changes, which I'm going to send tomorrow. Regards, Marko Tiikkaja *** a/src/backend/storage/lmgr/lock.c ---

[HACKERS] plperlu problem with utf8 [REVIEW]

2011-01-15 Thread Andy Colson
This is a review of plperl encoding issues https://commitfest.postgresql.org/action/patch_view?id=452 Purpose: Your database uses one encoding, and passes data to perl in the same encoding, which perl is not prepared for (it assumes UTF-8). This patch makes sure data is encoded

Re: [HACKERS] [PATCH] Return command tag 'REPLACE X' for CREATE OR REPLACE statements.

2011-01-15 Thread Marti Raudsepp
Here's an updated patch that reports command status back to ProcessUtility via 'bool' return value. I was a bit unsure about using bool return values because it's not immediately obvious what true or false refer to, but defining a new enum seemed like overkill, so I went with bool anyway. Any

Re: [HACKERS] Spread checkpoint sync

2011-01-15 Thread Robert Haas
On Sat, Jan 15, 2011 at 10:31 AM, Greg Smith g...@2ndquadrant.com wrote: That's going to give worse performance than the current code in some cases. OK. How does the checkpoint target give you any time to sync them?  Unless you squeeze the writes together more tightly, but that seems sketchy.

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-15 Thread Euler Taveira de Oliveira
Em 15-01-2011 15:10, Magnus Hagander escreveu: One thing I'm thinking about - right now the tool just takes -c conninfo to connect to the database. Should it instead be taught to take the connection parameters that for example pg_dump does - one for each of host, port, user, password?

Re: [HACKERS] Sync Rep for 2011CF1

2011-01-15 Thread Magnus Hagander
On Sat, Jan 15, 2011 at 22:40, Simon Riggs si...@2ndquadrant.com wrote: Here's the latest patch for sync rep. From here, I will be developing the patch further on public git repository towards commit. My expectation is that commit is at least 2 That's great. Just one tiny detail - which

Re: [HACKERS] ALTER TABLE ... REPLACE WITH

2011-01-15 Thread Simon Riggs
On Tue, 2010-12-14 at 19:48 +, Simon Riggs wrote: On Tue, 2010-12-14 at 11:34 -0800, Josh Berkus wrote: As for the utility of this command: there is no question that I would use it. I'm not sure I like the syntax (I'd prefer REPLACE TABLE WITH _), but that's painting the bike

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-15 Thread Dimitri Fontaine
Magnus Hagander mag...@hagander.net writes: Not sure pg_ctl clone would be the proper name, since it's not actually a clone at this point (it might be with the second patch I ust posted that includes the WAL files) Let's keep the clone name for the client that makes it all then :) That

Re: [HACKERS] ALTER TYPE 0: Introduction; test cases

2011-01-15 Thread Noah Misch
On Sat, Jan 15, 2011 at 02:31:21PM -0500, Robert Haas wrote: On Sat, Jan 15, 2011 at 10:25 AM, Noah Misch n...@leadboat.com wrote: Do you value test coverage so little? If you're asking whether I think real-world usability is more important than test coverage, then yes. No, I wasn't asking

[HACKERS] What happened to open_sync_without_odirect?

2011-01-15 Thread Josh Berkus
Last I remember, we were going to add this as an option. But I don't see a patch in the queue. Am I missing it? Was I supposed to write it? -- -- Josh Berkus PostgreSQL Experts Inc.

Re: [HACKERS] Include WAL in base backup

2011-01-15 Thread Dimitri Fontaine
Magnus Hagander mag...@hagander.net writes: Here's a cutdown version of the idea about including WAL in the base backup. What I initially wanted was to introduce a way to guarantee that the required WAL (with some sort of cutoff of course) would be available for the backup, but I ran out of

Re: [HACKERS] auto-sizing wal_buffers

2011-01-15 Thread Josh Berkus
On 1/14/11 10:51 PM, Greg Smith wrote: ! Since the data is written out to disk at every transaction commit, ! the setting many only need to be be large enough to hold the amount ! of WAL data generated by one typical transaction. Larger values, ! typically

Re: [HACKERS] arrays as pl/perl input arguments [PATCH]

2011-01-15 Thread Alex Hunsaker
On Wed, Jan 12, 2011 at 13:04, Alexey Klyukin al...@commandprompt.com wrote: On Jan 12, 2011, at 8:52 PM, David E. Wheeler wrote: On Jan 12, 2011, at 5:14 AM, Alexey Klyukin wrote: You mean packing both a string representation and a reference to a single SV * value? Dunno, I'm not a guts

Re: [HACKERS] Spread checkpoint sync

2011-01-15 Thread Greg Smith
Robert Haas wrote: That seems like a bad idea - don't we routinely recommend that people crank this up to 0.9? You'd be effectively bounding the upper range of this setting to a value to the less than the lowest value we recommend anyone use today. I was just giving an example of how I

Re: [HACKERS] LAST CALL FOR 9.1

2011-01-15 Thread Josh Berkus
On 1/15/11 6:50 AM, Robert Haas wrote: It is really already too late for us to be seriously considering integrating sync rep into 9.1. It will lead to another enormous beta period during which the tree will be closed to new patches and everyone will complain, or else we'll open the tree for

Re: [HACKERS] limiting hint bit I/O

2011-01-15 Thread Josh Berkus
If the problem is that all the freezing happens at once, then ISTM the solution is to add a random factor. Say, when a tuple just passes the lower threshold it has a 1% chance of being frozen. The chance grows until it is 100% as it reaches the upper threshold. Doesn't have to be random; it

Re: [HACKERS] Include WAL in base backup

2011-01-15 Thread Magnus Hagander
On Sat, Jan 15, 2011 at 23:32, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Magnus Hagander mag...@hagander.net writes: Here's a cutdown version of the idea about including WAL in the base backup. What I initially wanted was to introduce a way to guarantee that the required WAL (with some

[HACKERS] .gitignore file needed for new replication parser

2011-01-15 Thread Kevin Grittner
diff --git a/src/backend/replication/.gitignore b/src/backend/replication/.gitignore new file mode 100644 index 000..a0332b2 --- /dev/null +++ b/src/backend/replication/.gitignore @@ -0,0 +1,3 @@ +/repl_gram.c +/repl_gram.h +/repl_scanner.c -- Sent via pgsql-hackers mailing list

[HACKERS] New test_fsync messages for direct I/O

2011-01-15 Thread Bruce Momjian
Josh Berkus wrote: Greg, This is interesting, because test_fsync consistently reported a rate of about half this when using open_datasync instead of the equal performance I'm getting from the database. I'll see if I can reproduce that further, but it's no reason to be concerned about

Re: [HACKERS] What happened to open_sync_without_odirect?

2011-01-15 Thread Bruce Momjian
Josh Berkus wrote: Last I remember, we were going to add this as an option. But I don't see a patch in the queue. Am I missing it? Was I supposed to write it? I don't know, but let me add that I am confused how this would look to users. In many cases, kernels don't even support O_DIRECT, so

Re: [HACKERS] textarray option for file FDW

2011-01-15 Thread Andrew Dunstan
On 01/15/2011 12:29 PM, Andrew Dunstan wrote: I've been waiting for the latest FDW patches as patiently as I can, and I've been reviewing them this morning, in particular the file_fdw patch and how it interacts with the newly exposed COPY API. Overall it seems to be a whole lot cleaner,

[HACKERS] review: FDW API

2011-01-15 Thread Jan Urbański
Hi, what follows is a review of the FDW API patch from http://archives.postgresql.org/message-id/20110114212358.82c7.69899...@metrosystems.co.jp All three patches apply cleanly and compile without warnings. Regression tests pass. Let me go patch by patch, starting with the first one that adds

Re: [HACKERS] psql: Add \dL to show languages

2011-01-15 Thread Andreas Karlsson
Hi Josh, Here is my review of this patch for the commitfest. Review of https://commitfest.postgresql.org/action/patch_view?id=439 Contents and Purpose This patch adds the \dL command in psql to list the procedual languages. To me this seems like a useful addition to the

Re: [HACKERS] Streaming base backups

2011-01-15 Thread Tatsuo Ishii
When do the standby launch its walreceiver? It would be extra-nice for the base backup tool to optionally continue streaming WALs until the standby starts doing it itself, so that wal_keep_segments is really deprecated. No idea how feasible that is, though. Good point. I have been always

Re: [HACKERS] Bug in pg_describe_object, patch v2

2011-01-15 Thread Andreas Karlsson
On Sat, 2011-01-15 at 10:36 -0500, Tom Lane wrote: But I can read the handwriting on the wall: if I want this done right, I'm going to have to do it myself. regards, tom lane Do I understand you correctly if I interpret what you would like to see is the same format used

Re: [HACKERS] Spread checkpoint sync

2011-01-15 Thread Marti Raudsepp
On Sat, Jan 15, 2011 at 14:05, Robert Haas robertmh...@gmail.com wrote: Idea #4: For ext3 filesystems that like to dump the entire buffer cache instead of only the requested file, write a little daemon that runs alongside of (and completely indepdently of) PostgreSQL.  Every 30 s, it opens a

Re: [HACKERS] Spread checkpoint sync

2011-01-15 Thread Robert Haas
On Sat, Jan 15, 2011 at 5:57 PM, Greg Smith g...@2ndquadrant.com wrote: I was just giving an example of how I might do an initial split.  There's a checkpoint happening now at time T; we have a rough idea that it needs to be finished before some upcoming time T+D.  Currently with default

Re: [HACKERS] Spread checkpoint sync

2011-01-15 Thread Greg Smith
Robert Haas wrote: What is the basis for thinking that the sync should get the same amount of time as the writes? That seems pretty arbitrary. Right now, you're allowing 3 seconds per fsync, which could be a lot more or a lot less than 40% of the total checkpoint time... Just that it's where