Re: [HACKERS] Doc patch making firm recommendation for setting the value of commit_delay

2012-11-15 Thread Albe Laurenz
Peter Geoghegan wrote: Some of you will be aware that I've tried to formulate a good general recommendation for setting the value of commit_delay, in light of the improvements made in 9.3. I previously asked for help finding a methodology for optimising commit_delay [1]. The documentation

Re: [HACKERS] feature proposal - triggers by semantics

2012-11-15 Thread Craig Ringer
On 11/15/2012 03:44 PM, Darren Duncan wrote: As they currently exist, triggers always fire based on certain SQL syntax used, rather than on the semantics of what is actually going on. That's not quite right. COPY fires INSERT triggers, despite never using an explicit INSERT statement. There

Re: [HACKERS] Doc patch making firm recommendation for setting the value of commit_delay

2012-11-15 Thread Greg Smith
On 11/15/12 12:19 AM, Albe Laurenz wrote: If there is an agreement that half the sync time as reported by pg_test_fsync is a good value, would it make sense to have initdb test sync time and preset commit_delay? Peter has validated this against a good range of systems, but it would be

Re: [HACKERS] Doc patch making firm recommendation for setting the value of commit_delay

2012-11-15 Thread Craig Ringer
On 11/15/2012 04:56 PM, Greg Smith wrote: I would rather see this just turn into one of the things a more general tuning tool knew how to do, executing against a fully setup system. Having a useful implementation of commit_delay and useful docs on it seems like enough of a jump forward for

Re: [HACKERS] logical changeset generation v3

2012-11-15 Thread Andres Freund
Hi, On Thursday, November 15, 2012 05:08:26 AM Michael Paquier wrote: Looks like cool stuff @-@ I might be interested in looking at that a bit as I think I will hopefully be hopefully be able to grab some time in the next couple of weeks. Are some of those patches already submitted to a CF?

[HACKERS] Timing events WIP v1

2012-11-15 Thread Greg Smith
Attached is a first WIP saving Timing Events via a new hook, grabbed by a new pg_timing_events contrib module. This only implements a small portion of the RFP spec I presented earlier this month, and is by no means finished code looking for a regular review. This is just enough framework to

Re: [HACKERS] Doc patch making firm recommendation for setting the value of commit_delay

2012-11-15 Thread Magnus Hagander
On Thu, Nov 15, 2012 at 9:56 AM, Greg Smith g...@2ndquadrant.com wrote: On 11/15/12 12:19 AM, Albe Laurenz wrote: If there is an agreement that half the sync time as reported by pg_test_fsync is a good value, would it make sense to have initdb test sync time and preset commit_delay? Peter

Re: [HACKERS] feature proposal - triggers by semantics

2012-11-15 Thread Michael Paquier
A row-level trigger for TRUNCATE does not really make sense, as it would mean that TRUNCATE needs to scan each tuple of the table it needs to interact with to fire its trigger, so it would more or less achieve the same performance as a plain DELETE FROM table;. TRUNCATE is performant because it

Re: [HACKERS] feature proposal - triggers by semantics

2012-11-15 Thread Hannu Krosing
On 11/15/2012 09:48 AM, Craig Ringer wrote: If you want to prevent TRUNCATE, deny the privilege or add a trigger that aborts the command. You can abort the transaction but not skip action as currently it is only possible to skip in ROW level triggers. So I'd modify this request to allow BEFORE

Re: [HACKERS] Materialized views WIP patch

2012-11-15 Thread Greg Smith
On 11/14/12 6:28 PM, Kevin Grittner wrote: - Documentation is incomplete. ... - There are no regression tests yet. Do you have any simple test cases you've been using you could attach? With epic new features like this, when things don't work it's hard to distinguish between that just

Re: [HACKERS] WIP patch for hint bit i/o mitigation

2012-11-15 Thread Amit Kapila
On Thursday, November 15, 2012 2:02 AM Atri Sharma wrote: On Thu, Nov 15, 2012 at 12:42 AM, Atri Sharma atri.j...@gmail.com wrote: On Wed, Nov 7, 2012 at 9:47 PM, Merlin Moncure mmonc...@gmail.com wrote: On Wed, Nov 7, 2012 at 6:01 AM, Amit Kapila amit.kap...@huawei.com wrote: Following the

Re: [HACKERS] Doc patch making firm recommendation for setting the value of commit_delay

2012-11-15 Thread Peter Geoghegan
On 15 November 2012 10:04, Magnus Hagander mag...@hagander.net wrote: I would rather see this just turn into one of the things a more general tuning tool knew how to do, executing against a fully setup system. Having a useful implementation of commit_delay and useful docs on it seems like

Re: [HACKERS] feature proposal - triggers by semantics

2012-11-15 Thread Craig Ringer
On 11/15/2012 06:25 PM, Hannu Krosing wrote: On 11/15/2012 09:48 AM, Craig Ringer wrote: If you want to prevent TRUNCATE, deny the privilege or add a trigger that aborts the command. You can abort the transaction but not skip action as currently it is only possible to skip in ROW level

Re: [HACKERS] Switching timeline over streaming replication

2012-11-15 Thread Heikki Linnakangas
Here's an updated version of this patch, rebased with master, including the recent replication timeout changes, and some other cleanup. On 12.10.2012 09:34, Amit Kapila wrote: The test is finished from myside. one more issue: ... ./pg_basebackup -P -D ../../data_sub -X fetch -p 2303

Re: [HACKERS] Doc patch making firm recommendation for setting the value of commit_delay

2012-11-15 Thread Peter Geoghegan
On 15 November 2012 08:56, Greg Smith g...@2ndquadrant.com wrote: My main concern with this would be the relatively common practice of moving the pg_xlog directory after initdb time. I probably should have increased the default number of seconds that pg_test_fsync runs for, in light of the fact

Re: [HACKERS] Switching timeline over streaming replication

2012-11-15 Thread Heikki Linnakangas
On 10.10.2012 17:26, Amit Kapila wrote: 36.+SendTimeLineHistory(TimeLineHistoryCmd *cmd) { .. if (nread= 0) +ereport(ERROR, +(errcode_for_file_access(), + errmsg(could not read file \%s\:

Re: [HACKERS] Switching timeline over streaming replication

2012-11-15 Thread Heikki Linnakangas
On 15.11.2012 12:44, Heikki Linnakangas wrote: Here's an updated version of this patch, rebased with master, including the recent replication timeout changes, and some other cleanup. On 12.10.2012 09:34, Amit Kapila wrote: The test is finished from myside. one more issue: ...

Re: [HACKERS] Switching timeline over streaming replication

2012-11-15 Thread Amit Kapila
On Thursday, November 15, 2012 6:05 PM Heikki Linnakangas wrote: On 15.11.2012 12:44, Heikki Linnakangas wrote: Here's an updated version of this patch, rebased with master, including the recent replication timeout changes, and some other cleanup. On 12.10.2012 09:34, Amit Kapila wrote:

Re: [HACKERS] Hash id in pg_stat_statements

2012-11-15 Thread Magnus Hagander
On Tue, Oct 2, 2012 at 8:22 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: On 2 October 2012 18:16, Tom Lane t...@sss.pgh.pa.us wrote 1. Why isn't something like md5() on the reported query text an equally good solution for users who want a query hash? Because that does not uniquely

Re: [HACKERS] Hash id in pg_stat_statements

2012-11-15 Thread Peter Geoghegan
On 15 November 2012 13:10, Magnus Hagander mag...@hagander.net wrote: Well, forgive me for pointing this out, but I did propose that the hash be a 64-bit value (which would have necessitated adopting hash_any() to produce 64-bit values), but you rejected the proposal. I arrived at the same

[HACKERS] Re: [BUGS] BUG #7534: walreceiver takes long time to detect n/w breakdown

2012-11-15 Thread Amit kapila
On Monday, November 12, 2012 8:23 PM Fujii Masao wrote: On Fri, Nov 9, 2012 at 3:03 PM, Amit Kapila amit.kap...@huawei.com wrote: On Thursday, November 08, 2012 10:42 PM Fujii Masao wrote: On Thu, Nov 8, 2012 at 5:53 PM, Amit Kapila amit.kap...@huawei.com wrote: On Thursday, November 08, 2012

Re: [HACKERS] [PATCH] Patch to compute Max LSN of Data Pages

2012-11-15 Thread Robert Haas
On Thu, Nov 15, 2012 at 12:08 AM, Amit Kapila amit.kap...@huawei.com wrote: Okay. So as Robert and Alvaro suggested to have it separate utility rather than having options in pg_resetxlog to print MAX LSN seems to be quite appropriate. I am planning to update the patch to make it a separate

Re: [HACKERS] [PATCH 08/14] Store the number of subtransactions in xl_running_xacts separately from toplevel xids

2012-11-15 Thread Simon Riggs
On 14 November 2012 22:17, Andres Freund and...@2ndquadrant.com wrote: To avoid complicating logic we store both, the toplevel and the subxids, in -xip, first -xcnt toplevel ones, and then -subxcnt subxids. That looks good, not much change. Will apply in next few days. Please add me as

Re: [HACKERS] [PATCH 08/14] Store the number of subtransactions in xl_running_xacts separately from toplevel xids

2012-11-15 Thread Andres Freund
On 2012-11-15 09:07:23 -0300, Simon Riggs wrote: On 14 November 2012 22:17, Andres Freund and...@2ndquadrant.com wrote: To avoid complicating logic we store both, the toplevel and the subxids, in -xip, first -xcnt toplevel ones, and then -subxcnt subxids. That looks good, not much change.

Re: [HACKERS] [PATCH 02/14] Add support for a generic wal reading facility dubbed XLogReader

2012-11-15 Thread Heikki Linnakangas
On 15.11.2012 03:17, Andres Freund wrote: Features: - streaming reading/writing - filtering - reassembly of records Reusing the ReadRecord infrastructure in situations where the code that wants to do so is not tightly integrated into xlog.c is rather hard and would require changes to rather

Re: [HACKERS] add -Wlogical-op to standard compiler options?

2012-11-15 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: I think it might be worth adding -Wlogical-op to the standard warning options (for supported compilers, determined by configure test). Does that add any new warnings with the current source code, and if so what? regards, tom lane

[HACKERS] Re: [PATCH 02/14] Add support for a generic wal reading facility dubbed XLogReader

2012-11-15 Thread Andres Freund
On 2012-11-15 16:22:56 +0200, Heikki Linnakangas wrote: On 15.11.2012 03:17, Andres Freund wrote: Features: - streaming reading/writing - filtering - reassembly of records Reusing the ReadRecord infrastructure in situations where the code that wants to do so is not tightly integrated

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-11-15 Thread Heikki Linnakangas
On 23.10.2012 00:29, Alvaro Herrera wrote: Here's an updated version of this patch, which also works in an EXEC_BACKEND environment. (I haven't tested this at all on Windows, but I don't see anything that would create a portability problem there.) Looks good at first glance. Fails on Windows,

Re: [HACKERS] [PATCH 02/14] Add support for a generic wal reading facility dubbed XLogReader

2012-11-15 Thread Alvaro Herrera
Heikki Linnakangas wrote: On 15.11.2012 03:17, Andres Freund wrote: Features: - streaming reading/writing - filtering - reassembly of records Reusing the ReadRecord infrastructure in situations where the code that wants to do so is not tightly integrated into xlog.c is rather hard and

Re: [HACKERS] Switching timeline over streaming replication

2012-11-15 Thread Tom Lane
Heikki Linnakangas hlinnakan...@vmware.com writes: This is a fairly general issue, actually. Looking around, I can see at least two similar cases in existing code, with BasicOpenFile, where we will leak file descriptors on error: Um, don't we automatically clean those up during transaction

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-15 Thread Amit kapila
On Wednesday, November 14, 2012 12:24 AM Robert Haas wrote: On Mon, Nov 12, 2012 at 10:59 PM, Amit kapila amit.kap...@huawei.com wrote: Uh, no, I don't think that's a good idea. IMHO, what we should do is: 1. Read postgresql.conf.auto and remember all the settings we saw. If we see

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-11-15 Thread Alvaro Herrera
Heikki Linnakangas escribió: On 23.10.2012 00:29, Alvaro Herrera wrote: Here's an updated version of this patch, which also works in an EXEC_BACKEND environment. (I haven't tested this at all on Windows, but I don't see anything that would create a portability problem there.) Looks good at

Re: [HACKERS] [PATCH] binary heap implementation

2012-11-15 Thread Robert Haas
On Wed, Nov 14, 2012 at 8:11 AM, Abhijit Menon-Sen a...@2ndquadrant.com wrote: Comments? Suggestions? It could use a run through pgindent. And the header comments are separated by a blank line from the functions to which they are attached, which is not project style. + if (heap-size + 1

Re: [HACKERS] [PATCH 02/14] Add support for a generic wal reading facility dubbed XLogReader

2012-11-15 Thread Andres Freund
On 2012-11-15 11:50:37 -0300, Alvaro Herrera wrote: Heikki Linnakangas wrote: On 15.11.2012 03:17, Andres Freund wrote: Features: - streaming reading/writing - filtering - reassembly of records Reusing the ReadRecord infrastructure in situations where the code that wants to

Re: [HACKERS] Switching timeline over streaming replication

2012-11-15 Thread Heikki Linnakangas
On 15.11.2012 16:55, Tom Lane wrote: Heikki Linnakangashlinnakan...@vmware.com writes: This is a fairly general issue, actually. Looking around, I can see at least two similar cases in existing code, with BasicOpenFile, where we will leak file descriptors on error: Um, don't we automatically

Re: [HACKERS] [PATCH 02/14] Add support for a generic wal reading facility dubbed XLogReader

2012-11-15 Thread Alvaro Herrera
Andres Freund wrote: On 2012-11-15 11:50:37 -0300, Alvaro Herrera wrote: I worked a bit more on that patch of yours, but I neglected to submit it. Did you have something in particular that you wanted changed in it? Could you push your newest version to your git repository or similar?

Re: [HACKERS] [PATCH] binary heap implementation

2012-11-15 Thread Alvaro Herrera
Robert Haas escribió: On Wed, Nov 14, 2012 at 8:11 AM, Abhijit Menon-Sen a...@2ndquadrant.com wrote: Comments? Suggestions? It could use a run through pgindent. And the header comments are separated by a blank line from the functions to which they are attached, which is not project

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-11-15 Thread Heikki Linnakangas
On 15.11.2012 17:10, Alvaro Herrera wrote: Heikki Linnakangas escribió: On 23.10.2012 00:29, Alvaro Herrera wrote: Here's an updated version of this patch, which also works in an EXEC_BACKEND environment. (I haven't tested this at all on Windows, but I don't see anything that would create a

Re: [HACKERS] [PATCH] binary heap implementation

2012-11-15 Thread Will Crawford
Assert(!description of error) is an idiom I've seen more than once, although I'm not sure I understand why it's not written as Robert says with the condition in the brackets (or as a print to STDERR followed by abort() instead). On 15 November 2012 15:11, Robert Haas robertmh...@gmail.com wrote:

Re: [HACKERS] [PATCH 02/14] Add support for a generic wal reading facility dubbed XLogReader

2012-11-15 Thread Heikki Linnakangas
On 15.11.2012 16:50, Alvaro Herrera wrote: Heikki Linnakangas wrote: On 15.11.2012 03:17, Andres Freund wrote: Features: - streaming reading/writing - filtering - reassembly of records Reusing the ReadRecord infrastructure in situations where the code that wants to do so is not tightly

Re: [HACKERS] WIP patch for hint bit i/o mitigation

2012-11-15 Thread Merlin Moncure
On Thu, Nov 15, 2012 at 4:39 AM, Amit Kapila amit.kap...@huawei.com wrote: In each visibility function (except HeapTupleSatisfiesVacuum() ), an addition check has been added to check if the commit status of Xmin or Xmax of a tuple can be retrieved from the cache. 1. From your

Re: [HACKERS] [PATCH] binary heap implementation

2012-11-15 Thread Andrew Dunstan
On 11/15/2012 10:11 AM, Robert Haas wrote: + { + sift_down(heap, i); + } Project style is to omit braces for a single-line body. This comes up a few other places as well. I thought we modified that some years ago, although my memory of it is a bit hazy.

Re: [HACKERS] tuplesort memory usage: grow_memtuples

2012-11-15 Thread Robert Haas
On Tue, Oct 16, 2012 at 4:47 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: The same basic strategy for sizing the tuplesort memtuples array in also exists in tuplestore. I wonder if we should repeat this there? I suppose that that could follow later. I think it'd be a good idea to either

Re: [HACKERS] [PATCH] binary heap implementation

2012-11-15 Thread Robert Haas
On Thu, Nov 15, 2012 at 10:21 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Other than the coding style issues, I think this looks fine. If you can fix that up, I believe I could go ahead and commit this, unless anyone else objects. Does this include the changes in nodeMergeappend.c? I

Re: [HACKERS] [PATCH] binary heap implementation

2012-11-15 Thread Andres Freund
On 2012-11-14 18:41:12 +0530, Abhijit Menon-Sen wrote: There are two or three places in the Postgres source that implement heap sort (e.g. tuplesort.c, nodeMergeAppend.c), and it's also needed by the BDR code. It seemed reasonable to factor out the functionality. pg_dump also contains a binary

Re: [HACKERS] [PATCH 02/14] Add support for a generic wal reading facility dubbed XLogReader

2012-11-15 Thread Alvaro Herrera
Heikki Linnakangas wrote: On 15.11.2012 16:50, Alvaro Herrera wrote: I worked a bit more on that patch of yours, but I neglected to submit it. Did you have something in particular that you wanted changed in it? Off the top of my head, there were a two open items with the patch as I

Re: [HACKERS] WIP patch for hint bit i/o mitigation

2012-11-15 Thread Amit Kapila
On Thursday, November 15, 2012 9:27 PM Merlin Moncure wrote: On Thu, Nov 15, 2012 at 4:39 AM, Amit Kapila amit.kap...@huawei.com wrote: In each visibility function (except HeapTupleSatisfiesVacuum() ), an addition check has been added to check if the commit status of Xmin or Xmax of a

Re: [HACKERS] [PATCH] binary heap implementation

2012-11-15 Thread Alvaro Herrera
Andrew Dunstan escribió: On 11/15/2012 10:11 AM, Robert Haas wrote: +{ +sift_down(heap, i); +} Project style is to omit braces for a single-line body. This comes up a few other places as well. I thought we modified that some years ago, although my memory of it

Re: [HACKERS] [PATCH 03/14] Add simple xlogdump tool

2012-11-15 Thread Peter Eisentraut
On 11/14/12 8:17 PM, Andres Freund wrote: diff --git a/src/bin/Makefile b/src/bin/Makefile index b4dfdba..9992f7a 100644 --- a/src/bin/Makefile +++ b/src/bin/Makefile @@ -14,7 +14,7 @@ top_builddir = ../.. include $(top_builddir)/src/Makefile.global SUBDIRS = initdb pg_ctl pg_dump \ -

Re: [HACKERS] [PATCH] binary heap implementation

2012-11-15 Thread Robert Haas
On Thu, Nov 15, 2012 at 11:22 AM, Andres Freund and...@2ndquadrant.com wrote: On 2012-11-14 18:41:12 +0530, Abhijit Menon-Sen wrote: There are two or three places in the Postgres source that implement heap sort (e.g. tuplesort.c, nodeMergeAppend.c), and it's also needed by the BDR code. It

Re: [HACKERS] [PATCH 03/14] Add simple xlogdump tool

2012-11-15 Thread Andres Freund
On 2012-11-15 11:31:55 -0500, Peter Eisentraut wrote: On 11/14/12 8:17 PM, Andres Freund wrote: diff --git a/src/bin/Makefile b/src/bin/Makefile index b4dfdba..9992f7a 100644 --- a/src/bin/Makefile +++ b/src/bin/Makefile @@ -14,7 +14,7 @@ top_builddir = ../.. include

Re: [HACKERS] WIP patch for hint bit i/o mitigation

2012-11-15 Thread Merlin Moncure
On Thu, Nov 15, 2012 at 10:25 AM, Amit Kapila amit.kap...@huawei.com wrote: IMNSHO. deferring non-critical i/o from foreground process to background process is generally good. Yes, in regard of deferring you are right. However in this case may be when foreground process has to mark page dirty

Re: [HACKERS] [PATCH] binary heap implementation

2012-11-15 Thread Andres Freund
On 2012-11-15 11:37:16 -0500, Robert Haas wrote: On Thu, Nov 15, 2012 at 11:22 AM, Andres Freund and...@2ndquadrant.com wrote: On 2012-11-14 18:41:12 +0530, Abhijit Menon-Sen wrote: There are two or three places in the Postgres source that implement heap sort (e.g. tuplesort.c,

Re: [HACKERS] [PATCH 03/14] Add simple xlogdump tool

2012-11-15 Thread Jeff Janes
On Wed, Nov 14, 2012 at 5:17 PM, Andres Freund and...@2ndquadrant.com wrote: --- src/bin/Makefile| 2 +- src/bin/xlogdump/Makefile | 25 +++ src/bin/xlogdump/xlogdump.c | 468 3 files changed, 494 insertions(+), 1 deletion(-)

Re: [HACKERS] [PATCH] binary heap implementation

2012-11-15 Thread Robert Haas
On Thu, Nov 15, 2012 at 11:50 AM, Andres Freund and...@2ndquadrant.com wrote: Me neither. I was thinking about letting the user allocate enough memory like: binaryheap *heap = palloc(binaryheap_size(/*capacity*/ 10)); binaryheap_init(heap, 10, comparator); But thats pretty ugly. Yeah. I

Re: [HACKERS] [PATCH 03/14] Add simple xlogdump tool

2012-11-15 Thread Andres Freund
On 2012-11-15 09:06:23 -0800, Jeff Janes wrote: On Wed, Nov 14, 2012 at 5:17 PM, Andres Freund and...@2ndquadrant.com wrote: --- src/bin/Makefile| 2 +- src/bin/xlogdump/Makefile | 25 +++ src/bin/xlogdump/xlogdump.c | 468

Re: [HACKERS] tuplesort memory usage: grow_memtuples

2012-11-15 Thread Peter Geoghegan
On 15 November 2012 16:09, Robert Haas robertmh...@gmail.com wrote: [Lots of complicated commentary on tuplesort variables] Whew. In the attached version, I updated the comment to reflect this, and also reversed the order of the if/else block to put the shorter and more common case first,

Re: [HACKERS] tuplesort memory usage: grow_memtuples

2012-11-15 Thread Peter Geoghegan
On 15 November 2012 16:09, Robert Haas robertmh...@gmail.com wrote: I'm still not too sure why this part is OK: /* This may not be our first time through */ if (newmemtupsize = memtupsize) return false; Suppose we enlarge the memtuples array by something other

[HACKERS] another idea for changing global configuration settings from SQL

2012-11-15 Thread Peter Eisentraut
Independent of the discussion of how to edit configuration files from SQL, I had another idea how many of the use cases for this could be handled. We already have the ability to store in pg_db_role_setting configuration settings for specific user, specific database specific user, any database

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-15 Thread Cédric Villemain
Le jeudi 15 novembre 2012 15:48:14, Amit kapila a écrit : On Wednesday, November 14, 2012 12:24 AM Robert Haas wrote: On Mon, Nov 12, 2012 at 10:59 PM, Amit kapila amit.kap...@huawei.com wrote: Uh, no, I don't think that's a good idea. IMHO, what we should do is: 1. Read

Re: [HACKERS] another idea for changing global configuration settings from SQL

2012-11-15 Thread Robert Haas
On Thu, Nov 15, 2012 at 12:53 PM, Peter Eisentraut pete...@gmx.net wrote: Independent of the discussion of how to edit configuration files from SQL, I had another idea how many of the use cases for this could be handled. We already have the ability to store in pg_db_role_setting configuration

Re: [HACKERS] another idea for changing global configuration settings from SQL

2012-11-15 Thread Cédric Villemain
Le jeudi 15 novembre 2012 18:53:15, Peter Eisentraut a écrit : Independent of the discussion of how to edit configuration files from SQL, I had another idea how many of the use cases for this could be handled. We already have the ability to store in pg_db_role_setting configuration settings

Re: [HACKERS] tuplesort memory usage: grow_memtuples

2012-11-15 Thread Robert Haas
On Thu, Nov 15, 2012 at 12:14 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: On 15 November 2012 16:09, Robert Haas robertmh...@gmail.com wrote: [Lots of complicated commentary on tuplesort variables] Whew. In the attached version, I updated the comment to reflect this, and also reversed

[HACKERS] pg_ctl reload -o ....

2012-11-15 Thread Jeff Janes
If I want to change a parameter that affects an auxiliary process (like bgwriter), I can usually get away with doing; pg_ctl restart -o '--setting=new' But sometimes I really need to avoid the restart, because it blows away shared_buffers or for other reasons. I can do pg_ctl reload, but that

Re: [HACKERS] Timing events WIP v1

2012-11-15 Thread Josh Berkus
Extending this to save the key/value set and most of the other data I mentioned before is pretty straightforward. Why not use Hstore? Seriously? It would require merging Hstore into core, but I don't necessarily see that as a bad thing. -- Josh Berkus PostgreSQL Experts Inc.

Re: [HACKERS] Materialized views WIP patch

2012-11-15 Thread Josh Berkus
Kevin, Attached is a patch that is still WIP but that I think is getting pretty close to completion. It is not intended to be the be-all and end-all for materialized views, but the minimum useful feature set -- which is all that I've had time to do for this release. In particular, the view

Re: [HACKERS] another idea for changing global configuration settings from SQL

2012-11-15 Thread Josh Berkus
ApplySetting(InvalidOid, InvalidOid, relsetting, PGC_S_$SOMETHING); in postinit.c, and have some SQL command to modify this setting. The only thing you couldn't handle that way are SIGHUP settings, but the often-cited use cases work_mem, logging, etc. would work. There would also be

Re: [HACKERS] WIP patch: add (PRE|POST)PROCESSOR options to COPY

2012-11-15 Thread Robert Haas
On Wed, Nov 14, 2012 at 10:14 PM, Craig Ringer cr...@2ndquadrant.com wrote: So? You're already handing the keys to the kingdom to anybody who can control the contents of that command line, even if it's only to point at the wrong program. And one man's unexpected side-effect is another man's

Re: [HACKERS] tuplesort memory usage: grow_memtuples

2012-11-15 Thread Peter Geoghegan
On 15 November 2012 18:13, Robert Haas robertmh...@gmail.com wrote: Ah, yeah. I wondered in passing about that but forgot to follow up on it. The problem specifically is that the intermediate result memtupsize * newmemtuples might overflow. I believe that the old memtupsize can never be

Re: [HACKERS] tuplesort memory usage: grow_memtuples

2012-11-15 Thread Robert Haas
On Thu, Nov 15, 2012 at 2:13 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: On 15 November 2012 18:13, Robert Haas robertmh...@gmail.com wrote: Ah, yeah. I wondered in passing about that but forgot to follow up on it. The problem specifically is that the intermediate result memtupsize *

Re: [HACKERS] pg_ctl reload -o ....

2012-11-15 Thread Tom Lane
Jeff Janes jeff.ja...@gmail.com writes: Is there a reason pg_ctl reload shouldn't honor -o ? -o means pass these switches on the postmaster's command line. If you're not restarting the postmaster, you don't get to change its command line. IMO setting stuff on the command line is pretty evil

Re: [HACKERS] WIP patch: add (PRE|POST)PROCESSOR options to COPY

2012-11-15 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Yeah. If we're going to do this at all, and I'm not convinced it's worth the work, I think it's definitely good to support a variant where we specify exactly the things that will be passed to exec(). There's just too many ways to accidentally shoot

Re: [HACKERS] tuplesort memory usage: grow_memtuples

2012-11-15 Thread Peter Geoghegan
On 15 November 2012 19:16, Robert Haas robertmh...@gmail.com wrote: So what's next here? Do you want to work on these issue some more? Or does Jeff? I'd like to see this go in, but I'm not sure I have the bandwidth to do the legwork myself. I'll take another look. No elegant solution

[HACKERS] pg_trgm partial-match

2012-11-15 Thread Fujii Masao
Hi, I'd like to propose to extend pg_trgm so that it can compare a partial-match query key to a GIN index. IOW, I'm thinking to implement the 'comparePartial' GIN method for pg_trgm. Currently, when the query key is less than three characters, we cannot use a GIN index (+ pg_trgm) efficiently,

Re: [HACKERS] Dumping an Extension's Script

2012-11-15 Thread Robert Haas
On Mon, Nov 12, 2012 at 11:00 AM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Please find attached to this email an RFC patch implementing the basics of the pg_dump --extension-script option. After much discussion around the concept of an inline extension, we decided last year that a good

Re: [HACKERS] support for LDAP URLs

2012-11-15 Thread Robert Haas
On Mon, Nov 12, 2012 at 10:38 PM, Peter Eisentraut pete...@gmx.net wrote: Here is a patch to support RFC 2255 LDAP URLs in pg_hba.conf. So, instead of, say host ... ldap ldapserver=ldap.example.net ldapbasedn=dc=example, dc=net ldapsearchattribute=uid you could write host ... ldap

Re: [HACKERS] Enabling Checksums

2012-11-15 Thread Jeff Davis
On Wed, 2012-11-14 at 21:22 -0500, Robert Haas wrote: But there are some practical issues, as Tom points out. Another one is that it's harder for external utilities (like pg_basebackup) to verify checksums. Well, I think the invariant we'd need to maintain is as follows: every page for

Re: [HACKERS] feature proposal - triggers by semantics

2012-11-15 Thread Darren Duncan
Craig Ringer wrote: On 11/15/2012 06:25 PM, Hannu Krosing wrote: On 11/15/2012 09:48 AM, Craig Ringer wrote: If you want to prevent TRUNCATE, deny the privilege or add a trigger that aborts the command. You can abort the transaction but not skip action as currently it is only possible to skip

Re: [HACKERS] tuplesort memory usage: grow_memtuples

2012-11-15 Thread Greg Stark
On Thu, Nov 15, 2012 at 7:36 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: On 15 November 2012 19:16, Robert Haas robertmh...@gmail.com wrote: So what's next here? Do you want to work on these issue some more? Or does Jeff? I'd like to see this go in, but I'm not sure I have the bandwidth

Re: [HACKERS] feature proposal - triggers by semantics

2012-11-15 Thread Christopher Browne
On Thu, Nov 15, 2012 at 2:53 PM, Darren Duncan dar...@darrenduncan.net wrote: I still think the syntax of TRUNCATE FOR EACH ROW would be useful, but if no one agrees... I'm compelled to disagree. What was useful about TRUNCATE in the first place was that it quickly operated against the entire

Re: [HACKERS] Dumping an Extension's Script

2012-11-15 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: Please find attached to this email an RFC patch implementing the basics of the pg_dump --extension-script option. After much discussion around the concept of an inline extension, we decided last year that a good first step would be pg_dump support for

Re: [HACKERS] Timing events WIP v1

2012-11-15 Thread Alvaro Herrera
Greg Smith wrote: -The event queue into a simple array accessed in circular fashion. After realizing that output needed to navigate in the opposite order of element addition, I ended up just putting all the queue navigation code directly into the add/output routines. I'd be happy to use a

Re: [HACKERS] pg_ctl reload -o ....

2012-11-15 Thread Jeff Janes
On Thu, Nov 15, 2012 at 11:26 AM, Tom Lane t...@sss.pgh.pa.us wrote: Jeff Janes jeff.ja...@gmail.com writes: Is there a reason pg_ctl reload shouldn't honor -o ? -o means pass these switches on the postmaster's command line. If you're not restarting the postmaster, you don't get to change

[HACKERS] User control over psql error stream

2012-11-15 Thread Karl O. Pinc
Hi, This patch gives the end user control over psql's error stream. This allows a single psql session to use \o to send both errors and table output to multiple files. Useful when capturing test output, etc. Control is provided via a new estream \pset. Here's the docs.

Re: [HACKERS] Further pg_upgrade analysis for many tables

2012-11-15 Thread Dimitri Fontaine
Alvaro Herrera alvhe...@2ndquadrant.com writes: Could we use some adaptive mechanism here? Say we use a list for the first ten entries, and if an eleventh one comes in, we create a hash table for that one and all subsequent ones. All future calls would have to examine both the list for the

Re: [HACKERS] pg_ctl reload -o ....

2012-11-15 Thread Andrew Dunstan
On 11/15/2012 03:41 PM, Jeff Janes wrote: On Thu, Nov 15, 2012 at 11:26 AM, Tom Lane t...@sss.pgh.pa.us wrote: Jeff Janes jeff.ja...@gmail.com writes: Is there a reason pg_ctl reload shouldn't honor -o ? -o means pass these switches on the postmaster's command line. If you're not restarting

Re: [HACKERS] tuplesort memory usage: grow_memtuples

2012-11-15 Thread Peter Geoghegan
On 15 November 2012 19:54, Greg Stark st...@mit.edu wrote: The only concern I had was about the behaviour after it did the special case. I didn't want it to keep doing the math and trying to grow again a little bit every tuple. I think I was leaning to putting the magic flag back. The

[HACKERS] Partial-index predicate proofs got dumber in 9.2

2012-11-15 Thread Tom Lane
I looked into the optimization regression reported here: http://archives.postgresql.org/pgsql-performance/2012-11/msg00140.php It's easy to reproduce the problem in the regression database: create index ti on tenk1 (fivethous) where fivethous is not null; explain select * from int4_tbl, tenk1

Re: [HACKERS] pg_ctl reload -o ....

2012-11-15 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 11/15/2012 03:41 PM, Jeff Janes wrote: OK. You see pg_ctl as a wrapper around postmaster, while I was viewing it as a thing in itself. You're possibly not aware of its history. Before 8.0 it was a shell script and its use was seen as rather

Re: [HACKERS] another idea for changing global configuration settings from SQL

2012-11-15 Thread Dimitri Fontaine
Peter Eisentraut pete...@gmx.net writes: The existing infrastructure would also support any user, any database (= all the time) There would also be the advantage that pg_dumpall would save these settings. Thoughts? That's brilliant. +1. Regards, -- Dimitri Fontaine http://2ndQuadrant.fr

Re: [HACKERS] feature proposal - triggers by semantics

2012-11-15 Thread Dimitri Fontaine
Darren Duncan dar...@darrenduncan.net writes: So, I'm partly proposing a specific narrow new feature, TRUNCATE FOR EACH ROW Kevin has been proposing that we consider an alternative approach in some other cases that I think would work better for you, too. Namely, to have access to OLD and NEW in

Re: [HACKERS] Hash id in pg_stat_statements

2012-11-15 Thread Dimitri Fontaine
Peter Geoghegan pe...@2ndquadrant.com writes: should expose the hash. The need to aggregate historical statistics just doesn't appreciably alter things here, I feel. The number of discrete queries that an application will execute in a week just isn't that different from the number that it will

Re: [HACKERS] Further pg_upgrade analysis for many tables

2012-11-15 Thread Tom Lane
Dimitri Fontaine dimi...@2ndquadrant.fr writes: Alvaro Herrera alvhe...@2ndquadrant.com writes: Could we use some adaptive mechanism here? Say we use a list for the first ten entries, and if an eleventh one comes in, we create a hash table for that one and all subsequent ones. All future

Re: [HACKERS] add -Wlogical-op to standard compiler options?

2012-11-15 Thread Peter Eisentraut
On 11/15/12 9:40 AM, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: I think it might be worth adding -Wlogical-op to the standard warning options (for supported compilers, determined by configure test). Does that add any new warnings with the current source code, and if so what?

Re: [HACKERS] feature proposal - triggers by semantics

2012-11-15 Thread Darren Duncan
Dimitri Fontaine wrote: Darren Duncan dar...@darrenduncan.net writes: So, I'm partly proposing a specific narrow new feature, TRUNCATE FOR EACH ROW Kevin has been proposing that we consider an alternative approach in some other cases that I think would work better for you, too. Namely, to

Re: [HACKERS] add -Wlogical-op to standard compiler options?

2012-11-15 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On 11/15/12 9:40 AM, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: I think it might be worth adding -Wlogical-op to the standard warning options (for supported compilers, determined by configure test). Does that add any new warnings with

Re: [HACKERS] another idea for changing global configuration settings from SQL

2012-11-15 Thread Magnus Hagander
On Thu, Nov 15, 2012 at 6:53 PM, Peter Eisentraut pete...@gmx.net wrote: Independent of the discussion of how to edit configuration files from SQL, I had another idea how many of the use cases for this could be handled. We already have the ability to store in pg_db_role_setting configuration

Re: [HACKERS] Index only scans wiki page

2012-11-15 Thread Peter Geoghegan
On 13 November 2012 01:25, Peter Geoghegan pe...@2ndquadrant.com wrote: Attached is a piece I wrote on the feature. That might form the basis of a new wiki page. Since no one else moved on this, I've completely replaced the existing page with the contents of the user-orientated document I

Re: [HACKERS] autovacuum truncate exclusive lock round two

2012-11-15 Thread Dimitri Fontaine
Jan Wieck janwi...@yahoo.com writes: Use this lmgr feature inside count_nondeletable_pages() of vacuumlazy.c to periodically check, if there is a conflicting lock request waiting. If not, keep going. If there is a waiter, truncate the relation to the point checked thus far, release the

Re: [HACKERS] autovacuum truncate exclusive lock round two

2012-11-15 Thread Alvaro Herrera
Dimitri Fontaine wrote: Jan Wieck janwi...@yahoo.com writes: Use this lmgr feature inside count_nondeletable_pages() of vacuumlazy.c to periodically check, if there is a conflicting lock request waiting. If not, keep going. If there is a waiter, truncate the relation to the point checked

  1   2   >