Re: [HACKERS] automating CF submissions (was xlog location arithmetic)

2012-01-15 Thread Magnus Hagander
On Sun, Jan 15, 2012 at 05:44, Greg Smith g...@2ndquadrant.com wrote: On 01/14/2012 10:49 PM, Gurjeet Singh wrote: So lets make it easy for the patch submitter to start the process. I propose that we have a page in the CF application where people can upload/attach the patch, and the app posts

[HACKERS] Vacuum rate limit in KBps

2012-01-15 Thread Greg Smith
So far the reaction I've gotten from my recent submission to make autovacuum log its read/write in MB/s has been rather positive. I've been surprised at the unprecedented (to me at least) amount of backporting onto big production systems it's gotten. There is a whole lot of pent up

Re: [HACKERS] Our poll() based WaitLatch implementation is broken

2012-01-15 Thread Peter Geoghegan
On 15 January 2012 07:26, Peter Geoghegan pe...@2ndquadrant.com wrote: Build Postgres master, on Linux or another platform that will use the poll() implementation rather than the older select(). Send the Postmaster SIGKILL. Observe that the WAL Writer lives on, representing a denial of service

Re: [HACKERS] automating CF submissions (was xlog location arithmetic)

2012-01-15 Thread Greg Smith
On 01/15/2012 03:17 AM, Magnus Hagander wrote: And FWIW, I'd find it a lot more useful for the CF app to have the ability to post *reviews* in it, that would end up being properly threaded. Next you'll be saying we should have some sort of web application to help with the whole review

Re: [HACKERS] automating CF submissions (was xlog location arithmetic)

2012-01-15 Thread Magnus Hagander
On Sun, Jan 15, 2012 at 09:37, Greg Smith g...@2ndquadrant.com wrote: On 01/15/2012 03:17 AM, Magnus Hagander wrote: And FWIW, I'd find it a lot more useful for the CF app to have the ability to post *reviews* in it, that would end up being properly threaded. Next you'll be saying we

[HACKERS] pgstat documentation tables

2012-01-15 Thread Magnus Hagander
Right now we have a single table on http://www.postgresql.org/docs/devel/static/monitoring-stats.html#MONITORING-STATS-VIEWS that lists all our statistics views. It makes for difficult to parse descriptions like One row only, showing cluster-wide statistics from the background writer: number of

Re: [HACKERS] IDLE in transaction introspection

2012-01-15 Thread Greg Smith
On 01/12/2012 11:57 AM, Scott Mead wrote: Pretty delayed, but please find the attached patch that addresses all the issues discussed. The docs on this v4 look like they suffered a patch order problem here. In the v3, you added a whole table describing the pg_stat_activity documentation in

Re: [HACKERS] pgstat documentation tables

2012-01-15 Thread Greg Smith
On 01/15/2012 03:51 AM, Magnus Hagander wrote: I'd like to turn that into one table for each view, with two columns, one being the name the other one being the description. That'll also make it possible to expand on the descriptions without making it completley unreadable, should we want to.

[HACKERS] Review of: pg_stat_statements with query tree normalization

2012-01-15 Thread Daniel Farina
I've *finally* gotten around to reviewing this patch. My first step was to de-bitrot it very slightly. More on that in a moment. After that, I tried using it. Installation worked nicely -- I did CREATE EXTENSION and then tried reading from pg_stat_statements. I was then given an error message

Re: [HACKERS] Review of: pg_stat_statements with query tree normalization

2012-01-15 Thread Peter Geoghegan
On 15 January 2012 11:41, Daniel Farina dan...@heroku.com wrote: I've *finally* gotten around to reviewing this patch. My first step was to de-bitrot it very slightly.  More on that in a moment. Thanks. Prepared statements are less informative, unless one knows their naming convention:

Re: [HACKERS] exit() calls in libraries

2012-01-15 Thread Peter Eisentraut
On mån, 2011-12-05 at 15:04 -0300, Alvaro Herrera wrote: Having had to battle some exit() calls in the PHP interpreter back when I was working in PL/php, I agree that they shouldn't be there -- abort() seems more appropriate if the system is truly busted. As for the fr-print.c code, I'm not

Re: [HACKERS] age(xid) on hot standby

2012-01-15 Thread Peter Eisentraut
On ons, 2011-12-28 at 14:35 -0500, Tom Lane wrote: Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Peter Eisentraut's message of mié dic 28 15:04:09 -0300 2011: On a hot standby, this fails with: ERROR: cannot assign TransactionIds during recovery I think we could just

Re: [HACKERS] lots of unused variable warnings in assert-free builds

2012-01-15 Thread Andrew Dunstan
On 01/15/2012 01:37 AM, Tom Lane wrote: Peter Eisentrautpete...@gmx.net writes: I see that in some places our code already uses #ifdef USE_ASSERT_CHECKING, presumably to hide similar issues. But in most cases using this would significantly butcher the code. I found that adding

Re: [HACKERS] lots of unused variable warnings in assert-free builds

2012-01-15 Thread Simon Riggs
On Sun, Jan 15, 2012 at 1:14 PM, Andrew Dunstan and...@dunslane.net wrote: On 01/15/2012 01:37 AM, Tom Lane wrote: Peter Eisentrautpete...@gmx.net  writes: I see that in some places our code already uses #ifdef USE_ASSERT_CHECKING, presumably to hide similar issues.  But in most cases

Re: [HACKERS] pgstat documentation tables

2012-01-15 Thread Magnus Hagander
On Sun, Jan 15, 2012 at 10:35, Greg Smith g...@2ndquadrant.com wrote: On 01/15/2012 03:51 AM, Magnus Hagander wrote: I'd like to turn that into one table for each view, with two columns, one being the name the other one being the description. That'll also make it possible to expand on the

Re: [HACKERS] Patch to allow users to kill their own queries

2012-01-15 Thread Magnus Hagander
On Fri, Jan 13, 2012 at 14:46, Magnus Hagander mag...@hagander.net wrote: On Fri, Jan 13, 2012 at 14:42, Greg Smith g...@2ndquadrant.com wrote: On 01/03/2012 12:59 PM, Tom Lane wrote: Noah Mischn...@leadboat.com  writes: Regarding the other message, avoid composing a translated message from

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2012-01-15 Thread Hitoshi Harada
On Sat, Jan 14, 2012 at 6:48 AM, Robert Haas robertmh...@gmail.com wrote: On Sat, Jan 14, 2012 at 5:25 AM, Hitoshi Harada umi.tan...@gmail.com wrote: The patch looks ok, though I wonder if we could have a way to release the lock on namespace much before the end of transaction. Well, that wold

Re: [HACKERS] JSON for PG 9.2

2012-01-15 Thread Andrew Dunstan
On 01/14/2012 03:06 PM, Andrew Dunstan wrote: OK, here's a patch that does both query_to_json and array_to_json, along with docs and regression tests. It include Robert's original patch, although I can produce a differential patch if required. It can also be pulled from

[HACKERS] WAL Restore process during recovery

2012-01-15 Thread Simon Riggs
WALRestore process asynchronously executes restore_command while recovery continues working. Overlaps downloading of next WAL file to reduce time delays in file based archive recovery. Handles cases of file-only and streaming/file correctly. --  Simon Riggs  

Re: [HACKERS] pgstat documentation tables

2012-01-15 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: Right now we have a single table on http://www.postgresql.org/docs/devel/static/monitoring-stats.html#MONITORING-STATS-VIEWS that lists all our statistics views ... I'd like to turn that into one table for each view, Please follow the style already

Re: [HACKERS] 9.2 Reviewfest - logging hooks patch

2012-01-15 Thread Martin Pihlak
On 01/13/2012 05:29 AM, Christopher Maujean wrote: Patch http://archives.postgresql.org/message-id/4edbefbb.9070...@gmail.com In reviewing the referenced patch, we found that it has no developer documentation and no regression tests. While it compiles and installs, there is nothing to

Re: [HACKERS] libpq: PQcmdStatus, PQcmdTuples signatures can be painlessly improved

2012-01-15 Thread Peter Eisentraut
On tis, 2012-01-10 at 14:10 -0500, Alex Goncharov wrote: | Note that const PGresult * would only warn against changing the | fields It would not warn, it would err (the compilation should fail). No, const violations generally only produce warnings. | of the PGresult struct. It doesn't

Re: [HACKERS] Dry-run mode for pg_archivecleanup

2012-01-15 Thread Gabriele Bartolini
Hi Josh, Il 15/01/12 01:13, Josh Kupershmidt ha scritto: I have signed on to review this patch for the 2012-01 CF. The patch applies cleanly, includes the necessary documentation, and implements a useful feature. Thank you for dedicating your time to this matter. I think the actual debugging

Re: [HACKERS] pg_trigger_depth() v3 (was: TG_DEPTH)

2012-01-15 Thread Kevin Grittner
Kevin Grittner wrote: Florian Pflug wrote: The trigger depth is incremented before calling the trigger function in ExecCallTriggerFunc() and decremented right afterwards, which seems fine - apart from the fact that the decrement is skipped in case of an error. The patch handles that by

Re: [HACKERS] Our poll() based WaitLatch implementation is broken

2012-01-15 Thread Heikki Linnakangas
On 15.01.2012 09:26, Peter Geoghegan wrote: Build Postgres master, on Linux or another platform that will use the poll() implementation rather than the older select(). Send the Postmaster SIGKILL. Observe that the WAL Writer lives on, representing a denial of service as it stays attached to

Re: [HACKERS] pg_basebackup is not checking IDENTIFY_SYSTEM numbre of columns

2012-01-15 Thread Jaime Casanova
On Wed, Jan 11, 2012 at 5:11 PM, Magnus Hagander mag...@hagander.net wrote: No, no reason. Adding such a check would be a good idea. ok. patch attached, it also adds a few PQclear() calls before disconnect_and_exit(). btw, in BaseBackup() in line 1149 (after the patch is applied) there is an

Re: [HACKERS] [PATCH] Support for foreign keys with arrays

2012-01-15 Thread Marco Nenciarini
Hello, Il giorno dom, 11/12/2011 alle 19.45 -0500, Noah Misch ha scritto: On Sat, Dec 10, 2011 at 09:47:53AM +0100, Gabriele Bartolini wrote: So, here is a summary: --- - - | ON| ON| Action | DELETE | UPDATE |

Re: [HACKERS] Vacuum rate limit in KBps

2012-01-15 Thread Heikki Linnakangas
On 15.01.2012 10:24, Greg Smith wrote: That got me thinking: if MB/s is what everyone wants to monitor, can we provide a UI to set these parameters that way too? The attached patch is a bit rough still, but it does that. The key was recognizing that the cost delay plus cost limit can be

Re: [HACKERS] [COMMITTERS] pgsql: Fix breakage from earlier plperl fix.

2012-01-15 Thread Andrew Dunstan
On 01/13/2012 07:09 PM, Alex Hunsaker wrote: On Fri, Jan 13, 2012 at 16:07, Andrew Dunstanand...@dunslane.net wrote: On 01/12/2012 09:28 PM, Alex Hunsaker wrote: Util.c/o not depending on plperl_helpers.h was also throwing me for a loop so I fixed it and SPI.c... Thoughts? Basically

Re: [HACKERS] Standalone synchronous master

2012-01-15 Thread Jeff Janes
On Fri, Jan 13, 2012 at 9:50 AM, Tom Lane t...@sss.pgh.pa.us wrote: Jeff Janes jeff.ja...@gmail.com writes: I don't understand why this is controversial.  In the current code, if you have a master and a single sync standby, and the master disappears and you promote the standby, now the new

Re: [HACKERS] Standalone synchronous master

2012-01-15 Thread Jeff Janes
On Fri, Jan 13, 2012 at 10:12 AM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Jeff Janes jeff.ja...@gmail.com wrote:\ I don't understand why this is controversial. I'm having a hard time seeing why this is considered a feature.  It seems to me what is being proposed is a mode with no

Re: [HACKERS] Dry-run mode for pg_archivecleanup

2012-01-15 Thread Josh Kupershmidt
On Sun, Jan 15, 2012 at 3:02 PM, Gabriele Bartolini gabriele.bartol...@2ndquadrant.it wrote: My actual intention was to have the filename as output of the command, in order to easily pipe it to another script. Hence my first choice was to use the stdout channel, considering also that

[HACKERS] Group commit, revised

2012-01-15 Thread Peter Geoghegan
Attached is a patch that myself and Simon Riggs collaborated on. I took the group commit patch that Simon posted to the list back in November, and partially rewrote it. Here is that original thread: http://archives.postgresql.org/pgsql-hackers/2011-11/msg00802.php I've also attached the results

[HACKERS] SKIP LOCKED DATA

2012-01-15 Thread Thomas Munro
Hi Apologies for posting about new vapourware features for distant future releases at a very busy time in the cycle for 9.2... I am wondering out loud whether I am brave enough to try to propose SKIP LOCKED DATA support and would be grateful for any feedback and/or {en|dis}couragement. I don't

Re: [HACKERS] disable prompting by default in createuser

2012-01-15 Thread Josh Kupershmidt
On Thu, Dec 22, 2011 at 2:26 PM, Peter Eisentraut pete...@gmx.net wrote: On lör, 2011-11-26 at 01:28 +0200, Peter Eisentraut wrote: I propose that we change createuser so that it does not prompt for anything by default.  We can arrange options so that you can get prompts for whatever is

[HACKERS] Memory usage during sorting

2012-01-15 Thread Jeff Janes
In tuplesort.c, it initially reads tuples into memory until availMem is exhausted. It then switches to the tape sort algorithm, and allocates buffer space for each tape it will use. This substantially over-runs the allowedMem, and drives availMem negative. It works off this deficit by writing

Re: [HACKERS] pgstat documentation tables

2012-01-15 Thread Greg Smith
On 01/15/2012 12:20 PM, Tom Lane wrote: Please follow the style already used for system catalogs; ie I think there should be a summary table with one entry per view, and then a separate description and table-of-columns for each view. Yes, that's a perfect precedent. I think the easiest path

Re: [HACKERS] archive_keepalive_command

2012-01-15 Thread Simon Riggs
On Fri, Dec 16, 2011 at 3:01 PM, Simon Riggs si...@2ndquadrant.com wrote: archive_command and restore_command describe how to ship WAL files to/from an archive. When there is nothing to ship, we delay sending WAL files. When no WAL files, the standby has no information at all. To provide

Re: [HACKERS] Vacuum rate limit in KBps

2012-01-15 Thread Greg Smith
On 01/15/2012 04:17 PM, Heikki Linnakangas wrote: I think it makes more sense to use the max read rate as the main knob, rather than write rate. That's because the max read rate is higher than the write rate, when you don't need to dirty pages. Or do you think saturating the I/O system with

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-01-15 Thread Greg Smith
On 01/15/2012 05:14 PM, Ants Aasma wrote: I hope that having a tool to measure the overhead and check the sanity of clock sources is enough to answer the worries about the potential performance hit. We could also check that the clock source is fast enough on start-up/when the guc is changed, but

Re: [HACKERS] separate initdb -A options for local and host

2012-01-15 Thread Robert Haas
On Sat, Jan 14, 2012 at 5:18 PM, Peter Eisentraut pete...@gmx.net wrote: On lör, 2011-11-26 at 01:20 +0200, Peter Eisentraut wrote: I think it would be useful to have separate initdb -A options for local and host entries.  In 9.1, we went out of our way to separate the peer and ident methods,

[HACKERS] triggered_change_notification v3

2012-01-15 Thread Kevin Grittner
Attached is a version of a previously posted patch which has been modified based on on-list feedback from Álvaro. This is a generalized trigger function which can be used as an AFTER EACH ROW trigger on any table which has a primary key, and will send notifications of operations for which the

Re: [HACKERS] SKIP LOCKED DATA

2012-01-15 Thread Tom Lane
Thomas Munro mu...@ip9.org writes: I am wondering out loud whether I am brave enough to try to propose SKIP LOCKED DATA support and would be grateful for any feedback and/or {en|dis}couragement. I don't see it on the todo list, and didn't find signs of others working on this (did I miss

Re: [HACKERS] patch : Allow toast tables to be moved to a different tablespace

2012-01-15 Thread Jaime Casanova
On Tue, Dec 13, 2011 at 12:29 PM, Julien Tachoires jul...@gmail.com wrote: OK, considering that, I don't see any way to handle the case raised by Jaime :( Did you consider what Álvaro suggested? anyway, seems is too late for this commitfest. are you intending to resume work on this for the

[HACKERS] Publish checkpoint timing and sync files summary data to pg_stat_bgwriter

2012-01-15 Thread Greg Smith
One of the most useful bits of feedback on how well checkpoint I/O is going is the amount of time taken to sync files to disk. Right now the only way to get that is to parse the logs. The attached patch publishes the most useful three bits of data you could only get from log_checkpoints

Re: [HACKERS] Publish checkpoint timing and sync files summary data to pg_stat_bgwriter

2012-01-15 Thread Greg Smith
On 01/16/2012 01:28 AM, Greg Smith wrote: -I can't tell for sure if this is working properly when log_checkpoints is off. This now collects checkpoint end time data in all cases, whereas before it ignored that work if log_checkpoints was off. ...and there's at least one I missed located

[HACKERS] Checkpoint sync pause

2012-01-15 Thread Greg Smith
Last year at this point, I submitted an increasingly complicated checkpoint sync spreading feature. I wasn't able to prove any repeatable drop in sync time latency from those patches. While that was going on, and continuing into recently, the production server that started all this with its