Re: [HACKERS] On partitioning

2014-12-04 Thread Amit Langote
Hi, From: Alvaro Herrera [mailto:alvhe...@2ndquadrant.com] Amit Langote wrote: From: Robert Haas [mailto:robertmh...@gmail.com] What is an overflow partition and why do we want that? That would be a default partition. That is, where the tuples that don't belong elsewhere (other

Re: [HACKERS] libpq pipelining

2014-12-04 Thread Heikki Linnakangas
On 12/04/2014 03:11 AM, Matt Newell wrote: The recent discussion about pipelining in the jodbc driver prompted me to look at what it would take for libpq. Great! I have a proof of concept patch working. The results are even more promising than I expected. While it's true that many

Re: [HACKERS] [COMMITTERS] pgsql: Keep track of transaction commit timestamps

2014-12-04 Thread Heikki Linnakangas
On 12/03/2014 04:54 PM, Alvaro Herrera wrote: ir commit timestamp directly as they commit, or an external transaction c Sorry, I'm late to the party, but here's some random comments on this after a quick review: * The whole concept of a node ID seems to be undocumented, and unused. No-one

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-12-04 Thread Anssi Kääriäinen
On Wed, 2014-11-26 at 16:59 -0800, Peter Geoghegan wrote: On Mon, Nov 24, 2014 at 1:03 PM, Peter Geoghegan p...@heroku.com wrote: Looks like the consensus is that we should have RETURNING project updated tuples too, then. Attached revision, v1.5, establishes this behavior (as always, there

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-12-04 Thread Craig Ringer
On 12/04/2014 07:07 PM, Anssi Kääriäinen wrote: On Wed, 2014-11-26 at 16:59 -0800, Peter Geoghegan wrote: On Mon, Nov 24, 2014 at 1:03 PM, Peter Geoghegan p...@heroku.com wrote: Looks like the consensus is that we should have RETURNING project updated tuples too, then. Attached revision,

Re: [HACKERS] [COMMITTERS] pgsql: Keep track of transaction commit timestamps

2014-12-04 Thread Petr Jelinek
On 04/12/14 10:42, Heikki Linnakangas wrote: On 12/03/2014 04:54 PM, Alvaro Herrera wrote: ir commit timestamp directly as they commit, or an external transaction c Sorry, I'm late to the party, but here's some random comments on this after a quick review: * The whole concept of a node ID

Re: [HACKERS] [COMMITTERS] pgsql: Keep track of transaction commit timestamps

2014-12-04 Thread Heikki Linnakangas
On 12/04/2014 01:16 PM, Petr Jelinek wrote: On 04/12/14 10:42, Heikki Linnakangas wrote: On 12/03/2014 04:54 PM, Alvaro Herrera wrote: ir commit timestamp directly as they commit, or an external transaction c Sorry, I'm late to the party, but here's some random comments on this after a quick

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-04 Thread Michael Paquier
On Thu, Dec 4, 2014 at 7:36 PM, Rahila Syed rahilasyed...@gmail.com wrote: IIUC, forcibly written fpws are not exposed to user , so is it worthwhile to add a GUC similar to full_page_writes in order to control a feature which is unexposed to user in first place? If full page writes is set

Re: [HACKERS] [COMMITTERS] pgsql: Keep track of transaction commit timestamps

2014-12-04 Thread Petr Jelinek
On 04/12/14 12:26, Heikki Linnakangas wrote: On 12/04/2014 01:16 PM, Petr Jelinek wrote: On 04/12/14 10:42, Heikki Linnakangas wrote: On 12/03/2014 04:54 PM, Alvaro Herrera wrote: ir commit timestamp directly as they commit, or an external transaction c Sorry, I'm late to the party, but

Re: [HACKERS] [COMMITTERS] pgsql: Keep track of transaction commit timestamps

2014-12-04 Thread Heikki Linnakangas
On 12/04/2014 01:47 PM, Petr Jelinek wrote: On 04/12/14 12:26, Heikki Linnakangas wrote: On 12/04/2014 01:16 PM, Petr Jelinek wrote: On 04/12/14 10:42, Heikki Linnakangas wrote: On 12/03/2014 04:54 PM, Alvaro Herrera wrote: ir commit timestamp directly as they commit, or an external

Re: [HACKERS] SSL regression test suite

2014-12-04 Thread Heikki Linnakangas
On 10/06/2014 04:21 PM, Heikki Linnakangas wrote: Here's a new version of the SSL regression suite I wrote earlier. It now specifies both host and hostaddr in the connection string as Andres suggested, so it no longer requires changes to network configuration. I added a bunch of tests for the

Re: [HACKERS] Add shutdown_at_recovery_target option to recovery.conf

2014-12-04 Thread Petr Jelinek
On 02/12/14 18:59, Robert Haas wrote: On Fri, Nov 28, 2014 at 11:59 AM, Petr Jelinek p...@2ndquadrant.com wrote: I'm a bit late to the party, but wouldn't recovery_target_action = ... have been a better name for this? It'd be in line with the other recovery_target_* parameters, and also a bit

Re: [HACKERS] Add shutdown_at_recovery_target option to recovery.conf

2014-12-04 Thread Simon Riggs
On 4 December 2014 at 22:13, Petr Jelinek p...@2ndquadrant.com wrote: On 02/12/14 18:59, Robert Haas wrote: On Fri, Nov 28, 2014 at 11:59 AM, Petr Jelinek p...@2ndquadrant.com wrote: I'm a bit late to the party, but wouldn't recovery_target_action = ... have been a better name for this?

Re: [HACKERS] Add shutdown_at_recovery_target option to recovery.conf

2014-12-04 Thread Michael Paquier
On Thu, Dec 4, 2014 at 10:13 PM, Petr Jelinek p...@2ndquadrant.com wrote: On 02/12/14 18:59, Robert Haas wrote: On Fri, Nov 28, 2014 at 11:59 AM, Petr Jelinek p...@2ndquadrant.com wrote: I'm a bit late to the party, but wouldn't recovery_target_action = ... have been a better name for

Re: [HACKERS] Add shutdown_at_recovery_target option to recovery.conf

2014-12-04 Thread Michael Paquier
On Thu, Dec 4, 2014 at 10:45 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Thu, Dec 4, 2014 at 10:13 PM, Petr Jelinek p...@2ndquadrant.com wrote: On 02/12/14 18:59, Robert Haas wrote: On Fri, Nov 28, 2014 at 11:59 AM, Petr Jelinek p...@2ndquadrant.com wrote: I'm a bit

Re: [HACKERS] [COMMITTERS] pgsql: Keep track of transaction commit timestamps

2014-12-04 Thread Alex Shulgin
Heikki Linnakangas hlinnakan...@vmware.com writes: On 12/03/2014 04:54 PM, Alvaro Herrera wrote: ir commit timestamp directly as they commit, or an external transaction c Sorry, I'm late to the party, but here's some random comments on this after a quick review: Also this commit breaks

Re: [HACKERS] New wal format distorts pg_xlogdump --stats

2014-12-04 Thread Heikki Linnakangas
On 11/25/2014 05:36 AM, Andres Freund wrote: Hi, The new WAL format calculates FPI vs plain record data like: rec_len = XLogRecGetDataLen(record) + SizeOfXLogRecord; fpi_len = record-decoded_record-xl_tot_len - rec_len; Due to the amount of data now handled outside the main

Re: [HACKERS] libpq pipelining

2014-12-04 Thread Craig Ringer
On 12/04/2014 05:08 PM, Heikki Linnakangas wrote: A good API is crucial for this. It should make it easy to write an application that does pipelining, and to handle all the error conditions in a predictable way. I'd suggest that you write the documentation first, before writing any code, so

Re: [HACKERS] [COMMITTERS] pgsql: Keep track of transaction commit timestamps

2014-12-04 Thread Alvaro Herrera
Alex Shulgin wrote: Also this commit breaks initdb of `make check' for me: creating template1 database in /home/ash/build/postgresql/HEAD/src/test/regress/./tmp_check/data/base/1 ... TRAP: FailedAssertion(!(((xmax) = ((TransactionId) 3))), File:

Re: [HACKERS] make check-world regress failed

2014-12-04 Thread Heikki Linnakangas
On 11/23/2014 08:37 PM, Vladimir Koković wrote: PostgreSQL check-world regress failed with current GIT HEAD on my Kubuntu 14.10. uname -a Linux vlD-kuci 3.16.0-24-generic #32-Ubuntu SMP Tue Oct 28 13:13:18 UTC 2014 i686 athlon i686 GNU/Linux gdb -d

Re: [HACKERS] [COMMITTERS] pgsql: Keep track of transaction commit timestamps

2014-12-04 Thread Alex Shulgin
Alvaro Herrera alvhe...@2ndquadrant.com writes: Alex Shulgin wrote: Also this commit breaks initdb of `make check' for me: creating template1 database in /home/ash/build/postgresql/HEAD/src/test/regress/./tmp_check/data/base/1 ... TRAP: FailedAssertion(!(((xmax) = ((TransactionId) 3))),

Re: [HACKERS] [COMMITTERS] pgsql: Keep track of transaction commit timestamps

2014-12-04 Thread Alex Shulgin
Alvaro Herrera alvhe...@2ndquadrant.com writes: Uh, that's odd. Can you please get a stack trace? Do you have unusual settings or a patched build? Is there a way to pause the bootstrap process so I can attach gdb to it? -- Alex -- Sent via pgsql-hackers mailing list

Re: [HACKERS] [COMMITTERS] pgsql: Keep track of transaction commit timestamps

2014-12-04 Thread Craig Ringer
On 12/04/2014 10:50 PM, Alex Shulgin wrote: Is there a way to pause the bootstrap process so I can attach gdb to it? With a newer gdb, you can instead tell gdb to follow all forks. I wrote some notes on it recently. http://blog.2ndquadrant.com/processes-breakpoints-watchpoints-postgresql/ I've

Re: [HACKERS] [COMMITTERS] pgsql: Keep track of transaction commit timestamps

2014-12-04 Thread Alex Shulgin
Craig Ringer cr...@2ndquadrant.com writes: On 12/04/2014 10:50 PM, Alex Shulgin wrote: Is there a way to pause the bootstrap process so I can attach gdb to it? With a newer gdb, you can instead tell gdb to follow all forks. I wrote some notes on it recently.

[HACKERS] Postgres 9.4.0 to be released week after next

2014-12-04 Thread Tom Lane
After due consideration, the core committee has agreed it's time to push this puppy out the door. We will wrap 9.4.0 on Monday Dec 15 for public announcement Thursday 18th. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] SSL regression test suite

2014-12-04 Thread David Fetter
On Thu, Dec 04, 2014 at 02:42:41PM +0200, Heikki Linnakangas wrote: On 10/06/2014 04:21 PM, Heikki Linnakangas wrote: This probably needs some further cleanup before it's ready for committing. One issues is that it creates a temporary cluster that listens for TCP connections on localhost,

Re: [HACKERS] SSL regression test suite

2014-12-04 Thread Tom Lane
Heikki Linnakangas hlinnakan...@vmware.com writes: On 10/06/2014 04:21 PM, Heikki Linnakangas wrote: This probably needs some further cleanup before it's ready for committing. One issues is that it creates a temporary cluster that listens for TCP connections on localhost, which isn't safe on a

Re: [HACKERS] Sequence Access Method WIP

2014-12-04 Thread Jim Nasby
On 12/3/14, 8:50 AM, José Luis Tallón wrote: May I possibly suggest a file-per-schema model instead? This approach would certainly solve the excessive i-node consumption problem that --I guess-- Andres is trying to address here. I don't think that really has any advantages. Just spreading

Re: [HACKERS] why is PG_AUTOCONF_FILENAME is pg_config_manual.h?

2014-12-04 Thread Peter Eisentraut
On 11/27/14 9:58 AM, Peter Eisentraut wrote: Surely that's not a value that we expect users to be able to edit. Is pg_config_manual.h just abused as a place that's included everywhere? (I suggest utils/guc.h as a better place.) This has been fixed. -- Sent via pgsql-hackers mailing

Re: [HACKERS] Bugfix and new feature for PGXS

2014-12-04 Thread Peter Eisentraut
On 11/19/14 11:11 PM, Peter Eisentraut wrote: I noticed this item was still in the 9.4 code. Looking at the original submission (http://www.postgresql.org/message-id/201306181552.36673.ced...@2ndquadrant.com, patch 0001), I think the original reason for adding this was wrong to begin with.

Re: [HACKERS] [COMMITTERS] pgsql: Keep track of transaction commit timestamps

2014-12-04 Thread Alvaro Herrera
Alex Shulgin wrote: DEBUG: inserting column 7 value varchar_transform Breakpoint 1, GetSnapshotData (snapshot=0xdb2d60 CatalogSnapshotData) at /home/ash/src/postgresql/src/backend/storage/ipc/procarray.c:1413 1413 xmax = ShmemVariableCache-latestCompletedXid; (gdb) p

Re: [HACKERS] [COMMITTERS] pgsql: Keep track of transaction commit timestamps

2014-12-04 Thread Alex Shulgin
Alvaro Herrera alvhe...@2ndquadrant.com writes: Alex Shulgin wrote: DEBUG: inserting column 7 value varchar_transform Breakpoint 1, GetSnapshotData (snapshot=0xdb2d60 CatalogSnapshotData) at /home/ash/src/postgresql/src/backend/storage/ipc/procarray.c:1413 1413 xmax =

Re: [HACKERS] [COMMITTERS] pgsql: Keep track of transaction commit timestamps

2014-12-04 Thread Alex Shulgin
Alex Shulgin a...@commandprompt.com writes: Figured it out with a pg_usleep in bootstrap.c anyway. Does this look sane? DEBUG: inserting column 6 value 0 DEBUG: inserted - 0 DEBUG: inserting column 7 value varchar_transform TRAP: FailedAssertion(!(((xmax) = ((TransactionId) 3))),

Re: [HACKERS] [COMMITTERS] pgsql: Keep track of transaction commit timestamps

2014-12-04 Thread Alex Shulgin
Alvaro Herrera alvhe...@2ndquadrant.com writes: Alex Shulgin wrote: DEBUG: inserting column 7 value varchar_transform Breakpoint 1, GetSnapshotData (snapshot=0xdb2d60 CatalogSnapshotData) at /home/ash/src/postgresql/src/backend/storage/ipc/procarray.c:1413 1413 xmax =

Re: [HACKERS] Sequence Access Method WIP

2014-12-04 Thread Andres Freund
May I possibly suggest a file-per-schema model instead? This approach would certainly solve the excessive i-node consumption problem that --I guess-- Andres is trying to address here. I don't think that really has any advantages. Just spreading the I/O load, nothing more, it seems: Just

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-12-04 Thread Peter Geoghegan
On Thu, Dec 4, 2014 at 3:04 AM, Craig Ringer cr...@2ndquadrant.com wrote: Yes, I think that's pretty important. With a negative attno so it's treated as a hidden col that must be explicitly named to be shown and won't be confused with user columns. I think that the standard for adding a new

Re: [HACKERS] SSL regression test suite

2014-12-04 Thread Alvaro Herrera
Heikki Linnakangas wrote: How do people feel about including this test suite in the source tree? +1 It's probably not suitable for running as part of make check-world, but it's extremely handy if you're working on a patch related to SSL. I'd like to commit this, even if it has some rough

Re: [HACKERS] libpq pipelining

2014-12-04 Thread Matt Newell
On Thursday, December 04, 2014 10:30:46 PM Craig Ringer wrote: On 12/04/2014 05:08 PM, Heikki Linnakangas wrote: A good API is crucial for this. It should make it easy to write an application that does pipelining, and to handle all the error conditions in a predictable way. I'd suggest that

Re: [HACKERS] libpq pipelining

2014-12-04 Thread Claudio Freire
On Thu, Dec 4, 2014 at 4:11 PM, Matt Newell newe...@blur.com wrote: With the API i am proposing, only 2 new functions (PQgetFirstQuery, PQgetLastQuery) are required to be able to match each result to the query that caused it. Another function, PQgetNextQuery allows iterating through the

Re: [HACKERS] Bugfix and new feature for PGXS

2014-12-04 Thread Peter Eisentraut
On 12/4/14 11:38 AM, Peter Eisentraut wrote: On 11/19/14 11:11 PM, Peter Eisentraut wrote: I noticed this item was still in the 9.4 code. Looking at the original submission (http://www.postgresql.org/message-id/201306181552.36673.ced...@2ndquadrant.com, patch 0001), I think the original

Re: [HACKERS] libpq pipelining

2014-12-04 Thread Matt Newell
On Thursday, December 04, 2014 04:30:27 PM Claudio Freire wrote: On Thu, Dec 4, 2014 at 4:11 PM, Matt Newell newe...@blur.com wrote: With the API i am proposing, only 2 new functions (PQgetFirstQuery, PQgetLastQuery) are required to be able to match each result to the query that caused it.

Re: [HACKERS] superuser() shortcuts

2014-12-04 Thread Peter Eisentraut
On 11/21/14 10:28 AM, Stephen Frost wrote: Specifically: --- One curious item to note is that the current if(!superuser()) {} block approach has masked an inconsistency in at least the FDW code which required a change to the regression tests- namely, we normally force FDW

Re: [HACKERS] superuser() shortcuts

2014-12-04 Thread Robert Haas
On Tue, Dec 2, 2014 at 4:11 PM, Stephen Frost sfr...@snowman.net wrote: It's pretty clear that the current message is complaining about a permissions problem, so the fact that it doesn't specifically include the words permission and denied doesn't bother me. Let me ask the question again:

Re: [HACKERS] superuser() shortcuts

2014-12-04 Thread Stephen Frost
* Peter Eisentraut (pete...@gmx.net) wrote: On 11/21/14 10:28 AM, Stephen Frost wrote: Specifically: --- One curious item to note is that the current if(!superuser()) {} block approach has masked an inconsistency in at least the FDW code which required a change to the

Re: [HACKERS] superuser() shortcuts

2014-12-04 Thread Peter Eisentraut
On 11/26/14 10:24 AM, Stephen Frost wrote: The implementation detail is that it's not part of the normal GRANT/REVOKE privilege system, which is why it's useful to note it in the detail and why we don't need to add an errdetail along the lines of 'You must have SELECT rights on relation X to

Re: [HACKERS] Bugfix and new feature for PGXS

2014-12-04 Thread Andrew Dunstan
On 12/04/2014 02:44 PM, Peter Eisentraut wrote: On 12/4/14 11:38 AM, Peter Eisentraut wrote: On 11/19/14 11:11 PM, Peter Eisentraut wrote: I noticed this item was still in the 9.4 code. Looking at the original submission

Re: [HACKERS] superuser() shortcuts

2014-12-04 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: On Tue, Dec 2, 2014 at 4:11 PM, Stephen Frost sfr...@snowman.net wrote: It's pretty clear that the current message is complaining about a permissions problem, so the fact that it doesn't specifically include the words permission and denied

Re: [HACKERS] superuser() shortcuts

2014-12-04 Thread Stephen Frost
* Peter Eisentraut (pete...@gmx.net) wrote: I will produce a generic 'permission denied' error, and if the reason for the lack of permission is anything other than GRANT/REVOKE, then I will add it to the detail message. That's what I had been thinking, on the assumption that individuals with

Re: [HACKERS] Bugfix and new feature for PGXS

2014-12-04 Thread Andrew Dunstan
On 12/04/2014 03:47 PM, Andrew Dunstan wrote: On 12/04/2014 02:44 PM, Peter Eisentraut wrote: On 12/4/14 11:38 AM, Peter Eisentraut wrote: On 11/19/14 11:11 PM, Peter Eisentraut wrote: I noticed this item was still in the 9.4 code. Looking at the original submission

[HACKERS] Re: [COMMITTERS] pgsql: Support frontend-backend protocol communication using a shm_mq.

2014-12-04 Thread Alvaro Herrera
Robert Haas wrote: Support frontend-backend protocol communication using a shm_mq. I just noticed that this patch broke the case where a standalone backend is sent a query that throws an error -- instead, we get a segmentation fault. Example: echo foobar | postgres --single PostgreSQL

[HACKERS] duplicated partial-column assigns allowed by checkInsertTargets

2014-12-04 Thread Alvaro Herrera
While perusing checkInsertTargets I noticed that it allows duplicated assignments to the same member of a composite: alvherre=# create type f as (a int, b int); CREATE TYPE alvherre=# create table t (col f); CREATE TABLE alvherre=# insert into t (col.a, col.b, col.a) values (42, 43, 44); INSERT 0

Re: [HACKERS] controlling psql's use of the pager a bit more

2014-12-04 Thread Alvaro Herrera
Andrew Dunstan wrote: However, there is more work to do. As Tom noted upthread, psql's calculation of the number of lines is pretty bad. For example, if I do: \pset pager_min_lines 100 select * from generate_series(1,50); the pager still gets invoked, which is unfortunate to say

Re: [HACKERS] superuser() shortcuts

2014-12-04 Thread Andres Freund
On 2014-12-04 15:59:17 -0500, Stephen Frost wrote: I have a hard time wrapping my head around what a *lot* of our users ask when they see a given error message, but if our error message is 'you must have a pear-shaped object to run this command' then I imagine that a new-to-PG user might think

Re: [HACKERS] libpq pipelining

2014-12-04 Thread Heikki Linnakangas
On 12/04/2014 09:11 PM, Matt Newell wrote: With the API i am proposing, only 2 new functions (PQgetFirstQuery, PQgetLastQuery) are required to be able to match each result to the query that caused it. Another function, PQgetNextQuery allows iterating through the pending queries, and

Re: [HACKERS] controlling psql's use of the pager a bit more

2014-12-04 Thread Andrew Dunstan
On 12/04/2014 03:53 PM, Alvaro Herrera wrote: Andrew Dunstan wrote: However, there is more work to do. As Tom noted upthread, psql's calculation of the number of lines is pretty bad. For example, if I do: \pset pager_min_lines 100 select * from generate_series(1,50); the pager still

Re: [HACKERS] superuser() shortcuts

2014-12-04 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: On 2014-12-04 15:59:17 -0500, Stephen Frost wrote: I have a hard time wrapping my head around what a *lot* of our users ask when they see a given error message, but if our error message is 'you must have a pear-shaped object to run this

Re: [HACKERS] superuser() shortcuts

2014-12-04 Thread Alvaro Herrera
Stephen Frost wrote: * Andres Freund (and...@2ndquadrant.com) wrote: On 2014-12-04 15:59:17 -0500, Stephen Frost wrote: I just don't understand why you want to pointlessly tinker with this. Because I don't feel it's pointless to improve the consistency of the error messaging and

Re: [HACKERS] superuser() shortcuts

2014-12-04 Thread Andres Freund
On 2014-12-04 17:06:02 -0500, Stephen Frost wrote: * Andres Freund (and...@2ndquadrant.com) wrote: On 2014-12-04 15:59:17 -0500, Stephen Frost wrote: I have a hard time wrapping my head around what a *lot* of our users ask when they see a given error message, but if our error message is

Re: [HACKERS] superuser() shortcuts

2014-12-04 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: On 2014-12-04 17:06:02 -0500, Stephen Frost wrote: Would you agree with Peter that, rather than focus on if the errdetail() involves an implementation detail or not, we should go ahead and add the You must have SELECT rights ... to the

[HACKERS] Minor documentation tweak to pg_stat_all_tables view description

2014-12-04 Thread Peter Geoghegan
Attached patch makes minor modification to the pg_stat_all_tables documentation. This clarifies that pg_stat_*_tables.n_tup_upd includes HOT updates. -- Peter Geoghegan diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index b29e5e6..3ce7e80 100644 ---

Re: [HACKERS] superuser() shortcuts

2014-12-04 Thread Stephen Frost
Alvaro, * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: Several dozens messages ago in this thread I would have dropped this item, TBH. I'm getting to that point, though it's quite frustrating. I didn't much care initially but it's gotten to the point where the current situation just

Re: [HACKERS] Minor documentation tweak to pg_stat_all_tables view description

2014-12-04 Thread Andres Freund
On 2014-12-04 14:43:43 -0800, Peter Geoghegan wrote: Attached patch makes minor modification to the pg_stat_all_tables documentation. This clarifies that pg_stat_*_tables.n_tup_upd includes HOT updates. Ah. Good idea. I've been asked that by others and myself quite a number of times and had to

Re: [HACKERS] Minor documentation tweak to pg_stat_all_tables view description

2014-12-04 Thread Peter Geoghegan
On Thu, Dec 4, 2014 at 2:56 PM, Andres Freund and...@2ndquadrant.com wrote: I wondered for a sec whether it'd be better to refer to n_tup_hot_upd here, but decided you were right. Pushed. Thanks. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] New wal format distorts pg_xlogdump --stats

2014-12-04 Thread Andres Freund
On 2014-12-04 16:26:02 +0200, Heikki Linnakangas wrote: Yeah, that's broken. I propose the attached. Or does anyone want to argue for adding an XLogRecGetFPILen() accessor macro for the hole_length in xlogreader.h. It's not something a redo routine would need, nor most XLOG-reading

Re: [HACKERS] libpq pipelining

2014-12-04 Thread Matt Newell
On Thursday, December 04, 2014 11:39:02 PM Heikki Linnakangas wrote: Adding the ability to set a user supplied pointer on the PGquery struct might make it much easier for some frameworks, and other users might want a callback, but I don't think either are required. I don't like exposing

Re: [HACKERS] duplicated partial-column assigns allowed by checkInsertTargets

2014-12-04 Thread Jim Nasby
On 12/4/14, 2:23 PM, Alvaro Herrera wrote: While perusing checkInsertTargets I noticed that it allows duplicated assignments to the same member of a composite: alvherre=# create type f as (a int, b int); CREATE TYPE alvherre=# create table t (col f); CREATE TABLE alvherre=# insert into t

Re: [HACKERS] New wal format distorts pg_xlogdump --stats

2014-12-04 Thread Michael Paquier
On Fri, Dec 5, 2014 at 8:09 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-12-04 16:26:02 +0200, Heikki Linnakangas wrote: Yeah, that's broken. I propose the attached. Or does anyone want to argue for adding an XLogRecGetFPILen() accessor macro for the hole_length in

[HACKERS] pg_basebackup -x/X doesn't play well with archive_mode wal_keep_segments

2014-12-04 Thread Andres Freund
Hi, We've recently observed a case where, after a promotion, a postgres server suddenly started to archive a large amount of old WAL. After some digging the problem is this: pg_basebackup -X creates files in pg_xlog/ without creating the corresponding .done file. Note that walreceiver *does*

Re: [HACKERS] libpq pipelining

2014-12-04 Thread Matt Newell
The explanation of PQgetFirstQuery makes it sound pretty hard to match up the result with the query. You have to pay attention to PQisBusy. PQgetFirstQuery should also be valid after calling PQgetResult and then you don't have to worry about PQisBusy, so I should probably change the

Re: [HACKERS] New wal format distorts pg_xlogdump --stats

2014-12-04 Thread Andres Freund
On 2014-12-05 08:58:33 +0900, Michael Paquier wrote: On Fri, Dec 5, 2014 at 8:09 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-12-04 16:26:02 +0200, Heikki Linnakangas wrote: Yeah, that's broken. I propose the attached. Or does anyone want to argue for adding an

[HACKERS] Re: [COMMITTERS] pgsql: Support frontend-backend protocol communication using a shm_mq.

2014-12-04 Thread Robert Haas
On Thu, Dec 4, 2014 at 4:15 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Robert Haas wrote: Support frontend-backend protocol communication using a shm_mq. I just noticed that this patch broke the case where a standalone backend is sent a query that throws an error -- instead, we get a

Re: [HACKERS] Bugfix and new feature for PGXS

2014-12-04 Thread Peter Eisentraut
On 12/4/14 3:47 PM, Andrew Dunstan wrote: You have broken two buildfarm instances that build and test external modules - in one case the Redis FDW module and in the other the File Text Array FDW. I will see what can be retrieved. This has been fixed. One thing I'll look into sometime is

Re: [HACKERS] superuser() shortcuts

2014-12-04 Thread Robert Haas
On Thu, Dec 4, 2014 at 3:59 PM, Stephen Frost sfr...@snowman.net wrote: I have a hard time wrapping my head around what a *lot* of our users ask when they see a given error message, but if our error message is 'you must have a pear-shaped object to run this command' then I imagine that a

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-04 Thread Robert Haas
On Thu, Dec 4, 2014 at 5:36 AM, Rahila Syed rahilasyed...@gmail.com wrote: The only scenario in which a user would not want to compress forcibly written pages is when CPU utilization is high. Or if they think the code to compress full pages is buggy. But according to measurements done

[HACKERS] XLOG_PARAMETER_CHANGE (WAL record) missing two params in its desc routine

2014-12-04 Thread Michael Paquier
Hi all, While reading the code in this area this morning, I noticed that wal_log_hints and track_commit_timestamp are not mentioned in the desc routine of XLOG_CHANGE_PARAMETER. Also, it is not mentioned in postgresql.conf.sample that a value update of wal_log_hints requires a system restart. In

[HACKERS] check-world failure: dummy_seclabel

2014-12-04 Thread Adam Brightwell
All, I've noticed that 'check-world' fails for dummy_seclabel after a 'clean'. I believe that in commit da34731, the EXTRA_CLEAN statement should have been removed from 'src/test/modules/dummy_seclabel/Makefile' as well. Attached is a proposed patch that addresses this issue. Thanks, Adam --

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-12-04 Thread Anssi Kääriäinen
On Thu, 2014-12-04 at 10:27 -0800, Peter Geoghegan wrote: I think that the standard for adding a new system attribute ought to be enormous. The only case where a new one was added post-Postgres95 was tableoid. I'm pretty sure that others aren't going to want to do it that way. Besides, I'm

Re: [HACKERS] On partitioning

2014-12-04 Thread Amit Kapila
On Thu, Dec 4, 2014 at 10:46 AM, Amit Langote langote_amit...@lab.ntt.co.jp wrote: Hi, From: Jim Nasby [mailto:jim.na...@bluetreble.com] On 12/2/14, 9:43 PM, Amit Langote wrote: What are you going to do if the partitioning key has two columns of different data types? Sorry,

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-04 Thread Rahila Syed
If that's really true, we could consider having no configuration any time, and just compressing always. But I'm skeptical that it's actually true. I was referring to this for CPU utilization: http://www.postgresql.org/message-id/1410414381339-5818552.p...@n5.nabble.com http:// The above

Re: [HACKERS] On partitioning

2014-12-04 Thread Amit Kapila
On Wed, Dec 3, 2014 at 6:30 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Amit Langote wrote: From: Robert Haas [mailto:robertmh...@gmail.com] What is an overflow partition and why do we want that? That would be a default partition. That is, where the tuples that don't belong

Re: [HACKERS] On partitioning

2014-12-04 Thread Amit Langote
From: Amit Kapila [mailto:amit.kapil...@gmail.com] On Thu, Dec 4, 2014 at 10:46 AM, Amit Langote langote_amit...@lab.ntt.co.jp wrote: The more SQL way would be records (composite types). That would make catalog inspection a LOT easier and presumably make it easier to change the

Re: [HACKERS] On partitioning

2014-12-04 Thread Amit Kapila
On Tue, Dec 2, 2014 at 8:53 PM, Robert Haas robertmh...@gmail.com wrote: On Tue, Nov 25, 2014 at 8:20 PM, Amit Langote langote_amit...@lab.ntt.co.jp wrote: Before going too much further with this I'd mock up schemas for your proposed catalogs and a list of DDL operations to be supported,

Re: [HACKERS] pg_basebackup -x/X doesn't play well with archive_mode wal_keep_segments

2014-12-04 Thread Fujii Masao
On Fri, Dec 5, 2014 at 9:28 AM, Andres Freund and...@2ndquadrant.com wrote: Hi, We've recently observed a case where, after a promotion, a postgres server suddenly started to archive a large amount of old WAL. After some digging the problem is this: pg_basebackup -X creates files in

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-04 Thread Fujii Masao
On Thu, Dec 4, 2014 at 8:37 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Thu, Dec 4, 2014 at 7:36 PM, Rahila Syed rahilasyed...@gmail.com wrote: IIUC, forcibly written fpws are not exposed to user , so is it worthwhile to add a GUC similar to full_page_writes in order to control a

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-04 Thread Michael Paquier
On Fri, Dec 5, 2014 at 10:53 AM, Robert Haas robertmh...@gmail.com wrote: On Thu, Dec 4, 2014 at 5:36 AM, Rahila Syed rahilasyed...@gmail.com wrote: The only scenario in which a user would not want to compress forcibly written pages is when CPU utilization is high. Or if they think the