Re: [HACKERS] Exposing PG_VERSION_NUM in pg_config

2015-04-06 Thread Jim Nasby
On 4/1/15 1:25 AM, Andrew Gierth wrote: Michael == Michael Paquier michael.paqu...@gmail.com writes: Michael For an extension that has a single branch compatible with a Michael set of multiple major versions of Postgres, the cases are Michael custom values for REGRESS_OPTS and REGRESS

Re: [HACKERS] pg_dump / copy bugs with big lines ?

2015-04-06 Thread Jim Nasby
On 3/31/15 3:46 AM, Ronan Dunklau wrote: StringInfo uses int's to store length, so it could possibly be changed, but then you'd just error out due to MaxAllocSize. Now perhaps those could both be relaxed, but certainly not to the extent that you can shove an entire 1.6TB row into an output

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-06 Thread Josh Berkus
On 04/06/2015 10:07 AM, Jim Nasby wrote: Actually, I would start simply with ReadOnly and ReadWrite. As I understand it, the goal here is to prevent huge amounts of periodic freeze work due to XID wraparound. I don't think we need the Freeze state to accomplish that. With a single bit per

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-06 Thread Alvaro Herrera
Josh Berkus wrote: I agree with Jim that if we have a trustworthy Frozen Map, having a ReadOnly flag is of marginal value, unless such a ReadOnly flag allowed us to skip updating the individual row XIDs entirely. I can think of some ways to do that, but they have severe tradeoffs. If you're

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-06 Thread k...@rice.edu
On Mon, Apr 06, 2015 at 12:07:47PM -0500, Jim Nasby wrote: ... As I understand it, the goal here is to prevent huge amounts of periodic freeze work due to XID wraparound. I don't think we need the Freeze state to accomplish that. With a single bit per page in the Frozen Map, checking a

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-06 Thread Jim Nasby
On 4/6/15 12:29 PM, k...@rice.edu wrote: On Mon, Apr 06, 2015 at 12:07:47PM -0500, Jim Nasby wrote: ... As I understand it, the goal here is to prevent huge amounts of periodic freeze work due to XID wraparound. I don't think we need the Freeze state to accomplish that. With a single bit per

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-06 Thread Josh Berkus
On 04/06/2015 11:35 AM, Alvaro Herrera wrote: Josh Berkus wrote: I agree with Jim that if we have a trustworthy Frozen Map, having a ReadOnly flag is of marginal value, unless such a ReadOnly flag allowed us to skip updating the individual row XIDs entirely. I can think of some ways to do

Re: [HACKERS] Auditing extension for PostgreSQL (Take 2)

2015-04-06 Thread Simon Riggs
On 6 April 2015 at 16:34, Peter Eisentraut pete...@gmx.net wrote: On 2/14/15 9:34 PM, David Steele wrote: The patch I've attached satisfies the requirements that I've had from customers in the past. What I'm missing is a more precise description/documentation of what those requirements might

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-06 Thread Jim Nasby
On 4/6/15 1:28 PM, Josh Berkus wrote: On 04/06/2015 10:07 AM, Jim Nasby wrote: Actually, I would start simply with ReadOnly and ReadWrite. As I understand it, the goal here is to prevent huge amounts of periodic freeze work due to XID wraparound. I don't think we need the Freeze state to

Re: [HACKERS] Auditing extension for PostgreSQL (Take 2)

2015-04-06 Thread Peter Eisentraut
On 2/14/15 9:34 PM, David Steele wrote: The patch I've attached satisfies the requirements that I've had from customers in the past. What I'm missing is a more precise description/documentation of what those requirements might be. Audit is a big word. It might imply regulatory or standards

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-06 Thread Jim Nasby
On 4/6/15 5:18 PM, Greg Stark wrote: Only I would suggest thinking of it in terms of two orthogonal boolean flags rather than three states. It's easier to reason about whether a table has a specific property than trying to control a state machine in a predefined pathway. So I would say the two

Re: [HACKERS] Auditing extension for PostgreSQL (Take 2)

2015-04-06 Thread Alvaro Herrera
Simon Riggs wrote: The present version can trigger an audit trail event for a statement, without tracking the object that was being audited. This prevents you from searching for all SQL that touches table X, i.e. we know the statements were generated, but not which ones they were. IMHO that

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-06 Thread Greg Stark
On 6 Apr 2015 09:17, Jim Nasby jim.na...@bluetreble.com wrote: No. You would be free to set a table as ReadOnly any time you wanted, without scanning anything. All that setting does is disable any DML on the table. The Frozen state would only be set by the vacuum code, IFF: - The table state

Re: [HACKERS] BRIN range operator class

2015-04-06 Thread Alvaro Herrera
Thanks for the updated patch; I will at it as soon as time allows. (Not really all that soon, regrettably.) Judging from a quick look, I think patches 1 and 5 can be committed quickly; they imply no changes to other parts of BRIN. (Not sure why 1 and 5 are separate. Any reason for this?) Also

Re: [HACKERS] Auditing extension for PostgreSQL (Take 2)

2015-04-06 Thread David Steele
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 4/6/15 4:34 PM, Peter Eisentraut wrote: On 2/14/15 9:34 PM, David Steele wrote: The patch I've attached satisfies the requirements that I've had from customers in the past. What I'm missing is a more precise description/documentation of

Re: [HACKERS] pg_rewind and log messages

2015-04-06 Thread Michael Paquier
On Mon, Apr 6, 2015 at 10:01 PM, Alvaro Herrera wrote: I'm not sure about translation of generic strings such as %s: %s. My first impression is that they shouldn't be translated, but maybe it is important that they are for languages I don't know nothing about such as Japanese. I

[HACKERS] Typo in a comment in set_rel_size()

2015-04-06 Thread Amit Langote
Hi, Attached fixes what I suppose is a typo: * so set up a single dummy path for it. Here we only check this for * regular baserels; if it's an otherrel, CE was already checked in -* set_append_rel_pathlist(). +* set_append_rel_size(). * Thanks, Amit

Re: [HACKERS] Auditing extension for PostgreSQL (Take 2)

2015-04-06 Thread David Steele
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 4/6/15 4:47 PM, Simon Riggs wrote: On 6 April 2015 at 16:34, Peter Eisentraut pete...@gmx.net wrote: Audit is a big word. It might imply regulatory or standards compliance on some level. We already have ways to log everything. If

[HACKERS] PATCH: use foreign keys to improve join estimates v1

2015-04-06 Thread Tomas Vondra
Hi, attached is a first version of a patch that aims to improve cardinality estimates of joins by matching foreign keys between the tables (which was ignored by the planner until now). This significantly improves estimates when joining two tables using multi-column conditions, matching a

[HACKERS] Ignoring some binaries generated in src/test

2015-04-06 Thread Michael Paquier
Hi all, A couple of binaries in src/test, that are not part of the main make flow, can be built but they are actually not ignored in the tree: examples/testlibpq examples/testlibpq2 examples/testlibpq3 examples/testlibpq4 examples/testlo examples/testlo64

Re: [HACKERS] Proposal : REINDEX xxx VERBOSE

2015-04-06 Thread Fabrízio de Royes Mello
On Mon, Apr 6, 2015 at 10:21 AM, Sawada Masahiko sawada.m...@gmail.com wrote: On Fri, Mar 13, 2015 at 5:10 PM, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: Hello, I have some trivial comments about the latest patch. At Thu, 12 Mar 2015 21:15:14 +0900, Sawada Masahiko

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-06 Thread Sawada Masahiko
On Tue, Apr 7, 2015 at 7:53 AM, Jim Nasby jim.na...@bluetreble.com wrote: On 4/6/15 5:18 PM, Greg Stark wrote: Only I would suggest thinking of it in terms of two orthogonal boolean flags rather than three states. It's easier to reason about whether a table has a specific property than trying

Re: [HACKERS] Auditing extension for PostgreSQL (Take 2)

2015-04-06 Thread Simon Riggs
On 6 April 2015 at 20:38, David Steele da...@pgmasters.net wrote: The earlier version of pg_audit generated different output. Specifically, it allowed you to generate output for each object tracked; one line per object. That discussion covers recursive SQL. That is important too, but not what

Re: [HACKERS] pg_rewind and log messages

2015-04-06 Thread Michael Paquier
On Mon, Apr 6, 2015 at 9:10 PM, Fujii Masao masao.fu...@gmail.com wrote: I'm not familiar with native language support (sorry), but don't we need to add the shortcut of gettext into every calls of pg_log and pg_fatal, e.g., change pg_fatal(xxx) to pg_fatal(_(xxx))? I know that fprintf() in

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-04-06 Thread Kouhei Kaigai
Hanada-san, Thanks for your dedicated efforts for remote-join feature. Below are the comments from my side. * Bug - mixture of ctid system column and whole row-reference In case when ctid system column is required, deparseSelectSql() adds ctid reference on the base relation scan level. On the

Re: [HACKERS] Auditing extension for PostgreSQL (Take 2)

2015-04-06 Thread Sawada Masahiko
On Fri, Apr 3, 2015 at 10:01 PM, David Steele da...@pgmasters.net wrote: On 4/3/15 3:59 AM, Sawada Masahiko wrote: On Thu, Apr 2, 2015 at 2:46 AM, David Steele da...@pgmasters.net wrote: Let me know if you see any other issues. I pulled HEAD, and then tried to compile source code after

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-06 Thread Sawada Masahiko
On Sun, Apr 5, 2015 at 8:21 AM, Jeff Janes jeff.ja...@gmail.com wrote: On Sat, Apr 4, 2015 at 3:10 PM, Jim Nasby jim.na...@bluetreble.com wrote: On 4/3/15 12:59 AM, Sawada Masahiko wrote: + case HEAPTUPLE_LIVE: + case

Re: [HACKERS] TABLESAMPLE patch

2015-04-06 Thread Amit Kapila
On Sat, Apr 4, 2015 at 8:25 PM, Petr Jelinek p...@2ndquadrant.com wrote: On 04/04/15 14:57, Amit Kapila wrote: 1. +tablesample_clause: +TABLESAMPLE ColId '(' func_arg_list ')' opt_repeatable_clause It seems to me that you want to allow it to make it extendable to user defined Tablesample

Re: [HACKERS] initdb -S and tablespaces

2015-04-06 Thread Abhijit Menon-Sen
Hi Álvaro. Thanks for taking a look at the patch. At 2015-04-03 13:32:32 -0300, alvhe...@2ndquadrant.com wrote: But then, since the name is already telling us that it's all about PGDATA, maybe we can remove the recursively part of the name? Sure, that makes sense too. Since you and Andres

Re: [HACKERS] pg_rewind and log messages

2015-04-06 Thread Michael Paquier
On Mon, Apr 6, 2015 at 1:41 PM, Michael Paquier wrote: I guess that you are working on a patch? If not, you are looking for one? Code-speaking, this gives the patch attached. I eliminated a bunch of newlines in the log messages that seemed really unnecessary to me, simplifying a bit the whole.

Re: [HACKERS] TABLESAMPLE patch

2015-04-06 Thread Simon Riggs
On 2 April 2015 at 17:36, Petr Jelinek p...@2ndquadrant.com wrote: so here is version 11. Looks great. Comment on docs: The SELECT docs refer only to SYSTEM and BERNOULLI. It doesn't mention that if other methods are available they could be used also. The phrasing was sampling method can be

Re: [HACKERS] TABLESAMPLE patch

2015-04-06 Thread Petr Jelinek
On 06/04/15 12:33, Amit Kapila wrote: On Sat, Apr 4, 2015 at 8:25 PM, Petr Jelinek p...@2ndquadrant.com mailto:p...@2ndquadrant.com wrote: Yes I want extensibility here. And I think the tablesample method arguments are same thing as function arguments given that in the end they are arguments

Re: [HACKERS] pg_rewind and log messages

2015-04-06 Thread Alvaro Herrera
Fujii Masao wrote: On Mon, Apr 6, 2015 at 5:33 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Mon, Apr 6, 2015 at 1:41 PM, Michael Paquier wrote: I guess that you are working on a patch? If not, you are looking for one? Code-speaking, this gives the patch attached. Thanks!

Re: [HACKERS] initdb -S and tablespaces

2015-04-06 Thread Alvaro Herrera
Abhijit Menon-Sen wrote: Hi, At 2015-04-03 13:32:32 -0300, alvhe...@2ndquadrant.com wrote: I also noticed that walkdir() thinks it is completely agnostic on what the passed action is; except that the comment at the bottom talks about how fsync on directories is important, which seems out

Re: [HACKERS] Proposal : REINDEX xxx VERBOSE

2015-04-06 Thread Sawada Masahiko
On Fri, Mar 13, 2015 at 5:10 PM, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: Hello, I have some trivial comments about the latest patch. At Thu, 12 Mar 2015 21:15:14 +0900, Sawada Masahiko sawada.m...@gmail.com wrote in

Re: [HACKERS] TABLESAMPLE patch

2015-04-06 Thread Petr Jelinek
On 06/04/15 15:07, Amit Kapila wrote: On Mon, Apr 6, 2015 at 5:56 PM, Petr Jelinek p...@2ndquadrant.com mailto:p...@2ndquadrant.com wrote: On 06/04/15 12:33, Amit Kapila wrote: But I think the Update on target table with sample scan is supported via views which doesn't seem to be the

Re: [HACKERS] Auditing extension for PostgreSQL (Take 2)

2015-04-06 Thread Sawada Masahiko
On Fri, Apr 3, 2015 at 10:01 PM, David Steele da...@pgmasters.net wrote: On 4/3/15 3:59 AM, Sawada Masahiko wrote: On Thu, Apr 2, 2015 at 2:46 AM, David Steele da...@pgmasters.net wrote: Let me know if you see any other issues. I pulled HEAD, and then tried to compile source code after

Re: [HACKERS] TABLESAMPLE patch

2015-04-06 Thread Amit Kapila
On Mon, Apr 6, 2015 at 5:56 PM, Petr Jelinek p...@2ndquadrant.com wrote: On 06/04/15 12:33, Amit Kapila wrote: But I think the Update on target table with sample scan is supported via views which doesn't seem to be the right thing in case you just want to support it via FROM/USING, example

Re: [HACKERS] pg_rewind and log messages

2015-04-06 Thread Fujii Masao
On Mon, Apr 6, 2015 at 5:33 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Mon, Apr 6, 2015 at 1:41 PM, Michael Paquier wrote: I guess that you are working on a patch? If not, you are looking for one? Code-speaking, this gives the patch attached. Thanks! Here are the review comments:

Re: [HACKERS] TABLESAMPLE patch

2015-04-06 Thread Petr Jelinek
On 06/04/15 11:02, Simon Riggs wrote: On 2 April 2015 at 17:36, Petr Jelinek p...@2ndquadrant.com wrote: so here is version 11. Looks great. Comment on docs: The SELECT docs refer only to SYSTEM and BERNOULLI. It doesn't mention that if other methods are available they could be used also.

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-06 Thread Jim Nasby
On 4/6/15 1:46 AM, Sawada Masahiko wrote: On Sun, Apr 5, 2015 at 8:21 AM, Jeff Janes jeff.ja...@gmail.com wrote: On Sat, Apr 4, 2015 at 3:10 PM, Jim Nasby jim.na...@bluetreble.com wrote: On 4/3/15 12:59 AM, Sawada Masahiko wrote: + case HEAPTUPLE_LIVE: +

Re: [HACKERS] Auditing extension for PostgreSQL (Take 2)

2015-04-06 Thread David Steele
On 4/6/15 8:40 AM, Sawada Masahiko wrote: On Fri, Apr 3, 2015 at 10:01 PM, David Steele da...@pgmasters.net wrote: On 4/3/15 3:59 AM, Sawada Masahiko wrote: On Thu, Apr 2, 2015 at 2:46 AM, David Steele da...@pgmasters.net wrote: Let me know if you see any other issues. I pulled HEAD, and

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-06 Thread Sawada Masahiko
On Mon, Apr 6, 2015 at 10:17 PM, Jim Nasby jim.na...@bluetreble.com wrote: On 4/6/15 1:46 AM, Sawada Masahiko wrote: On Sun, Apr 5, 2015 at 8:21 AM, Jeff Janes jeff.ja...@gmail.com wrote: On Sat, Apr 4, 2015 at 3:10 PM, Jim Nasby jim.na...@bluetreble.com wrote: On 4/3/15 12:59 AM, Sawada

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-06 Thread Jim Nasby
On 4/6/15 11:12 AM, Sawada Masahiko wrote: On Mon, Apr 6, 2015 at 10:17 PM, Jim Nasby jim.na...@bluetreble.com wrote: On 4/6/15 1:46 AM, Sawada Masahiko wrote: On Sun, Apr 5, 2015 at 8:21 AM, Jeff Janes jeff.ja...@gmail.com wrote: On Sat, Apr 4, 2015 at 3:10 PM, Jim Nasby