Re: [HACKERS] pg_basebackup vs. Windows and tablespaces

2014-11-13 Thread Amit Kapila
On Fri, Nov 14, 2014 at 9:11 AM, Robert Haas wrote: > On Thu, Nov 13, 2014 at 10:37 PM, Amit Kapila wrote: > >> > > > > I have mentioned that this can be usable for Linux users as well on that > > thread, however I think we might want to provide it with an option for > > linux users. In general,

Re: [HACKERS] Race condition between hot standby and restoring a FPW

2014-11-13 Thread Heikki Linnakangas
On 11/13/2014 01:06 AM, Jim Nasby wrote: On 11/12/14, 9:47 AM, Tom Lane wrote: Heikki Linnakangas writes: On 11/12/2014 05:20 PM, Tom Lane wrote: On reconsideration I think the "RBM_ZERO returns page already locked" alternative may be the less ugly. That has the advantage that any code that

Re: [HACKERS] using custom scan nodes to prototype parallel sequential scan

2014-11-13 Thread Jim Nasby
On 11/12/14, 1:54 AM, David Rowley wrote: On Tue, Nov 11, 2014 at 9:29 PM, Simon Riggs mailto:si...@2ndquadrant.com>> wrote: This plan type is widely used in reporting queries, so will hit the mainline of BI applications and many Mat View creations. This will allow SELECT count(*) F

Re: [HACKERS] using custom scan nodes to prototype parallel sequential scan

2014-11-13 Thread Kouhei Kaigai
> On Fri, Nov 14, 2014 at 2:12 PM, Kouhei Kaigai wrote: > > > > On 12 November 2014 07:54, David Rowley > > > Take int4_avg_accum() for example it does: > > > > > > transdata->count++; > > transdata->sum += newval; > > > > The merge function

Re: [HACKERS] WAL format and API changes (9.5)

2014-11-13 Thread Amit Kapila
On Thu, Nov 13, 2014 at 7:03 PM, Heikki Linnakangas wrote: > > On 11/11/2014 04:42 PM, Amit Kapila wrote: >> >> I have done some performance testing of this patch using attached >> script and data is as below: >> >> ... >> >> It seems to me that there is a regression of (4 ~ 8%) for small records,

Re: [HACKERS] Re: Segmentation fault in pg_dumpall from master down to 9.1 and other bug introduced by RLS

2014-11-13 Thread Tom Lane
Noah Misch writes: > On Thu, Nov 13, 2014 at 08:24:36PM -0500, Stephen Frost wrote: >> Agreed. I'll take care of both and we'll make sure the new role >> attributes being added will do the same for upgrades also. > That would make pg_dumpall less faithful for every role other than the > bootstra

Re: [HACKERS] Unintended restart after recovery error

2014-11-13 Thread Fujii Masao
On Thu, Nov 13, 2014 at 8:30 AM, Robert Haas wrote: > On Wed, Nov 12, 2014 at 4:52 PM, Antonin Houska wrote: >> Fujii Masao wrote: >> >>> On Wed, Nov 12, 2014 at 6:52 PM, Antonin Houska wrote: >>> > While looking at postmaster.c:reaper(), one problematic case occurred to >>> > me. >>> > >>> >

[HACKERS] Re: Segmentation fault in pg_dumpall from master down to 9.1 and other bug introduced by RLS

2014-11-13 Thread Noah Misch
On Thu, Nov 13, 2014 at 08:24:36PM -0500, Stephen Frost wrote: > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > What's bothering me is that I see this in pg_dumpall output from a 9.4 > > or earlier database: > > > > ALTER ROLE postgres WITH SUPERUSER INHERIT CREATEROLE CREATEDB LOGIN > > REPLICATION

Re: [HACKERS] pg_basebackup vs. Windows and tablespaces

2014-11-13 Thread Robert Haas
On Thu, Nov 13, 2014 at 10:37 PM, Amit Kapila wrote: > On Fri, Nov 14, 2014 at 3:03 AM, Peter Eisentraut wrote: >> >> On 11/13/14 11:52 AM, Alvaro Herrera wrote: >> > Amit Kapila wrote: >> > >> >> This patch is in "Ready for committer" stage for more than 1.5 months. >> >> I believe this is an im

Re: [HACKERS] pg_basebackup vs. Windows and tablespaces

2014-11-13 Thread Amit Kapila
On Fri, Nov 14, 2014 at 3:03 AM, Peter Eisentraut wrote: > > On 11/13/14 11:52 AM, Alvaro Herrera wrote: > > Amit Kapila wrote: > > > >> This patch is in "Ready for committer" stage for more than 1.5 months. > >> I believe this is an important functionality such that without this tar > >> format o

[HACKERS] Re: [GENERAL] Performance issue with libpq prepared queries on 9.3 and 9.4

2014-11-13 Thread Sam Saffron
Thank you so much! So to recap the general way to reproduce this issue is: create table products(id int primary key, type varchar); insert into products select generate_series(1,1), 'aaa'; insert into products select generate_series(10001,2), 'bbb'; create index idx on products(type); pre

Re: [HACKERS] logical decoding - reading a user catalog table

2014-11-13 Thread Steve Singer
On 11/13/2014 02:44 PM, Andres Freund wrote: H I've pushed a fix for a bug that could possibly also cause this. Although it'd be odd that it always hits the user catalog table. Except if your tests mostly modify the slony tables, but do not do much DDL otherwise? The test I was running doesn

Re: [HACKERS] On the warpath again about ill-considered inclusion nests

2014-11-13 Thread Stephen Frost
Tom, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Well, if you *only* move RowSecurityDesc and not RowSecurityPolicy, > okay, but that seems a bit useless/inconsistent if I'm reading it > right that RowSecurityDesc contains a List of RowSecurityPolicy structs. Yes, good point. > What seems possibly

Re: [HACKERS] On partitioning

2014-11-13 Thread Stephen Frost
Robert, * Robert Haas (robertmh...@gmail.com) wrote: > On Thu, Nov 13, 2014 at 1:39 AM, Stephen Frost wrote: > > but > > with declarative partitioning, I expect us to eventually be able to > > eliminate complete partitions from consideration on both sides of a > > partition-table join and optimiz

Re: [HACKERS] what does this mean: "running xacts with xcnt == 0"

2014-11-13 Thread Andres Freund
On 2014-11-13 21:03:57 -0500, Peter Eisentraut wrote: > On 11/12/14 1:04 PM, Robert Haas wrote: > > I suggest: > > > > 1. Logical decoding will begin using saved snapshot. > > 2. Transaction ID %u finished; no more running transactions. > > 3. There are no running transactions. > > I went with th

Re: [HACKERS] what does this mean: "running xacts with xcnt == 0"

2014-11-13 Thread Peter Eisentraut
On 11/12/14 1:04 PM, Robert Haas wrote: > I suggest: > > 1. Logical decoding will begin using saved snapshot. > 2. Transaction ID %u finished; no more running transactions. > 3. There are no running transactions. I went with that. Thank you. -- Sent via pgsql-hackers mailing list (pgsql-hacke

Re: [HACKERS] EXPLAIN ANALYZE output weird for Top-N Sort

2014-11-13 Thread Tom Lane
David G Johnston writes: > Tom Lane-2 wrote >> [ shrug... ] The estimated value is the planner's estimate of what would >> happen *if you ran the node to completion*, which in practice doesn't >> happen because of the LIMIT. > I don't see how a sort node cannot run to completion... The sort m

Re: [HACKERS] using custom scan nodes to prototype parallel sequential scan

2014-11-13 Thread David Rowley
On Fri, Nov 14, 2014 at 2:12 PM, Kouhei Kaigai wrote: > > On 12 November 2014 07:54, David Rowley > > Take int4_avg_accum() for example it does: > > > > > > transdata->count++; > > transdata->sum += newval; > > > > The merge function would need to perform something like: > > > > transdata-

Re: [HACKERS] EXPLAIN ANALYZE output weird for Top-N Sort

2014-11-13 Thread David G Johnston
Tom Lane-2 wrote > [ shrug... ] The estimated value is the planner's estimate of what would > happen *if you ran the node to completion*, which in practice doesn't > happen because of the LIMIT. I don't see how a sort node cannot run to completion...raising the thought that the "actual" row cou

Re: [HACKERS] EXPLAIN ANALYZE output weird for Top-N Sort

2014-11-13 Thread Tom Lane
David G Johnston writes: > Tom Lane-2 wrote >> [ shrug... ] The estimated value is the planner's estimate of what would >> happen *if you ran the node to completion*, which in practice doesn't >> happen because of the LIMIT. The actual value is, well, the actual value. >> We certainly should not

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2014-11-13 Thread Andreas Karlsson
On 11/14/2014 02:25 AM, Peter Eisentraut wrote: There is something odd about your patch. I claims that all files are new files, e.g.: diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c new file mode 100644 index d61af92..98183b4 *** a/src/backend/utils/adt/numeric.c

Re: [HACKERS] EXPLAIN ANALYZE output weird for Top-N Sort

2014-11-13 Thread David G Johnston
Tom Lane-2 wrote > Simon Riggs < > simon@ > > writes: >> Limit (cost= rows=20 width=175) (actual time= rows=20 loops=1) >>-> Sort (cost= rows=568733 width=175) (actual time= >> rows=20 loops=1) >> Sort Method: top-N heapsort > >> The Sort estimate shows 568733 row

Re: [HACKERS] Re: [GENERAL] Performance issue with libpq prepared queries on 9.3 and 9.4

2014-11-13 Thread Tom Lane
David Johnston writes: > ​While "planner hints" comes to mind...on the SQL side can we extend the > "PREPARE" command with two additional keywords?​ > ​PREPARE > name [ ( data_type [, ...] ) ] [ > [NO] GENERIC > ​] ​ > ​AS statement Don't really see the point. The OP's problem is t

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2014-11-13 Thread Peter Eisentraut
On 11/13/14 7:57 PM, Andreas Karlsson wrote: > On 11/13/2014 03:38 AM, Alvaro Herrera wrote: >> configure is a generated file. If your patch touches it but not >> configure.in, there is a problem. > > Thanks for pointing it out, I have now fixed it. There is something odd about your patch. I cl

Re: [HACKERS] Segmentation fault in pg_dumpall from master down to 9.1 and other bug introduced by RLS

2014-11-13 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > What's bothering me is that I see this in pg_dumpall output from a 9.4 > or earlier database: > > ALTER ROLE postgres WITH SUPERUSER INHERIT CREATEROLE CREATEDB LOGIN > REPLICATION NOBYPASSRLS; Ah, yeah, good point. > That means that if you do a pg_upgra

Re: [HACKERS] using custom scan nodes to prototype parallel sequential scan

2014-11-13 Thread Kouhei Kaigai
> On 12 November 2014 07:54, David Rowley > wrote: > > On Tue, Nov 11, 2014 at 9:29 PM, Simon Riggs > wrote: > >> > >> > >> This plan type is widely used in reporting queries, so will hit > the > >> mainline of BI applications and many Mat View creations. >

Re: [HACKERS] Re: [GENERAL] Performance issue with libpq prepared queries on 9.3 and 9.4

2014-11-13 Thread David Johnston
On Thu, Nov 13, 2014 at 5:47 PM, Tom Lane wrote: > David G Johnston writes: > > Tom Lane-2 wrote > >> In the meantime, I assume that your real data contains a small > percentage > >> of values other than these two? If so, maybe cranking up the statistics > >> target would help. If the planner

Re: [HACKERS] what does this mean: "running xacts with xcnt == 0"

2014-11-13 Thread Peter Eisentraut
On 11/12/14 10:59 AM, Andres Freund wrote: > No, I really don't want to do that. When trying to see whether logical > replication started that's imo quite an importantdetail. Especially when > first seing > ereport(LOG, > (errmsg("logical decoding found initial s

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2014-11-13 Thread Andreas Karlsson
On 11/13/2014 03:38 AM, Alvaro Herrera wrote: configure is a generated file. If your patch touches it but not configure.in, there is a problem. Thanks for pointing it out, I have now fixed it. -- Andreas Karlsson diff --git a/configure b/configure new file mode 100755 index c4f70e8..bb801b4 *

Re: [HACKERS] using custom scan nodes to prototype parallel sequential scan

2014-11-13 Thread David Rowley
On Fri, Nov 14, 2014 at 1:19 PM, Simon Riggs wrote: > On 12 November 2014 07:54, David Rowley wrote: > > On Tue, Nov 11, 2014 at 9:29 PM, Simon Riggs > wrote: > >> > >> > >> This plan type is widely used in reporting queries, so will hit the > >> mainline of BI applications and many Mat View cr

Re: [HACKERS] EXPLAIN ANALYZE output weird for Top-N Sort

2014-11-13 Thread Tom Lane
Simon Riggs writes: > Limit (cost= rows=20 width=175) (actual time= rows=20 loops=1) >-> Sort (cost= rows=568733 width=175) (actual time= > rows=20 loops=1) > Sort Method: top-N heapsort > The Sort estimate shows 568733 rows, whereas the actual rows are 20. [ shru

Re: [HACKERS] Re: [GENERAL] Performance issue with libpq prepared queries on 9.3 and 9.4

2014-11-13 Thread Tom Lane
David G Johnston writes: > Tom Lane-2 wrote >> In the meantime, I assume that your real data contains a small percentage >> of values other than these two? If so, maybe cranking up the statistics >> target would help. If the planner knows that there are more than two >> values in the column, I t

[HACKERS] EXPLAIN ANALYZE output weird for Top-N Sort

2014-11-13 Thread Simon Riggs
Limit (cost= rows=20 width=175) (actual time= rows=20 loops=1) -> Sort (cost= rows=568733 width=175) (actual time= rows=20 loops=1) Sort Method: top-N heapsort The Sort estimate shows 568733 rows, whereas the actual rows are 20. Both are correct, in a way. The node

[HACKERS] Re: [GENERAL] Performance issue with libpq prepared queries on 9.3 and 9.4

2014-11-13 Thread David G Johnston
Tom Lane-2 wrote > In the meantime, I assume that your real data contains a small percentage > of values other than these two? If so, maybe cranking up the statistics > target would help. If the planner knows that there are more than two > values in the column, I think it would be less optimistic

Re: [HACKERS] [GENERAL] Performance issue with libpq prepared queries on 9.3 and 9.4

2014-11-13 Thread Tom Lane
Sam Saffron writes: > I have hit a rather odd issue with prepared queries on both pg 9.3 and 9.4 > beta. > I have this table (copy at http://samsaffron.com/testing.db.gz) with a > very odd performance profile: Interesting case. The issue seems to be that your statistics look like this: select

Re: [HACKERS] Teaching pg_dump to use NOT VALID constraints

2014-11-13 Thread Peter Eisentraut
On 11/13/14 5:07 AM, Simon Riggs wrote: > On 13 November 2014 00:20, Jim Nasby wrote: > >> Isn't the real use-case here that if constraints were valid when you dumped >> then we shouldn't have to *any* re-validate when we load? (Though, we'd have >> to be careful of that with CHECK because that c

Re: [HACKERS] using custom scan nodes to prototype parallel sequential scan

2014-11-13 Thread Simon Riggs
On 12 November 2014 00:54, Robert Haas wrote: > On Tue, Nov 11, 2014 at 3:29 AM, Simon Riggs wrote: >> * only functions marked as "CONTAINS NO SQL" >> We don't really know what proisparallel is, but we do know what >> CONTAINS NO SQL means and can easily check for it. >> Plus I already have a pat

Re: [HACKERS] using custom scan nodes to prototype parallel sequential scan

2014-11-13 Thread Simon Riggs
On 12 November 2014 07:54, David Rowley wrote: > On Tue, Nov 11, 2014 at 9:29 PM, Simon Riggs wrote: >> >> >> This plan type is widely used in reporting queries, so will hit the >> mainline of BI applications and many Mat View creations. >> This will allow SELECT count(*) FROM foo to go faster al

Re: [HACKERS] using custom scan nodes to prototype parallel sequential scan

2014-11-13 Thread Simon Riggs
On 12 November 2014 00:54, Robert Haas wrote: > I'm going to aim for the simpler: > > Hash Aggregate > -> Parallel Seq Scan > Workers: 4 > > Yeah, I know that won't perform as well as what you're proposing, but > I'm fairly sure it's simpler. Simple is best, so +1. -- Simon Riggs

Re: [HACKERS] tracking commit timestamps

2014-11-13 Thread Simon Riggs
On 13 November 2014 21:24, Robert Haas wrote: > On Thu, Nov 13, 2014 at 8:18 AM, Simon Riggs wrote: >> Ordering transactions in LSN order is very precisly the remit of the >> existing logical decoding API. Any user that wishes to see a commits >> in sequence can do so using that API. BDR already

Re: [HACKERS] Segmentation fault in pg_dumpall from master down to 9.1 and other bug introduced by RLS

2014-11-13 Thread Andres Freund
On 2014-11-13 18:24:53 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2014-11-13 18:00:45 -0500, Stephen Frost wrote: > >> Err. That might have come off poorly- I didn't mean that sarcastically > >> but was really wondering how far back you test (or how far back we > >> feel pg_dumpall nee

Re: [HACKERS] Segmentation fault in pg_dumpall from master down to 9.1 and other bug introduced by RLS

2014-11-13 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> Actually, I think that code is not just under-tested but poorly thought >> out. It will dump ALL roles from a pre-9.5 database with NOBYPASSRLS; >> even superusers. > Superusers are always considered to have it, regardless of if t

Re: [HACKERS] Segmentation fault in pg_dumpall from master down to 9.1 and other bug introduced by RLS

2014-11-13 Thread Tom Lane
Andres Freund writes: > On 2014-11-13 18:00:45 -0500, Stephen Frost wrote: >> Err. That might have come off poorly- I didn't mean that sarcastically >> but was really wondering how far back you test (or how far back we >> feel pg_dumpall needs to work against..). > pg_dump currently errors out b

Re: [HACKERS] Segmentation fault in pg_dumpall from master down to 9.1 and other bug introduced by RLS

2014-11-13 Thread Andres Freund
On 2014-11-13 18:00:45 -0500, Stephen Frost wrote: > * Stephen Frost (sfr...@snowman.net) wrote: > > I'm happy to fix it either way (and fix it for 8.1, and back to.. what? > > Postgres95?) > > Err. That might have come off poorly- I didn't mean that sarcastically > but was really wondering how f

Re: [HACKERS] Segmentation fault in pg_dumpall from master down to 9.1 and other bug introduced by RLS

2014-11-13 Thread Stephen Frost
* Stephen Frost (sfr...@snowman.net) wrote: > I'm happy to fix it either way (and fix it for 8.1, and back to.. what? > Postgres95?) Err. That might have come off poorly- I didn't mean that sarcastically but was really wondering how far back you test (or how far back we feel pg_dumpall needs to w

Re: [HACKERS] Missing FIN_CRC32 calls in logical replication code

2014-11-13 Thread Andres Freund
On 2014-11-14 00:04:52 +0200, Heikki Linnakangas wrote: > On 11/11/2014 06:55 PM, Andres Freund wrote: > >On 2014-11-03 21:58:51 +0200, Heikki Linnakangas wrote: > >>PS. I find the name "ReplicationSlotOnDiskDynamicSize" confusing, as it is > >>in fact a fixed size struct. I gather it's expected th

Re: [HACKERS] Segmentation fault in pg_dumpall from master down to 9.1 and other bug introduced by RLS

2014-11-13 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Gilles Darold writes: > > In the same query there is another bug introduced by commit 491c029 > > that add Row-Level Security Policies. Current master code has a broken > > pg_dumpall when trying to dump from a backend lower than 8.1. > > Actually, I thin

Re: [HACKERS] Segmentation fault in pg_dumpall from master down to 9.1 and other bug introduced by RLS

2014-11-13 Thread Tom Lane
Gilles Darold writes: > In the same query there is another bug introduced by commit 491c029 > that add Row-Level Security Policies. Current master code has a broken > pg_dumpall when trying to dump from a backend lower than 8.1. Actually, I think that code is not just under-tested but poorly tho

Re: [HACKERS] Inconsistencies in documentation of row-level locking

2014-11-13 Thread Michael Paquier
On Fri, Nov 14, 2014 at 2:47 AM, Alvaro Herrera wrote: > Thanks, I applied it to master and 9.4 after some slight > editorialization. Thanks. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpre

Re: [HACKERS] Missing FIN_CRC32 calls in logical replication code

2014-11-13 Thread Heikki Linnakangas
On 11/11/2014 06:55 PM, Andres Freund wrote: On 2014-11-03 21:58:51 +0200, Heikki Linnakangas wrote: PS. I find the name "ReplicationSlotOnDiskDynamicSize" confusing, as it is in fact a fixed size struct. I gather it's expected that the size of that part might change across versions, but by that

Re: [HACKERS] group locking: incomplete patch, just for discussion

2014-11-13 Thread Jeff Davis
On Thu, Nov 13, 2014 at 11:26 AM, Robert Haas wrote: > > On Thu, Nov 13, 2014 at 3:38 AM, Jeff Davis wrote: > > If two backends both have an exclusive lock on the relation for a join > > operation, that implies that they need to do their own synchronization, > > because obviously the lock manager

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2014-11-13 Thread Andres Freund
On November 13, 2014 10:23:41 PM CET, Peter Eisentraut wrote: >On 11/12/14 7:31 PM, Andres Freund wrote: >> Yes, it sucks. But it beats not being able to reindex a relation with >a >> primary key (referenced by a fkey) without waiting several hours by a >> couple magnitudes. And that's the current

Re: [HACKERS] pg_basebackup vs. Windows and tablespaces

2014-11-13 Thread Robert Haas
On Thu, Nov 13, 2014 at 4:33 PM, Peter Eisentraut wrote: > I'm not fully on board with that premise. (Get a better tar tool. > Submit a patch.) Noah was unable to find one that works: http://www.postgresql.org/message-id/20130801161519.ga334...@tornado.leadboat.com If most tar tools worked, an

Re: [HACKERS] pg_basebackup vs. Windows and tablespaces

2014-11-13 Thread Peter Eisentraut
On 11/13/14 11:52 AM, Alvaro Herrera wrote: > Amit Kapila wrote: > >> This patch is in "Ready for committer" stage for more than 1.5 months. >> I believe this is an important functionality such that without this tar >> format of pg_basebackup is not usable on Windows. I feel this >> will add a va

[HACKERS] Size of regression database

2014-11-13 Thread Tom Lane
I was testing backwards compatibility of pg_dumpall just now, and was somewhat astonished to notice the size of the output for the regression database compared to what it was not too long ago: -rw-rw-r--. 1 tgl tgl 4509135 Nov 13 16:19 dumpall.83 -rw-rw-r--. 1 tgl tgl 4514441 Nov 13 16:24 dumpal

Re: [HACKERS] tracking commit timestamps

2014-11-13 Thread Robert Haas
On Thu, Nov 13, 2014 at 8:18 AM, Simon Riggs wrote: > Ordering transactions in LSN order is very precisly the remit of the > existing logical decoding API. Any user that wishes to see a commits > in sequence can do so using that API. BDR already does this, as do > other users of the decoding API.

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2014-11-13 Thread Peter Eisentraut
On 11/12/14 7:31 PM, Andres Freund wrote: > Yes, it sucks. But it beats not being able to reindex a relation with a > primary key (referenced by a fkey) without waiting several hours by a > couple magnitudes. And that's the current situation. That's fine, but we have, for better or worse, defined

Re: [HACKERS] array exclusion constraints

2014-11-13 Thread Antonin Houska
franc...@hush.com wrote: > > CREATE TABLE test ( >foo text[], >EXCLUDE USING gist (foo WITH &&) > ); > > ERROR: data type text[] has no default operator class for access method > "gist" > HINT: You must specify an operator class for the index or define a default > operator class for t

[HACKERS] alternative model for handling locking in parallel groups

2014-11-13 Thread Robert Haas
Discussion of my incomplete group locking patch seems to have converged around two points: (1) Everybody agrees that undetected deadlocks are unacceptable. (2) Nobody agrees with my proposal to treat locks held by group members as mutually non-conflicting. As was probably evident from the emails

Re: [HACKERS] group locking: incomplete patch, just for discussion

2014-11-13 Thread Andres Freund
On November 13, 2014 8:50:18 PM CET, Tom Lane wrote: >Robert Haas writes: >> On Thu, Nov 13, 2014 at 3:38 AM, Jeff Davis >wrote: >>> If two backends both have an exclusive lock on the relation for a >join >>> operation, that implies that they need to do their own >synchronization, >>> because ob

Re: [HACKERS] alter user/role CURRENT_USER

2014-11-13 Thread Alvaro Herrera
Kyotaro HORIGUCHI wrote: > - Storage for new information > > The new struct NameId stores an identifier which telling what it > logically is using the new enum NameIdTypes. I think NameId is a bad name for this. My point is that NameId, as it stands, might be a name for anything, not just a rol

Re: [HACKERS] group locking: incomplete patch, just for discussion

2014-11-13 Thread Tom Lane
Robert Haas writes: > On Thu, Nov 13, 2014 at 3:38 AM, Jeff Davis wrote: >> If two backends both have an exclusive lock on the relation for a join >> operation, that implies that they need to do their own synchronization, >> because obviously the lock manager is not doing it for them. > This doe

Re: [HACKERS] Segmentation fault in pg_dumpall from master down to 9.1 and other bug introduced by RLS

2014-11-13 Thread Tom Lane
Gilles Darold writes: > There's a segfault when trying to dump global object from a running > 7.4.27 with a pg_dumpall of version 9.3.5 but also 9.2.9. Hm ... I make a practice of checking pg_dump's backwards compatibility from time to time, but I confess I've not tested pg_dumpall lately. Will t

Re: [HACKERS] logical decoding - reading a user catalog table

2014-11-13 Thread Andres Freund
Hi Steve, On 2014-10-28 22:38:29 -0400, Steve Singer wrote: > On 10/28/2014 01:31 PM, Andres Freund wrote: > >On 2014-10-25 18:18:07 -0400, Steve Singer wrote: > >>My logical decoding plugin is occasionally getting this error > >> > >>"could not resolve cmin/cmax of catalog tuple" > >> > >>I get

Re: [HACKERS] Another logical decoding assertion failure

2014-11-13 Thread Andres Freund
On 2014-08-15 14:53:45 +0200, Antonin Houska wrote: > http://www.postgresql.org/message-id/blu436-smtp12682d628f61ab9736099c3dc...@phx.gbl > recalls me that I also saw an assertion failure recently. Although I > wanted to isolate and report my issue when my vacation is over, this > report made me c

Re: [HACKERS] group locking: incomplete patch, just for discussion

2014-11-13 Thread Robert Haas
On Thu, Nov 13, 2014 at 3:38 AM, Jeff Davis wrote: > If two backends both have an exclusive lock on the relation for a join > operation, that implies that they need to do their own synchronization, > because obviously the lock manager is not doing it for them. This doesn't make sense to me. Why

[HACKERS] array exclusion constraints

2014-11-13 Thread francisk
CREATE TABLE test ( foo text[], EXCLUDE USING gist (foo WITH &&) ); ERROR: data type text[] has no default operator class for access method "gist" HINT: You must specify an operator class for the index or define a default operator class for the data type. It looks like exclusion constra

Re: [HACKERS] controlling psql's use of the pager a bit more

2014-11-13 Thread Merlin Moncure
On Thu, Nov 13, 2014 at 12:03 PM, David Johnston wrote: > On Thu, Nov 13, 2014 at 10:55 AM, Merlin Moncure wrote: >> >> On Thu, Nov 13, 2014 at 11:39 AM, Robert Haas >> wrote: >> > On Thu, Nov 13, 2014 at 11:54 AM, David G Johnston >> > wrote: >> >>> Because I might be quite happy with 100 or 2

Re: [HACKERS] controlling psql's use of the pager a bit more

2014-11-13 Thread Jeff Janes
On Thu, Nov 13, 2014 at 8:14 AM, Andres Freund wrote: > On 2014-11-13 11:09:06 -0500, Tom Lane wrote: > > Andrew Dunstan writes: > > > I often get annoyed because psql is a bit too aggressive when it > decides > > > whether to put output through the pager, and the only way to avoid this > > > is

Re: [HACKERS] controlling psql's use of the pager a bit more

2014-11-13 Thread Jeff Janes
On Thu, Nov 13, 2014 at 7:52 AM, Andrew Dunstan wrote: > > I often get annoyed because psql is a bit too aggressive when it decides > whether to put output through the pager, and the only way to avoid this is > to turn the pager off (in which case your next query might dump many > thousands of li

Re: [HACKERS] controlling psql's use of the pager a bit more

2014-11-13 Thread David Johnston
On Thu, Nov 13, 2014 at 10:55 AM, Merlin Moncure wrote: > On Thu, Nov 13, 2014 at 11:39 AM, Robert Haas > wrote: > > On Thu, Nov 13, 2014 at 11:54 AM, David G Johnston > > wrote: > >>> Because I might be quite happy with 100 or 200 lines I can just scroll > >>> in my terminal's scroll buffer, b

Re: [HACKERS] controlling psql's use of the pager a bit more

2014-11-13 Thread Merlin Moncure
On Thu, Nov 13, 2014 at 11:39 AM, Robert Haas wrote: > On Thu, Nov 13, 2014 at 11:54 AM, David G Johnston > wrote: >>> Because I might be quite happy with 100 or 200 lines I can just scroll >>> in my terminal's scroll buffer, but want to use the pager for more than >>> that. This is useful especi

Re: [HACKERS] Doing better at HINTing an appropriate column within errorMissingColumn()

2014-11-13 Thread Peter Geoghegan
On Thu, Nov 13, 2014 at 9:36 AM, Robert Haas wrote: > Committed. I changed varstr_leven() to varstr_levenshtein() because > abbrvs cn mk the code hrd to undstnd. And to grep. Thanks. I'll produce a revision of patch 2/2 soon. > And I removed the > StaticAssertStmt you added, because it's not a

Re: [HACKERS] Inconsistencies in documentation of row-level locking

2014-11-13 Thread Alvaro Herrera
Michael Paquier wrote: > Hi all, > > Currently all the row-level lock modes are described in the page for > SELECT query: > http://www.postgresql.org/docs/devel/static/explicit-locking.html#LOCKING-ROWS > However, after browsing the documentation, I noticed in the page > describing all the explici

Re: [HACKERS] controlling psql's use of the pager a bit more

2014-11-13 Thread Robert Haas
On Thu, Nov 13, 2014 at 11:54 AM, David G Johnston wrote: >> Because I might be quite happy with 100 or 200 lines I can just scroll >> in my terminal's scroll buffer, but want to use the pager for more than >> that. This is useful especially if I want to scroll back and see the >> results from a q

Re: [HACKERS] recovery_target_time and standby_mode

2014-11-13 Thread Robert Haas
On Thu, Nov 13, 2014 at 2:45 AM, Josh Berkus wrote: > On 11/12/2014 10:06 AM, Robert Haas wrote: >>> hat *appears* to be happening is that the pause_at_recovery_target, >>> > followed by the restart, on the replica causes it to advance one commit >>> > on timeline 1. But *not all the time*; this

Re: [HACKERS] Doing better at HINTing an appropriate column within errorMissingColumn()

2014-11-13 Thread Robert Haas
On Wed, Nov 12, 2014 at 8:00 PM, Peter Geoghegan wrote: > On Wed, Nov 12, 2014 at 4:54 PM, Peter Geoghegan wrote: >> Attached patch moves the Levenshtein distance implementation into core. > > Oops. Somehow managed to send a *.patch.swp file. :-) > > Here is the actual patch. Committed. I chan

Re: [HACKERS] Doing better at HINTing an appropriate column within errorMissingColumn()

2014-11-13 Thread Peter Geoghegan
On Thu, Nov 13, 2014 at 8:57 AM, Robert Haas wrote: >> My point is: I am not sure I can be defined as a reviewer of this >> patch or take any credit in this patch review knowing that the latest >> version submitted is a simple rebase of the version I did my first >> review on. Hence, code speaking

Re: [HACKERS] Doing better at HINTing an appropriate column within errorMissingColumn()

2014-11-13 Thread Robert Haas
On Wed, Nov 12, 2014 at 10:42 PM, Michael Paquier wrote: > On Tue, Nov 11, 2014 at 3:52 PM, Peter Geoghegan wrote: >> I'm pretty puzzled by this. Other than our "agree to disagree and >> defer to committer" position on the question of whether or not more >> than one suggestion can come from a sin

Re: [HACKERS] controlling psql's use of the pager a bit more

2014-11-13 Thread David G Johnston
Andrew Dunstan wrote > On 11/13/2014 11:09 AM, Tom Lane wrote: >> Andrew Dunstan < > andrew@ > > writes: >>> I often get annoyed because psql is a bit too aggressive when it decides >>> whether to put output through the pager, and the only way to avoid this >>> is to turn the pager off (in which

Re: [HACKERS] pg_basebackup vs. Windows and tablespaces

2014-11-13 Thread Alvaro Herrera
Amit Kapila wrote: > This patch is in "Ready for committer" stage for more than 1.5 months. > I believe this is an important functionality such that without this tar > format of pg_basebackup is not usable on Windows. I feel this > will add a value to pg_basebackup utility and moreover the need >

Re: [HACKERS] On partitioning

2014-11-13 Thread Robert Haas
On Thu, Nov 13, 2014 at 1:39 AM, Stephen Frost wrote: > Agreed- a node tree seems a bit too far to make this really work well.. > But, I'm curious what you were thinking specifically? I gave a pretty specific example in my email. > A node tree which > accepts an "argument" of the constant used i

Re: [HACKERS] controlling psql's use of the pager a bit more

2014-11-13 Thread Michael Banck
On Thu, Nov 13, 2014 at 05:14:47PM +0100, Andres Freund wrote: > On 2014-11-13 11:09:06 -0500, Tom Lane wrote: > > Andrew Dunstan writes: > > > I often get annoyed because psql is a bit too aggressive when it decides > > > whether to put output through the pager, and the only way to avoid this >

Re: [HACKERS] WIP: multivariate statistics / proof of concept

2014-11-13 Thread Tomas Vondra
Dne 13 Listopad 2014, 16:51, Katharina Büchse napsal(a): > On 13.11.2014 14:11, Tomas Vondra wrote: > >> The only place where I think this might work are the associative rules. >> It's simple to specify rules like ("ZIP code" implies "city") and we >> could >> even do some simple check against the

Re: [HACKERS] controlling psql's use of the pager a bit more

2014-11-13 Thread Andrew Dunstan
On 11/13/2014 11:09 AM, Tom Lane wrote: Andrew Dunstan writes: I often get annoyed because psql is a bit too aggressive when it decides whether to put output through the pager, and the only way to avoid this is to turn the pager off (in which case your next query might dump many thousands of l

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2014-11-13 Thread Robert Haas
On Wed, Nov 12, 2014 at 7:31 PM, Andres Freund wrote: > But I think it won't work realistically. We have a *lot* of > infrastructure that refers to indexes using it's primary key. I don't > think we want to touch all those places to also disambiguate on some > other factor. All the relevant APIs a

[HACKERS] Segmentation fault in pg_dumpall from master down to 9.1 and other bug introduced by RLS

2014-11-13 Thread Gilles Darold
Hi, There's a segfault when trying to dump global object from a running 7.4.27 with a pg_dumpall of version 9.3.5 but also 9.2.9. $ pg_dumpall -g -h localhost -p 5474 column number -1 is out of range 0..12 Segmentation fault (core dumped) The problem comes from the first columns of the

Re: [HACKERS] controlling psql's use of the pager a bit more

2014-11-13 Thread Andres Freund
On 2014-11-13 11:09:06 -0500, Tom Lane wrote: > Andrew Dunstan writes: > > I often get annoyed because psql is a bit too aggressive when it decides > > whether to put output through the pager, and the only way to avoid this > > is to turn the pager off (in which case your next query might dump m

Re: [HACKERS] controlling psql's use of the pager a bit more

2014-11-13 Thread Tom Lane
Andrew Dunstan writes: > I often get annoyed because psql is a bit too aggressive when it decides > whether to put output through the pager, and the only way to avoid this > is to turn the pager off (in which case your next query might dump many > thousands of lines to the screen). I'd like a w

Re: [HACKERS] controlling psql's use of the pager a bit more

2014-11-13 Thread Thom Brown
On 13 November 2014 15:52, Andrew Dunstan wrote: > > I often get annoyed because psql is a bit too aggressive when it decides > whether to put output through the pager, and the only way to avoid this is > to turn the pager off (in which case your next query might dump many > thousands of lines to

[HACKERS] controlling psql's use of the pager a bit more

2014-11-13 Thread Andrew Dunstan
I often get annoyed because psql is a bit too aggressive when it decides whether to put output through the pager, and the only way to avoid this is to turn the pager off (in which case your next query might dump many thousands of lines to the screen). I'd like a way to be able to specify a mi

Re: [HACKERS] WIP: multivariate statistics / proof of concept

2014-11-13 Thread Katharina Büchse
On 13.11.2014 14:11, Tomas Vondra wrote: Dne 13 Listopad 2014, 12:31, Simon Riggs napsal(a): On 12 October 2014 23:00, Tomas Vondra wrote: It however seems to be working sufficiently well at this point, enough to get some useful feedback. So here we go. This looks interesting and useful. Wh

Re: [HACKERS] On the warpath again about ill-considered inclusion nests

2014-11-13 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> has any business including. And if that weren't bad enough, it's >> been included into utils/rel.h (relcache), > This is for the definition of RowSecurityDesc. I'm happy to move that > to a utils/rowsecurity.h instead, following

Re: [HACKERS] WAL format and API changes (9.5)

2014-11-13 Thread Andres Freund
On 2014-11-13 15:33:44 +0200, Heikki Linnakangas wrote: > Here's a new version, with big changes again to the record format. Have a > look at xlogrecord.h for the details, but in a nutshell: > > 1. The overall format is now: XLogRecord, per-block headers, header for main > data portion, per-block

Re: [HACKERS] Failback to old master

2014-11-13 Thread Ants Aasma
On Thu, Nov 13, 2014 at 10:05 AM, Heikki Linnakangas wrote: >> In this case the >> old master will request recovery from a point after the timeline >> switch and the new master will reply with an error. So it is safe to >> try re-adding a crashed master as a slave, but this might fail. > > > Are

Re: [HACKERS] tracking commit timestamps

2014-11-13 Thread Petr Jelinek
On 13/11/14 14:18, Simon Riggs wrote: So in summary... the components are * Commit LSN order is useful for applying changes - available by logical decoding * Commit timestamps and nodeid are useful for conflict resolution - available from this patch Both components have been designed in ways th

Re: [HACKERS] tracking commit timestamps

2014-11-13 Thread Simon Riggs
On 9 November 2014 16:57, Steve Singer wrote: > On 11/07/2014 07:07 PM, Petr Jelinek wrote: > > >> The list of what is useful might be long, but we can't have everything >> there as there are space constraints, and LSN is another 8 bytes and I still >> want to have some bytes for storing the "orig

Re: [HACKERS] WIP: multivariate statistics / proof of concept

2014-11-13 Thread Tomas Vondra
Dne 13 Listopad 2014, 12:31, Simon Riggs napsal(a): > On 12 October 2014 23:00, Tomas Vondra wrote: > >> It however seems to be working sufficiently well at this point, enough >> to get some useful feedback. So here we go. > > This looks interesting and useful. > > What I'd like to check before a

Re: [HACKERS] tracking commit timestamps

2014-11-13 Thread Petr Jelinek
On 13/11/14 07:04, Michael Paquier wrote: On Wed, Nov 12, 2014 at 10:06 PM, Petr Jelinek wrote: Brief list of changes: - the commit timestamp record now stores timestamp, lsn and nodeid Now that not only the commit timestamp is stored, calling that "commit timestamp", "committs" or "commit_t

Re: [HACKERS] Add CREATE support to event triggers

2014-11-13 Thread Andres Freund
On 2014-11-13 07:17:32 -0500, Robert Haas wrote: > On Wed, Nov 12, 2014 at 4:58 PM, Andres Freund wrote: > > That's already the situation today with all the logical replication > > solutions. They *constantly* break in the field. Most commonly because > > of DDL differences. > > Right. And that'

  1   2   >