Re: [HACKERS] Change GetLastImportantRecPtr's definition? (wasSkip checkpoints, archiving on idle systems.)

2017-05-04 Thread Andres Freund
Hi, On 2017-05-05 11:50:12 +0530, Amit Kapila wrote: > I see that EndPos can be updated in one of the cases after releasing > the lock (refer below code). Won't that matter? I can't see how it'd in the cases that'd matter for GetLastImportantRecPtr() - but it'd probably good to note it in the com

Re: [HACKERS] Adding support for Default partition in partitioning

2017-05-04 Thread Rajkumar Raghuwanshi
Hi Rahila, pg_restore is failing for default partition, dump file still storing old syntax of default partition. create table lpd (a int, b int, c varchar) partition by list(a); create table lpd_d partition of lpd DEFAULT; create database bkp owner 'edb'; grant all on DATABASE bkp to edb; --tak

Re: [HACKERS] Change GetLastImportantRecPtr's definition? (wasSkip checkpoints, archiving on idle systems.)

2017-05-04 Thread Amit Kapila
On Fri, May 5, 2017 at 11:43 AM, Andres Freund wrote: > On 2017-05-05 11:04:14 +0530, Amit Kapila wrote: >> On Fri, May 5, 2017 at 6:54 AM, Andres Freund wrote: >> > Hi, >> > >> > On 2016-12-22 19:33:30 +, Andres Freund wrote: >> >> Skip checkpoints, archiving on idle systems. >> > >> > As pa

Re: [HACKERS] Change GetLastImportantRecPtr's definition? (wasSkip checkpoints, archiving on idle systems.)

2017-05-04 Thread Andres Freund
On 2017-05-05 11:04:14 +0530, Amit Kapila wrote: > On Fri, May 5, 2017 at 6:54 AM, Andres Freund wrote: > > Hi, > > > > On 2016-12-22 19:33:30 +, Andres Freund wrote: > >> Skip checkpoints, archiving on idle systems. > > > > As part of an independent bugfix I noticed that Michael & I appear to

Re: [HACKERS] modeling parallel contention (was: Parallel Append implementation)

2017-05-04 Thread Amit Kapila
On Fri, May 5, 2017 at 7:07 AM, Andres Freund wrote: > On 2017-05-02 15:13:58 -0400, Robert Haas wrote: >> On Tue, Apr 18, 2017 at 2:48 AM, Amit Khandekar >> wrote: >> The main things that keeps this from being a crippling issue right now >> is the fact that we tend not to use that many parallel

Re: [HACKERS] modeling parallel contention (was: Parallel Append implementation)

2017-05-04 Thread Amit Khandekar
On 5 May 2017 at 07:50, David Rowley wrote: > On 3 May 2017 at 07:13, Robert Haas wrote: >> It is of course possible that the Parallel Seq Scan could run into >> contention problems if the number of workers is large, but in my >> experience there are bigger problems here. The non-parallel Seq S

Re: [HACKERS] Change GetLastImportantRecPtr's definition? (wasSkip checkpoints, archiving on idle systems.)

2017-05-04 Thread Amit Kapila
On Fri, May 5, 2017 at 6:54 AM, Andres Freund wrote: > Hi, > > On 2016-12-22 19:33:30 +, Andres Freund wrote: >> Skip checkpoints, archiving on idle systems. > > As part of an independent bugfix I noticed that Michael & I appear to > have introduced an off-by-one here. A few locations do compa

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-05-04 Thread Michael Paquier
On Wed, May 3, 2017 at 12:25 AM, Peter Eisentraut wrote: > On 5/2/17 10:08, Michael Paquier wrote: >> On Tue, May 2, 2017 at 9:30 PM, Peter Eisentraut >> wrote: >>> On 5/2/17 03:11, Petr Jelinek wrote: logical decoding can theoretically do HOT pruning (even if the chance is really small

Re: [HACKERS] statement_timeout is not working as expected with postgres_fdw

2017-05-04 Thread Amit Kapila
On Thu, May 4, 2017 at 10:40 PM, Robert Haas wrote: > On Thu, May 4, 2017 at 1:04 PM, Amit Kapila wrote: >> On Thu, May 4, 2017 at 10:18 PM, Robert Haas wrote: >>> On Thu, May 4, 2017 at 12:18 PM, Amit Kapila >>> wrote: As soon as the first command fails due to timeout, we will set '

Re: [HACKERS] statement_timeout is not working as expected with postgres_fdw

2017-05-04 Thread Amit Kapila
On Thu, May 4, 2017 at 8:08 PM, Robert Haas wrote: > On Thu, May 4, 2017 at 7:13 AM, Amit Kapila wrote: >> In pgfdw_xact_callback, if the execution of ABORT TRANSACTION fails >> due to any reason then I think it will close the connection. The >> relavant code is: >> if (PQstatus(entry->conn) !=

Re: [HACKERS] compiler warning with VS 2017

2017-05-04 Thread Tom Lane
Haribabu Kommi writes: > I am getting a compiler warning when I build the latest HEAD PostgreSQL with > visual studio 2017. > The code at the line is, > tuple->values[i] = (char *) (Size)0xdeadbeef; /* make bad usage more obvious > */ Yeah, you're not the first to complain about this. To my min

Re: [HACKERS] CTE inlining

2017-05-04 Thread Andres Freund
Hi, On 2017-05-04 19:57:21 -0700, Joe Conway wrote: > One thought, is that we treat a CTE in a similar way to foreign tables, > with the same set of push downs. A bit surprised about that suggestion - there seems to be very little similarity between the cases. What'd be the benefit of that? With

Re: [HACKERS] CTE inlining

2017-05-04 Thread Joe Conway
On 05/04/2017 07:04 PM, Tom Lane wrote: > Craig Ringer writes: >> We're carefully maintaining this bizarre cognitive dissonance where we >> justify the need for using this as a planner hint at the same time as >> denying that we have a hint. That makes it hard to make progress here. >> I think the

Re: [HACKERS] modeling parallel contention (was: Parallel Append implementation)

2017-05-04 Thread Thomas Munro
On Fri, May 5, 2017 at 2:23 PM, David Rowley wrote: > On 5 May 2017 at 13:37, Andres Freund wrote: >> On 2017-05-02 15:13:58 -0400, Robert Haas wrote: >>> Multiple people (including David Rowley >>> as well as folks here at EnterpriseDB) have demonstrated that for >>> certain queries, we can actu

Re: [HACKERS] modeling parallel contention (was: Parallel Append implementation)

2017-05-04 Thread Andres Freund
On 2017-05-04 19:45:33 -0700, Andres Freund wrote: > Increment phs_cblock without checking rs_nblocks, but outside of the > lock do a % scan->rs_nblocks, to get the "actual" position. Finish if > (phs_cblock - phs_startblock) / scan->rs_nblocks >= 1. Err, as I've been pointed to: It should be s/l

Re: [HACKERS] modeling parallel contention (was: Parallel Append implementation)

2017-05-04 Thread Andres Freund
On 2017-05-05 14:40:43 +1200, David Rowley wrote: > On 5 May 2017 at 14:36, Andres Freund wrote: > > I wonder how much doing the atomic ops approach alone can help, that > > doesn't have the issue that the work might be unevenly distributed > > between pages. > > I wondered that too, since I thou

Re: [HACKERS] modeling parallel contention (was: Parallel Append implementation)

2017-05-04 Thread David Rowley
On 5 May 2017 at 14:36, Andres Freund wrote: > I wonder how much doing the atomic ops approach alone can help, that > doesn't have the issue that the work might be unevenly distributed > between pages. I wondered that too, since I though the barrier for making this change would be lower by doing

Re: [HACKERS] modeling parallel contention (was: Parallel Append implementation)

2017-05-04 Thread David Rowley
On 3 May 2017 at 07:13, Robert Haas wrote: > Multiple people (including David Rowley > as well as folks here at EnterpriseDB) have demonstrated that for > certain queries, we can actually use a lot more workers and everything > works great. The problem is that for other queries, using a lot of >

Re: [HACKERS] modeling parallel contention (was: Parallel Append implementation)

2017-05-04 Thread Andres Freund
Hi, On 2017-05-05 14:20:48 +1200, David Rowley wrote: > Yeah, I did get some time to look over the contention in Parallel Seq > Scan a while back and I discovered that on the machine that I was > testing on. the lock obtained in heap_parallelscan_nextpage() was > causing workers to have to wait fo

Re: [HACKERS] modeling parallel contention (was: Parallel Append implementation)

2017-05-04 Thread David Rowley
On 5 May 2017 at 13:37, Andres Freund wrote: > On 2017-05-02 15:13:58 -0400, Robert Haas wrote: >> Multiple people (including David Rowley >> as well as folks here at EnterpriseDB) have demonstrated that for >> certain queries, we can actually use a lot more workers and everything >> works great.

[HACKERS] compiler warning with VS 2017

2017-05-04 Thread Haribabu Kommi
I am getting a compiler warning when I build the latest HEAD PostgreSQL with visual studio 2017. src/backend/replication/logical/proto.c(482): warning C4312: 'type cast': conversion from 'unsigned int' to 'char *' of greater size Details of the warning is available in the link [1]. The code at t

Re: [HACKERS] modeling parallel contention (was: Parallel Append implementation)

2017-05-04 Thread David Rowley
On 3 May 2017 at 07:13, Robert Haas wrote: > It is of course possible that the Parallel Seq Scan could run into > contention problems if the number of workers is large, but in my > experience there are bigger problems here. The non-parallel Seq Scan > can also contend -- not of course over the s

Re: [HACKERS] CTE inlining

2017-05-04 Thread Stephen Frost
Tom, all, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Craig Ringer writes: > > We're carefully maintaining this bizarre cognitive dissonance where we > > justify the need for using this as a planner hint at the same time as > > denying that we have a hint. That makes it hard to make progress here. >

Re: [HACKERS] CTE inlining

2017-05-04 Thread Serge Rielau
In my past life when I was faced with such debates I argued that the number of customers We are hoping to attract in the future is much bigger than the ones we risk offending. Doesn't mean I wanted to piss everyone off. Just that I didn't want to be held hostage by history. Cheers Serge PS: On

Re: [HACKERS] CTE inlining

2017-05-04 Thread Tom Lane
Craig Ringer writes: > We're carefully maintaining this bizarre cognitive dissonance where we > justify the need for using this as a planner hint at the same time as > denying that we have a hint. That makes it hard to make progress here. > I think there's fear that we're setting some kind of prec

Re: [HACKERS] modeling parallel contention (was: Parallel Append implementation)

2017-05-04 Thread Andres Freund
On 2017-05-02 15:13:58 -0400, Robert Haas wrote: > On Tue, Apr 18, 2017 at 2:48 AM, Amit Khandekar > wrote: > The main things that keeps this from being a crippling issue right now > is the fact that we tend not to use that many parallel workers in the > first place. We're trying to scale a quer

Re: [HACKERS] CTE inlining

2017-05-04 Thread Craig Ringer
On 5 May 2017 at 08:17, Joe Conway wrote: > On 05/04/2017 05:03 PM, Craig Ringer wrote: >> On 5 May 2017 02:52, "Tom Lane" wrote: >> I haven't been keeping close tabs either, but surely we still have >> to have >> the optimization fence in (at least) all these cases: >> >> * CTE co

Re: [HACKERS] PG 10 release notes

2017-05-04 Thread Andres Freund
On 2017-05-04 18:23:38 -0700, Peter Geoghegan wrote: > On Thu, May 4, 2017 at 6:08 PM, Robert Haas wrote: > >> E.g. to power amazon's data migration service (yes, that scares me). > > > > If you recall, I did predict prior to commit that test_decoding would > > get put into production use regardle

[HACKERS] Change GetLastImportantRecPtr's definition? (wasSkip checkpoints, archiving on idle systems.)

2017-05-04 Thread Andres Freund
Hi, On 2016-12-22 19:33:30 +, Andres Freund wrote: > Skip checkpoints, archiving on idle systems. As part of an independent bugfix I noticed that Michael & I appear to have introduced an off-by-one here. A few locations do comparisons like: /* * Only log if enough tim

Re: [HACKERS] PG 10 release notes

2017-05-04 Thread Peter Geoghegan
On Thu, May 4, 2017 at 6:08 PM, Robert Haas wrote: >> E.g. to power amazon's data migration service (yes, that scares me). > > If you recall, I did predict prior to commit that test_decoding would > get put into production use regardless of the name. I thought you were completely wrong when you s

Re: [HACKERS] PG 10 release notes

2017-05-04 Thread Robert Haas
On Thu, May 4, 2017 at 8:46 PM, Bruce Momjian wrote: > On Thu, May 4, 2017 at 05:09:40PM -0700, Andres Freund wrote: >> > > I would not in any way refer to logical decoding as being only a proof >> > > of concept, even before logical replication. >> > >> > The community ships a reliable logical _

Re: [HACKERS] PG 10 release notes

2017-05-04 Thread Andres Freund
On 2017-05-04 21:08:41 -0400, Robert Haas wrote: > On Thu, May 4, 2017 at 8:09 PM, Andres Freund wrote: > > On Mon, May 1, 2017 at 08:02:46AM -0400, Robert Haas wrote: > >> Even test_decoding is (perhaps regrettably) being used to build production > >> solutions. > > > > E.g. to power amazon's d

Re: [HACKERS] PG 10 release notes

2017-05-04 Thread Robert Haas
On Thu, May 4, 2017 at 8:09 PM, Andres Freund wrote: > On Mon, May 1, 2017 at 08:02:46AM -0400, Robert Haas wrote: >> Even test_decoding is (perhaps regrettably) being used to build production >> solutions. > > E.g. to power amazon's data migration service (yes, that scares me). If you recall,

Re: [HACKERS] PG 10 release notes

2017-05-04 Thread Andres Freund
On 2017-05-04 20:46:24 -0400, Bruce Momjian wrote: > On Thu, May 4, 2017 at 05:09:40PM -0700, Andres Freund wrote: > > > > I would not in any way refer to logical decoding as being only a proof > > > > of concept, even before logical replication. > > > > > > The community ships a reliable logical

Re: [HACKERS] PG 10 release notes

2017-05-04 Thread Robert Haas
On Thu, May 4, 2017 at 7:56 PM, Bruce Momjian wrote: > The community ships a reliable logical _encoding_, and a test logical > _decoding_. As far as I am aware, logical encoding is a term you just made up, because it's not referenced anywhere in the commit log or the source tree, unlike logical d

Re: [HACKERS] PG 10 release notes

2017-05-04 Thread Bruce Momjian
On Thu, May 4, 2017 at 05:09:40PM -0700, Andres Freund wrote: > > > I would not in any way refer to logical decoding as being only a proof > > > of concept, even before logical replication. > > > > The community ships a reliable logical _encoding_, and a test logical > > _decoding_. > > Yes, so

Re: [HACKERS] snapbuild woes

2017-05-04 Thread Andres Freund
On 2017-05-04 17:00:04 -0700, Andres Freund wrote: > Attached is a prototype patch for that. Oops. Andres >From b6eb46e376e40f3e2e9a55d16b1b37b27904564b Mon Sep 17 00:00:00 2001 From: Andres Freund Date: Thu, 4 May 2017 16:40:52 -0700 Subject: [PATCH 1/2] WIP: Fix off-by-one around GetLastImport

[HACKERS] no test coverage for ALTER FOREIGN DATA WRAPPER name HANDLER ...

2017-05-04 Thread Mark Dilger
Hackers, just FYI, I cannot find any regression test coverage of this part of the grammar, not even in the contrib/ directory or TAP tests. I was going to submit a patch to help out, and discovered it is not so easy to do, and perhaps that is why the coverage is missing. My apologies for the

Re: [HACKERS] PG 10 release notes

2017-05-04 Thread Andres Freund
On 2017-05-04 19:56:21 -0400, Bruce Momjian wrote: > On Mon, May 1, 2017 at 08:02:46AM -0400, Robert Haas wrote: > This came up from discussion related to this item: > > the ability of logical decoding to follow timeline switches > > My point was that based on the text it is test_decoding

Re: [HACKERS] PG 10 release notes

2017-05-04 Thread Bruce Momjian
On Thu, May 4, 2017 at 07:01:17PM -0300, Alvaro Herrera wrote: > Thanks for doing this, looks great. A few notes: > > > > >Add the ability to compute a correlation ratio and the number of >distinct values on several columns (Tomas Vondra, David Rowley) >

Re: [HACKERS] CTE inlining

2017-05-04 Thread Joe Conway
On 05/04/2017 05:03 PM, Craig Ringer wrote: > On 5 May 2017 02:52, "Tom Lane" wrote: > I haven't been keeping close tabs either, but surely we still have > to have > the optimization fence in (at least) all these cases: > > * CTE contains INSERT/UPDATE/DELETE > * CTE contains S

Re: [HACKERS] CTE inlining

2017-05-04 Thread Craig Ringer
On 5 May 2017 06:04, "Andreas Karlsson" wrote: On 05/04/2017 06:22 PM, Andrew Dunstan wrote: > I wrote this query: > > select (json_populate_record(null::mytype, myjson)).* > from mytable; > > > It turned out that this was an order of magnitude faster: > > with r as > ( >

Re: [HACKERS] PG 10 release notes

2017-05-04 Thread Bruce Momjian
On Thu, May 4, 2017 at 06:02:58PM -0300, Alvaro Herrera wrote: > > I can't see how this can be added to an existing BRIN entry, so it would > > have to be new. The text would be: > > > > Improve accuracy in determining if a BRIN index scan is beneficial > > > > though this not something I w

Re: [HACKERS] PG 10 release notes

2017-05-04 Thread Andres Freund
On 2017-05-04 19:56:21 -0400, Bruce Momjian wrote: > On Mon, May 1, 2017 at 08:02:46AM -0400, Robert Haas wrote: > > On Tue, Apr 25, 2017 at 11:01 AM, Bruce Momjian wrote: > > >> Or the ability of logical decoding to follow timeline switches. > > > > > > I didn't think logical decoding was really

Re: [HACKERS] PG 10 release notes

2017-05-04 Thread Bruce Momjian
On Mon, May 1, 2017 at 10:20:38AM -0400, Robert Haas wrote: > I'm pretty sure this is not the first year in which your policy of > excluding certain performance-related items has met with opposition. > I agree that there are some improvements that are sufficiently small > and boring that they do n

Re: [HACKERS] CTE inlining

2017-05-04 Thread Craig Ringer
On 5 May 2017 02:52, "Tom Lane" wrote: Tomas Vondra writes: > On 5/4/17 8:03 PM, Joe Conway wrote: >>> I haven't been able to follow this incredibly long thread, so please >>> excuse me if way off base, but are we talking about that a CTE would be >>> silently be rewritten as an inline expressio

Re: [HACKERS] snapbuild woes

2017-05-04 Thread Andres Freund
Hi, On 2017-05-02 08:55:53 +0200, Petr Jelinek wrote: > Aah, now I understand we talked about slightly different things, I > considered the running thing to be first step towards tracking aborted > txes everywhere. > I think > we'll have to revisit tracking of aborted transactions in PG11 then > t

Re: [HACKERS] PG 10 release notes

2017-05-04 Thread Bruce Momjian
On Mon, May 1, 2017 at 08:02:46AM -0400, Robert Haas wrote: > On Tue, Apr 25, 2017 at 11:01 AM, Bruce Momjian wrote: > >> Or the ability of logical decoding to follow timeline switches. > > > > I didn't think logical decoding was really more than a proof-of-concept > > until now. > > /me searche

Re: [HACKERS] Potential issue with alter system

2017-05-04 Thread Joshua D. Drake
On 05/04/2017 12:49 PM, Tom Lane wrote: "Joshua D. Drake" writes: So I did this: If you have other entries you want to keep in the postgresql.auto.conf file, you could get away with manually editing it to remove the newline. Got it. Thanks for digging in. This is actually a very real and

Re: [HACKERS] PG 10 release notes

2017-05-04 Thread Bruce Momjian
On Fri, Apr 28, 2017 at 01:12:34PM +0900, Masahiko Sawada wrote: > On Tue, Apr 25, 2017 at 10:31 AM, Bruce Momjian wrote: > > I have committed the first draft of the Postgres 10 release notes. They > > are current as of two days ago, and I will keep them current. Please > > give me any feedback

Re: [HACKERS] PG 10 release notes

2017-05-04 Thread Bruce Momjian
On Thu, Apr 27, 2017 at 04:05:09PM +0100, Dagfinn Ilmari Mannsåker wrote: > Bruce Momjian writes: > > > I have committed the first draft of the Postgres 10 release notes. They > > are current as of two days ago, and I will keep them current. Please > > give me any feedback you have. > > I noti

Re: [HACKERS] PG 10 release notes

2017-05-04 Thread Bruce Momjian
On Thu, Apr 27, 2017 at 03:04:57PM +0200, Daniel Verite wrote: > Fabien COELHO wrote: > > >Fix psql \p to always print what would be executed by \g or \w (Daniel > >Vérité) > > > >Previously \p didn't properly print the reverted-to command after a > >buffer contents reset. C

Re: [HACKERS] PG 10 release notes

2017-05-04 Thread Bruce Momjian
On Thu, Apr 27, 2017 at 10:43:34AM +0200, Fabien COELHO wrote: > > Hello Bruce, > > >I have committed the first draft of the Postgres 10 release notes. They > >are current as of two days ago, and I will keep them current. Please > >give me any feedback you have. > > About: > > """ > Fix psq

Re: [HACKERS] PG 10 release notes

2017-05-04 Thread Bruce Momjian
On Thu, Apr 27, 2017 at 10:21:44AM +0500, Andrew Borodin wrote: > Hi, Bruce! > > 2017-04-25 6:31 GMT+05:00 Bruce Momjian : > > The only unusual thing is that this release has ~180 items while most > > recent release have had ~220. The pattern I see that there are more > > large features in this r

Re: [HACKERS] logical replication syntax (was DROP SUBSCRIPTION, query cancellations and slot handling)

2017-05-04 Thread Petr Jelinek
On 04/05/17 23:29, Tom Lane wrote: > Robert Haas writes: >> On Wed, May 3, 2017 at 12:38 AM, Petr Jelinek >> wrote: >>> Ok, Let me be clear, I actually happen to agree with your proposal. The >>> reason I am moaning is that I always seem to find myself doing tons of >>> mechanical work to rewrite

Re: [HACKERS] PG 10 release notes

2017-05-04 Thread Andres Freund
On 2017-05-04 17:33:13 -0500, Merlin Moncure wrote: > On Mon, May 1, 2017 at 7:02 AM, Robert Haas wrote: > > On Tue, Apr 25, 2017 at 11:01 AM, Bruce Momjian wrote: > >> I didn't think logical decoding was really more than a proof-of-concept > >> until now. > > > > /me searches for jaw on floor.

Re: [HACKERS] PG 10 release notes

2017-05-04 Thread Andres Freund
On 2017-04-25 15:29:01 -0400, Bruce Momjian wrote: > Uh, the only logical decoding code that I know we ship pre-PG 10 is > contrib/test_decoding/. That's completely wrong. src/backend/replication/logical/ is a a bit bigger than that... - Andres -- Sent via pgsql-hackers mailing list (pgsql-ha

Re: [HACKERS] PG 10 release notes

2017-05-04 Thread Merlin Moncure
On Mon, May 1, 2017 at 7:02 AM, Robert Haas wrote: > On Tue, Apr 25, 2017 at 11:01 AM, Bruce Momjian wrote: >> I didn't think logical decoding was really more than a proof-of-concept >> until now. > > /me searches for jaw on floor. > > I would not in any way refer to logical decoding as being onl

Re: [HACKERS] WITH clause in CREATE STATISTICS

2017-05-04 Thread Sven R. Kunze
On 04.05.2017 23:13, Tom Lane wrote: I'm not against what you've done here, because I had no love for USING in this context anyway; it conveys approximately nothing to the mind about what is in the list it's introducing. But I'm concerned whether we're boxing ourselves in by using ON. Actually,

Re: [HACKERS] CTE inlining

2017-05-04 Thread Andreas Karlsson
On 05/04/2017 06:22 PM, Andrew Dunstan wrote: I wrote this query: select (json_populate_record(null::mytype, myjson)).* from mytable; It turned out that this was an order of magnitude faster: with r as ( select json_populate_record(null::mytype, myjson) as x from

Re: [HACKERS] what's up with IDENTIFIER_LOOKUP_EXPR?

2017-05-04 Thread Tom Lane
Robert Haas writes: > The PLPGSQL_DTYPE_* constants are another thing that's not really > documented. Yeah :-(. Complain to Jan sometime. > You've mentioned that we should get rid of > PLPGSQL_DTYPE_ROW in favor of, uh, whatever's better than that, but > it's not clear to me what that really me

Re: [HACKERS] WIP Patch: Precalculate stable functions, infrastructure v1

2017-05-04 Thread Alexander Korotkov
On Thu, May 4, 2017 at 7:51 PM, Marina Polyakova wrote: > and here I send infrastructure patch which includes <...> >> > > Next 2 patches: > > Patch 'planning and execution', which includes: > - replacement nonvolatile functions and operators by appropriate cached > expressions; > - planning and

Re: [HACKERS] PG 10 release notes

2017-05-04 Thread Alvaro Herrera
Thanks for doing this, looks great. A few notes: Add the ability to compute a correlation ratio and the number of distinct values on several columns (Tomas Vondra, David Rowley) I think this should be worded in terms of "extended data statistics" or suc

Re: [HACKERS] Row Level Security UPDATE Confusion

2017-05-04 Thread Stephen Frost
Robert, all, * Robert Haas (robertmh...@gmail.com) wrote: > On Fri, Apr 14, 2017 at 9:16 AM, Stephen Frost wrote: > > I agreed already up-thread that there's an issue there and will be > > looking to fix it. That comment was simply replying to Rod's point that > > the documentation could also be

Re: [HACKERS] pg_dump emits ALTER TABLE ONLY partitioned_table

2017-05-04 Thread Stephen Frost
Amit, * Amit Langote (amitlangot...@gmail.com) wrote: > On Wed, May 3, 2017 at 12:05 PM, Stephen Frost wrote: > > Assuming this looks good to you, I'll push it tomorrow, possibly with > > other minor adjustments and perhaps a few more tests. > > Your latest patch looks good to me. Found a few m

Re: [HACKERS] logical replication syntax (was DROP SUBSCRIPTION, query cancellations and slot handling)

2017-05-04 Thread Tom Lane
Robert Haas writes: > On Wed, May 3, 2017 at 12:38 AM, Petr Jelinek > wrote: >> Ok, Let me be clear, I actually happen to agree with your proposal. The >> reason I am moaning is that I always seem to find myself doing tons of >> mechanical work to rewrite some cosmetic aspect of some patch based

Re: [HACKERS] Missing feature in Phrase Search?

2017-05-04 Thread Sven R. Kunze
Hi everybody, On 21.04.2017 20:47, Josh Berkus wrote: Oleg, Teodor, folks: I was demo'ing phrase search for a meetup yesterday, and the user feedback I got showed that there's a missing feature with phrase search. Let me explain by example: 'fix <-> error' will match 'fixed error', 'fixing

Re: [HACKERS] CTE inlining

2017-05-04 Thread Gavin Flower
On 05/05/17 06:39, Tomas Vondra wrote: On 5/4/17 8:03 PM, Joe Conway wrote: On 05/04/2017 10:56 AM, Andrew Dunstan wrote: On 05/04/2017 01:52 PM, Joe Conway wrote: On 05/04/2017 10:33 AM, Alvaro Herrera wrote: I'm not sure what your point is. We know that for some cases the optimization

Re: [HACKERS] WITH clause in CREATE STATISTICS

2017-05-04 Thread Tom Lane
Alvaro Herrera writes: > Here's a patch implementing this idea. From gram.y's comment, the > support syntax is now: > > /* >* >*QUERY : > ! *CREATE STATISTICS stats_name [(stat types)]

Re: [HACKERS] what's up with IDENTIFIER_LOOKUP_EXPR?

2017-05-04 Thread Robert Haas
On Thu, May 4, 2017 at 4:21 PM, Tom Lane wrote: > Robert Haas writes: >> plpgsql has an enum called IdentifierLookup which includes a value >> IDENTIFIER_LOOKUP_EXPR which is declared like this: >> IDENTIFIER_LOOKUP_EXPR /* In SQL expression --- special >> case */ >> It regretta

Re: [HACKERS] PG 10 release notes

2017-05-04 Thread Alvaro Herrera
Bruce Momjian wrote: > On Tue, Apr 25, 2017 at 04:03:53PM +1200, David Rowley wrote: > > ..On 25 April 2017 at 13:31, Bruce Momjian wrote: > > > The only unusual thing is that this release has ~180 items while most > > > recent release have had ~220. The pattern I see that there are more > > > l

Re: [HACKERS] PG 10 release notes

2017-05-04 Thread Alvaro Herrera
Claudio Freire wrote: > On Tue, Apr 25, 2017 at 2:45 PM, Bruce Momjian wrote: > > However, given your explanation, I have added the item: > > > >Improve speed of VACUUM's removal of trailing empty > >heap pages (Alvaro Herrera) > > That's enough for me, thanks. Thanks! I amende

Re: [HACKERS] Adding support for Default partition in partitioning

2017-05-04 Thread Sven R. Kunze
Hi Rahila, still thinking about the syntax (sorry): On 04.05.2017 13:44, Rahila Syed wrote: [...] The syntax implemented in this patch is as follows, CREATE TABLE p11 PARTITION OF p1 DEFAULT; Rewriting the following: On Thu, May 4, 2017 at 4:02 PM, amul sul > wrot

Re: [HACKERS] transition table behavior with inheritance appears broken (was: Declarative partitioning - another take)

2017-05-04 Thread Thomas Munro
On Fri, May 5, 2017 at 2:40 AM, Robert Haas wrote: > On Thu, May 4, 2017 at 4:46 AM, Thomas Munro > wrote: >> On Thu, May 4, 2017 at 4:02 AM, Alvaro Herrera >> wrote: >>> Robert Haas wrote: I suspect that most users would find it more useful to capture all of the rows that the stateme

Re: [HACKERS] Adding support for Default partition in partitioning

2017-05-04 Thread Jeevan Ladhe
While reviewing the code I was trying to explore more cases, and I here comes an open question to my mind: should we allow the default partition table to be partitioned further? If we allow it(as in the current case) then observe following case, where I have defined a default partitioned which is

Re: [HACKERS] delta relations in AFTER triggers

2017-05-04 Thread Thomas Munro
On Fri, May 5, 2017 at 12:39 AM, Neha Sharma wrote: > While testing the feature we encountered one more crash,below is the > scenario to reproduce. > > create table t1 ( a int); > create table t2 ( a int); > insert into t1 values (11),(12),(13); > > create or replace function my_trig() returns tri

Re: [HACKERS] what's up with IDENTIFIER_LOOKUP_EXPR?

2017-05-04 Thread Tom Lane
Robert Haas writes: > plpgsql has an enum called IdentifierLookup which includes a value > IDENTIFIER_LOOKUP_EXPR which is declared like this: > IDENTIFIER_LOOKUP_EXPR /* In SQL expression --- special case > */ > It regrettably does not explain what exactly is special about it, a

Re: [HACKERS] Adding support for Default partition in partitioning

2017-05-04 Thread Jeevan Ladhe
Hi Rahila, I have started reviewing your latest patch, and here are my initial comments: 1. In following block, we can just do with def_index, and we do not need found_def flag. We can check if def_index is -1 or not to decide if default partition is present. @@ -166,6 +172,8 @@ RelationBuildPar

[HACKERS] what's up with IDENTIFIER_LOOKUP_EXPR?

2017-05-04 Thread Robert Haas
plpgsql has an enum called IdentifierLookup which includes a value IDENTIFIER_LOOKUP_EXPR which is declared like this: IDENTIFIER_LOOKUP_EXPR /* In SQL expression --- special case */ It regrettably does not explain what exactly is special about it, and AFAICT, neither does any ot

Re: [HACKERS] Fix freeing of dangling IndexScanDesc.xs_hitup in GiST

2017-05-04 Thread Nikita Glukhov
On 04.05.2017 22:16, Tom Lane wrote: Nikita Glukhov writes: In gistrescan() IndexScanDesc.xs_hitup is not reset after MemoryContextReset() of so->queueCxt in which xs_hitup was allocated, then getNextNearest() tries to pfree() dangling xs_hitup, which results in the reuse of this pointer and

Re: [HACKERS] Potential issue with alter system

2017-05-04 Thread Tom Lane
"Joshua D. Drake" writes: > So I did this: > postgres=# alter system set archive_command to 'rsynv -av %p > postgres@52.3.141.224:/data/archive/%f > '; > Note the new line. It properly created in postgresql.auto.conf: > archive_command = 'rsynv -av %p postgres@52.3.141.224:/data/archive/%f > '

Re: [HACKERS] Fix freeing of dangling IndexScanDesc.xs_hitup in GiST

2017-05-04 Thread Tom Lane
Nikita Glukhov writes: > In gistrescan() IndexScanDesc.xs_hitup is not reset after > MemoryContextReset() of > so->queueCxt in which xs_hitup was allocated, then getNextNearest() tries to > pfree() > dangling xs_hitup, which results in the reuse of this pointer and the > subsequent crash. Righ

Re: [HACKERS] json_agg produces nonstandard json

2017-05-04 Thread Tom Lane
Jordan Deitch writes: > However, I don't see consistency between the results of these two > statements: > select jsonb_agg((select 1 where false)); > select sum((select 1 where false)); Well, SUM() is defined to ignore null input values, which is not too surprising as it couldn't do anything ver

Re: [HACKERS] CTE inlining

2017-05-04 Thread Serge Rielau
I haven't been keeping close tabs either, but surely we still have to have the optimization fence in (at least) all these cases: * CTE contains INSERT/UPDATE/DELETE * CTE contains SELECT FOR UPDATE/SHARE (else the set of rows that get locked might change) * CTE contains volatile functions I'm wil

Re: [HACKERS] CTE inlining

2017-05-04 Thread Tom Lane
Tomas Vondra writes: > On 5/4/17 8:03 PM, Joe Conway wrote: >>> I haven't been able to follow this incredibly long thread, so please >>> excuse me if way off base, but are we talking about that a CTE would be >>> silently be rewritten as an inline expression potentially unless it is >>> decorated

Re: [HACKERS] logical replication syntax (was DROP SUBSCRIPTION, query cancellations and slot handling)

2017-05-04 Thread Robert Haas
On Wed, May 3, 2017 at 12:38 AM, Petr Jelinek wrote: > Ok, Let me be clear, I actually happen to agree with your proposal. The > reason I am moaning is that I always seem to find myself doing tons of > mechanical work to rewrite some cosmetic aspect of some patch based on > which committer is payi

Re: [HACKERS] json_agg produces nonstandard json

2017-05-04 Thread Jordan Deitch
Thank you for responding! Good points. However, I don't see consistency between the results of these two statements: select jsonb_agg((select 1 where false)); select sum((select 1 where false)); Therefore another option I would like to suggest is returning the same null value-types for the sum(

Re: [HACKERS] WITH clause in CREATE STATISTICS

2017-05-04 Thread Alvaro Herrera
Here's a patch implementing this idea. From gram.y's comment, the support syntax is now: /* * *QUERY : ! *CREATE STATISTICS stats_name [(stat types)] arguments ! ! *where 'ar

[HACKERS] Fix freeing of dangling IndexScanDesc.xs_hitup in GiST

2017-05-04 Thread Nikita Glukhov
Hello, hackers! The last query in the following script crashes Postgres: create table t (id serial, amount int); insert into t (amount) select random() * 1000 from generate_series(1, 100); create extension btree_gist; create index t_gist_idx on t using gist(id, amount); select p.id, p.amount, s

[HACKERS] Potential issue with alter system

2017-05-04 Thread Joshua D. Drake
Folks, So I did this: postgres=# alter system set archive_command to 'rsynv -av %p postgres@52.3.141.224:/data/archive/%f '; Note the new line. It properly created in postgresql.auto.conf: archive_command = 'rsynv -av %p postgres@52.3.141.224:/data/archive/%f ' (note the new line) I noticed

Re: [HACKERS] CTE inlining

2017-05-04 Thread Tomas Vondra
On 5/4/17 8:03 PM, Joe Conway wrote: On 05/04/2017 10:56 AM, Andrew Dunstan wrote: On 05/04/2017 01:52 PM, Joe Conway wrote: On 05/04/2017 10:33 AM, Alvaro Herrera wrote: I'm not sure what your point is. We know that for some cases the optimization barrier semantics are useful, which is w

Re: [HACKERS] CTE inlining

2017-05-04 Thread Tom Lane
Alvaro Herrera writes: > I'm not sure what your point is. We know that for some cases the > optimization barrier semantics are useful, which is why the proposal is > to add a keyword to install one explicitely: > with materialized r as > ( > select json_populate_rec

Re: [HACKERS] Reducing runtime of stats regression test

2017-05-04 Thread Tom Lane
Robert Haas writes: > On Thu, May 4, 2017 at 10:22 AM, Tom Lane wrote: >> Yes, but that would be getting into the realm of new features, not >> post-feature-freeze test adjustments. It certainly couldn't be >> a candidate for back-patching. > I'm not sure there's some bright line between adding

Re: [HACKERS] json_agg produces nonstandard json

2017-05-04 Thread Tom Lane
Jordan Deitch writes: > A json(b)_agg() will produce the following result when no results are > passed in: > "[null]" > per: > select jsonb_agg((select 1 where false)); Looks fine to me. > I believe, generally speaking, '[]' would be the more appropriate output. Why? What you gave it was one n

Re: [HACKERS] CTE inlining

2017-05-04 Thread Joe Conway
On 05/04/2017 10:56 AM, Andrew Dunstan wrote: > > > On 05/04/2017 01:52 PM, Joe Conway wrote: >> On 05/04/2017 10:33 AM, Alvaro Herrera wrote: >>> I'm not sure what your point is. We know that for some cases the >>> optimization barrier semantics are useful, which is why the proposal is >>> to a

Re: [HACKERS] CTE inlining

2017-05-04 Thread Tomas Vondra
On 5/4/17 7:56 PM, Andrew Dunstan wrote: On 05/04/2017 01:52 PM, Joe Conway wrote: On 05/04/2017 10:33 AM, Alvaro Herrera wrote: I'm not sure what your point is. We know that for some cases the optimization barrier semantics are useful, which is why the proposal is to add a keyword to instal

Re: [HACKERS] CTE inlining

2017-05-04 Thread Andrew Dunstan
On 05/04/2017 01:52 PM, Joe Conway wrote: > On 05/04/2017 10:33 AM, Alvaro Herrera wrote: >> I'm not sure what your point is. We know that for some cases the >> optimization barrier semantics are useful, which is why the proposal is >> to add a keyword to install one explicitely: >> >>

Re: [HACKERS] CTE inlining

2017-05-04 Thread Joe Conway
On 05/04/2017 10:33 AM, Alvaro Herrera wrote: > I'm not sure what your point is. We know that for some cases the > optimization barrier semantics are useful, which is why the proposal is > to add a keyword to install one explicitely: > > with materialized r as > ( >

Re: [HACKERS] WIP Patch: Precalculate stable functions, infrastructure v1

2017-05-04 Thread Marina Polyakova
and here I send infrastructure patch which includes <...> Next 2 patches: Patch 'planning and execution', which includes: - replacement nonvolatile functions and operators by appropriate cached expressions; - planning and execution cached expressions; - regression tests. Patch 'costs', which

Re: [HACKERS] CTE inlining

2017-05-04 Thread Andrew Dunstan
On 05/04/2017 01:33 PM, Alvaro Herrera wrote: > Andrew Dunstan wrote: > >> Hadn't though about LATERAL, good point. Still, there will be other cases. > I'm not sure what your point is. We know that for some cases the > optimization barrier semantics are useful, which is why the proposal is > to

  1   2   >