Re: [HACKERS] Logical replication existing data copy

2017-02-25 Thread Erik Rijkers
On 2017-02-25 00:40, Petr Jelinek wrote: 0001-Use-asynchronous-connect-API-in-libpqwalreceiver.patch 0002-Fix-after-trigger-execution-in-logical-replication.patch 0003-Add-RENAME-support-for-PUBLICATIONs-and-SUBSCRIPTION.patch snapbuild-v3-0001-Reserve-global-xmin-for-create-slot-snasphot-export

Re: [HACKERS] Proposal : Parallel Merge Join

2017-02-25 Thread Dilip Kumar
On Sat, Feb 25, 2017 at 11:29 AM, Amit Kapila wrote: > It seems you have forgotten to change in the below check: > > + if (innerpath != NULL && > + innerpath->parallel_safe && > + (cheapest_startup_inner == NULL || > + cheapest_startup_inner->parallel_safe == false || > + compare_path_costs(innerp

Re: [HACKERS] Allow pg_dumpall to work without pg_authid

2017-02-25 Thread Simon Riggs
On 22 February 2017 at 07:33, Robins Tharakan wrote: > Stephen, > > On 20 February 2017 at 08:50, Stephen Frost wrote: >> >> The other changes to use pg_roles instead of pg_authid when rolpassword >> isn't being used look like they should just be changed to use pg_roles >> instead of using one or

Re: [HACKERS] Enabling parallelism for queries coming from SQL or other PL functions

2017-02-25 Thread Amit Kapila
On Fri, Feb 24, 2017 at 11:30 AM, Dilip Kumar wrote: > On Fri, Feb 24, 2017 at 10:06 AM, Amit Kapila wrote: >> We have a below check in standard_planner() (!IsParallelWorker()) >> which should prohibit generating parallel plan inside worker, if that >> is what you are seeing, then we might need a

Re: [HACKERS] Proposal : Parallel Merge Join

2017-02-25 Thread Amit Kapila
On Sat, Feb 25, 2017 at 3:22 PM, Dilip Kumar wrote: > On Sat, Feb 25, 2017 at 11:29 AM, Amit Kapila wrote: >> It seems you have forgotten to change in the below check: >> >> + if (innerpath != NULL && >> + innerpath->parallel_safe && >> + (cheapest_startup_inner == NULL || >> + cheapest_startup_i

Re: [HACKERS] logical replication access control patches

2017-02-25 Thread Peter Eisentraut
On 2/18/17 18:06, Stephen Frost wrote: > I'm not convinced that it really makes sense to have PUBLICATION of a > table be independent from the rights an owner of a table has. We don't > allow other ALTER commands on objects based on GRANT'able rights, in > general, so I'm not really sure that it m

Re: [HACKERS] Automatic cleanup of oldest WAL segments with pg_receivexlog

2017-02-25 Thread Magnus Hagander
On Fri, Feb 24, 2017 at 3:56 AM, Jim Nasby wrote: > On 2/23/17 8:47 PM, Michael Paquier wrote: > >> Anything else than measured in bytes either requires a lookup at the >> file timestamp, which is not reliable with noatime or a lookup at WAL >> itself to decide when is the commit timestamp that m

Re: [HACKERS] Automatic cleanup of oldest WAL segments with pg_receivexlog

2017-02-25 Thread Magnus Hagander
On Fri, Feb 24, 2017 at 3:52 AM, Michael Paquier wrote: > On Fri, Feb 24, 2017 at 1:10 AM, Magnus Hagander > wrote: > > I'm not sure this logic belongs in pg_receivexlog. If we put the decision > > making there, then we lock ourselves into one "type of policy". > > > > Wouldn't this one, along w

Re: [HACKERS] Proposal for changes to recovery.conf API

2017-02-25 Thread Michael Paquier
On Fri, Feb 24, 2017 at 7:39 PM, Simon Riggs wrote: > New patch version implementing everything you requested, incl docs and > tap tests. The TAP changes look good to me. I thought that more diffs would have been necessary. > The patch as offered here is what I've been asked to do by everybody >

Re: [HACKERS] Automatic cleanup of oldest WAL segments with pg_receivexlog

2017-02-25 Thread Michael Paquier
On Sat, Feb 25, 2017 at 10:32 PM, Magnus Hagander wrote: > Oh, I definitely think such a command should be able to take a placeholder > like %f telling which segment it has just processed. In fact, I'd consider > it one of the most important features of it :) I cannot think about any other meanin

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-02-25 Thread Bruce Momjian
On Sat, Feb 25, 2017 at 10:50:57AM +0530, Pavan Deolasee wrote: > > On Fri, Feb 24, 2017 at 9:47 PM, Robert Haas wrote: > And there are no bugs, right?  :-) > > Yeah yeah absolutely nothing. Just like any other feature committed to > Postgres > so far ;-) > > I need to polish the chain con

Re: [HACKERS] Automatic cleanup of oldest WAL segments with pg_receivexlog

2017-02-25 Thread Magnus Hagander
On Feb 25, 2017 15:00, "Michael Paquier" wrote: On Sat, Feb 25, 2017 at 10:32 PM, Magnus Hagander wrote: > Oh, I definitely think such a command should be able to take a placeholder > like %f telling which segment it has just processed. In fact, I'd consider > it one of the most important featur

Re: [HACKERS] Enabling parallelism for queries coming from SQL or other PL functions

2017-02-25 Thread Dilip Kumar
On Sat, Feb 25, 2017 at 5:12 PM, Amit Kapila wrote: > Sure, but that should only happen if the function is *not* declared as > parallel safe (aka in parallel safe functions, we should not generate > parallel plans). So basically we want to put a restriction that parallel-safe function can not use

Re: [HACKERS] btree_gin and btree_gist for enums

2017-02-25 Thread Andrew Dunstan
On 02/24/2017 05:34 PM, Andrew Dunstan wrote: > > On 02/24/2017 02:55 PM, Andrew Dunstan wrote: >> On 02/24/2017 11:02 AM, Tom Lane wrote: I don't know what to call it either. In my test I used CallerContextFunctionCall2 - not sure if that's quite right, but should be close. >>> Ca

Re: [HACKERS] btree_gin and btree_gist for enums

2017-02-25 Thread Tom Lane
Andrew Dunstan writes: > On 02/24/2017 05:34 PM, Andrew Dunstan wrote: >> It's occurred to me that we could reduce the code clutter in fmgr.c a >> bit by turning the DirectFunctionCall{n]Coll functions into macros >> calling these functions and passing NULL as the flinfo param. > here's a patch a

Re: [HACKERS] btree_gin and btree_gist for enums

2017-02-25 Thread Emre Hasegeli
> The reason this is kind of scary is that it's just blithely assuming > that the function won't look at the *other* fields of the FmgrInfo. > If it did, it would likely get very confused, since those fields > would be describing the GIN support function, not the function we're > calling. I am sor

Re: [HACKERS] btree_gin and btree_gist for enums

2017-02-25 Thread Andrew Dunstan
On 02/25/2017 12:04 PM, Tom Lane wrote: > I think it'd be better to leave DirectFunctionCallN alone and just invent > a small number of CallerFInfoFunctionCallN support functions (maybe N=1 > and N=2 would be enough, at least for now). > > See attached. cheers andrew -

[HACKERS] Reduce lock levels for reloptions

2017-02-25 Thread Simon Riggs
Patch to reduce lock levels of various relation-level options, following on from earlier work by Fabrizio, with additional work and documentation by me. Submitted to Sept CF, requested to start new thread to discuss https://www.postgresql.org/message-id/CA%2BTgmoYe5eDhjRodo3uOtVFGiDWwO2zGUp_mDHeSx

Re: [HACKERS] [PATCH] SortSupport for macaddr type

2017-02-25 Thread Julien Rouhaud
On Sun, Feb 05, 2017 at 01:56:41PM -0800, Brandur Leach wrote: Hello Brandur, thanks for the updated patch! > > > * you used macaddr_cmp_internal() function name, for uuid > > the same function is named uuid_internal_cmp(). Using > > the same naming pattern is probably better. > > I was a

Re: [HACKERS] UPDATE of partition key

2017-02-25 Thread Greg Stark
On 24 February 2017 at 14:57, David G. Johnston wrote: > I dislike an error. I'd say that making partition "just work" here is > material for another patch. In this one an update of the partition key can > be documented as shorthand for delete-returning-insert with all the > limitations that go

[HACKERS] I propose killing PL/Tcl's "modules" infrastructure

2017-02-25 Thread Tom Lane
Over in https://www.postgresql.org/message-id/alpine.DEB.2.11.1702251701030.3920@Sandal.Woodpecker it's pointed out that pltcl_loadmod was never updated for the switch to standard_conforming_strings (and the patch proposed there doesn't begin to cover all the places that would need fixed for that).

[HACKERS] PL/Python: Add cursor and execute methods to plan object

2017-02-25 Thread Peter Eisentraut
Something that has been bothering me in PL/Python for a long time is the non-object-oriented way in which plans are prepared and executed: plan = plpy.prepare(...) res = plpy.execute(plan, ...) where plpy.execute() takes either a plan or a query string. I think a better style would be

Re: [HACKERS] btree_gin and btree_gist for enums

2017-02-25 Thread Tom Lane
Andrew Dunstan writes: > On 02/25/2017 12:04 PM, Tom Lane wrote: >> I think it'd be better to leave DirectFunctionCallN alone and just invent >> a small number of CallerFInfoFunctionCallN support functions (maybe N=1 >> and N=2 would be enough, at least for now). > See attached. Yeah, I like thi

Re: [HACKERS] I propose killing PL/Tcl's "modules" infrastructure

2017-02-25 Thread Andrew Dunstan
On 02/25/2017 01:14 PM, Tom Lane wrote: > Over in > https://www.postgresql.org/message-id/alpine.DEB.2.11.1702251701030.3920@Sandal.Woodpecker > it's pointed out that pltcl_loadmod was never updated for the switch > to standard_conforming_strings (and the patch proposed there doesn't > begin to c

Re: [HACKERS] I propose killing PL/Tcl's "modules" infrastructure

2017-02-25 Thread Joshua D. Drake
On 02/25/2017 10:14 AM, Tom Lane wrote: Now, we could try to fix this bug, and add the regression test coverage that the code clearly lacks, and upgrade the documentation about it from its currently very sad state. But I think the right answer is just to remove the feature altogether. It's evi

Re: [HACKERS] btree_gin and btree_gist for enums

2017-02-25 Thread Andrew Dunstan
On 02/25/2017 01:34 PM, Tom Lane wrote: > Andrew Dunstan writes: >> On 02/25/2017 12:04 PM, Tom Lane wrote: >>> I think it'd be better to leave DirectFunctionCallN alone and just invent >>> a small number of CallerFInfoFunctionCallN support functions (maybe N=1 >>> and N=2 would be enough, at le

Re: [HACKERS] Should `pg_upgrade --check` check relation filenodes are present?

2017-02-25 Thread Bruce Momjian
On Wed, Feb 1, 2017 at 10:57:01AM +1300, Craig de Stigter wrote: > Hi list > > We attempted to pg_upgrade a database on a replication slave, and got the > error: > > > error while creating link for relation "." ("/var/ > lib/postgresql-ext/PG_9.2_201204301/19171/141610397" to "/

Re: [HACKERS] I propose killing PL/Tcl's "modules" infrastructure

2017-02-25 Thread Tom Lane
Andrew Dunstan writes: > On 02/25/2017 01:14 PM, Tom Lane wrote: >> Now, we could try to fix this bug, and add the regression test coverage >> that the code clearly lacks, and upgrade the documentation about it from >> its currently very sad state. But I think the right answer is just to >> remov

Re: [HACKERS] I propose killing PL/Tcl's "modules" infrastructure

2017-02-25 Thread Tom Lane
"Joshua D. Drake" writes: > I don't see a reason to keep pl/tcl in core at all so ripping out the > functionality seems the least disruptive and perhaps even a deprecation > of the PL (at least from a core perspective) in v10. I'm not in any hurry to remove or deprecate PL/Tcl as a whole. It's

Re: [HACKERS] I propose killing PL/Tcl's "modules" infrastructure

2017-02-25 Thread Andrew Dunstan
On 02/25/2017 01:44 PM, Tom Lane wrote: > Andrew Dunstan writes: >> On 02/25/2017 01:14 PM, Tom Lane wrote: >>> Now, we could try to fix this bug, and add the regression test coverage >>> that the code clearly lacks, and upgrade the documentation about it from >>> its currently very sad state.

Re: [HACKERS] UPDATE of partition key

2017-02-25 Thread David G. Johnston
On Sat, Feb 25, 2017 at 11:11 AM, Greg Stark wrote: > On 24 February 2017 at 14:57, David G. Johnston > wrote: > > I dislike an error. I'd say that making partition "just work" here is > > material for another patch. In this one an update of the partition key > can > > be documented as shortha

Re: [HACKERS] I propose killing PL/Tcl's "modules" infrastructure

2017-02-25 Thread Tom Lane
Andrew Dunstan writes: > On 02/25/2017 01:44 PM, Tom Lane wrote: >> Yeah, the only part that's even a bit hard to replicate in userland is >> initializing the autoloading mechanism in each session. It would be >> cleaner to provide a feature similar to what you describe that could >> be used for

Re: [HACKERS] Proposal for changes to recovery.conf API

2017-02-25 Thread Simon Riggs
On 25 February 2017 at 13:58, Michael Paquier wrote: > - trigger_file is removed. > FWIW, my only complain is about the removal of trigger_file, this is > useful to detect a trigger file on a different partition that PGDATA! > Keeping it costs also nothing.. Sorry, that is just an error of imple

Re: [HACKERS] I propose killing PL/Tcl's "modules" infrastructure

2017-02-25 Thread Andrew Dunstan
On 02/25/2017 02:21 PM, Tom Lane wrote: > Andrew Dunstan writes: >> On 02/25/2017 01:44 PM, Tom Lane wrote: >>> Yeah, the only part that's even a bit hard to replicate in userland is >>> initializing the autoloading mechanism in each session. It would be >>> cleaner to provide a feature similar

Re: [HACKERS] dropping partitioned tables without CASCADE

2017-02-25 Thread Simon Riggs
On 23 February 2017 at 16:33, Simon Riggs wrote: > I'll be happy to review Patch looks OK so far, but fails on a partition that has partitions, probably because of the way we test relkind in the call to StoreCatalogInheritance1(). Please add a test for that so we can check automatically. Than

[HACKERS] Should logtape.c blocks be of type long?

2017-02-25 Thread Peter Geoghegan
logtape.c stores block numbers on disk. These block numbers are represented in memory as being of type long. This is why BufFile clients that use the block-based interface (currently limited to logtape.c and gistbuildbuffers.c) must live with a specific limitation, as noted within buffile.c: /* *

Re: [HACKERS] Documentation improvements for partitioning

2017-02-25 Thread Bruce Momjian
On Wed, Feb 22, 2017 at 07:44:41AM +0530, Robert Haas wrote: > On Tue, Feb 21, 2017 at 2:03 AM, Bruce Momjian wrote: > > I have to admit my reaction was similar to Simon's, meaning that the > > lack of docs is a problem, and that the limitations are kind of a > > surprise, and I wonder what other

Re: [HACKERS] Automatic cleanup of oldest WAL segments with pg_receivexlog

2017-02-25 Thread Michael Paquier
On Sun, Feb 26, 2017 at 12:41 AM, Magnus Hagander wrote: > On Feb 25, 2017 15:00, "Michael Paquier" wrote: > > On Sat, Feb 25, 2017 at 10:32 PM, Magnus Hagander > wrote: >> Oh, I definitely think such a command should be able to take a placeholder >> like %f telling which segment it has just pro

Re: [HACKERS] bytea_output output of base64

2017-02-25 Thread Peter Eisentraut
On 2/23/17 17:55, Bruce Momjian wrote: > On Thu, Feb 23, 2017 at 04:08:58PM -0500, Tom Lane wrote: >> Bruce Momjian writes: >>> Is there a reason we don't support base64 as a bytea_output output >>> option, except that no one has implemented it? >> >> How about "we already have one too many bytea

Re: [HACKERS] bytea_output output of base64

2017-02-25 Thread Bruce Momjian
On Sat, Feb 25, 2017 at 06:49:01PM -0500, Peter Eisentraut wrote: > On 2/23/17 17:55, Bruce Momjian wrote: > > On Thu, Feb 23, 2017 at 04:08:58PM -0500, Tom Lane wrote: > >> Bruce Momjian writes: > >>> Is there a reason we don't support base64 as a bytea_output output > >>> option, except that no

Re: [HACKERS] snapbuild woes

2017-02-25 Thread Petr Jelinek
On 24/02/17 22:56, Petr Jelinek wrote: > On 22/02/17 03:05, Petr Jelinek wrote: >> On 13/12/16 00:38, Petr Jelinek wrote: >>> On 12/12/16 23:33, Andres Freund wrote: On 2016-12-12 23:27:30 +0100, Petr Jelinek wrote: > On 12/12/16 22:42, Andres Freund wrote: >> Hi, >> >> On 2016

Re: [HACKERS] Logical replication existing data copy

2017-02-25 Thread Petr Jelinek
On 25/02/17 09:40, Erik Rijkers wrote: > On 2017-02-25 00:40, Petr Jelinek wrote: > >> 0001-Use-asynchronous-connect-API-in-libpqwalreceiver.patch >> 0002-Fix-after-trigger-execution-in-logical-replication.patch >> 0003-Add-RENAME-support-for-PUBLICATIONs-and-SUBSCRIPTION.patch >> snapbuild-v3-000

Re: [HACKERS] Enabling parallelism for queries coming from SQL or other PL functions

2017-02-25 Thread Amit Kapila
On Sat, Feb 25, 2017 at 9:47 PM, Dilip Kumar wrote: > On Sat, Feb 25, 2017 at 5:12 PM, Amit Kapila wrote: >> Sure, but that should only happen if the function is *not* declared as >> parallel safe (aka in parallel safe functions, we should not generate >> parallel plans). > > So basically we want

Re: [HACKERS] \if, \elseif, \else, \endif (was Re: PSQL commands: \quit_if, \quit_unless)

2017-02-25 Thread Corey Huinker
> > Typo "unterminted". > Fixed. > The \if within the \if false branch is not tallied properly? Am I missing > something? > Nope, you found a bug. FIxed. Test-case added. > I changed the paragraph to >> > >Lines within false branches are parsed normally, however, any >> completed >>

Re: [HACKERS] WIP: Covering + unique indexes.

2017-02-25 Thread Amit Kapila
On Thu, Feb 16, 2017 at 6:43 PM, Anastasia Lubennikova wrote: > 14.02.2017 15:46, Amit Kapila: > > >> 4. >> + IDENTITY_P IF_P ILIKE IMMEDIATE IMMUTABLE IMPLICIT_P IMPORT_P IN_P >> INCLUDE >>INCLUDING INCREMENT INDEX INDEXES INHERIT INHERITS INITIALLY INLINE_P >>INNER_P INOUT INPUT_P INSENS

Re: [HACKERS] Documentation improvements for partitioning

2017-02-25 Thread Robert Haas
On Sun, Feb 26, 2017 at 4:31 AM, Bruce Momjian wrote: > I think you are right. I was only guessing on a possible cause of > Simon's reaction since I had the same reaction. When traveling, it is > hard to get excited about reading a 100+ post thread that has reached a > conclusion. I found Simon

Re: [HACKERS] Proposal : Parallel Merge Join

2017-02-25 Thread Robert Haas
On Fri, Feb 24, 2017 at 3:54 PM, Amit Kapila wrote: > I agree in some cases it could be better, but I think benefits are not > completely clear, so probably we can leave it as of now and if later > any one comes with a clear use case or can see the benefits of such > path then we can include it.

Re: [HACKERS] Making clausesel.c Smarter

2017-02-25 Thread Robert Haas
On Fri, Feb 24, 2017 at 3:30 PM, David Rowley wrote: > It would be good to improve the situation here in the back branches > too, but I'm thinking that the attached might be a little invasive for > that? My experience has been that customers - at least EnterpriseDB customers - do not appreciate p

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-25 Thread Robert Haas
On Sat, Feb 25, 2017 at 3:40 AM, Peter Geoghegan wrote: > On Fri, Feb 24, 2017 at 9:26 AM, Robert Haas wrote: >> I think this thread is pretty short on evidence that would let us make >> a smart decision about what to do here. I see three possibilities. >> The first is that this patch is a good

Re: [HACKERS] UPDATE of partition key

2017-02-25 Thread Robert Haas
On Sat, Feb 25, 2017 at 11:41 PM, Greg Stark wrote: > What I'm concerned about is that silently giving "wrong" answers in > regular queries -- not even ones doing the partition key updates -- is > something the user can't really manage. They have no way to rewrite > the query to avoid the problem

Re: [HACKERS] case_preservation_and_insensitivity = on

2017-02-25 Thread Robert Haas
On Fri, Feb 24, 2017 at 11:04 PM, Joel Jacobson wrote: > On Thu, Feb 23, 2017 at 8:04 AM, Robert Haas wrote: >> It doesn't sound like a good solution to me, because there can be SQL >> code inside stored procedures that clients never see. > > In our code base, we use CamelCase in all PL/pgSQL fun

Re: [HACKERS] \if, \elseif, \else, \endif (was Re: PSQL commands: \quit_if, \quit_unless)

2017-02-25 Thread Fabien COELHO
Hello Corey, About v18: Patch applies, make check ok, psql tap tests ok. ISTM that contrary to the documentation "\elif something" is not evaluated in all cases, and the resulting code is harder to understand with a nested switch and condition structure: switch default read if