Re: Internal error XX000 with enable_partition_pruning=on, pg 11 beta1 on Debian

2018-08-07 Thread Amit Langote
On 2018/08/08 8:09, Tom Lane wrote: > Rushabh Lathia writes: >> Consider the below case: > > I initially thought the rule might be messing stuff up, but you can get > the same result without the rule by writing out the transformed query > by hand: > > regression=# explain UPDATE pt_p1 SET a = 3

Re: Early WIP/PoC for inlining CTEs

2018-08-07 Thread Andres Freund
Hi, On 2018-08-08 16:55:22 +1200, Thomas Munro wrote: > On Fri, Jul 27, 2018 at 8:10 PM, David Fetter wrote: > > On Fri, Jul 27, 2018 at 02:55:26PM +1200, Thomas Munro wrote: > >> On Thu, Jul 26, 2018 at 7:14 AM, David Fetter wrote: > >> > Please find attached the next version, which passes

Re: Early WIP/PoC for inlining CTEs

2018-08-07 Thread Thomas Munro
On Fri, Jul 27, 2018 at 8:10 PM, David Fetter wrote: > On Fri, Jul 27, 2018 at 02:55:26PM +1200, Thomas Munro wrote: >> On Thu, Jul 26, 2018 at 7:14 AM, David Fetter wrote: >> > Please find attached the next version, which passes 'make check'. >> >> ... but not 'make check-world'

Why do we expand tuples in execMain.c?

2018-08-07 Thread Andres Freund
Hi, I noticed if (HeapTupleHeaderGetNatts(tuple.t_data) < RelationGetDescr(erm->relation)->natts) { copyTuple = heap_expand_tuple(,

Re: Shared buffer access rule violations?

2018-08-07 Thread Peter Geoghegan
On Tue, Aug 7, 2018 at 6:43 PM, Asim R P wrote: > Please find attached a patch to mark a shared buffer as read-write or > read-only using mprotect(). The idea is to catch violations of shared > buffer access rules. This patch was useful to detect the access > violations reported in this thread.

Re: Shared buffer access rule violations?

2018-08-07 Thread Asim R P
Please find attached a patch to mark a shared buffer as read-write or read-only using mprotect(). The idea is to catch violations of shared buffer access rules. This patch was useful to detect the access violations reported in this thread. The mprotect() calls are enabled by -DMPROTECT_BUFFER

Re: Internal error XX000 with enable_partition_pruning=on, pg 11 beta1 on Debian

2018-08-07 Thread Tom Lane
Rushabh Lathia writes: > Consider the below case: I initially thought the rule might be messing stuff up, but you can get the same result without the rule by writing out the transformed query by hand: regression=# explain UPDATE pt_p1 SET a = 3 from pt WHERE pt.a = 2 and pt.a = pt_p1.a;

Re: Page freezing, FSM, and WAL replay

2018-08-07 Thread Tom Lane
Michael Paquier writes: > On Tue, Aug 07, 2018 at 12:19:13PM -0400, Alvaro Herrera wrote: >> Now that the minors have been tagged, I'm considering pushing this >> shortly to 9.6 - master. > I may be missing something, but the next round of minor releases is not > tagged yet, and only stamped.

Re: Make foo=null a warning by default.

2018-08-07 Thread Bruce Momjian
hOn Mon, Jul 16, 2018 at 11:37:28AM -0400, Tom Lane wrote: > Heikki Linnakangas writes: > > On 16/07/18 18:10, Tom Lane wrote: > >> TBH I'm not really excited about investing any work in this area at all. > >> Considering how seldom we hear any questions about transform_null_equals > >>

Re: Page freezing, FSM, and WAL replay

2018-08-07 Thread Michael Paquier
On Tue, Aug 07, 2018 at 12:19:13PM -0400, Alvaro Herrera wrote: > Now that the minors have been tagged, I'm considering pushing this > shortly to 9.6 - master. I may be missing something, but the next round of minor releases is not tagged yet, and only stamped. So I think that you should hold on

Re: Negotiating the SCRAM channel binding type

2018-08-07 Thread Heikki Linnakangas
On 07/08/18 22:34, Bruce Momjian wrote: On Thu, Jul 12, 2018 at 11:26:30AM +0300, Heikki Linnakangas wrote: On 12/07/18 07:14, Michael Paquier wrote: On Wed, Jul 11, 2018 at 03:01:03PM +0300, Heikki Linnakangas wrote: I started digging into this more closely, and ran into a little problem. If

Re: Negotiating the SCRAM channel binding type

2018-08-07 Thread Bruce Momjian
On Thu, Jul 12, 2018 at 11:26:30AM +0300, Heikki Linnakangas wrote: > On 12/07/18 07:14, Michael Paquier wrote: > >On Wed, Jul 11, 2018 at 03:01:03PM +0300, Heikki Linnakangas wrote: > >>I started digging into this more closely, and ran into a little problem. If > >>channel binding is not used,

Re: Page freezing, FSM, and WAL replay

2018-08-07 Thread Alvaro Herrera
On 2018-Aug-02, Alvaro Herrera wrote: > After considering several possible solutions, I propose to have > heap_xlog_visible compute free space for any page being marked frozen; > Pavan adds to that to have heap_xlog_clean compute free space for all > pages also. This means that if we later

Re: pg_dump: sortDumpableObjectsByTypeName() doesn't always do that

2018-08-07 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Jacob Champion writes: > > Thanks! We have made some progress towards a repro but we're having > > problems putting it into the pg_dump suite. > > Yeah, I find the pg_dump test suite to be pretty much unreadable too. > Don't worry about it ---

Re: pg_dump: sortDumpableObjectsByTypeName() doesn't always do that

2018-08-07 Thread Tom Lane
Jacob Champion writes: > Thanks! We have made some progress towards a repro but we're having > problems putting it into the pg_dump suite. Yeah, I find the pg_dump test suite to be pretty much unreadable too. Don't worry about it --- I don't think we need to memorialize a test case for this. >

Re: pg_dump: sortDumpableObjectsByTypeName() doesn't always do that

2018-08-07 Thread Jacob Champion
On Tue, Aug 7, 2018 at 10:24 AM Tom Lane wrote: > I don't see any reason to insist on a test case before pushing this > fix, so I did that. (As I expected, the fix doesn't change any existing > regression test results.) Thanks! We have made some progress towards a repro but we're having

Re: Typo in doc or wrong EXCLUDE implementation

2018-08-07 Thread Bruce Momjian
This email was sent to docs, but I think it is a hackers issue. The person is asking why exclusion constraints aren't marked as UNIQUE indexes that can be used for referential integrity. I think the reason is that non-equality exclusion constraints, like preventing overlap, but don't uniquely

Re: [PATCH] Include application_name in "connection authorized" log message

2018-08-07 Thread Tom Lane
Don Seiler writes: > On Tue, Aug 7, 2018 at 11:29 AM, Tom Lane wrote: >> Well, if you're going to insist on that part, it's probably not worth >> making the application_name GUC have inconsistent behavior. > OK so just to make sure I understand: > 1. We want to make a generic, central

Re: pg_dump: sortDumpableObjectsByTypeName() doesn't always do that

2018-08-07 Thread Tom Lane
Jacob Champion writes: > On Mon, Aug 6, 2018 at 12:45 PM Tom Lane wrote: >> Ah, gotcha. But whether the behavior is sane or not, it'd be reproducible >> for any specific input dataset on any specific platform (unless you've got >> a quicksort that actually uses randomized pivots; but ours

Re: [PATCH] Include application_name in "connection authorized" log message

2018-08-07 Thread Don Seiler
On Tue, Aug 7, 2018 at 11:29 AM, Tom Lane wrote: > Stephen Frost writes: > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > >> But having said that, I don't exactly see why you couldn't force it > >> with an ultimately-redundant SetConfigOption call to put the value > >> in place before the ereport

Re: Internal error XX000 with enable_partition_pruning=on, pg 11 beta1 on Debian

2018-08-07 Thread Rushabh Lathia
Hi, Consider the below case: CREATE TABLE pt (a INT, b INT, c INT) PARTITION BY RANGE(a); CREATE TABLE pt_p1 PARTITION OF pt FOR VALUES FROM (1) to (6) PARTITION BY RANGE (b); CREATE TABLE pt_p1_p1 PARTITION OF pt_p1 FOR VALUES FROM (11) to (44); CREATE TABLE pt_p1_p2 PARTITION OF pt_p1 FOR

Re: [PATCH] Include application_name in "connection authorized" log message

2018-08-07 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> But having said that, I don't exactly see why you couldn't force it >> with an ultimately-redundant SetConfigOption call to put the value >> in place before the ereport happens. The GUC machinery is surely >> functional before we

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-08-07 Thread Fabien COELHO
Hello Marina, v10-0001-Pgbench-errors-use-the-RandomState-structure-for.patch - a patch for the RandomState structure (this is used to reset a client's random seed during the repeating of transactions after serialization/deadlock failures). About this v10 part 1: Patch applies cleanly,

Re: Standby trying "restore_command" before local WAL

2018-08-07 Thread Stephen Frost
Greetings, * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote: > That's how I read this part of RestoreArchivedFile: > > https://github.com/postgres/postgres/blob/master/src/backend/access/transam/xlogarchive.c#L110 > > The very first thing it does is checking if the local file exists, and if

Re: Standby trying "restore_command" before local WAL

2018-08-07 Thread Stephen Frost
Greetings, * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote: > On 08/06/2018 09:32 PM, Stephen Frost wrote: > >* Tomas Vondra (tomas.von...@2ndquadrant.com) wrote: > >>On 08/06/2018 06:11 PM, Stephen Frost wrote: > >WAL checksums are per WAL record, not across the whole file... And, > >yes,

Percona Live Europe Open Source Database Conference 2018 Call For Papers

2018-08-07 Thread Lorraine Pocklington
*Hi The Percona Live Europe Open Source Database Conference is being held this year in Frankfurt from November 5-7, 2018. In case you are not familiar with the conference, our brief is to offer a platform for the presentation of technical and business papers on any topic that’s relevant to open

Re: [Patch] Create a new session in postmaster by calling setsid()

2018-08-07 Thread Paul Guo
On Thu, Aug 2, 2018 at 10:30 PM, Tom Lane wrote: > Paul Guo writes: > > [ make the postmaster execute setsid() too ] > > I'm a bit skeptical of this proposal. Forcing the postmaster to > dissociate from its controlling terminal is a good thing in some > scenarios, but probably less good in

Re: [PATCH] Include application_name in "connection authorized" log message

2018-08-07 Thread Stephen Frost
Greetings Tom, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > >> Moreover, if you don't check it then the appname recorded > >> by log_connections would not match appearances for the same session > >> later in the log, which puts the

Re: Negotiating the SCRAM channel binding type

2018-08-07 Thread Stephen Frost
Greetings, * Michael Paquier (mich...@paquier.xyz) wrote: > On Tue, Aug 07, 2018 at 02:32:27PM +0530, Robert Haas wrote: > > On Sun, Aug 5, 2018 at 4:30 PM, Heikki Linnakangas wrote: > >> Well, it'd be useless for users, there is no reason to switch off channel > >> binding if both the client

Re: [PATCH] Include application_name in "connection authorized" log message

2018-08-07 Thread Stephen Frost
Greetings Don, * Don Seiler (d...@seiler.us) wrote: > On Tue, Aug 7, 2018 at 8:46 AM, Stephen Frost wrote: > > * Don Seiler (d...@seiler.us) wrote: > > > On Mon, Jul 30, 2018 at 5:20 AM, Peter Eisentraut < > > > peter.eisentr...@2ndquadrant.com> wrote: > > > > > > > On 13/07/2018 20:20, Don

Re: [PATCH] Include application_name in "connection authorized" log message

2018-08-07 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > * Don Seiler (d...@seiler.us) wrote: > >> Is the concern that any user can set their client's application name value > >> to any string they want? Is there a reason we can't call > >> check_application_name() before

Re: [PATCH] Include application_name in "connection authorized" log message

2018-08-07 Thread Don Seiler
On Tue, Aug 7, 2018 at 8:46 AM, Stephen Frost wrote: > Greetings, > > * Don Seiler (d...@seiler.us) wrote: > > On Mon, Jul 30, 2018 at 5:20 AM, Peter Eisentraut < > > peter.eisentr...@2ndquadrant.com> wrote: > > > > > On 13/07/2018 20:20, Don Seiler wrote: > > > > See attached for latest

Re: garbage variable in GNUmakefile.in

2018-08-07 Thread Tom Lane
Daniel Gustafsson writes: > I happened to notice that there is a variable 'garbage' in GNUmakefile.in, > which was defined in 32163099d7c43a0244f463eb4e525c711e6e97a3 ~18 years ago, > but which seems to not be used? Is 'garbage' a special make variable or is a > leftover which can be removed?

Re: [PATCH] Include application_name in "connection authorized" log message

2018-08-07 Thread Tom Lane
Stephen Frost writes: > * Don Seiler (d...@seiler.us) wrote: >> Is the concern that any user can set their client's application name value >> to any string they want? Is there a reason we can't call >> check_application_name() before setting it in the Port struct in >> postmaster.c? > I've not

Re: [HACKERS] logical decoding of two-phase transactions

2018-08-07 Thread Nikhil Sontakke
Hi Arseny, > - Decoding transactions at PREPARE record changes rules of the "we ship > all commits after lsn 'x'" game. Namely, it will break initial > tablesync: what if consistent snapshot was formed *after* PREPARE, but > before COMMIT PREPARED, and the plugin decides to employ 2pc?

Re: [PATCH] Include application_name in "connection authorized" log message

2018-08-07 Thread Stephen Frost
Greetings, * Don Seiler (d...@seiler.us) wrote: > On Mon, Jul 30, 2018 at 5:20 AM, Peter Eisentraut < > peter.eisentr...@2ndquadrant.com> wrote: > > > On 13/07/2018 20:20, Don Seiler wrote: > > > See attached for latest revision. > > > > This doesn't compile with SSL enabled because there is a

Re: Reopen logfile on SIGHUP

2018-08-07 Thread Alexander Kuzmenkov
I think the latest v4 patch addresses the concerns raised upthread. I'm marking the commitfest entry as RFC. -- Alexander Kuzmenkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [PATCH] Include application_name in "connection authorized" log message

2018-08-07 Thread Don Seiler
On Mon, Jul 30, 2018 at 5:20 AM, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 13/07/2018 20:20, Don Seiler wrote: > > See attached for latest revision. > > This doesn't compile with SSL enabled because there is a comma missing. > Hmm I'll check this out tonight. Sorry I

Re: ATTACH/DETACH PARTITION CONCURRENTLY

2018-08-07 Thread Andres Freund
On 2018-08-08 01:23:51 +1200, David Rowley wrote: > On 8 August 2018 at 00:47, Andres Freund wrote: > > On 2018-08-08 00:40:12 +1200, David Rowley wrote: > >> 1. Obtain a ShareUpdateExclusiveLock on the partitioned table rather > >> than an AccessExclusiveLock. > >> 2. Do all the normal partition

Re: ATTACH/DETACH PARTITION CONCURRENTLY

2018-08-07 Thread David Rowley
On 8 August 2018 at 00:47, Andres Freund wrote: > On 2018-08-08 00:40:12 +1200, David Rowley wrote: >> 1. Obtain a ShareUpdateExclusiveLock on the partitioned table rather >> than an AccessExclusiveLock. >> 2. Do all the normal partition attach partition validation. >> 3. Insert pg_partition

garbage variable in GNUmakefile.in

2018-08-07 Thread Daniel Gustafsson
I happened to notice that there is a variable 'garbage' in GNUmakefile.in, which was defined in 32163099d7c43a0244f463eb4e525c711e6e97a3 ~18 years ago, but which seems to not be used? Is 'garbage' a special make variable or is a leftover which can be removed? cheers ./daniel garbage.diff

Re: ATTACH/DETACH PARTITION CONCURRENTLY

2018-08-07 Thread Simon Riggs
On 7 August 2018 at 13:47, Andres Freund wrote: > Hi, > > On 2018-08-08 00:40:12 +1200, David Rowley wrote: >> 1. Obtain a ShareUpdateExclusiveLock on the partitioned table rather >> than an AccessExclusiveLock. >> 2. Do all the normal partition attach partition validation. >> 3. Insert

Re: ATTACH/DETACH PARTITION CONCURRENTLY

2018-08-07 Thread Andres Freund
Hi, On 2018-08-08 00:40:12 +1200, David Rowley wrote: > 1. Obtain a ShareUpdateExclusiveLock on the partitioned table rather > than an AccessExclusiveLock. > 2. Do all the normal partition attach partition validation. > 3. Insert pg_partition record with partvalid = true. > 4. Invalidate relcache

Re: ATTACH/DETACH PARTITION CONCURRENTLY

2018-08-07 Thread David Rowley
On 3 August 2018 at 01:25, David Rowley wrote: > 1. Do all the normal partition attach partition validation. > 2. Insert a record into pg_partition with partisvalid=false > 3. Obtain a session-level ShareUpdateExclusiveLock on the partitioned table. > 4. Obtain a session-level AccessExclusiveLock

Re: [HACKERS] Cached plans and statement generalization

2018-08-07 Thread Konstantin Knizhnik
On 07.08.2018 13:02, Yamaji, Ryo wrote: I want to confirm one point. If I will have reviewed the autoprepare patch, then are you ready to register the patch at commit fest in the near future? I fear that autoprepare patch do not registered at commit fest in the future (for example, you are

Re: Constraint documentation

2018-08-07 Thread Lætitia Avrot
Hi Peter, I understand what you're pointing at and I agree that it could be a good thing to be able to dump/restore a table without problem. My point was that check constraints weren't supposed to be used that way theorically (or maybe i'm mistaken ?) so I thought maybe we should just inform the

Re: Standby trying "restore_command" before local WAL

2018-08-07 Thread Tomas Vondra
On 08/06/2018 09:32 PM, Stephen Frost wrote: Greetings, * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote: On 08/06/2018 06:11 PM, Stephen Frost wrote: * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote: On 08/06/2018 05:19 PM, Stephen Frost wrote: * David Steele (da...@pgmasters.net)

Re: partition tree inspection functions

2018-08-07 Thread Jesper Pedersen
Hi, On 08/07/2018 03:32 AM, Amit Langote wrote: Do we need a pg_partition_level that expects the individual partition OID to be passed to it or can we do with the information we get from the revised pg_partition_children? In earlier revisions, pg_partition_children returned only the partition

Re: Negotiating the SCRAM channel binding type

2018-08-07 Thread Michael Paquier
On Tue, Aug 07, 2018 at 02:32:27PM +0530, Robert Haas wrote: > On Sun, Aug 5, 2018 at 4:30 PM, Heikki Linnakangas wrote: >> Well, it'd be useless for users, there is no reason to switch off channel >> binding if both the client and server support it. It might not add any >> security you care

RE: [HACKERS] Cached plans and statement generalization

2018-08-07 Thread Yamaji, Ryo
> -Original Message- > From: Konstantin Knizhnik [mailto:k.knizh...@postgrespro.ru] > Sent: Friday, August 3, 2018 7:02 AM > To: Yamaji, Ryo/山地 亮 > Cc: PostgreSQL mailing lists > Subject: Re: [HACKERS] Cached plans and statement generalization > > Thank you. > Unfortunately

Re: [HACKERS] logical decoding of two-phase transactions

2018-08-07 Thread Arseny Sher
Andres Freund writes: >> - On decoding of aborted xacts. The idea to throw an error once we >> detect the abort is appealing, however I think you will have problems >> with subxacts in the current implementation. What if subxact issues >> DDL and then aborted, but main transaction

Re: Negotiating the SCRAM channel binding type

2018-08-07 Thread Robert Haas
On Sun, Aug 5, 2018 at 4:30 PM, Heikki Linnakangas wrote: > Well, it'd be useless for users, there is no reason to switch off channel > binding if both the client and server support it. It might not add any > security you care about, but it won't do any harm either. The > non-channel-binding

Re: Fix hints on CREATE PROCEDURE errors

2018-08-07 Thread Amit Langote
Hi. On 2018/08/07 3:32, Jeremy Evans wrote: > When testing out CREATE PROCEDURE with 11 beta 2, I noticed, the hints > in the errors reference DROP FUNCTION, which doesn't work for > procedures. Good catch. > DROP ROUTINE works for both functions and procedures, so > this patch should work for

Re: partition tree inspection functions

2018-08-07 Thread Amit Langote
Hi, On 2018/08/03 21:35, Jesper Pedersen wrote: > Hi Amit, > > On 08/03/2018 04:28 AM, Amit Langote wrote: >> That's a good idea, thanks. >> >> Actually, by the time I sent the last version of the patch or maybe few >> versions before that, I too had started thinking if we shouldn't just have >>

Re: [WIP] [B-Tree] Retail IndexTuple deletion

2018-08-07 Thread Andrey Lepikhov
Hi, I wrote a background worker (hcleaner) to demonstrate application of Retail IndexTuple deletion (see patch at attachment). Like Autovacuum it utilizes concept of one launcher and many workers. But one worker correspond to one database. Short description: Backend collects dirty block

Re: Optimizer misses big in 10.4 with BRIN index

2018-08-07 Thread Arcadiy Ivanov
On 07/26/2018 07:27 AM, Tomas Vondra wrote: Arcadiy, can you provide plans with parallel query disabled? Or even better, produce a test case that reproduces this (using synthetic data, anonymized data or something like that, if needed). So I basically spent most of the time trying to create a

Re: BUG #15212: Default values in partition tables don't work as expected and allow NOT NULL violation

2018-08-07 Thread Amit Langote
Thanks Ashutosh, and sorry that I somehow missed replying to this. On 2018/07/13 22:50, Ashutosh Bapat wrote: > On Thu, Jul 12, 2018 at 2:29 PM, Amit Langote wrote: >> I have modified the comments around this code in the updated patch. > > +/* > + * Each member in 'saved_schema'