Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2018-06-21 Thread Masahiko Sawada
On Thu, Jun 21, 2018 at 2:53 PM, Nico Williams wrote: > On Thu, Jun 21, 2018 at 10:05:41AM +0900, Masahiko Sawada wrote: >> On Thu, Jun 21, 2018 at 6:57 AM, Nico Williams wrote: >> > On Wed, Jun 20, 2018 at 05:16:46PM -0400, Bruce Momjian wrote: >> >> On Mon, Jun 18, 2018 at 12:29:57PM -0500,

Re: Push down Aggregates below joins

2018-06-21 Thread Heikki Linnakangas
On 21/06/18 09:11, Antonin Houska wrote: Tomas Vondra wrote: On 06/20/2018 10:12 PM, Heikki Linnakangas wrote: Currently, the planner always first decides the scan/join order, and adds Group/Agg nodes on top of the joins. Sometimes it would be legal, and beneficial, to perform the

"wal receiver" process hang in syslog() while exiting after receiving SIGTERM while the postgres has been promoted.

2018-06-21 Thread Chen, Yan-Jack (NSB - CN/Hangzhou)
Hi Hackers, We encounter one problem happened while we try to promote standby postgres(version 9.6.9) server to active. From the trace(we triggered the process abort). We can see the process was hang in syslog() while handling SIGTERM. According to below article. Looks it is risky to write

Re: Fix slot's xmin advancement and subxact's lost snapshots in decoding.

2018-06-21 Thread Arseny Sher
Hello, Thank you for your time. Alvaro Herrera writes: > On 2018-Jun-11, Antonin Houska wrote: > >> >> One comment about the coding: since you introduce a new transaction list and >> it's sorted by LSN, I think you should make the function AssertTXNLsnOrder() >> more generic and use it to

Re: libpq compression

2018-06-21 Thread Konstantin Knizhnik
On 20.06.2018 23:34, Robbie Harwood wrote: Konstantin Knizhnik writes: My idea was the following: client want to use compression. But server may reject this attempt (for any reasons: it doesn't support it, has no proper compression library, do not want to spend CPU for decompression,...)

Re: bug with expression index on partition

2018-06-21 Thread Amit Langote
On 2018/06/21 15:35, Amit Langote wrote: > So, CompareIndexInfo and generateClonedIndexStmt are both doing the right > thing, but DefineIndex is not. Attached is a patch to fix DefineIndex so > that it converts indexParams before recursing to create the index on a > partition. I noticed that

Buildfarm failure in rolenames.sql

2018-06-21 Thread Michael Paquier
Hi all, Some of your may have noticed, but the buildfarm has thrown up the following failure today: https://buildfarm.postgresql.org/cgi-bin/show_history.pl?nm=gull=HEAD The failure is the following one: -- ALTER TABLE OWNER TO \c - ! \connect: FATAL: could not read block 3 in file

partition table and stddev() /variance() behaviour

2018-06-21 Thread Rajkumar Raghuwanshi
Hi, I am getting different output for stddev/variance functions with partition tables. CREATE TABLE part (c1 INT,c2 INT) PARTITION BY RANGE (c1); CREATE TABLE part_p1 PARTITION OF part FOR VALUES FROM (1) TO (3); CREATE TABLE part_p2 PARTITION OF part FOR VALUES FROM (3) TO (5); INSERT INTO

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2018-06-21 Thread Bruce Momjian
On Wed, Jun 20, 2018 at 04:57:18PM -0500, Nico Williams wrote: > On Wed, Jun 20, 2018 at 05:16:46PM -0400, Bruce Momjian wrote: > > On Mon, Jun 18, 2018 at 12:29:57PM -0500, Nico Williams wrote: > > > Note that unless the pg_catalog is protected against manipulation by > > > remote storage, then

Re: partition table and stddev() /variance() behaviour

2018-06-21 Thread David Rowley
On 22 June 2018 at 00:18, Rajkumar Raghuwanshi wrote: > CREATE TABLE part (c1 INT,c2 INT) PARTITION BY RANGE (c1); > CREATE TABLE part_p1 PARTITION OF part FOR VALUES FROM (1) TO (3); > CREATE TABLE part_p2 PARTITION OF part FOR VALUES FROM (3) TO (5); > > INSERT INTO part VALUES

Re: server crashed with TRAP: FailedAssertion("!(!parallel_aware || pathnode->path.parallel_safe)"

2018-06-21 Thread Amit Kapila
On Thu, Jun 21, 2018 at 11:51 AM, Amit Khandekar wrote: > On 20 June 2018 at 14:28, Amit Khandekar wrote: >>> >>> Did you get a chance to look at it? >> >> Not yet, but I have planned to do this by tomorrow. > > > After list_concat, the subpaths no longer has only non-partial paths, > which

Re: [PATCH] Include application_name in "connection authorized" log message

2018-06-21 Thread Don Seiler
On Wed, Jun 20, 2018 at 2:45 PM, Stephen Frost wrote: > > diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h > index 7698cd1f88..088ef346a8 100644 > --- a/src/include/libpq/libpq-be.h > +++ b/src/include/libpq/libpq-be.h > @@ -135,6 +135,7 @@ typedef struct Port >

Re: Sample values for pg_stat_statements

2018-06-21 Thread Daniel Gustafsson
> On 17 Apr 2018, at 17:58, Vik Fearing wrote: > Thanks! Attached is a patch addressing your concerns. I happened to notice that this patch was moved from Returned with Feedback to Needs Review after the CF closed, which means it’s now sitting open in a closed CF. The intended flow after RWF

Re: Fast default stuff versus pg_upgrade

2018-06-21 Thread Andrew Dunstan
On 06/20/2018 09:04 PM, Andres Freund wrote: Probably couldn't hurt to run the changed files through pgindent and fix the damage... Looks reasonable to me, but I've not tested it. Thanks Patch after pgindent attached. This will involve a catversion bump since we're adding a new

Re: Possible bug in logical replication.

2018-06-21 Thread Petr Jelinek
On 20/06/18 09:59, Arseny Sher wrote: > > Michael Paquier writes: > >> On Mon, Jun 18, 2018 at 09:42:36PM +0900, Michael Paquier wrote: >>> On Fri, Jun 15, 2018 at 06:27:56PM +0300, Arseny Sher wrote: >>> It seems to me that we still want to have the slot forwarding finish in >>> this case even

Re: PATCH: backtraces for error messages

2018-06-21 Thread Kyotaro HORIGUCHI
Hello, I basically like this. At Thu, 21 Jun 2018 12:35:10 +0800, Craig Ringer wrote in > This is what the stacks look like btw > > > [2018-06-21 12:26:45.309 AWST] [7293] [] [] [:0] DEBUG: 0: > find_in_dynamic_libpath: trying > "/home/craig/pg/10/lib/postgresql/pglogical.so" >

Re: PATCH: backtraces for error messages

2018-06-21 Thread Pavan Deolasee
On Thu, Jun 21, 2018 at 11:02 AM, Michael Paquier wrote: > On Thu, Jun 21, 2018 at 12:35:10PM +0800, Craig Ringer wrote: > > I wrote it because I got sick of Assert(false) debugging, and I was > chasing > > down some "ERROR: 08P01: insufficient data left in message" errors. > Then I > > got

Re: WAL prefetch

2018-06-21 Thread Konstantin Knizhnik
I continue my experiments with WAL prefetch. I have embedded prefetch in Postgres: now walprefetcher is started together with startup process and is able to help it to speedup recovery. The patch is attached. Unfortunately result is negative (at least at my desktop: SSD, 16Gb RAM). Recovery

Re: partition table and stddev() /variance() behaviour

2018-06-21 Thread Tom Lane
David Rowley writes: > Well, that's quite surprising. It appears to be a bug in > numeric_poly_combine for machines without a working int128 type. The > parameters in accum_sum_copy are in the incorrect order. Ouch. > The very minimal fix is attached, but I'll need to go look at where > the

Re: New function pg_stat_statements_reset_query() to reset statistics of a specific query

2018-06-21 Thread Ashutosh Sharma
On Wed, Jun 20, 2018 at 1:00 PM, Haribabu Kommi wrote: > The pg_stat_statements contains the statistics of the queries that are > cumulative. > I find that any optimizations that are done to improve the performance of a > query > are not be visible clearly until the stats are reset. Currently

Re: Postgres 11 release notes

2018-06-21 Thread Alexander Korotkov
On Tue, Jun 19, 2018 at 1:40 PM Alexander Korotkov wrote: > On Tue, Jun 19, 2018 at 12:15 PM Alexander Korotkov > wrote: > > On Sat, Jun 16, 2018 at 3:57 PM Darafei "Komяpa" Praliaskouski > > wrote: > > >> > > >> > I'm not sure it is usefull in release notes since it is more about > > >> >

Re: server crashed with TRAP: FailedAssertion("!(!parallel_aware || pathnode->path.parallel_safe)"

2018-06-21 Thread Tom Lane
Amit Kapila writes: > On Thu, Jun 21, 2018 at 11:51 AM, Amit Khandekar > wrote: >> After list_concat, the subpaths no longer has only non-partial paths, >> which it is supposed to have. So it anyways should not be used in the >> subsequent code in that function. So I think the following change

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2018-06-21 Thread Bruce Momjian
On Wed, Jun 20, 2018 at 05:28:43PM -0500, Nico Williams wrote: > On Wed, Jun 20, 2018 at 06:19:40PM -0400, Joe Conway wrote: > > On 06/20/2018 05:12 PM, Bruce Momjian wrote: > > > On Mon, Jun 18, 2018 at 11:06:20AM -0400, Joe Conway wrote: > > > Even if they are encrypted with the same key, they

Wrong cost estimation for foreign tables join with use_remote_estimate disabled

2018-06-21 Thread Konstantin Knizhnik
Hi hackers, I hope that somebody understand postgres_fdw cost calculation magic better than I;) The following very simple test reduce the problem with wrong cost estimation: create table t1(x integer primary key, y integer); create index on t1(y); insert into t1 values

Re: Fast default stuff versus pg_upgrade

2018-06-21 Thread Andrew Dunstan
On 06/21/2018 01:18 PM, Tom Lane wrote: Andrew Dunstan writes: On 06/21/2018 12:39 PM, Tom Lane wrote: Andres Freund writes: On June 21, 2018 9:04:28 AM PDT, Tom Lane wrote: This isn't really ready to go. One clear problem is that you broke pg_dump'ing from any pre-v11 version, because

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2018-06-21 Thread Bruce Momjian
On Wed, Jun 20, 2018 at 06:19:40PM -0400, Joe Conway wrote: > On 06/20/2018 05:12 PM, Bruce Momjian wrote: > > On Mon, Jun 18, 2018 at 11:06:20AM -0400, Joe Conway wrote: > >>> At the same time, having to have a bunch of independently-decipherable > >>> short field values is not real secure

Re: Fast default stuff versus pg_upgrade

2018-06-21 Thread Tom Lane
Andres Freund writes: > On June 21, 2018 9:04:28 AM PDT, Tom Lane wrote: >> This isn't really ready to go. One clear problem is that you broke >> pg_dump'ing from any pre-v11 version, because you did not add suitable >> null outputs to the pre-v11 query variants in getTableAttrs. > Thought the

Re: Wrong cost estimation for foreign tables join with use_remote_estimate disabled

2018-06-21 Thread Tom Lane
Konstantin Knizhnik writes: > The following very simple test reduce the problem with wrong cost > estimation: > create foreign table t1_fdw(x integer, y integer) server pg_fdw options > (table_name 't1', use_remote_estimate 'false'); > create foreign table t2_fdw(x integer) server pg_fdw

Re: Fast default stuff versus pg_upgrade

2018-06-21 Thread Andres Freund
On 2018-06-21 13:44:19 -0400, Tom Lane wrote: > Andrew Dunstan writes: > > On 06/21/2018 01:18 PM, Tom Lane wrote: > >> I might be OK with a patch that converts *all* of pg_dump's cross-version > >> difference handling code to depend on PQfnumber silently returning -1 > >> rather than failing,

Re: Fast default stuff versus pg_upgrade

2018-06-21 Thread Tom Lane
Andres Freund writes: > On 2018-06-21 13:44:19 -0400, Tom Lane wrote: >> Actually, now that I think about it, there is a concrete reason for the >> historical pattern: it provides a cross-check that you did not fat-finger >> the query, ie misspell the column alias vs the PQfnumber parameter.

Re: partition table and stddev() /variance() behaviour

2018-06-21 Thread Tom Lane
David Rowley writes: > On 22 June 2018 at 02:01, Tom Lane wrote: >> coverage.postgresql.org shows that numeric_poly_serialize/combine() >> aren't exercised at all by the regression tests. Which is embarrassing >> for this case, but I'm a bit leery of trying to insist on 100% coverage. >> It

Re: Fast default stuff versus pg_upgrade

2018-06-21 Thread Tom Lane
Andrew Dunstan writes: > Patch after pgindent attached. This will involve a catversion bump since > we're adding a new function. This isn't really ready to go. One clear problem is that you broke pg_dump'ing from any pre-v11 version, because you did not add suitable null outputs to the pre-v11

Re: libpq compression

2018-06-21 Thread Konstantin Knizhnik
On 21.06.2018 17:56, Robbie Harwood wrote: Konstantin Knizhnik writes: On 20.06.2018 23:34, Robbie Harwood wrote: Konstantin Knizhnik writes: My idea was the following: client want to use compression. But server may reject this attempt (for any reasons: it doesn't support it, has no

Re: Fast default stuff versus pg_upgrade

2018-06-21 Thread Tom Lane
Andrew Dunstan writes: > On 06/21/2018 12:39 PM, Tom Lane wrote: >> Andres Freund writes: > On June 21, 2018 9:04:28 AM PDT, Tom Lane wrote: > This isn't really ready to go. One clear problem is that you broke > pg_dump'ing from any pre-v11 version, because you did not add suitable > null

Re: Fast default stuff versus pg_upgrade

2018-06-21 Thread Tom Lane
Andrew Dunstan writes: > On 06/21/2018 01:18 PM, Tom Lane wrote: >> I might be OK with a patch that converts *all* of pg_dump's cross-version >> difference handling code to depend on PQfnumber silently returning -1 >> rather than failing, but I don't want to see it done like that in just >> one

Re: Fast default stuff versus pg_upgrade

2018-06-21 Thread Andrew Dunstan
On 06/21/2018 01:44 PM, Tom Lane wrote: Andrew Dunstan writes: On 06/21/2018 01:18 PM, Tom Lane wrote: I might be OK with a patch that converts *all* of pg_dump's cross-version difference handling code to depend on PQfnumber silently returning -1 rather than failing, but I don't want to

Re: partition table and stddev() /variance() behaviour

2018-06-21 Thread David Rowley
On 22 June 2018 at 03:11, David Rowley wrote: > I think some coverage of the numerical aggregates is a good idea, so > I've added some in the attached. I managed to get a parallel plan > going with a query to onek, which is pretty cheap to execute. I didn't > touch the bool aggregates. Maybe I

Re: Fast default stuff versus pg_upgrade

2018-06-21 Thread Tom Lane
I wrote: > We could do without the unrelated whitespace changes in dumpDefaultACL, > too (or did pgindent introduce those? Seems surprising ... oh, looks > like "type" has somehow snuck into typedefs.list. Time for another > blacklist entry.) Hmm .. actually, "type" already is in pgindent's

Re: WAL prefetch

2018-06-21 Thread Tomas Vondra
On 06/21/2018 04:01 PM, Konstantin Knizhnik wrote: I continue my experiments with WAL prefetch. I have embedded prefetch in Postgres: now walprefetcher is started together with startup process and is able to help it to speedup recovery. The patch is attached. Unfortunately result is

Re: Considering signal handling in plpython again

2018-06-21 Thread Heikki Linnakangas
Hi Hubert, Are you working on this, or should I pick this up? Would be nice to get this done as soon as v12 development begins. - Heikki

Re: Fast default stuff versus pg_upgrade

2018-06-21 Thread Andrew Dunstan
On 06/21/2018 12:08 PM, Tom Lane wrote: I wrote: We could do without the unrelated whitespace changes in dumpDefaultACL, too (or did pgindent introduce those? Seems surprising ... oh, looks like "type" has somehow snuck into typedefs.list. Time for another blacklist entry.) Hmm ..

Re: Fast default stuff versus pg_upgrade

2018-06-21 Thread Andrew Dunstan
On 06/21/2018 12:39 PM, Tom Lane wrote: Andres Freund writes: On June 21, 2018 9:04:28 AM PDT, Tom Lane wrote: This isn't really ready to go. One clear problem is that you broke pg_dump'ing from any pre-v11 version, because you did not add suitable null outputs to the pre-v11 query

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2018-06-21 Thread Nico Williams
On Thu, Jun 21, 2018 at 10:14:54AM -0400, Bruce Momjian wrote: > On Wed, Jun 20, 2018 at 04:57:18PM -0500, Nico Williams wrote: > > Client-side crypto is hard to do well and still get decent performance. > > So on the whole I think that crypto is a poor fit for the DBAs-are-the- > > threat threat

Re: libpq compression

2018-06-21 Thread Robbie Harwood
Konstantin Knizhnik writes: > On 21.06.2018 17:56, Robbie Harwood wrote: >> Konstantin Knizhnik writes: >>> On 20.06.2018 23:34, Robbie Harwood wrote: Konstantin Knizhnik writes: Well, that's a design decision you've made. You could put lengths on chunks that are sent out

Re: location reporting in TAP test failures

2018-06-21 Thread Heikki Linnakangas
On 05/06/18 18:28, Peter Eisentraut wrote: Right now, when a TAP test reports a failure, it looks something like this: # Failed test 'creating a replication slot' # at /../postgresql/src/bin/pg_basebackup/../../../src/test/perl/TestLib.pm line 371. That file location is where we

Re: Speedup of relation deletes during recovery

2018-06-21 Thread Andres Freund
On 2018-06-21 14:40:58 +0900, Michael Paquier wrote: > On Wed, Jun 20, 2018 at 08:43:11PM -0700, Andres Freund wrote: > > On 2018-06-18 11:13:47 -0700, Andres Freund wrote: > >> We could do that - but add_to_unowned_list() is actually a bottleneck in > >> other places during recovery too. We

Re: Fast default stuff versus pg_upgrade

2018-06-21 Thread Tom Lane
Andrew Dunstan writes: > On 06/21/2018 01:44 PM, Tom Lane wrote: >> So I'm thinking that the attidentity code is just wrong, and you should >> change that too while you're at it. > That should be backpatched if changed, no? I don't think we'd want this > to get out of sync between the branches.

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2018-06-21 Thread Bruce Momjian
On Thu, Jun 21, 2018 at 04:49:34PM +0900, Masahiko Sawada wrote: > >> On Thu, Jun 21, 2018 at 6:57 AM, Nico Williams > >> wrote: > >> So on the whole I think that crypto is a poor fit for the DBAs-are-the- > >> threat threat model. It's better to reduce the number of DBAs/sysadmins > >> and

Re: Fast default stuff versus pg_upgrade

2018-06-21 Thread Andres Freund
On June 21, 2018 9:04:28 AM PDT, Tom Lane wrote: >Andrew Dunstan writes: >> Patch after pgindent attached. This will involve a catversion bump >since >> we're adding a new function. > >This isn't really ready to go. One clear problem is that you broke >pg_dump'ing from any pre-v11 version,

Re: phraseto_tsquery design

2018-06-21 Thread Arthur Zakirov
Hello, On Thu, Jun 21, 2018 at 11:02:32AM -0400, Sagiv Some wrote: > 2. However, it seems impossible to bypass the performance problem of phrase > searching. I conduct quite a bit of phrase searching, and although > postgres' "phraseto_tsquery" performs great on phrases with uncommon words, > it

Re: Spilling hashed SetOps and aggregates to disk

2018-06-21 Thread David Gershuni
> On Jun 19, 2018, at 10:36 PM, Jeff Davis wrote: > > But I am worried that I am missing something, because it appears that > for AGG_MIXED, we wait until the end of the last phase to emit the > contents of the hash tables. Wouldn't they be complete after the first > phase? You're right.

Re: libpq compression

2018-06-21 Thread Robbie Harwood
Konstantin Knizhnik writes: > On 20.06.2018 23:34, Robbie Harwood wrote: >> Konstantin Knizhnik writes: >> >> >> My idea was the following: client want to use compression. But server >> may reject this attempt (for any reasons: it doesn't support it, has >> no proper compression library, do not

Re: partition table and stddev() /variance() behaviour

2018-06-21 Thread David Rowley
On 22 June 2018 at 02:01, Tom Lane wrote: > David Rowley writes: >> Well, that's quite surprising. It appears to be a bug in >> numeric_poly_combine for machines without a working int128 type. The >> parameters in accum_sum_copy are in the incorrect order. > > Ouch. Yeah. Looks like this

Re: Spilling hashed SetOps and aggregates to disk

2018-06-21 Thread David Gershuni
On Jun 21, 2018, at 1:04 PM, Jeff Davis wrote: > On Thu, 2018-06-21 at 11:04 -0700, David Gershuni wrote: >> This approach seems functionally correct, but I don't like the idea >> of >> transforming O(N) tuples of disk I/O into O(S*N) tuples of disk I/O >> (in the worst case). > > It's the same

Re: libpq compression

2018-06-21 Thread Nico Williams
On Thu, Jun 21, 2018 at 10:12:17AM +0300, Konstantin Knizhnik wrote: > On 20.06.2018 23:34, Robbie Harwood wrote: > >Konstantin Knizhnik writes: > >Well, that's a design decision you've made. You could put lengths on > >chunks that are sent out - then you'd know exactly how much is needed. >

Re: Fast default stuff versus pg_upgrade

2018-06-21 Thread Alvaro Herrera
In terms of pgindent, I'm surprised about these lines: +missingval = OidFunctionCall3( + F_ARRAY_IN, Why did you put a newline there? In ancient times there was a reason for that in some cases, because pgindent would move the argument to the left of the open

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2018-06-21 Thread Nico Williams
On Fri, May 25, 2018 at 08:41:46PM +0900, Moon, Insung wrote: > Issues on data encryption of PostgreSQL > == > Currently, in PostgreSQL, data encryption can be using pgcrypto Tool. > However, it is inconvenient to use pgcrypto to encrypts data in some cases. > > There are two significant

PSA: --enable-coverage interferes with parallel query scheduling

2018-06-21 Thread Tom Lane
I've been poking at why coverage.postgresql.org shows that the second stanza in int8_avg_combine isn't exercised, when it clearly should be. I can reproduce the problem here, so it's fairly robust. Eventually it occurred to me to try a straight EXPLAIN ANALYZE VERBOSE, and what I find is that

Re: XML/XPath issues: text/CDATA in XMLTABLE, XPath evaluated with wrong context

2018-06-21 Thread Alvaro Herrera
I have pushed the patch now (in your original form rather than my later formulation) -- let's see how the buildfarm likes it. There are (at least) three issues remaining, as per below; Pavel, do you have any insight on these? First one is about array indexes not working sanely (I couldn't get

Continue work on changes to recovery.conf API

2018-06-21 Thread Sergei Kornilov
Hello all I would like to continue work on new recovery api proposed in thread [1]. We have some form of consensus but thread has been inactive for a long time and i hope i can help. I start from last published patch [2] and make some changes: - updated to current HEAD - made the patch pass

Re: Spilling hashed SetOps and aggregates to disk

2018-06-21 Thread Jeff Davis
On Thu, 2018-06-21 at 11:04 -0700, David Gershuni wrote: > This approach seems functionally correct, but I don't like the idea > of > transforming O(N) tuples of disk I/O into O(S*N) tuples of disk I/O > (in the worst case). It's the same amount of I/O as the idea you suggested as putting the

Re: Fast default stuff versus pg_upgrade

2018-06-21 Thread Tom Lane
Andrew Dunstan writes: > I left that for a separate exercise. I think this does things the way > you want. I must admit to being a bit surprised, I was expecting you to > have more to say about the upgrade function than the pg_dump changes :-) Well, the upgrade function is certainly a bit ugly

Re: Fast default stuff versus pg_upgrade

2018-06-21 Thread Tom Lane
Alvaro Herrera writes: > In terms of pgindent, I'm surprised about these lines: > +missingval = OidFunctionCall3( > + F_ARRAY_IN, > Why did you put a newline there? In ancient times there was a reason > for that in some cases, because pgindent would move the

Re: Fast default stuff versus pg_upgrade

2018-06-21 Thread Andrew Dunstan
On 06/21/2018 01:53 PM, Tom Lane wrote: Andrew Dunstan writes: On 06/21/2018 01:44 PM, Tom Lane wrote: So I'm thinking that the attidentity code is just wrong, and you should change that too while you're at it. That should be backpatched if changed, no? I don't think we'd want this to get

Re: Sample values for pg_stat_statements

2018-06-21 Thread Michael Paquier
On Thu, Jun 21, 2018 at 12:28:01PM +0200, Daniel Gustafsson wrote: > I happened to notice that this patch was moved from Returned with Feedback to > Needs Review after the CF closed, which means it’s now sitting open in a > closed > CF. The intended flow after RWF is that the patch is

Incorrect comments in commit_ts.c

2018-06-21 Thread shao bret
Hi, Here is the comment for CommitTsPagePrecedes in commit_ts.c /* * Decide which of two CLOG page numbers is "older" for truncation purposes. … */ I think there is some mistake for this comment. It should be ‘commitTS page’, not ‘CLOG page’. Thanks, Br. Bret 发送自 Windows 10

Re: Spilling hashed SetOps and aggregates to disk

2018-06-21 Thread Jeff Davis
On Thu, 2018-06-21 at 13:44 -0700, David Gershuni wrote: > To handle hash collisions, we can do the following: > > 1) We track the current hash code we’re processing, in ascending > order. > > 2) Instead of combining one group at at time, we’ll maintain a list > of > all groups we’ve seen that

Re: PATCH: backtraces for error messages

2018-06-21 Thread Craig Ringer
On Thu., 21 Jun. 2018, 19:26 Pavan Deolasee, wrote: > > > On Thu, Jun 21, 2018 at 11:02 AM, Michael Paquier > wrote: > >> On Thu, Jun 21, 2018 at 12:35:10PM +0800, Craig Ringer wrote: >> > I wrote it because I got sick of Assert(false) debugging, and I was >> chasing >> > down some "ERROR:

Re: partition table and stddev() /variance() behaviour

2018-06-21 Thread David Rowley
On 22 June 2018 at 03:30, Tom Lane wrote: >> I think some coverage of the numerical aggregates is a good idea, so >> I've added some in the attached. I managed to get a parallel plan >> going with a query to onek, which is pretty cheap to execute. I didn't >> touch the bool aggregates. Maybe I

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2018-06-21 Thread Bruce Momjian
On Thu, Jun 21, 2018 at 12:12:40PM -0500, Nico Williams wrote: > On Thu, Jun 21, 2018 at 10:14:54AM -0400, Bruce Momjian wrote: > > On Wed, Jun 20, 2018 at 04:57:18PM -0500, Nico Williams wrote: > > > Client-side crypto is hard to do well and still get decent performance. > > > So on the whole I

Re: PATCH: backtraces for error messages

2018-06-21 Thread Andres Freund
On 2018-06-22 08:24:45 +0800, Craig Ringer wrote: > On Thu., 21 Jun. 2018, 19:26 Pavan Deolasee, > wrote: > > > > > > > On Thu, Jun 21, 2018 at 11:02 AM, Michael Paquier > > wrote: > > > >> On Thu, Jun 21, 2018 at 12:35:10PM +0800, Craig Ringer wrote: > >> > I wrote it because I got sick of

Re: Considering signal handling in plpython again

2018-06-21 Thread Hubert Zhang
Hi Heikki, Not working on it now, you can go ahead. On Fri, Jun 22, 2018 at 12:56 AM, Heikki Linnakangas wrote: > Hi Hubert, > > Are you working on this, or should I pick this up? Would be nice to get > this done as soon as v12 development begins. > > - Heikki > -- Thanks Hubert Zhang

Re: PANIC during crash recovery of a recently promoted standby

2018-06-21 Thread Michael Paquier
On Fri, Jun 22, 2018 at 10:08:24AM +0530, Pavan Deolasee wrote: > On Fri, Jun 22, 2018 at 9:28 AM, Michael Paquier > wrote: >> So an extra pair of eyes from another committer would be >> welcome. I am letting that cool down for a couple of days now. > > I am not a committer, so don't know if my

Re: PANIC during crash recovery of a recently promoted standby

2018-06-21 Thread Michael Paquier
On Thu, Jun 07, 2018 at 07:58:29PM +0900, Kyotaro HORIGUCHI wrote: > (I believe that) By definition recovery doesn't end until the > end-of-recovery check point ends so from the viewpoint I think it > is wrong to clear ControlFile->minRecoveryPoint before the end. > > Invalid-page checking during

Re: Incorrect comments in commit_ts.c

2018-06-21 Thread Michael Paquier
On Fri, Jun 22, 2018 at 03:45:01AM +, shao bret wrote: > I think there is some mistake for this comment. It should be > ‘commitTS page’, not ‘CLOG page’. You are right, so pushed. -- Michael signature.asc Description: PGP signature

Re: Server crashed with TRAP: FailedAssertion("!(parallel_workers > 0)" when partitionwise_aggregate true.

2018-06-21 Thread Jeevan Chalke
On Wed, Jun 20, 2018 at 7:11 PM, Ashutosh Bapat < ashutosh.ba...@enterprisedb.com> wrote: > On Tue, Jun 19, 2018 at 2:13 PM, Jeevan Chalke > wrote: > > > > > > In the reported testcase, parallel_workers is set to 0 for all partition > > (child) relations. Which means partial parallel paths are

Re: Server crashed with TRAP: FailedAssertion("!(parallel_workers > 0)" when partitionwise_aggregate true.

2018-06-21 Thread Jeevan Chalke
Hi, Off-list Ashutosh Bapat has suggested using a flag instead of counting number of dummy rels and then manipulating on it. That will be simple and smoother. I agree with his suggestion and updated my patch accordingly. Thanks -- Jeevan Chalke Technical Architect, Product Development

Re: PATCH: backtraces for error messages

2018-06-21 Thread Pavan Deolasee
On Fri, Jun 22, 2018 at 6:18 AM, Andres Freund wrote: > On 2018-06-22 08:24:45 +0800, Craig Ringer wrote: > > > > > > Huge +1 from me on being able to selectively manage logging on a > > module/subsystem, file, or line level. > > > > I don't think I saw the post. > > > > Such a thing would

Threat models for DB cryptography (Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key) Management Service (KMS)

2018-06-21 Thread Nico Williams
On Thu, Jun 21, 2018 at 07:46:35PM -0400, Bruce Momjian wrote: > Agreed. I can see from this discussion that we have a long way to go > before we can produce something clearly useful, but it will be worth it. Let's start with a set of threat models then. I'll go first: 1) storage devices as

Re: [HACKERS] Pluggable storage

2018-06-21 Thread Haribabu Kommi
On Thu, Jun 14, 2018 at 12:25 PM Amit Kapila wrote: > On Thu, Jun 14, 2018 at 1:50 AM, Haribabu Kommi > wrote: > > > > On Fri, Apr 20, 2018 at 4:44 PM Haribabu Kommi > > > wrote: > > > > VACUUM: > > Not much changes are done in this apart moving the Vacuum visibility > > functions as part of

Re: PANIC during crash recovery of a recently promoted standby

2018-06-21 Thread Pavan Deolasee
On Fri, Jun 22, 2018 at 9:28 AM, Michael Paquier wrote: > > > This is not really a complicated patch, and it took a lot of energy from > me the last couple of days per the nature of the many scenarios to think > about... Thanks for the efforts. It wasn't an easy bug to chase to begin with. So

Re: PANIC during crash recovery of a recently promoted standby

2018-06-21 Thread Kyotaro HORIGUCHI
Hello, sorry for the absense and I looked the second patch. At Fri, 22 Jun 2018 13:45:21 +0900, Michael Paquier wrote in <20180622044521.gc5...@paquier.xyz> > On Fri, Jun 22, 2018 at 10:08:24AM +0530, Pavan Deolasee wrote: > > On Fri, Jun 22, 2018 at 9:28 AM, Michael Paquier > > wrote: > >> So

RE: Threat models for DB cryptography (Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key) Management Service (KMS)

2018-06-21 Thread Tsunakawa, Takayuki
From: Nico Williams [mailto:n...@cryptonector.com] > Let's start with a set of threat models then. I'll go first: Thank you so much for summarizing the current situation. I'd appreciate it if you could write this on the PostgreSQL wiki, when the discussion has settled somehow. > - local

Re: Push down Aggregates below joins

2018-06-21 Thread Antonin Houska
Tomas Vondra wrote: > On 06/20/2018 10:12 PM, Heikki Linnakangas wrote: > > Currently, the planner always first decides the scan/join order, and > > adds Group/Agg nodes on top of the joins. Sometimes it would be legal, > > and beneficial, to perform the aggregation below a join. I've been > >

Re: server crashed with TRAP: FailedAssertion("!(!parallel_aware || pathnode->path.parallel_safe)"

2018-06-21 Thread Amit Khandekar
On 20 June 2018 at 14:28, Amit Khandekar wrote: > On 20 June 2018 at 14:24, Amit Kapila wrote: >> On Mon, Jun 18, 2018 at 3:11 PM, Amit Khandekar >> wrote: >>> On 16 June 2018 at 10:44, Amit Kapila wrote: On Thu, Jun 14, 2018 at 10:05 PM, Tom Lane wrote: > > It looks to me like

Re: Fix some error handling for read() and errno

2018-06-21 Thread Michael Paquier
On Wed, Jun 20, 2018 at 02:52:23PM +0200, Magnus Hagander wrote: > On Mon, Jun 18, 2018 at 6:42 AM, Michael Paquier > wrote: >> Okay, so this makes two votes in favor of keeping the messages simple >> without context, shaped like "could not read file %s", with Robert and >> Alvaro, and two votes

bug with expression index on partition

2018-06-21 Thread Amit Langote
Hi. I noticed a bug with creating an expression index on a partitioned table. The fact that a partition may have differing attribute numbers is not accounted for when recursively *creating* the index on partition. The partition index gets created but is unusable. create table p (a int)