Re: Threat models for DB cryptography (Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key) Management Service (KMS)

2018-06-22 Thread Nico Williams
On Fri, Jun 22, 2018 at 05:31:44AM +, Tsunakawa, Takayuki wrote: > From: Nico Williams [mailto:n...@cryptonector.com] > > Let's start with a set of threat models then. I'll go first: > > Thank you so much for summarizing the current situation. I'd > appreciate it if you could write this on

Re: I'd like to discuss scaleout at PGCon

2018-06-22 Thread Merlin Moncure
On Fri, Jun 22, 2018 at 12:34 PM Bruce Momjian wrote: > > On Fri, Jun 1, 2018 at 11:29:43AM -0500, Merlin Moncure wrote: > > FWIW, Distributed analytical queries is the right market to be in. > > This is the field in which I work, and this is where the action is at. > > I am very, very, sure

Re: bug with expression index on partition

2018-06-22 Thread Alvaro Herrera
On 2018-Jun-21, Amit Langote wrote: > On 2018/06/21 15:35, Amit Langote wrote: > > So, CompareIndexInfo and generateClonedIndexStmt are both doing the right > > thing, but DefineIndex is not. Attached is a patch to fix DefineIndex so > > that it converts indexParams before recursing to create

Re: libpq compression

2018-06-22 Thread Konstantin Knizhnik
On 22.06.2018 19:05, Nico Williams wrote: On Fri, Jun 22, 2018 at 10:18:12AM +0300, Konstantin Knizhnik wrote: On 22.06.2018 00:34, Nico Williams wrote: So I think you just have to have lengths. Now, this being about compression, I understand that you might now want to have 4-byte lengths,

Re: Concurrency bug in UPDATE of partition-key

2018-06-22 Thread Amit Khandekar
On 20 June 2018 at 18:54, Amit Kapila wrote: > On Tue, Jun 19, 2018 at 9:33 PM, Amit Khandekar > wrote: >> >> Could not add RAISE statement, because the isolation test does not >> seem to print those statements in the output. Instead, I have dumped >> some rows in a new table through the

Re: [PATCH] Include application_name in "connection authorized" log message

2018-06-22 Thread Andres Freund
On 2018-06-22 12:16:18 -0400, Robert Haas wrote: > On Fri, Jun 22, 2018 at 11:17 AM, Stephen Frost wrote: > > The issue here is exactly that at the point where we emit the > > 'connection authorized' message, we haven't processed generic GUCs from > > the startup packet yet and therefore

Re: Speeding up INSERTs and UPDATEs to partitioned tables

2018-06-22 Thread David Rowley
On 22 June 2018 at 18:28, David Rowley wrote: > I've written fixes for items 1-6 above. > > I did: > > 1. Use an array instead of a List. > 2. Don't do this loop. palloc0() the partitions array instead. Let > UPDATE add whatever subplans exist to the zeroed array. > 3. Track what we initialize in

Re: [WIP] [B-Tree] Retail IndexTuple deletion

2018-06-22 Thread Peter Geoghegan
On Fri, Jun 22, 2018 at 12:43 PM, Peter Geoghegan wrote: > On Fri, Jun 22, 2018 at 4:24 AM, Andrey V. Lepikhov > wrote: >> According to your feedback, i develop second version of the patch. >> In this version: >> 1. High-level functions index_beginscan(), index_rescan() not used. Tree >> descent

Re: Using JSONB directly from application

2018-06-22 Thread Christian Ohler
(continuing an old thread from https://www.postgresql.org/message-id/CAMsr%2BYEtamQYZ5EocsuthQCvyvmRnQrucDP6GZynPtf0gsMbuw%40mail.gmail.com ) Craig Ringer , 2018-02-26: > On 26 February 2018 at 04:05, Anthony Communier wrote: > > > It would be nice if application connected to a Postrgesql

Re: bug with expression index on partition

2018-06-22 Thread Alvaro Herrera
On 2018-Jun-21, Amit Langote wrote: > explain (costs off) select p from p order by p; > QUERY PLAN > --- > Merge Append >Sort Key: ((p1.*)::p) >-> Index Scan using p1_p_idx on p1 >-> Index Scan using p2_p_idx on p2 >-> Index

Re: Fix some error handling for read() and errno

2018-06-22 Thread Michael Paquier
On Fri, Jun 22, 2018 at 09:51:30AM -0400, Alvaro Herrera wrote: > On 2018-Jun-22, Robert Haas wrote: >> I think this should be split into two patches. Fixing failure to save >> and restore errno is a different issue from cleaning up short write >> messages. I agree that the former should be

Re: [PATCH] Include application_name in "connection authorized" log message

2018-06-22 Thread Alvaro Herrera
On 2018-Jun-22, Andres Freund wrote: > On 2018-06-22 15:50:06 -0400, Alvaro Herrera wrote: > > On 2018-Jun-22, Andres Freund wrote: > > > > > On 2018-06-22 15:26:06 -0400, Alvaro Herrera wrote: > > > > On 2018-Jun-22, Andres Freund wrote: > > > > > I think a fair argument could be made that

Re: Using JSONB directly from application

2018-06-22 Thread Nico Williams
On Mon, Feb 26, 2018 at 09:36:52PM +0800, Craig Ringer wrote: > On 26 February 2018 at 04:05, Anthony Communier > wrote: > > It would be nice if application connected to a Postrgesql database could > > send and receive JSONB in binary. It could save some useless text > > conversion. All works

Re: I'd like to discuss scaleout at PGCon

2018-06-22 Thread Bruce Momjian
On Fri, Jun 22, 2018 at 01:28:58PM -0500, Merlin Moncure wrote: > On Fri, Jun 22, 2018 at 12:34 PM Bruce Momjian wrote: > > > > What we don't want to do is to add a bunch of sharding-specific code > > without knowing which workloads it benefits, and how many of our users > > will actually use

Re: New function pg_stat_statements_reset_query() to reset statistics of a specific query

2018-06-22 Thread Ashutosh Sharma
On Sat, Jun 23, 2018 at 7:36 AM, Haribabu Kommi wrote: > On Sat, Jun 23, 2018 at 5:45 AM Euler Taveira wrote: >> >> 2018-06-22 12:06 GMT-03:00 Robert Haas : >> > On Wed, Jun 20, 2018 at 10:19 AM, Euler Taveira >> > wrote: >> >> 2018-06-20 4:30 GMT-03:00 Haribabu Kommi : >> >>> Attached is a

Re: New function pg_stat_statements_reset_query() to reset statistics of a specific query

2018-06-22 Thread Haribabu Kommi
On Sat, Jun 23, 2018 at 5:45 AM Euler Taveira wrote: > 2018-06-22 12:06 GMT-03:00 Robert Haas : > > On Wed, Jun 20, 2018 at 10:19 AM, Euler Taveira > wrote: > >> 2018-06-20 4:30 GMT-03:00 Haribabu Kommi : > >>> Attached is a simple patch with implementation. Comments? > >>> > >> Why don't you

Re: I'd like to discuss scaleout at PGCon

2018-06-22 Thread Haribabu Kommi
On Sat, Jun 23, 2018 at 12:17 PM Bruce Momjian wrote: > On Fri, Jun 22, 2018 at 01:28:58PM -0500, Merlin Moncure wrote: > > On Fri, Jun 22, 2018 at 12:34 PM Bruce Momjian wrote: > > > > > > What we don't want to do is to add a bunch of sharding-specific code > > > without knowing which

Re: I'd like to discuss scaleout at PGCon

2018-06-22 Thread Bruce Momjian
On Sat, Jun 23, 2018 at 12:41:00PM +1000, Haribabu Kommi wrote: > > On Sat, Jun 23, 2018 at 12:17 PM Bruce Momjian wrote: > > On Fri, Jun 22, 2018 at 01:28:58PM -0500, Merlin Moncure wrote: > > On Fri, Jun 22, 2018 at 12:34 PM Bruce Momjian wrote: > > > > > > What we don't want

Re: SCRAM with channel binding downgrade attack

2018-06-22 Thread Bruce Momjian
On Sun, Jun 17, 2018 at 10:21:27PM +0900, Michael Paquier wrote: > On Fri, Jun 15, 2018 at 05:23:27PM -0400, Robert Haas wrote: > > On Thu, Jun 14, 2018 at 7:43 AM, Magnus Hagander > > wrote: > >> I still think that the fact that we are still discussing what is basically > >> the *basic

Re: [PATCH] Include application_name in "connection authorized" log message

2018-06-22 Thread Robert Haas
On Fri, Jun 22, 2018 at 11:17 AM, Stephen Frost wrote: > The issue here is exactly that at the point where we emit the > 'connection authorized' message, we haven't processed generic GUCs from > the startup packet yet and therefore application_name isn't set as a > GUC and, as a result, isn't

Re: [PATCH] Include application_name in "connection authorized" log message

2018-06-22 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Fri, Jun 22, 2018 at 11:17 AM, Stephen Frost wrote: > > The issue here is exactly that at the point where we emit the > > 'connection authorized' message, we haven't processed generic GUCs from > > the startup packet yet and therefore

Re: libpq compression

2018-06-22 Thread Nico Williams
On Fri, Jun 22, 2018 at 10:18:12AM +0300, Konstantin Knizhnik wrote: > On 22.06.2018 00:34, Nico Williams wrote: > >So I think you just have to have lengths. > > > >Now, this being about compression, I understand that you might now want > >to have 4-byte lengths, especially given that most

Re: libpq compression

2018-06-22 Thread Robbie Harwood
Konstantin Knizhnik writes: > On 22.06.2018 18:59, Robbie Harwood wrote: >> Konstantin Knizhnik writes: >>> On 21.06.2018 20:14, Robbie Harwood wrote: Konstantin Knizhnik writes: > On 21.06.2018 17:56, Robbie Harwood wrote: >> Konstantin Knizhnik writes: >>> On 20.06.2018

Re: [PATCH] Include application_name in "connection authorized" log message

2018-06-22 Thread Andres Freund
On 2018-06-22 15:50:06 -0400, Alvaro Herrera wrote: > On 2018-Jun-22, Andres Freund wrote: > > > On 2018-06-22 15:26:06 -0400, Alvaro Herrera wrote: > > > On 2018-Jun-22, Andres Freund wrote: > > > > I think a fair argument could be made that you'd want to have > > > > application_name logged

Re: libpq compression

2018-06-22 Thread Robbie Harwood
Konstantin Knizhnik writes: > On 21.06.2018 20:14, Robbie Harwood wrote: >> Konstantin Knizhnik writes: >>> On 21.06.2018 17:56, Robbie Harwood wrote: Konstantin Knizhnik writes: > On 20.06.2018 23:34, Robbie Harwood wrote: >> Konstantin Knizhnik writes: >> >> Well,

Re: [PATCH] Include application_name in "connection authorized" log message

2018-06-22 Thread Alvaro Herrera
On 2018-Jun-22, Andres Freund wrote: > On 2018-06-22 12:16:18 -0400, Robert Haas wrote: > > OK, that makes more sense, but I'm still skeptical of adding a special > > case particularly for application_name. > > I think a fair argument could be made that you'd want to have > application_name

Re: New function pg_stat_statements_reset_query() to reset statistics of a specific query

2018-06-22 Thread Euler Taveira
2018-06-22 12:06 GMT-03:00 Robert Haas : > On Wed, Jun 20, 2018 at 10:19 AM, Euler Taveira wrote: >> 2018-06-20 4:30 GMT-03:00 Haribabu Kommi : >>> Attached is a simple patch with implementation. Comments? >>> >> Why don't you extend the existing function pg_stat_statements_reset()? > > Well, the

Re: found xmin from before relfrozenxid on pg_catalog.pg_authid

2018-06-22 Thread Matheus de Oliveira
Hello again... On Tue, Jun 19, 2018 at 12:53 PM, Andres Freund wrote: > ... > > After that, yes, deleting the > global/pg_internal.init file is the way to go, and I can't think of a > case where it's problematic, even without stopping the server. > > Just to let you know. I applied the work

Re: I'd like to discuss scaleout at PGCon

2018-06-22 Thread Joshua D. Drake
On 06/22/2018 11:28 AM, Merlin Moncure wrote: Key features from my perspective: *) fdw in parallel. how do i do it today? ghetto implemented parallel queries with asynchronous dblink *) column store Although not in core, we do have this as an extension through Citus don't we? JD --

Re: Incorrect errno used with %m for backend code

2018-06-22 Thread Andres Freund
Hi, On 2018-06-22 11:15:53 -0400, Alvaro Herrera wrote: > I don't understand the logic in RestoreSlotFromDisk that fsync the file > prior to reading it. What are we protecting against? we could previously have crashed before fsyncing. But we can only rely on slot contents being durable if we

Re: [PATCH] Include application_name in "connection authorized" log message

2018-06-22 Thread Andres Freund
On 2018-06-22 15:26:06 -0400, Alvaro Herrera wrote: > On 2018-Jun-22, Andres Freund wrote: > > > On 2018-06-22 12:16:18 -0400, Robert Haas wrote: > > > > OK, that makes more sense, but I'm still skeptical of adding a special > > > case particularly for application_name. > > > > I think a fair

Re: New function pg_stat_statements_reset_query() to reset statistics of a specific query

2018-06-22 Thread Rui DeSousa
Why not just parameterize it with the three key fields; userid, dbid, and queryid? i.e It would then allow it be limited to only records associated with a given user and/or database as well. pg_stat_statements_reset(dbid oid, userid oid, queryid bigint) pg_stat_statements_reset(null, null,

Re: I'd like to discuss scaleout at PGCon

2018-06-22 Thread Bruce Momjian
On Fri, Jun 1, 2018 at 11:29:43AM -0500, Merlin Moncure wrote: > FWIW, Distributed analytical queries is the right market to be in. > This is the field in which I work, and this is where the action is at. > I am very, very, sure about this. My view is that many of the > existing solutions to

Re: [WIP] [B-Tree] Retail IndexTuple deletion

2018-06-22 Thread Peter Geoghegan
On Fri, Jun 22, 2018 at 4:24 AM, Andrey V. Lepikhov wrote: > According to your feedback, i develop second version of the patch. > In this version: > 1. High-level functions index_beginscan(), index_rescan() not used. Tree > descent made by _bt_search(). _bt_binsrch() used for positioning on the >

Re: [PATCH] Include application_name in "connection authorized" log message

2018-06-22 Thread Alvaro Herrera
On 2018-Jun-22, Andres Freund wrote: > On 2018-06-22 15:26:06 -0400, Alvaro Herrera wrote: > > On 2018-Jun-22, Andres Freund wrote: > > > > > On 2018-06-22 12:16:18 -0400, Robert Haas wrote: > > > > > > OK, that makes more sense, but I'm still skeptical of adding a special > > > > case

Re: libpq compression

2018-06-22 Thread Konstantin Knizhnik
On 22.06.2018 18:59, Robbie Harwood wrote: Konstantin Knizhnik writes: On 21.06.2018 20:14, Robbie Harwood wrote: Konstantin Knizhnik writes: On 21.06.2018 17:56, Robbie Harwood wrote: Konstantin Knizhnik writes: On 20.06.2018 23:34, Robbie Harwood wrote: Konstantin Knizhnik

Re: PANIC during crash recovery of a recently promoted standby

2018-06-22 Thread Michael Paquier
On Fri, Jun 22, 2018 at 02:34:02PM +0900, Kyotaro HORIGUCHI wrote: > Hello, sorry for the absense and I looked the second patch. Thanks for the review! > At Fri, 22 Jun 2018 13:45:21 +0900, Michael Paquier > wrote in <20180622044521.gc5...@paquier.xyz> >> long as crash recovery runs. And

Re: Wrong cost estimation for foreign tables join with use_remote_estimate disabled

2018-06-22 Thread Konstantin Knizhnik
On 21.06.2018 20:08, Tom Lane wrote: Konstantin Knizhnik writes: The following very simple test reduce the problem with wrong cost estimation: create foreign table t1_fdw(x integer, y integer) server pg_fdw options (table_name 't1', use_remote_estimate 'false'); create foreign table

Re: libpq compression

2018-06-22 Thread Konstantin Knizhnik
On 21.06.2018 20:14, Robbie Harwood wrote: Konstantin Knizhnik writes: On 21.06.2018 17:56, Robbie Harwood wrote: Konstantin Knizhnik writes: On 20.06.2018 23:34, Robbie Harwood wrote: Konstantin Knizhnik writes: Well, that's a design decision you've made. You could put lengths on

Re: Incorrect errno used with %m for backend code

2018-06-22 Thread Ashutosh Sharma
Hi, It seems like in case of few system calls for e.g. write system call, errno is not set even if the number of bytes written is smaller than the bytes requested and for such cases we explicitly set an errno to ENOSPC. Something like this, /* * if write didn't

Incorrect errno used with %m for backend code

2018-06-22 Thread Michael Paquier
Hi all, I have been reviewing the use of errno in the backend code when %m is used in the logs, and found more places than when I looked at improving the error messages for read() calls which bloat the errno value because of intermediate system calls. As the problem is separate and should be

Re: partition table and stddev() /variance() behaviour

2018-06-22 Thread Rajkumar Raghuwanshi
Thanks for commit. I have verified reported case. it is fixed now. Thanks & Regards, Rajkumar Raghuwanshi QMG, EnterpriseDB Corporation On Fri, Jun 22, 2018 at 8:38 AM, David Rowley wrote: > On 22 June 2018 at 03:30, Tom Lane wrote: > >> I think some coverage of the numerical aggregates is a

Re: Push down Aggregates below joins

2018-06-22 Thread Antonin Houska
Heikki Linnakangas wrote: > Ah, cool! I missed that thread earlier. Yes, seems like we've been hacking on > the same feature. Let's compare! > > I've been using this paper as a guide: > > "Including Group-By in Query Optimization", by Surajit Chaudhuri and Kyuseok > Shim: >

Re: Continue work on changes to recovery.conf API

2018-06-22 Thread Craig Ringer
On 22 June 2018 at 02:48, Sergei Kornilov wrote: > I completely forgot attach patch, sorry. Attached now Sergei, It's great to see you picking this up. I'll try to find time to review this for the next CF cycle. Please poke me if you don't hear from me, I do get distracted. It's long past

Re: Server crashed with TRAP: FailedAssertion("!(parallel_workers > 0)" when partitionwise_aggregate true.

2018-06-22 Thread Rajkumar Raghuwanshi
On Fri, Jun 22, 2018 at 11:15 AM, Jeevan Chalke < jeevan.cha...@enterprisedb.com> wrote: > Hi, > > Off-list Ashutosh Bapat has suggested using a flag instead of counting > number of > dummy rels and then manipulating on it. That will be simple and smoother. > > I agree with his suggestion and

Speeding up INSERTs and UPDATEs to partitioned tables

2018-06-22 Thread David Rowley
Hi, As part of my efforts to make partitioning scale better for larger numbers of partitions, I've been looking at primarily INSERT VALUES performance. Here the overheads are almost completely in the executor. Planning of this type of statement is very simple since there is no FROM clause to

Re: server crashed with TRAP: FailedAssertion("!(!parallel_aware || pathnode->path.parallel_safe)"

2018-06-22 Thread Rajkumar Raghuwanshi
Thanks for commit. I have verified reported case. it is fixed now. Thanks & Regards, Rajkumar Raghuwanshi QMG, EnterpriseDB Corporation On Thu, Jun 21, 2018 at 7:21 PM, Tom Lane wrote: > Amit Kapila writes: > > On Thu, Jun 21, 2018 at 11:51 AM, Amit Khandekar > wrote: > >> After list_concat,

Re: PATCH: backtraces for error messages

2018-06-22 Thread Craig Ringer
On 22 June 2018 at 08:48, Andres Freund wrote: > On 2018-06-22 08:24:45 +0800, Craig Ringer wrote: > > On Thu., 21 Jun. 2018, 19:26 Pavan Deolasee, > > wrote: > > > > > > > > > > > On Thu, Jun 21, 2018 at 11:02 AM, Michael Paquier > > > > wrote: > > > > > >> On Thu, Jun 21, 2018 at 12:35:10PM

Re: libpq compression

2018-06-22 Thread Konstantin Knizhnik
On 22.06.2018 00:34, Nico Williams wrote: On Thu, Jun 21, 2018 at 10:12:17AM +0300, Konstantin Knizhnik wrote: On 20.06.2018 23:34, Robbie Harwood wrote: Konstantin Knizhnik writes: Well, that's a design decision you've made. You could put lengths on chunks that are sent out - then you'd

Re: ERROR: ORDER/GROUP BY expression not found in targetlist

2018-06-22 Thread Rajkumar Raghuwanshi
Thanks for commit. I have verified reported case. it is fixed now. Thanks & Regards, Rajkumar Raghuwanshi QMG, EnterpriseDB Corporation On Thu, Jun 21, 2018 at 1:54 AM, Tom Lane wrote: > I wrote: > > Thanks for the report. I traced through this, and the problem seems to > > be that

Re: Possible bug in logical replication.

2018-06-22 Thread Kyotaro HORIGUCHI
Hello. At Thu, 14 Jun 2018 16:06:43 -0400, Alvaro Herrera wrote in <20180614200643.3my362zmfiwitrni@alvherre.pgsql> > Can somebody (Arseny, Konstantin, horiguti, Sawada) please confirm that > Michaël's commit fixes the reported bug? pg_advance_replication_slots can advance uninitialized

答复: Incorrect comments in commit_ts.c

2018-06-22 Thread shao bret
Thanks, I have seen the code change. Br. Bret. HighGo Software Website:http://www.highgo.com/?l=en 发送自 Windows 10 版邮件应用 发件人: Michael Paquier 发送时间: 2018年6月22日 12:32 收件人: shao bret 抄送:

utilities to rebuild commit logs from wal

2018-06-22 Thread Chris Travers
Before we reinvent the wheel here, does anyone know of utilities to build commit logs from wal segments? Or even to just fill with commits? I figure it is worth asking before I write one.

Re: server crashed with TRAP: FailedAssertion("!(!parallel_aware || pathnode->path.parallel_safe)"

2018-06-22 Thread Amit Kapila
On Thu, Jun 21, 2018 at 7:21 PM, Tom Lane wrote: > Amit Kapila writes: >> On Thu, Jun 21, 2018 at 11:51 AM, Amit Khandekar >> wrote: >>> After list_concat, the subpaths no longer has only non-partial paths, >>> which it is supposed to have. So it anyways should not be used in the >>>

Re: [HACKERS] GUC for cleanup indexes threshold.

2018-06-22 Thread Alexander Korotkov
On Wed, Jun 20, 2018 at 12:00 PM Alexander Korotkov wrote: > On Wed, Jun 20, 2018 at 8:32 AM Masahiko Sawada wrote: > > On Wed, Jun 20, 2018 at 1:00 AM, Alexander Korotkov > > > Ok. I've rephrased comment a bit. Also, you created "index vacuum" > > > subsection in the "resource usage" section.

Re: Wrong cost estimation for foreign tables join with use_remote_estimate disabled

2018-06-22 Thread Etsuro Fujita
Hi Konstantin, (2018/06/22 15:26), Konstantin Knizhnik wrote: On 21.06.2018 20:08, Tom Lane wrote: Konstantin Knizhnik writes: The following very simple test reduce the problem with wrong cost estimation: create foreign table t1_fdw(x integer, y integer) server pg_fdw options (table_name

Re: Incorrect errno used with %m for backend code

2018-06-22 Thread Michael Paquier
On Fri, Jun 22, 2018 at 01:00:45PM +0530, Ashutosh Sharma wrote: > It seems like in case of few system calls for e.g. write system call, > errno is not set even if the number of bytes written is smaller than > the bytes requested and for such cases we explicitly set an errno to > ENOSPC. Something

Re: Possible bug in logical replication.

2018-06-22 Thread Michael Paquier
On Fri, Jun 22, 2018 at 04:33:12PM +0900, Kyotaro HORIGUCHI wrote: > pg_advance_replication_slots can advance uninitialized physical > slots and that might not be good. (Logical slots always have > initial invalid values in thw two lsn columns.) The current logic is careful that users willing to

Re: Wrong cost estimation for foreign tables join with use_remote_estimate disabled

2018-06-22 Thread Ashutosh Bapat
On Fri, Jun 22, 2018 at 11:56 AM, Konstantin Knizhnik wrote: > > > On 21.06.2018 20:08, Tom Lane wrote: >> >> Konstantin Knizhnik writes: >>> >>> The following very simple test reduce the problem with wrong cost >>> estimation: >>> create foreign table t1_fdw(x integer, y integer) server pg_fdw

Re: Wrong cost estimation for foreign tables join with use_remote_estimate disabled

2018-06-22 Thread Ashutosh Bapat
Sorry here's preview link [1] [1] https://ashutoshpg.blogspot.com/b/post-preview?token=TCTIKGQBAAA.2iKpIUItkwZLkXiujvs0zad-DtDdKbwIdRFCGbac9--XbqcA-xnCdz4wmbD4hIaEHuyg5Xrz8eZq8ZNmw83yfQ.HXi__guM-7SzdIWi27QkjA=4019325618679658571=POST On Fri, Jun 22, 2018 at 4:00 PM, Ashutosh Bapat wrote: > On

Lifetime of commit timestamps

2018-06-22 Thread Kyotaro HORIGUCHI
Hello. I don't find any description in the documentation about the guaranteed lifetime of commit timestamps. I think they are preserved until corresponding xid goes beyond the freeze horizen, even though they are actually preserved longer for several reasons. If it is not, I think such

Re: Wrong cost estimation for foreign tables join with use_remote_estimate disabled

2018-06-22 Thread Amit Langote
On 2018/06/22 18:15, Etsuro Fujita wrote: > I'd vote for #2.  One idea for that is to introduce CREATE FOREIGN INDEX > to have information on remote indexes on the local side, which I proposed > before.  I have been putting it on hold since then, though. Sorry to hijack this thread, but I'd like

Re: WAL prefetch

2018-06-22 Thread Konstantin Knizhnik
On 21.06.2018 19:57, Tomas Vondra wrote: On 06/21/2018 04:01 PM, Konstantin Knizhnik wrote: I continue my experiments with WAL prefetch. I have embedded prefetch in Postgres: now walprefetcher is started together with startup process and is able to help it to speedup recovery. The patch

Re: add default parallel query to v10 release notes? (Re: [PERFORM] performance drop after upgrade (9.6 > 10))

2018-06-22 Thread Amit Kapila
On Wed, Jun 20, 2018 at 8:43 PM, Bruce Momjian wrote: > On Thu, May 24, 2018 at 08:00:25PM -0500, Justin Pryzby wrote: > > So I did some research on this, particularly to find out how it was > missed in the PG 10 release notes. It turns out that > max_parallel_workers_per_gather has always

Re: Incorrect errno used with %m for backend code

2018-06-22 Thread Ashutosh Sharma
On Fri, Jun 22, 2018 at 2:44 PM, Michael Paquier wrote: > On Fri, Jun 22, 2018 at 01:00:45PM +0530, Ashutosh Sharma wrote: >> It seems like in case of few system calls for e.g. write system call, >> errno is not set even if the number of bytes written is smaller than >> the bytes requested and

Re: Wrong cost estimation for foreign tables join with use_remote_estimate disabled

2018-06-22 Thread Etsuro Fujita
(2018/06/22 18:49), Amit Langote wrote: On 2018/06/22 18:15, Etsuro Fujita wrote: I'd vote for #2. One idea for that is to introduce CREATE FOREIGN INDEX to have information on remote indexes on the local side, which I proposed before. I have been putting it on hold since then, though.

Re: Wrong cost estimation for foreign tables join with use_remote_estimate disabled

2018-06-22 Thread Konstantin Knizhnik
On 22.06.2018 13:30, Ashutosh Bapat wrote: On Fri, Jun 22, 2018 at 11:56 AM, Konstantin Knizhnik wrote: On 21.06.2018 20:08, Tom Lane wrote: Konstantin Knizhnik writes: The following very simple test reduce the problem with wrong cost estimation: create foreign table t1_fdw(x integer, y

Re: [WIP] [B-Tree] Retail IndexTuple deletion

2018-06-22 Thread Andrey V. Lepikhov
Hi, According to your feedback, i develop second version of the patch. In this version: 1. High-level functions index_beginscan(), index_rescan() not used. Tree descent made by _bt_search(). _bt_binsrch() used for positioning on the page. 2. TID list introduced in amtargetdelete() interface.

Re: Fast default stuff versus pg_upgrade

2018-06-22 Thread Andrew Dunstan
On 06/21/2018 04:41 PM, Tom Lane wrote: Andrew Dunstan writes: I left that for a separate exercise. I think this does things the way you want. I must admit to being a bit surprised, I was expecting you to have more to say about the upgrade function than the pg_dump changes :-) Well, the

Re: PSA: --enable-coverage interferes with parallel query scheduling

2018-06-22 Thread Robert Haas
On Thu, Jun 21, 2018 at 4:00 PM, Tom Lane wrote: > We could probably fix it by using a significantly larger test case, > but that's not very attractive to put into the regression tests. > Anybody have a better idea about how to improve this? Or even a > clear explanation for what's causing it?

Psql patch to show access methods info

2018-06-22 Thread Sergey Cherkashin
Hello! There are command in psql to list access methods, but there are no fast way to look detailed info about them. So here a patch with new commands: \dAp [PATTERN]   list access methods with properties (Table pg_am) \dAf[+]  [AMPTRN [OPFPTRN]]  list operator families of access

Re: Keeping temporary tables in shared buffers

2018-06-22 Thread Robert Haas
On Mon, May 28, 2018 at 4:25 AM, Amit Kapila wrote: > On Fri, May 25, 2018 at 6:33 AM, Asim Praveen wrote: >> We are evaluating the use of shared buffers for temporary tables. The >> advantage being queries involving temporary tables can make use of parallel >> workers. > This is one way, but I

Re: Fix some error handling for read() and errno

2018-06-22 Thread Robert Haas
On Thu, Jun 21, 2018 at 2:32 AM, Michael Paquier wrote: > Sure. I have looked at that and I am finishing with the updated version > attached. > > I removed the portion about slru in the code, but I would like to add at > least a mention about it in the commit message. The simplifications are >

Re: "wal receiver" process hang in syslog() while exiting after receiving SIGTERM while the postgres has been promoted.

2018-06-22 Thread Robert Haas
On Thu, Jun 21, 2018 at 1:11 AM, Chen, Yan-Jack (NSB - CN/Hangzhou) wrote: > Hi Hackers, > We encounter one problem happened while we try to promote standby > postgres(version 9.6.9) server to active. From the trace(we triggered the > process abort). We can see the process was hang in syslog()

Re: Server crashed with TRAP: FailedAssertion("!(parallel_workers > 0)" when partitionwise_aggregate true.

2018-06-22 Thread Robert Haas
On Fri, Jun 22, 2018 at 2:26 AM, Rajkumar Raghuwanshi wrote: > I have applied patch and checked reported issue. Patch applied cleanly and > issues not reproducible any more. Committed, with a few changes: - Renamed found_partially_grouped_child to partial_grouping_valid. The old name seemed to

Re: Fix some error handling for read() and errno

2018-06-22 Thread Alvaro Herrera
On 2018-Jun-22, Robert Haas wrote: > I think this should be split into two patches. Fixing failure to save > and restore errno is a different issue from cleaning up short write > messages. I agree that the former should be back-patched and the > latter not. Hmm, yeah, good thought, +1. --

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-06-22 Thread Robert Haas
On Tue, Jun 19, 2018 at 8:46 AM, Etsuro Fujita wrote: > Here is a patch for that. > > * As I said upthread, the patch makes code much more simple; I removed all > the changes to setrefs.c added by the partitionwise-join patch. I also > simplified the logic for building a tlist for a child-join

Re: [PATCH] Include application_name in "connection authorized" log message

2018-06-22 Thread Robert Haas
On Wed, Jun 20, 2018 at 3:45 PM, Stephen Frost wrote: > * Don Seiler (d...@seiler.us) wrote: >> In trying to troubleshoot the source of a recent connection storm, I was >> frustrated to find that the app name was not included in the connection >> messages. It is there in the log_line_prefix on

Re: Incorrect errno used with %m for backend code

2018-06-22 Thread Alvaro Herrera
On 2018-Jun-22, Michael Paquier wrote: > A couple of places use CloseTransientFile without bothering much that > this can overwrite errno. I was tempted in keeping errno saved and kept > if set to a non-zero value, but refrained from doing so as some callers > may rely on the existing behavior,

Re: [PATCH] Include application_name in "connection authorized" log message

2018-06-22 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Jun 20, 2018 at 3:45 PM, Stephen Frost wrote: > > * Don Seiler (d...@seiler.us) wrote: > >> In trying to troubleshoot the source of a recent connection storm, I was > >> frustrated to find that the app name was not included in the

Re: PATCH: backtraces for error messages

2018-06-22 Thread Korry Douglas
A few misc comments: In general, +1. It might be nice to move the stack trace code into a separate function (not static to elog.c) - I have often found it useful to attach backtraces to objects so I can debug complex allocation/deallocation problems. The major expense in capturing a stack trace