Re: [PATCH] Log CSV by default

2018-12-03 Thread Gilles Darold
Le 04/12/2018 à 03:37, Michael Paquier a écrit : > On Mon, Dec 03, 2018 at 09:20:01PM +0100, Gilles Darold wrote: >> pgBadger is able to parse jsonlog >> (https://github.com/michaelpq/pg_plugins/tree/master/jsonlog) output in >> multi-process mode because it do not use an external library to parse

Re: Use durable_unlink for .ready and .done files for WAL segment removal

2018-12-03 Thread Michael Paquier
On Thu, Nov 29, 2018 at 03:00:42PM +, Bossart, Nathan wrote: > +1 Okay, here is an updated patch for this stuff, which does the following: - Check for a WAL segment if it has a ".ready" status file, an orphaned status file is removed only on ENOENT. - If durable_unlink fails, retry 3 times.

RE: [Todo item] Add entry creation timestamp column to pg_stat_replication

2018-12-03 Thread myungkyu.lim
>> I have been looking at this patch more in-depth, and you missed one >> critical thing: hot standby feedback messages also include the >> timestamp the client used when sending the message, so if we want to >> track the latest time when a message has been sent we should track it >> as much as

Re: doc - improve description of default privileges

2018-12-03 Thread Fabien COELHO
I looked at the psql manpage and the HTML rendering of section 5.6 and it all looks good to me. Indeed, this looks great, a precise and full description of privileges just in one place. Pushed (with a little bit more tweaking). Thanks for the rewrite, extensions, improvements and final

slow queries over information schema.tables

2018-12-03 Thread Pavel Stehule
Hi one customer reported slow queries over information_schema.tables. There is newer used a index over relname probably due casting to information_schema.sql_identifier. Slow query select * from information_schema.tables where table_name = 'pg_class'; Usually, there is hard to fix it on

Re: [Todo item] Add entry creation timestamp column to pg_stat_replication

2018-12-03 Thread Michael Paquier
On Tue, Dec 04, 2018 at 12:56:25PM +0900, Michael Paquier wrote: > I have been looking at this patch more in-depth, and you missed one > critical thing: hot standby feedback messages also include the timestamp > the client used when sending the message, so if we want to track the > latest time

Re: Tab completion for ALTER INDEX|TABLE ALTER COLUMN SET STATISTICS

2018-12-03 Thread Tatsuro Yamada
At Wed, 28 Nov 2018 14:41:40 +0900, Tatsuro Yamada wrote in <54bd214b-d0d3-8654- * tab_completion_alter_index_set_statistics.patch === There are two problems. You can use these DDL before testing. #create table hoge (a integer, b integer); #create index ind_hoge on hoge (a, (a

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2018-12-03 Thread Peter Geoghegan
On Mon, Dec 3, 2018 at 7:10 PM Peter Geoghegan wrote: > Attached is v9, which does things that way. There are no interesting > changes, though I have set things up so that a later patch in the > series can add "dynamic prefix truncation" -- I do not include any > such patch in v9, though. I'm

RE: idle-in-transaction timeout error does not give a hint

2018-12-03 Thread Ideriha, Takeshi
>From: Tatsuo Ishii [mailto:is...@sraoss.co.jp] >Subject: Re: idle-in-transaction timeout error does not give a hint > >>Alternative HINT message would be something like: >> >>HINT: In a moment you should be able to reconnect to the >> database and restart your transaction.

Re: [Todo item] Add entry creation timestamp column to pg_stat_replication

2018-12-03 Thread Michael Paquier
On Fri, Nov 30, 2018 at 05:54:15PM +0900, Michael Paquier wrote: > Looks pretty to me at quick glance, unfortunately I have not spent much > time on it, particularly testing it. > > + > + reply_time > + timestamp with time zone > + Send time of last message received from WAL >

Re: Index Skip Scan

2018-12-03 Thread Peter Geoghegan
On Wed, Nov 21, 2018 at 12:55 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > Well, no, it's callled with ScanDirectionIsForward(dir). But as far as I > remember from the previous discussions the entire topic of backward scan is > questionable for this patch, so I'll try to invest some time in

Dynamic prefix truncation for nbtree, Lanin & Shasha design issue

2018-12-03 Thread Peter Geoghegan
Attached is a prototype patch for dynamic prefix truncation that applies on top of v9 of the nbtree patch series [1] I've been working on. This results in considerable performance improvements in some cases, since it's (almost!) safe to skip attributes that we know must be redundant/equal based on

Re: [HACKERS] CLUSTER command progress monitor

2018-12-03 Thread Michael Paquier
On Mon, Dec 03, 2018 at 02:17:25PM -0300, Alvaro Herrera wrote: > I think we should mark it as Returned with Feedback then. +1. -- Michael signature.asc Description: PGP signature

Re: error message when subscription target is a partitioned table

2018-12-03 Thread Amit Langote
On 2018/12/04 11:23, Michael Paquier wrote: > On Tue, Dec 04, 2018 at 10:51:40AM +0900, Amit Langote wrote: >> Okay, here is a patch. I didn't find any tests in subscription.sql >> related to unsupported relkinds, so didn't bother adding one for this case >> either. > > Should we care about

Re: [PATCH] Log CSV by default

2018-12-03 Thread Michael Paquier
On Mon, Dec 03, 2018 at 09:20:01PM +0100, Gilles Darold wrote: > pgBadger is able to parse jsonlog > (https://github.com/michaelpq/pg_plugins/tree/master/jsonlog) output in > multi-process mode because it do not use an external library to parse > the json. In this minimalist implementation I

Re: error message when subscription target is a partitioned table

2018-12-03 Thread Michael Paquier
On Tue, Dec 04, 2018 at 10:51:40AM +0900, Amit Langote wrote: > Okay, here is a patch. I didn't find any tests in subscription.sql > related to unsupported relkinds, so didn't bother adding one for this case > either. Should we care about other relkinds as well? -- Michael signature.asc

Re: error message when subscription target is a partitioned table

2018-12-03 Thread Amit Langote
On 2018/12/03 17:51, Magnus Hagander wrote: > On Mon, Dec 3, 2018 at 7:39 AM Tatsuo Ishii wrote: >>> Could we improve the error message that's output when the subscription >>> target relation is a partitioned table? Currently, we get: >>> >>> ERROR: logical replication target relation

GIN predicate locking slows down valgrind isolationtests tremendously

2018-12-03 Thread Andres Freund
Hi, https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink=2018-03-31%2014%3A40%3A02 isolation-check (02:19:36) https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink=2018-03-28%2010%3A40%3A02 isolation-check (00:18:44) As far as I can tell that increase comes laregely from the

don't mark indexes invalid unnecessarily

2018-12-03 Thread Alvaro Herrera
While working on FKs pointing to partitioned tables, I noticed that in PG11 we fail to produce a working dump in the case of a partitioned table that doesn't have partitions. The attached patch fixes that. In doing so, it breaks a test ... and analyzing that, it turns out that the test was

Re: [PATCH v19] GSSAPI encryption support

2018-12-03 Thread Thomas Munro
On Tue, Dec 4, 2018 at 10:20 AM Stephen Frost wrote: > (and I have to wonder- if we want nearly all callers of > WaitLatch/WaitLatchOrSocket to use WL_EXIT_ON_PM_DEATH, maybe we should > make that the default and allow it to be overridden..? ... That is what I proposed. It was Heikki who talked

Re: [PATCH v19] GSSAPI encryption support

2018-12-03 Thread Stephen Frost
Greetings Robbie, * Dmitry Dolgov (9erthali...@gmail.com) wrote: > > On Tue, Oct 2, 2018 at 11:12 PM Robbie Harwood wrote: > > > > Michael Paquier writes: > > > > > On Mon, Aug 06, 2018 at 05:23:28PM -0400, Robbie Harwood wrote: > > >> If you're in a position where you're using Kerberos (or

Re: [PATCH] Log CSV by default

2018-12-03 Thread Gilles Darold
Le 03/12/2018 à 19:25, David Fetter a écrit : > On Mon, Dec 03, 2018 at 07:18:31PM +0100, Gilles Darold wrote: >> Le 30/11/2018 à 19:53, David Fetter a écrit : >>> This makes it much simpler for computers to use the logs while not >>> making it excessively difficult for humans to use them. >> I'm

citelny zajimavy clanek venovany fyzicke replikaci

2018-12-03 Thread Pavel Stehule
Ahoj viz https://www.percona.com/blog/2018/11/30/postgresql-streaming-physical-replication-with-slots/ Pavel

Re: Bug fix for glibc broke freebsd build in REL_11_STABLE

2018-12-03 Thread Tom Lane
Andres Freund writes: > On 2018-12-01 17:28:54 -0500, Tom Lane wrote: >> I guess I wasn't precise enough: I meant add -msse2 if on x86 and >> compiler doesn't take -fexcess-precision=standard. > Hm, I still don't like that: It'd silently bump the minimum required > architecture. Like in the case

Re: [PATCH] Log CSV by default

2018-12-03 Thread David Fetter
On Mon, Dec 03, 2018 at 07:18:31PM +0100, Gilles Darold wrote: > Le 30/11/2018 à 19:53, David Fetter a écrit : > > This makes it much simpler for computers to use the logs while not > > making it excessively difficult for humans to use them. > > I'm also not very comfortable with this change. For

Re: [PATCH] Log CSV by default

2018-12-03 Thread Gilles Darold
Le 30/11/2018 à 19:53, David Fetter a écrit : > This makes it much simpler for computers to use the logs while not > making it excessively difficult for humans to use them. I'm also not very comfortable with this change. For a pgBadger point of view I don't know an existing library to parse csv

Re: [proposal] Add an option for returning SQLSTATE in psql error message

2018-12-03 Thread Pavel Stehule
po 3. 12. 2018 v 18:57 odesílatel didier napsal: > On Mon, Dec 3, 2018 at 5:51 PM Tom Lane wrote: > > > No, it's in libpq, so you'd have to touch that not the server. > libpq, not as bad as the server but nonetheless maybe a bit too much for > this. > Is adding value in library contract? > >

Re: [proposal] Add an option for returning SQLSTATE in psql error message

2018-12-03 Thread didier
On Mon, Dec 3, 2018 at 5:51 PM Tom Lane wrote: > No, it's in libpq, so you'd have to touch that not the server. libpq, not as bad as the server but nonetheless maybe a bit too much for this. Is adding value in library contract? Anyway. attached a POC adding a new value to VERBOSITY (hopefully

Re: [HACKERS] CLUSTER command progress monitor

2018-12-03 Thread Alvaro Herrera
On 2018-Dec-03, Tatsuro Yamada wrote: > > In the meantime I'm moving it to the next CF. > > Thank you for managing the CF and Sorry for the late reply. > I'll rebase it for the next CF and also I'll clear my head because the patch > needs design change to address the feedbacks, I guess.

Re: [proposal] Add an option for returning SQLSTATE in psql error message

2018-12-03 Thread Tom Lane
didier writes: > Yep, name is bad, but I'm not sure about VERBOSITY, isn't it > controlling output from the server not the client? No, it's in libpq, so you'd have to touch that not the server. I agree with Andrew's thought, and would further say that just "\set VERBOSITY sqlstate" would be a

Re: doc - improve description of default privileges

2018-12-03 Thread Tom Lane
Fabien COELHO writes: >>> I feel if we're going to do anything, we should put a unified description >>> of privileges and aclitem-reading into section 5.6, and take that material >>> out of the various places where it lives now. Like the attached, in which >>> I failed to resist the temptation

Re: SQL/JSON: documentation

2018-12-03 Thread Liudmila Mantrova
On 11/29/18 7:34 PM, Dmitry Dolgov wrote: Hi, Any progress on that? It would be nice to have a new version of the documentation, and I would even advocate to put it into the json path patch [1] (especially, since there were already requests for that, and I personally don't see any reason to

Re: [proposal] Add an option for returning SQLSTATE in psql error message

2018-12-03 Thread didier
Yep, name is bad, but I'm not sure about VERBOSITY, isn't it controlling output from the server not the client? You may want to set both VERBOSITY to 'verbose' and ECHO_ERROR to 'none' then in script do SELECT -- no error output \if :ERROR -- do something with LAST_ERROR_MESSAGE On Mon,

Re: [proposal] Add an option for returning SQLSTATE in psql error message

2018-12-03 Thread Pavel Stehule
po 3. 12. 2018 v 16:49 odesílatel Andrew Gierth napsal: > > "didier" == didier writes: > > didier> Attached a POC adding a new variable ECHO_ERROR > didier> \set ECHO_ERROR text|none|psqlstate > > I wouldn't have called it that. Possibly another option to the existing > VERBOSITY

Re: Commitfest 2018-11

2018-12-03 Thread David Steele
On 12/1/18 1:46 PM, Lætitia Avrot wrote: Yeah, I've been annoyed by that too (not that I've not been guilty of it myself).  We should probably tell people not to add themselves as reviewers unless they're actively planning to review soon, because it discourages other people

Re: [proposal] Add an option for returning SQLSTATE in psql error message

2018-12-03 Thread Andrew Gierth
> "didier" == didier writes: didier> Attached a POC adding a new variable ECHO_ERROR didier> \set ECHO_ERROR text|none|psqlstate I wouldn't have called it that. Possibly another option to the existing VERBOSITY variable? \set VERBOSITY sqlstate_only or something of that ilk (it's already

Re: [proposal] Add an option for returning SQLSTATE in psql error message

2018-12-03 Thread didier
Attached a POC adding a new variable ECHO_ERROR \set ECHO_ERROR text|none|psqlstate On Mon, Dec 3, 2018 at 2:47 AM Andrew Gierth wrote: > > > "Tom" == Tom Lane writes: > > Tom> I don't buy that argument. We use psql's normal display in all the > Tom> regular regression tests, and it's not

JSON Merge-Patch

2018-12-03 Thread Charles Leifer
I wanted to request that you all consider implementing a "merge" or "update" function for performing in-place updates of JSON data-structures. The relevant algorithm is described here: https://tools.ietf.org/html/rfc7396 SQLite's json1 extension has an implementation and it's quite useful:

Re: Undo worker and transaction rollback

2018-12-03 Thread Dilip Kumar
On Fri, 30 Nov 2018, 20:42 Dmitry Dolgov <9erthali...@gmail.com wrote: > > On Mon, Nov 5, 2018 at 1:32 PM Dilip Kumar > wrote: > > > > Updated patch, include defect fix from Kuntal posted on [1]. > > > > [1] >

Delay locking partitions during query execution

2018-12-03 Thread David Rowley
Over on [1] I'm proposing to delay locking partitions of a partitioned table that's the target of an INSERT or UPDATE command until we first route a tuple to the partition. Currently, we go and lock all partitions, even if we just insert a single tuple to a single partition. The patch in [1]

Re: Reviving the "Stopping logical replication protocol" patch from Vladimir Gordichuk

2018-12-03 Thread Dave Cramer
Dmitry, Please see attached rebased patches Dave Cramer On Fri, 30 Nov 2018 at 18:52, Dmitry Dolgov <9erthali...@gmail.com> wrote: > >On Sat, Dec 1, 2018 at 12:49 AM Dave Cramer wrote: > > > > Thanks, I have done a preliminary check and it seems pretty > straightforward. > > > > I will clean

Re: postgres_fdw: oddity in costing aggregate pushdown paths

2018-12-03 Thread Etsuro Fujita
(2018/11/30 18:51), Etsuro Fujita wrote: > (2018/11/28 13:38), Etsuro Fujita wrote: >> BTW another thing I noticed is this comment on costing aggregate >> pushdown paths using local statistics in estimate_path_cost_size: >> >>* Also, core does not care about costing HAVING

Re: [HACKERS] CLUSTER command progress monitor

2018-12-03 Thread Tatsuro Yamada
On 2018/11/29 21:20, Dmitry Dolgov wrote: On Fri, Aug 24, 2018 at 7:06 AM Tatsuro Yamada wrote: On 2017/11/22 6:07, Peter Geoghegan wrote: On Mon, Oct 2, 2017 at 6:04 AM, Robert Haas wrote: Progress reporting on sorts seems like a tricky problem to me, as I said before. In most cases, a

Re: make installcheck-world in a clean environment

2018-12-03 Thread Alexander Lakhin
Hello, 01.12.2018 09:12, Alexander Lakhin wrote: > 30.11.2018 23:59, Dmitry Dolgov wrote: >> Hi, >> >> I've noticed that for this patch cfbot show strange error >> >> USE_INSTALLED_ASSETS=1 make all > I've fixed the patch. Rebased the patch once more after d3c09b9b. Best regards, Alexander diff

Re: error message when subscription target is a partitioned table

2018-12-03 Thread Magnus Hagander
On Mon, Dec 3, 2018 at 7:39 AM Tatsuo Ishii wrote: > > Hi, > > > > Could we improve the error message that's output when the subscription > > target relation is a partitioned table? Currently, we get: > > > > ERROR: logical replication target relation "public.foo" is not a table > > > > I