Re: Problems with plan estimates in postgres_fdw

2019-03-03 Thread Antonin Houska
Etsuro Fujita wrote: > (2019/03/01 20:00), Antonin Houska wrote: > > Etsuro Fujita wrote: > > It's still unclear to me why add_foreign_ordered_paths() passes the input > > relation (input_rel) to estimate_path_cost_size(). If it passed the output > > rel > > (i.e. ordered_rel in this case)

RE: pgbench - doCustom cleanup

2019-03-03 Thread Jamison, Kirk
Hi Fabien and Alvaro, I found that I have already reviewed this thread before, so I tried to apply the patch, but part of the chunk failed, because of the unused line below which was already removed in the recent related commit. > PGResult*res; I removed the line and fixed the

Re: Online verification of checksums

2019-03-03 Thread Magnus Hagander
On Mon, Mar 4, 2019, 04:10 Michael Paquier wrote: > On Sun, Mar 03, 2019 at 07:58:26AM +0100, Fabien COELHO wrote: > > I agree that having a server function (extension?) to do a full checksum > > verification, possibly bandwidth-controlled, would be a good thing. > However > > it would have side

Re: [WIP] CREATE SUBSCRIPTION with FOR TABLES clause (table filter)

2019-03-03 Thread Michael Paquier
On Sun, Feb 17, 2019 at 03:33:12PM +0500, Andrey Borodin wrote: > I've made some more iterations looking for ideas how to improve the > patch and found non. > Code style, docs, tests, make-check worlds, bit status, everything > seems OK. A little bit of copied code from dblink (there is no >

Re: Online verification of checksums

2019-03-03 Thread Fabien COELHO
Bonjour Michaël, I agree that having a server function (extension?) to do a full checksum verification, possibly bandwidth-controlled, would be a good thing. However it would have side effects, such as interfering deeply with the server page cache, which may or may not be desirable. In what

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2019-03-03 Thread Heikki Linnakangas
Some comments on v13-0002-make-heap-TID-a-tie-breaker-nbtree-index-column.patch below. Mostly about code comments. In general, I think a round of copy-editing the comments, to use simpler language, would do good. The actual code changes look good to me. /* * _bt_findinsertloc() --

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-03-03 Thread Masahiko Sawada
On Sat, Mar 2, 2019 at 6:23 AM Robert Haas wrote: > > On Fri, Mar 1, 2019 at 3:52 PM Haribabu Kommi > wrote: > > The Cybertec proposed patches are doing the encryption at the instance > > level, AFAIK, the current discussion is also trying to reduce the scope of > > the > > encryption to

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-03-03 Thread Masahiko Sawada
On Sat, Mar 2, 2019 at 5:27 AM Robert Haas wrote: > > On Thu, Feb 7, 2019 at 3:28 AM Masahiko Sawada wrote: > > WAL encryption will follow as an additional feature. > > I don't think WAL encryption is an optional feature. You can argue > about whether it's useful to encrypt the disk files in

Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

2019-03-03 Thread Masahiko Sawada
On Sat, Mar 2, 2019 at 4:34 AM Tom Lane wrote: > > Andrew Dunstan writes: > > On 3/1/19 2:14 PM, Tom Lane wrote: > >> Indeed, but I'm not sure that the use-cases are the same. In particular, > >> unless somebody has done some rather impossible magic, it would be > >> disastrous to apply

Re: [bug fix] Produce a crash dump before main() on Windows

2019-03-03 Thread Kyotaro HORIGUCHI
Hello. At Tue, 6 Nov 2018 15:53:37 +1100, Haribabu Kommi wrote in > Thanks for confirmation of that PostgreSQL runs as service. > > Based on the following details, we can decide whether this fix is required > or not. > 1. Starting of Postgres server using pg_ctl without service is of >

Re: Rare SSL failures on eelpout

2019-03-03 Thread Tom Lane
Thomas Munro writes: > With a simple socket test program I can see that if you send a single > packet after the remote end has closed and after it had already read > everything you sent it up to now, you get EPIPE. If there was some > outstanding data from a previous send that it hadn't read yet

Re: Rare SSL failures on eelpout

2019-03-03 Thread Thomas Munro
On Wed, Jan 23, 2019 at 11:23 AM Thomas Munro wrote: > On Wed, Jan 23, 2019 at 4:07 AM Tom Lane wrote: > > The whole thing reminds me of the recent bug #15598: > > > > https://www.postgresql.org/message-id/87k1iy44fd.fsf%40news-spur.riddles.org.uk > > Yeah, if errors get moved to later exchanges

Re: [HACKERS] Removing [Merge]Append nodes which contain a single subpath

2019-03-03 Thread Tom Lane
David Rowley writes: > [ v13-0001-Forgo-generating-single-subpath-Append-and-Merge.patch ] I continue to think that this is the wrong way to go about it, and as proof of concept present the attached, which reproduces all of the regression-test plan changes appearing in v13 --- with a whole lot

Re: pg_dump multi VALUES INSERT

2019-03-03 Thread Michael Paquier
On Sat, Mar 02, 2019 at 08:01:50AM +0100, Fabien COELHO wrote: > About the output: I'd suggest to indent one line per row, something like: > > INSERT INTO foo VALUES > (..., ..., ..., ...), > (..., ..., ..., ...), > (..., ..., ..., ...); > > so as to avoid very very very very very

Re: [HACKERS] WAL logging problem in 9.4.3?

2019-03-03 Thread Kyotaro HORIGUCHI
Rebased. No commit hit this but I fixed one space error. regards. -- Kyotaro Horiguchi NTT Open Source Software Center >From d048aedbee48a1a0d91ae6e009b7a7903f272720 Mon Sep 17 00:00:00 2001 From: Kyotaro Horiguchi Date: Thu, 11 Oct 2018 10:03:21 +0900 Subject: [PATCH 1/4] TAP test for

Re: allow online change primary_conninfo

2019-03-03 Thread Michael Paquier
On Sat, Mar 02, 2019 at 01:49:51PM +0300, Sergei Kornilov wrote: > This might be not the right way, but I can't think of a better way > to not switch to a different method than split of lastSourceFailed > processing and starting new source. Something like refactoring in > first attached patch. I

Re: [HACKERS] EvalPlanQual behaves oddly for FDW queries involving system columns

2019-03-03 Thread Etsuro Fujita
Hi Andres, (2019/03/02 3:57), Andres Freund wrote: FWIW, I pushed the EPQ patch, doing this conversion blindly. It'd be awesome if you'd check that it actually works... I'll start the work later this week. I think I can post an (initial) report on that next week, maybe. Best regards,

Re: Online verification of checksums

2019-03-03 Thread Michael Paquier
On Sun, Mar 03, 2019 at 07:58:26AM +0100, Fabien COELHO wrote: > I agree that having a server function (extension?) to do a full checksum > verification, possibly bandwidth-controlled, would be a good thing. However > it would have side effects, such as interfering deeply with the server page >

Re: Online verification of checksums

2019-03-03 Thread Michael Paquier
On Sun, Mar 03, 2019 at 11:51:48AM +0100, Michael Banck wrote: > I still think that an external checksum verification tool has some > merit, given that basebackup does it and the current offline requirement > is really not useful in practise. I am not going to argue again about the way checksum

RE: Protect syscache from bloating with negative cache entries

2019-03-03 Thread Ideriha, Takeshi
>From: Tsunakawa, Takayuki [mailto:tsunakawa.ta...@jp.fujitsu.com] >> [Size=800, iter=1,000,000] >> Master |15.763 >> Patched|16.262 (+3%) >> >> [Size=32768, iter=1,000,000] >> Master |61.3076 >> Patched|62.9566 (+2%) > >What's the unit, second or millisecond? Millisecond. >Why is the number of

Re: pg_partition_tree crashes for a non-defined relation

2019-03-03 Thread Michael Paquier
On Mon, Mar 04, 2019 at 10:44:10AM +0900, Amit Langote wrote: > Thanks for committing and adding me as an author. Sure. A good portion of the changes suggested on the backend was mainly yours, so that looked right to me. -- Michael signature.asc Description: PGP signature

Re: Temporal Table Proposal

2019-03-03 Thread Paul Jungwirth
On 2/25/19 4:21 AM, Ibrar Ahmed wrote: Great, to hear that you are working on that. Do you think I can help you with this? I did some groundwork to make it possible. I can help in coding/reviewing or even can take lead if you want to. Hi Ibrar, I'd love some help with this! I submitted my

Re: pg_partition_tree crashes for a non-defined relation

2019-03-03 Thread Amit Langote
On 2019/03/02 18:21, Michael Paquier wrote: > On Fri, Mar 01, 2019 at 11:38:20AM -0500, Tom Lane wrote: >> Right, while you'd get zero rows out for a non-partitioned table. >> WFM. > > Exactly. I have committed a patch doing exactly that, and I have > added test cases with a partitioned table

Re: jsonpath

2019-03-03 Thread Andrew Dunstan
On 3/3/19 1:08 PM, Tomas Vondra wrote: > > > jsonb_util.c > > > I see we're now handling NaN values in convertJsonbScalar(). Isn't it > actually a bug that we don't do this already? Or is it not needed for > some reason? > JSON standard numerics don't support NaN, Infinity etc., so

Re: [HACKERS] Block level parallel vacuum

2019-03-03 Thread Masahiko Sawada
On Sat, Mar 2, 2019 at 3:54 AM Robert Haas wrote: > > On Fri, Mar 1, 2019 at 12:19 AM Masahiko Sawada wrote: > > > I wonder if we really want this behavior. Should a setting that > > > controls the degree of parallelism when scanning the table also affect > > > VACUUM? I tend to think that we

Re: POC: converting Lists into arrays

2019-03-03 Thread David Rowley
On Mon, 4 Mar 2019 at 07:29, Tom Lane wrote: > > Andres Freund writes: > > I still regularly see list overhead matter in production workloads. A > > lot of it being memory allocator overhead, which is why I'm concerned > > with a rewrite that doesn't reduce the number of memory allocations. > >

Re: Online verification of checksums

2019-03-03 Thread Michael Paquier
On Sun, Mar 03, 2019 at 03:12:51AM +0100, Tomas Vondra wrote: > You and Andres may be right that trying to verify checksums online > without close interaction with the server is ultimately futile (or at > least overly complex). But I'm not sure those issues (torn pages and > partial reads) are

Re: readdir is incorrectly implemented at Windows

2019-03-03 Thread Michael Paquier
On Fri, Mar 01, 2019 at 04:43:13PM +0900, Michael Paquier wrote: > Thanks for confirming, Konstantin. Let's wait a couple of days to see > if anybody has objections or comments, and I'll try to commit this > patch. Done and backpatched down to 9.4, with Andrew's suggestion from upthread

RE: [PATCH] Fix Proposal - Deadlock Issue in Single User Mode When IO Failure Occurs

2019-03-03 Thread Chengchao Yu
Hi Amit, Greetings! Thank you so much for your previous feedbacks! It seems the old patch is broken with latest master branch. So, I have rebased the patch, now it can be applied to the latest master without conflicts. Btw, since the commitfest was created:

RE: Timeout parameters

2019-03-03 Thread Jamison, Kirk
On Sunday, March 3, 2019 4:09PM (GMT+9), Fabien COELHO wrote: >Basically same thing about the tcp_user_timeout guc v8, especially: > do you have any advice about how I can test the feature, i.e. > trigger a timeout? > >> Patch applies & compiles cleanly. Global check is ok, although there >> are

Fsync-before-close thought experiment

2019-03-03 Thread Thomas Munro
Hello, We corrected our previously held belief that it's safe to retry fsync(). We still haven't dealt with the possibility that some kernels can forget about write-back errors due to inode cache pressure, if there is a time when no process has the file open. To prevent that we need to hold

Re: NOT IN subquery optimization

2019-03-03 Thread David Rowley
On Mon, 4 Mar 2019 at 11:06, Tom Lane wrote: > > David Rowley writes: > > On Mon, 4 Mar 2019 at 04:42, Tom Lane wrote: > >> You absolutely will get errors during btree insertions and searches > >> if a datatype's btree comparison functions ever return NULL (for > >> non-NULL inputs). > > > I

Re: NOT IN subquery optimization

2019-03-03 Thread Tom Lane
David Rowley writes: > On Mon, 4 Mar 2019 at 04:42, Tom Lane wrote: >> You absolutely will get errors during btree insertions and searches >> if a datatype's btree comparison functions ever return NULL (for >> non-NULL inputs). > I understand this is the case if an index happens to be used, but

Re: NOT IN subquery optimization

2019-03-03 Thread David Rowley
On Mon, 4 Mar 2019 at 04:42, Tom Lane wrote: > > David Rowley writes: > > On Sun, 3 Mar 2019 at 17:11, Tom Lane wrote: > >> (At the code level, this is implicit in the fact that the comparison > >> function will be called via FunctionCall2Coll or a sibling, and those > >> all throw an error if

Re: psql show URL with help

2019-03-03 Thread David Fetter
On Sun, Mar 03, 2019 at 09:57:25PM +0100, Magnus Hagander wrote: > On Sun, Mar 3, 2019 at 7:14 PM David Fetter wrote: > > > On Wed, Feb 27, 2019 at 09:14:59AM +0100, Peter Eisentraut wrote: > > > + url = psprintf(" > >

Re: psql show URL with help

2019-03-03 Thread Magnus Hagander
On Sun, Mar 3, 2019 at 7:14 PM David Fetter wrote: > On Wed, Feb 27, 2019 at 09:14:59AM +0100, Peter Eisentraut wrote: > > + url = psprintf(" > https://www.postgresql.org/docs/%s/%s.html;, > > + > strstr(PG_VERSION, "devel") ? "devel" : PG_MAJORVERSION, > > +

Re: psql show URL with help

2019-03-03 Thread Ramanarayana
Hi, Is there any documentation change required for this patch? Cheers Ram 4.0

Re: [HACKERS] Incomplete startup packet errors

2019-03-03 Thread Tom Lane
I wrote: > Andrew Dunstan writes: >> Patch proposed by Christoph Berg is here: >> https://www.postgresql.org/message-id/20190228151336.GB7550%40msg.df7cb.de > Meh. That doesn't silence only the zero-bytes case, and I'm also > rather afraid of the fact that it's changing COMMERROR to something >

Re: [PATCH] kNN for btree

2019-03-03 Thread Anastasia Lubennikova
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed Hi, thank you for your work on this patch. Patch #1 is

Re: POC: converting Lists into arrays

2019-03-03 Thread Tom Lane
Andres Freund writes: > On 2019-03-02 18:11:43 -0500, Tom Lane wrote: >> I wonder what test cases Andres has been looking at that convince >> him that we need a reimplementation of Lists. > My main observation was from when the expression evaluation was using > lists all over. List iteration

Re: psql show URL with help

2019-03-03 Thread David Fetter
On Wed, Feb 27, 2019 at 09:14:59AM +0100, Peter Eisentraut wrote: > + url = > psprintf("https://www.postgresql.org/docs/%s/%s.html;, > + > strstr(PG_VERSION, "devel") ? "devel" : PG_MAJORVERSION, >

Re: jsonpath

2019-03-03 Thread Tomas Vondra
Hi, Here are some initial comments from a review of the 0001 part. I plan to do more testing on a large data set and additional round of review over the next week. FWIW I've passed this through valgrind and the usual battery of regression tests, and there were no issues. I haven't looked at 0002

Re: jsonpath

2019-03-03 Thread Pavel Stehule
so 2. 3. 2019 v 6:15 odesílatel Alexander Korotkov < a.korot...@postgrespro.ru> napsal: > Hi! > > On Fri, Mar 1, 2019 at 3:36 AM Nikita Glukhov > wrote: > > > > Attached 34th version of the patches. > > > > 1. Partial jsonpath support: > >- Fixed copying of jsonb with vars jsonb_path_query()

Re: NOT IN subquery optimization

2019-03-03 Thread Tom Lane
David Rowley writes: > On Sun, 3 Mar 2019 at 17:11, Tom Lane wrote: >> (At the code level, this is implicit in the fact that the comparison >> function will be called via FunctionCall2Coll or a sibling, and those >> all throw an error if the called function returns NULL.) > Ah okay. I can get

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2019-03-03 Thread Heikki Linnakangas
On 26/02/2019 12:31, Peter Geoghegan wrote: On Mon, Jan 28, 2019 at 7:32 AM Heikki Linnakangas wrote: I spent some time first trying to understand the current algorithm, and then rewriting it in a way that I find easier to understand. I came up with the attached. I think it optimizes for the

Re: NOT IN subquery optimization

2019-03-03 Thread David Rowley
On Sun, 3 Mar 2019 at 17:11, Tom Lane wrote: > (At the code level, this is implicit in the fact that the comparison > function will be called via FunctionCall2Coll or a sibling, and those > all throw an error if the called function returns NULL.) > > Now, it doesn't say in so many words that the

Re: Online verification of checksums

2019-03-03 Thread Michael Banck
Hi, Am Samstag, den 02.03.2019, 11:08 -0500 schrieb Stephen Frost:h > * Michael Banck (michael.ba...@credativ.de) wrote: > > Am Freitag, den 01.03.2019, 18:03 -0500 schrieb Robert Haas: > > > On Tue, Sep 18, 2018 at 10:37 AM Michael Banck > > > wrote: > > > > I have added a retry for this as

Re: pgbench - add pseudo-random permutation function

2019-03-03 Thread Fabien COELHO
Indeed, the patch needs a rebase & conflit resolution. I'll do it. Later. Here is an update: - take advantage of pg_bitutils (although I noted that the "slow" popcount there could be speeded-up and shorten with a bitwise operator implementation that I just removed from pgbench). -

Re: libpq environment variables in the server

2019-03-03 Thread Noah Misch
On Mon, Jan 21, 2019 at 11:42:16AM +0100, Peter Eisentraut wrote: > For example, the TAP test infrastructure sets PGAPPNAME to allow > identifying clients in the server log. But this environment variable is > also inherited by temporary servers started with pg_ctl and is then in > turn used by