Re: progress report for ANALYZE

2019-07-21 Thread Tatsuro Yamada
Hi Horiguchi-san! On 2019/07/11 19:56, Kyotaro Horiguchi wrote: Hello. At Tue, 9 Jul 2019 17:38:44 +0900, Tatsuro Yamada wrote in <244cb241-168b-d6a9-c45f-a80c34cdc...@nttcom.co.jp> Hi Alvaro, Anthony, Julien and Robert, On 2019/07/09 3:47, Julien Rouhaud wrote: On Mon, Jul 8, 2019 at 8:4

Re: Index Skip Scan

2019-07-21 Thread David Rowley
On Wed, 17 Jul 2019 at 04:53, Jesper Pedersen wrote: > Here is a patch more in that direction. Thanks. I've just had a look over this and it roughly what I have in mind. Here are the comments I noted down during the review: cost_index: I know you've not finished here, but I think it'll need to

Re: Fix typos and inconsistencies for HEAD (take 7)

2019-07-21 Thread Alexander Lakhin
Hello Tom, 22.07.2019 7:14, Tom Lane wrote: >> Fixing both places sounds adapted to me. An alternative we could use >> here is just to say something like that: >> The effective resolution is only 1/HZ, which can be configured with >> kernel parameter (see man 7 time), and is 4 milliseconds by >> d

Re: POC: converting Lists into arrays

2019-07-21 Thread Tom Lane
David Rowley writes: > On Mon, 22 Jul 2019 at 16:37, Tom Lane wrote: >> Interesting. I wonder if bms_next_member() could be made any quicker? > I had a quick look earlier and the only thing I saw was maybe to do > the first loop differently from subsequent ones. The "w &= mask;" > does nothing

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2019-07-21 Thread David Rowley
On Mon, 22 Jul 2019 at 16:36, Tsunakawa, Takayuki wrote: > > From: David Rowley [mailto:david.row...@2ndquadrant.com] > > For the use case we've been measuring with partitioned tables and the > > generic plan generation causing a sudden spike in the number of > > obtained locks, then having plan_c

Re: POC: converting Lists into arrays

2019-07-21 Thread David Rowley
On Mon, 22 Jul 2019 at 16:37, Tom Lane wrote: > > David Rowley writes: > > So the bms_next_member() loop is slower when the bitmapset is fully > > populated with all subplans, but way faster when there's just 1 > > member. > > Interesting. I wonder if bms_next_member() could be made any quicker?

Re: POC: converting Lists into arrays

2019-07-21 Thread Tom Lane
David Rowley writes: > On Mon, 22 Jul 2019 at 02:45, Tom Lane wrote: >> One small question is whether it loses if most of the subplans >> are present in the bitmap. I imagine that would be close enough >> to break-even, but it might be worth trying to test to be sure. > ... > -- Test 2 (all mat

RE: Speed up transaction completion faster after many relations are accessed in a transaction

2019-07-21 Thread Tsunakawa, Takayuki
From: David Rowley [mailto:david.row...@2ndquadrant.com] > For the use case we've been measuring with partitioned tables and the > generic plan generation causing a sudden spike in the number of > obtained locks, then having plan_cache_mode = force_custom_plan will > cause the lock table not to bec

Re: Fix typos and inconsistencies for HEAD (take 7)

2019-07-21 Thread Alexander Lakhin
Hello Michael, 22.07.2019 4:05, Michael Paquier wrote: >> Also, I found e-mail headers in optimizer/plan/README not relevant, so I >> propose to remove them. > Not sure about that part. I agree that the proposed fix is not complete, but just raises the demand for a subsequent fix. If you don't mind

Re: Fix typos and inconsistencies for HEAD (take 7)

2019-07-21 Thread Tom Lane
Michael Paquier writes: > On Sun, Jul 21, 2019 at 08:28:53AM +0300, Alexander Lakhin wrote: >> And another finding is related to the sleep effective resolution. `man 7 >> time` says "Since kernel 2.6.13, the HZ value is a kernel configuration  >> parameter  and  can  be 100, 250 (the default) ..."

Re: Add parallelism and glibc dependent only options to reindexdb

2019-07-21 Thread Michael Paquier
On Fri, Jul 19, 2019 at 08:29:27AM +0200, Julien Rouhaud wrote: > On Fri, Jul 19, 2019 at 2:35 AM Michael Paquier wrote: >> For the second patch, could you send a rebase with a fix for the >> connection slot when processing the reindex commands? > > Attached, I also hopefully removed all the now

Re: Psql patch to show access methods info

2019-07-21 Thread Alvaro Herrera
On 2019-Jul-21, Alexander Korotkov wrote: > I've one note. Behavior of "\dA" and "\dA pattern" look > counter-intuitive to me. I would rather expect that "\dA pattern" > would just filter results of "\dA", but it displays different > information. I suggest rename displaying access method proper

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2019-07-21 Thread David Rowley
On Mon, 22 Jul 2019 at 14:21, Tsunakawa, Takayuki wrote: > > From: David Rowley [mailto:david.row...@2ndquadrant.com] > > I personally don't think that's true. The only way you'll notice the > > LockReleaseAll() overhead is to execute very fast queries with a > > bloated lock table. It's pretty

[Patch] PQconnectPoll() is blocked if target_session_attrs is read-write

2019-07-21 Thread Matsumura, Ryo
Hi # I rewrote my previous mail. PQconnectPoll() is used as method for asynchronous using externally or internally. If a caller confirms a socket ready for writing or reading that is requested by return value of previous PQconnectPoll(), next PQconnectPoll() must not be blocked. But if the calle

RE: Speed up transaction completion faster after many relations are accessed in a transaction

2019-07-21 Thread Tsunakawa, Takayuki
From: David Rowley [mailto:david.row...@2ndquadrant.com] > I personally don't think that's true. The only way you'll notice the > LockReleaseAll() overhead is to execute very fast queries with a > bloated lock table. It's pretty hard to notice that a single 0.1ms > query is slow. You'll need to e

Re: pg_receivewal documentation

2019-07-21 Thread Michael Paquier
On Fri, Jul 19, 2019 at 02:04:03PM -0400, Robert Haas wrote: > You could just say something like: > > Since pg_receivewal does not apply WAL, you should not allow it to > become a synchronous standby when synchronous_commit = remote_apply. > If it does, it will appear to be a standby which never c

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2019-07-21 Thread David Rowley
On Mon, 22 Jul 2019 at 12:48, Tsunakawa, Takayuki wrote: > > From: David Rowley [mailto:david.row...@2ndquadrant.com] > > I went back to the drawing board on this and I've added some code that > > counts > > the number of times we've seen the table to be oversized and just shrinks > > the table b

Re: POC: converting Lists into arrays

2019-07-21 Thread David Rowley
On Mon, 22 Jul 2019 at 02:45, Tom Lane wrote: > One small question is whether it loses if most of the subplans > are present in the bitmap. I imagine that would be close enough > to break-even, but it might be worth trying to test to be sure. > (I'd think about breaking out just the loops in ques

Re: Fix typos and inconsistencies for HEAD (take 7)

2019-07-21 Thread Michael Paquier
On Sun, Jul 21, 2019 at 08:28:53AM +0300, Alexander Lakhin wrote: > Please consider fixing the next pack of typos and inconsistencies in the > tree: Thanks, all those things look fine. I have noticed one mistake. > 7.44. json_plperl -> jsonb_plperlu The path was incorrect here. > Also, I found

RE: Speed up transaction completion faster after many relations are accessed in a transaction

2019-07-21 Thread Tsunakawa, Takayuki
From: David Rowley [mailto:david.row...@2ndquadrant.com] > I went back to the drawing board on this and I've added some code that counts > the number of times we've seen the table to be oversized and just shrinks > the table back down on the 1000th time. 6.93% / 1000 is not all that much. I'm afr

Re: new function for tsquery creartion

2019-07-21 Thread Where is Where
Hello everyone, I am wondering if AROUND(N) or is still possible? I found this thread below and the original post https://www.postgresql.org/message-id/fe93ff7e9ad79196486ada79e268%40postgrespro.ru mentioned the proposed feature: 'New operator AROUND(N). It matches if the distance between word

Re: Performance issue in foreign-key-aware join estimation

2019-07-21 Thread David Rowley
On Mon, 22 Jul 2019 at 01:50, David Rowley wrote: > > On Mon, 22 Jul 2019 at 00:44, Andreas Seltenreich wrote: > > sqlsmith triggers an assertion in this commit (3373c7155350): > > > > TRAP: FailedAssertion("!(rel->reloptkind == RELOPT_BASEREL)", File: > > "equivclass.c", Line: 764) > > Thanks f

Re: The flinfo->fn_extra question, from me this time.

2019-07-21 Thread Tom Lane
Chapman Flack writes: > Until now, I had assumed that SFRM_ValuePerCall mode might offer some > benefits, such as the possibility of pipelining certain queries and not > building up a whole tuplestore in advance. > But looking in the code, I'm getting the impression that those > benefits are only

Re: The flinfo->fn_extra question, from me this time.

2019-07-21 Thread Chapman Flack
On 06/15/19 21:46, Chapman Flack wrote: > On 06/15/19 21:21, Tom Lane wrote: >> Yup. (Of course, you don't have to use the SRF_FIRSTCALL_INIT >> infrastructure.) > > That had crossed my mind ... but it seems there's around 80 or 100 > lines of good stuff there that'd be a shame to duplicate. If o

Re: POC: converting Lists into arrays

2019-07-21 Thread Tom Lane
I wrote: >> * Rationalize places that are using combinations of list_copy and >> list_concat, probably by inventing an additional list-concatenation >> primitive that modifies neither input. > I poked around to see what we have in this department. There seem to > be several identifiable use-cases

Re: Bad canonicalization for dateranges with 'infinity' bounds

2019-07-21 Thread Stephen Frost
Greetings, * Jeff Davis (pg...@j-davis.com) wrote: > On Thu, 2019-07-18 at 17:36 -0400, Tom Lane wrote: > > (The commit message doesn't seem to have made it to the pgsql- > > committers > > list either, but that's probably an independent issue.) > > I was curious about that as well. The whitelis

Queries on QMF to POSTGRE

2019-07-21 Thread JVM .
HI Team, I’m looking to convert QMF Queries , QMF forms and QMF procedure to the POSTGRESQL will it support all of them. If yes please help us with the sample example. Or any Documentation. Thanking in anticipation . Regards Jotiram More

Re: Fix typos and inconsistencies for HEAD (take 7)

2019-07-21 Thread Tom Lane
Alexander Lakhin writes: > Also, I found e-mail headers in optimizer/plan/README not relevant, so I > propose to remove them. FWIW, I think they're highly relevant, because they put a date on that text. I've not gone through that README lately, but I wouldn't be surprised if it's largely obsolet

Re: POC: converting Lists into arrays

2019-07-21 Thread Tom Lane
David Rowley writes: > On Wed, 17 Jul 2019 at 11:06, Tom Lane wrote: >> (Actually, I doubt that any of these changes will really move the >> performance needle in the real world. It's more a case of wanting >> the code to present good examples not bad ones.) > In spirit with the above, I'd quit

Re: Performance issue in foreign-key-aware join estimation

2019-07-21 Thread David Rowley
On Mon, 22 Jul 2019 at 00:44, Andreas Seltenreich wrote: > sqlsmith triggers an assertion in this commit (3373c7155350): > > TRAP: FailedAssertion("!(rel->reloptkind == RELOPT_BASEREL)", File: > "equivclass.c", Line: 764) Thanks for the report. It looks like this is caused by the join removal c

Re: Compiler warnings with MinGW

2019-07-21 Thread Michael Paquier
On Sat, Jul 20, 2019 at 06:19:34PM +0900, Michael Paquier wrote: > Just wanted to double-check something. We usually don't bother > back-patching warning fixes like this one, right? I have double-checked the thing, and applied it only on HEAD as we have that for some time (since 9.1 actually and

Re: POC: converting Lists into arrays

2019-07-21 Thread David Rowley
On Wed, 17 Jul 2019 at 11:06, Tom Lane wrote: > (Actually, I doubt that any of these changes will really move the > performance needle in the real world. It's more a case of wanting > the code to present good examples not bad ones.) In spirit with the above, I'd quite like to fix a small bad exa

Re: Compile from source using latest Microsoft Windows SDK

2019-07-21 Thread Andrew Dunstan
On 7/19/19 9:10 PM, Michael Paquier wrote: > On Fri, Jul 19, 2019 at 08:30:38AM -0400, Andrew Dunstan wrote: >> My tests of the VS2017 stuff used this install mechanism on a fresh >> Windows instance: >> >> choco install -y visualstudio2017-workload-vctools --package-parameters >> "--includeOptio

Re: Psql patch to show access methods info

2019-07-21 Thread Alexander Korotkov
On Mon, Jul 15, 2019 at 10:05 PM Nikita Glukhov wrote: > On 01.07.2019 14:06, Thomas Munro wrote: > > > On Sun, Mar 31, 2019 at 2:13 PM wrote: > >> Thanks for review. > > Hi Sergey, > > > > A new Commitfest is here and this doesn't apply -- could you please > > post a rebase? > > > > Thanks, > >

Re: Performance issue in foreign-key-aware join estimation

2019-07-21 Thread Andreas Seltenreich
David Rowley writes: > On Thu, 18 Jul 2019 at 19:24, David Rowley > wrote: >> Unless there's some objection, I'll be looking into pushing both 0001 >> and 0002 in a single commit in the next few days. > > I've pushed this after doing a bit of final tweaking. sqlsmith triggers an assertion in th

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2019-07-21 Thread Tomas Vondra
On Sat, Jul 20, 2019 at 07:37:08PM -0400, James Coleman wrote: .. Yes, you're right - an extra sort node would be necessary. But I don't think that changes the idea behind that example. The question is whether the extra sorts below the gather would be cheaper than doing a large sort on top of it

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2019-07-21 Thread David Rowley
On Thu, 18 Jul 2019 at 14:53, David Rowley wrote: > Is anyone particularly concerned about the worst-case slowdown here > being about 1.54%? The best case, and arguably a more realistic case > above showed a 34% speedup for the best case. I took a bit more time to test the performance on this. I