Re: [HACKERS] CLUSTER command progress monitor

On 2017/11/22 6:07, Peter Geoghegan wrote: On Mon, Oct 2, 2017 at 6:04 AM, Robert Haas wrote: Progress reporting on sorts seems like a tricky problem to me, as I said before. In most cases, a sort is going to involve an initial stage where it reads all the input tuples and writes out

libpq debug log

Hi, I'm going to propose libpq debug log for analysis of queries on the application side. I think that it is useful to determine whether the cause is on the application side or the server side when a slow query occurs. The provided information is "date and time" at which execution of

document that MergeAppend can now prune

Hi. It seems that the following commit missed updating ddl.sgml: commit 5220bb7533f9891b1e071da6461d5c387e8f7b09 Author: Heikki Linnakangas Date: Thu Jul 19 13:49:43 2018 +0300 Expand run-time partition pruning to work with MergeAppend Attached patch fixes that. Thanks, Amit diff --git

Re: Fix help option of contrib/oid2name

On 2018/08/21 12:57, Tatsuro Yamada wrote: On 2018/08/21 12:40, Michael Paquier wrote: On Tue, Aug 21, 2018 at 12:26:15PM +0900, Tatsuro Yamada wrote: BTW, can I add the patch to the Commitfest September? You should. Thanks, I'll do that. I'll send 2 patches in this week, probably. I

Re: Removing useless DISTINCT clauses

On 24 August 2018 at 14:12, Stephen Frost wrote: > This is happening at the same time as some optimizations around GROUP > BY, so either there's something different about what's happening there > and I didn't appreciate it, or does that optimization suffer from a > similar issue? There are two

Re: Pluggable Storage - Andres's take

Hi, On 2018-08-24 11:55:41 +1000, Haribabu Kommi wrote: > On Tue, Aug 21, 2018 at 6:59 PM Andres Freund wrote: > > > On 2018-08-21 16:55:47 +1000, Haribabu Kommi wrote: > > > On Sun, Aug 5, 2018 at 7:48 PM Andres Freund wrote: > > > > I'm currently in the process of rebasing zheap onto the

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

On Thu, Aug 23, 2018 at 10:00:49PM +0900, Etsuro Fujita wrote: > I tried this today, but doing git behind the corporate firewall doesn't > work. I don't know the clear cause of that, so I'll investigate that > tomorrow. You may be able to tweak that by using https as origin point or proper git

Re: C99 compliance for src/port/snprintf.c

On Thu, Aug 16, 2018 at 12:24 PM, Thomas Munro wrote: > FWIW cfbot is using Visual Studio 2010 right now. Appveyor provides > 2008, 2010, 2012, 2013 (AKA 12.0), 2015, 2017, and to test with a > different toolchain you can take the example patch from >

RE: [HACKERS] Proposal to add work_mem option to postgres_fdw module

Hi, Hackers. I updated the patch that I attached the other day. Added release of dynamically allocated memory and modified source according to coding rule. Regards, Noriyoshi Shinoda -- From: Shinoda, Noriyoshi (PN Japan GCS Delivery) Sent: Friday, August 17, 2018 3:07 PM To:

Re: Windows vs C99 (was Re: C99 compliance for src/port/snprintf.c)

On Fri, Aug 24, 2018 at 1:44 PM, Andres Freund wrote: > On 2018-08-23 22:02:26 +0200, Peter Eisentraut wrote: >> On 23/08/2018 03:31, Andres Freund wrote: >> > On 2018-08-22 17:09:05 -0700, Andres Freund wrote: >> >> Attached is a version doing so. >> > >> > Mildly updated version attached. This

Re: Hint to set owner for tablespace directory

On Thu, Aug 23, 2018 at 02:24:25PM +0300, Maksim Milyutin wrote: > I want to add patch that prints hint to set required owner for the > tablespace directory if this is the cause of the problem (*errno == EPERM* > after calling *chmod*). Please do not forget to add this patch to the next commit

Re: Improve behavior of concurrent ANALYZE/VACUUM

On Thu, Aug 23, 2018 at 09:53:57PM +, Bossart, Nathan wrote: > This seems reasonable to me. I think establishing the expected > behavior here is a good idea. Thanks, I have pushed the new test series, and reused it to check the new version of the main patch, which is attached. I have added

Re: Removing useless DISTINCT clauses

Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > * David Rowley (david.row...@2ndquadrant.com) wrote: > >> On 24 August 2018 at 11:34, Stephen Frost wrote: > >>> * David Rowley (david.row...@2ndquadrant.com) wrote: > My personal opinion of only being able to

Re: Removing useless DISTINCT clauses

Stephen Frost writes: > * David Rowley (david.row...@2ndquadrant.com) wrote: >> On 24 August 2018 at 11:34, Stephen Frost wrote: >>> * David Rowley (david.row...@2ndquadrant.com) wrote: My personal opinion of only being able to completely remove the DISTINCT when there's a single item

Re: Pluggable Storage - Andres's take

On Tue, Aug 21, 2018 at 6:59 PM Andres Freund wrote: > On 2018-08-21 16:55:47 +1000, Haribabu Kommi wrote: > > On Sun, Aug 5, 2018 at 7:48 PM Andres Freund wrote: > > > I'm currently in the process of rebasing zheap onto the pluggable > > > storage work. The goal, which seems to work

Re: Windows vs C99 (was Re: C99 compliance for src/port/snprintf.c)

On 2018-08-23 22:02:26 +0200, Peter Eisentraut wrote: > On 23/08/2018 03:31, Andres Freund wrote: > > On 2018-08-22 17:09:05 -0700, Andres Freund wrote: > >> Attached is a version doing so. > > > > Mildly updated version attached. This adds an explanatory commit > > message, removes one stray

Re: [HACKERS] Optional message to user when terminating/cancelling backend

Thomas Munro writes: > On Fri, Aug 24, 2018 at 6:53 AM, Tom Lane wrote: >> I think this is just a timing problem: the signal gets sent, >> but it might or might not get received before the current statement ends. > How about we just wait forever if the function manages to return? > select case

Re: [HACKERS] Re: Improve OR conditions on joined columns (common star schema problem)

On Thu, Aug 23, 2018 at 5:20 PM, Peter Geoghegan wrote: > This patch adds an enhancement that is an example of a broader class > of optimizer enhancement primarily aimed at making star-schema queries > have more efficient plans, by arranging to use several independent > nested loop joins based on

Re: TupleTableSlot abstraction

Hi, On 2018-08-20 19:51:33 +0530, Ashutosh Bapat wrote: > Sorry, forgot about that. Here's the patch set with that addressed. Btw, you attach files as tar.zip, but they're actually gzip compressed... > From 838a463646a048b3dccff95079a514fdc86effb3 Mon Sep 17 00:00:00 2001 > From: Ashutosh

Re: [HACKERS] Optional message to user when terminating/cancelling backend

On Fri, Aug 24, 2018 at 6:53 AM, Tom Lane wrote: > Thomas Munro writes: >> On Wed, Jul 25, 2018 at 7:27 PM, Daniel Gustafsson wrote: >>> Seems the build of the updated patch built and tested Ok. Still have no >>> idea >>> why the previous one didn’t. > >> That problem apparently didn't go

Re: Removing useless DISTINCT clauses

Greetings, * David Rowley (david.row...@2ndquadrant.com) wrote: > On 24 August 2018 at 11:34, Stephen Frost wrote: > > * David Rowley (david.row...@2ndquadrant.com) wrote: > >> My personal opinion of only being able to completely remove the > >> DISTINCT when there's a single item in the rtable

Re: [HACKERS] Re: Improve OR conditions on joined columns (common star schema problem)

On Thu, Aug 23, 2018 at 11:10 AM, Tom Lane wrote: > Rebased up to HEAD, per cfbot nagging. Still no substantive change from > v2. I happened to have the opportunity to talk to Tom about this patch in person. I expressed some very general concerns that are worth repeating publicly. This patch

Re: Removing useless DISTINCT clauses

Greetings, * Jim Finnerty (jfinn...@amazon.com) wrote: > I was thinking about this last night and I realized that putting the new > hasModifiedDistinct flag on the PlannerInfo struct eliminates the need to > deal with the serialization issues, and makes it simpler. > > Here's a new patch (v7)

Re: Removing useless DISTINCT clauses

On 24 August 2018 at 11:34, Stephen Frost wrote: > Greetings, > > * David Rowley (david.row...@2ndquadrant.com) wrote: >> My personal opinion of only being able to completely remove the >> DISTINCT when there's a single item in the rtable (or a single base >> table) is that it's just too poor to

Re: Removing useless DISTINCT clauses

Greetings, * David Rowley (david.row...@2ndquadrant.com) wrote: > My personal opinion of only being able to completely remove the > DISTINCT when there's a single item in the rtable (or a single base > table) is that it's just too poor to bother with. I think such a > solution is best left to

Re: Removing useless DISTINCT clauses

On 24 August 2018 at 03:29, Finnerty, Jim wrote: > Thank you Álvaro. Here's the distinct_opt_v7 patch. Determining if there's just 1 base relation by checking the length the rtable is not a good way. If you want to see why, try creating a primary key on a partitioned table. My personal opinion

Re: Make executor's Range Table an array instead of a List

On 24 August 2018 at 02:26, Amit Langote wrote: > One of the patches I sent last week does the same thing, among a > couple of other things with regard to handling relations in the > executor. On a cursory look at the patch, your take of it looks > better than mine. Will test tomorrow. Here is

Re: Flexible configuration for full-text search

Alexander Korotkov writes: > On Fri, Apr 6, 2018 at 10:52 AM Aleksandr Parfenov > wrote: >> The key point of the patch is to process stopwords the same way as >> others at the level of the PostgreSQL internals and give users an >> instrument to process them in a special way via configurations.

Re: Flexible configuration for full-text search

On Fri, Apr 6, 2018 at 10:52 AM Aleksandr Parfenov wrote: > On Thu, 5 Apr 2018 17:26:10 +0300 > Teodor Sigaev wrote: > > 4) Initial approach suggested to distinguish three state of > > dictionary result: null (unknown word), stopword and usual word. Now > > only two, we lost possibility to catch

Re: Improve behavior of concurrent ANALYZE/VACUUM

On 8/23/18, 12:08 AM, "Michael Paquier" wrote: > Normal regression tests are less costly than isolation tests, so let's > use them as possible. What you attached is covering only a portion of > all the scenarios though, as it is as well interesting to see what > happens if another user owns only

Re: Windows vs C99 (was Re: C99 compliance for src/port/snprintf.c)

On 23/08/2018 03:31, Andres Freund wrote: > On 2018-08-22 17:09:05 -0700, Andres Freund wrote: >> Attached is a version doing so. > > Mildly updated version attached. This adds an explanatory commit > message, removes one stray docs C89 reference, and fixes a mis-squashing > of a patch. Let's do

Re: JIT explain output

Hi, On 2018-08-23 21:10:19 +0200, Peter Eisentraut wrote: > Some minor comments on the JIT explain output. For example: > Seq Scan on tenk1 (cost=0.00..483.00 rows= width=244) (actual > time=5.564..8.755 rows=4960 loops=1) >Filter: (unique1 > unique2) >Rows Removed by Filter: 5040

JIT explain output

Some minor comments on the JIT explain output. For example: Seq Scan on tenk1 (cost=0.00..483.00 rows= width=244) (actual time=5.564..8.755 rows=4960 loops=1) Filter: (unique1 > unique2) Rows Removed by Filter: 5040 Planning Time: 0.857 ms JIT: Functions: 2 Generation Time:

Re: A really subtle lexer bug

Andrew Gierth writes: > Here's what I will push unless there's something important I missed. Stylistic nitpick: I don't especially like "continue" as the body of a for-loop. How about instead of this: for (nchars--; nchars > 1 &&

Re: [HACKERS] Optional message to user when terminating/cancelling backend

Thomas Munro writes: > On Wed, Jul 25, 2018 at 7:27 PM, Daniel Gustafsson wrote: >> Seems the build of the updated patch built and tested Ok. Still have no idea >> why the previous one didn’t. > That problem apparently didn't go away. cfbot tested it 7 times in > the past week, and it passed

Re: A really subtle lexer bug

Here's what I will push unless there's something important I missed. I split the regression tests between create_operator.sql and polymorphism.sql, because => is really "named argument syntax" rather than an operator as such, and polymorphism.sql is where the existing tests were for that. I

Re: [HACKERS] Re: Improve OR conditions on joined columns (common star schema problem)

I wrote: > [ join-or-to-union-4.patch ] Rebased up to HEAD, per cfbot nagging. Still no substantive change from v2. regards, tom lane diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c index 6269f47..8935503 100644 ***

Re: A really subtle lexer bug

Andrew Gierth writes: > "Tom" == Tom Lane writes: > Tom> * Some regression tests exercising these code paths might be a > Tom> good thing. > Agreed. Any preferences where they should go? There's not really a "lexer/grammar" test script. Maybe you could drop it into create_operator.sql,

Re: Getting NOT NULL constraint from pg_attribute

Andres Freund writes: > And arguably there's a fair bit of redundancy in pg_attribute, just > because it's convenient for tupledescs. Given that pg_attribute very > commonly is the largest catalog table by far, that very well could use > some attention. Without tupdescs in mind, there's really

Re: proposal: schema private functions

2018-08-23 17:18 GMT+02:00 Nico Williams : > Couldn't this be handled by having a new permission on FUNCTIONs > ("CALL"?) to distinguish EXECUTE? > I don't understand to this questions. Private functions will be functions still - there is not necessity to call these functions differently. >

Re: Memory leak with CALL to Procedure with COMMIT.

> On Aug 23, 2018, at 9:34 AM, Peter Eisentraut > wrote: > > I think I've found a reasonable fix for this. > > The problem arises with the combination of CALL with output parameters > and doing a COMMIT inside the procedure. When a CALL has output > parameters, the portal uses the strategy

Re: A really subtle lexer bug

> "Tom" == Tom Lane writes: >> Patch attached. >> This fixes two bugs: first the mis-lexing of two-char ops as mentioned >> originally; second, the O(N^3) lexing time of strings of - or + >> characters is reduced to O(N^2) (in practice it's better than O(N^2) >> once N gets large

Re: proposal: schema private functions

Couldn't this be handled by having a new permission on FUNCTIONs ("CALL"?) to distinguish EXECUTE? This would have to be made backwards-compatible, possibly by automatically granting CALL ON ALL FUNCTIONS to public at schema create time and/or PG upgrade time, which the schema owner could then

Re: Getting NOT NULL constraint from pg_attribute

On 2018-08-23 11:04:30 -0400, Tom Lane wrote: > "David G. Johnston" writes: > > On Monday, August 20, 2018, Wu Ivy wrote: > >> Why are SELECT query never marked nullable? > > > Basically the nullability property is used by the planner for optimization > > during the joining of physical tables.

Re: Getting NOT NULL constraint from pg_attribute

"David G. Johnston" writes: > On Monday, August 20, 2018, Wu Ivy wrote: >> Why are SELECT query never marked nullable? > Basically the nullability property is used by the planner for optimization > during the joining of physical tables. As soon as you try outputting > columns the ability to

Re: A really subtle lexer bug

Andrew Gierth writes: > "Andrew" == Andrew Gierth writes: > Andrew> I guess the fix is to extend the existing special case code > Andrew> that checks for one character left after removing trailing [+-] > Andrew> and also check for the two-character ops "<>" ">=" "<=" "=>" > Andrew> "!=". >

Re: Removing useless DISTINCT clauses

On 2018-Aug-23, Jim Finnerty wrote: > distinct_opt_v7.patch > Are you posting this via postgresql-archive.org? Please don't do that. These posts of yours are mangled by postgresql-archive and do not contain the file you're

Re: Make executor's Range Table an array instead of a List

(sorry David for the same email twice, I didn't hit Reply All at first) On Thu, Aug 23, 2018 at 7:58 PM, David Rowley wrote: > Continuing along on my adventures of making performance improvements > for partitioned tables, I discovered that getrelid is quite slow in > InitPlan() during UPDATEs

Re: Conflict handling for COPY FROM

Hello, The attached patch add error handling for Extra data missing data invalid oid null oid and row count mismatch And the record that field on the above case write to the file with appended error message in it and in case of unique violation or exclusion constraint violation error the

Re: Memory leak with CALL to Procedure with COMMIT.

I think I've found a reasonable fix for this. The problem arises with the combination of CALL with output parameters and doing a COMMIT inside the procedure. When a CALL has output parameters, the portal uses the strategy PORTAL_UTIL_SELECT instead of PORTAL_MULTI_QUERY. Using

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

(2018/08/22 20:08), Etsuro Fujita wrote: (2018/08/16 12:00), Etsuro Fujita wrote: (2018/08/15 23:40), Robert Haas wrote: Given the comments from the RMT, and also on general principle, it seems like we really need to get on with committing something here. It's my understanding you plan to do

Re: [HACKERS] proposal: schema variables

On 23.08.2018 12:46, Fabien COELHO wrote: I do not understand your point, as usual. I raise a factual issue about security, and you do not answer how this can be solved with your proposal, but appeal to argument of authority and declare your "strong opinion". I do not see any intrinsic

Hint to set owner for tablespace directory

Hi! I have noticed the novice users are stuck trying to create tablespace over a directory whose owner is not the system postgres user. They observed the message "could not set permissions on directory ...: permission denied". I want to add patch that prints hint to set required owner for

Make executor's Range Table an array instead of a List

Continuing along on my adventures of making performance improvements for partitioned tables, I discovered that getrelid is quite slow in InitPlan() during UPDATEs and DELETEs when there are many resultRelations to process. A List is a pretty poor choice for this data structure since we already

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

> On Fri, 27 Jul 2018 at 20:13, Robert Haas wrote: > > On Fri, Jul 27, 2018 at 3:17 AM, Ashutosh Bapat > wrote: > > Apart from the complexity there's also a possibility that this > > skipping will reduce the efficiency actually in normal cases. Consider > > a case where A and B have exactly

Re: [HACKERS] proposal: schema variables

Security vs "good enough in some cases" looks bad to me. We don't find a agreement, because you are concentrated on transation, me on session. And we have different expectations. I do not understand your point, as usual. I raise a factual issue about security, and you do not answer how

Re: [HACKERS] proposal: schema variables

2018-08-23 10:17 GMT+02:00 Fabien COELHO : > > Hello Pavel, > > 2. holding some session based informations, that can be used in security >> definer functions. >> > > Hmmm, I see our disagreement. My point is that this feature is *NOT* fit > for security-related uses because if the transaction

Re: Adding TCP_USER_TIMEOUT support for libpq/psqlodbc

Hello , Did you take a look into the patches from the previous mail ? What do you think about my proposal ? Could you please say if there are any objections in regard to it? Thank you in advance, Andrei Yahorau From: Andrei Yahorau/IBA To: pgsql-hack...@postgresql.org, Cc:

Re: csv format for psql

Bonjour Daniel, '\n' gets translated by libc when the output is in text mode. We discussed this upthread, but maybe it should be a code comment: added now. My point was more about the documentation which should be clear about what is the EOL. I understand from your point above that the EOL

Re: [HACKERS] proposal: schema variables

Hello Pavel, 2. holding some session based informations, that can be used in security definer functions. Hmmm, I see our disagreement. My point is that this feature is *NOT* fit for security-related uses because if the transaction fails the variable would keep the value it had if the

Re: [PATCH] Add regress test for pg_read_all_stats role

On Tue, Aug 21, 2018 at 05:48:49PM +0100, Alexandra Ryzhevich wrote: > Just to check if changes broke something. I haven't find these checks in > other regress tests. In other way we get only positive tests. If this > is not needed then should I remove all the checks for > regress_role_nopriv role

Re: Query is over 2x slower with jit=on

På torsdag 23. august 2018 kl. 03:00:42, skrev Jonathan S. Katz < jk...@postgresql.org >: > On Aug 22, 2018, at 7:13 PM, Andres Freund wrote: [snip] > For the archives sake: This likely largely is the consequence of > building with LLVM's expensive assertions

Re: proposal: schema private functions

2018-08-14 14:00 GMT+02:00 Pavel Stehule : > > > 2018-08-14 13:56 GMT+02:00 Pavel Stehule : > >> Hi >> >> I would to introduce new flag for routines - PRIVATE. Routines with this >> flag can be called only from other routines assigned with same schema. >> Because these routines are not available