Re: A patch for get origin from commit_ts.

2020-06-29 Thread movead...@highgo.ca
>> A second thing is that TransactionIdGetCommitTsData() was introdued in >> core(73c986add). It has only one caller pg_xact_commit_timestamp() which >> passes RepOriginId as NULL, making last argument to the >> TransactionIdGetCommitTsData() a dead code in core. >> >> Quick code search shows

Re: POC: postgres_fdw insert batching

2020-06-29 Thread Amit Langote
On Tue, Jun 30, 2020 at 1:22 PM Ashutosh Bapat wrote: > On Tue, 30 Jun 2020 at 08:47, Etsuro Fujita wrote: >> On Mon, Jun 29, 2020 at 7:52 PM Ashutosh Bapat >> wrote: >> > On Sun, Jun 28, 2020 at 8:40 PM Tomas Vondra >> > wrote: >> >> > > 3) What about the other DML operations (DELETE/UPDATE)?

Re: track_planning causing performance regression

2020-06-29 Thread Fujii Masao
On 2020/06/29 22:23, Ants Aasma wrote: On Mon, 29 Jun 2020 at 12:17, Julien Rouhaud mailto:rjuju...@gmail.com>> wrote: On Mon, Jun 29, 2020 at 10:55 AM Fujii Masao mailto:masao.fu...@oss.nttdata.com>> wrote: > > On 2020/06/29 16:05, Julien Rouhaud wrote: > > On Mon, Jun

Re: refactoring basebackup.c

2020-06-29 Thread Dipesh Pandit
Hi, I have repeated the experiment with 8K block size and found that the results are not varying much after applying the patch. Please find the details below. *Backup type*: local backup using pg_basebackup *Data size*: Around 200GB (200 tables - each table around 1.05 GB) *TAR_SEND_SIZE value*:

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-06-29 Thread Dilip Kumar
On Tue, Jun 30, 2020 at 9:20 AM Amit Kapila wrote: > > On Mon, Jun 29, 2020 at 4:24 PM Dilip Kumar wrote: > > > > On Mon, Jun 22, 2020 at 4:30 PM Amit Kapila wrote: > > > > > > > > > Other than above tests, can we somehow verify that the invalidations > > > generated at commit time are the same

Re: A patch for get origin from commit_ts.

2020-06-29 Thread Michael Paquier
On Mon, Jun 29, 2020 at 06:17:27PM -0700, Madan Kumar wrote: > We already have pg_xact_commit_timestamp() that returns the timestamp of > the commit. It may be better to have one single function returning both > timestamp and origin for a given transaction ID. > > A second thing is that

Re: truncating timestamps on arbitrary intervals

2020-06-29 Thread John Naylor
In v9, I've simplified the patch somewhat to make it easier for future work to build on. - When truncating on month-or-greater intervals, require the origin to align on month. This removes the need to handle weird corner cases that have no straightforward behavior. - Remove hackish and possibly

Re: Use of "long" in incremental sort code

2020-06-29 Thread David Rowley
On Tue, 30 Jun 2020 at 16:20, Tom Lane wrote: > There is a fairly widespread issue that memory-size-related GUCs and > suchlike variables are limited to represent sizes that fit in a "long". > Although Win64 is the *only* platform where that's an issue, maybe > it's worth doing something about.

Re: POC: postgres_fdw insert batching

2020-06-29 Thread Ashutosh Bapat
On Tue, 30 Jun 2020 at 08:47, Etsuro Fujita wrote: > On Mon, Jun 29, 2020 at 7:52 PM Ashutosh Bapat > wrote: > > On Sun, Jun 28, 2020 at 8:40 PM Tomas Vondra > > wrote: > > > > 3) What about the other DML operations (DELETE/UPDATE)? > > > > > > The other DML operations could probably benefit

Re: Use of "long" in incremental sort code

2020-06-29 Thread Tom Lane
David Rowley writes: > I noticed the incremental sort code makes use of the long datatype a > few times, e.g in TuplesortInstrumentation and > IncrementalSortGroupInfo. (64-bit windows machines have sizeof(long) > == 4). I understand that the values are in kilobytes and it would > take 2TB to

Re: [PATCH] Better cleanup in TLS tests for -13beta2

2020-06-29 Thread Michael Paquier
On Mon, Jun 29, 2020 at 03:51:48PM -0400, Tom Lane wrote: > Daniel Gustafsson writes: >> That being said, we do retain temporary files on such failures on purpose in >> our TestLib since 88802e068017bee8cea7a5502a712794e761c7b5 and a few >> follow-up >> commits since, should these be handled

Re: pgsql: Enable Unix-domain sockets support on Windows

2020-06-29 Thread Amit Kapila
On Mon, Jun 29, 2020 at 8:48 PM Andrew Dunstan wrote: > > > On 6/28/20 4:33 AM, Peter Eisentraut wrote: > > On 2020-06-27 13:57, Amit Kapila wrote: > >> Fair enough, but what should be the behavior in the Windows versions > >> (<10) where Unix-domain sockets are not supported? > > > > You get an

Use of "long" in incremental sort code

2020-06-29 Thread David Rowley
Hi, I noticed the incremental sort code makes use of the long datatype a few times, e.g in TuplesortInstrumentation and IncrementalSortGroupInfo. (64-bit windows machines have sizeof(long) == 4). I understand that the values are in kilobytes and it would take 2TB to cause them to wrap.

Re: Backpatch b61d161c14 (Introduce vacuum errcontext ...)

2020-06-29 Thread Amit Kapila
On Fri, Jun 26, 2020 at 9:19 AM Amit Kapila wrote: > > On Fri, Jun 26, 2020 at 7:25 AM Justin Pryzby wrote: > > > > > > Comments: > > > > > * The index name is saved only during this phase and restored > > > immediately > > > > => I wouldn't say "only" since it's saved during

RE: [Bug fix]There is the case archive_timeout parameter is ignored after recovery works.

2020-06-29 Thread higuchi.dais...@fujitsu.com
>> We don't want change checkpoint interval during recovery, that means >> we cannot cnosider archive_timeout at the fist checkpointer after >> recovery ends. So I think that the suggestion from Fujii-san is the >> direction. >+1 >If this idea has some problems, we can revisit Daisuke-san's idea.

Re: Resetting spilled txn statistics in pg_stat_replication

2020-06-29 Thread Amit Kapila
On Tue, Jun 30, 2020 at 6:38 AM Masahiko Sawada wrote: > > On Mon, 29 Jun 2020 at 20:37, Amit Kapila wrote: > > > > On Mon, Jun 29, 2020 at 10:26 AM Masahiko Sawada > > wrote: > > > > > > I agree that it would not be a common case that the user sets > > > different values for different

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-06-29 Thread Amit Kapila
On Mon, Jun 29, 2020 at 4:24 PM Dilip Kumar wrote: > > On Mon, Jun 22, 2020 at 4:30 PM Amit Kapila wrote: > > > > > > Other than above tests, can we somehow verify that the invalidations > > generated at commit time are the same as what we do with this patch? > > We have verified with individual

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2020-06-29 Thread Masahiko Sawada
On Mon, 22 Jun 2020 at 14:56, Bharath Rupireddy wrote: > > Hi, > > When a query on foreign table is executed from a local session using > postgres_fdw, as expected the local postgres backend opens a > connection which causes a remote session/backend to be opened on the > remote postgres server

Re: POC: postgres_fdw insert batching

2020-06-29 Thread Etsuro Fujita
On Mon, Jun 29, 2020 at 7:52 PM Ashutosh Bapat wrote: > On Sun, Jun 28, 2020 at 8:40 PM Tomas Vondra > wrote: > > 3) What about the other DML operations (DELETE/UPDATE)? > > > > The other DML operations could probably benefit from the batching too. > > INSERT was good enough for a PoC, but

Re: pg_bsd_indent compiles bytecode

2020-06-29 Thread Tom Lane
Andres Freund writes: > It's easy enough to fix by just adding a ifndef PROGRAM around the piece > adding the depency to the .bc files: > ifeq ($(with_llvm), yes) > ifndef PROGRAM > all: $(addsuffix .bc, $(MODULES)) $(patsubst %.o,%.bc, $(OBJS)) > endif # PROGRAM > endif # with_llvm > but it's

Re: pg_bsd_indent compiles bytecode

2020-06-29 Thread Tom Lane
I wrote: > Andres Freund writes: >> The way that pg_bsd_indent defines its variables isn't standard C, as >> far as I can tell, which explains the errors I was getting. All the >> individual files include indent_globs.h, which declares/defines a bunch >> of variables. Since it doesn't use extern,

Re: More efficient RI checks - take 2

2020-06-29 Thread Andres Freund
Hi, I was looking at this patch with Corey during a patch-review session. So these are basically our "combined" comments. On 2020-06-05 17:16:43 +0200, Antonin Houska wrote: > From 6c1cb8ae7fbf0a8122d8c6637c61b9915bc57223 Mon Sep 17 00:00:00 2001 > From: Antonin Houska > Date: Fri, 5 Jun 2020

Re: A patch for get origin from commit_ts.

2020-06-29 Thread Madan Kumar
Hello hackers, We already have pg_xact_commit_timestamp() that returns the timestamp of the commit. It may be better to have one single function returning both timestamp and origin for a given transaction ID. A second thing is that TransactionIdGetCommitTsData() was introdued in core(73c986add).

Re: Resetting spilled txn statistics in pg_stat_replication

2020-06-29 Thread Masahiko Sawada
On Mon, 29 Jun 2020 at 20:37, Amit Kapila wrote: > > On Mon, Jun 29, 2020 at 10:26 AM Masahiko Sawada > wrote: > > > > On Fri, 26 Jun 2020 at 17:53, Amit Kapila wrote: > > > > > > On Fri, Jun 26, 2020 at 11:31 AM Masahiko Sawada > > > wrote: > > > > > > > > On Thu, 25 Jun 2020 at 19:35, Amit

Re: [Bug fix]There is the case archive_timeout parameter is ignored after recovery works.

2020-06-29 Thread Fujii Masao
On 2020/06/30 9:14, Kyotaro Horiguchi wrote: Opps! I misunderstood that. At Mon, 29 Jun 2020 13:00:25 +, "higuchi.dais...@fujitsu.com" wrote in Fujii-san, thank you for comments. The cause of this problem is that the checkpointer's sleep time is calculated >from both

Re: [Bug fix]There is the case archive_timeout parameter is ignored after recovery works.

2020-06-29 Thread Kyotaro Horiguchi
Opps! I misunderstood that. At Mon, 29 Jun 2020 13:00:25 +, "higuchi.dais...@fujitsu.com" wrote in > Fujii-san, thank you for comments. > > >The cause of this problem is that the checkpointer's sleep time is calculated > >from both checkpoint_timeout and archive_timeout during normal

Re: Hybrid Hash/Nested Loop joins and caching results from subplans

2020-06-29 Thread David Rowley
On Fri, 12 Jun 2020 at 16:10, Andy Fan wrote: > I first tried to see if we can have a row estimation before the subplan > is created and it looks very complex. The subplan was created during > preprocess_qual_conditions, at that time, we even didn't create the base > RelOptInfo , to say nothing

Re: Open Item: Should non-text EXPLAIN always show properties?

2020-06-29 Thread David Rowley
On Fri, 26 Jun 2020 at 04:33, Tom Lane wrote: > > Robert Haas writes: > > On Thu, Jun 25, 2020 at 8:42 AM James Coleman wrote: > >> Yesterday I'd replied [1] to Justin's proposal for this WRT > >> incremental sort and expressed my opinion that including both > >> unnecessarily (i.e., including

Re: pgstattuple: Have pgstattuple_approx accept TOAST tables

2020-06-29 Thread Peter Eisentraut
On 2020-06-17 13:39, Laurenz Albe wrote: On Fri, 2020-04-17 at 13:01 +0200, Peter Eisentraut wrote: I alluded to this in [0], but it's better discussed in its own thread. I think the check that makes pgstattuple_approx reject TOAST tables is a mistake. They have visibility and free space map,

Re: track_planning causing performance regression

2020-06-29 Thread Andres Freund
Hi, On 2020-06-29 17:55:28 +0900, Fujii Masao wrote: > One idea to reduce that lock contention is to separate per-query spinlock > into two; one is for planning, and the other is for execution. pgss_store() > determines which lock to use based on the given "kind" argument. > To make this idea

Re: track_planning causing performance regression

2020-06-29 Thread Andres Freund
Hi, On 2020-06-29 09:05:18 +0200, Julien Rouhaud wrote: > I can't reproduce this on my laptop, but I can certainly believe that > running the same 3 queries using more connections than available cores > will lead to extra overhead. > I disagree with the conclusion though. It seems to me that if

Re: track_planning causing performance regression

2020-06-29 Thread Peter Geoghegan
On Mon, Jun 29, 2020 at 3:23 PM Peter Geoghegan wrote: > +1 -- this regression seems unacceptable to me. I added an open item to track this. Thanks -- Peter Geoghegan

Re: track_planning causing performance regression

2020-06-29 Thread Peter Geoghegan
On Mon, Jun 29, 2020 at 1:55 AM Fujii Masao wrote: > > I disagree with the conclusion though. It seems to me that if you > > really have this workload that consists in these few queries and want > > to get better performance, you'll anyway use a connection pooler > > and/or use prepared

Re: Default setting for enable_hashagg_disk

2020-06-29 Thread Peter Geoghegan
On Mon, Jun 29, 2020 at 2:22 PM Tomas Vondra wrote: > Can you give and example of what you mean by being too permissive or too > conservative? Do you mean the possibility of unlimited memory usage in > v12, and strict enforcement in v13? Yes -- that's all I meant. > IMO enforcing the work_mem

Re: Default setting for enable_hashagg_disk

2020-06-29 Thread Tomas Vondra
On Mon, Jun 29, 2020 at 01:31:40PM -0400, Bruce Momjian wrote: On Mon, Jun 29, 2020 at 10:20:14AM -0700, Peter Geoghegan wrote: I have no reason to believe that the planner is any more or any less likely to conclude that the hash table will fit in memory in v13 as things stand (I don't know if

Re: Default setting for enable_hashagg_disk

2020-06-29 Thread Tomas Vondra
On Mon, Jun 29, 2020 at 10:20:14AM -0700, Peter Geoghegan wrote: On Mon, Jun 29, 2020 at 8:07 AM Tomas Vondra wrote: Not sure I follow. Which cases do you mean when you say that 12 could safely do them, but 13 won't? I see the following two cases: a) Planner in 12 and 13 disagree about

Re: Parallell hashjoin sometimes ignores temp_tablespaces

2020-06-29 Thread Daniel Gustafsson
> On 29 Jun 2020, at 17:02, Magnus Hagander wrote: > I think the fix is the attached one (tested on version 11 which is what > $customer is using). To me it looks like this may have been a copy/paste > error all the way back in 98e8b480532 which added default_tablespace back in > 2004. (And

Re: [PATCH] Better cleanup in TLS tests for -13beta2

2020-06-29 Thread Tom Lane
Daniel Gustafsson writes: >> On 29 Jun 2020, at 21:27, Tom Lane wrote: >> Hmm ... so I guess my reaction to both of these is "what guarantees >> that we get to the part of the script that does the unlinks?". > That being said, we do retain temporary files on such failures on purpose in > our

Re: [PATCH] Better cleanup in TLS tests for -13beta2

2020-06-29 Thread Daniel Gustafsson
> On 29 Jun 2020, at 21:27, Tom Lane wrote: > Hmm ... so I guess my reaction to both of these is "what guarantees > that we get to the part of the script that does the unlinks?". > I've certainly seen lots of TAP tests fail to complete. Could we > do the cleanup in an END block or the like?

Re: [PATCH] Better cleanup in TLS tests for -13beta2

2020-06-29 Thread Tom Lane
Daniel Gustafsson writes: > The proposed patch admittedly seems a bit like a hack, and the client_tmo.key > handling is wrong as mentioned above. I propose that we instead add the key > to > the @keys array and have clean up handle all files uniformly. The attached > fixes both of the files.

Re: [PATCH] Better cleanup in TLS tests for -13beta2

2020-06-29 Thread Daniel Gustafsson
> On 29 Jun 2020, at 17:52, Felix Lechner wrote: > This patch removes two temporary files that are not removed. In > Debian, repeated builds fail. We do not allow builds from modified > sources. Aha, nice catch! > The first file was clearly an oversight. It was created separately. I > am not

Re: pg_bsd_indent compiles bytecode

2020-06-29 Thread Tom Lane
Andres Freund writes: > The way that pg_bsd_indent defines its variables isn't standard C, as > far as I can tell, which explains the errors I was getting. All the > individual files include indent_globs.h, which declares/defines a bunch > of variables. Since it doesn't use extern, they'll all

Re: Default setting for enable_hashagg_disk

2020-06-29 Thread Peter Geoghegan
On Mon, Jun 29, 2020 at 8:29 AM Bruce Momjian wrote: > Is this something we want to codify for all node types, > i.e., choose a non-spill node type if we need a lot more than work_mem, > but then let work_mem be a soft limit if we do choose it, e.g., allow > 50% over work_mem in the executor for

Re: Default setting for enable_hashagg_disk

2020-06-29 Thread Bruce Momjian
On Mon, Jun 29, 2020 at 10:20:14AM -0700, Peter Geoghegan wrote: > I have no reason to believe that the planner is any more or any less > likely to conclude that the hash table will fit in memory in v13 as > things stand (I don't know if the BufFile issue matters). > > In general, grouping

Re: Default setting for enable_hashagg_disk

2020-06-29 Thread Peter Geoghegan
On Mon, Jun 29, 2020 at 8:07 AM Tomas Vondra wrote: > Not sure I follow. Which cases do you mean when you say that 12 could > safely do them, but 13 won't? I see the following two cases: > a) Planner in 12 and 13 disagree about whether the hash table will fit > into work_mem. > > I don't quite

Re: pg_bsd_indent compiles bytecode

2020-06-29 Thread Andres Freund
Hi, On 2020-06-27 18:54:04 -0400, Tom Lane wrote: > Having now checked, there isn't any such problem. No .bc files are > getting built except in src/backend and in other modules that feed > into the backend, such as src/timezone and most of contrib. > I do see .bc files getting built for

Re: pg_bsd_indent compiles bytecode

2020-06-29 Thread Andres Freund
Hi, On 2020-06-27 18:43:40 -0400, Tom Lane wrote: > Andres Freund writes: > > I can test that with another program, but for some reason pg_bsd_indent > > fails to build against 13/HEAD, but builds fine against 12. Not sure yet > > what's up: > > Huh. Works here on RHEL8 ... what platform are

Re: [HACKERS] Custom compression methods

2020-06-29 Thread Andres Freund
Hi, On 2020-06-24 07:59:47 -0400, Robert Haas wrote: > On Tue, Jun 23, 2020 at 4:00 PM Andres Freund wrote: > > https://postgr.es/m/20130621000900.GA12425%40alap2.anarazel.de is a > > thread with more information / patches further along. > > > > I confused this patch with the approach in > >

Re: +(pg_lsn, int8) and -(pg_lsn, int8) operators

2020-06-29 Thread Fujii Masao
On 2020/06/30 1:03, Asif Rehman wrote: Hi, The patch looks fine to me, however there is one hunk failing for the test case, so it needs to be rebased. Thanks for the check! Attached is the updated version of the patch. The new status of this patch is: Waiting on Author I will change

Re: +(pg_lsn, int8) and -(pg_lsn, int8) operators

2020-06-29 Thread Asif Rehman
Hi, The patch looks fine to me, however there is one hunk failing for the test case, so it needs to be rebased. The new status of this patch is: Waiting on Author

[PATCH] Better cleanup in TLS tests for -13beta2

2020-06-29 Thread Felix Lechner
Hi, This patch removes two temporary files that are not removed. In Debian, repeated builds fail. We do not allow builds from modified sources. The first file was clearly an oversight. It was created separately. I am not sure why the loop over @keys did not remove the second. For the record,

Re: pgbench and timestamps (bounced)

2020-06-29 Thread Fabien COELHO
[Resent on hackers for CF registration, sorry for the noise] Hello Tom, The attached patch fixes some of the underlying problems reported by delaying the :var to $1 substitution to the last possible moments, so that what variables are actually defined is known. PREPARE-ing is also delayed to

Re: bugfix: invalid bit/varbit input causes the log file to be unreadable

2020-06-29 Thread Tom Lane
Quan Zongliang writes: > I tested it, and it looks fine. Pushed, thanks for reporting the issue! regards, tom lane

Re: Default setting for enable_hashagg_disk

2020-06-29 Thread Bruce Momjian
On Sun, Jun 28, 2020 at 05:40:16PM -0700, Peter Geoghegan wrote: > I think any problem that might come up with the costing is best > thought of as a distinct problem. This thread is mostly about the > problem of users getting fewer in-memory hash aggregates compared to a > previous release running

Re: pgsql: Enable Unix-domain sockets support on Windows

2020-06-29 Thread Andrew Dunstan
On 6/28/20 4:33 AM, Peter Eisentraut wrote: > On 2020-06-27 13:57, Amit Kapila wrote: >> Fair enough, but what should be the behavior in the Windows versions >> (<10) where Unix-domain sockets are not supported? > > You get an error about an unsupported address family, similar to > trying to use

Re: Default setting for enable_hashagg_disk

2020-06-29 Thread Tomas Vondra
On Sun, Jun 28, 2020 at 06:39:40PM -0700, Peter Geoghegan wrote: On Sat, Jun 27, 2020 at 3:41 AM Tomas Vondra wrote: Well, there are multiple ideas discussed in this (sub)thread, one of them being a per-query memory limit. That requires decisions how much memory should different nodes get,

Parallell hashjoin sometimes ignores temp_tablespaces

2020-06-29 Thread Magnus Hagander
If a database (a) has a default tablespace set, Reproduction: CREATE TABLESPACE t LOCATION '/tmp/t'; CREATE DATABASE dumb TABLESPACE t; \c dumb SET temp_tablespaces=t; At this point if you run a query with a parallel hash join in it, the tempfiles go in base/pgsql_tmp instead of the temporary

Re: Fwd: PostgreSQL: WolfSSL support

2020-06-29 Thread Stephen Frost
Greetings, * Christoph Berg (m...@debian.org) wrote: > Re: Peter Eisentraut > > What would be the advantage of using wolfSSL over OpenSSL? > > Avoiding the OpenSSL-vs-GPL linkage problem with readline. I'd further say "folks are interested in an alternative to OpenSSL" as being a generally good

Re: Fwd: PostgreSQL: WolfSSL support

2020-06-29 Thread Stephen Frost
Greetings, * Felix Lechner (felix.lech...@lease-up.com) wrote: > Attached please find a WIP patch for wolfSSL support in postgresql-12. Would really be best to have this off of HEAD if we're going to be looking at it rather than v12. We certainly aren't going to add new support for something

Re: Why forbid "INSERT INTO t () VALUES ();"

2020-06-29 Thread Stephen Frost
Greetings, * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: > On 2020-06-25 18:07, Tom Lane wrote: > >So I'm still not convinced we should do this. "MySQL is incapable > >of conforming to the standard" is a really lousy reason for us to do > >something. > > Conformance to the

Re: track_planning causing performance regression

2020-06-29 Thread Ants Aasma
On Mon, 29 Jun 2020 at 12:17, Julien Rouhaud wrote: > On Mon, Jun 29, 2020 at 10:55 AM Fujii Masao > wrote: > > > > On 2020/06/29 16:05, Julien Rouhaud wrote: > > > On Mon, Jun 29, 2020 at 7:49 AM Tharakan, Robins > wrote: > > >> > > >> During fully-cached SELECT-only test using pgbench,

Re: some more pg_dump refactoring

2020-06-29 Thread Peter Eisentraut
On 2020-06-25 08:58, Fabien COELHO wrote: You changed the query strings to use "\n" instead of " ". I would not have changed that, because it departs from the style around, and I do not think it improves readability at the C code level. This was the style that was introduced by

RE: [Bug fix]There is the case archive_timeout parameter is ignored after recovery works.

2020-06-29 Thread higuchi.dais...@fujitsu.com
Fujii-san, thank you for comments. >The cause of this problem is that the checkpointer's sleep time is calculated >from both checkpoint_timeout and archive_timeout during normal running, >but calculated only from checkpoint_timeout during recovery. So Daisuke-san's >patch tries to change that so

Re: xid wraparound danger due to INDEX_CLEANUP false

2020-06-29 Thread Masahiko Sawada
On Sun, 28 Jun 2020 at 02:44, Peter Geoghegan wrote: > > On Fri, Jun 26, 2020 at 10:15 PM Masahiko Sawada > wrote: > > Regarding to the extent of the impact, this bug will affect the user > > who turned vacuum_index_cleanup off or executed manually vacuum with > > INDEX_CLEANUP off for a long

Re: Allow CURRENT_ROLE in GRANTED BY

2020-06-29 Thread Peter Eisentraut
On 2020-06-24 23:08, Alvaro Herrera wrote: On 2020-Jun-24, Peter Eisentraut wrote: I was checking some loose ends in SQL conformance, when I noticed: We support GRANT role ... GRANTED BY CURRENT_USER, but we don't support CURRENT_ROLE in that place, even though in PostgreSQL they are

RE: [Bug fix]There is the case archive_timeout parameter is ignored after recovery works.

2020-06-29 Thread higuchi.dais...@fujitsu.com
Thank you for comments. >Unfortunately the diff command in your test script doesn't show me >anything, but I can understand what you are thinking is a problem, >maybe. I'm sorry but I might have confused you... I explain how to use my test script. I use diff command to check if the archiver has

Re: Index Skip Scan (new UniqueKeys)

2020-06-29 Thread Dmitry Dolgov
> On Thu, Jun 11, 2020 at 04:14:07PM +0800, Andy Fan wrote: > > I just get the rough idea of patch, looks we have to narrow down the > user cases where we can use this method. Consider the below example: Hi Not exactly narrow down, but rather get rid of wrong usage of skipping for index scan.

Re: track_planning causing performance regression

2020-06-29 Thread Julien Rouhaud
On Mon, Jun 29, 2020 at 1:14 PM Fujii Masao wrote: > > On 2020/06/29 18:56, Fujii Masao wrote: > > > > > > On 2020/06/29 18:53, Julien Rouhaud wrote: > >> On Mon, Jun 29, 2020 at 11:38 AM Fujii Masao > >> wrote: > >>> > > Your benchmark result seems to suggest that the cause of the problem

Re: tar-related code in PostgreSQL

2020-06-29 Thread Robert Haas
On Sun, Jun 28, 2020 at 11:24 AM Hamid Akhtar wrote: > The following review has been posted through the commitfest application: > make installcheck-world: tested, passed > Implements feature: tested, passed > Spec compliant: not tested > Documentation:not tested > >

Re: Resetting spilled txn statistics in pg_stat_replication

2020-06-29 Thread Amit Kapila
On Mon, Jun 29, 2020 at 10:26 AM Masahiko Sawada wrote: > > On Fri, 26 Jun 2020 at 17:53, Amit Kapila wrote: > > > > On Fri, Jun 26, 2020 at 11:31 AM Masahiko Sawada > > wrote: > > > > > > On Thu, 25 Jun 2020 at 19:35, Amit Kapila wrote: > > > > > > > > > > > > Today, I thought about it again,

Re: track_planning causing performance regression

2020-06-29 Thread Fujii Masao
On 2020/06/29 18:56, Fujii Masao wrote: On 2020/06/29 18:53, Julien Rouhaud wrote: On Mon, Jun 29, 2020 at 11:38 AM Fujii Masao wrote: Your benchmark result seems to suggest that the cause of the problem is the contention of per-query spinlock in pgss_store(). Right? This lock

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-06-29 Thread Dilip Kumar
On Mon, Jun 22, 2020 at 4:30 PM Amit Kapila wrote: > > On Mon, Jun 22, 2020 at 4:26 PM Amit Kapila wrote: > > > > On Thu, Jun 18, 2020 at 9:02 PM Dilip Kumar wrote: > > > > > > Yes, I have made the changes. Basically, now I am only using the > > > XLOG_XACT_INVALIDATIONS for generating all the

Re: POC: postgres_fdw insert batching

2020-06-29 Thread Ashutosh Bapat
On Sun, Jun 28, 2020 at 8:40 PM Tomas Vondra wrote: > > FDW batching: 4584 ms > > So, rather nice improvement, I'd say ... Very nice. > > Before I spend more time hacking on this, I have a couple open questions > about the design, restrictions etc. > > > 1) Extend the FDW API? > > In the

Re: bugfix: invalid bit/varbit input causes the log file to be unreadable

2020-06-29 Thread Quan Zongliang
Good. I tested it, and it looks fine. Thank you. On 2020/6/29 1:10 上午, Tom Lane wrote: I wrote: Even granting the premise, the proposed patch seems like a significant decrease in user-friendliness for typical cases. I'd rather see us make an effort to print one valid-per-the-DB-encoding

Re: [Bug fix]There is the case archive_timeout parameter is ignored after recovery works.

2020-06-29 Thread Fujii Masao
On 2020/06/29 16:41, Kyotaro Horiguchi wrote: Hello. At Mon, 29 Jun 2020 04:35:11 +, "higuchi.dais...@fujitsu.com" wrote in Hi, I found the bug about archive_timeout parameter. There is the case archive_timeout parameter is ignored after recovery works. ... [Problem] When the value

Bug with indexes on whole-row expressions

2020-06-29 Thread Laurenz Albe
CREATE TABLE boom (a integer, b integer); -- index on whole-row expression CREATE UNIQUE INDEX ON boom ((boom)); INSERT INTO boom VALUES (1, 2), (1, 3); ALTER TABLE boom DROP b; TABLE boom; a --- 1 1 (2 rows) REINDEX TABLE boom; ERROR: could not create unique index "boom_boom_idx"

Re: Asymmetry in opening and closing indices for partition routing

2020-06-29 Thread Ashutosh Bapat
On Mon, 22 Jun 2020 at 23:22, Alvaro Herrera wrote: > > I'm not sure that expecting the relcache entry's refcount drops to zero > at the right time is a good approach; that may cause leaks some other > place might have refcounts you're not expecting (say, an open cursor > that's not fully read).

Re: estimation problems for DISTINCT ON with FDW

2020-06-29 Thread Bharath Rupireddy
> It doesn't cost out the plan of pushing the DISTINCT ON down to the foreign > side, which is probably the best way to run the query. I guess it makes > sense that FDW machinery in general doesn't want to try to push a PostgreSQL > specific construct. I think you are right, the DISTINCT

Re: update substring pattern matching syntax

2020-06-29 Thread Peter Eisentraut
On 2020-06-28 08:13, Fabien COELHO wrote: v2 patches apply cleanly, compile, global check ok, citext check ok, doc gen ok. No further comments. committed, thanks As I did not find an entry in the CF, so I did nothing about tagging it "ready". Right, I had not registered it yet. -- Peter

Re: track_planning causing performance regression

2020-06-29 Thread Fujii Masao
On 2020/06/29 18:53, Julien Rouhaud wrote: On Mon, Jun 29, 2020 at 11:38 AM Fujii Masao wrote: Your benchmark result seems to suggest that the cause of the problem is the contention of per-query spinlock in pgss_store(). Right? This lock contention is likely to happen when multiple

Re: track_planning causing performance regression

2020-06-29 Thread Julien Rouhaud
On Mon, Jun 29, 2020 at 11:38 AM Fujii Masao wrote: > > >> Your benchmark result seems to suggest that the cause of the problem is > >> the contention of per-query spinlock in pgss_store(). Right? > >> This lock contention is likely to happen when multiple sessions run > >> the same queries. > >>

Re: track_planning causing performance regression

2020-06-29 Thread Fujii Masao
On 2020/06/29 18:17, Julien Rouhaud wrote: On Mon, Jun 29, 2020 at 10:55 AM Fujii Masao wrote: On 2020/06/29 16:05, Julien Rouhaud wrote: On Mon, Jun 29, 2020 at 7:49 AM Tharakan, Robins wrote: During fully-cached SELECT-only test using pgbench, Postgres v13Beta1 shows Thanks for the

Re: track_planning causing performance regression

2020-06-29 Thread Julien Rouhaud
On Mon, Jun 29, 2020 at 10:55 AM Fujii Masao wrote: > > On 2020/06/29 16:05, Julien Rouhaud wrote: > > On Mon, Jun 29, 2020 at 7:49 AM Tharakan, Robins wrote: > >> > >> During fully-cached SELECT-only test using pgbench, Postgres v13Beta1 shows > > Thanks for the benchmark! > > > >> ~45%

Re: ModifyTable overheads in generic plans

2020-06-29 Thread Amit Langote
On Mon, Jun 29, 2020 at 10:39 AM David Rowley wrote: > > On Sat, 27 Jun 2020 at 00:36, Amit Langote wrote: > > 2. ExecCheckRTPerms(): checks permissions of *all* partitions before > > executing the plan tree, but maybe it's okay to check only the ones > > that will be accessed > > I don't think

Re: track_planning causing performance regression

2020-06-29 Thread Fujii Masao
On 2020/06/29 16:05, Julien Rouhaud wrote: Hi, On Mon, Jun 29, 2020 at 7:49 AM Tharakan, Robins wrote: During fully-cached SELECT-only test using pgbench, Postgres v13Beta1 shows Thanks for the benchmark! ~45% performance drop [2] at high DB connection counts (when compared with

Re: Add Information during standby recovery conflicts

2020-06-29 Thread Masahiko Sawada
On Thu, 18 Jun 2020 at 16:28, Drouvot, Bertrand wrote: > > Hi hackers, > > I've attached a patch to add information during standby recovery conflicts. > The motive behind is to be able to get some information: > > On the apply side > On the blocker(s) side > > Motivation: > > When a standby

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2020-06-29 Thread Bharath Rupireddy
Thanks for the responses. > > If this is just going to apply to postgres_fdw why not just have that module > provide a function "disconnect_open_sessions()" or the like that does this > upon user command? I suppose there would be some potential value to having > this be set per-user but that

Re: [Bug fix]There is the case archive_timeout parameter is ignored after recovery works.

2020-06-29 Thread Kyotaro Horiguchi
At Mon, 29 Jun 2020 16:41:11 +0900 (JST), Kyotaro Horiguchi wrote in > Anyway, the attached patch would resolve your problem. I found another issue related to my last patch. For the current master (and older versions) if walreceiver is signaled to exit just after a segment is completed,

Re: TAP tests and symlinks on Windows

2020-06-29 Thread Michael Paquier
On Fri, Jun 26, 2020 at 02:00:37PM +0200, Peter Eisentraut wrote: > We should be more accurate about things like this: > > +# The following tests test symlinks. Windows may not have symlinks, so > +# skip there. > > The issue isn't whether Windows has symlinks, since all versions of Windows >

Re: pg_bsd_indent compiles bytecode

2020-06-29 Thread Michael Paquier
On Sat, Jun 27, 2020 at 06:54:04PM -0400, Tom Lane wrote: > Having now checked, there isn't any such problem. No .bc files are > getting built except in src/backend and in other modules that feed > into the backend, such as src/timezone and most of contrib. > > I do see .bc files getting built

Re: POC and rebased patch for CSN based snapshots

2020-06-29 Thread Andrey V. Lepikhov
On 6/12/20 2:41 PM, movead...@highgo.ca wrote: Hello hackers, Currently, I do some changes based on the last version: 1. Catch up to the current  commit (c2bd1fec32ab54). 2. Add regression and document. 3. Add support to switch from xid-base snapshot to csn-base snapshot, and the same with

Re: [Bug fix]There is the case archive_timeout parameter is ignored after recovery works.

2020-06-29 Thread Kyotaro Horiguchi
Hello. At Mon, 29 Jun 2020 04:35:11 +, "higuchi.dais...@fujitsu.com" wrote in > Hi, > > I found the bug about archive_timeout parameter. > There is the case archive_timeout parameter is ignored after recovery works. ... > [Problem] > When the value of archive_timeout is smaller than that

Re: [PATCH] COPY command's data format option allows only lowercase csv, text or binary

2020-06-29 Thread Michael Paquier
On Thu, Jun 25, 2020 at 11:07:33AM +0900, Michael Paquier wrote: > 0d8c9c1 has introduced some in parse_basebackup_options() for the > new manifest option, and fe30e7e for AlterType(), no? Please forget this one. I had a moment of brain fade. Those have been added for the option values, and on

Re: track_planning causing performance regression

2020-06-29 Thread Julien Rouhaud
Hi, On Mon, Jun 29, 2020 at 7:49 AM Tharakan, Robins wrote: > > During fully-cached SELECT-only test using pgbench, Postgres v13Beta1 shows > ~45% performance drop [2] at high DB connection counts (when compared with > v12.3) > > Disabling pg_stat_statements.track_planning (which is 'On' by

Re: Ought to use heap_multi_insert() for pg_attribute/depend insertions?

2020-06-29 Thread Michael Paquier
On Fri, Jun 26, 2020 at 02:26:50PM +0200, Daniel Gustafsson wrote: > On 26 Jun 2020, at 10:11, Michael Paquier wrote: >> + /* TODO is nreferenced a reasonable allocation of slots? */ >> + slot = palloc(sizeof(TupleTableSlot *) * nreferenced); >> It seems to me that we could just apply the

Re: Creating a function for exposing memory usage of backend process

2020-06-29 Thread Bharath Rupireddy
> > If there is no such way to know the cache sizes and other info such as > > statistics, number of entries, cache misses, hits etc. can the > > approach discussed here be applied? > I think it's partially yes. > > > If the user knows the cache statistics and other information, may be > > we

Re: Creating a function for exposing memory usage of backend process

2020-06-29 Thread Fujii Masao
On 2020/06/29 12:01, torikoshia wrote: On 2020-06-20 03:11, Robert Haas wrote: On Wed, Jun 17, 2020 at 11:56 PM Fujii Masao wrote: > As a first step, to deal with (3) I'd like to add > pg_stat_get_backend_memory_context() which target is limited to the > local backend process. +1 +1

Re: [PATCH] COPY command's data format option allows only lowercase csv, text or binary

2020-06-29 Thread Bharath Rupireddy
> As that shows, there's already a round of lowercasing done by the parser. > The only way that strcasecmp in copy.c would be useful is if you wanted to > accept things like > copy foo from stdin (format "CSV"); > I don't find that to be a terribly good idea. The normal implication > of