Re: Skipping logical replication transactions on subscriber side

2021-07-06 Thread Masahiko Sawada
On Mon, Jul 5, 2021 at 6:46 PM Amit Kapila wrote: > > On Thu, Jul 1, 2021 at 6:31 PM Masahiko Sawada wrote: > > > > On Thu, Jul 1, 2021 at 12:56 PM Amit Kapila wrote: > > > > > > > > > Don't we want to clear stats at drop subscription as well? We do drop > > > database stats in dropdb via pgstat

Re: Can a child process detect postmaster death when in pg_usleep?

2021-07-06 Thread Bharath Rupireddy
On Tue, Jul 6, 2021 at 6:15 AM Michael Paquier wrote: > On Mon, Jul 05, 2021 at 09:42:29PM +0530, Bharath Rupireddy wrote: > > I agree. I'm attaching the patch that replaces pg_usleep with > > WaitLatch for {pre, post}_auth_delay. I'm also attaching Michael's > > latest patch stop-backup-latch-v2.

Re: rand48 replacement

2021-07-06 Thread Yura Sokolov
Fabien COELHO писал 2021-07-06 09:13: Hello Yura, I believe most "range" values are small, much smaller than UINT32_MAX. In this case, according to [1] fastest method is Lemire's one (I'd take original version from [2]) [...] Yep. I share your point that the range is more often 32 bits. H

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2021-07-06 Thread David Rowley
On Mon, 5 Jul 2021 at 18:38, Ronan Dunklau wrote: > I think the overhead occurs because in the ExecAgg case, we use the > tuplesort_*_datum API as an optimization when we have a single column as an > input, which the ExecSort code doesn't. Maybe it would be worth it to try to > use that API in sor

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2021-07-06 Thread Yugo NAGATA
Hello Ishii-san, On Fri, 02 Jul 2021 09:25:03 +0900 (JST) Tatsuo Ishii wrote: > I have found an interesting result from patched pgbench (I have set > the isolation level to REPEATABLE READ): > > $ pgbench -p 11000 -c 10 -T 30 --max-tries=0 test > pgbench (15devel, server 13.3) > starting vacu

Re: ECPG bug fix: DECALRE STATEMENT and DEALLOCATE, DESCRIBE

2021-07-06 Thread Michael Paquier
On Fri, Jul 02, 2021 at 12:53:02PM +, kuroda.hay...@fujitsu.com wrote: > I added such a message and some tests, but I began to think this is strange. > Now I'm wondering why the connection is checked in some DESCRIPTOR-related > statements? In my understanding connection name is not used in >

Re: ECPG bug fix: DECALRE STATEMENT and DEALLOCATE, DESCRIBE

2021-07-06 Thread Michael Paquier
On Tue, Jul 06, 2021 at 04:58:03PM +0900, Michael Paquier wrote: > I have been chewing on this comment and it took me some time to > understand what you meant here. It is true that the ecpglib part, aka > all the routines you are quoting above, don't rely at all on the > connection names. However

Re: Can a child process detect postmaster death when in pg_usleep?

2021-07-06 Thread Michael Paquier
On Tue, Jul 06, 2021 at 12:42:21PM +0530, Bharath Rupireddy wrote: > I'm sorry to say that I didn't get what was said above. We reset the > latch after we come out of WaitLatch but not before going to wait. And > the reason to have WL_LATCH_SET, is to exit the wait loop if MyLatch > is set for that

Re: .ready and .done files considered harmful

2021-07-06 Thread Dilip Kumar
On Tue, Jul 6, 2021 at 11:36 AM Dipesh Pandit wrote: > > Hi, > > We have addressed the O(n^2) problem which involves directory scan for > archiving individual WAL files by maintaining a WAL counter to identify > the next WAL file in a sequence. > > WAL archiver scans the status directory to identi

Re: Refactor "mutually exclusive options" error reporting code in parse_subscription_options

2021-07-06 Thread Amit Kapila
On Fri, Jul 2, 2021 at 12:36 PM Amit Kapila wrote: > > On Fri, Jul 2, 2021 at 8:35 AM Alvaro Herrera wrote: > > > > > The latest patch sent by Bharath looks good to me. Would you like to > > > commit it or shall I take care of it? > > > > Please, go ahead. > > > > Okay, I'll push it early next we

Re: ECPG bug fix: DECALRE STATEMENT and DEALLOCATE, DESCRIBE

2021-07-06 Thread Kyotaro Horiguchi
At Fri, 2 Jul 2021 12:53:02 +, "kuroda.hay...@fujitsu.com" wrote in > Dear Hackers, > > I revised my patch. Thanks for the new version. > > However, I perfectly agree that it's very difficult for users to find a > > problem from the message. > > I will try to add information to it in the

Re: More time spending with "delete pending"

2021-07-06 Thread Michael Paquier
On Sun, Mar 14, 2021 at 06:00:00PM +0300, Alexander Lakhin wrote: > I believe that the patch attached to [1] should fix this issue. The > patch still applies to master and makes the demotest (attached to [2]) > pass. Also I've prepared a trivial patch that makes pgwin32_open() use > the original st

Re: ECPG doesn't compile CREATE AS EXECUTE properly.

2021-07-06 Thread Kyotaro Horiguchi
Thanks for the comment. At Tue, 6 Jul 2021 11:17:47 +0900, Michael Paquier wrote in > On Thu, Jul 01, 2021 at 06:45:25PM +0900, Kyotaro Horiguchi wrote: > > Separating "CREATE TABLE AS EXECUTE" from ExecuteStmt would be cleaner > > but I avoided to change the syntax tree. Instead the attched ma

Re: Skipping logical replication transactions on subscriber side

2021-07-06 Thread Amit Kapila
On Tue, Jul 6, 2021 at 11:28 AM Masahiko Sawada wrote: > > On Mon, Jul 5, 2021 at 7:33 PM Alexey Lesovsky wrote: > > > > Hi, > > Have a few notes about pg_stat_logical_replication_error from the DBA point > > of view (which will use this view in the future). > > Thank you for the comments! > > >

Re: [PATCH] expand the units that pg_size_pretty supports on output

2021-07-06 Thread David Rowley
On Mon, 5 Jul 2021 at 20:00, David Rowley wrote: > I don't really like the fact that I had to add the doHalfRound field > to get the same rounding behaviour as the original functions. I'm > wondering if it would just be too clever just to track how many bits > we've shifted right by in pg_size_pr

Re: Asymmetric partition-wise JOIN

2021-07-06 Thread Andrey Lepikhov
On 5/7/21 23:15, Zhihong Yu wrote: On Mon, Jul 5, 2021 at 2:57 AM Andrey Lepikhov mailto:a.lepik...@postgrespro.ru>> wrote: +            * Can't imagine situation when join relation already exists. But in +            * the 'partition_join' regression test it happens. +            * It may be a

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-07-06 Thread Dilip Kumar
On Fri, Jun 18, 2021 at 12:50 AM Robert Haas wrote: > > On Thu, Jun 17, 2021 at 2:48 PM Andres Freund wrote: > > Or do you mean that looking at the filesystem at all is bypassing shared > > buffers? > > This is what I mean. I think we will end up in a better spot if we can > avoid doing that with

Re: Skipping logical replication transactions on subscriber side

2021-07-06 Thread Amit Kapila
On Tue, Jul 6, 2021 at 12:30 PM Masahiko Sawada wrote: > > On Mon, Jul 5, 2021 at 6:46 PM Amit Kapila wrote: > > > > On Thu, Jul 1, 2021 at 6:31 PM Masahiko Sawada > > wrote: > > > > > > On Thu, Jul 1, 2021 at 12:56 PM Amit Kapila > > > wrote: > > > > > > > > > > > > Don't we want to clear st

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-07-06 Thread David Rowley
On Sun, 4 Jul 2021 at 02:08, Andy Fan wrote: > I'd start to work on UniqueKey again, it would be great that we can target it > to PG 15. The attached patch is just for the notnull_attrs. Since we can't > say > a column is nullable or not without saying in which resultset, so I think > attachi

Re: "debug_invalidate_system_caches_always" is too long

2021-07-06 Thread Andrew Dunstan
On 7/5/21 11:46 PM, Bharath Rupireddy wrote: > On Tue, Jul 6, 2021 at 12:43 AM Tom Lane wrote: >> Noah Misch writes: >>> On Sun, Jul 04, 2021 at 04:27:13PM -0400, Tom Lane wrote: However, I think we should also give serious consideration to "debug_clobber_cache" or "debug_clobber_cach

Why is XLOG_FPI_FOR_HINT always need backups?

2021-07-06 Thread zwj
Hi all, When I read the source code file src/backend/access/transam/xloginsert.c, I get something confused me. In the function XLogSaveBufferForHint, the flags are always REGBUF_FORCE_IMAGE which means it is always need backups. Is it right? Why do not check the full_page_writes? -- Zhang W

Re: Skipping logical replication transactions on subscriber side

2021-07-06 Thread Alexey Lesovsky
On Tue, Jul 6, 2021 at 10:58 AM Masahiko Sawada wrote: > > Also, I'd like to suggest thinking twice about the view name (and > function used in view DDL) - "pg_stat_logical_replication_error" contains > very common "logical replication" words, but the view contains errors > related to subscripti

Re: Can a child process detect postmaster death when in pg_usleep?

2021-07-06 Thread Bharath Rupireddy
On Tue, Jul 6, 2021 at 1:38 PM Michael Paquier wrote: > > On Tue, Jul 06, 2021 at 12:42:21PM +0530, Bharath Rupireddy wrote: > > I'm sorry to say that I didn't get what was said above. We reset the > > latch after we come out of WaitLatch but not before going to wait. And > > the reason to have WL

Re: Refactor "mutually exclusive options" error reporting code in parse_subscription_options

2021-07-06 Thread Bharath Rupireddy
On Tue, Jul 6, 2021 at 1:51 PM Amit Kapila wrote: > > Okay, I'll push it early next week (by Tuesday) unless there are more > > comments or suggestions. Thanks! > > > > Pushed! Thanks, Amit. I'm posting the 0002 patch which removes extra ereport calls using local variables. Please review it. Reg

Re: Commit fest manager

2021-07-06 Thread Ibrar Ahmed
On Fri, Jul 2, 2021 at 7:15 PM Ibrar Ahmed wrote: > > > On Fri, 2 Jul 2021 at 7:06 PM, vignesh C wrote: > >> On Fri, Jul 2, 2021 at 6:05 PM Ibrar Ahmed wrote: >> > >> > >> > >> > On Fri, 2 Jul 2021 at 1:47 PM, David Rowley >> wrote: >> >> >> >> On Fri, 2 Jul 2021 at 15:04, vignesh C wrote: >>

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-07-06 Thread David Rowley
I've been looking at the NOT IN hashing patch again and after making a few minor tweaks I think it's pretty much ready to go. If anyone feels differently, please let me know in the next couple of days. Otherwise, I plan on taking a final look and pushing it soon. David v5-0001-Use-hash-table-to

Re: Remove useless int64 range checks on BIGINT sequence MINVALUE/MAXVALUE values

2021-07-06 Thread David Rowley
On Sun, 4 Jul 2021 at 20:53, David Rowley wrote: > > On Sat, 3 Jul 2021 at 22:44, Peter Eisentraut > wrote: > > I don't think this is a good change. > > > I think we should leave it as is. > > I'm inclined to agree. Does anyone object to marking this patch as rejected in the CF app? David

Re: Removing unneeded self joins

2021-07-06 Thread Hywel Carver
On Mon, Jul 5, 2021 at 2:20 PM Andrey Lepikhov wrote: > On 2/7/21 01:56, Hywel Carver wrote: > > On Wed, Jun 30, 2021 at 12:21 PM Andrey Lepikhov > > mailto:a.lepik...@postgrespro.ru>> wrote: > > I think, here we could ask more general question: do we want to > > remove a > > 'IS NOT

Re: Commit fest manager

2021-07-06 Thread Michael Paquier
On Tue, Jul 06, 2021 at 03:38:23PM +0500, Ibrar Ahmed wrote: > Any update and decision on this? so I can start working on this. Working on the CF does not strongly require the admin permissions. I have already switched the current CF as in progress, so most of the admin job is done for this month

Re: Can a child process detect postmaster death when in pg_usleep?

2021-07-06 Thread Michael Paquier
On Tue, Jul 06, 2021 at 03:54:07PM +0530, Bharath Rupireddy wrote: > Thanks. You are right. The issue is due to the MyLatch being set by > SwitchToSharedLatch before WaitLatch. If we use (WL_TIMEOUT | > WL_EXIT_ON_PM_DEATH), then the backends will honour the > post_auth_delay as well as detect the

Re: Why is XLOG_FPI_FOR_HINT always need backups?

2021-07-06 Thread Japin Li
On Tue, 06 Jul 2021 at 17:58, zwj <757634...@qq.com> wrote: > Hi all, > > When I read the source code file src/backend/access/transam/xloginsert.c, I > get something confused me. > In the function XLogSaveBufferForHint, the flags are always > REGBUF_FORCE_IMAGE which means it is always need bac

Re: ECPG doesn't compile CREATE AS EXECUTE properly.

2021-07-06 Thread Michael Paquier
On Tue, Jul 06, 2021 at 05:47:34PM +0900, Kyotaro Horiguchi wrote: > More accurately, I didn't come up with the way to split out some of > the rule-components in a rule out as another rule using the existing > infrastructure. > > [...] > > Then add the following component to the rule "stmt". I se

Re: [PATCH] Use optimized single-datum tuplesort in ExecSort

2021-07-06 Thread Ranier Vilela
Em ter., 6 de jul. de 2021 às 03:15, Ronan Dunklau escreveu: > Hello, > > While testing the patch "Add proper planner support for ORDER BY / > DISTINCT > aggregates" [0] I discovered the performance penalty from adding a sort > node > essentially came from not using the single-datum tuplesort opt

Re: Commit fest manager

2021-07-06 Thread Ibrar Ahmed
On Tue, Jul 6, 2021 at 3:58 PM Michael Paquier wrote: > On Tue, Jul 06, 2021 at 03:38:23PM +0500, Ibrar Ahmed wrote: > > Any update and decision on this? so I can start working on this. > > Working on the CF does not strongly require the admin permissions. I > have already switched the current C

Re: Can a child process detect postmaster death when in pg_usleep?

2021-07-06 Thread Bharath Rupireddy
On Tue, Jul 6, 2021 at 4:33 PM Michael Paquier wrote: > > On Tue, Jul 06, 2021 at 03:54:07PM +0530, Bharath Rupireddy wrote: > > Thanks. You are right. The issue is due to the MyLatch being set by > > SwitchToSharedLatch before WaitLatch. If we use (WL_TIMEOUT | > > WL_EXIT_ON_PM_DEATH), then the

Re: [PATCH] expand the units that pg_size_pretty supports on output

2021-07-06 Thread Dean Rasheed
On Tue, 6 Jul 2021 at 10:20, David Rowley wrote: > > I made another pass over this and ended up removing the doHalfRound > field in favour of just doing rounding based on the previous > bitshifts. > When I first read this: +/* half-round until we get down to unitBits */ +

Re: [PATCH] Use optimized single-datum tuplesort in ExecSort

2021-07-06 Thread Ranier Vilela
Em ter., 6 de jul. de 2021 às 08:25, Ranier Vilela escreveu: > Em ter., 6 de jul. de 2021 às 03:15, Ronan Dunklau > escreveu: > >> Hello, >> >> While testing the patch "Add proper planner support for ORDER BY / >> DISTINCT >> aggregates" [0] I discovered the performance penalty from adding a sor

Re: Remove useless int64 range checks on BIGINT sequence MINVALUE/MAXVALUE values

2021-07-06 Thread Greg Nancarrow
On Tue, Jul 6, 2021 at 8:43 PM David Rowley wrote: > > On Sun, 4 Jul 2021 at 20:53, David Rowley wrote: > > > > On Sat, 3 Jul 2021 at 22:44, Peter Eisentraut > > wrote: > > > I don't think this is a good change. > > > > > I think we should leave it as is. > > > > I'm inclined to agree. > > Does

Re: [PATCH] expand the units that pg_size_pretty supports on output

2021-07-06 Thread David Rowley
On Tue, 6 Jul 2021 at 23:39, Dean Rasheed wrote: > When I first read this: > > +/* half-round until we get down to unitBits */ > +while (rightshifts++ < unit->unitBits) > +size = half_rounded(size); > > it looked to me like it would be invoking half_rounded(

Re: Removing redundant check for transaction in progress in check_safe_enum_use

2021-07-06 Thread Matthias van de Meent
On Sun, 4 Jul 2021, 03:40 Zhihong Yu, wrote: > > Hi, > I was looking at : > Relax transactional restrictions on ALTER TYPE ... ADD VALUE (redux). > > In check_safe_enum_use(): > > + if (!TransactionIdIsInProgress(xmin) && > + TransactionIdDidCommit(xmin)) > + return; > > Since the co

?????? Why is XLOG_FPI_FOR_HINT always need backups?

2021-07-06 Thread zwj
Thank you for reply. You are right and the PostgreSQL server writes the entire content of each disk page to WAL during the first modification of that page after a checkpoint while data checksum is on. But I wonder whether it is necessary or not while my file system can protect the blocks of d

Re: [PATCH] expand the units that pg_size_pretty supports on output

2021-07-06 Thread Dean Rasheed
On Tue, 6 Jul 2021 at 13:15, David Rowley wrote: > > Can you give an example where calling half_rounded too many times will > give the wrong value? Keeping in mind we call half_rounded the number > of times that the passed in value would need to be left-shifted by to > get the equivalent truncated

Re: Asymmetric partition-wise JOIN

2021-07-06 Thread Alexander Pyhalov
Andrey Lepikhov писал 2021-07-06 12:28: On 5/7/21 23:15, Zhihong Yu wrote: On Mon, Jul 5, 2021 at 2:57 AM Andrey Lepikhov mailto:a.lepik...@postgrespro.ru>> wrote: +            * Can't imagine situation when join relation already exists. But in +            * the 'partition_join' regression te

Re: Fix pkg-config file for static linking

2021-07-06 Thread Peter Eisentraut
On 21.06.21 15:47, Filip Gospodinov wrote: -PKG_CONFIG_REQUIRES_PRIVATE = libssl libcrypto +PKG_CONFIG_REQUIRES_PRIVATE = libpgcommon libpgport libssl libcrypto This doesn't work. This patch adds libpgcommon and libpgport to Requires.private. But they are not pkg-config names but library nam

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-07-06 Thread Tom Lane
David Rowley writes: > Tom, I'm wondering if you might get a chance to draw up a design for > what you've got in mind with this? I assume adding a new field in > Var, but I'm drawing a few blanks on how things might work for equal() > when one Var has the field set and another does not. As I sai

Re: [PATCH] Use optimized single-datum tuplesort in ExecSort

2021-07-06 Thread James Coleman
Adding David since this patch is likely a precondition for [1]. On Tue, Jul 6, 2021 at 2:15 AM Ronan Dunklau wrote: > > Hello, > > While testing the patch "Add proper planner support for ORDER BY / DISTINCT > aggregates" [0] I discovered the performance penalty from adding a sort node > essential

Re: "debug_invalidate_system_caches_always" is too long

2021-07-06 Thread Tom Lane
Andrew Dunstan writes: > On 7/5/21 11:46 PM, Bharath Rupireddy wrote: >> On Tue, Jul 6, 2021 at 12:43 AM Tom Lane wrote: >>> I like "debug_flush_caches" --- it's short and accurate. >> Do we always flush the cache entries into the disk? Sometimes we just >> invalidate the cache entries in the re

Re: .ready and .done files considered harmful

2021-07-06 Thread Stephen Frost
Greetings, * Dipesh Pandit (dipesh.pan...@gmail.com) wrote: > We have addressed the O(n^2) problem which involves directory scan for > archiving individual WAL files by maintaining a WAL counter to identify > the next WAL file in a sequence. This seems to have missed the concerns raised in https:

Re: [PATCH] Use optimized single-datum tuplesort in ExecSort

2021-07-06 Thread Ranier Vilela
Em ter., 6 de jul. de 2021 às 10:19, James Coleman escreveu: > Adding David since this patch is likely a precondition for [1]. > > On Tue, Jul 6, 2021 at 2:15 AM Ronan Dunklau > wrote: > > > > Hello, > > > > While testing the patch "Add proper planner support for ORDER BY / > DISTINCT > > aggreg

Re: [PATCH] Use optimized single-datum tuplesort in ExecSort

2021-07-06 Thread Dilip Kumar
On Tue, Jul 6, 2021 at 6:49 PM James Coleman wrote: > > Adding David since this patch is likely a precondition for [1]. > > On Tue, Jul 6, 2021 at 2:15 AM Ronan Dunklau wrote: > > > > Hello, > > > > While testing the patch "Add proper planner support for ORDER BY / DISTINCT > > aggregates" [0] I

Re: proposal - log_full_scan

2021-07-06 Thread Daniel Gustafsson
Looking at this I like the idea in principle, but I'm not convinced that auto_explain is the right tool for this. auto_explain is for identifying slow queries, and what you are proposing is to identify queries with a certain "shape" (for lack of a better term) even if they aren't slow as per the l

Re: .ready and .done files considered harmful

2021-07-06 Thread Dipesh Pandit
> specifically about history files being given higher priority for > archiving. If we go with this change then we'd at least want to rewrite > or remove those comments, but I don't actually agree that we should > remove that preference to archive history files ahead of WAL, for the > reasons broug

Re: Identify missing publications from publisher while create/alter subscription.

2021-07-06 Thread vignesh C
On Wed, Jun 30, 2021 at 8:23 PM vignesh C wrote: > > On Sun, Jun 6, 2021 at 11:55 AM vignesh C wrote: > > > > On Fri, May 7, 2021 at 6:44 PM vignesh C wrote: > > > > > > Thanks for the comments, the attached patch has the fix for the same. > > > > The patch was not applying on the head, attached

Re: [PATCH] expand the units that pg_size_pretty supports on output

2021-07-06 Thread David Christensen
David Rowley writes: > On Mon, 5 Jul 2021 at 20:00, David Rowley wrote: >> I don't really like the fact that I had to add the doHalfRound field >> to get the same rounding behaviour as the original functions. I'm >> wondering if it would just be too clever just to track how many bits >> we've

Re: .ready and .done files considered harmful

2021-07-06 Thread Jeevan Ladhe
> I have a few suggestions on the patch > 1. > + > + /* > + * Found the oldest WAL, reset timeline ID and log segment number to > generate > + * the next WAL file in the sequence. > + */ > + if (found && !historyFound) > + { > + XLogFromFileName(xlog, &curFileTLI, &nextLogSegNo, wal_segment_size);

Re: [PATCH] expand the units that pg_size_pretty supports on output

2021-07-06 Thread Tom Lane
David Rowley writes: > Does anyone want to have a look over this? If not, I plan to push it > in the next day or so. Minor nit: use "const char *text" in the struct declaration, so that all of the static data can be placed in fixed storage. > (I'm not sure why pgindent removed the space between

Re: [PATCH] Use optimized single-datum tuplesort in ExecSort

2021-07-06 Thread Ronan Dunklau
Thank you for the review, I will address those shortly, but will answer some questions in the meantime. > > First, the changes are lacking any explanatory comments. Probably we > > should follow how nodeAgg does this and add both comments to the > > ExecSort function header as well as specific co

Re: Enhanced error message to include hint messages for redundant options error

2021-07-06 Thread vignesh C
On Wed, Jun 30, 2021 at 7:48 PM vignesh C wrote: > > On Thu, May 13, 2021 at 8:09 PM vignesh C wrote: > > > > On Thu, May 13, 2021 at 4:58 AM Alvaro Herrera > > wrote: > > > > > > > Thanks for the comments, Attached patch has the changes for the same. > > > > The Patch was not applying on Head,

Re: [PATCH] Use optimized single-datum tuplesort in ExecSort

2021-07-06 Thread James Coleman
On Tue, Jul 6, 2021 at 11:03 AM Ronan Dunklau wrote: > > Thank you for the review, I will address those shortly, but will answer some > questions in the meantime. > > > > First, the changes are lacking any explanatory comments. Probably we > > > should follow how nodeAgg does this and add both com

Re: Refactor "mutually exclusive options" error reporting code in parse_subscription_options

2021-07-06 Thread Alvaro Herrera
On 2021-Jul-06, Bharath Rupireddy wrote: > Thanks, Amit. I'm posting the 0002 patch which removes extra ereport > calls using local variables. Please review it. I looked at this the other day and I'm not sure I like it very much. It's not making anything any simpler, it's barely saving two lines

Re: proposal - log_full_scan

2021-07-06 Thread Pavel Stehule
Hi út 6. 7. 2021 v 16:07 odesílatel Daniel Gustafsson napsal: > Looking at this I like the idea in principle, but I'm not convinced that > auto_explain is the right tool for this. auto_explain is for identifying > slow > queries, and what you are proposing is to identify queries with a certain

Re: logical replication worker accesses catalogs in error context callback

2021-07-06 Thread Tom Lane
Bharath Rupireddy writes: > How about making the below else if statement and the attname > assignment into a single line? They are falling below the 80 char > limit. > else if (colno > 0 && colno <= list_length(rte->eref->colnames)) > attname = strVal(list_nth(rte->eref->colnam

Re: Delegating superuser tasks to new security roles (Was: Granting control of SUSET gucs to non-superusers)

2021-07-06 Thread Mark Dilger
> On Jul 5, 2021, at 1:50 AM, Andrey Borodin wrote: > > I'm not sure, but maybe we should allow replication role to change > session_replication_role? Thanks, Andrey, for taking a look. Yes, there is certainly some logic to that suggestion. The patch v4-0005 only delegates authority to pe

Re: Grammar railroad diagram

2021-07-06 Thread Bruce Momjian
On Sat, Jul 3, 2021 at 10:39:02AM +0200, Domingo Alvarez Duarte wrote: > I've done a experimental tool to convert bison grammars to a kind of EBNF > understood by https://www.bottlecaps.de/rr/ui to generate railroad diagrams > see > bellow the converted 'postgresql-13.3/src/backend/parser/gram.y'

Re: Grammar railroad diagram

2021-07-06 Thread Domingo Alvarez Duarte
Hello Bruce ! You can download the railroad generator to generate offline using Java here -> https://www.bottlecaps.de/rr/download/rr-1.63-java8.zip (link from the https://www.bottlecaps.de/rr/ui on tab Welcome). java -jar rr.war -out:Dafny.atg.xhtml grammar.txt Cheers ! On 6/7/21 18:51, Br

Re: visibility map corruption

2021-07-06 Thread Bruce Momjian
On Sun, Jul 4, 2021 at 10:28:25PM +, Floris Van Nee wrote: > > > > I wonder if it's related to this issue: > > > > https://www.postgresql.org/message- > > id/20210423234256.hwopuftipdmp3...@alap3.anarazel.de > > > > Have you increased autovacuum_freeze_max_age from its default? This > > already

Re: Planning time grows exponentially with levels of nested views

2021-07-06 Thread Dean Rasheed
On Sun, 18 Apr 2021 at 21:42, Tom Lane wrote: > > > If multiple references are actually possible then this'd break it. > > I think this patch doesn't make things any worse for such a case though. > If we re-introduced such a bug, the result would be an immediate null > pointer crash while trying t

Re: visibility map corruption

2021-07-06 Thread Peter Geoghegan
On Tue, Jul 6, 2021 at 10:27 AM Bruce Momjian wrote: > OK, this is confirmation that the pg_resetwal bug, and its use by > pg_upgrade, is a serious issue that needs to be addressed. I am > prepared to work on it now. To be clear, I'm not 100% sure that this is related to the pg_upgrade + "pg_res

Re: [CLOBBER_CACHE]Server crashed with segfault 11 while executing clusterdb

2021-07-06 Thread Tom Lane
Amul Sul writes: > On Tue, Apr 20, 2021 at 6:59 AM Kyotaro Horiguchi > wrote: >> I don't mind RelationGetSmgr(index)->smgr_rnode alone or >> &variable->member alone and there's not the previous call to >> RelationGetSmgr just above. How about using a temporary variable? >> >> SMgrRelation srel =

Re: Planning time grows exponentially with levels of nested views

2021-07-06 Thread Tom Lane
Dean Rasheed writes: > I took a look at this and wasn't able to find any way to break it, and > your argument that it can't really make such rewriter bugs any worse > makes sense. Thanks for looking! > Would it make sense to update the comment prior to copying the subquery? Yeah, I hadn't touch

Re: visibility map corruption

2021-07-06 Thread Bruce Momjian
On Tue, Jul 6, 2021 at 10:32:24AM -0700, Peter Geoghegan wrote: > On Tue, Jul 6, 2021 at 10:27 AM Bruce Momjian wrote: > > OK, this is confirmation that the pg_resetwal bug, and its use by > > pg_upgrade, is a serious issue that needs to be addressed. I am > > prepared to work on it now. > > To

Re: Pipeline mode and PQpipelineSync()

2021-07-06 Thread Alvaro Herrera
On 2021-Jul-06, Boris Kolpackov wrote: > Alvaro Herrera writes: > > > Ah, yes it does. I can reproduce this now. I thought PQconsumeInput > > was sufficient, but it's not: you have to have the PQgetResult in there > > too. Looking ... > > Any progress on fixing this? Can you please try with

Re: Pipeline mode and PQpipelineSync()

2021-07-06 Thread Alvaro Herrera
On 2021-Jul-06, Boris Kolpackov wrote: > Alvaro Herrera writes: > > > Ah, yes it does. I can reproduce this now. I thought PQconsumeInput > > was sufficient, but it's not: you have to have the PQgetResult in there > > too. Looking ... > > Any progress on fixing this? Yeah ... the problem as

Re: [PATCH] Pull general SASL framework out of SCRAM

2021-07-06 Thread Jacob Champion
On Mon, 2021-07-05 at 17:17 +0900, Michael Paquier wrote: > On Wed, Jun 30, 2021 at 10:30:12PM +, Jacob Champion wrote: > > Done in v3, with a second patch for the code motion. > > I have gone through that, tweaking the documentation you have added as > that's the meat of the patch, reworking

Re: visibility map corruption

2021-07-06 Thread Peter Geoghegan
On Tue, Jul 6, 2021 at 10:58 AM Bruce Momjian wrote: > Well, pg_upgrade corruptions are rare, but so is modifying > autovacuum_freeze_max_age. If we have a corruption and we know > autovacuum_freeze_max_age was modified, odds are that is the cause. My point is that there isn't necessarily that m

Re: visibility map corruption

2021-07-06 Thread Tom Lane
Peter Geoghegan writes: > ... We should just carry forward the original oldestXid. Yup. It's a bit silly that we recognized the need to do that for oldestMultiXid yet not for oldestXid. BTW, is it really necessary for copy_xact_xlog_xid to invoke pg_resetwal so many times? Why can't we pass al

Re: prion failed with ERROR: missing chunk number 0 for toast value 14334 in pg_toast_2619

2021-07-06 Thread Justin Pryzby
On Wed, Jun 30, 2021 at 03:29:41PM +0900, Michael Paquier wrote: > On Tue, May 18, 2021 at 01:04:18PM +0200, Drouvot, Bertrand wrote: > > On 5/17/21 8:56 PM, Andres Freund wrote: > >> On 2021-05-17 20:14:40 +0200, Drouvot, Bertrand wrote: > >>> I was also wondering if: > >>> > >>> * We should ke

Re: proposal - log_full_scan

2021-07-06 Thread Daniel Gustafsson
> On 6 Jul 2021, at 18:14, Pavel Stehule wrote: > I thought about it more, and sometimes bitmap index scans are problematic > too, index scans in nested loops can be a problem too. Right. Depending on the circumstances, pretty much anything in a plan can be something deemed problematic in some

Re: Hook for extensible parsing.

2021-07-06 Thread Jim Mlodgenski
On Sat, Jun 12, 2021 at 4:29 AM Julien Rouhaud wrote: > I'd like to propose an alternative approach, which is to allow multiple > parsers > to coexist, and let third-party parsers optionally fallback on the core > parsers. I'm sending this now as a follow-up of [1] and to avoid duplicated > eff

Re: Using COPY FREEZE in pgbench

2021-07-06 Thread Dean Rasheed
On Sun, 4 Jul 2021 at 09:32, Tatsuo Ishii wrote: > > >> So overall gain by the patch is around 15%, whereas the last test > >> before the commit was 14%. It seems the patch is still beneficial > >> after the commit. > > > > Yes, that's good! > > Yeah! > I tested this with -s100 and got similar r

Re: [PATCH v3 1/1] Fix detection of preadv/pwritev support for OSX.

2021-07-06 Thread Tom Lane
Peter Eisentraut writes: > I think this change is perfectly appropriate (modulo some small cleanups). I think there are a couple of issues here. 1. People who are already using MACOSX_DEPLOYMENT_TARGET to control their builds would like to keep on doing so, but the AC_CHECK_FUNCS probe doesn't w

Re: rand48 replacement

2021-07-06 Thread Fabien COELHO
Hello Yura, However, I'm not enthousiastic at combining two methods depending on the range, the function looks complex enough without that, so I would suggest not to take this option. Also, the decision process adds to the average cost, which is undesirable. Given 99.99% cases will be in the

Re: [PATCH v3 1/1] Fix detection of preadv/pwritev support for OSX.

2021-07-06 Thread James Hilliard
On Tue, Jul 6, 2021 at 2:34 PM Tom Lane wrote: > > Peter Eisentraut writes: > > I think this change is perfectly appropriate (modulo some small cleanups). > > I think there are a couple of issues here. > > 1. People who are already using MACOSX_DEPLOYMENT_TARGET to control > their builds would li

Re: [PATCH v3 1/1] Fix detection of preadv/pwritev support for OSX.

2021-07-06 Thread Tom Lane
James Hilliard writes: > On Tue, Jul 6, 2021 at 2:34 PM Tom Lane wrote: >> As far as I can tell, the only way to really deal with #2 is to >> perform a runtime dlsym() probe to see whether pwritev exists, and >> then fall back to our src/port/ implementation if not. This does >> not look particu

Re: visibility map corruption

2021-07-06 Thread Peter Geoghegan
On Tue, Jul 6, 2021 at 11:57 AM Tom Lane wrote: > Peter Geoghegan writes: > > ... We should just carry forward the original oldestXid. > > Yup. It's a bit silly that we recognized the need to do that > for oldestMultiXid yet not for oldestXid. True. But at the same time it somehow doesn't seem

Re: trivial improvement to system_or_bail

2021-07-06 Thread Alvaro Herrera
On 2021-Jun-30, Daniel Gustafsson wrote: > + BAIL_OUT("system $_[0] failed: $!\n"); > I wonder if we should take more inspiration from the Perl manual and change it > to "failed to execute" to make it clear that the failure was in executing the > program, not from the program i

Re: visibility map corruption

2021-07-06 Thread Mark Dilger
> On Jul 6, 2021, at 2:27 PM, Peter Geoghegan wrote: > > It looks like amcheck's verify_heapam.c functionality almost catches > bugs like this one. Something for Mark (CC'd) to consider. Does it > matter that we usually "ctx.oldest_xid = ctx.relfrozenxid", and so > usually use pg_class.relfroz

Re: [PATCH] Allow CustomScan nodes to signal projection support

2021-07-06 Thread Tom Lane
Aleksander Alekseev writes: >> I named the flag CUSTOMPATH_SUPPORT_PROJECTION similar to the other >> custom node flags, but this would revert the current logic > This seems to be a typical Kobayashi Maru situation, i.e any choice is > a bad one. I suggest keeping the patch as is and hoping that

[PATCH][postgres_fdw] Add push down of CASE WHEN clauses

2021-07-06 Thread Gilles Darold
Hi, I have noticed that postgres_fdw do not push down the CASE WHEN clauses. In the following case this normal: contrib_regression=# EXPLAIN (ANALYZE, VERBOSE) SELECT (CASE WHEN mod(c1, 4) = 0 THEN 1 ELSE 2 END) FROM ft1;    QUERY PLAN

Re: visibility map corruption

2021-07-06 Thread Peter Geoghegan
On Tue, Jul 6, 2021 at 3:12 PM Mark Dilger wrote: > Thanks, Peter, for drawing my attention to this. I had already been > following this thread, but had not yet thought about the problem in terms of > amcheck. > > I will investigate possible solutions in verify_heapam(). Thanks! Great that we

Re: visibility map corruption

2021-07-06 Thread Bruce Momjian
On Tue, Jul 6, 2021 at 02:27:34PM -0700, Peter Geoghegan wrote: > > BTW, is it really necessary for copy_xact_xlog_xid to invoke pg_resetwal > > so many times? Why can't we pass all of the update-this options in one > > call? > > No opinion here. > > > Who's going to do the legwork on this? >

Re: visibility map corruption

2021-07-06 Thread Peter Geoghegan
On Tue, Jul 6, 2021 at 3:30 PM Bruce Momjian wrote: > Yes, I can, though it seems like a much bigger issue than pg_upgrade. > I will be glad to dig into it. I'm not sure what you mean by that. Technically this would be an issue for any program that uses "pg_resetwal -x" in the way that pg_upgrade

Re: visibility map corruption

2021-07-06 Thread Bruce Momjian
On Tue, Jul 6, 2021 at 06:30:41PM -0400, Bruce Momjian wrote: > On Tue, Jul 6, 2021 at 02:27:34PM -0700, Peter Geoghegan wrote: > > > BTW, is it really necessary for copy_xact_xlog_xid to invoke pg_resetwal > > > so many times? Why can't we pass all of the update-this options in one > > > call?

Re: visibility map corruption

2021-07-06 Thread Bruce Momjian
On Tue, Jul 6, 2021 at 03:46:48PM -0700, Peter Geoghegan wrote: > On Tue, Jul 6, 2021 at 3:30 PM Bruce Momjian wrote: > > Yes, I can, though it seems like a much bigger issue than pg_upgrade. > > I will be glad to dig into it. > > I'm not sure what you mean by that. Technically this would be an

Re: visibility map corruption

2021-07-06 Thread Peter Geoghegan
On Tue, Jul 6, 2021 at 3:49 PM Bruce Momjian wrote: > My point is that there are a lot internals involved here that are not > part of pg_upgrade, though it probably only affects pg_upgrade. Anyway, > Bertrand patch seems to have what I need. I was confused by your remarks because I am kind of lo

Re: Column Filtering in Logical Replication

2021-07-06 Thread Alvaro Herrera
Hello, here are a few comments on this patch. The patch adds a function get_att_num_by_name; but we have a lsyscache.c function for that purpose, get_attnum. Maybe that one should be used instead? get_tuple_columns_map() returns a bitmapset of the attnos of the columns in the given list, so its

Re: Refactor "mutually exclusive options" error reporting code in parse_subscription_options

2021-07-06 Thread Peter Smith
On Tue, Jul 6, 2021 at 6:21 PM Amit Kapila wrote: > > On Fri, Jul 2, 2021 at 12:36 PM Amit Kapila wrote: > > > > On Fri, Jul 2, 2021 at 8:35 AM Alvaro Herrera > > wrote: > > > > > > > The latest patch sent by Bharath looks good to me. Would you like to > > > > commit it or shall I take care of

Re: visibility map corruption

2021-07-06 Thread Bruce Momjian
On Tue, Jul 6, 2021 at 06:49:10PM -0400, Bruce Momjian wrote: > On Tue, Jul 6, 2021 at 03:46:48PM -0700, Peter Geoghegan wrote: > > On Tue, Jul 6, 2021 at 3:30 PM Bruce Momjian wrote: > > > Yes, I can, though it seems like a much bigger issue than pg_upgrade. > > > I will be glad to dig into it.

Re: Re[3]: On login trigger: take three

2021-07-06 Thread Greg Nancarrow
On Sun, Jul 4, 2021 at 1:21 PM vignesh C wrote: > > CFBot shows the following failure: > # poll_query_until timed out executing this query: > # SELECT '0/3046250' <= replay_lsn AND state = 'streaming' FROM > pg_catalog.pg_stat_replication WHERE application_name = 'standby_1'; > # expecting this ou

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-07-06 Thread Andy Fan
On Tue, Jul 6, 2021 at 9:14 PM Tom Lane wrote: > David Rowley writes: > > Tom, I'm wondering if you might get a chance to draw up a design for > > what you've got in mind with this? I assume adding a new field in > > Var, but I'm drawing a few blanks on how things might work for equal() > > whe

  1   2   >