Re: Teach pg_receivewal to use lz4 compression

2021-07-11 Thread Michael Paquier
On Mon, Jul 12, 2021 at 11:10:24AM +0530, Dilip Kumar wrote: > On Thu, Jul 8, 2021 at 7:48 PM wrote: >> We can, though I am not in favour of doing so. There is seemingly >> little benefit for added complexity. > > I am really not sure what complexity you are talking about, do you > mean since

Re: More time spending with "delete pending"

2021-07-11 Thread Michael Paquier
On Mon, Jul 12, 2021 at 02:09:41PM +0900, Michael Paquier wrote: > And fairywren, that uses MinGW, is unhappy: > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fairywren=2021-07-12%2004%3A47%3A13 > Looking at it now. I am not sure what to do here to cool down MinGW, so the patch has been

Re: Skipping logical replication transactions on subscriber side

2021-07-11 Thread Masahiko Sawada
On Mon, Jul 12, 2021 at 1:15 PM Amit Kapila wrote: > > On Mon, Jul 12, 2021 at 9:37 AM Alexey Lesovsky wrote: > > > > On Mon, Jul 12, 2021 at 8:36 AM Amit Kapila wrote: > >> > >> > > >> > Ok, looks nice. But I am curious how this will work in the case when > >> > there are two (or more) errors

Re: Teach pg_receivewal to use lz4 compression

2021-07-11 Thread Dilip Kumar
On Thu, Jul 8, 2021 at 7:48 PM wrote: > > Dilip Kumar wrote: > > > Wouldn't it be better to call it compression method instead of > > compression program? > > Agreed. This is inline with the suggestions of other reviewers. > Find the change in the attached patch. Thanks, I will have a look. > >

Re: More time spending with "delete pending"

2021-07-11 Thread Michael Paquier
On Mon, Jul 12, 2021 at 01:07:17PM +0900, Michael Paquier wrote: > Thanks, that matches my impression. There was one comment at the top > of _pgstat64() that was still incorrect. I have spent more time > checking and tested this stuff today, and that looks fine. One > question pending is if we

Re: [CLOBBER_CACHE]Server crashed with segfault 11 while executing clusterdb

2021-07-11 Thread Amul Sul
On Fri, Jul 9, 2021 at 7:30 PM Alvaro Herrera wrote: > > On 2021-Jul-09, Amul Sul wrote: > > > > On Tue, Jul 6, 2021 at 11:06 PM Tom Lane wrote: > > > > > The point of the static-inline function idea was to be cheap enough > > > > that it isn't worth worrying about this sort of risky

Re: Remove useless int64 range checks on BIGINT sequence MINVALUE/MAXVALUE values

2021-07-11 Thread Greg Nancarrow
On Mon, Jul 12, 2021 at 2:26 PM David Rowley wrote: > > > It seems strange to add a comment to explain why it's there. If we're > > going to the trouble of doing that, then we should just remove it and > > add a very small comment to mention why INT8 sequences don't need to > > be checked. > >

Re: row filtering for logical replication

2021-07-11 Thread Amit Kapila
On Mon, Jul 12, 2021 at 7:19 AM Alvaro Herrera wrote: > > Hi > > Andres complained about the safety of doing general expression > evaluation in pgoutput; that was first in > > https://postgr.es/m/20210128022032.eq2qqc6zxkqn5...@alap3.anarazel.de > where he described a possible approach to handle

Re: Remove useless int64 range checks on BIGINT sequence MINVALUE/MAXVALUE values

2021-07-11 Thread David Rowley
On Wed, 7 Jul 2021 at 20:37, David Rowley wrote: > > On Wed, 7 Jul 2021 at 00:06, Greg Nancarrow wrote: > > I think if you're going to reject this patch, a brief comment should > > be added to that code to justify why that existing superfluous check > > is worthwhile. > > It seems strange to add

Re: Skipping logical replication transactions on subscriber side

2021-07-11 Thread Amit Kapila
On Mon, Jul 12, 2021 at 9:37 AM Alexey Lesovsky wrote: > > On Mon, Jul 12, 2021 at 8:36 AM Amit Kapila wrote: >> >> > >> > Ok, looks nice. But I am curious how this will work in the case when there >> > are two (or more) errors in the same subscription, but different relations? >> > >> >> We

Re: Skipping logical replication transactions on subscriber side

2021-07-11 Thread Alexey Lesovsky
On Mon, Jul 12, 2021 at 8:36 AM Amit Kapila wrote: > > > > Ok, looks nice. But I am curious how this will work in the case when > there are two (or more) errors in the same subscription, but different > relations? > > > > We can't proceed unless the first error is resolved, so there > shouldn't

Re: More time spending with "delete pending"

2021-07-11 Thread Michael Paquier
On Fri, Jul 09, 2021 at 09:00:00PM +0300, Alexander Lakhin wrote: > Thank you! I agree with your improvement. I can't remember why did I > inject 'include "port.h"' in genfile.c. > Today I've rechecked all the chain of includes and I see that stat() is > redefined as _pgstat64() in win32_port.h,

RE: ECPG bug fix: DECALRE STATEMENT and DEALLOCATE, DESCRIBE

2021-07-11 Thread kuroda.hay...@fujitsu.com
Dear Horiguchi-san, Thank you for reviewing! I attached new version. Sorry for delaying reply. > Since we don't allow descriptors with the same name even if they are > for the different connections, I think we can set the current > connection if any (which is set either by AT option or

Re: [HACKERS] logical decoding of two-phase transactions

2021-07-11 Thread Peter Smith
On Sun, Jul 11, 2021 at 8:20 PM Amit Kapila wrote: > > On Fri, Jul 9, 2021 at 4:43 AM Peter Smith wrote: > > > > > The patch looks good to me, I don't have any comments. > > > > I tried the v95-0001 patch. > > > > - The patch applied cleanly and all build / testing was OK. > > - The

Re: Skipping logical replication transactions on subscriber side

2021-07-11 Thread Amit Kapila
On Fri, Jul 9, 2021 at 9:02 AM Alexey Lesovsky wrote: > > On Fri, Jul 9, 2021 at 5:43 AM Masahiko Sawada wrote: >> >> > I also would like to clarify, when conflict is resolved - the error record >> > is cleared or kept in the view? If it is cleared, the error counter is >> > required (because

Re: Skipping logical replication transactions on subscriber side

2021-07-11 Thread Amit Kapila
On Fri, Jul 9, 2021 at 5:58 AM Masahiko Sawada wrote: > > On Thu, Jul 8, 2021 at 6:28 PM Amit Kapila wrote: > > > > On Wed, Jul 7, 2021 at 11:47 AM Masahiko Sawada > > wrote: > > > > > > On Tue, Jul 6, 2021 at 6:33 PM Amit Kapila > > > wrote: > > > > > > > > > > > > > > According to the doc,

Re: Diagnostic comment in LogicalIncreaseXminForSlot

2021-07-11 Thread Amit Kapila
On Mon, Jul 5, 2021 at 12:54 PM Masahiko Sawada wrote: > > On Fri, May 21, 2021 at 6:00 PM Ashutosh Bapat > wrote: > > > > It's there in CF. I am fine with PG-15. It will be good to patch the > > back-branches to have this extra diagnostic information available. > > The patch looks to me. > {

Re: Record a Bitmapset of non-pruned partitions

2021-07-11 Thread David Rowley
On Sat, 10 Jul 2021 at 03:24, David Rowley wrote: > The good news is that the code in partitions_are_ordered() became even > more simple as a result of this change. We can do ordered scan simply > when !bms_overlap(live_parts, boundinfo->interleaved_parts). I've spent a bit more time revising

Re: bugfix: when the blocksize is 32k, the function page_header of pageinspect returns negative numbers.

2021-07-11 Thread Michael Paquier
On Sat, Jul 10, 2021 at 07:09:10PM +0530, Bharath Rupireddy wrote: > Thanks. The patch looks good to me, except a minor comment - isn't it > "int2 for these fields" as the fields still exist? + /* pageinspect >= > 1.10 uses int4 instead of int2 for those fields */ This comment looks fine to me

Re: row filtering for logical replication

2021-07-11 Thread Greg Nancarrow
On Mon, Jul 12, 2021 at 9:31 AM Euler Taveira wrote: > > cfbot seems to be unhappy with v18 on some of the hosts. Cirrus/FreeBSD failed > in the test 010_truncate. It also failed in a Cirrus/Linux box. I failed to > reproduce in my local FreeBSD box. Since it passes appveyor and Cirrus/macos, >

Re: Cosmic ray hits integerset

2021-07-11 Thread Greg Stark
Fwiw, yes it could be a cosmic ray. It could also just be marginally bad ram. Bad ram is notoriously hard to reliably test for. It can be very sensitive to the exact bit pattern stored in it, the timing of reads and writes, and other factors. The whole point of the rowhammer attacks is to push

Re: row filtering for logical replication

2021-07-11 Thread Alvaro Herrera
Hi Andres complained about the safety of doing general expression evaluation in pgoutput; that was first in https://postgr.es/m/20210128022032.eq2qqc6zxkqn5...@alap3.anarazel.de where he described a possible approach to handle it by restricting expressions to have limited shape; and later in

Re: enable_resultcache confusion

2021-07-11 Thread David Rowley
On Mon, 12 Jul 2021 at 03:22, Justin Pryzby wrote: > |This is useful if only a small percentage of rows is checked on > |the inner side and is controlled by |linkend="guc-enable-resultcache"/>. You might be right there, but I'm not too sure if I changed that that it

Re: Protect against possible memory corruption (src/backend/access/nbtree/nbtxlog.c)

2021-07-11 Thread Ranier Vilela
Em dom., 11 de jul. de 2021 às 19:19, Heikki Linnakangas escreveu: > On 11/07/2021 22:51, Ranier Vilela wrote: > > Hi, > > > > While analyzing a possible use of an uninitialized variable, I checked > that > > *_bt_restore_page* can lead to memory corruption, > > by not checking the maximum limit

Re: row filtering for logical replication

2021-07-11 Thread Euler Taveira
On Sun, Jul 11, 2021, at 4:39 PM, Euler Taveira wrote: > with cache (v18) > --- > > mean: 0.63 us > stddev: 1.07 us > median: 0.55 us > min-max:[0.29 .. 844.87] us > percentile(99): 1.38 us > mode: 0.41 us > > It represents -57%. It

Re: row filtering for logical replication

2021-07-11 Thread Tomas Vondra
Hi, I took a look at this patch, which seems to be in CF since 2018. I have only some basic comments and observations at this point: 1) alter_publication.sgml I think "expression is executed" sounds a bit strange, perhaps "evaluated" would be better? 2) create_publication.sgml Why is the

Re: proposal - psql - use pager for \watch command

2021-07-11 Thread Thomas Munro
On Sun, Jul 11, 2021 at 1:18 AM vignesh C wrote: > On Wed, May 12, 2021 at 5:45 PM Pavel Stehule wrote: > > looks so with your patch psql doesn't work on ms Here's a fix for Windows. The pqsignal() calls are #ifdef'd out. I also removed a few lines that were added after commit 3a513067 but

Re: Protect against possible memory corruption (src/backend/access/nbtree/nbtxlog.c)

2021-07-11 Thread Heikki Linnakangas
On 11/07/2021 22:51, Ranier Vilela wrote: Hi, While analyzing a possible use of an uninitialized variable, I checked that *_bt_restore_page* can lead to memory corruption, by not checking the maximum limit of array items which is MaxIndexTuplesPerPage. + /* Protect against corrupted

Protect against possible memory corruption (src/backend/access/nbtree/nbtxlog.c)

2021-07-11 Thread Ranier Vilela
Hi, While analyzing a possible use of an uninitialized variable, I checked that *_bt_restore_page* can lead to memory corruption, by not checking the maximum limit of array items which is MaxIndexTuplesPerPage. It can also generate a dangling pointer by incrementing it beyond the limits it can

Re: row filtering for logical replication

2021-07-11 Thread Euler Taveira
On Mon, Jul 5, 2021, at 12:14 AM, Greg Nancarrow wrote: > I have some review comments on the "Row filter for logical replication" patch: > > (1) Suggested update to patch comment: > (There are some missing words and things which could be better expressed) I incorporated all your wording

Re: row filtering for logical replication

2021-07-11 Thread Euler Taveira
On Fri, Jul 2, 2021, at 4:29 AM, Peter Smith wrote: > Hi. > > I have been looking at the latest patch set (v16). Below are my review > comments and some patches. > Peter, thanks for your detailed review. Comments are inline. > 1. Patch 0001 comment - typo > > you can optionally filter rows

Re: pg_stats and range statistics

2021-07-11 Thread Soumyadeep Chakraborty
Hello, This should have been added with [1]. Excerpt from the documentation: "pg_stats is also designed to present the information in a more readable format than the underlying catalog — at the cost that its schema must be extended whenever new slot types are defined for pg_statistic." [2] So,

Re: enable_resultcache confusion

2021-07-11 Thread Justin Pryzby
On Mon, Jul 12, 2021 at 02:56:58AM +1200, David Rowley wrote: > On Sat, 10 Jul 2021 at 07:30, Robert Haas wrote: > > > > On Fri, Jul 9, 2021 at 11:35 AM David Rowley wrote: > > > I really like that name. > > > > > > I'll wait to see if anyone else wants to voice their opinion before I > > > do

Re: Slow standby snapshot

2021-07-11 Thread Michail Nikolaev
Hello, Kirill. > Also, maybe it is better to reduce the invasivity by using a more > simple approach. For example, use the first bit to mark xid as valid > and the last 7 bit (128 values) as an optimistic offset to the next > valid xid (jump by 127 steps in the worse scenario). > What do you

Re: Enhanced error message to include hint messages for redundant options error

2021-07-11 Thread vignesh C
.On Sun, Jul 11, 2021 at 2:57 PM Dean Rasheed wrote: > > On Sat, 10 Jul 2021 at 17:03, vignesh C wrote: > > > > > Also, I don't think this function should be marked inline -- using a > > > normal function ought to help make the compiled code smaller. > > > > inline instructs the compiler to

Re: pgbench logging broken by time logic changes

2021-07-11 Thread Fabien COELHO
Hello Thomas, I committed the code change without the new TAP tests, because I didn't want to leave the open item hanging any longer. Ok. Good. As for the test, ... [...] Argh, so there are no tests that would have caught the regressions:-( ... I know it can fail, and your v18 didn't

Re: psql - factor out echo code

2021-07-11 Thread vignesh C
On Sat, Jul 10, 2021 at 10:25 PM Fabien COELHO wrote: > > > Hello Vignesh, > > > I am changing the status to "Needs review" as the review is not > > completed for this patch and also there are some tests failing, that > > need to be fixed: > > test test_extdepend ... FAILED 50

Re: Enhanced error message to include hint messages for redundant options error

2021-07-11 Thread Dean Rasheed
On Sat, 10 Jul 2021 at 18:09, vignesh C wrote: > > I'm planning to handle conflicting errors separately after this > current work is done, once the patch is changed to have just the valid > scenarios(removing the scenarios you have pointed out) existing > function can work as is without any

Re: Enhanced error message to include hint messages for redundant options error

2021-07-11 Thread Dean Rasheed
On Sat, 10 Jul 2021 at 17:03, vignesh C wrote: > > > Also, I don't think this function should be marked inline -- using a > > normal function ought to help make the compiled code smaller. > > inline instructs the compiler to attempt to embed the function content > into the calling code instead of

Re: pgbench logging broken by time logic changes

2021-07-11 Thread Thomas Munro
Hi Fabien, I committed the code change without the new TAP tests, because I didn't want to leave the open item hanging any longer. As for the test, ... On Sat, Jul 10, 2021 at 9:36 PM Fabien COELHO wrote: > >> I hoped we were done here but I realised that your check for 1-3 log > >> lines will