Re: [HACKERS] Custom compression methods

2021-03-24 Thread Dilip Kumar
On Wed, Mar 24, 2021 at 3:40 PM Dilip Kumar wrote: > > 0001 ->shows compression method for the index attribute in index describe > 0002 -> fix the reported bug (test case included) > > Apart from this, I was thinking that currently, we are allowing to > ALTER SET COMPRE

Re: [HACKERS] Custom compression methods

2021-03-24 Thread Dilip Kumar
On Wed, Mar 24, 2021 at 8:41 PM Robert Haas wrote: > > On Wed, Mar 24, 2021 at 7:45 AM Dilip Kumar wrote: > > I have anyway created a patch for this as well. Including all three > > patches so we don't lose track. > > > > 0001 ->shows compression method for

Re: [HACKERS] Custom compression methods

2021-03-24 Thread Dilip Kumar
On Wed, Mar 24, 2021 at 9:32 PM Robert Haas wrote: > > On Wed, Mar 24, 2021 at 11:41 AM Dilip Kumar wrote: > > Okay, that sounds like a reasonable design idea. But the problem is > > that in index_form_tuple we only have index tuple descriptor, not the > > heap tuple d

Re: [HACKERS] Custom compression methods

2021-03-25 Thread Dilip Kumar
On Wed, Mar 24, 2021 at 10:57 PM Robert Haas wrote: > > On Wed, Mar 24, 2021 at 12:14 PM Dilip Kumar wrote: > > Actually, we are already doing this, I mean ALTER TABLE .. ALTER > > COLUMN .. SET COMPRESSION is already updating the compression method > > of the index att

Re: Allow ERROR from heap_prepare_freeze_tuple to be downgraded to WARNING

2020-07-20 Thread Dilip Kumar
On Mon, Jul 20, 2020 at 10:14 PM Alvaro Herrera wrote: > > On 2020-Jul-20, Dilip Kumar wrote: > > > On Fri, Jul 17, 2020 at 4:16 PM Dilip Kumar wrote: > > > > So if the vacuum_tolerate_damage is set then in > > > all the cases in heap_prepare_fre

Re: Allow ERROR from heap_prepare_freeze_tuple to be downgraded to WARNING

2020-07-20 Thread Dilip Kumar
On Tue, Jul 21, 2020 at 2:00 AM Andres Freund wrote: > > Hi, > > On 2020-07-17 16:16:23 +0530, Dilip Kumar wrote: > > The attached patch allows the vacuum to continue by emitting WARNING > > for the corrupted tuple instead of immediately error out as discussed > >

Re: Allow ERROR from heap_prepare_freeze_tuple to be downgraded to WARNING

2020-07-21 Thread Dilip Kumar
On Tue, Jul 21, 2020 at 11:00 AM Dilip Kumar wrote: > > On Tue, Jul 21, 2020 at 2:00 AM Andres Freund wrote: > > > > Hi, > > > > On 2020-07-17 16:16:23 +0530, Dilip Kumar wrote: > > > The attached patch allows the vacuum to continue by emitting WARNING &

Re: Allow ERROR from heap_prepare_freeze_tuple to be downgraded to WARNING

2020-07-21 Thread Dilip Kumar
On Tue, Jul 21, 2020 at 4:08 PM Dilip Kumar wrote: > > On Tue, Jul 21, 2020 at 11:00 AM Dilip Kumar wrote: > > > > On Tue, Jul 21, 2020 at 2:00 AM Andres Freund wrote: > > > > > > Hi, > > > > > > On 2020-07-17 16:16:23 +0530, Dilip Kumar

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-07-25 Thread Dilip Kumar
On Sat, Jul 25, 2020 at 5:08 PM Amit Kapila wrote: > > On Fri, Jul 24, 2020 at 7:17 PM Dilip Kumar wrote: > > > > Your changes look fine to me. Additionally, I have changed a test > > case of getting the streaming changes in 0002. Instead of just > > showing th

Parallel bitmap index scan

2020-07-26 Thread Dilip Kumar
BitmapAnd or BitmapOr path then I did not parallelize the underlying bitmap index scan. I think for BitmapAnd and BitmapOr we should use a completely different design, something similar to what we are doing in parallel append so I don't think BitmapAnd and BitmapOr we need to cover under this

Re: Parallel bitmap index scan

2020-07-26 Thread Dilip Kumar
On Sun, Jul 26, 2020 at 6:42 PM Dilip Kumar wrote: > > I would like to propose a patch for enabling the parallelism for the > bitmap index scan path. > > Background: > Currently, we support only a parallel bitmap heap scan path. Therein, > the underlying bitmap index sca

Re: Parallel bitmap index scan

2020-07-26 Thread Dilip Kumar
On Mon, 27 Jul 2020 at 3:48 AM, Thomas Munro wrote: > On Mon, Jul 27, 2020 at 1:58 AM Dilip Kumar wrote: > > On Sun, Jul 26, 2020 at 6:42 PM Dilip Kumar > wrote: > > > > > > I would like to propose a patch for enabling the parallelism for th

Re: Parallel bitmap index scan

2020-07-28 Thread Dilip Kumar
On Mon, Jul 27, 2020 at 3:48 AM Thomas Munro wrote: > > On Mon, Jul 27, 2020 at 1:58 AM Dilip Kumar wrote: > > On Sun, Jul 26, 2020 at 6:42 PM Dilip Kumar wrote: > > > > > > I would like to propose a patch for enabling the parallelism for th

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-08-07 Thread Dilip Kumar
On Thu, Aug 6, 2020 at 2:43 PM Amit Kapila wrote: > > On Wed, Aug 5, 2020 at 7:37 PM Dilip Kumar wrote: > > > > On Wed, Aug 5, 2020 at 6:25 PM Amit Kapila wrote: > > > > > > > > Can we add a test for incomplete changes (probably with toast > > &

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-08-15 Thread Dilip Kumar
On Thu, Aug 13, 2020 at 6:47 PM Amit Kapila wrote: > > On Thu, Aug 13, 2020 at 12:08 PM Amit Kapila wrote: > > > > On Fri, Aug 7, 2020 at 2:04 PM Dilip Kumar wrote: > > > > > > On Thu, Aug 6, 2020 at 2:43 PM Amit Kapila > > > wrote: > > &

Re: Parallel bitmap index scan

2020-08-17 Thread Dilip Kumar
On Mon, 17 Aug 2020 at 7:42 PM, Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > On Sun, Jul 26, 2020 at 6:43 PM Dilip Kumar wrote: > > > > > > I would like to propose a patch for enabling the parallelism for the > > > bitmap ind

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-08-19 Thread Dilip Kumar
On Wed, Aug 19, 2020 at 10:11 AM Amit Kapila wrote: > > On Mon, Aug 17, 2020 at 6:29 PM Dilip Kumar wrote: > > > > > > In last patch v49-0001, there is one issue, Basically, I have called > > BufFileFlush in all the cases. But, ideally, we can not call this i

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-08-19 Thread Dilip Kumar
On Wed, Aug 19, 2020 at 12:20 PM Amit Kapila wrote: > > On Wed, Aug 19, 2020 at 10:10 AM Amit Kapila wrote: > > > > On Mon, Aug 17, 2020 at 6:29 PM Dilip Kumar wrote: > > > > > > > > > In last patch v49-0001, there is one issue, Basically, I ha

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-08-20 Thread Dilip Kumar
On Wed, Aug 19, 2020 at 1:35 PM Dilip Kumar wrote: > > On Wed, Aug 19, 2020 at 12:20 PM Amit Kapila wrote: > > > > On Wed, Aug 19, 2020 at 10:10 AM Amit Kapila > > wrote: > > > > > > On Mon, Aug 17, 2020 at 6:29 PM Dilip Kumar wrote: > > >

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-08-20 Thread Dilip Kumar
On Fri, Aug 21, 2020 at 9:14 AM Amit Kapila wrote: > > On Thu, Aug 20, 2020 at 5:42 PM Dilip Kumar wrote: > > > > On Thu, Aug 20, 2020 at 2:30 PM Amit Kapila wrote: > > > > > > > > > Right, I think this can happen if one has changed those by BufFi

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-08-20 Thread Dilip Kumar
On Fri, Aug 21, 2020 at 10:20 AM Amit Kapila wrote: > > On Fri, Aug 21, 2020 at 9:14 AM Amit Kapila wrote: > > > > On Thu, Aug 20, 2020 at 5:42 PM Dilip Kumar wrote: > > > > > > On Thu, Aug 20, 2020 at 2:30 PM Amit Kapila > > > wrote: > > &g

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-08-21 Thread Dilip Kumar
On Fri, Aug 21, 2020 at 3:13 PM Amit Kapila wrote: > > On Fri, Aug 21, 2020 at 10:33 AM Dilip Kumar wrote: > > > > On Fri, Aug 21, 2020 at 9:14 AM Amit Kapila wrote: > > > > > > 2. > > > + /* > > > + * If the new location is smaller then

Re: Re: [HACKERS] Custom compression methods

2020-08-23 Thread Dilip Kumar
On Thu, Aug 13, 2020 at 5:18 PM Dilip Kumar wrote: > There was some question which Robert has asked in this mail, please find my answer inline. Also, I have a few questions regarding further splitting up this patch. > On Fri, Jun 19, 2020 at 10:33 PM Robert Haas wrote: > > &

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-08-24 Thread Dilip Kumar
On Sat, Aug 22, 2020 at 8:38 AM Amit Kapila wrote: > > On Fri, Aug 21, 2020 at 5:10 PM Dilip Kumar wrote: > > > > I have reviewed and tested the patch and the changes look fine to me. > > > > Thanks, I will push the next patch early next week (by Tuesday) unless

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-08-24 Thread Dilip Kumar
On Tue, Aug 25, 2020 at 9:31 AM Amit Kapila wrote: > > On Mon, Aug 24, 2020 at 9:41 PM Dilip Kumar wrote: > > > > On Sat, Aug 22, 2020 at 8:38 AM Amit Kapila wrote: > > > > > > On Fri, Aug 21, 2020 at 5:10 PM Dilip Kumar wrote: > > > > >

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-08-25 Thread Dilip Kumar
On Tue, Aug 25, 2020 at 6:27 PM Amit Kapila wrote: > > On Tue, Aug 25, 2020 at 10:41 AM Dilip Kumar wrote: > > > > On Tue, Aug 25, 2020 at 9:31 AM Amit Kapila wrote: > > > > > > > > > I think the existing design is superior as it allows the fle

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-08-28 Thread Dilip Kumar
ound PG_USED_FOR_ASSERTS_ONLY = false; > > > > Thanks for the report. Tom Lane has already fixed this [1]. > > [1] - > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=e942af7b8261cd8070d0eeaf518dbc1a664859fd As discussed, I have added a another test ca

Re: Allow ERROR from heap_prepare_freeze_tuple to be downgraded to WARNING

2020-08-29 Thread Dilip Kumar
On Fri, Aug 28, 2020 at 9:49 PM Robert Haas wrote: > > On Tue, Jul 21, 2020 at 9:21 AM Dilip Kumar wrote: > > In the previous version, the feature was enabled for cluster/vacuum > > full command as well. in the attached patch I have enabled it only > > if we are run

Re: Allow ERROR from heap_prepare_freeze_tuple to be downgraded to WARNING

2020-08-29 Thread Dilip Kumar
f the second tuple as we have frozen the second tuple. But, I feel this is easily fixable right? I mean instead of not doing anything to the corrupted tuple we can partially freeze it? I mean we can just leave the corrupted xid alone but mark the other xid as frozen if that is smaller then cutoff_xid. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-09-02 Thread Dilip Kumar
On Tue, Sep 1, 2020 at 8:33 PM Amit Kapila wrote: > > On Tue, Sep 1, 2020 at 9:28 AM Amit Kapila wrote: > > > > On Mon, Aug 31, 2020 at 7:28 PM Dilip Kumar wrote: > > > > > > On Mon, Aug 31, 2020 at 1:24 PM Amit Kapila > > > wrote: &g

Re: [HACKERS] Custom compression methods

2020-09-02 Thread Dilip Kumar
On Wed, Sep 2, 2020 at 4:57 AM Mark Dilger wrote: > > > > > On Aug 13, 2020, at 4:48 AM, Dilip Kumar wrote: > > > > v1-0001: As suggested by Robert, it provides the syntax support for > > setting the compression method for a column while creating a table and

Re: Re: [HACKERS] Custom compression methods

2020-09-02 Thread Dilip Kumar
On Mon, Aug 31, 2020 at 10:45 AM Amit Khandekar wrote: > > On Thu, 13 Aug 2020 at 17:18, Dilip Kumar wrote: > > I have rebased the patch on the latest head and currently, broken into 3 > > parts. > > > > v1-0001: As suggested by Robert, it provides the synt

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-09-02 Thread Dilip Kumar
On Wed, Sep 2, 2020 at 7:19 PM Amit Kapila wrote: > On Wed, Sep 2, 2020 at 3:41 PM Dilip Kumar wrote: > > > > On Wed, Sep 2, 2020 at 10:55 AM Amit Kapila > wrote: > > > > > > > > > > > > > We can combine the tests in 015_stream_simpl

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-09-05 Thread Dilip Kumar
d that in one of the existing tests in > > 018_stream_subxact_abort.pl. I have added one test for Rollback, > > changed few messages, removed one test case which was not making any > > sense in the patch. See attached and let me know what you think about > > it? I have reviewed the changes and looks fine to me. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-09-07 Thread Dilip Kumar
On Mon, Sep 7, 2020 at 12:00 PM Amit Kapila wrote: > On Sat, Sep 5, 2020 at 8:55 PM Dilip Kumar wrote: > > > > > > I have reviewed the changes and looks fine to me. > > > > Thanks, I have pushed the last patch. Let's wait for a day or so to > see the b

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-09-09 Thread Dilip Kumar
typo in one of the comments > > Patch attached. > Looks good to me. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Bug in logical decoding of in-progress transactions

2020-09-09 Thread Dilip Kumar
a -- opening a streamed block for transaction TXN 508 closing a streamed block for transaction TXN 508 opening a streamed block for transaction TXN 510 streaming change for TXN 510 closing a streamed block for transaction TXN 510 (5 rows) After patch d

Re: Bug in logical decoding of in-progress transactions

2020-09-09 Thread Dilip Kumar
On Thu, Sep 10, 2020 at 11:47 AM Amit Kapila wrote: > On Thu, Sep 10, 2020 at 11:42 AM Dilip Kumar > wrote: > > > > On Thu, Sep 10, 2020 at 11:29 AM Amit Kapila > wrote: > >> > >> Hi, > >> > >> There is a recent build farm failure [1]

Re: Bug in logical decoding of in-progress transactions

2020-09-09 Thread Dilip Kumar
On Thu, Sep 10, 2020 at 11:53 AM Dilip Kumar wrote: > On Thu, Sep 10, 2020 at 11:47 AM Amit Kapila > wrote: > >> On Thu, Sep 10, 2020 at 11:42 AM Dilip Kumar >> wrote: >> > >> > On Thu, Sep 10, 2020 at 11:29 AM Amit Kapila >> wrote: >> &g

Re: Bug in logical decoding of in-progress transactions

2020-09-10 Thread Dilip Kumar
On Thu, Sep 10, 2020 at 2:48 PM Amit Kapila wrote: > On Thu, Sep 10, 2020 at 12:00 PM Dilip Kumar > wrote: > > > > On Thu, Sep 10, 2020 at 11:53 AM Dilip Kumar > wrote: > >> > >>> > > >>> > I have written a test case to reprodu

Error "initial slot snapshot too large" in create replication slot

2021-10-10 Thread Dilip Kumar
transaction abort, for this, a simple fix is we can clear this state on the transaction abort, maybe I will raise this as a separate issue? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com test.sql Description: application/sql

Re: Error "initial slot snapshot too large" in create replication slot

2021-10-11 Thread Dilip Kumar
On Mon, Oct 11, 2021 at 4:29 PM Kyotaro Horiguchi wrote: > > At Mon, 11 Oct 2021 11:49:41 +0530, Dilip Kumar wrote > in > > While creating an "export snapshot" I don't see any protection why the > > number of xids in the snapshot can not c

Re: row filtering for logical replication

2021-10-11 Thread Dilip Kumar
e for expression evaluation machinery and later the expression evaluation we do the deform again. So why don't you use the deformed tuple as it is to store as a virtual tuple? Infact, if newtup_changed is true then you are forming back the tuple just to get it deformed again in the expres

Reset snapshot export state on the transaction abort

2021-10-11 Thread Dilip Kumar
tch clears this state if the transaction is aborted. [1] https://www.postgresql.org/message-id/CAFiTN-tqopqpfS6HHug2nnOGieJJ_nm-Nvy0WBZ=zpo-lqt...@mail.gmail.com -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com From 673bea5121f0a6acadb1304e4a2b1027f5914e72 Mon Sep 17 00:00:00

Re: Error "initial slot snapshot too large" in create replication slot

2021-10-11 Thread Dilip Kumar
in mind, I will do a detailed review in a day or two. Thanks for working on this. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Question about building an exportable snapshop

2021-10-12 Thread Dilip Kumar
subxid)) { sub_needs_timetravel = true; needs_snapshot = true; [3] [1] https://www.postgresql.org/message-id/CAFiTN-tqopqpfS6HHug2nnOGieJJ_nm-Nvy0WBZ%3DZpo-LqtSJA%40mail.gmail.com -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Gather performance analysis

2021-10-12 Thread Dilip Kumar
On Tue, Oct 12, 2021 at 6:41 PM Tomas Vondra wrote: > > On 9/28/21 14:00, Dilip Kumar wrote: > > > > I think that would be great, can we just test this specific target > > where we are seeing a huge dip with the patch, e.g. > > with 1000 rows, 10 columns and 4 th

Re: Reset snapshot export state on the transaction abort

2021-10-12 Thread Dilip Kumar
On Wed, Oct 13, 2021 at 10:17 AM Michael Paquier wrote: > > On Mon, Oct 11, 2021 at 08:46:32PM +0530, Dilip Kumar wrote: > > As reported at [1], if the transaction is aborted during export > > snapshot then ExportInProgress and SavedResourceOwnerDuringExport are > > not g

Re: BUG #17220: ALTER INDEX ALTER COLUMN SET (..) with an optionless opclass makes index and table unusable

2021-10-13 Thread Dilip Kumar
for the index and if someone tries to set with the column name then it should throw an error. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Reset snapshot export state on the transaction abort

2021-10-14 Thread Dilip Kumar
On Thu, Oct 14, 2021 at 12:24 PM Michael Paquier wrote: > > On Wed, Oct 13, 2021 at 10:53:24AM +0530, Dilip Kumar wrote: > > Actually, it is not required because 1) Snapshot export can not be > > allowed within a transaction block, basically, it starts its own > > tran

Re: [Bug] Logical Replication failing if the DateStyle is different in Publisher & Subscriber

2021-10-14 Thread Dilip Kumar
21-10-14 00:59:23.067 PDT [38262] HINT: Perhaps you need a > different "datestyle" setting. > > Is this an expected behavior? Looks like a problem to me, I think for fixing this, on logical replication connection always set subscriber's DateStlyle, with that the

Re: Reset snapshot export state on the transaction abort

2021-10-16 Thread Dilip Kumar
ildClearExportedSnapshot because that is anyway being called from the AbortTransaction. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com From 062a84dcd821fa5e41998c8714f8ec16befcf983 Mon Sep 17 00:00:00 2001 From: Dilip Kumar Date: Mon, 11 Oct 2021 20:38:58 +0530 Subject: [PATC

Re: [Bug] Logical Replication failing if the DateStyle is different in Publisher & Subscriber

2021-10-18 Thread Dilip Kumar
7; on subscriber, it works fine. > Please try v3 patch and let me know if they work as unexpected. > Thanks in advance. I think the idea of setting the standard DateStyle and the IntervalStyle on the walsender process looks fine to me. As this will avoid extra network round trips as Tom m

Re: Data is copied twice when specifying both child and parent table in publication

2021-10-18 Thread Dilip Kumar
sider this table only once and there will be no duplicate data. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Reset snapshot export state on the transaction abort

2021-10-18 Thread Dilip Kumar
ckpatched the fix. Thanks! -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Error "initial slot snapshot too large" in create replication slot

2021-10-19 Thread Dilip Kumar
On Tue, Oct 12, 2021 at 11:30 AM Dilip Kumar wrote: > > On Tue, Oct 12, 2021 at 10:35 AM Kyotaro Horiguchi > wrote: > > > > At Tue, 12 Oct 2021 13:59:59 +0900 (JST), Kyotaro Horiguchi > > wrote in > > > So I came up with the attached version. > >

Re: pgsql: Document XLOG_INCLUDE_XID a little better

2021-10-19 Thread Dilip Kumar
caller should also be changed accordingly. Other changes look fine. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Data is copied twice when specifying both child and parent table in publication

2021-10-20 Thread Dilip Kumar
ot=true. Note that the corresponding table > in the subscriber may well be a non-partitioned table (or the > partitions arranged differently) so the data does need to be > replicated again. I don't think this behavior is consistent, I mean for the initial sync we will replicate the duplicate data, whereas for later streaming we will only replicate it once. From the user POW, this behavior doesn't look correct. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Question about building an exportable snapshop

2021-10-20 Thread Dilip Kumar
On Wed, Oct 20, 2021 at 5:06 PM Amit Kapila wrote: > > On Tue, Oct 12, 2021 at 6:21 PM Dilip Kumar wrote: > > > > While working on the issue [1], I realize that if a subtransaction > > hasn't done any catalog change then we don't add this in the commit > >

Re: pgsql: Document XLOG_INCLUDE_XID a little better

2021-10-20 Thread Dilip Kumar
On Wed, Oct 20, 2021 at 4:17 PM Amit Kapila wrote: > > On Wed, Oct 20, 2021 at 10:25 AM Dilip Kumar wrote: > > > > I have one comment here, basically, you have changed the function name > > to "IsTopTransactionIdLogged", but it still behaves like > > IsT

Re: pgsql: Document XLOG_INCLUDE_XID a little better

2021-10-20 Thread Dilip Kumar
e inside XLogInsertRecord's > critical section? I think this function is doing somewhat similar things to what we are doing in MarkCurrentTransactionIdLoggedIfAny() so put at the same place. But I don't see any reason for this to be in the critical section. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: pgsql: Document XLOG_INCLUDE_XID a little better

2021-10-20 Thread Dilip Kumar
On Wed, Oct 20, 2021 at 9:46 PM Alvaro Herrera wrote: > > On 2021-Oct-20, Dilip Kumar wrote: > > > On Wed, Oct 20, 2021 at 7:09 PM Alvaro Herrera > > wrote: > > > > In IsTopTransactionIdLogPending(), I suggest to reorder the tests so > > > that the fast

Re: pgsql: Document XLOG_INCLUDE_XID a little better

2021-10-20 Thread Dilip Kumar
On Thu, Oct 21, 2021 at 9:11 AM Amit Kapila wrote: > > On Wed, Oct 20, 2021 at 8:49 PM Dilip Kumar wrote: > > > > On Wed, Oct 20, 2021 at 7:09 PM Alvaro Herrera > > wrote: > > > > > > Does MarkTopTransactionIdLogged() have to be inside XLogInsertRecord

Re: [Bug] Logical Replication failing if the DateStyle is different in Publisher & Subscriber

2021-10-20 Thread Dilip Kumar
On Thu, Oct 21, 2021 at 11:16 AM Masahiko Sawada wrote: > > On Wed, Oct 20, 2021 at 8:12 PM Japin Li wrote: > > > > > > On Mon, 18 Oct 2021 at 17:27, Dilip Kumar wrote: > > > On Mon, Oct 18, 2021 at 1:41 PM Japin Li wrote: > > > > > >> I

Re: Gather performance analysis

2021-10-26 Thread Dilip Kumar
On Tue, Oct 26, 2021 at 2:31 PM Amit Kapila wrote: > > On Fri, Oct 15, 2021 at 1:48 AM Robert Haas wrote: > > > > On Tue, Oct 12, 2021 at 10:14 AM Dilip Kumar wrote: > > > Thanks, yeah now it looks in line with other results. > > > > Since it seems there

Re: pgsql: Document XLOG_INCLUDE_XID a little better

2021-10-27 Thread Dilip Kumar
On Tue, Oct 26, 2021 at 9:19 AM Amit Kapila wrote: > > On Mon, Oct 25, 2021 at 4:21 PM Amit Kapila wrote: > > > > On Thu, Oct 21, 2021 at 11:20 AM Dilip Kumar wrote: > > > > > > On Thu, Oct 21, 2021 at 9:11 AM Amit Kapila > > > wrote: > > &g

Re: Isn't it better with "autovacuum worker...." instead of "worker took too long to start; canceled" specific to "auto

2021-10-27 Thread Dilip Kumar
cific and say "Autovacuum". Attached > patch does that. Please review it. +1, the error message and other improvements look good. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Add connection active, idle time to pg_stat_activity

2021-11-02 Thread Dilip Kumar
TED should be accumulated in the "pgStatTransactionIdleInTxnTime" counter or there should be a separate counter for that. But after your patch we can not accumulate this in the "pgStatTransactionIdleTime" counter. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Error "initial slot snapshot too large" in create replication slot

2021-11-02 Thread Dilip Kumar
On Tue, Oct 19, 2021 at 2:25 PM Dilip Kumar wrote: > > On Tue, Oct 12, 2021 at 11:30 AM Dilip Kumar wrote: > > > > On Tue, Oct 12, 2021 at 10:35 AM Kyotaro Horiguchi > > wrote: > > > > > > At Tue, 12 Oct 2021 13:59:59 +0900 (JST), Kyotaro Horiguchi >

Re: Skipping logical replication transactions on subscriber side

2021-11-08 Thread Dilip Kumar
al view does not contain this column. 6. + +Resets statistics of a single subscription worker statistics. /Resets statistics of a single subscription worker statistics/Resets statistics of a single subscription worker -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Skipping logical replication transactions on subscriber side

2021-11-08 Thread Dilip Kumar
On Tue, Nov 9, 2021 at 11:57 AM Amit Kapila wrote: > > On Tue, Nov 9, 2021 at 11:37 AM Dilip Kumar wrote: > > 1. > > I don't like the fact that this view is very specific for showing the > > errors but the name of the view is very generic. So are we keeping > &g

Re: Replication & recovery_min_apply_delay

2021-11-09 Thread Dilip Kumar
on is returning the size, but actually it either returns -1 or returns XLOG_BLCKSZ. So logically this function should just be returning boolean, true if it is able to read the first page, or false otherwise. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Failed transaction statistics to measure the logical replication progress

2021-11-09 Thread Dilip Kumar
is why are we trying to use this as rvalue here? This is clearly an issue. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Add connection active, idle time to pg_stat_activity

2021-11-10 Thread Dilip Kumar
On Tue, Nov 9, 2021 at 8:28 PM Rafia Sabih wrote: > > On Tue, 2 Nov 2021 at 09:00, Dilip Kumar wrote: > > > > About the patch, IIUC earlier all the idle time was accumulated in the > > "pgStatTransactionIdleTime" counter, now with your patch you have >

Re: Replication & recovery_min_apply_delay

2021-11-13 Thread Dilip Kumar
then we might fetch the WAL using both walreceiver as well as from archive right? because we are not changing the currentsource. Is this intentional or do we want to change the currentsource as well? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Printing backtrace of postgres processes

2021-11-14 Thread Dilip Kumar
ace generation is not supported by this installation + pg_print_backtrace + + f Should we give some hints on how to enable that? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Printing backtrace of postgres processes

2021-11-14 Thread Dilip Kumar
On Mon, Nov 15, 2021 at 11:53 AM Bharath Rupireddy wrote: > > On Mon, Nov 15, 2021 at 11:37 AM Dilip Kumar wrote: > > > > On Mon, Nov 15, 2021 at 10:34 AM vignesh C wrote: > > > > > > > > Thanks for the comments, the attached v12 patch has the c

Re: row filtering for logical replication

2021-11-15 Thread Dilip Kumar
lation, HeapTuple oldtuple, HeapTuple newtuple, RelationSyncEntry *entry, ReorderBufferChangeType *action) +{ This function definition header is too long to fit in one line, so better to break it. I think running will be a good idea. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Add connection active, idle time to pg_stat_activity

2021-11-15 Thread Dilip Kumar
d to "pgStatIdleTime" and pgStatTransactionIdleInTxnTime should be renamed to "pgStatTransactionIdleTime" -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: row filtering for logical replication

2021-11-15 Thread Dilip Kumar
On Mon, 15 Nov 2021 at 3:07 PM, Amit Kapila wrote: > On Mon, Nov 15, 2021 at 2:44 PM Dilip Kumar wrote: > > > > On Fri, Nov 12, 2021 at 3:49 PM Ajin Cherian wrote: > > > > > > > This function definition header is too long to fit in one line, so > > be

Re: Add connection active, idle time to pg_stat_activity

2021-11-15 Thread Dilip Kumar
On Mon, Nov 15, 2021 at 4:46 PM Rafia Sabih wrote: > > On Mon, 15 Nov 2021 at 10:24, Dilip Kumar wrote: > > > > On Wed, Nov 10, 2021 at 1:47 PM Rafia Sabih > > wrote: > > > > > > > It seems that in beentry->st_idle_time, you want to compute the &

Re: row filtering for logical replication

2021-11-16 Thread Dilip Kumar
On Mon, Nov 15, 2021 at 2:44 PM Dilip Kumar wrote: > > On Fri, Nov 12, 2021 at 3:49 PM Ajin Cherian wrote: > > > > Attaching version 39- I have reviewed, 0001* and I have a few comments on it --- >If you choose to do the initial table synchronization, only data that satis

Re: Replication & recovery_min_apply_delay

2021-11-22 Thread Dilip Kumar
On Mon, Nov 22, 2021 at 4:25 PM Bharath Rupireddy wrote: > > On Sun, Nov 14, 2021 at 11:45 AM Dilip Kumar wrote: > > > > On Wed, Oct 27, 2021 at 1:01 AM Mahendra Singh Thalor > > wrote: > > > > I was going through and patch and trying to understand the

Re: row filtering for logical replication

2021-11-22 Thread Dilip Kumar
o can you add the comments explaining why we are making a copy here. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Sequence's value can be rollback after a crashed recovery.

2021-11-22 Thread Dilip Kumar
> If you can show that the sequence goes back after a commit, that'd be an > actual durability issue. I think at this thread[1], which claimed to get this issue even after commit, I haven't tried it myself though. [1] https://www.postgresql.org/message-id/flat/ea6485e3-98d0-24a7-094c-87f9d5f9b18f%40amazon.com#4cfe7217c829419b769339465e8c2915 -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-11-23 Thread Dilip Kumar
nning the raw file to identify which files to copy so seems cleaner to me 2) with 0007 if we directly scan directory we don't know the relation oid, so before acquiring the buffer lock there is no way to acquire the relation lock. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Rename dead_tuples to dead_items in vacuumlazy.c

2021-11-23 Thread Dilip Kumar
LEXIBLE_ARRAY_MEMBER]; Instead of just dead, why not deadtid or deaditem? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [BUG]Missing REPLICA IDENTITY check when DROP NOT NULL

2021-11-24 Thread Dilip Kumar
m. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: pg_dump, pg_basebackup don't error out with wrong option for "--format"

2021-11-24 Thread Dilip Kumar
file and --format will conflict, --foo is also not allowed because it is not a valid prefix string of any valid option string. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-11-25 Thread Dilip Kumar
On Thu, Nov 25, 2021 at 1:07 PM Greg Nancarrow wrote: > > On Tue, Oct 5, 2021 at 7:07 PM Dilip Kumar wrote: > > > > Patch details: > > 0001 to 0006 implements an approach1 > > 0007 removes the code of pg_class scanning and adds the directory scan. > > > >

Re: Add connection active, idle time to pg_stat_activity

2021-11-26 Thread Dilip Kumar
dle_in_xact_time. > > Got it. > Updated Okay, thanks, I will look into it one more time early next week and if I see no issues then I will move it to RFC. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Synchronizing slots from primary to standby

2021-11-28 Thread Dilip Kumar
ary/publisher? How can it ensure that the > latest information exists say when the subscriber is down/crashed > before it picks up the latest slot information? Yeah that is a good question that how frequently the subscriber should fetch the slot information, I think that should be configurable values. And the time delay is more, the chances of losing the latest slot is more. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Add connection active, idle time to pg_stat_activity

2021-11-28 Thread Dilip Kumar
hey get overflowed? This is a 64-bit variable so I am not sure do we really need to worry about overflow? I mean if we are storing microseconds then also this will be able to last for ~300,000 years no? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Synchronizing slots from primary to standby

2021-11-28 Thread Dilip Kumar
On Mon, Nov 29, 2021 at 12:19 PM Bharath Rupireddy wrote: > > On Mon, Nov 29, 2021 at 11:14 AM Dilip Kumar wrote: > > > > On Mon, Nov 29, 2021 at 9:40 AM Bharath Rupireddy > > wrote: > > > > > > On Mon, Nov 29, 2021 at 1:48 AM SATYANARAYANA NARLAPURAM

Re: row filtering for logical replication

2021-11-29 Thread Dilip Kumar
t;, so in this case, we will apply only the update filter i.e. a > 1 so as per that this will become the INSERT operation because the old row was not passing the filter. So now we will insert a new row in the subscriber-side with value (2,3). Looks a bit odd to me that the value b=3 would have been rejected with the direct insert but it is allowed due to indirect insert done by update. Is this behavior looks odd only to me? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: row filtering for logical replication

2021-11-29 Thread Dilip Kumar
think for inserts (new row) we can consider the insert > filter as well but that makes it tricky to explain. I feel we can > change it later as well if there is a valid use case for this. What do > you think? Yeah, that makes sense. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Switching XLog source from archive to streaming when primary available

2021-11-29 Thread Dilip Kumar
en if the WAL is locally available and we don't really need more WAL but that is controlled by a GUC. [1] https://www.postgresql.org/message-id/CAKYtNApe05WmeRo92gTePEmhOM4myMpCK_%2BceSJtC7-AWLw1qw%40mail.gmail.com -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Can I assume relation would not be invalid during from ExecutorRun to ExecutorEnd

2021-11-29 Thread Dilip Kumar
ore we access them they are initialized, and most of them are allocated while building the relation descriptor see RelationBuildDesc(). So if you keep some random pointer in RelationData without ensuring that is getting reinitialized while building the relation cache then I think that's not the correct way to do it. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: row filtering for logical replication

2021-11-29 Thread Dilip Kumar
ng pubactions for it at the > current location in the v42* patch (in pgoutput_row_filter). It is > okay to combine them at a later stage during execution when we can't > do it at the time of forming cache entry. What about the initial table sync? during that, we are going to combine all the filters or we are going to apply only the insert filters? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Can I assume relation would not be invalid during from ExecutorRun to ExecutorEnd

2021-11-29 Thread Dilip Kumar
DDL on this particular relation so this recache entry should not be invalidated at least in the example you have given. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

<    1   2   3   4   5   6   7   8   9   10   >