Re: Proposal: Global Index for PostgreSQL

2025-06-18 Thread Dilip Kumar
On Wed, Jun 18, 2025 at 4:38 AM Masahiko Sawada wrote: > > On Sat, Jun 14, 2025 at 2:32 AM Dilip Kumar wrote: > > Thanks for your opinion, Sawada-San. > Does it need to keep holding dead TIDs for each partition until it > completes vacuuming all partitions that are cov

Re: Replication slot is not able to sync up

2025-06-14 Thread Dilip Kumar
up. We could later add a timeout parameter to control maximum > wait time if this approach seems acceptable. > > I tested that, when pgbench TPC-B is running on the primary, calling > pg_sync_replication_slots() on the standby correctly blocks until I advance > the > primary

Re: Proposal: Global Index for PostgreSQL

2025-06-14 Thread Dilip Kumar
On Mon, Jun 9, 2025 at 3:28 PM Dilip Kumar wrote: > > On Mon, Jun 9, 2025 at 2:03 PM Nikita Malakhov wrote: > > 4) Update-heavy partitioned tables that should run vacuum frequently. > > Significant > > vacuum slowdown would result in going beyond SLAs without corre

Re: Question on error code selection in conflict detection

2025-06-13 Thread Dilip Kumar
On Wed, Jun 11, 2025 at 8:02 PM Dilip Kumar wrote: > > On Wed, Jun 11, 2025 at 7:33 PM Robert Haas wrote: > > > > On Tue, Jun 10, 2025 at 2:09 AM Amit Kapila wrote: > > > Can we instead try to use other suitable existing error codes? > > > > Why? >

Re: Question on error code selection in conflict detection

2025-06-11 Thread Dilip Kumar
gical replication conflicts, I suggest we consider defining a dedicated class of error codes, much like we have for FDWs. IMHO this would be a more future-proof approach, given the potential for many new conflict detection types in the future. -- Regards, Dilip Kumar Google

Re: failover logical replication slots

2025-06-10 Thread Dilip Kumar
erwrite > existing slots though not sure if it's worth it. > > From a database user's perspective, it's necessary to clean up any leftover > slots on a new standby following a switchover, regardless of whether the > failover slot feature is supported. Because those leftover slots could lead to > excessive WAL accumulation. It's logical for users to clean up existing replication slots on a new standby. Therefore, the current behavior might not be overly inconvenient. However, providing a GUC to force slot overwrites could streamline switchovers, allowing users to do nothing post-switchover. I'm curious to hear others' thoughts on this. -- Regards, Dilip Kumar Google

Re: Question on error code selection in conflict detection

2025-06-10 Thread Dilip Kumar
On Tue, Jun 10, 2025 at 12:14 PM Dilip Kumar wrote: > > On Tue, Jun 10, 2025 at 11:39 AM Amit Kapila wrote: > > > > On Mon, Jun 9, 2025 at 7:14 PM Dilip Kumar wrote: > > > > > > I was reviewing the code for conflict reporting and beca

Re: Proposal: Global Index for PostgreSQL

2025-06-10 Thread Dilip Kumar
On Wed, Jun 11, 2025 at 1:08 AM Bruce Momjian wrote: > > On Mon, Jun 9, 2025 at 05:51:25PM -0400, Bruce Momjian wrote: > > On Mon, Jun 9, 2025 at 03:28:38PM +0530, Dilip Kumar wrote: > > There are certainly use cases where this would be helpful, but I think > > the bi

Re: Proposal: Global Index for PostgreSQL

2025-06-10 Thread Dilip Kumar
On Tue, Jun 10, 2025 at 3:21 AM Bruce Momjian wrote: Thanks Bruce for your thoughts on this. > On Mon, Jun 9, 2025 at 03:28:38PM +0530, Dilip Kumar wrote: > > On Mon, Jun 9, 2025 at 2:03 PM Nikita Malakhov wrote: > > > Global Indexes is a very interesting functiona

Re: Question on error code selection in conflict detection

2025-06-10 Thread Dilip Kumar
On Tue, Jun 10, 2025 at 1:25 AM Robert Haas wrote: > > On Mon, Jun 9, 2025 at 9:45 AM Dilip Kumar wrote: > > I was reviewing the code for conflict reporting and became curious > > about the choice of ERRCODE_T_R_SERIALIZATION_FAILURE. This error code > > typically s

Re: Question on error code selection in conflict detection

2025-06-09 Thread Dilip Kumar
On Tue, Jun 10, 2025 at 11:39 AM Amit Kapila wrote: > > On Mon, Jun 9, 2025 at 7:14 PM Dilip Kumar wrote: > > > > I was reviewing the code for conflict reporting and became curious > > about the choice of ERRCODE_T_R_SERIALIZATION_FAILURE. This error code > > typic

Re: Add new wait event to XactLockTableWait

2025-06-09 Thread Dilip Kumar
On Mon, Jun 9, 2025 at 6:55 PM Xuneng Zhou wrote: > > Hi Dilip, > > Thanks for looking into this! > > On Mon, Jun 9, 2025 at 6:56 PM Dilip Kumar wrote: >> >> > Thanks for updating the patch! It looks good to me. >> > >> > I think we can

Question on error code selection in conflict detection

2025-06-09 Thread Dilip Kumar
rcode(ERRCODE_T_R_SERIALIZATION_FAILURE); } Assert(false); return 0; /* silence compiler warning */ } -- Regards, Dilip Kumar Google

Re: Add new wait event to XactLockTableWait

2025-06-09 Thread Dilip Kumar
before > > submitting patches and sending emails.😂 > > Thanks for updating the patch! It looks good to me. > > I think we can mark it as "Ready for Committer" in the CommitFest. > Unless there are any objections, I'll commit it once v19 development opens. LGTM, except I suggest using WAIT_EVENT_XACT_COMPLETE instead of WAIT_EVENT_XACT_DONE. I think it sounds better. -- Regards, Dilip Kumar Google

Re: Proposal: Global Index for PostgreSQL

2025-06-09 Thread Dilip Kumar
rd with prioritizing a VACUUM optimization solution for partitioned tables with global indexes. My initial proposal touched on a proof-of-concept experiment, which indicated no significant performance hit with global index after the optimization. I'll share the detailed VACUUM optimization proposal in this thread within the next couple of days. -- Regards, Dilip Kumar Google

Re: Proposal: Global Index for PostgreSQL

2025-06-06 Thread Dilip Kumar
On Fri, Jun 6, 2025 at 1:01 PM wenhui qiu wrote: > > Hi Dilip Kumar >Thank you for your working on this ,I remember six years ago there was > talk about global index ,You can see if this mailing list has any references > to > (https://www.postgresql.org/message-id/CAL

Re: Fix slot synchronization with two_phase decoding enabled

2025-06-06 Thread Dilip Kumar
On Thu, Jun 5, 2025 at 3:59 PM Amit Kapila wrote: > > On Thu, Jun 5, 2025 at 2:53 PM Dilip Kumar wrote: > > > > On Tue, Jun 3, 2025 at 11:05 AM Nisha Moond > > wrote: > > > > > > > > > Attached v17 patches. Added a top-up patch 0002

Proposal: Global Index for PostgreSQL

2025-06-06 Thread Dilip Kumar
all at once, then (2) is clearly better. I'm aiming to submit the first WIP patch set before the July commitfest. It won't have all the issues ironed out yet, but the main design will be functional. Thanks, Robert, for many of the key design ideas and regular discussion throughout designing this. I'd also like to thank Joe, Peter Geoghegan, Alvaro, and Masahiko Sawada for discussing the independent issues with me offlist. -- Regards, Dilip Kumar Google

Re: Fix slot synchronization with two_phase decoding enabled

2025-06-05 Thread Dilip Kumar
On Thu, Jun 5, 2025 at 3:59 PM Amit Kapila wrote: > > On Thu, Jun 5, 2025 at 2:53 PM Dilip Kumar wrote: > > > > On Tue, Jun 3, 2025 at 11:05 AM Nisha Moond > > wrote: > > > > > > > > > Attached v17 patches. Added a top-up patch 0002

Re: Fix slot synchronization with two_phase decoding enabled

2025-06-05 Thread Dilip Kumar
On Thu, Jun 5, 2025 at 2:53 PM Dilip Kumar wrote: > > On Tue, Jun 3, 2025 at 11:05 AM Nisha Moond wrote: > > > > > > Attached v17 patches. Added a top-up patch 0002 implementing the idea > > suggested by Amit above. > > I have started reviewing this, althoug

Re: Fix slot synchronization with two_phase decoding enabled

2025-06-05 Thread Dilip Kumar
ility? because if a user wants to use both the options together then after creating the slot they need to track when is the right time to enable the other option? Not sure if anyone else has this concern or it's just me? -- Regards, Dilip Kumar Google

Re: Potential issue in listExtensions()

2025-06-05 Thread Dilip Kumar
_extension entries from > pg_description. > Attaching the patch for the same. > > Thoughts? > Attached patch seems to be fixing the issue. -- Regards, Dilip Kumar Google

Re: [PATCH] Allow parallelism for plpgsql return expression after commit 556f7b7

2025-06-03 Thread Dilip Kumar
On Tue, May 20, 2025 at 1:45 PM DIPESH DHAMELIYA wrote: > > > On Tue, May 20, 2025 at 11:57 AM Dilip Kumar wrote: > > > > I don't think we can remove the 'maxtuples' parameter from > > exec_run_select(). In this particular case, the query itself is &g

Re: Speedup truncations of temporary relation forks

2025-06-01 Thread Dilip Kumar
On Sun, Jun 1, 2025 at 5:51 PM Daniil Davydov <3daniss...@gmail.com> wrote: > > Hi, > > On Sun, Jun 1, 2025 at 5:31 PM Dilip Kumar wrote: > > > > On Sun, Jun 1, 2025 at 7:52 AM Michael Paquier wrote: > > > > > > I doubt that it would be a good ide

Re: Speedup truncations of temporary relation forks

2025-06-01 Thread Dilip Kumar
from generate_series(1,100) as i; ANALYZE ; select relpages from pg_class where relname='test'; TRUNCATE TABLE test; ROLLBACK; -- Regards, Dilip Kumar Google

Re: Conflict detection for update_deleted in logical replication

2025-05-28 Thread Dilip Kumar
On Tue, May 27, 2025 at 11:45 AM Amit Kapila wrote: > > On Mon, May 26, 2025 at 12:46 PM Zhijie Hou (Fujitsu) > wrote: > > > > On Sun, May 25, 2025 at 4:36 PM Dilip Kumar wrote: > > > > > > > > I am thinking can't we make it more deterministic su

Re: Hash table scans outside transactions

2025-05-25 Thread Dilip Kumar
s. > Maybe any thoughts on that? I haven't reviewed the complete patch or tested it, but I don't see any issues with it. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Understanding when VM record needs snapshot conflict horizon

2025-05-25 Thread Dilip Kumar
it is not necessary, don't you think it will impact performance on standby? because now it has to loop through the procarray on standby to check whether there is any conflict before applying this WAL. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Conflict detection for update_deleted in logical replication

2025-05-25 Thread Dilip Kumar
case is not harmful? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Conflict detection for update_deleted in logical replication

2025-05-24 Thread Dilip Kumar
On Sat, May 24, 2025 at 11:00 AM Amit Kapila wrote: > > On Sat, May 24, 2025 at 10:29 AM Dilip Kumar wrote: > > > > On Sat, May 24, 2025 at 10:04 AM Dilip Kumar wrote: > > > > > > On Fri, May 23, 2025 at 9:21 PM Xuneng Zhou wrote: > > > > > &

Re: Conflict detection for update_deleted in logical replication

2025-05-23 Thread Dilip Kumar
On Sat, May 24, 2025 at 10:04 AM Dilip Kumar wrote: > > On Fri, May 23, 2025 at 9:21 PM Xuneng Zhou wrote: > > > Looking at v31-0001 again, most of it looks fine except this logic of > getting the commit_ts after marking the transaction in commit. I see > in RecordTransa

Re: Conflict detection for update_deleted in logical replication

2025-05-23 Thread Dilip Kumar
nmsgs, invalMessages, RelcacheInitFileInval, MyXactFlags, InvalidTransactionId, NULL /* plain commit */ ); -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [PATCH] Allow parallelism for plpgsql return expression after commit 556f7b7

2025-05-19 Thread Dilip Kumar
SELECT email FROM users); RETURN user_email; END; $$ LANGUAGE plpgsql; -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-05-18 Thread Dilip Kumar
ver restart. If we can address that by making the GUCs reloadable via SIGHUP, that might be enough. On the other hand, if the goal is to make the behavior fully dynamic, then we should go all the way, decouple it from wal_level. For example, we could start logging the extra WAL needed for logical decoding as soon as a logical slot is created, and stop once all logical slots are dropped, even if wal_level is still set to logical. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Assertion failure in smgr.c when using pg_prewarm with partitioned tables

2025-05-15 Thread Dilip Kumar
btree" index, yet also states the > relation > is a "btree" index, which can seem contradictory. The actual issue is > that > the index is a btree partitioned index, but this detail is missing, > causing > possible confusion. Yes, I found the error message confusing during testing as well, so it makes sense to improve it. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Assertion failure in smgr.c when using pg_prewarm with partitioned tables

2025-05-15 Thread Dilip Kumar
On Thu, May 15, 2025 at 3:14 PM Fujii Masao wrote: > > > > On 2025/05/15 18:20, Dilip Kumar wrote: > > On Thu, May 15, 2025 at 2:22 PM Masahiro Ikeda > > wrote: > >> > >> Hi, > >> > >> I encountered an assertion failure when a part

Re: Assertion failure in smgr.c when using pg_prewarm with partitioned tables

2025-05-15 Thread Dilip Kumar
k that instead of checking the relnumber is valid, we shall check whether it's a partitioned rel and give a separate error that prewarm is not supported for partitioned tables? And in fact, we can think of supporting this for the partitioned tables as well in the future, where we c

Re: Backward movement of confirmed_flush resulting in data duplication.

2025-05-14 Thread Dilip Kumar
On Wed, May 14, 2025 at 9:16 AM Amit Kapila wrote: > > On Tue, May 13, 2025 at 4:22 PM Dilip Kumar wrote: > > > > On Tue, May 13, 2025 at 3:48 PM shveta malik wrote: > > > > > > Hi All, > > > > > > It is a spin-off thread from earlier discu

Re: Backward movement of confirmed_flush resulting in data duplication.

2025-05-14 Thread Dilip Kumar
On Wed, May 14, 2025 at 12:15 PM Amit Kapila wrote: > > On Wed, May 14, 2025 at 11:59 AM Dilip Kumar wrote: > > > > On Wed, May 14, 2025 at 9:16 AM Amit Kapila wrote: > > > > > > On Tue, May 13, 2025 at 4:22 PM Dilip Kumar wrote: > > > > >

Re: Backward movement of confirmed_flush resulting in data duplication.

2025-05-13 Thread Dilip Kumar
y, have we analyzed if there’s a way to prevent the subscriber from moving the LSN backward in the first place? That might lead to a cleaner and more robust solution overall. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: make VALIDATE domain constraint lock on related relations as ShareUpdateExclusiveLock

2025-05-12 Thread Dilip Kumar
omain constraints. > > The attached patch addresses this problem. This makes sense, and the patch also looks good to me. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Suggestion to add --continue-client-on-abort option to pgbench

2025-05-12 Thread Dilip Kumar
You can just try to apply using git am, and it will report those whitespace warnings. And for fixing, you can just use "--whitespace=fix" along with git am. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Should shared_preload_libraries be loaded during binary upgrade?

2025-05-01 Thread Dilip Kumar
On Thu, May 1, 2025 at 7:35 PM Tom Lane wrote: > > Bruce Momjian writes: > > On Thu, May 1, 2025 at 11:05:56AM +0530, Dilip Kumar wrote: > >> Does it make sense to load "shared_preload_libraries" during binary > >> upgrade mode? > > > Well, the

Should shared_preload_libraries be loaded during binary upgrade?

2025-04-30 Thread Dilip Kumar
ase. While it's true that extensions should ideally handle this themselves, wouldn't it be safer if we could avoid loading them at all during the binary upgrade mode? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: New committer: Jacob Champion

2025-04-16 Thread Dilip Kumar
Congratulations Jacob! Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com On Sat, 12 Apr 2025 at 1:56 AM, Jonathan S. Katz wrote: > The Core Team would like to extend our congratulations to Jacob > Champion, who has accepted an invitation to become our newest Post

Re: Add an option to skip loading missing publication to avoid logical replication failure

2025-03-13 Thread Dilip Kumar
On Thu, Mar 13, 2025 at 3:20 PM Amit Kapila wrote: > > On Thu, Mar 13, 2025 at 11:43 AM Dilip Kumar wrote: > > > > Looks fine, shall we add the missing publication point as well > > something like below > > > > /* > > * In operations like '

Re: Add an option to skip loading missing publication to avoid logical replication failure

2025-03-12 Thread Dilip Kumar
On Thu, Mar 13, 2025 at 10:49 AM vignesh C wrote: > > On Thu, 13 Mar 2025 at 09:18, Dilip Kumar wrote: > > > > Thanks looks good to me. > > > > While looking at the patch, I have a few comments/questions > > > > + if (pub) >

Re: Add an option to skip loading missing publication to avoid logical replication failure

2025-03-12 Thread Dilip Kumar
On Thu, Mar 13, 2025 at 7:38 AM vignesh C wrote: > > On Wed, 12 Mar 2025 at 16:15, Dilip Kumar wrote: > > > > Thanks, Vignesh, for adding the test. I believe you've tested the > > effect of DROP PUBLICATION. However, I think we should also test the > > b

Re: Add an option to skip loading missing publication to avoid logical replication failure

2025-03-12 Thread Dilip Kumar
or dilip's comment from [2]. > The attached v7 version patch has the changes for the same. > Thanks, Vignesh, for adding the test. I believe you've tested the effect of DROP PUBLICATION. However, I think we should also test the behavior of ALTER SUBSCRIPTION...SET PUBLICATION before creating the PUBLICATION, and then create the PUBLICATION at a later stage. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Parallel heap vacuum

2025-03-12 Thread Dilip Kumar
On Wed, Mar 12, 2025 at 3:40 PM Amit Kapila wrote: > > On Wed, Mar 12, 2025 at 11:24 AM Dilip Kumar wrote: > > > > On Wed, Mar 12, 2025 at 3:17 AM Masahiko Sawada > > wrote: > > > > > > On Tue, Mar 11, 2025 at 5:51 AM Amit Kapila > > > wr

Re: Parallel heap vacuum

2025-03-11 Thread Dilip Kumar
if all tables are small, we could just apply table-level parallelism without needing block-level parallelism at all. This approach could offer more flexibility, isn't it? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Parallel heap vacuum

2025-03-11 Thread Dilip Kumar
On Wed, Mar 12, 2025 at 11:24 AM Dilip Kumar wrote: > > On Wed, Mar 12, 2025 at 3:17 AM Masahiko Sawada wrote: > > Some more specific comments on the patch set -- v11-0001 1. This introduces functions like parallel_vacuum_estimate(), parallel_vacuum_initialize(), etc., but the

Re: Conflict detection for multiple_unique_conflicts in logical replication

2025-03-11 Thread Dilip Kumar
On Tue, Mar 11, 2025 at 2:28 PM Nisha Moond wrote: > > On Tue, Mar 11, 2025 at 11:10 AM Dilip Kumar wrote: > The goal of introducing a new conflict type is to handle multiple-key > conflicts separately. It will not only allow users to apply different > resolution methods for sing

Re: change on_exit_nicely_list array to the dynamic array to increase slots at run time for pg_restore

2025-03-11 Thread Dilip Kumar
ation[1] so that, instead of holding a single "Archive," it holds a list, allowing you to append different Archive handles to the list. [1] typedef struct ShutdownInformation { ParallelState *pstate; Archive *AHX; -> change this to List of Archive* } ShutdownInformation; -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Conflict detection for multiple_unique_conflicts in logical replication

2025-03-10 Thread Dilip Kumar
the logs. If only a single key > conflict is detected, the existing 'insert_exists' conflict will be > raised as it is now. I think it makes sense to report all the unique key conflicts for a single row at once, rather than stopping after the first one. However, I don't under

Re: Add an option to skip loading missing publication to avoid logical replication failure

2025-03-10 Thread Dilip Kumar
On Mon, Mar 10, 2025 at 10:54 AM Amit Kapila wrote: > > On Mon, Mar 10, 2025 at 10:15 AM Dilip Kumar wrote: > > > > On Mon, Mar 10, 2025 at 9:33 AM Amit Kapila wrote: > >> > >> On Tue, Mar 4, 2025 at 6:54 PM vignesh C wrote: > >> &g

Re: Add an option to skip loading missing publication to avoid logical replication failure

2025-03-09 Thread Dilip Kumar
this behavior looks fine. BTW, I am planning to commit this only on HEAD as this is a behavior > change. Please let me know if you guys think otherwise. > Somehow this looks like a bug fix which should be backported no? Am I missing something? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Conflict detection for update_deleted in logical replication

2025-02-10 Thread Dilip Kumar
On Mon, Feb 10, 2025 at 2:45 PM Amit Kapila wrote: > > On Mon, Feb 10, 2025 at 10:26 AM Dilip Kumar wrote: > > > > On Fri, Feb 7, 2025 at 11:17 AM Amit Kapila wrote: > > > > > > > > > I'm not really sure that these behaviors

Re: Conflict detection for update_deleted in logical replication

2025-02-09 Thread Dilip Kumar
ions, waiting too long could lead to excessive delays. The idea here is that the Node1 subscriber can wait for the full max_conflict_retention_duration set by the GUC when necessary, while the Node2 subscriber can choose a shorter wait time to avoid unnecessary delays caused by frequent long transactions. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Conflict detection for update_deleted in logical replication

2025-02-04 Thread Dilip Kumar
pped after reaching 'max_conflict_retention_duration' will it resume back? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Conflict detection for update_deleted in logical replication

2025-02-04 Thread Dilip Kumar
revent. But OTOH, the > implementation could be slightly simpler if we provide these options > as GUC though I am not completely sure of that point. Having said > that, I am open to changing it to a non-subscription level. Do you > think it would be better to provide one or both of

Re: Parallel heap vacuum

2025-01-18 Thread Dilip Kumar
On Fri, Jan 17, 2025 at 10:43 PM Masahiko Sawada wrote: > On Fri, Jan 17, 2025 at 1:43 AM Dilip Kumar wrote: > > > > On Fri, Jan 17, 2025 at 6:37 AM Masahiko Sawada > wrote: > >> > >> On Sun, Jan 12, 2025 at 1:34 AM Masahiko Sawada > wrote: > &

Re: Parallel heap vacuum

2025-01-17 Thread Dilip Kumar
this patch set? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Conflict detection for update_deleted in logical replication

2025-01-16 Thread Dilip Kumar
On Thu, Jan 16, 2025 at 4:02 PM Amit Kapila wrote: > On Thu, Jan 16, 2025 at 3:45 PM Dilip Kumar wrote: > > > > On Fri, Jan 3, 2025 at 4:31 PM Amit Kapila > wrote: > >> > >> On Thu, Jan 2, 2025 at 2:57 PM vignesh C wrote: > >> > > >>

Re: Conflict detection for update_deleted in logical replication

2025-01-16 Thread Dilip Kumar
E works. At least that's what my opinion about this case. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Conflict detection for update_deleted in logical replication

2024-12-17 Thread Dilip Kumar
On Tue, Dec 17, 2024 at 8:54 AM Zhijie Hou (Fujitsu) wrote: > > On Monday, December 16, 2024 7:21 PM Dilip Kumar > wrote: > > > So IIUC in step 2) we send the message and get the list of all the > > transactions which are in the commit phase? What do you exactly mean by

Re: Conflict detection for update_deleted in logical replication

2024-12-16 Thread Dilip Kumar
ction we were interested on the publisher? At high level the overall idea looks promising to me but wanted to put more thought on lower level details about what transactions exactly we are waiting for and what WAL LSN we are waiting to get flushed. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Skip collecting decoded changes of already-aborted transactions

2024-12-14 Thread Dilip Kumar
he approach you followed in the current patch. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Skip collecting decoded changes of already-aborted transactions

2024-12-11 Thread Dilip Kumar
On Thu, Dec 12, 2024 at 11:08 AM Amit Kapila wrote: > > On Wed, Dec 11, 2024 at 8:21 AM Dilip Kumar wrote: > > > > On Wed, Dec 11, 2024 at 3:18 AM Masahiko Sawada > > wrote: > > > > > > On Mon, Dec 9, 2024 at 10:19 PM Dilip Kumar wrote: > > &

Re: Track the amount of time waiting due to cost_delay

2024-12-10 Thread Dilip Kumar
oo. > However, parallel workers report their sleep time no more frequently > than once per second, so the reported value may be slightly stale. > This description looks good. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Skip collecting decoded changes of already-aborted transactions

2024-12-10 Thread Dilip Kumar
On Wed, Dec 11, 2024 at 3:18 AM Masahiko Sawada wrote: > > On Mon, Dec 9, 2024 at 10:19 PM Dilip Kumar wrote: > > > > > > If the largest transaction is non-streamable, won't the transaction > > > returned by ReorderBufferLargestTXN() in the other case alr

Re: Fix bank selection logic in SLRU

2024-12-10 Thread Dilip Kumar
On Tue, 10 Dec 2024 at 6:32 PM, Andrey M. Borodin wrote: > > > > On 10 Dec 2024, at 15:39, Yura Sokolov wrote: > > > > It is not critical bug, since it doesn't hurt correctness just > performance. In worst case only one bank will be used. > > Ugh... yeah. IMO the problem is that we do not have p

Re: Fix bank selection logic in SLRU

2024-12-10 Thread Dilip Kumar
On Tue, 10 Dec 2024 at 7:30 PM, Robert Haas wrote: > On Tue, Dec 10, 2024 at 8:58 AM Dilip Kumar wrote: > >> Bank selection code assumes that number of buffers is power of 2. > >> If the number of buffers is not power of 2 - only subset of buffers > will be used. In wor

Re: Fix bank selection logic in SLRU

2024-12-10 Thread Dilip Kumar
On Tue, 10 Dec 2024 at 7:24 PM, Andrey M. Borodin wrote: > > > > On 10 Dec 2024, at 16:26, Dilip Kumar wrote: > > > > IIUC, we do check that it should be in multiple of bank size (i.e.) > > which is multiple of 2, right? Am I missing something? > > Bank s

Re: Fix bank selection logic in SLRU

2024-12-10 Thread Dilip Kumar
ck_errdetail("\"%s\" must be a multiple of %d", name, SLRU_BANK_SIZE); return false; } -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Skip collecting decoded changes of already-aborted transactions

2024-12-09 Thread Dilip Kumar
On Tue, Dec 10, 2024 at 11:09 AM Amit Kapila wrote: > > On Tue, Dec 10, 2024 at 10:59 AM Dilip Kumar wrote: > > > > On Tue, Nov 26, 2024 at 3:02 AM Masahiko Sawada > > wrote: > > > > > > > > I've attached a new version patch that incorpo

Re: Skip collecting decoded changes of already-aborted transactions

2024-12-09 Thread Dilip Kumar
we could repeat this process for a few more transactions. In contrast, it might be more efficient to simply discard the large aborted transaction, even if it’s not streamable, to avoid this issue. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Track the amount of time waiting due to cost_delay

2024-12-09 Thread Dilip Kumar
On Mon, Dec 9, 2024 at 10:11 PM Bertrand Drouvot wrote: > > Hi, > > On Mon, Dec 09, 2024 at 08:34:13PM +0530, Dilip Kumar wrote: > > On Mon, Dec 9, 2024 at 6:55 PM Bertrand Drouvot > > > Yeah. I think we could change the wording that way: > > > &

Re: Track the amount of time waiting due to cost_delay

2024-12-09 Thread Dilip Kumar
On Mon, Dec 9, 2024 at 6:55 PM Bertrand Drouvot wrote: > > Hi, > > On Mon, Dec 09, 2024 at 05:18:30PM +0530, Dilip Kumar wrote: > > On Mon, Dec 9, 2024 at 2:51 PM Masahiro Ikeda > > wrote: > > > > > > On 2024-12-06 18:31, Bertrand Drouvot wrote: > &

Re: Track the amount of time waiting due to cost_delay

2024-12-09 Thread Dilip Kumar
in a shared variable instead of local in individual workers and whenever the shared variable crosses WORKER_REPORT_DELAY_INTERVAL we can report this would avoid individual reporting from different workers. Since we are already computing the cost balance in shared variable i.e. VacuumSharedC

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-12-09 Thread Dilip Kumar
h an made a number of stylistic changes > to conform more closely to existing styles (adding back more rs_ > prefixes and typedef'ing BitmapHeapScanDescData * as > BitmapHeapScanDesc [despite how sad it makes me]). > > > BTW would it be a good idea for heapam_scan_bitmap_next_block to check > > the passed "scan" has SO_TYPE_BITMAPSCAN? We haven't been checking that > > so far I think, but we only had a single struct ... > > Good point. I've done this. > > I plan to commit 0002 and 0003 next week. I'm interested if you think > 0001 is correct. > I may also commit 0004-0006 as I feel they are ready too. Are we planning to commit this refactoring? I think this refactoring makes the overall code of BitmapHeapNext() quite clean and readable. I haven't read all patches but 0001-0006 including 0009 makes this code quite clean and readable. I like the refactoring of merging of the shared iterator and the private iterator also. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Question about read_stream_look_ahead()

2024-11-26 Thread Dilip Kumar
Can we merge it with the pending read? */ if (stream->pending_read_nblocks > 0 && stream->pending_read_blocknum + stream->pending_read_nblocks == blocknum) { stream->pending_read_nblocks++; continue; } -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Fix small typo, use InvalidRelFileNumber instead of InvalidOid

2024-11-10 Thread Dilip Kumar
On Sat, Nov 9, 2024 at 12:41 PM Dilip Kumar wrote: > > On Fri, Nov 8, 2024 at 8:36 PM Tom Lane wrote: >> >> Dilip Kumar writes: >> > IIRC, In catalog we intentionally left it as Oid because RelFileNumber is >> > an internal typedef bug, it is not an exposed d

Re: Fix small typo, use InvalidRelFileNumber instead of InvalidOid

2024-11-08 Thread Dilip Kumar
On Fri, Nov 8, 2024 at 8:36 PM Tom Lane wrote: > Dilip Kumar writes: > > IIRC, In catalog we intentionally left it as Oid because RelFileNumber is > > an internal typedef bug, it is not an exposed datatype, so probably we > can > > not use it in catalog. > > We cou

Re: Fix small typo, use InvalidRelFileNumber instead of InvalidOid

2024-11-08 Thread Dilip Kumar
On Fri, Nov 8, 2024 at 4:30 PM Alvaro Herrera wrote: > On 2024-Nov-08, Dilip Kumar wrote: > > > It appears that we are passing InvalidOid instead of InvalidRelFileNumber > > when calling index_create(). While this isn’t technically a bug, since > > InvalidRelFileNumber

Fix small typo, use InvalidRelFileNumber instead of InvalidOid

2024-11-08 Thread Dilip Kumar
It appears that we are passing InvalidOid instead of InvalidRelFileNumber when calling index_create(). While this isn’t technically a bug, since InvalidRelFileNumber is defined as InvalidOid, it’s preferable to use the correct identifier for clarity and consistency. -- Regards, Dilip Kumar

Re: Can rs_cindex be < 0 for bitmap heap scans?

2024-10-28 Thread Dilip Kumar
On Sat, Oct 26, 2024 at 5:30 PM Melanie Plageman wrote: > On Sat, Oct 26, 2024 at 12:17 AM Dilip Kumar > wrote: > > > > On Fri, Oct 25, 2024 at 10:07 PM Melanie Plageman < > melanieplage...@gmail.com> wrote: > >> > >> On Fri, Oct 25,

Re: Can rs_cindex be < 0 for bitmap heap scans?

2024-10-25 Thread Dilip Kumar
ly(scan->rs_inited)) { /* continue from previously returned page/tuple */ page = BufferGetPage(scan->rs_cbuf); lineindex = scan->rs_cindex + dir; if (ScanDirectionIsForward(dir)) --Refer definition of ScanDirection typedef enum ScanDirection { BackwardScanDirection = -1, NoMovemen

Re: Can rs_cindex be < 0 for bitmap heap scans?

2024-10-24 Thread Dilip Kumar
On Thu, Oct 24, 2024 at 7:11 PM Melanie Plageman wrote: > Thanks for the reply, Dilip! > > On Thu, Oct 24, 2024 at 12:50 AM Dilip Kumar > wrote: > > > > On Thu, Oct 24, 2024 at 3:45 AM Melanie Plageman < > melanieplage...@gmail.com> wrote: > >> >

Re: Can rs_cindex be < 0 for bitmap heap scans?

2024-10-24 Thread Dilip Kumar
lock. [1] @@ -378,6 +378,7 @@ initscan(HeapScanDesc scan, ScanKey key, bool keep_startblock) ItemPointerSetInvalid(&scan->rs_ctup.t_self); scan->rs_cbuf = InvalidBuffer; scan->rs_cblock = InvalidBlockNumber; + scan->rs_cindex = 0; -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Make default subscription streaming option as Parallel

2024-10-19 Thread Dilip Kumar
On Fri, 18 Oct 2024 at 5:24 PM, Amit Kapila wrote: > On Fri, Oct 18, 2024 at 9:48 AM Dilip Kumar wrote: > > > > On Tue, Oct 8, 2024 at 3:38 PM Amit Kapila > wrote: > >> > >> On Tue, Oct 8, 2024 at 2:25 PM shveta malik > wrote: > >> > >

Re: Make default subscription streaming option as Parallel

2024-10-17 Thread Dilip Kumar
reasonable new defaults. I think we should make parallel streaming as > default and then wait for some user feedback before changing other > defaults. > > I agree, actually streaming of in progress transactions is a useful feature for performance in case of large transaction

Re: Invalid Assert while validating REPLICA IDENTITY?

2024-09-08 Thread Dilip Kumar
On Fri, Sep 6, 2024 at 4:48 PM vignesh C wrote: > > On Mon, 2 Sept 2024 at 18:22, Dilip Kumar wrote: > > > > On Mon, Sep 2, 2024 at 3:32 PM Amit Kapila wrote: > > > > > > On Mon, Sep 2, 2024 at 11:21 AM Dilip Kumar wrote: > > > > > >

Deadlock due to locking order violation while inserting into a leaf relation

2024-09-08 Thread Dilip Kumar
for query details. LOCATION: DeadLockReport, deadlock.c:1135 -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Invalid Assert while validating REPLICA IDENTITY?

2024-09-02 Thread Dilip Kumar
On Mon, Sep 2, 2024 at 3:32 PM Amit Kapila wrote: > > On Mon, Sep 2, 2024 at 11:21 AM Dilip Kumar wrote: > > > > While working on some other code I noticed that in > > FindReplTupleInLocalRel() there is an assert [1] that seems to be > > passing IndexRelation to Get

Invalid Assert while validating REPLICA IDENTITY?

2024-09-01 Thread Dilip Kumar
idxoid || IsIndexUsableForReplicaIdentityFull(BuildIndexInfo(idxrel), edata->targetRel->attrmap)); index_close(idxrel, AccessShareLock); -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Conflict Detection and Resolution

2024-07-30 Thread Dilip Kumar
On Tue, Jul 30, 2024 at 4:56 PM shveta malik wrote: > > On Tue, Jul 30, 2024 at 4:04 PM Dilip Kumar wrote: > > > > On Fri, Jul 26, 2024 at 9:50 AM Ajin Cherian wrote: > > > > Comment in 0002, > > > > 1) I do not see any test case that set a proper co

Re: Conflict Detection and Resolution

2024-07-30 Thread Dilip Kumar
ode-1. But maybe it would be more difficult to get a consistent value if we are setting up a mess replication topology right? Maybe there I think a more advanced timestamp-based option would work better IMHO. I am doing code level review as well and will share my comments soon on 0003 and 0004

Re: Conflict detection and logging in logical replication

2024-07-30 Thread Dilip Kumar
On Tue, Jul 30, 2024 at 1:49 PM Zhijie Hou (Fujitsu) wrote: > > > On Monday, July 29, 2024 6:59 PM Dilip Kumar > > wrote: > > > > > > On Mon, Jul 29, 2024 at 11:44 AM Zhijie Hou (Fujitsu) > > > wrote: > > > > > > > >

Re: Conflict detection and logging in logical replication

2024-07-29 Thread Dilip Kumar
'ri_onConflictArbiterIndexes' and checking if any of these is a subset of the indexes that is returned by ExecInsertIndexTuples(). Why are we doing that, I think we can directly use the 'recheckIndexes' which is returned by ExecInsertIndexTuples(), and those index

  1   2   3   4   5   6   7   8   9   10   >