Re: Parallel Index Scan vs BTP_DELETED and BTP_HALF_DEAD

On Tue, Dec 12, 2017 at 5:20 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Tue, Dec 12, 2017 at 4:00 PM, Kuntal Ghosh > <kuntalghosh.2...@gmail.com> wrote: >> On Mon, Dec 11, 2017 at 2:26 PM, Thomas Munro >> <thomas.mu...@enterprisedb.com> wrote: >&

Re: [HACKERS] parallelize queries containing initplans

ated a text column named 'vartext', inserted some random length texts(max length 100) and tweaked the above query as follows: select ten,count(*) from tenk1 a where a.ten in (select b.ten from tenk1 b where (select a.vartext from tenk1 c where c.ten = a.ten limit 1) = b.vartext limit

Re: Parallel Index Scan vs BTP_DELETED and BTP_HALF_DEAD

an extra if condition. _bt_parallel_release can be included in the next one. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: Loaded footgun open_datasync on Windows

l.org/message-id/flat/D90A5A6C612A39408103E6ECDD77B829408D4F%40voyager.corporate.connx.com#d90a5a6c612a39408103e6ecdd77b829408...@voyager.corporate.connx.com -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: Index maintenance function for BRIN doesn't check RecoveryInProgress()

than erroring due to lock modes. +1 -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: New committers announced at PGCon 2018

hael Paquier > Tomas Vondra > > Congratulations to all! > > regards, tom lane > > -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: Loaded footgun open_datasync on Windows

On Wed, Jun 20, 2018 at 7:36 PM, Laurenz Albe wrote: > Kuntal Ghosh wrote: > [pg_test_fsync doesn't use pgwin32_open and hence doesn't respect O_DSYNC] >> On Wed, Jun 6, 2018 at 2:39 PM, Amit Kapila wrote: >> > On Wed, Jun 6, 2018 at 10:18 AM, Michael Paquier >> >

Re: Incorrect fsync handling in pg_basebackup's tar_finish

ks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: Incorrect fsync handling in pg_basebackup's tar_finish

gt; + if (fsync(tar_data->fd) != 0) >> + return false; >> + } >> >> That looks incorrect to me, hence shouldn't something like the attached >> be done? Magnus and others, any opinions? In the same note, in tar_close(), we fsync on close. We'

In pageinspect, perform clean-up after testing gin-related functions

Hello all, In pageinspect/sql/gin.sql, we don't drop the table test1 at the end of the test. IMHO, we should clean-up at the end of a test. I've attached the patch to perform the same. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Inconsistent behavior in serializable snapshot

owever, if I perform "VACUUM FREEZE bank_account" after the setup step, s2wx throws a conflict error: ERROR: could not serialize access due to read/write dependencies among transactions DETAIL: Reason code: Canceled on identification as a pivot, during write. HINT: The transactio

Re: Inconsistent behavior in serializable snapshot

On Sun, Mar 11, 2018 at 7:52 PM, Kuntal Ghosh <kuntalghosh.2...@gmail.com> wrote: > Hello hackers, > > While working on serializable transaction isolation, I've noticed some > strange behavior in the first permutation mentioned in > isolation/specs/read-only-anomaly-2

Re: zheap: a new storage format for PostgreSQL

ion open for 15 minutes. 2. Combination of ROLLBACK and COMMIT (As suggested by Fabien) 3. PGbench tests for fixed number of transaction. 4. Modify the distribution (As suggested by Alexander Korotkov) Do let me know if any other tests are required. -- Thanks & Regards, Kuntal Ghos

Re: [WIP] [B-Tree] Retail IndexTuple deletion

be reclaimed by PageRepairFragmentation. Now, if you do byte-by-byte comparison with wal_consistency tool, it may fail even for normal tuple as well. Please let me know if you feel the same way. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: [WIP] [B-Tree] Retail IndexTuple deletion

On Fri, Jun 29, 2018 at 11:04 AM, Andrey V. Lepikhov wrote: > On 29.06.2018 10:00, Kuntal Ghosh wrote: >> >> On Wed, Jun 27, 2018 at 12:10 PM, Andrey V. Lepikhov >> wrote: >>> >>> I prepare third version of the patches. Summary: >>> 1. Mask DEAD t

Re: POC: Cleaning up orphaned files using undo logs

ession failures when the tap-tests are enabled. It seems that we're not estimating and allocating the shared memory for rollback-hash tables correctly. I've added a patch to fix the same. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com 0001-Fix-shared-memory-siz

Re: Unordered wait event ClogGroupUpdate

gt; And those ones are also incorrect after another lookup: > - WAIT_EVENT_PARALLEL_FINISH > - WAIT_EVENT_HASH_GROW_BATCHES_DECIDING > - WAIT_EVENT_LOGICAL_APPLY_MAIN > I don't see more of them.. Nice. Same here. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: Unordered wait event ClogGroupUpdate

for HEAD, for v11 I propose to only fix the > documentation side of things to avoid an ABI breakage. +1 -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

In-place updates and serializable transactions

In brief, due to in-place updates, in some cases, the false positives may increase for serializable transactions. Any thoughts? [1] src/backend/storage/lmgr/README-SSI [2] src/test/isolation/specs/multiple-row-versions.spec -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: In-place updates and serializable transactions

though it might look bad in > benchmarks which include a mix of blind writes and rmw operations). > Particularly if it only happens if you explicitly opt into zheap storage. > Thanks Joshua for sharing your input on this. I'm not aware of any realistic workloads for serializa

Re: In-place updates and serializable transactions

T3->T4) Step 5: T3-> COMMIT; Step 6: T1-> UPDATE t where id=1; COMMIT; (creates T4->T1,) At step 6, when the update statement is executed, T1 is rolled back because of T3->T4->T1. But for zheap, step 3 also creates a dependency T1->T3 because of in-place update. When T4 commits in

Re: zheap: a new storage format for PostgreSQL

BlockNumber curblkno = InvalidBlockNumber; >^ Thanks Daniel for testing zheap and reporting the issue. We'll push a fix for the same. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

On Tue, Feb 26, 2019 at 6:46 PM Simon Riggs wrote: > > On Thu, 21 Feb 2019 at 15:38, Kuntal Ghosh wrote: > >> >> Thank you for the patch. It seems to me that while performing COPY >> FREEZE, if we've copied tuples in a previously emptied page > > > There w

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

ee08NT2FQFmz_pQ%40mail.gmail.com > > -- > Pavan Deolasee http://www.2ndQuadrant.com/ > PostgreSQL Development, 24x7 Support, Training & Services -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: PANIC :Call AbortTransaction when transaction id is no normal

levant those steps are since the database is anyway shutting down. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: Adaptive query optimization

hat's doable. > That'll be an interesting work. For the above query, we can definitely calculate the correction coefficient of t1-t2 join given (t1.a = ? AND t1.b = ? AND t1.c < ?) and (t2.x = ? AND t2.y = ?) are true. But, I'm not sure how we can extrapolate that value for t1-t2 join. > &g

Re: Questions of 'for update'

On Mon, Jun 10, 2019 at 12:42 PM Etsuro Fujita wrote: > Hi, > > On Mon, Jun 10, 2019 at 3:50 PM Kuntal Ghosh > wrote: > > On Mon, Jun 10, 2019 at 11:31 AM Zhenghua Lyu wrote: > >> 2. Is the case above a bug or a feature? > >> > > IMHO, it loo

Re: [pg_rewind] cp: cannot stat ‘pg_wal/RECOVERYHISTORY’: No such file or directory

th the comment in the file which forbids the user from editing the file. Any thoughts? -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: Questions of 'for update'

plementation. The argument can be that the snapshot is consistent throughout all the nodes. Whatever tuple you've fetched from the bottom level is locked correctly. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: Why to index a "Recently DEAD" tuple when creating index

clause to detect serializable conflicts: Transaction A precedes transaction B. (Because, transaction A has deleted a tuple and it's not visible to transaction B) -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: Some reloptions non-initialized when loaded

out. > Do we also need to initialize vacuum_cleanup_index_scale_factor? -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: Why to index a "Recently DEAD" tuple when creating index

On Mon, Jun 10, 2019 at 5:26 PM Tom Lane wrote: > > Kuntal Ghosh writes: > >> 2. If we only support "Read Committed" isolation level, is there a safe > >> way to not index such data? > > > I can't think of a case where the RECENTLY_DELETED tuple

Re: [pg_rewind] cp: cannot stat ‘pg_wal/RECOVERYHISTORY’: No such file or directory

On Mon, Jun 10, 2019 at 7:19 PM Robert Haas wrote: > > On Mon, Jun 10, 2019 at 7:08 AM Kuntal Ghosh > wrote: > > This can surely be fixed from the script. While configuring the old > > master as a standby server, clear/modify the settings in > > postgresql.aut

Re: Adaptive query optimization

predicate selectivities of t1 and t2 should have an impact on the calculation of the correction coefficient. If those selectivities are low, the misestimation (which is actual/estimate) should not affect the t1-t2 join correction coefficient much. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: Why to index a "Recently DEAD" tuple when creating index

On Mon, Jun 10, 2019 at 1:30 PM Alex wrote: > > > > On Mon, Jun 10, 2019 at 3:28 PM Kuntal Ghosh > wrote: >> >> On Mon, Jun 10, 2019 at 12:15 PM Alex wrote: >>> >>> HEAPTUPLE_RECENTLY_DEAD, /* tuple is dead, but not deletable yet */ >>> &

Re: Why to index a "Recently DEAD" tuple when creating index

On Mon, Jun 10, 2019 at 2:12 PM Alex wrote: > On Mon, Jun 10, 2019 at 4:10 PM Kuntal Ghosh > wrote: >> I think what I'm trying to say is different. >> >> For my case, the sequence is as following: >> 1. Transaction A has deleted a tuple, say t1 and got committed.

Re: PANIC :Call AbortTransaction when transaction id is no normal

On Mon, May 13, 2019 at 7:07 PM Tom Lane wrote: > Michael Paquier writes: > > On Mon, May 13, 2019 at 01:25:19PM +0530, Kuntal Ghosh wrote: > >> If we fix the issue in this way, we're certainly not going to do all > >> those portal,locks,memory,resource owner cleanup

Re: POC: Cleaning up orphaned files using undo logs

e the default max_worker_processes > from 8 to 12, because otherwise a couple of tests run with fewer > parallel workers than they expect, due to undo worker processes using > up slots. There is probably a better solution to that problem. > > I put the patches in a tarball here, but they are a

Re: Avoiding deadlock errors in CREATE INDEX CONCURRENTLY

s index */ } Please let me know if I'm missing something. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Regression test PANICs with master-standby setup on same machine

the issue with the attached script. sh standby-server-setup.sh make installcheck I accept that configuring master-standby on the same machine for this test is not okay. But, can we avoid the PANIC somehow? Or, is this intentional and I should not include testtablespace in this case? -- Thanks & R

Re: Regression test PANICs with master-standby setup on same machine

ster. > 4. Create a standby using pg_basebackup --tablespace_mapping== > 5. Start the standby. > > [1] > https://www.postgresql.org/message-id/20190422.211933.156769089.horiguchi.kyot...@lab.ntt.co.jp > Thank you for the info. I'll try the same. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: POC: Cleaning up orphaned files using undo logs

tatus as UNDO_LOG_STATUS_FULL. If we don't that, after crash recovery, some new transaction may use that undo log which is wrong, IMHO. Am I missing something? -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: [Patch] Invalid permission check in pg_stats for functional indexes

Yeah, it'll be good to have some regression tests for the same. I'm also not sure which regression file best suites for these tests. [1] http://cfbot.cputube.org/patch_24_2274.log -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: [PATCH] psql: add tab completion for \df slash command suffixes

Pg94. I guess the same patch should be applied on Pg10 and Pg96. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: [Patch] Invalid permission check in pg_stats for functional indexes

g the status to ready for committer. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: POC: Cleaning up orphaned files using undo logs

er. But I see what you mean: the next modification of > the page that DOES depend on the insert pointer might not log the > meta-data if it's not the first WAL record to touch it after a > checkpoint. Rats. I'll have to think about that some more. Cool. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: subscriptionCheck failures on nightjar

rsion 5.0.0-23-generic), but couldn't reproduce the same. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: POC: Cleaning up orphaned files using undo logs

it'll not be able to WAL log the backup image of the meta unlogged part. IMHO, this breaks the recovery logic of unlogged part of undo meta. Thoughts? On Mon, Sep 2, 2019 at 9:47 AM Thomas Munro wrote: > > On Fri, Aug 30, 2019 at 8:27 PM Kuntal Ghosh > wrote: > > I'm getting

Re: POC: Cleaning up orphaned files using undo logs

; inline function that gets the element once, stores it in a local > variable, and then updates all the fields. > Noted. Earlier, Robert also raised the point of using so many macros. He also suggested to use a single type of object that stores all the information we need. It'll make things simpler and easier to understand. In the upcoming patch set, we're removing all these changes. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

ely we shouldn't make that a > binary search for each subtrans level, but just have a small > simplehash hashtable for xids. A check for top transaction id first and usage of simple sound like good optimizations. But, I'm not sure whether these changes should be part of this patch or a separate one. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

T4->T1,) At step 6, when the update statement is executed, T1 is rolled back because of T3->T4->T1. But for zheap, step 3 also creates a dependency T1->T3 because of in-place update. When T4 commits in step 4, it marks T3 as doomed because of T1 --> T3 --> T4. Hence, in step 5, T3 is rolled back. [1] Re: In-place updates and serializable transactions: https://www.postgresql.org/message-id/CAGz5QCJzreUqJqHeXrbEs6xb0zCNKBHhOj6D9Tjd3btJTzydxg%40mail.gmail.com -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: POC: Cleaning up orphaned files using undo logs

+ /* + * You've used up all 16 exabytes of undo log addressing space. + * This is a difficult state to reach using only 16 exabytes of + * WAL. + */ + elog(ERROR, "undo log address space exhausted"); + } looks like a potential unlikely() condition. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: subscriptionCheck failures on nightjar

tory While subscription 3 is created, it eventually reaches to a consistent snapshot point and prints the WAL location corresponding to it. It seems sub1/sub2 immediately fails to serialize the snapshot to the .snap file having the same WAL location. Is this helpful? -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: Custom reloptions and lock modes

gt; > + DEFAULT_BLOOM_BITS, 1, MAX_BLOOM_BITS, > > + AccessExclusiveLock); > > Do we need a comment to explain why we're using AccessExclusiveLock in > > this case? > > Because that's the safest default to use here? That seemed obvious to > me. Okay. Sounds good. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: Custom reloptions and lock modes

clusiveLock); Do we need a comment to explain why we're using AccessExclusiveLock in this case? -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: auxiliary processes in pg_stat_ssl

ter approach than checking the backend_type. The patch looks good to me. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

On Tue, Nov 12, 2019 at 4:12 PM Alexey Kondratov wrote: > > On 04.11.2019 13:05, Kuntal Ghosh wrote: > > On Mon, Nov 4, 2019 at 3:32 PM Dilip Kumar wrote: > >> So your result shows that with "streaming on", performance is > >> degrading? By any chance di

Re: Ordering of header file inclusion

he latest HEAD. Apart from that, the changes looks good to me. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

: 90079.286 ms (01:30.079) -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

ves. I've not analyzed the bottleneck yet. I'm looking into the same. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: ALTER TABLE ... SET STORAGE does not propagate to indexes

size correctly which is around 8KB. But, the above changes will make the tuple size around 3KB. So, it'll not be able to test that particular scenario.Thoughts? -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: ALTER TABLE ... SET STORAGE does not propagate to indexes

On Tue, Feb 25, 2020 at 1:09 PM Peter Eisentraut wrote: > > On 2020-02-24 12:21, Kuntal Ghosh wrote: > > I've reproduced the issue on head. And, the patch seems to solve the > > problem. The patch looks good to me. But, I've a small doubt regarding > > the changes in test_

Re: logical decoding : exceeded maxAllocatedDescs for .spill files

mment whether we can back-patch only this part of the code. But, this seems to allocate a huge amount of memory per chunk although the tuple is small. Thoughts? -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: logical decoding : exceeded maxAllocatedDescs for .spill files

hed a patch that implements the same. It solves the problem reported earlier. This solution will at least slow down the process of going OOM even for very small sized tuples. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com 0001-Restrict-memory-allocation

Fix comment for max_cached_tuplebufs definition

to keep 8MB cache only. In that case, I can come up with another patch. Thoughts? -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com 0001-Fix-comment-for-max_cached_tuplebufs-definition.patch Description: Binary data

Re: logical decoding : exceeded maxAllocatedDescs for .spill files

Hello, On Sat, Feb 8, 2020 at 1:18 AM Andres Freund wrote: > > Hi, > > On 2020-02-07 20:02:01 +0100, Tomas Vondra wrote: > > On Fri, Feb 07, 2020 at 10:33:48AM -0800, Andres Freund wrote: > > > Hi, > > > > > > On 2020-02-04 10:15:01 +0530, K

Re: logical decoding : exceeded maxAllocatedDescs for .spill files

on tuple size, say 100 bytes, approach 1 will allocate 390 MB of memory (approx.) whereas approach 2 will allocate 4GB of memory approximately. If there is no obvious error that I'm missing, I think we should implement the first approach. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

te in deadlock cycles. And, the number of locks under this category is likely to increase in future with new parallel features. Hence, it could be used in multiple places. Should we move the condition inside a macro and just call it from here? -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

indeed use that value a lot in the code. IMHO, it looks okay to me to have that assumption here as well. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: backend type in log_line_prefix?

uot;. In v3-0004-Remove-am_syslogger-global-variable.patch, + * This is exported so that elog.c can call it when BackendType is B_LOGGER. s/BackendType/MyBackendType? Done some basic testing. Working as expected. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: backend type in log_line_prefix?

On Tue, Mar 10, 2020 at 9:11 PM Peter Eisentraut wrote: > On 2020-03-09 16:20, Kuntal Ghosh wrote: > > In v3-0001-Refactor-ps_status.c-API.patch, > > - * postgres: walsender > > This part is still valid, right? > Sure but I figured this comment was in the context of

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

Hello Ashutosh, Fujita, On Wed, Apr 8, 2020 at 3:49 PM Ashutosh Bapat wrote: > On Wed, 8 Apr 2020 at 15:42, Etsuro Fujita wrote: >> On Wed, Apr 8, 2020 at 4:30 PM Kuntal Ghosh >> wrote: >> > I'm getting the following warning during compilation. >> &g

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

On Mon, Apr 13, 2020 at 5:20 PM Dilip Kumar wrote: > On Mon, Apr 13, 2020 at 4:14 PM Kuntal Ghosh > wrote: > > > > +#define SizeOfTransactionId (sizeof(TransactionId) + sizeof(char)) > > This looks wrong. We should change the name of this Macro or we can >

Re: Parallel copy

is just impractical for PG due to certain limitations. Thoughts? [1] https://www.microsoft.com/en-us/research/uploads/prod/2019/04/chunker-sigmod19.pdf [2] ParaText. https://github.com/wiseio/paratext. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: Parallel copy

On Wed, Apr 15, 2020 at 10:45 PM Andres Freund wrote: > > Hi, > > On 2020-04-15 20:36:39 +0530, Kuntal Ghosh wrote: > > I was thinking from this point of view - the sooner we introduce > > parallelism in the process, the greater the benefits. > > I don't really

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

tion() to dump the (streaming = 'on') option. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: SLRU statistics

update hit_count if we can find the buffer in SimpleLruReadPage_ReadOnly directly. Am I missing something? Attached a patch for the same. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com v1-0001-Update-stats-in-SimpleLruReadPage_ReadOnly.patch Description: Binary data

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

g: unused variable ‘inner_binfo’ [-Wunused-variable] PartitionBoundInfo inner_binfo = inner_rel->boundinfo; ^ For fixing the same, we can declare inner_binfo as PG_USED_FOR_ASSERTS_ONLY as it is not used for any other purpose. -- Thanks & Regards, Kuntal Ghosh En

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

ic bool +ReorderBufferCanStream(ReorderBuffer *rb) +{ + LogicalDecodingContext *ctx = rb->private_data; + + return ctx->streaming; +} Potential inline function. +static void +ReorderBufferStreamTXN(ReorderBuffer *rb, ReorderBufferTXN *txn) +{ + volatile Snapshot snapshot_now; + volatile CommandId com

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

NAPBUILD_FULL_SNAPSHOT && - info != XLOG_XACT_ASSIGNMENT) + !TransactionIdIsValid(r->toplevel_xid)) Perhaps, XLogRecGetTopXid() can be used. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

On Fri, Mar 13, 2020 at 8:42 AM Dilip Kumar wrote: > > On Thu, Mar 12, 2020 at 7:50 PM Kuntal Ghosh > > wrote: > > > > > + /* > > > + * The relation extension or page lock can never participate in actual > > > + * deadlock cy

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

On Fri, Mar 13, 2020 at 8:29 AM Amit Kapila wrote: > > On Thu, Mar 12, 2020 at 7:50 PM Kuntal Ghosh > wrote: > > I think moving them inside a macro is a good idea. Also, I think we > > should move all the Assert related code inside some debugging macro > > similar to

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

except for relation extension lock. Same as above Other than that, the patches look good to me. I've also done some testing after applying the Test-group-deadlock patch provided by Amit earlier in the thread. It works as expected. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: WAL usage calculation patch

Pos == InvalidXLogRecPtr); I think there are some issues in the num_fpw calculation. For some cases, we have to return from XLogInsert without inserting a record. Basically, we've to recompute/reassemble the same record. In those cases, num_fpw should be reset. Thoughts? -- Thanks & Regards, Kunta

Re: WAL usage calculation patch

On Tue, Mar 31, 2020 at 7:39 PM Julien Rouhaud wrote: > > On Tue, Mar 31, 2020 at 12:21 PM Kuntal Ghosh > wrote: > > > > On Mon, Mar 30, 2020 at 6:14 PM Julien Rouhaud wrote: > > > > > @@ -448,6 +449,7 @@ XLogInsert(RmgrId rmid, uint8 info) > > b

Re: Parallel copy

On Wed, Apr 15, 2020 at 2:15 PM Ants Aasma wrote: > > On Tue, 14 Apr 2020 at 22:40, Kuntal Ghosh wrote: > > 1. Each worker scans a distinct fixed sized chunk of the CSV file and > > collects the following three stats from the chunk: > > a) number of quotes > > b)

Re: Incorrect assumption in heap_prepare_freeze_tuple

ldestXmin. Thus the only tuples that the HTSV() we're > talking about can return DEAD for are ones that were RECENTLY_DEAD > in heap_page_prune(). > Got it. Thank you for the explanations. :-) -- Thanks & Regards, Kuntal Ghosh

Incorrect assumption in heap_prepare_freeze_tuple

Attached a patch for the same. Thoughts? -- Thanks & Regards, Kuntal Ghosh 0001-Fix-sanity-check-for-HOT-updated-tuple-when-freezing.patch Description: Binary data

Re: Incorrect assumption in heap_prepare_freeze_tuple

you're right that any tuple older than cutoff_xid (since it was set earlier) will be pruned by heap_page_prune and hence we shouldn't encounter the error. I'll study the rewrite_heap_tuple path as well. > What made you look at this? Did you hit the error? Nope, I haven't encountered the error. Just trying to understand the code. :-) -- Thanks & Regards, Kuntal Ghosh

Re: Incorrect assumption in heap_prepare_freeze_tuple

On Sat, Oct 3, 2020 at 1:06 PM Andres Freund wrote: > On 2020-10-03 12:58:01 +0530, Kuntal Ghosh wrote: > > IIUC, there can be a HOT-updated tuple which is not initially pruned > > by heap_page_prune but later diagnosed HEAPTUPLE_DEAD by > > HeapTupleSatisfiesVacuum

Re: [BUG]Update Toast data failure in logical replication

e to detoast the key. + */ This doesn't really mention why we need to detoast the key even when the key remains the same. I guess we can add some more details here. -- Thanks & Regards, Kuntal Ghosh

Re: Extensibility of the PostgreSQL wire protocol

ail in the same function: /* Should only be called by old-style COPY OUT */ Assert(DoingCopyOut); -- Thanks & Regards, Kuntal Ghosh Amazon Web Services

Re: Add connection active, idle time to pg_stat_activity

backend is idle for a long time since the last execution and the execution time of the last query (query_end - query_start). You also need to update the documentation. -- Thanks & Regards, Kuntal Ghosh

Re: Invalid memory access in pg_stat_get_subscription

Hello Tom, On Wed, Jun 8, 2022 at 12:44 AM Tom Lane wrote: > > Kuntal Ghosh writes: > > While exploring some code in logical replication worker > > implementation, I noticed that we're accessing an invalid memory while > > traversing LogicalRepCtx->workers[i]. > &g

Invalid memory access in pg_stat_get_subscription

ory in ApplyLauncherShmemSize. But, in the for loop, we're accessing the max_logical_replication_workers + 1 location which is resulting in random crashes. Please find the patch that fixes the issue. I'm not sure whether we should add a regression test for the same. -- Thanks & Regards, Kunta

Re: Allow logical replication to copy tables in binary format

that behaviour in a major version release. For the existing applications that are using (or unknowingly misusing) the feature, as Amit mentioned, they have a workaround. -- Thanks & Regards, Kuntal Ghosh

Re: Improve WALRead() to suck data directly from WAL buffers when possible

rmance is a concern, we can introduce a GUC to enable/disable this feature. -- Thanks & Regards, Kuntal Ghosh

Re: Avoid multiple SetLatch() calls in procsignal_sigusr1_handler()

it seems unnecessary. > +1 -- Thanks & Regards, Kuntal Ghosh