Re: speed up a logical replica setup

2023-11-02 Thread Ashutosh Bapat
On Wed, Nov 1, 2023 at 7:10 PM Ashutosh Bapat wrote: > > I noticed some differences between this and a similar utility > https://github.com/2ndQuadrant/pglogical/blob/REL2_x_STABLE/pglogical_create_subscriber.c. > I will be reviewing these differences next to see if we are missing >

Re: speed up a logical replica setup

2023-11-01 Thread Ashutosh Bapat
On Thu, Oct 26, 2023 at 5:17 PM Ashutosh Bapat wrote: > > On Mon, Oct 23, 2023 at 9:34 AM Euler Taveira wrote: > > > > > It is still a WIP but I would like to share it and get some feedback. > > > > > > I have started reviewing the patch. I have just

Relids instead of Bitmapset * in plannode.h

2023-10-31 Thread Ashutosh Bapat
Hi, For some reason plannode.h has declared variable to hold RTIs as Bitmapset * instead of Relids like other places. Here's patch to fix it. This is superficial change as Relids is typedefed to Bitmapset *. Build succeeds for me and also make check passes. -- Best Wishes, Ashutosh Bapat From

Re: Reuse child_relids in try_partitionwise_join was Re: Assert failure on bms_equal(child_joinrel->relids, child_joinrelids)

2023-10-30 Thread Ashutosh Bapat
On Fri, Sep 8, 2023 at 3:22 PM Ashutosh Bapat wrote: > > On Fri, Jul 28, 2023 at 3:16 PM Ashutosh Bapat > wrote: > > > > Hi Tom, Richard, > > > > On Mon, Jul 24, 2023 at 8:17 AM Richard Guo wrote: > > > > > > Thanks for pushing it!

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2023-10-30 Thread Ashutosh Bapat
On Thu, Sep 14, 2023 at 4:39 PM Ashutosh Bapat wrote: > > The patch set is thus > 0001 - patch used to measure memory used during planning > > 0002 - Patch to free intermediate Relids computed by > adjust_child_relids_multilevel(). I didn't test memory consumption for > mul

Re: Memory consumed by child SpecialJoinInfo in partitionwise join planning

2023-10-30 Thread Ashutosh Bapat
Rebased patchset. No changes in actual patches. 0001 is squashed version of patches submitted at https://www.postgresql.org/message-id/CAExHW5sCJX7696sF-OnugAiaXS=Ag95=-m1csrjcmyyj8pd...@mail.gmail.com. On Tue, Oct 3, 2023 at 6:42 PM Ashutosh Bapat wrote: > > On Fri, Sep 29, 2023 at 8:36 A

Re: Report planning memory in EXPLAIN ANALYZE

2023-10-30 Thread Ashutosh Bapat
hing is wrong. > Thanks Andy. Here's rebased patches. A conflict in explain.out resolved. -- Best Wishes, Ashutosh Bapat From 171ce2bd03f846e7ba3e6972b1f51a432d5f75c5 Mon Sep 17 00:00:00 2001 From: Ashutosh Bapat Date: Wed, 23 Aug 2023 16:23:12 +0530 Subject: [PATCH 2/3] Report memory a

Re: Not deleted mentions of the cleared path

2023-10-30 Thread Ashutosh Bapat
elements > [0].ptr_value)->subpath)->path->parent->cheapest_startup_path > $20 = (struct Path *) 0x55bf9ef7f4e0 This looks familiar though. There was some nearby thread where Tom Lane, if my memory serves well, provided a case where a path from lower rel was added to an upper rel without copying or changing its parent. This very much looks like that case. -- Best Wishes, Ashutosh Bapat

Re: Why is DEFAULT_FDW_TUPLE_COST so insanely low?

2023-10-30 Thread Ashutosh Bapat
ike socket. This will also mean that operations which reduce the number of rows will be favoured and pushed down. That's what is desired. -- Best Wishes, Ashutosh Bapat On Mon, Oct 30, 2023 at 6:52 AM David Rowley wrote: > > On Sun, 29 Oct 2023 at 12:45, Bruce Momjian wrote: > > Has an

Re: BRIN minmax multi - incorrect distance for infinite timestamp/date

2023-10-29 Thread Ashutosh Bapat
On Fri, Oct 27, 2023 at 10:32 PM Tomas Vondra wrote: > > FWIW I've cleaned up and pushed all the patches we came up with this > thread. And I've backpatched all of them to 14+. > Thanks a lot Tomas. -- Best Wishes, Ashutosh Bapat

Re: btree_gin: Incorrect leftmost interval value

2023-10-27 Thread Ashutosh Bapat
e interval patches to commit but I guess, the wait won't be too long and the outcome will be better. I can include this change in those patches. -- Best Wishes, Ashutosh Bapat

Re: Infinite Interval

2023-10-27 Thread Ashutosh Bapat
On Fri, Oct 27, 2023 at 2:07 PM Dean Rasheed wrote: > > On Wed, 4 Oct 2023 at 14:29, Ashutosh Bapat > wrote: > > > > I think we should introduce interval_out_of_range_error() function on > > the lines of float_overflow_error(). Later patches introduce more > >

partitioning and identity column

2023-10-27 Thread Ashutosh Bapat
t/70be435b-05db-06f2-7c01-9bb8ee2fccce%40enterprisedb.com [2] https://www.postgresql.org/message-id/caexhw5toasjc7uwseszx6sgvktfxsv7pd606zp6dntx7y6o...@mail.gmail.com -- Best Wishes, Ashutosh Bapat

Re: Use virtual tuple slot for Unique node

2023-10-27 Thread Ashutosh Bapat
On Fri, Oct 27, 2023 at 8:48 AM David Rowley wrote: > > On Wed, 25 Oct 2023 at 22:48, Ashutosh Bapat > wrote: > > We may save the size of data in VirtualTupleTableSlot, thus avoiding > > the first loop. I assume that when allocating > > VirtualTupleTableSlot->dat

Re: speed up a logical replica setup

2023-10-26 Thread Ashutosh Bapat
nstead I see create_logical_replication_slot() in pg_subscriber.c. Which of these two you intend to use finally? -- Best Wishes, Ashutosh Bapat

Re: CDC/ETL system on top of logical replication with pgoutput, custom client

2023-10-25 Thread Ashutosh Bapat
ed. Please take a look at XLogSendLogical(void) and the places where WalSndKeepaliveIfNecessary() is called. -- Best Wishes, Ashutosh Bapat

Re: CDC/ETL system on top of logical replication with pgoutput, custom client

2023-10-25 Thread Ashutosh Bapat
On Tue, Oct 24, 2023 at 8:53 PM José Neves wrote: > > Hi there, hope to find you well. > > I have a follow-up question to this already long thread. > > Upon deploying my PostgreSQL logical replication fed application on a stale > database, I ended up running out of space, as the replication slot

Re: Use virtual tuple slot for Unique node

2023-10-25 Thread Ashutosh Bapat
On Tue, Oct 24, 2023 at 4:30 AM David Rowley wrote: > > On Fri, 20 Oct 2023 at 22:30, Ashutosh Bapat > wrote: > > I ran my experiments again. It seems on my machine the execution times > > do vary a bit. I ran EXPLAIN ANALYZE on the query 5 times and took > > averag

Re: RFC: Logging plan of the running query

2023-10-24 Thread Ashutosh Bapat
t; However perhaps there is still an opportunity for moving some of the > auto_explain code into core so as to enable deduplicating the code. > Right. That's what I also think. -- Best Wishes, Ashutosh Bapat

Re: BRIN minmax multi - incorrect distance for infinite timestamp/date

2023-10-20 Thread Ashutosh Bapat
On Thu, Oct 19, 2023 at 6:11 PM Ashutosh Bapat wrote: > > I think we should provide generate_series(date, date, integer) which > will use date + integer -> date. Just to be clear, I don't mean that this patch should add it. -- Best Wishes, Ashutosh Bapat

Re: Use virtual tuple slot for Unique node

2023-10-20 Thread Ashutosh Bapat
pposed to be faster. We use same slot types for input and output in other places where as well. May be we should fix the above said inefficiency in tt_virtual_copyslot()? -- Best Wishes, Ashutosh Bapat

Re: BRIN minmax multi - incorrect distance for infinite timestamp/date

2023-10-19 Thread Ashutosh Bapat
On Thu, Oct 19, 2023 at 4:51 PM Tomas Vondra wrote: > > On 10/19/23 11:22, Ashutosh Bapat wrote: > > On Thu, Oct 19, 2023 at 2:31 PM Tomas Vondra > > wrote: > > > >> > >> Does that explain the algorithm? I'm not against clarifying the com

Re: BRIN minmax multi - incorrect distance for infinite timestamp/date

2023-10-19 Thread Ashutosh Bapat
9 BC 4713-01-08 BC 4713-01-07 BC 4713-01-06 BC 4713-01-05 BC 4713-01-04 BC 4713-01-03 BC 4713-01-02 BC 4713-01-01 BC (32 rows) -- Best Wishes, Ashutosh Bapat

Re: BRIN minmax multi - incorrect distance for infinite timestamp/date

2023-10-18 Thread Ashutosh Bapat
that in the commit message so that the person > > writing release notes picks it up from there? > > > > Yes, I think I'll mention what impact each issue can have on indexes. Thanks. -- Best Wishes, Ashutosh Bapat

Re: BRIN minmax multi - incorrect distance for infinite timestamp/date

2023-10-18 Thread Ashutosh Bapat
), so the > distances are negated. Which means we actually merge the values from the > most distant ones, and thus are "guaranteed" to build very a very > inefficient summary. People with multi-minmax indexes on "date" columns > probably will need to reindex. > R

Re: RFC: Logging plan of the running query

2023-10-18 Thread Ashutosh Bapat
t it in core, you will need to port relevant parts of auto_explain code to core. -- Best Wishes, Ashutosh Bapat

Re: Asymmetric partition-wise JOIN

2023-10-18 Thread Ashutosh Bapat
f > RelOptInfo ? See Scan::scanrelid -- Best Wishes, Ashutosh Bapat

Re: Asymmetric partition-wise JOIN

2023-10-17 Thread Ashutosh Bapat
On Tue, Oct 17, 2023 at 2:05 PM Andrei Lepikhov wrote: > > On 16/10/2023 23:21, Ashutosh Bapat wrote: > > On Mon, Oct 16, 2023 at 10:24 AM Andrei Lepikhov > > Whenever I visited this idea, I hit one issue prominently - how would > > we differentiate different scans of the

serial and partitioned table

2023-10-17 Thread Ashutosh Bapat
titioned tables. [1] https://www.postgresql.org/message-id/CAExHW5uRUtDfU0R8zXofQxCV3S1B%2BPa%2BX%2BNrpMwzKraLc25%3DEg%40mail.gmail.com [2] https://www.postgresql.org/message-id/flat/70be435b-05db-06f2-7c01-9bb8ee2fccce%40enterprisedb.com -- Best Wishes, Ashutosh Bapat

odd behaviour with serial, non null and partitioned table

2023-10-17 Thread Ashutosh Bapat
as attached to tpart as a partition. Is this backward compatibility break intentional? I haven't followed NOT NULL constraint thread closely. I might have missed some discussion. -- Best Wishes, Ashutosh Bapat

Re: Asymmetric partition-wise JOIN

2023-10-16 Thread Ashutosh Bapat
to know the number of such relations involved in the query unless we start planning such a join. It's late to add new base relations and assign them new Relids. Of course I haven't thought hard about it. I haven't looked at the patch to see whether this problem is solved and how. -- Best Wishes, Ashutosh Bapat

Re: BRIN minmax multi - incorrect distance for infinite timestamp/date

2023-10-16 Thread Ashutosh Bapat
On Mon, Oct 16, 2023 at 7:33 PM Tomas Vondra wrote: > > On 10/16/23 11:25, Ashutosh Bapat wrote: > > Thanks Tomas for bringing this discussion to hackers. > > > > > > On Fri, Oct 13, 2023 at 8:58 PM Dean Rasheed > > wrote: > >> > >>

Re: RFC: Logging plan of the running query

2023-10-16 Thread Ashutosh Bapat
On Thu, Oct 12, 2023 at 6:51 PM torikoshia wrote: > > On 2023-10-11 16:22, Ashutosh Bapat wrote: > > > > Considering the similarity with auto_explain I wondered whether this > > function should be part of auto_explain contrib module itself? If we > > do that users

Re: BRIN minmax multi - incorrect distance for infinite timestamp/date

2023-10-16 Thread Ashutosh Bapat
s not true. Tomas, you may want to incorporate this in your patch. If not, I will incorporate it in my infinite interval patchset in [1]. [1] https://www.postgresql.org/message-id/CAExHW5u1JE7dxK=wlzqhcszntoxqzjdiermhrepw6r8w6kc...@mail.gmail.com -- Best Wishes, Ashutosh Bapat

Re: Use virtual tuple slot for Unique node

2023-10-12 Thread Ashutosh Bapat
uple table slot is so narrow when large data needs to be copied that the execution times almost match. The patched and unpatched execution times differ by the margin of error either way. -- Best Wishes, Ashutosh Bapat

Re: RFC: Logging plan of the running query

2023-10-11 Thread Ashutosh Bapat
but may not be necessary for the first cut. -- Best Wishes, Ashutosh Bapat

Re: Check each of base restriction clauses for constant-FALSE-or-NULL

2023-10-09 Thread Ashutosh Bapat
ou have an example where this could be demonstrated? -- Best Wishes, Ashutosh Bapat

Re: Draft LIMIT pushdown to Append and MergeAppend patch

2023-10-09 Thread Ashutosh Bapat
On Mon, Oct 9, 2023 at 4:33 PM David Rowley wrote: > > On Mon, 9 Oct 2023 at 23:35, Ashutosh Bapat > wrote: > > > > On Mon, Oct 9, 2023 at 6:25 AM David Rowley wrote: > > > > > > However, it may also be worth you reading over [3] and the ultimate >

Re: Draft LIMIT pushdown to Append and MergeAppend patch

2023-10-09 Thread Ashutosh Bapat
turned ON. The solution we will develop here will solve problem with partitionwise join as well. It's hard to solve this problem. If there's a real case where LIMIT pushdown helps without fixing Sort pushdown case, it might help proceeding with the same. -- Best Wishes, Ashutosh Bapat

Re: Check each of base restriction clauses for constant-FALSE-or-NULL

2023-10-09 Thread Ashutosh Bapat
> -- patched > explain (costs off) > select * from t t1 left join (select * from t t2 where false) s on s.a = 1; >QUERY PLAN > > Nested Loop Left Join >-> Seq Scan on t t1 >-> Result > One-Time Filter: false > (4 rows) Does your code have any other benefits like deeming an inner join as empty? -- Best Wishes, Ashutosh Bapat

Re: FDW LIM IT pushdown

2023-10-09 Thread Ashutosh Bapat
's being used for costing or for pushing it down to the partitions. This isn't a full answer, but I hope these pointers would help you. -- Best Wishes, Ashutosh Bapat

Re: ALTER COLUMN ... SET EXPRESSION to alter stored generated column's expression

2023-10-06 Thread Ashutosh Bapat
; Should we treat it the same fashion as ALTER COLUMN ... TYPE which rewrites the column values? Of course that rewrites the whole table, but logically they are comparable. -- Best Wishes, Ashutosh Bapat

Re: Two Window aggregate node for logically same over clause

2023-10-06 Thread Ashutosh Bapat
ery execution. Is there any reason for creating two > different window aggregate node? I don't see any. https://www.postgresql.org/docs/16/sql-select.html description of ORDER BY clause clearly says that ASC is assumed when no direction is mentioned. The only place in code which is used to crea

Re: FDW pushdown of non-collated functions

2023-10-06 Thread Ashutosh Bapat
years at least, may be such a simple fix is not enough. Another solution would be to attach another attribute to a function indicating whether the output of that function depends upon the input collations or not. Doing that just for FDW may not be acceptable though. -- Best Wishes, Ashutosh Bapat

Re: Infinite Interval

2023-10-04 Thread Ashutosh Bapat
ostgresql.org/docs/current/datatype-datetime.html#DATATYPE-DATETIME [3] https://www.postgresql.org/message-id/CAAvxfHf0-T99i%3DOrve_xfonVCvsCuPy7C4avVm%3D%2Byu128ujSGg%40mail.gmail.com [4] https://www.postgresql.org/message-id/26022.1545087636%40sss.pgh.pa.us [5] https://www.postgresql.org/message-id/CAAvxfHdzd5JLRBXDAW7OPhsNNACvhsCP3f5R4LNhRVaDuQG0gg%40mail.gmail.com -- Best Wishes, Ashutosh Bapat

Re: Memory consumed by child SpecialJoinInfo in partitionwise join planning

2023-10-03 Thread Ashutosh Bapat
Ds and the list of expressions may be > -* referenced somewhere else. Do not free those. > -*/ > + /* semi_rhs_exprs may be referenced, so don't free. */ > } Works for me. PFA patchset with these changes. I have still left the changes addressing your c

Re: Doc: Minor update for enable_partitionwise_aggregate

2023-10-03 Thread Ashutosh Bapat
ly off to me to say the "partitioned tables" instead of the partitions. It's technically incorrect as well. Aggregation is performed on a single relation always - a join or subquery or simple relation. A join may have multiple tables in it but the aggregation is performed on its result and not individual tables and hence not on partitions of individual tables. -- Best Wishes, Ashutosh Bapat

Re: Memory consumed by child SpecialJoinInfo in partitionwise join planning

2023-09-27 Thread Ashutosh Bapat
ous patch set squashed together 0003 - changes addressing above comments. -- Best Wishes, Ashutosh Bapat From d7e65bf7c277c3dd3afbd1b7a152b86d24bd6904 Mon Sep 17 00:00:00 2001 From: Ashutosh Bapat Date: Wed, 27 Sep 2023 16:29:11 +0530 Subject: [PATCH 3/3] Address Amit's comments --- src/backend

Re: Avoid a possible out-of-bounds access (src/backend/optimizer/util/relnode.c)

2023-09-26 Thread Ashutosh Bapat
_rel() as this would make the case for using > PlannerInfo.simple_rel_array directly even stronger. I am curious, is the overhead in find_base_rel() impacting overall performance? -- Best Wishes, Ashutosh Bapat

Re: Avoid a possible out-of-bounds access (src/backend/optimizer/util/relnode.c)

2023-09-25 Thread Ashutosh Bapat
On Mon, Sep 25, 2023 at 7:14 PM Ranier Vilela wrote: > > Em seg., 25 de set. de 2023 às 08:23, Ashutosh Bapat > escreveu: >> >> On Sat, Sep 23, 2023 at 7:29 PM Ranier Vilela wrote: >> > >> > Hi, >> > >> > Per Coverity. &

Re: Avoid a possible out-of-bounds access (src/backend/optimizer/util/relnode.c)

2023-09-25 Thread Ashutosh Bapat
bms_make_singleton() has an assertion at the end: Assert(result >= 0); bms_make_singleton(), bms_add_member() explicitly forbids negative values. It looks like we have covered all the places which can add a negative value to a bitmapset. May be we are missing a place or two. It will be good to

Re: Infinite Interval

2023-09-22 Thread Ashutosh Bapat
NULL inputs if they are *not* strict. #select proisstrict from pg_proc where proname = 'interval_avg_serialize'; proisstrict - t (1 row) #select proisstrict from pg_proc where proname = 'interval_avg_deserialize'; proisstrict - t (1 row) -- Best Wishes, Ashutosh Bapat

Re: Report planning memory in EXPLAIN ANALYZE

2023-09-22 Thread Ashutosh Bapat
t terms indicating memory actually used vs memory allocated. > > Personally I am pretty like patch 1, we just need to refactor some words > to make everything clearer. Thanks. -- Best Wishes, Ashutosh Bapat

Re: Infinite Interval

2023-09-21 Thread Ashutosh Bapat
On Wed, Sep 20, 2023 at 8:23 PM Dean Rasheed wrote: > > On Wed, 20 Sept 2023 at 15:00, Ashutosh Bapat > wrote: > > > > 0005 - Refactored Jian's code fixing window functions. Does not > > contain the changes for serialization and deserialization. Jian, > > pl

Re: Memory consumed by child SpecialJoinInfo in partitionwise join planning

2023-09-20 Thread Ashutosh Bapat
On Thu, Sep 21, 2023 at 6:37 AM Amit Langote wrote: > > On Wed, Sep 20, 2023 at 10:24 PM Ashutosh Bapat > wrote: > > On Wed, Sep 20, 2023 at 5:24 PM Amit Langote > > wrote: > > > Just one comment on 0003: > > > > > > + /* > > > +*

Re: Infinite Interval

2023-09-20 Thread Ashutosh Bapat
at finite values don't sum to our representation of infinity, > as in interval_pl(). Fixed in the latest patch set. -- Best Wishes, Ashutosh Bapat

Re: Infinite Interval

2023-09-20 Thread Ashutosh Bapat
On Mon, Sep 18, 2023 at 5:09 PM Dean Rasheed wrote: > > On Wed, 13 Sept 2023 at 11:13, Ashutosh Bapat > wrote: > > > > On Tue, Sep 12, 2023 at 2:39 PM Dean Rasheed > > wrote: > > > > > > and it looks like the infinite interval > > > inp

Re: Memory consumed by child SpecialJoinInfo in partitionwise join planning

2023-09-20 Thread Ashutosh Bapat
tion build the rest of the code will free Relids which are referenced somewhere else causing dangling pointers. -- Best Wishes, Ashutosh Bapat

Re: persist logical slots to disk during shutdown checkpoint

2023-09-20 Thread Ashutosh Bapat
for the updated version. > > > > Pushed. Commitfest entry "https://commitfest.postgresql.org/44/4536/ is in "Ready for committer" state. Is there something remaining here? We should probably set it as "committed". -- Best Wishes, Ashutosh Bapat

Re: [PoC] Reducing planning time when tables have many partitions

2023-09-20 Thread Ashutosh Bapat
l be simplified by your approach. [1] https://www.postgresql.org/message-id/caexhw5u0yyyr2mwvlrvvy_qnld65kpc9u-bo0ox7bglkgba...@mail.gmail.com -- Best Wishes, Ashutosh Bapat

Re: Infinite Interval

2023-09-18 Thread Ashutosh Bapat
nfinity is right. But timestamp corresponds to both timestamp with and without timezone as per table 8.9 on the same page . https://www.postgresql.org/docs/current/datatype-datetime.html#DATATYPE-DATETIME-TABLE. So I don't see a need to specify timestamptz separately. [1] https://www.postgresql.org/message-id/CAExHW5ut4bR4KSNWAhXb_EZ8PyY=j100gua6zumnhvoia1z...@mail.gmail.com -- Best Wishes, Ashutosh Bapat

Re: ALTER COLUMN ... SET EXPRESSION to alter stored generated column's expression

2023-09-14 Thread Ashutosh Bapat
reasonable - following precedence is usually good. But I haven't reviewed the code to comment on it. [1] https://www.postgresql.org/docs/16/ddl-generated-columns.html -- Best Wishes, Ashutosh Bapat

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2023-09-14 Thread Ashutosh Bapat
Hi All, On Fri, Aug 11, 2023 at 6:24 PM Ashutosh Bapat wrote: > > Obtaining child clauses from parent clauses by translation and > tracking the translations is less complex and may be more efficient > too. I will post a patch on those lines soon. > PFA patch set to add infrastr

Re: logical decoding and replication of sequences, take 2

2023-09-13 Thread Ashutosh Bapat
On Fri, Aug 18, 2023 at 4:28 PM Amit Kapila wrote: > > On Fri, Aug 18, 2023 at 10:37 AM Amit Kapila wrote: > > > > On Thu, Aug 17, 2023 at 7:13 PM Ashutosh Bapat > > wrote: > > > > > > On Wed, Aug 16, 2023 at 7:56 PM Tomas Vondra > > &

Re: logical decoding and replication of sequences, take 2

2023-09-13 Thread Ashutosh Bapat
On Thu, Aug 17, 2023 at 7:13 PM Ashutosh Bapat wrote: > > > > The attached patch merges the earlier improvements, except for the part > > that experimented with adding a "fake" transaction (which turned out to > > have a number of difficult issues). > > 000

Reuse child_relids in try_partitionwise_join was Re: Assert failure on bms_equal(child_joinrel->relids, child_joinrelids)

2023-09-08 Thread Ashutosh Bapat
On Fri, Jul 28, 2023 at 3:16 PM Ashutosh Bapat wrote: > > Hi Tom, Richard, > > On Mon, Jul 24, 2023 at 8:17 AM Richard Guo wrote: > > > > Thanks for pushing it! > > With this fix, I saw a noticeable increase in the memory consumption > of planner. I was ru

Re: Oversight in reparameterize_path_by_child leading to executor crash

2023-09-08 Thread Ashutosh Bapat
ystery. I am not sure where we are taking the original bug fix with this investigation. Is it required to fix this problem in order to fix the original problem OR we should commit the fix for the original problem and then investigate this further? -- Best Wishes, Ashutosh Bapat

Re: [BackendXidGetPid] only access allProcs when xid matches

2023-09-07 Thread Ashutosh Bapat
On Thu, Sep 7, 2023 at 3:05 PM Junwang Zhao wrote: > > On Thu, Sep 7, 2023 at 5:07 PM Ashutosh Bapat > wrote: > > > > Forgot to attach the patch. > > LGTM > > Should I change the status to ready for committer now? > I would still love to see some numbers.

Re: persist logical slots to disk during shutdown checkpoint

2023-09-07 Thread Ashutosh Bapat
On Thu, Sep 7, 2023 at 4:11 PM Amit Kapila wrote: > > On Thu, Sep 7, 2023 at 3:38 PM Ashutosh Bapat > wrote: > > > > * This needn't actually be part of a checkpoint, but it's a convenient > > - * location. > > + * location. is_shutdown is true in case of a shu

Re: persist logical slots to disk during shutdown checkpoint

2023-09-07 Thread Ashutosh Bapat
is commented upon in the relevant code. * This needn't actually be part of a checkpoint, but it's a convenient - * location. + * location. is_shutdown is true in case of a shutdown checkpoint. Relying on the first sentence, if we decide to not persist the replication slot at the time of checkpoint, would that be OK? It doesn't look like a convenience thing to me any more. -- Best Wishes, Ashutosh Bapat

Re: [BackendXidGetPid] only access allProcs when xid matches

2023-09-07 Thread Ashutosh Bapat
Forgot to attach the patch. On Thu, Sep 7, 2023 at 1:22 PM Ashutosh Bapat wrote: > > Hi Junwang, > We leave a line blank after variable declaration as in the attached patch. > > Otherwise the patch looks good to me. > > The function modified by the patch is only used by e

Re: [BackendXidGetPid] only access allProcs when xid matches

2023-09-07 Thread Ashutosh Bapat
be able to catch some miniscule improvement per row. If the performance is measurable, we can mark the CF entry as ready for committer. -- Best Wishes, Ashutosh Bapat On Thu, Aug 10, 2023 at 1:48 PM Junwang Zhao wrote: > > On Thu, Aug 10, 2023 at 4:11 PM Ashutosh Bapat > wrote: >

Re: [PoC] Reducing planning time when tables have many partitions

2023-09-07 Thread Ashutosh Bapat
including what I > mentioned above. It seems that you are still investigating and fixing issues. But the CF entry is marked as "needs review". I think a better status is "WoA". Do you agree with that? -- Best Wishes, Ashutosh Bapat

Re: Optimize planner memory consumption for huge arrays

2023-09-06 Thread Ashutosh Bapat
to create a Const node for each element. > > The diff in the attachment is proof of concept showing how to reduce wasting > of memory. Having benchmarked a bit, I didn't find any overhead. > You might want to include your benchmarking results as well. -- Best Wishes, Ashutosh Bapat

Re: Speaker Bureau

2023-09-04 Thread Ashutosh Bapat
Added my name to the list. I am available to travel for meetups. -- Best Wishes, Ashutosh Bapat On Sat, Sep 2, 2023 at 12:30 AM Dave Cramer wrote: > > Greetings, > > If you are on the speaker list can you send an email to ugc...@postgresql.us > indicating whether you are avai

Re: [17] CREATE SUBSCRIPTION ... SERVER

2023-09-04 Thread Ashutosh Bapat
sense. But it's not. We will continue to support current mechanism which doesn't require postgres_fdw to be installed on every PostgreSQL cluster. What security problems do you foresee if postgres_fdw is used in addition to the current mechanism? -- Best Wishes, Ashutosh Bapat

Re: [17] CREATE SUBSCRIPTION ... SERVER

2023-09-04 Thread Ashutosh Bapat
On Sat, Sep 2, 2023 at 12:24 AM Jeff Davis wrote: > > On Fri, 2023-09-01 at 12:28 +0530, Ashutosh Bapat wrote: > > Thinking larger, how about we allow any FDW to be used here. > > That's a possibility, but I think that means the subscription would > need to constantly re

Re: persist logical slots to disk during shutdown checkpoint

2023-09-01 Thread Ashutosh Bapat
On Thu, Aug 31, 2023 at 7:28 PM Amit Kapila wrote: > > On Thu, Aug 31, 2023 at 6:12 PM Ashutosh Bapat > wrote: > > > > On Thu, Aug 31, 2023 at 2:52 PM Amit Kapila wrote: > > > > > > I > > > > > > think we should shut down

Re: [17] CREATE SUBSCRIPTION ... SERVER

2023-09-01 Thread Ashutosh Bapat
On Fri, Sep 1, 2023 at 2:47 AM Joe Conway wrote: > > On 8/31/23 12:52, Jeff Davis wrote: > > On Thu, 2023-08-31 at 10:59 +0530, Ashutosh Bapat wrote: > >> The server's FDW has to be postgres_fdw. So we have to handle the > >> awkward dependency between core

Re: persist logical slots to disk during shutdown checkpoint

2023-08-31 Thread Ashutosh Bapat
catioinSlotMarkDirty() followed by ReplicationSlotSave() > > at all the places, even those which are more frequent than this. > > > > All but one. Normally, the idea of marking dirty is to indicate that > we will actually write/flush the contents at a later point (except > when required for correctness) as even indicated in the comments atop > ReplicatioinSlotMarkDirty(). However, I see your point that we use > that protocol at all the current places including CreateSlotOnDisk(). > So, we can probably do it here as well. yes I didn't see this entry in commitfest. Since we are discussing it and the next CF is about to begin, probably it's good to add one there. -- Best Wishes, Ashutosh Bapat

Re: Statistics Import and Export

2023-08-31 Thread Ashutosh Bapat
building it locally. If it's known that the stats on foreign and local servers match for a foreign table, we will be one step closer to accurately estimating the cost of a foreign plan locally rather than through EXPLAIN. -- Best Wishes, Ashutosh Bapat

Re: persist logical slots to disk during shutdown checkpoint

2023-08-31 Thread Ashutosh Bapat
h benefit. Sure, it may not be performance-sensitive but still > adding another pair of lock/release doesn't seem like a better idea. We call ReplicatioinSlotMarkDirty() followed by ReplicationSlotSave() at all the places, even those which are more frequent than this. So I think it's better to stick to that protocol rather than adding a new flag. -- Best Wishes, Ashutosh Bapat

Re: [17] CREATE SUBSCRIPTION ... SERVER

2023-08-30 Thread Ashutosh Bapat
On Wed, Aug 30, 2023 at 9:00 PM Jeff Davis wrote: > > On Wed, 2023-08-30 at 19:11 +0530, Ashutosh Bapat wrote: > > Are you suggesting that SERVERs created with FDW can not be used as > > publishers? > > Correct. Without that, how would the subscription know that the FDW &

Re: [17] CREATE SUBSCRIPTION ... SERVER

2023-08-30 Thread Ashutosh Bapat
cting this feature to only CONNECTION ONLY, we are restricting the possibility of such a push down. -- Best Wishes, Ashutosh Bapat

Re: persist logical slots to disk during shutdown checkpoint

2023-08-30 Thread Ashutosh Bapat
On Tue, Aug 29, 2023 at 5:40 PM Ashutosh Bapat wrote: > > I am looking at it. If you can wait till the end of the week, that > will be great. /* * Successfully wrote, unset dirty bit, unless somebody dirtied again - * already. + * already and remember the confirmed_

Re: persist logical slots to disk during shutdown checkpoint

2023-08-29 Thread Ashutosh Bapat
u have more comments or suggestions, I would like to push > this in a day or two. I am looking at it. If you can wait till the end of the week, that will be great. -- Best Wishes, Ashutosh Bapat

Re: Report planning memory in EXPLAIN ANALYZE

2023-08-24 Thread Ashutosh Bapat
need a way to create a new memory context with the same properties as the CurrentMemoryContext, a functionality missing right now. Once we agree upon the approach, the patch will need to be merged into 0002 and in turn 0001. -- Best Wishes, Ashutosh Bapat explain_planning_memory.out Description: Bina

Re: Oversight in reparameterize_path_by_child leading to executor crash

2023-08-23 Thread Ashutosh Bapat
> For another query it would error out during planning. > > regression=# explain (verbose, costs off) > select count(*) from prt1 t1 left join lateral > (select t1.b as t1b, t2.* from prt2 t2) s > on t1.a = s.b where s.t1b = s.b; > ERROR: variable not found in subplan target list > > To fix it we may need to modify RelOptInfos for Path, BitmapHeapPath, > ForeignPath and CustomPath, and modify IndexOptInfos for IndexPath. It > seems that that is not easily done without postponing reparameterization > of paths until createplan.c. Maybe I am missing something here but why aren't we copying these restrictinfos in the path somewhere? I have a similar question for tablesample clause as well. -- Best Wishes, Ashutosh Bapat

Re: Oversight in reparameterize_path_by_child leading to executor crash

2023-08-22 Thread Ashutosh Bapat
e the macros and actually look better to me. Let me know what you think of this. FWIW I ran make check and all the tests pass. I agree that we can not consider this for backbranches but we are anyway thinking of disabling reparameterization for tablesample scans anyway. -- Be

Re: persist logical slots to disk during shutdown checkpoint

2023-08-22 Thread Ashutosh Bapat
ter to do that > optimization (persist confirm_flush_lsn at a regular interval) as a > separate patch as we need to test and prove its value separately. Fine with me. -- Best Wishes, Ashutosh Bapat

Re: persist logical slots to disk during shutdown checkpoint

2023-08-22 Thread Ashutosh Bapat
now, we might just forget to do that small incremental change in future. My preference is 1. Do both (1) and (2) together 2. Do (2) first and then (1) as a separate commit. 3. Just implement (2) if we don't have time at all for first two options. -- Best Wishes, Ashutosh Bapat

Re: persist logical slots to disk during shutdown checkpoint

2023-08-21 Thread Ashutosh Bapat
dirty and persist if the last confirm_flush_lsn when slot was persisted was too far from the current confirm_flush_lsn of the slot. 2. at shutdown checkpoint, persist all the slots which have these two confirm_flush_lsns different. -- Best Wishes, Ashutosh Bapat

Re: Oversight in reparameterize_path_by_child leading to executor crash

2023-08-21 Thread Ashutosh Bapat
combing reparameterize_path_by_child() and path_is_reparameterizable_by_child(). The name of such a function can be chosen appropriately. The mode will be controlled by a fourth argument to the function. When assessing a path no translations happen and no extra memory is allocated. -- Best Wishes, Ashutosh Bapat

Re: logical decoding and replication of sequences, take 2

2023-08-17 Thread Ashutosh Bapat
pact of not setting replorigin_session_origin_timestamp. What fake transaction experiment are you talking about? -- Best Wishes, Ashutosh Bapat

Re: Memory consumed by child SpecialJoinInfo in partitionwise join planning

2023-08-15 Thread Ashutosh Bapat
On Fri, Aug 4, 2023 at 2:11 PM Ashutosh Bapat wrote: > > Attached patchset fixing those. > 0001 - patch to report planning memory, with to explain.out regression output > fix. We may consider committing this as well. > 0002 - with your comment addressed above. 0003 - A

Re: AssertLog instead of Assert in some places

2023-08-14 Thread Ashutosh Bapat
two separate incantations for non-recoverable and recoverable Asserts respectively. I like Peter's idea of having a new elevel, however it still requires adding conditional USE_ASSERT, an if testing the condition and then writing an error message. AssertLog() in the patch uses just a few more letters. It won't help to expand the scope of the problem since that will reduce the chances of getting anything done. -- Best Wishes, Ashutosh Bapat

Re: AssertLog instead of Assert in some places

2023-08-14 Thread Ashutosh Bapat
und > process that just restarts. Fair point. Our regression doesn't check server error logs for unwanted errors. How about restricting it to only client backends? I don't know how to identify those from others but there must be a way. -- Best Wishes, Ashutosh Bapat

Re: Report planning memory in EXPLAIN ANALYZE

2023-08-14 Thread Ashutosh Bapat
acking itself might make the planning in-efficient and we might still miss the spikes in memory allocations, if they are very short lived. If the planner runs for more than a few minutes, maybe we could add some tracking. -- Best Wishes, Ashutosh Bapat

Re: Report planning memory in EXPLAIN ANALYZE

2023-08-14 Thread Ashutosh Bapat
On Mon, Aug 14, 2023 at 5:23 AM David Rowley wrote: > > On Thu, 10 Aug 2023 at 20:33, Ashutosh Bapat > wrote: > > My point is what's relevant here is how much net memory planner asked > > for. > > But that's not what your patch is reporting. All you're reporting is

Re: Report planning memory in EXPLAIN ANALYZE

2023-08-11 Thread Ashutosh Bapat
velopers can watch it easily in different > ways, if needed. So, I vote for the 'memory used' metric only. > > The patch looks good, passes the tests. Thanks Andrey. David, are you against reporting "memory used"? If you are not against reporting used memory and still think that

<    1   2   3   4   5   6   7   8   9   10   >