Re: Table refer leak in logical replication

2021-04-20 Thread Amit Langote
h a table would need at least 4 columns. Agree about adding tests along these lines. Will post in a bit. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Partitioned table permission question

2021-04-20 Thread Amit Langote
d directly without an additional grant. In a similar way, the parent table's row security policies (see Section 5.8) are applied to rows coming from child tables during an inherited query. A child table's policies, if any, are applied only when it is the table explicitly named in the query; and in that case, any policies attached to its parent(s) are ignored." -- Amit Langote EDB: http://www.enterprisedb.com [1] https://www.postgresql.org/docs/current/ddl-inherit.html

Re: Table refer leak in logical replication

2021-04-20 Thread Amit Langote
On Tue, Apr 20, 2021 at 4:22 PM Michael Paquier wrote: > On Tue, Apr 20, 2021 at 02:48:35PM +0900, Amit Langote wrote: > > Manipulating the contents of es_opened_result_relations directly in > > worker.c is admittedly a "hack", which I am reluctant to have other > &g

Re: Table refer leak in logical replication

2021-04-19 Thread Amit Langote
Thanks for taking a look. On Tue, Apr 20, 2021 at 2:09 PM Michael Paquier wrote: > On Mon, Apr 19, 2021 at 09:44:05PM +0900, Amit Langote wrote: > > Okay, how about the attached then? > > create_estate_for_relation() returns an extra resultRelInfo that's

Re: Forget close an open relation in ReorderBufferProcessTXN()

2021-04-19 Thread Amit Langote
d/CA%2BHiwqEeU19iQgjN6HF1HTPU0L5%2BJxyS5CmxaOVGNXBAfUY06Q%40mail.gmail.com I had proposed to move the map creation from maybe_send_schema() to get_rel_sync_entry(), mainly because the latter is where I realized it belongs, though a bit too late. Attached is the part of the patch for this particular is

Re: Commit 86dc90056 - Rework planning and execution of UPDATE and DELETE

2021-04-19 Thread Amit Langote
pg_attribute.attstorage. That's because the new tuple would be computed from the subplan's targetlist and the TupleDesc for that targetlist is computed with no regard to where the computed tuple will go; IOW ignoring the target table's actual TupleDesc. After 86dc90056, the new tuple is c

Re: Table refer leak in logical replication

2021-04-19 Thread Amit Langote
On Mon, Apr 19, 2021 at 6:32 PM Michael Paquier wrote: > On Mon, Apr 19, 2021 at 02:33:10PM +0530, Amit Kapila wrote: > > On Mon, Apr 19, 2021 at 12:32 PM Amit Langote > > wrote: > >> FWIW, I agree with fixing this bug of 1375422c in as least scary > >> mann

Re: Table refer leak in logical replication

2021-04-19 Thread Amit Langote
at 02:33:10PM +0530, Amit Kapila wrote: > > > > > On Mon, Apr 19, 2021 at 12:32 PM Amit Langote > > > > > wrote: > > > > >> FWIW, I agree with fixing this bug of 1375422c in as least scary > > > > >> manner as possible. Hou-

Re: Table refer leak in logical replication

2021-04-19 Thread Amit Langote
On Mon, Apr 19, 2021 at 6:03 PM Amit Kapila wrote: > On Mon, Apr 19, 2021 at 12:32 PM Amit Langote wrote: > > On Sat, Apr 17, 2021 at 10:32 PM Amit Kapila > > wrote: > > > On Fri, Apr 16, 2021 at 11:55 AM Michael Paquier > > > wrote: > > > > A

Re: Doubt with [ RANGE partition with TEXT datatype ]

2021-04-19 Thread Amit Langote
o follow numerical rules, use a numeric data type. > If this behavior is expected, Kindly let me know, how to represent the range > from '5' to '10' with text datatype column? Don't know why you want to use the text type for the column and these particular values for the partitions bounds, but one workaround would be to use '05' instead of '5'. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Table refer leak in logical replication

2021-04-19 Thread Amit Langote
On Fri, Apr 16, 2021 at 3:24 PM Michael Paquier wrote: > On Tue, Apr 06, 2021 at 02:25:05PM +0900, Amit Langote wrote: > > While updating the patch to do so, it occurred to me that maybe we > > could move the ExecInitResultRelation() call into > > create_estate_for_relation

Re: Table refer leak in logical replication

2021-04-19 Thread Amit Langote
On Sat, Apr 17, 2021 at 10:32 PM Amit Kapila wrote: > On Fri, Apr 16, 2021 at 11:55 AM Michael Paquier wrote: > > On Tue, Apr 06, 2021 at 02:25:05PM +0900, Amit Langote wrote: > > > > Attached is v5 that I am finishing with. Much more could be done but > > I don

Re: ModifyTable overheads in generic plans

2021-04-13 Thread Amit Langote
On Wed, Apr 7, 2021 at 5:18 PM Amit Langote wrote: > On Wed, Apr 7, 2021 at 8:24 AM Tom Lane wrote: > > I also could not get excited about postponing initialization of RETURNING > > or WITH CHECK OPTIONS expressions. I grant that that can be helpful > > when those features

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-04-12 Thread Amit Langote
arlier query in the transaction, which does include the detach-pending partition, thus allowing the insert trying to insert a row referencing that partition to succeed. There is a provision in RelationGetPartitionDesc() to rebuild if any detach-pending partitions in the existing copy of PartitionDesc are not to be seen by the current query, but a bug mentioned in my earlier reply prevents that from kicking in. -- Amit Langote EDB: http://www.enterprisedb.com

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-04-12 Thread Amit Langote
On Mon, Apr 12, 2021 at 4:42 PM Amit Langote wrote: > On Mon, Apr 12, 2021 at 6:20 AM Alvaro Herrera > wrote: > > On 2021-Mar-31, Tom Lane wrote: > > > > > diff -U3 > > > /home/buildfarm/trilobite/buildroot/HEAD/pgsql.build/src/test/isolation/expected

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-04-12 Thread Amit Langote
key of d4_fk being violated. It seems problematic to me that the logic of setting includes_detached is oblivious of the special check in find_inheritance_children() to decide whether "force"-include a detach-pending child based on cross-checking its pg_inherit tuple's xmin against the active snapshot. Maybe fixing that would help, although I haven't tried that myself yet. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Table refer leak in logical replication

2021-04-11 Thread Amit Langote
On Sat, Apr 10, 2021 at 10:39 AM Justin Pryzby wrote: > I added this as an Open Item. Thanks, Justin. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Wired if-statement in gen_partprune_steps_internal

2021-04-08 Thread Amit Langote
On Thu, Apr 8, 2021 at 7:41 PM David Rowley wrote: > On Thu, 8 Apr 2021 at 21:04, Amit Langote wrote: > > Maybe, we should also updated the description of node struct as > > follows to consider that last point: >> > > * PartitionPruneStepOp - Information to prune us

Re: Wired if-statement in gen_partprune_steps_internal

2021-04-08 Thread Amit Langote
On Thu, Apr 8, 2021 at 5:34 PM David Rowley wrote: > On Thu, 8 Apr 2021 at 00:49, Amit Langote wrote: > > > > Thanks David. Actually, I was busy updating the patch to revert to > > gen_partprune_steps_internal() returning a list and was almost done > > with it when I

Re: ModifyTable overheads in generic plans

2021-04-07 Thread Amit Langote
On Thu, Apr 8, 2021 at 1:54 PM David Rowley wrote: > On Thu, 8 Apr 2021 at 15:32, Amit Langote wrote: > > There's 10-20% improvement in this case too for various partition > > counts, which really has more to do with 86dc90056 than the work done > > here. > >

Re: ModifyTable overheads in generic plans

2021-04-07 Thread Amit Langote
0 9020 v14dev HEAD 64 14925 14648 13361 128 14379 14333 13138 256 14478 14246 13316 102412744 12621 11579 There's 10-20% improvement in this case too for various partition counts, which really has more to do with 86dc90056 than the work done here. -- Amit Langote EDB: http://www.enterprisedb.com

Re: ModifyTable overheads in generic plans

2021-04-07 Thread Amit Langote
On Thu, Apr 8, 2021 at 1:34 AM Tom Lane wrote: > Amit Langote writes: > > Also, I think we should update the commentary around ri_projectNew a > > bit to make it clear that noplace beside ExecGet{Insert|Update}Tuple > > should be touching it and the associated slots. &

Re: Wired if-statement in gen_partprune_steps_internal

2021-04-07 Thread Amit Langote
On Wed, Apr 7, 2021 at 6:53 PM David Rowley wrote: > On Wed, 7 Apr 2021 at 21:04, Amit Langote wrote: > > > > On Wed, Apr 7, 2021 at 4:43 PM David Rowley wrote: > > > However, it does change the meaning of what PARTCLAUSE_MATCH_STEPS > > > does.

Re: Wired if-statement in gen_partprune_steps_internal

2021-04-07 Thread Amit Langote
o go ahead and commit the v3. I've attached the patch I had ended up with and was about to post as v3, just in case you wanted to glance. -- Amit Langote EDB: http://www.enterprisedb.com v3_amit.diff Description: Binary data

Re: Wired if-statement in gen_partprune_steps_internal

2021-04-07 Thread Amit Langote
On Wed, Apr 7, 2021 at 4:43 PM David Rowley wrote: > On Thu, 4 Mar 2021 at 19:03, Amit Langote wrote: > > On Tue, Oct 20, 2020 at 9:46 PM Amit Langote > > wrote: > > > I had updated the patch last week to address Horiguchi-san's comments > > > but d

Re: ModifyTable overheads in generic plans

2021-04-07 Thread Amit Langote
On Wed, Apr 7, 2021 at 8:24 AM Tom Lane wrote: > Amit Langote writes: > > On Mon, Apr 5, 2021 at 1:43 AM Tom Lane wrote: > >> OK. Do you want to pull out the bits of the patch that we can still > >> do without postponing BeginDirectModify? > > > I

Re: Allow batched insert during cross-partition updates

2021-04-06 Thread Amit Langote
On Tue, Apr 6, 2021 at 10:52 PM Bharath Rupireddy wrote: > > On Tue, Apr 6, 2021 at 6:37 PM Amit Langote wrote: > > > 3) will the cmin in the output always be the same? > > > +SELECT cmin, * FROM batch_cp_upd_test3; > > > > TBH, I am not so sure. Maybe

Re: Allow batched insert during cross-partition updates

2021-04-06 Thread Amit Langote
On Tue, Apr 6, 2021 at 6:49 PM Bharath Rupireddy wrote: > On Tue, Apr 6, 2021 at 3:08 PM Amit Langote wrote: > > Updated patch attached. I had to adjust the test case a little bit to > > account for the changes of 86dc90056d, something I failed to notice > > yesterday.

Re: Allow batched insert during cross-partition updates

2021-04-06 Thread Amit Langote
otally unrelated code. Updated patch attached. I had to adjust the test case a little bit to account for the changes of 86dc90056d, something I failed to notice yesterday. Also, I expanded the test case added in postgres_fdw.sql a bit to show the batching in action more explicitly. -- Amit Langote EDB: http://www.enterprisedb.com v4-0001-Allow-batching-of-inserts-during-cross-partition-.patch Description: Binary data

Re: A reloption for partitioned tables - parallel_workers

2021-04-06 Thread Amit Langote
On Fri, Apr 2, 2021 at 11:36 PM Laurenz Albe wrote: > On Wed, 2021-03-24 at 14:14 +1300, David Rowley wrote: > > On Fri, 19 Mar 2021 at 02:07, Amit Langote wrote: > > > Attached a new version rebased over c8f78b616, with the grouping > > > relation partitioning enhan

Re: postgres_fdw: IMPORT FOREIGN SCHEMA ... LIMIT TO (partition)

2021-04-06 Thread Amit Langote
archy, this approach should > +allow access to all the data without creating extra objects. > > Now "this approach" in the above is not clear? What about replacing it with > something like "importing only partitioned tables"? +1, that wording is better. -- Amit Langote EDB: http://www.enterprisedb.com

Re: postgres_fdw: IMPORT FOREIGN SCHEMA ... LIMIT TO (partition)

2021-04-06 Thread Amit Langote
s limitation. IMPORT FORAIGN > SCHEMA really is great when it works, but limitations like these are > crippling for some more specific use cases (e.g. allowing > long-duration read-only access to one partition in the partition tree > while also allowing the partition layout of the parents to be > modified). FWIW, I agree that it would be nice to have this. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Table refer leak in logical replication

2021-04-05 Thread Amit Langote
On Tue, Apr 6, 2021 at 1:57 PM Masahiko Sawada wrote: > On Tue, Apr 6, 2021 at 1:15 PM Amit Langote wrote: > > On Tue, Apr 6, 2021 at 1:01 PM houzj.f...@fujitsu.com > > > The commit introduced a > > > > new function ExecInitResultRelation() that sets both > >

Re: Table refer leak in logical replication

2021-04-05 Thread Amit Langote
t; We should call ExecCloseRangeTableRelations at the end of apply_handle_xxx to > close the rel in "Estate->es_relations". Right, ExecCloseRangeTableRelations() was missing. I think it may be better to create a sibling function to create_estate_for_relation(),

Re: ModifyTable overheads in generic plans

2021-04-05 Thread Amit Langote
On Mon, Apr 5, 2021 at 1:43 AM Tom Lane wrote: > Amit Langote writes: > > On Sun, Apr 4, 2021 at 10:20 AM Tom Lane wrote: > >> In some desultory performance testing here, it seemed like a > >> significant part of the cost is ExecOpenIndices, and I don't see &g

Re: Allow batched insert during cross-partition updates

2021-04-04 Thread Amit Langote
On Tue, Mar 16, 2021 at 6:13 PM wrote: > Status updated to RfC in the commitfest app. Patch fails to apply per cfbot, so rebased. -- Amit Langote EDB: http://www.enterprisedb.com v3-0001-Allow-batching-of-inserts-during-cross-partition-.patch Description: Binary data

Re: ModifyTable overheads in generic plans

2021-04-04 Thread Amit Langote
On Sun, Apr 4, 2021 at 10:20 AM Tom Lane wrote: > Amit Langote writes: > > On Thu, Apr 1, 2021 at 3:12 AM Tom Lane wrote: > >> Amit Langote writes: > > [ v14-0002-Initialize-result-relation-information-lazily.patch ] > >> Needs YA rebase over 86dc90056.

Re: simplifying foreign key/RI checks

2021-04-04 Thread Amit Langote
in one place, the two if statements can be > combined into one. Thanks for taking a look. I agree with most of your suggestions and have incorporated them in the v8 just posted. -- Amit Langote EDB: http://www.enterprisedb.com

Re: simplifying foreign key/RI checks

2021-04-04 Thread Amit Langote
Hi Alvaro, On Sat, Apr 3, 2021 at 12:01 AM Alvaro Herrera wrote: > On 2021-Apr-02, Amit Langote wrote: > > > On Sat, Mar 20, 2021 at 10:21 PM Amit Langote > > wrote: > > > Updated patches attached. Sorry about the delay. > > > > Rebased over the re

Re: a misbehavior of partition row movement (?)

2021-04-02 Thread Amit Langote
On Thu, Apr 1, 2021 at 10:56 AM Masahiko Sawada wrote: > On Tue, Mar 23, 2021 at 6:27 PM Amit Langote wrote: > > Actually, I found a big hole in my assumptions around deferrable > > foreign constraints, invalidating the approach I took in 0002 to use a > > query-lifetime

Re: simplifying foreign key/RI checks

2021-04-02 Thread Amit Langote
On Sat, Mar 20, 2021 at 10:21 PM Amit Langote wrote: > Updated patches attached. Sorry about the delay. Rebased over the recent DETACH PARTITION CONCURRENTLY work. Apparently, ri_ReferencedKeyExists() was using the wrong snapshot. -- Amit Langote EDB: http://www.enterprisedb.com v8-0

Re: making update/delete of inheritance trees scale better

2021-04-02 Thread Amit Langote
On Wed, Mar 31, 2021 at 9:54 PM Amit Langote wrote: > I reran some of the performance tests I did earlier (I've attached the > modified test running script for reference): For archives' sake, noticing a mistake in my benchmarking script, I repeated the tests. Apparently, all p

Re: ModifyTable overheads in generic plans

2021-04-01 Thread Amit Langote
On Thu, Apr 1, 2021 at 10:12 PM Amit Langote wrote: > > On Thu, Apr 1, 2021 at 3:12 AM Tom Lane wrote: > > Amit Langote writes: > > > [ v14-0002-Initialize-result-relation-information-lazily.patch ] > > Needs YA rebase over 86dc90056. > > Done. I will post t

Re: ModifyTable overheads in generic plans

2021-04-01 Thread Amit Langote
On Thu, Apr 1, 2021 at 3:12 AM Tom Lane wrote: > Amit Langote writes: > > [ v14-0002-Initialize-result-relation-information-lazily.patch ] > Needs YA rebase over 86dc90056. Done. I will post the updated results for -Mprepared benchmarks I did in the other thread shortly. -- Amit

Re: making update/delete of inheritance trees scale better

2021-03-31 Thread Amit Langote
On Thu, Apr 1, 2021 at 12:58 AM Tom Lane wrote: > Amit Langote writes: > > On Tue, Mar 30, 2021 at 1:51 PM Tom Lane wrote: > >> Here's a v13 patchset that I feel pretty good about. > > > Thanks. After staring at this for a day now, I do too. > > Thanks for

Re: making update/delete of inheritance trees scale better

2021-03-31 Thread Amit Langote
an be attributed to the fact that with a generic plan, there are a bunch of steps that must be done across all partitions, such as AcauireExecutorLocks(), ExecCheckRTPerms(), per-result-rel initializations in ExecInitModifyTable(), etc., even with the patched. As mentioned upthread, [1] can help with

Re: making update/delete of inheritance trees scale better

2021-03-30 Thread Amit Langote
st pushdown for the traditional inheritance cases as well. I agree that rethinking the whole apply_scanjoin_target_to_paths() approach might be a better use of our time. It has a looping-over-the-whole-partition-array bottleneck for simple lookup queries that I have long wanted to propose doing something about. -- Amit Langote EDB: http://www.enterprisedb.com

Re: making update/delete of inheritance trees scale better

2021-03-30 Thread Amit Langote
ree that that's not too bad for a worse-case test case, nor something we couldn't optimize. I have yet to look closely at where the problem lies though. -- Amit Langote EDB: http://www.enterprisedb.com

Re: making update/delete of inheritance trees scale better

2021-03-29 Thread Amit Langote
On Mon, Mar 29, 2021 at 11:41 PM Tom Lane wrote: > Amit Langote writes: > > On Sun, Mar 28, 2021 at 1:30 AM Tom Lane wrote: > >> I wanted to give a data dump of where I am. I've reviewed and > >> nontrivially modified 0001 and the executor parts of 0002, an

Re: making update/delete of inheritance trees scale better

2021-03-29 Thread Amit Langote
On Sun, Mar 28, 2021 at 1:30 AM Tom Lane wrote: > Amit Langote writes: > > Attached updated version of the patch. I have forgotten to mention in > > my recent posts on this thread one thing about 0001 that I had > > mentioned upthread back in June. That it curr

Re: making update/delete of inheritance trees scale better

2021-03-29 Thread Amit Langote
e where a remote partition chosen to insert a moved row into is also an UPDATE target partition that will be updated later. === I think we will need to take out the "...table will be updated later" part at the end of the sentence. -- Amit Langote EDB: http://www.enterprisedb.com

Re: making update/delete of inheritance trees scale better

2021-03-25 Thread Amit Langote
compute, which is what your approach sounds like. However, I had abandoned it due to the concern that it possibly hurt the prepared statements because we would build the targetlist on every execution, whereas only once if the planner does it. I'm just about done addressing Robert's comments, so will post an update shortly. -- Amit Langote EDB: http://www.enterprisedb.com

Re: making update/delete of inheritance trees scale better

2021-03-24 Thread Amit Langote
ucting the full updated tuple at the outset in > ModifyTable, and then proceeding pretty much as updates > have done in the past. But there's more we can do later. Agreed. I'm addressing Robert's comments and will post an updated patch by tomorrow. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Wired if-statement in gen_partprune_steps_internal

2021-03-23 Thread Amit Langote
Hi Ryan, On Tue, Mar 23, 2021 at 2:24 AM Ryan Lambert wrote: > Should the status of this patch be updated to ready for comitter to get in > line for Pg 14 deadline? Yes, I've done that. Thanks for the reminder. -- Amit Langote EDB: http://www.enterprisedb.com

Re: a misbehavior of partition row movement (?)

2021-03-23 Thread Amit Langote
ded to change the if condition as follows: - if (pset.sversion >= 110000) + if (pset.sversion >= 11 && pset.sversion < 14) Note that without this change, this code ends up revealing partitions' foreign key triggers, because we will now be marking them dependent on their parent trigger, which wasn't the case before this patch. > I'll look at 0002 patch. Actually, I found a big hole in my assumptions around deferrable foreign constraints, invalidating the approach I took in 0002 to use a query-lifetime tuplestore to record root parent tuples. I'm trying to find a way to make the tuplestore transaction-lifetime so that the patch still works. In the meantime, I'm attaching an updated set with 0001 changed per your comments. -- Amit Langote EDB: http://www.enterprisedb.com v6-0002-Enforce-foreign-key-correctly-during-cross-partit.patch Description: Binary data v6-0001-Create-foreign-key-triggers-in-partitioned-tables.patch Description: Binary data

Re: [CLOBBER_CACHE]Server crashed with segfault 11 while executing clusterdb

2021-03-22 Thread Amit Langote
d by operations that would invalidate the SMgrRelation that those smgr* operations would be called with. For example, the smgrnblocks() in gistBuildCallback() may get done too late than a corresponding RelationOpenSmgr() on the index relation. -- Amit Langote EDB: http://www.enterprisedb.com

Re: problem with RETURNING and update row movement

2021-03-22 Thread Amit Langote
On Sat, Mar 6, 2021 at 12:52 AM David Steele wrote: > On 10/30/20 6:00 AM, Amit Langote wrote: > > The question > > I guess is whether that thread must conclude before the fix here can > > be committed. > > Indeed. But it seems like there is a candidate patch on [1] th

Re: simplifying foreign key/RI checks

2021-03-20 Thread Amit Langote
On Mon, Mar 8, 2021 at 11:41 PM Amit Langote wrote: > On Thu, Mar 4, 2021 at 5:15 AM Tom Lane wrote: > > I guess I'm disturbed by the idea > > that we'd totally replace the implementation technology for only one > > variant of foreign key checks. That means that

Re: A reloption for partitioned tables - parallel_workers

2021-03-18 Thread Amit Langote
On Fri, Mar 5, 2021 at 11:06 PM Laurenz Albe wrote: > On Fri, 2021-03-05 at 22:55 +0900, Amit Langote wrote: > > On Fri, Mar 5, 2021 at 10:47 PM Laurenz Albe > > wrote: > > > On Wed, 2021-03-03 at 17:58 +0900, Amit Langote wrote: > > > > For example, with t

Re: crash during cascaded foreign key update

2021-03-18 Thread Amit Langote
On Wed, Mar 17, 2021 at 11:01 AM Amit Langote wrote: > On Tue, Mar 16, 2021 at 11:17 PM Tom Lane wrote: > > Amit Langote writes: > > > With HEAD (I think v12 and greater), I see $subject when trying out > > > the following scenario: Actually, the crash is reproducibl

Re: crash during cascaded foreign key update

2021-03-16 Thread Amit Langote
On Tue, Mar 16, 2021 at 11:17 PM Tom Lane wrote: > Amit Langote writes: > > With HEAD (I think v12 and greater), I see $subject when trying out > > the following scenario: > > I wonder if this is related to > > https://www.postgresql.org/message-id/flat/89429.1584443

crash during cascaded foreign key update

2021-03-16 Thread Amit Langote
) at postmaster.c:1736 #32 0x00891a4b in PostmasterMain (argc=3, argv=0x1b3cc20) at postmaster.c:1408 #33 0x0079360f in main (argc=3, argv=0x1b3cc20) at main.c:209 I haven't checked the failure in more detail yet other than that the failed Assert was added in 5db6df0c0117. -- Amit Langote EDB: http://www.enterprisedb.com

Re: simplifying foreign key/RI checks

2021-03-16 Thread Amit Langote
On Mon, Mar 8, 2021 at 11:41 PM Amit Langote wrote: > On Thu, Mar 4, 2021 at 5:15 AM Tom Lane wrote: > > Lastly, ri_PerformCheck is pretty careful about not only which > > snapshot it uses, but which *pair* of snapshots it uses, because > > sometimes it needs to worry abo

Re: Allow batched insert during cross-partition updates

2021-03-16 Thread Amit Langote
Hi Georgios, On Tue, Mar 16, 2021 at 5:12 PM wrote: > On Tuesday, March 16, 2021 6:13 AM, Amit Langote > wrote: > > On Fri, Mar 12, 2021 at 7:59 PM gkokola...@pm.me wrote: > > > On Friday, March 12, 2021 3:45 AM, Amit Langote amitlangot...@gmail.com > > > wrote

Re: Allow batched insert during cross-partition updates

2021-03-15 Thread Amit Langote
Hi Georgios, On Fri, Mar 12, 2021 at 7:59 PM wrote: > On Friday, March 12, 2021 3:45 AM, Amit Langote > wrote: > > On Thu, Mar 11, 2021 at 8:36 PM gkokola...@pm.me wrote: > > > On Thursday, March 11, 2021 9:42 AM, Amit Langote amitlangot...@gmail.com > > > wro

Re: Parallel INSERT (INTO ... SELECT ...)

2021-03-11 Thread Amit Langote
the patch for adding parallel_workers option to partitioned tables does? -- Amit Langote EDB: http://www.enterprisedb.com

Re: Allow batched insert during cross-partition updates

2021-03-11 Thread Amit Langote
On Thu, Mar 11, 2021 at 8:36 PM wrote: > On Thursday, March 11, 2021 9:42 AM, Amit Langote > wrote: > > On Wed, Mar 10, 2021 at 9:30 PM Georgios gkokola...@protonmail.com wrote: > > > > > I continued looking a bit at the patch, yet I am either failing to see > &g

Re: Allow batched insert during cross-partition updates

2021-03-11 Thread Amit Langote
is patch is for teaching those INSERTs to use batched mode if allowed, which is currently prohibited. So with this patch, if an UPDATE moves 10 rows from a local partition to a remote partition, then they will be inserted with a single INSERT command containing all 10 rows, instead of 10 separate IN

Re: Allow batched insert during cross-partition updates

2021-03-11 Thread Amit Langote
or functions would be better, I will change the patch that way. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Parallel INSERT (INTO ... SELECT ...)

2021-03-10 Thread Amit Langote
On Wed, Mar 10, 2021 at 6:18 PM Amit Kapila wrote: > On Mon, Mar 8, 2021 at 7:19 PM Amit Langote wrote: > > I just read through v25 and didn't find anything to complain about. > > Thanks a lot, pushed now! Amit L., your inputs are valuable for this work. Glad I could

Re: Huge memory consumption on partitioned table with FKs

2021-03-10 Thread Amit Langote
On Thu, Mar 11, 2021 at 4:25 AM Tom Lane wrote: > Amit Langote writes: > > On Wed, Mar 10, 2021 at 8:37 AM Tom Lane wrote: > >> Hmm. So, the key point is that the values coming from the partitioned > >> child table are injected into the test query as parameters, no

Re: Huge memory consumption on partitioned table with FKs

2021-03-10 Thread Amit Langote
On Wed, Mar 10, 2021 at 8:37 AM Tom Lane wrote: > Amit Langote writes: > > On Fri, Mar 5, 2021 at 6:00 AM Tom Lane wrote: > >> This claim seems false on its face: > >>> All child constraints of a given foreign key constraint can use the > >>> same RI

Re: TRUNCATE on foreign table

2021-03-08 Thread Amit Langote
) does. That leads to relids and relids_extra having different lengths, which trips the Assert in ExecuteTruncateGuts(). -- Amit Langote EDB: http://www.enterprisedb.com

Re: simplifying foreign key/RI checks

2021-03-08 Thread Amit Langote
s is a cute idea, and the speedup is pretty impressive, > but I don't think it's anywhere near committable. I also wonder > whether we really want ri_triggers.c having its own copy of > low-level stuff like the tuple-locking code you copied. Seems > like a likely maintenance hazard, so maybe some more refactoring > is needed. Okay, I will see if there's a way to avoid copying too much code. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Parallel INSERT (INTO ... SELECT ...)

2021-03-08 Thread Amit Langote
> 0001* patch or if you want to review it further? I just read through v25 and didn't find anything to complain about. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Huge memory consumption on partitioned table with FKs

2021-03-08 Thread Amit Langote
On Mon, Mar 8, 2021 at 9:53 PM Andy Fan wrote: > On Mon, Mar 8, 2021 at 8:42 PM Amit Langote wrote: >> On Mon, Mar 8, 2021 at 8:39 PM Andy Fan wrote: >> > On Mon, Mar 8, 2021 at 3:43 PM Andy Fan wrote: >> >> My point below is a bit off-topic, but I want to

Re: Huge memory consumption on partitioned table with FKs

2021-03-08 Thread Amit Langote
EATE TABLE p (a int) partitioned by list(a) RESTRICTED". We can add these > limitation to restricted partitioned relation only. I think you'd agree that the topics you want to discuss deserve a separate discussion thread. You may refer to this discussion in that new thread if you thi

Re: Parallel INSERT (INTO ... SELECT ...)

2021-03-07 Thread Amit Langote
ireExecutorLocksOnPartitions() will do in the generic plan case. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Wired if-statement in gen_partprune_steps_internal

2021-03-07 Thread Amit Langote
On Fri, Mar 5, 2021 at 7:50 AM Ryan Lambert wrote: > On Wed, Mar 3, 2021 at 11:03 PM Amit Langote wrote: >> >> Sorry, this seems to have totally slipped my mind. >> >> Attached is the patch I had promised. >> >> Also, I have updated the title of the CF ent

Re: Huge memory consumption on partitioned table with FKs

2021-03-07 Thread Amit Langote
On Fri, Mar 5, 2021 at 6:00 AM Tom Lane wrote: > Amit Langote writes: > > Updated patch attached. > > This claim seems false on its face: > > > All child constraints of a given foreign key constraint can use the > > same RI query and the resulting plan, that is,

Re: A reloption for partitioned tables - parallel_workers

2021-03-05 Thread Amit Langote
On Fri, Mar 5, 2021 at 10:47 PM Laurenz Albe wrote: > On Wed, 2021-03-03 at 17:58 +0900, Amit Langote wrote: > > For example, with the attached PoC patch: > > I have incorporated your POC patch and added a regression test. > > I didn't test it thoroughly though. Thanks.

Re: Extend more usecase for planning time partition pruning and init partition pruning.

2021-03-04 Thread Amit Langote
working with ScalarArrayOpExpr and BoolExpr to begin with? * Or maybe have you considered generalizing what build_implied_pruning_quals() does so that other places like indxpath.c can use the facility? -- Amit Langote EDB: http://www.enterprisedb.com

Re: Wired if-statement in gen_partprune_steps_internal

2021-03-03 Thread Amit Langote
On Tue, Oct 20, 2020 at 9:46 PM Amit Langote wrote: > On Tue, Oct 20, 2020 at 4:05 PM Andy Fan wrote: > > On Wed, Oct 14, 2020 at 11:26 AM Andy Fan wrote: > >> On Mon, Oct 12, 2020 at 4:37 PM Amit Langote > >> wrote: > >>> I think we should remov

Re: Huge memory consumption on partitioned table with FKs

2021-03-03 Thread Amit Langote
Hi David, On Wed, Mar 3, 2021 at 10:21 PM David Steele wrote: > On 12/7/20 10:59 PM, Amit Langote wrote: > > On Tue, Dec 8, 2020 at 12:04 PM Kyotaro Horiguchi > > wrote: > >> At Tue, 8 Dec 2020 01:16:00 +0900, Amit Langote > >> wrote in > >>>

Re: A reloption for partitioned tables - parallel_workers

2021-03-03 Thread Amit Langote
On Tue, Mar 2, 2021 at 5:47 PM Laurenz Albe wrote: > On Tue, 2021-03-02 at 11:23 +0900, Amit Langote wrote: > > +ALTER TABLE pagg_tab_ml SET (parallel_workers = 0); > > +EXPLAIN (COSTS OFF) > > +SELECT a FROM pagg_tab_ml WHERE b = 42; > > +

Re: Increase value of OUTER_VAR

2021-03-03 Thread Amit Langote
y mean we'd not > need to revisit this for quite a while. +1 Also, I got reminded of this discussion from not so long ago: https://www.postgresql.org/message-id/flat/16302-e45634e2c0e34e97%40postgresql.org -- Amit Langote EDB: http://www.enterprisedb.com

Re: A reloption for partitioned tables - parallel_workers

2021-03-01 Thread Amit Langote
rkers=0 on the parent table shouldn't really disable a regular (non-parallel-aware) Append running under Gather even if it does Parallel Append (parallel-aware)? So in this test case, there should have been a Gather atop Append, with individual partitions scanned using Parallel Seq Scan where app

Re: simplifying foreign key/RI checks

2021-03-01 Thread Amit Langote
about the attached? >> > > Sorry for the delay. > I see that the changes were made as described. > Passes make check and make check-world yet again. > I'm marking this Ready For Committer unless someone objects. Thank you Corey for the review. -- Amit Langote EDB: http://www.enterprisedb.com

Re: A reloption for partitioned tables - parallel_workers

2021-03-01 Thread Amit Langote
for partitioned tables, and I can see that it is actually wrong to access a partitioned table's parallel_workers through this macro as-is. Although I hadn't tested that, so thanks for pointing that out. > Like: > #define PartitionedTableGetParallelWorkers(relation, defaultpw) \ > xxx > (PartitionedTableRdOptions *) (relation)->rd_options)->parallel_workers : > (defaultpw)) I'm thinking it would be better to just modify the existing macro to check relkind to decide which struct pointer type to cast the value in rd_options to. I will post an updated patch later. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Parallel INSERT (INTO ... SELECT ...)

2021-02-28 Thread Amit Langote
On Mon, Mar 1, 2021 at 12:38 PM Greg Nancarrow wrote: > On Fri, Feb 26, 2021 at 5:50 PM Amit Langote wrote: > > > > On Fri, Feb 26, 2021 at 3:35 PM Greg Nancarrow wrote: > > > On Fri, Feb 26, 2021 at 4:07 PM Amit Langote > > > wrote: > > > >

Re: [BUG] segfault during delete

2021-02-28 Thread Amit Langote
the test case > to pg11. It worked fine for me (with no code changes), but it seems good to > have it there just to make sure the buildfarm agrees with us on this. Ah, good call. -- Amit Langote EDB: http://www.enterprisedb.com

Re: a misbehavior of partition row movement (?)

2021-02-25 Thread Amit Langote
such behaviour on a referencing table. > I think it is worth adding some explanation in this document. Thus, explaining > impact on referencing tables here, as it already describes behaviour of > UPDATE on a partitioned table. ISTM the description of the case that will now be prevented s

Re: Parallel INSERT (INTO ... SELECT ...)

2021-02-25 Thread Amit Langote
On Fri, Feb 26, 2021 at 3:35 PM Greg Nancarrow wrote: > On Fri, Feb 26, 2021 at 4:07 PM Amit Langote wrote: > > The attached patch fixes this, although I am starting to have second > > thoughts about how we're tracking partitions in this patch. Wondering > > if we shou

Re: Parallel INSERT (INTO ... SELECT ...)

2021-02-25 Thread Amit Langote
Hi Greg, Replying to an earlier email in the thread because I think I found a problem relevant to the topic that was brought up. On Wed, Feb 17, 2021 at 10:34 PM Amit Langote wrote: > On Wed, Feb 17, 2021 at 10:44 AM Greg Nancarrow wrote: > > Is the use of "table_clo

Re: Parallel INSERT (INTO ... SELECT ...)

2021-02-25 Thread Amit Langote
ses do the thing that produces a better plan [2]. While no compromises can be made for the former, whether or not to go for the latter probably involves some cost-benefit analysis, something we can probably revisit. I don't think we're compromising by not adding the partition OIDs when

Re: [BUG] segfault during delete

2021-02-24 Thread Amit Langote
On Wed, Feb 24, 2021 at 6:12 PM Drouvot, Bertrand wrote: > On 2/24/21 9:12 AM, Amit Langote wrote: > > I've attached a patch with my suggested fixes and also test cases. > > Please take a look. > > I had a look and it looks good to me. Also the new regression tests ar

Re: [BUG] segfault during delete

2021-02-24 Thread Amit Langote
the block for populating the transition NEW TABLE, because without doing so that block too can crash similarly: if (!TupIsNull(newslot) && ((event == TRIGGER_EVENT_INSERT && insert_new_table) || (event == TRIGGER_EVENT_UPDATE && update_new_table))) I've attached a patch with my suggested fixes and also test cases. Please take a look. -- Amit Langote EDB: http://www.enterprisedb.com v2-0001-Fix-use-after-tree-bug-with-AfterTriggersTableDat.patch Description: Binary data

Re: A reloption for partitioned tables - parallel_workers

2021-02-23 Thread Amit Langote
TableRdOptions +{ + int32 vl_len_; /* varlena header (do not touch directly!) */ + int parallel_workers; /* max number of parallel workers */ +} PartitionedTableRdOptions; + #define HEAP_MIN_FILLFACTOR 10 #define HEAP_DEFAULT_FILLFACTOR 100 -- Amit Langote EDB: http://www.enterprisedb.com

Re: a misbehavior of partition row movement (?)

2021-02-21 Thread Amit Langote
On Fri, Feb 19, 2021 at 5:04 PM Masahiko Sawada wrote: > On Mon, Feb 15, 2021 at 10:37 PM Amit Langote wrote: > > Regarding the patch, I would have liked if it only prevented the > > update when the foreign key that would be violated by the component > > delete references th

Re: Parallel INSERT (INTO ... SELECT ...)

2021-02-21 Thread Amit Langote
On Fri, Feb 19, 2021 at 7:38 PM Amit Kapila wrote: > On Thu, Feb 18, 2021 at 11:05 AM Amit Langote wrote: > > > > > > It also occurred to me that we can avoid pointless adding of > > > > partitions if the final plan won't use parallelism. For that

<    2   3   4   5   6   7   8   9   10   11   >