Re: RFC: Logging plan of the running query

2023-09-27 Thread Andrey Lepikhov
On 28/9/2023 09:04, torikoshia wrote: On 2023-09-25 18:49, Andrey Lepikhov wrote: On 25/9/2023 14:21, torikoshia wrote: On 2023-09-20 14:39, Lepikhov Andrei wrote: Hmm, as a test, I made sure to call ProcessLogQueryPlanInterrupt() on all CFI using v28-0002-Testing-attempt-logging-plan

Re: POC: GROUP BY optimization

2023-09-25 Thread Andrey Lepikhov
statistics on distinct values and these statistics cover some set of first columns in the grouping list, we can optimize these positions. It also looks reliable. Any thoughts? -- regards, Andrey Lepikhov Postgres Professional

Re: RFC: Logging plan of the running query

2023-09-25 Thread Andrey Lepikhov
either. I just feel uncomfortable if, at the moment of interruption, we have a descriptor of another query than the query have been executing and holding resources. -- regards, Andrey Lepikhov Postgres Professional

Re: POC: GUC option for skipping shared buffers in core dumps

2023-09-25 Thread Andrey Lepikhov
, Andrey Lepikhov Postgres Professional diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c index 5810f8825e..4d7bf2c0e4 100644 --- a/src/backend/bootstrap/bootstrap.c +++ b/src/backend/bootstrap/bootstrap.c @@ -325,7 +325,7 @@ BootstrapModeMain(int argc, char *argv

Re: Postgres picks suboptimal index after building of an extended statistics

2023-09-24 Thread Andrey Lepikhov
nge compare_path_costs_fuzzily() and add some heuristic, for example: "If selectivity of both paths gives us no more than 1 row, prefer to use a unique index or an index with least selectivity." -- regards, Andrey Lepikhov Postgres Professional

Re: [PATCH] Add extra statistics to explain for Nested Loop

2023-09-22 Thread Andrey Lepikhov
ers? And at the end. If someone wants a lot of additional statistics, why not give them that by extension? It is only needed to add a hook into the point of the node explanation and some efforts to make instrumentation extensible. But here, honestly, I don't have code/ideas so far. -- regards, And

Re: disfavoring unparameterized nested loops

2023-09-20 Thread Andrey Lepikhov
ntains too small tuples. It solves the issue, Isn't it? [1] https://techcommunity.microsoft.com/t5/sql-server-blog/introducing-batch-mode-adaptive-joins/ba-p/385411 -- regards, Andrey Lepikhov Postgres Professional

Re: Oversight in reparameterize_path_by_child leading to executor crash

2023-09-20 Thread Andrey Lepikhov
can build a more general view of the problem with this patch. [1] Asymmetric partition-wise JOIN https://www.postgresql.org/message-id/flat/CAOP8fzaVL_2SCJayLL9kj5pCA46PJOXXjuei6-3aFUV45j4LJQ%40mail.gmail.com -- regards, Andrey Lepikhov Postgres Professional

Re: [PoC] Reducing planning time when tables have many partitions

2023-09-19 Thread Andrey Lepikhov
of derives or ec_members, we should go through all the index lists and fix them, which is a non-trivial operation. [1] https://www.postgresql.org/message-id/flat/64486b0b-0404-e39e-322d-0801154901f3%40postgrespro.ru -- regards, Andrey Lepikhov Postgres Professional

Re: POC: GROUP BY optimization

2023-09-18 Thread Andrey Lepikhov
On 20/7/2023 18:46, Tomas Vondra wrote: On 7/20/23 08:37, Andrey Lepikhov wrote: On 3/10/2022 21:56, Tom Lane wrote: Revert "Optimize order of GROUP BY keys". This reverts commit db0d67db2401eb6238ccc04c6407a4fd4f985832 and several follow-on fixes. ... Since we're hard up against t

Re: POC: GROUP BY optimization

2023-09-12 Thread Andrey Lepikhov
in the query plan in attachment. -- regards, Andrey Lepikhov Postgres Professional From 33953655c9ac3f9ec64b80c9f2a2ff38bd178745 Mon Sep 17 00:00:00 2001 From: "Andrey V. Lepikhov" Date: Wed, 13 Sep 2023 11:20:03 +0700 Subject: [PATCH] Explore alternative orderings of group-by pathk

Re: Removing unneeded self joins

2023-09-12 Thread Andrey Lepikhov
On 5/7/2023 21:28, Andrey Lepikhov wrote: Hi, During the significant code revision in v.41 I lost some replacement operations. Here is the fix and extra tests to check this in the future. Also, Tom added the JoinDomain structure five months ago, and I added code to replace relids

Re: Report planning memory in EXPLAIN ANALYZE

2023-08-13 Thread Andrey Lepikhov
or typical queries. -- regards, Andrey Lepikhov Postgres Professional

Re: Report planning memory in EXPLAIN ANALYZE

2023-08-10 Thread Andrey Lepikhov
looks good, passes the tests. -- regards, Andrey Lepikhov Postgres Professional

Re: [PoC] Reducing planning time when tables have many partitions

2023-08-07 Thread Andrey Lepikhov
On 7/8/2023 19:15, Ashutosh Bapat wrote: On Mon, Aug 7, 2023 at 2:21 PM Andrey Lepikhov mailto:a.lepik...@postgrespro.ru>> wrote: >> Do you think that the memory measurement patch I have shared in those threads is useful in itself? If so, I will start another

Re: [PoC] Reducing planning time when tables have many partitions

2023-08-07 Thread Andrey Lepikhov
mory-related info in the output of EXPLAIN ANALYZE makes tests more complex because of architecture dependency. -- regards, Andrey Lepikhov Postgres Professional

Re: [PoC] Reducing planning time when tables have many partitions

2023-08-02 Thread Andrey Lepikhov
is one possible option. You introduced list_ptr_cmp as an extern function of a List, but use it the only under USE_ASSERT_CHECKING ifdef. Maybe you hide it under USE_ASSERT_CHECKING or remove all the stuff? -- regards, Andrey Lepikhov Postgres Professional

Re: [PoC] Reducing planning time when tables have many partitions

2023-07-27 Thread Andrey Lepikhov
ase in attachment. Here is three queries. Execution times: 1 - 8s; 2 - 30s; 3 - 131s (with your patch set). 1 - 5s; 2 - 10s; 3 - 33s (current master). Maybe it is a false alarm, but on my laptop I see this degradation at every launch. -- regards, Andrey Lepikhov Postgres Professional parts-problem.

Re: POC: GROUP BY optimization

2023-07-24 Thread Andrey Lepikhov
On 24/7/2023 16:56, Tomas Vondra wrote: On 7/24/23 04:10, Andrey Lepikhov wrote: On 20/7/2023 18:46, Tomas Vondra wrote: On 7/20/23 08:37, Andrey Lepikhov wrote: On 3/10/2022 21:56, Tom Lane wrote: Revert "Optimize order of GROUP BY keys". This reve

Re: POC: GROUP BY optimization

2023-07-23 Thread Andrey Lepikhov
On 20/7/2023 18:46, Tomas Vondra wrote: On 7/20/23 08:37, Andrey Lepikhov wrote: On 3/10/2022 21:56, Tom Lane wrote: Revert "Optimize order of GROUP BY keys". This reverts commit db0d67db2401eb6238ccc04c6407a4fd4f985832 and several follow-on fixes. ... Since we're hard up against t

Re: POC: GROUP BY optimization

2023-07-20 Thread Andrey Lepikhov
On 20/7/2023 18:46, Tomas Vondra wrote: On 7/20/23 08:37, Andrey Lepikhov wrote: On 3/10/2022 21:56, Tom Lane wrote: Revert "Optimize order of GROUP BY keys". This reverts commit db0d67db2401eb6238ccc04c6407a4fd4f985832 and several follow-on fixes. ... Since we're hard up against t

Re: POC: GROUP BY optimization

2023-07-20 Thread Andrey Lepikhov
cording to the reasons uttered in the revert commit. -- regards, Andrey Lepikhov Postgres Professional From 913d55ee887dccfeba360f5f44ed347dd1ba9044 Mon Sep 17 00:00:00 2001 From: "Andrey V. Lepikhov" Date: Fri, 14 Jul 2023 10:29:36 +0700 Subject: [PATCH] When evaluating a query with a

Re: POC, WIP: OR-clause support for indexes

2023-07-11 Thread Andrey Lepikhov
herit.out: (((a)::text = 'ab'::text) OR ((a)::text = ANY ('{NULL,cd}'::text[]))) to (((a)::text = ANY ('{NULL,cd}'::text[])) OR ((a)::text = 'ab'::text)) Transformations, mentioned above, are correct, of course. But it can be a sign of possible unstable behavior. -- regards, Andrey Lepikhov

Re: Generating code for query jumbling through gen_node_support.pl

2023-07-11 Thread Andrey Lepikhov
On 11/7/2023 12:35, Michael Paquier wrote: On Tue, Jul 11, 2023 at 12:29:29PM +0700, Andrey Lepikhov wrote: I vote for only one method based on a query tree structure. Noted BTW, did you think about different algorithms of queryId generation? Not really, except if you are referring

Re: Generating code for query jumbling through gen_node_support.pl

2023-07-10 Thread Andrey Lepikhov
open a way for extensions to have easy-supporting custom queryIds. -- regards, Andrey Lepikhov Postgres Professional

Re: POC, WIP: OR-clause support for indexes

2023-07-09 Thread Andrey Lepikhov
we don't need to free it at all. -- regards, Andrey Lepikhov Postgres Professional diff --git a/src/backend/parser/parse_expr.c b/src/backend/parser/parse_expr.c index 961ca3e482..f0fd63f05c 100644 --- a/src/backend/parser/parse_expr.c +++ b/src/backend/parser/parse_expr.c @@ -112,9 +112,6 @@ transf

Re: POC, WIP: OR-clause support for indexes

2023-07-06 Thread Andrey Lepikhov
of a clause. Constant side of the expression is detected by call of eval_const_expressions() and check each side on the Const type of node. See 'diff to diff' in attachment. -- regards, Andrey Lepikhov Postgres Professional diff --git a/src/backend/parser/parse_expr.c b/src/backend/parser/parse_expr.c

Re: POC, WIP: OR-clause support for indexes

2023-07-06 Thread Andrey Lepikhov
expressions like "F(X)=Const", not an 'F(X)=ConstExpression'. See delta.diff with mentioned changes in attachment. -- regards, Andrey Lepikhov Postgres Professional diff --git a/src/backend/parser/parse_expr.c b/src/backend/parser/parse_expr.c index c9193d826f..26648b0876 100644 --- a/s

Re: Removing unneeded self joins

2023-07-05 Thread Andrey Lepikhov
didn't replace SJs, defined by baserestrictinfos if no one self-join clause have existed for the join. Now, it is fixed, and the test has been added. To understand changes readily, see the delta file in the attachment. -- regards, Andrey Lepikhov Postgres Professional From

Re: Removing unneeded self joins

2023-06-30 Thread Andrey Lepikhov
Lepikhov Postgres Professional From 4a342b9789f5be209318c13fb7ec336fcbd2aee5 Mon Sep 17 00:00:00 2001 From: Andrey Lepikhov Date: Mon, 15 May 2023 09:04:51 +0500 Subject: [PATCH] Remove self-joins. A Self Join Elimination (SJE) feature removes inner join of plain table to itself in a query tree

Re: Problems with estimating OR conditions, IS NULL on LEFT JOINs

2023-06-26 Thread Andrey Lepikhov
; EXPLAIN ANALYZE SELECT * FROM l LEFT OUTER JOIN r ON (r.id = l.id) WHERE r.v IS NULL; Here you can see the same kind of underestimation: Hash Left Join (... rows=500 width=14) (... rows=9 ...) So the eqjoinsel_unmatch_left() function should be modified for the case where nd1 -- regards, Andrey

Re: eqjoinsel_semi still sucks ...

2023-06-23 Thread Andrey Lepikhov
and is limited. I've tried to understand the logic through commits 0d3b231eebf, 97930cf578e and 7f3eba30c9d. But it is still not clear. So, why the idea of clamping ndistinct is terrible in general? Could you explain your reasons a bit more? -- regards, Andrey Lepikhov Postgres Professional

MergeJoin beats HashJoin in the case of multiple hash clauses

2023-06-15 Thread Andrey Lepikhov
components. But as for me, here we should go the same way, as estimation of groups. The attached patch shows a sketch of the solution. -- regards, Andrey Lepikhov Postgres Professional q2.sql Description: application/sql diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer

Re: Removing unneeded self joins

2023-05-25 Thread Andrey Lepikhov
eless outer join' and 'Self join' elimination optimizations. Now, because of the 'ojrelid' field it looks too complicated. Do we need to split this routine again? -- Regards Andrey Lepikhov Postgres Professional From cb4340577dab0e8cf5531e9934f5734fda178490 Mon Sep 17 00:00:00 2001 From: Andrey L

Re: [POC] Allow an extension to add data into Query and PlannedStmt nodes

2023-03-30 Thread Andrey Lepikhov
and want to pass it along all planning and execution stages it should use extensible node with custom read/write/copy routines. -- regards, Andrey Lepikhov Postgres Professional From ab101322330684e9839e46c26f70ad5462e40dac Mon Sep 17 00:00:00 2001 From: "Andrey V. Lepikhov" Date: Thu, 30 Mar

[POC] Allow an extension to add data into Query and PlannedStmt nodes

2023-03-29 Thread Andrey Lepikhov
hooks for some purposes. So, any thoughts will be useful. -- Regards Andrey Lepikhov Postgres ProfessionalFrom 944ce61d7ff934727240d90ee7620bfb69ad3a5a Mon Sep 17 00:00:00 2001 From: Andrey Lepikhov Date: Wed, 22 Mar 2023 16:59:30 +0500 Subject: [PATCH] Add on more field into Query

Re: [PoC] Reducing planning time when tables have many partitions

2023-02-14 Thread Andrey Lepikhov
instead? -- Regards Andrey Lepikhov Postgres Professional

Re: [PATCH] random_normal function

2023-01-18 Thread Andrey Lepikhov
On 1/19/23 11:01, Tom Lane wrote: Andrey Lepikhov writes: On 1/9/23 23:52, Tom Lane wrote: BTW, if this does bring the probability of failure down to the one-in-a-billion range, I think we could also nuke the whole "ignore:" business, simplifying pg_regress and allowing the r

Re: [PATCH] random_normal function

2023-01-18 Thread Andrey Lepikhov
quot;isolation tests" option to create stable execution time-dependent tests now? Or I'm not aware about some test machinery? -- Regards Andrey Lepikhov Postgres Professional

Re: POC, WIP: OR-clause support for indexes

2022-12-27 Thread Andrey Lepikhov
-optimal BitmapOr in the case of many OR's possibly aggravated by many indexes on the relation. For example, such operation can be executed in create_index_paths() before passing rel->indexlist. -- Regards Andrey Lepikhov Postgres Professional demo.sql Description: application/sql

Re: Optimization issue of branching UNION ALL

2022-12-22 Thread Andrey Lepikhov
it. Thanks, I have written the letter because of some doubts too. But only one weak point I could imagine - if someday sql standard will be changed. Your code looks better, than previous attempt. -- regards, Andrey Lepikhov Postgres Professional

Optimization issue of branching UNION ALL

2022-12-20 Thread Andrey Lepikhov
it? In attachment you can see some sketch that reduces a number of planner cycles/copyings. -- Regards Andrey Lepikhov Postgres Professional t.sh Description: application/shellscript diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c index 08a73fb9d86

Re: Removing unneeded self joins

2022-12-15 Thread Andrey Lepikhov
sted Loop -> HashAggregate This change in the test behaviour is induced by the a5fc4641 "Avoid making commutatively-duplicate clauses in EquivalenceClasses." Nothing special, as I see. Attached patch fixes this. -- Regards Andrey Lepikhov Postgres Professional From 3e546637561bf4c6d1

Re: [PoC] Reducing planning time when tables have many partitions

2022-11-08 Thread Andrey Lepikhov
istake, maybe to add a comment why assertion here isn't failed? -- regards, Andrey Lepikhov Postgres Professional

Re: [PoC] Reducing planning time when tables have many partitions

2022-11-06 Thread Andrey Lepikhov
see, everywhere access to these lists guides by eclass_source_indexes and eclass_derive_indexes correspondingly. Maybe to merge them? -- regards, Andrey Lepikhov Postgres Professional

Re: explain analyze rows=%.0f

2022-11-06 Thread Andrey Lepikhov
ormat like X.XXEXX. I vote for second option. -- regards, Andrey Lepikhov Postgres Professional

Re: A new strategy for pull-up correlated ANY_SUBLINK

2022-11-01 Thread Andrey Lepikhov
such corner cases too ? [1] https://www.postgresql.org/message-id/flat/CALNJ-vTa5VgvV1NPRHnypdnbx-fhDu7vWp73EkMUbZRpNHTYQQ%40mail.gmail.com -- regards, Andrey Lepikhov Postgres Professional

Re: Fast COPY FROM based on batch insert

2022-10-28 Thread Andrey Lepikhov
studied performance of this code in comparison to bulk INSERTions. This patch seems to improve speed of insertion by about 20%. Also, this patch is very invasive. So, I don't have any plans to work on it now. -- regards, Andrey Lepikhov Postgres Professional

Re: Fast COPY FROM based on batch insert

2022-10-12 Thread Andrey Lepikhov
On 10/12/22 07:56, Etsuro Fujita wrote: On Tue, Oct 11, 2022 at 3:06 PM Andrey Lepikhov wrote: I reviewed the patch one more time. Only one question: bistate and ri_FdwRoutine are strongly bounded. Maybe to add some assertion on (ri_FdwRoutine XOR bistate) ? Just to prevent possible errors

Re: Fast COPY FROM based on batch insert

2022-10-11 Thread Andrey Lepikhov
o prevent possible errors in future. -- Regards Andrey Lepikhov Postgres Professional

Re: document the need to analyze partitioned tables

2022-10-06 Thread Andrey Lepikhov
test t1, test t2 WHERE t1.id = t2.val; VACUUM ANALYZE test; EXPLAIN (ANALYZE, TIMING OFF, SUMMARY OFF) SELECT * FROM test t1, test t2 WHERE t1.id = t2.val; Here without actual statistics on parent table we make wrong prediction. -- Regards Andrey Lepikhov Postgres Professional

Re: Removing unneeded self joins

2022-10-05 Thread Andrey Lepikhov
New version, rebased onto current master. Nothing special, just rebase. -- regards, Andrey Lepikhov Postgres Professional From 03aab7a2431032166c9ea5f52fbcccaf7168abec Mon Sep 17 00:00:00 2001 From: "Andrey V. Lepikhov" Date: Wed, 5 Oct 2022 16:58:34 +0500 Subject: [PATCH] Remove

Re: [POC] Allow flattening of subquery with a link to upper query

2022-10-05 Thread Andrey Lepikhov
be better if the fields have comments. Ok, I've added some comments. +                    * (for grouping, as an example). So, revert its status to +                    * a full valued entry. full valued -> fully valued Fixed -- regards, Andrey Lepikhov Postgres Professional F

Re: [POC] Allow flattening of subquery with a link to upper query

2022-10-03 Thread Andrey Lepikhov
On 9/13/22 16:40, Andrey Lepikhov wrote: On 5/9/2022 12:22, Richard Guo wrote: On Fri, Sep 2, 2022 at 7:09 PM Andrey Lepikhov mailto:a.lepik...@postgrespro.ru>> wrote: To resolve both issues, lower outer join passes through pull_sublinks_* into flattening routine (see attachment). I've

Re: [POC] Allow flattening of subquery with a link to upper query

2022-09-13 Thread Andrey Lepikhov
On 5/9/2022 12:22, Richard Guo wrote: On Fri, Sep 2, 2022 at 7:09 PM Andrey Lepikhov mailto:a.lepik...@postgrespro.ru>> wrote: > Hmm, I'm not sure this patch works correctly in all cases. It seems to > me this patch pulls up the subquery without checking the

Re: [POC] Allow flattening of subquery with a link to upper query

2022-09-05 Thread Andrey Lepikhov
On 9/5/22 12:22, Richard Guo wrote: On Fri, Sep 2, 2022 at 7:09 PM Andrey Lepikhov Yeah, it's not easy-to-solve problem. If I correctly understand the code, to fix this problem we must implement the same logic, as pull_up_subqueries (lowest_outer_join/safe_upper_varnos). Yeah

Re: [POC] Allow flattening of subquery with a link to upper query

2022-09-02 Thread Andrey Lepikhov
On 9/1/22 17:24, Richard Guo wrote: On Wed, Aug 31, 2022 at 2:35 PM Andrey Lepikhov mailto:a.lepik...@postgrespro.ru>> wrote: Before flattening procedure we just look through the quals of subquery, pull to the upper level OpExpr's containing variables from the upper re

Re: [HACKERS] PoC: custom signal handler for extensions

2022-09-01 Thread Andrey Lepikhov
concept of the auto_explain extension? -- Regards Andrey Lepikhov Postgres Professional

[POC] Allow flattening of subquery with a link to upper query

2022-08-31 Thread Andrey Lepikhov
, Won. “On optimizing an SQL-like nested query.” ACM Trans. Database Syst. 7 (1982): 443-469. -- Regards Andrey Lepikhov Postgres ProfessionalFrom 3f4247b23175388f8c6ee43740fb641d97e39d0b Mon Sep 17 00:00:00 2001 From: "Andrey V. Lepikhov" Date: Tue, 24 May 2022 15:59:02 +0500 Subje

Re: Removing unneeded self joins

2022-08-28 Thread Andrey Lepikhov
enseless search cycles of self-joins. And it may have higher limit than GUCs above. So I introduced a guc, called "self_join_search_limit" (so far undocumented) that is an explicit limit for a set of plain relations in FROM-list to search self-joins. -- Regards Andrey Lepikhov

Re: Removing unneeded self joins

2022-08-26 Thread Andrey Lepikhov
On 30/6/2022 17:11, Andrey Lepikhov wrote: On 19/5/2022 16:47, Ronan Dunklau wrote: I'll take a look at that one. New version of the patch, rebased on current master: 1. pgindent over the patch have passed. 2. number of changed files is reduced. 3. Some documentation and comments is added

Re: Fast COPY FROM based on batch insert

2022-08-22 Thread Andrey Lepikhov
, which I think makes the range information less useful. Maybe I'm too worried about that, though. I got your point. Indeed, perharps such info doesn't really needed to be included into the core, at least for now. -- regards, Andrey Lepikhov Postgres Professional

Re: Fast COPY FROM based on batch insert

2022-08-14 Thread Andrey Lepikhov
'COPY %s (buffered)' or 'COPY FOREIGN TABLE %s' or, if instead of relname_only field to save a MultiInsertBuffer pointer, we might add min/max linenos into the report: 'COPY %s, line between %llu and %llu' -- Regards Andrey Lepikhov Postgres Professional

Re: Fast COPY FROM based on batch insert

2022-07-26 Thread Andrey Lepikhov
On 7/22/22 13:14, Etsuro Fujita wrote: On Fri, Jul 22, 2022 at 3:39 PM Andrey Lepikhov wrote: Analyzing multi-level heterogeneous partitioned configurations I realized, that single write into a partition with a trigger will flush buffers for all other partitions of the parent table even

Re: Fast COPY FROM based on batch insert

2022-07-22 Thread Andrey Lepikhov
On 7/22/22 13:14, Etsuro Fujita wrote: On Fri, Jul 22, 2022 at 3:39 PM Andrey Lepikhov Why such cascade flush is really necessary, especially for BEFORE and INSTEAD OF triggers? BEFORE triggers on the chosen partition might query the parent table, not just the partition, so I think we need

Re: Fast COPY FROM based on batch insert

2022-07-22 Thread Andrey Lepikhov
On 7/20/22 13:10, Etsuro Fujita wrote: On Tue, Jul 19, 2022 at 6:35 PM Andrey Lepikhov wrote: On 18/7/2022 13:22, Etsuro Fujita wrote: I rewrote the decision logic to something much simpler and much less invasive, which reduces the patch size significantly. Attached is an updated patch

Re: [PoC] Reducing planning time when tables have many partitions

2022-07-21 Thread Andrey Lepikhov
to reverse the status if you need more feedback. -- Regards Andrey Lepikhov Postgres Professional

Re: Fast COPY FROM based on batch insert

2022-07-19 Thread Andrey Lepikhov
ribe in documentation, if the value of batch_size is more than 1, the ExecForeignBatchInsert routine have a chance to be called? -- regards, Andrey Lepikhov Postgres Professional

Re: Postgres picks suboptimal index after building of an extended statistics

2022-07-11 Thread Andrey Lepikhov
On 7/8/22 03:07, Tom Lane wrote: Andrey Lepikhov writes: On 12/8/21 04:26, Tomas Vondra wrote: I wonder if we should teach clauselist_selectivity about UNIQUE indexes, and improve the cardinality estimates directly, not just costing for index scans. I tried to implement this in different

Re: Fast COPY FROM based on batch insert

2022-07-10 Thread Andrey Lepikhov
On 11/7/2022 04:12, Ian Barwick wrote: On 09/07/2022 00:09, Andrey Lepikhov wrote: On 8/7/2022 05:12, Ian Barwick wrote: ERROR:  bind message supplies 0 parameters, but prepared statement "pgsql_fdw_prep_178" requires 6 CONTEXT:  remote SQL command: INSERT INTO public.fo

Re: Fast COPY FROM based on batch insert

2022-07-08 Thread Andrey Lepikhov
different change (see in attachment). -- regards, Andrey Lepikhov Postgres Professionaldiff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c index 245a260982..203289f7f2 100644 --- a/src/backend/commands/copyfrom.c +++ b/src/backend/commands/copyfrom.c @@ -32

Re: Fast COPY FROM based on batch insert

2022-07-07 Thread Andrey Lepikhov
On 7/7/2022 06:14, Ian Barwick wrote: 2022年3月24日(木) 15:44 Andrey V. Lepikhov : > > On 3/22/22 06:54, Etsuro Fujita wrote: > > On Fri, Jun 4, 2021 at 5:26 PM Andrey Lepikhov > > wrote: > >> We still have slow 'COPY FROM' operation for foreign tables in current

Re: Removing unneeded self joins

2022-06-30 Thread Andrey Lepikhov
On 19/5/2022 16:47, Ronan Dunklau wrote: I'll take a look at that one. New version of the patch, rebased on current master: 1. pgindent over the patch have passed. 2. number of changed files is reduced. 3. Some documentation and comments is added. -- regards, Andrey Lepikhov Postgres

Re: Postgres do not allow to create many tables with more than 63-symbols prefix

2022-06-26 Thread Andrey Lepikhov
On 6/27/22 06:38, Masahiko Sawada wrote: On Fri, Jun 24, 2022 at 2:12 PM Andrey Lepikhov wrote: On 6/23/22 07:03, Masahiko Sawada wrote: > On Sat, Jun 4, 2022 at 4:03 AM Andrey Lepikhov > wrote: >> It is very corner case, of course. But solution is easy and short.

Re: Implement hook for self-join simplification

2022-06-24 Thread Andrey Lepikhov
generalize parts of the core code and thus, reduce size of your code a lot. But if you want to use your code with many PG versions, even already working in production or you make just a research, without immediate practical result - your choice is an extension. -- regards, Andrey Lepikhov Postgres

Re: Implement hook for self-join simplification

2022-06-24 Thread Andrey Lepikhov
free to use it in your research. [1] https://www.postgresql.org/message-id/a1d6290c-44e0-0dfc-3fca-66a68b310...@postgrespro.ru -- regards, Andrey Lepikhov Postgres Professional

Postgres do not allow to create many tables with more than 63-symbols prefix

2022-06-23 Thread Andrey Lepikhov
Moved from the pgsql-bugs mailing list [1]. On 6/23/22 07:03, Masahiko Sawada wrote: > Hi, > > On Sat, Jun 4, 2022 at 4:03 AM Andrey Lepikhov > wrote: >> >> According to subj you can try to create many tables (induced by the case >> of partitioned table) wi

Re: [PROPOSAL] Detecting plan changes with plan_id in pg_stat_activity

2022-06-15 Thread Andrey Lepikhov
://www.postgresql.org/message-id/flat/e0de3423-4bba-1e69-c55a-f76bf18dbd74%40postgrespro.ru -- regards, Andrey Lepikhov Postgres Professional

Re: Compare variables of composite type with slightly different column types

2022-05-28 Thread Andrey Lepikhov
On 26/5/2022 14:25, Andrey Lepikhov wrote: I guess, here the compatible_oper() routine can be used to find a appropriate operator, or something like that can be invented. I looked into the 2cd7084 and a4424c5, but don't found any rationale. In accordance to this idea I prepared a code

Compare variables of composite type with slightly different column types

2022-05-26 Thread Andrey Lepikhov
ionale. -- Regards Andrey Lepikhov Postgres Professional

Re: Removing unneeded self joins

2022-05-19 Thread Andrey Lepikhov
On 5/17/22 19:14, Ronan Dunklau wrote: Le vendredi 13 mai 2022, 07:07:47 CEST Andrey Lepikhov a écrit : New version of the feature. Here a minor bug with RowMarks is fixed. A degenerated case is fixed, when uniqueness of an inner deduced not from join quals, but from a baserestrictinfo clauses

Re: Removing unneeded self joins

2022-03-04 Thread Andrey Lepikhov
demo in attachment). Also, in new version of the patch I fixed one stupid bug: checking a self-join candidate expression operator - we can remove only expressions like F(arg1) = G(arg2). -- regards, Andrey Lepikhov Postgres ProfessionalFrom 70398361a0a0d9c6c3c7ddd1fd305ac11138e7b1 Mon Sep 17 00

Re: Merging statistics from children instead of re-sampling everything

2022-02-10 Thread Andrey Lepikhov
you read tuples locally. Also, to get such parts of samples asynchronously, we can get size of each partition on a preliminary step of analysis. In my opinion, even this solution can reduce heaviness of a problem drastically. -- regards, Andrey Lepikhov Postgres Professional

Re: Multiple Query IDs for a rewritten parse tree

2022-01-31 Thread Andrey Lepikhov
that might be dealing with similar concerns. I think, it depends on a specific purpose of an extension. -- regards, Andrey Lepikhov Postgres Professional

Re: POC: GROUP BY optimization

2022-01-23 Thread Andrey Lepikhov
path, not the modified one we built in the last iteration. Or am I missing something You are right, I misunderstood the idea of path_save variable. -- regards, Andrey Lepikhov Postgres Professional

Re: POC: GROUP BY optimization

2022-01-21 Thread Andrey Lepikhov
n attachment patch to previous fixes. -- regards, Andrey Lepikhov Postgres Professionaldiff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c index 70af9c91d5..4e26cd275d 100644 --- a/src/backend/optimizer/path/costsize.c +++ b/src/backend/optimizer/path/

Re: Multiple Query IDs for a rewritten parse tree

2022-01-10 Thread Andrey Lepikhov
On 10/1/2022 15:39, Julien Rouhaud wrote: On Mon, Jan 10, 2022 at 03:24:46PM +0500, Andrey Lepikhov wrote: On 10/1/2022 13:56, Julien Rouhaud wrote: Yes. the same input query string doesn't prove that frozen query plan can be used, because rewrite rules could be changed. So we use only a query

Re: Multiple Query IDs for a rewritten parse tree

2022-01-10 Thread Andrey Lepikhov
for performance to collect pointers to all constant nodes during a process of hash generation. -- regards, Andrey Lepikhov Postgres Professional

Multiple Query IDs for a rewritten parse tree

2022-01-08 Thread Andrey Lepikhov
Any thoughts, comments, criticism ? -- regards, Andrey Lepikhov Postgres Professional

Re: Add index scan progress to pg_stat_progress_vacuum

2021-12-23 Thread Andrey Lepikhov
/receive/representation implementation of progress? So AM would define a set of parameters to send into stat collector and show to users. -- regards, Andrey Lepikhov Postgres Professional

Re: Clarifying/rationalizing Vars' varno/varattno/varnoold/varoattno

2021-12-22 Thread Andrey Lepikhov
On 22/12/2021 20:42, Tom Lane wrote: Andrey Lepikhov writes: On 5/2/2020 01:24, Tom Lane wrote: I've not written any actual code, but am close to being ready to. This thread gives us hope to get started on solving some of the basic planner problems. But there is no activity for a long time

Re: Clarifying/rationalizing Vars' varno/varattno/varnoold/varoattno

2021-12-22 Thread Andrey Lepikhov
now? -- regards, Andrey Lepikhov Postgres Professional

Re: Make query ID more portable

2021-10-14 Thread Andrey Lepikhov
On 14/10/21 10:40, Julien Rouhaud wrote: On Thu, Oct 14, 2021 at 12:37 PM Andrey Lepikhov wrote: On 12/10/21 18:45, Bruce Momjian wrote: On Tue, Oct 12, 2021 at 09:40:47AM -0400, Tom Lane wrote: Andrey Lepikhov writes: I think that there are just too many arbitrary decisions that could

Re: Make query ID more portable

2021-10-13 Thread Andrey Lepikhov
On 12/10/21 18:40, Tom Lane wrote: Andrey Lepikhov writes: But core jumbling code is good, fast and much easier in support. A bigger issue is that query ID stability isn't something we are going to promise on a large scale --- for example, what if a new release adds some new fields to struct

Re: Make query ID more portable

2021-10-13 Thread Andrey Lepikhov
On 12/10/21 18:45, Bruce Momjian wrote: On Tue, Oct 12, 2021 at 09:40:47AM -0400, Tom Lane wrote: Andrey Lepikhov writes: But core jumbling code is good, fast and much easier in support. Also, the current code handles renames of schemas and objects, but this would not. Yes, It is good option

Re: Make query ID more portable

2021-10-12 Thread Andrey Lepikhov
ds in this code. It would allow an extension to intercept this call and replace oid with an arbitrary value. -- regards, Andrey Lepikhov Postgres Professional

Re: Increase value of OUTER_VAR

2021-09-15 Thread Andrey Lepikhov
. -- regards, Andrey Lepikhov Postgres Professional

Re: Asymmetric partition-wise JOIN

2021-09-15 Thread Andrey Lepikhov
regression test added. I thought more and realized there isn't necessary to recurse in the adjust_child_relids_multilevel() routine if required_outer contains only normal_relids. Also, regression tests were improved a bit. -- regards, Andrey Lepikhov Postgres Professional >F

Re: Postgres picks suboptimal index after building of an extended statistics

2021-08-30 Thread Andrey Lepikhov
patch in attachment. -- regards, Andrey Lepikhov Postgres Professional From 41ce6007cd552afd1a73983f0b9c9cac0e125d58 Mon Sep 17 00:00:00 2001 From: "Andrey V. Lepikhov" Date: Mon, 30 Aug 2021 11:21:57 +0500 Subject: [PATCH] Estimating number of fetched rows in a btree index we save s

Representation of SubPlan testexpr in EXPLAIN

2021-08-24 Thread Andrey Lepikhov
a.x) It is a bit annoying when you don't have original query or don't trust competence of a user who sent you this explain. In attachment - patch which solves this problem. I'm not completely sure that this option really needed and patch presents a proof of concept only. -- regards, Andrey Lepik

  1   2   3   >