Optimize planner memory consumption for huge arrays

2023-09-03 Thread Lepikhov Andrei
Hi, hackers, Looking at the planner behaviour with the memory consumption patch [1], I figured out that arrays increase memory consumption by the optimizer significantly. See init.sql in attachment. The point here is that the planner does small memory allocations for each element during

Re: Report planning memory in EXPLAIN ANALYZE

2023-09-04 Thread Lepikhov Andrei
Using your patch I found out one redundant memory usage in the planner [1]. It can be interesting as an example of how this patch can detect problems. [1] Optimize planner memory consumption for huge arrays

Re: MergeJoin beats HashJoin in the case of multiple hash clauses

2023-09-11 Thread Lepikhov Andrei
On Mon, Sep 11, 2023, at 11:51 AM, Andy Fan wrote: > Hi, > > On Thu, Jun 15, 2023 at 4:30 PM Andrey Lepikhov > wrote: >> Hi, all. >> >> Some of my clients use JOIN's with three - four clauses. Quite >> frequently, I see complaints on unreasonable switch of JOIN algorithm to >> Merge Join

Re: POC: GUC option for skipping shared buffers in core dumps

2023-09-13 Thread Lepikhov Andrei
On Wed, Feb 12, 2020, at 7:55 AM, tsunakawa.ta...@fujitsu.com wrote: > From: Craig Ringer >> Currently my options are "dump all shmem including shared_buffers" or >> "dump no shmem". But I usually want "dump all shmem except >> shared_buffers". It's tolerable to just dump s_b on a test system

Re: Optimize planner memory consumption for huge arrays

2023-09-07 Thread Lepikhov Andrei
On Wed, Sep 6, 2023, at 8:09 PM, Ashutosh Bapat wrote: > Hi Lepikhov, > > Thanks for using my patch and I am glad that you found it useful. > > On Mon, Sep 4, 2023 at 10:56 AM Lepikhov Andrei > wrote: >> >> Hi, hackers, >> >> Looking at the planner beha

Re: RFC: Logging plan of the running query

2023-09-15 Thread Lepikhov Andrei
On Thu, Sep 7, 2023, at 1:09 PM, torikoshia wrote: > On 2023-09-06 11:17, James Coleman wrote: > It seems that we can know what queries were running from the stack > traces you shared. > As described above, I suspect a lock which was acquired prior to > ProcessLogQueryPlanInterrupt() caused the

Re: Optimize planner memory consumption for huge arrays

2023-09-04 Thread Lepikhov Andrei
On Mon, Sep 4, 2023, at 3:37 PM, Dilip Kumar wrote: > On Mon, Sep 4, 2023 at 11:58 AM Lepikhov Andrei > wrote: >> >> Hi, hackers, >> >> Looking at the planner behaviour with the memory consumption patch [1], I >> figured out that arrays increase

Re: RFC: Logging plan of the running query

2023-09-19 Thread Lepikhov Andrei
On Tue, Sep 19, 2023, at 8:39 PM, torikoshia wrote: > On 2023-09-15 15:21, Lepikhov Andrei wrote: >> On Thu, Sep 7, 2023, at 1:09 PM, torikoshia wrote: >> I have explored this patch and, by and large, agree with the code. But >> some questions I want to discuss: >>

Re: [PoC] Reducing planning time when tables have many partitions

2023-09-21 Thread Lepikhov Andrei
On Wed, Sep 20, 2023, at 5:04 PM, Yuya Watari wrote: > On Tue, Sep 19, 2023 at 5:21 PM Andrey Lepikhov > wrote: >> Working on self-join removal in the thread [1] nearby, I stuck into the >> problem, which made an additional argument to work in this new direction >> than a couple of previous ones.

Re: disfavoring unparameterized nested loops

2023-09-20 Thread Lepikhov Andrei
On Wed, Sep 20, 2023, at 4:49 PM, David Rowley wrote: > On Wed, 20 Sept 2023 at 19:56, Andrey Lepikhov > wrote: >> What could you say about a different way: hybrid join? In MS SQL Server, >> they have such a feature [1], and, according to their description, it >> requires low overhead. They

Re: Comment about set_join_pathlist_hook()

2023-09-20 Thread Lepikhov Andrei
On Wed, Sep 20, 2023, at 5:05 PM, Etsuro Fujita wrote: > Hi, > > What I am concerned about from the report [1] is that this comment is > a bit too terse; it might cause a misunderstanding that extensions can > do different things than we intend to allow: > > /* > * 6. Finally, give

Re: Comment about set_join_pathlist_hook()

2023-09-21 Thread Lepikhov Andrei
On Thu, Sep 21, 2023, at 12:53 PM, Etsuro Fujita wrote: > Hi, > > On Thu, Sep 21, 2023 at 11:49 AM Lepikhov Andrei > wrote: >> On Wed, Sep 20, 2023, at 5:05 PM, Etsuro Fujita wrote: >> > What I am concerned about from the report [1] is that this comment is >>