Re: Memory consumed by paths during partitionwise join planning

2025-07-25 Thread Ashutosh Bapat
On Tue, Jul 22, 2025 at 7:31 PM Andrei Lepikhov wrote: > > On 18/7/2025 13:48, Ashutosh Bapat wrote: > > On Mon, Jul 7, 2025 at 8:43 PM Andrei Lepikhov wrote: > > if (!IsA(new_path, IndexPath)) > > - pfree(new_path); > > + free_path(new_path, 0, false); > > > > Why don't we free the subpaths if t

Re: Memory consumed by paths during partitionwise join planning

2025-07-22 Thread Andrei Lepikhov
On 18/7/2025 13:48, Ashutosh Bapat wrote: On Mon, Jul 7, 2025 at 8:43 PM Andrei Lepikhov wrote: if (!IsA(new_path, IndexPath)) - pfree(new_path); + free_path(new_path, 0, false); Why don't we free the subpaths if they aren't referenced anymore? During testing, I discovered that we sometimes enc

Re: Memory consumed by paths during partitionwise join planning

2025-07-18 Thread Ashutosh Bapat
On Mon, Jul 7, 2025 at 8:43 PM Andrei Lepikhov wrote: > > On 27/6/2025 12:01, Andrei Lepikhov wrote: > > On 6/2/2024 13:51, Ashutosh Bapat wrote: > >> On Fri, Dec 15, 2023 at 5:22 AM Ashutosh Bapat > >> wrote: > >> First patch is no longer required. Here's rebased set > >> > >> The patches are ra

Re: Memory consumed by paths during partitionwise join planning

2025-07-07 Thread Andrei Lepikhov
On 27/6/2025 12:01, Andrei Lepikhov wrote: On 6/2/2024 13:51, Ashutosh Bapat wrote: On Fri, Dec 15, 2023 at 5:22 AM Ashutosh Bapat wrote: First patch is no longer required. Here's rebased set The patches are raw. make check has some crashes that I need to fix. I am waiting to hear whether this

Re: Memory consumed by paths during partitionwise join planning

2025-06-27 Thread Andrei Lepikhov
On 6/2/2024 13:51, Ashutosh Bapat wrote: On Fri, Dec 15, 2023 at 5:22 AM Ashutosh Bapat wrote: First patch is no longer required. Here's rebased set The patches are raw. make check has some crashes that I need to fix. I am waiting to hear whether this is useful and whether the design is on the

Re: Memory consumed by paths during partitionwise join planning

2024-09-19 Thread Andrei Lepikhov
On 19/9/2024 13:12, Ashutosh Bapat wrote: On Thu, Sep 19, 2024 at 4:18 PM Andrei Lepikhov wrote: At the same time, this technique (while highly useful in general) adds fragility and increases complexity: a developer needs to remember to link the path using the pointer in different places of the

Re: Memory consumed by paths during partitionwise join planning

2024-09-19 Thread Ashutosh Bapat
On Thu, Sep 19, 2024 at 4:18 PM Andrei Lepikhov wrote: > > On 6/2/2024 13:51, Ashutosh Bapat wrote: > > On Fri, Dec 15, 2023 at 5:22 AM Ashutosh Bapat > > wrote: > > > >>> > >>> That looks pretty small considering the benefits. What do you think? > >>> > >>> [1] > >>> https://www.postgresql.org/

Re: Memory consumed by paths during partitionwise join planning

2024-09-19 Thread Andrei Lepikhov
On 6/2/2024 13:51, Ashutosh Bapat wrote: On Fri, Dec 15, 2023 at 5:22 AM Ashutosh Bapat wrote: That looks pretty small considering the benefits. What do you think? [1] https://www.postgresql.org/message-id/caexhw5stmouobe55pmt83r8uxvfcph+pvo5dnpdrvcsbgxe...@mail.gmail.com If you want to e

Re: Memory consumed by paths during partitionwise join planning

2024-02-19 Thread Ashutosh Bapat
On Tue, Feb 20, 2024 at 8:19 AM Andrei Lepikhov wrote: > > On 19/2/2024 19:25, Ashutosh Bapat wrote: > > On Fri, Feb 16, 2024 at 8:42 AM Andrei Lepikhov > > wrote: > >> Live example: right now, I am working on the code like MSSQL has - a > >> combination of NestLoop and HashJoin paths and switchi

Re: Memory consumed by paths during partitionwise join planning

2024-02-19 Thread Andrei Lepikhov
On 19/2/2024 19:25, Ashutosh Bapat wrote: On Fri, Feb 16, 2024 at 8:42 AM Andrei Lepikhov wrote: Live example: right now, I am working on the code like MSSQL has - a combination of NestLoop and HashJoin paths and switching between them in real-time. It requires both paths in the path list at th

Re: Memory consumed by paths during partitionwise join planning

2024-02-19 Thread Ashutosh Bapat
On Fri, Feb 16, 2024 at 8:42 AM Andrei Lepikhov wrote: > Live example: right now, I am working on the code like MSSQL has - a > combination of NestLoop and HashJoin paths and switching between them in > real-time. It requires both paths in the path list at the moment when > extensions are coming.

Re: Memory consumed by paths during partitionwise join planning

2024-02-15 Thread Andrei Lepikhov
On 15/2/2024 19:06, Ashutosh Bapat wrote: On Thu, Feb 15, 2024 at 9:41 AM Andrei Lepikhov But I'm not sure about freeing unreferenced paths. I would have to see alternatives in the pathlist. I didn't understand this. Can you please elaborate? A path in any pathlist is referenced. An unreferenc

Re: Memory consumed by paths during partitionwise join planning

2024-02-15 Thread Ashutosh Bapat
On Thu, Feb 15, 2024 at 9:41 AM Andrei Lepikhov wrote: > > On 6/2/2024 19:51, Ashutosh Bapat wrote: > > On Fri, Dec 15, 2023 at 5:22 AM Ashutosh Bapat > > The patches are raw. make check has some crashes that I need to fix. I > > am waiting to hear whether this is useful and whether the design is

Re: Memory consumed by paths during partitionwise join planning

2024-02-14 Thread Andrei Lepikhov
On 6/2/2024 19:51, Ashutosh Bapat wrote: On Fri, Dec 15, 2023 at 5:22 AM Ashutosh Bapat The patches are raw. make check has some crashes that I need to fix. I am waiting to hear whether this is useful and whether the design is on the right track. Let me write words of opinion on that feature. I

Re: Memory consumed by paths during partitionwise join planning

2024-02-06 Thread Ashutosh Bapat
On Fri, Dec 15, 2023 at 5:22 AM Ashutosh Bapat wrote: > > > > That looks pretty small considering the benefits. What do you think? > > > > [1] > > https://www.postgresql.org/message-id/caexhw5stmouobe55pmt83r8uxvfcph+pvo5dnpdrvcsbgxe...@mail.gmail.com > > If you want to experiment, please use at

Re: Memory consumed by paths during partitionwise join planning

2023-12-14 Thread Ashutosh Bapat
Forgot to mention, On Thu, Dec 14, 2023 at 5:34 PM Ashutosh Bapat wrote: > > On Thu, Dec 7, 2023 at 6:19 PM David Rowley wrote: > > > > Maybe we can try to move forward with your refcount idea and see how > > the performance looks. If that's intolerable then that might help us > > decide on the

Re: Memory consumed by paths during partitionwise join planning

2023-12-14 Thread Ashutosh Bapat
On Thu, Dec 7, 2023 at 6:19 PM David Rowley wrote: > > Maybe we can try to move forward with your refcount idea and see how > the performance looks. If that's intolerable then that might help us > decide on the next best alternative solution. > Here are performance numbers setup create table t

Re: Memory consumed by paths during partitionwise join planning

2023-12-08 Thread Ashutosh Bapat
On Fri, Dec 8, 2023 at 1:02 PM David Rowley wrote: > > On Fri, 8 Dec 2023 at 18:02, Ashutosh Bapat > wrote: > > given path. E.g. we have three path chains as follows > > 1. joinpath_1->joinpath_2->seqpath_1, > > 2. joinpath_3->joinpath_4->seqpath_1, > > 3. joinpath_5->joinpath_2->seqpath_1 > > >

Re: Memory consumed by paths during partitionwise join planning

2023-12-07 Thread David Rowley
On Fri, 8 Dec 2023 at 18:02, Ashutosh Bapat wrote: > given path. E.g. we have three path chains as follows > 1. joinpath_1->joinpath_2->seqpath_1, > 2. joinpath_3->joinpath_4->seqpath_1, > 3. joinpath_5->joinpath_2->seqpath_1 > > Please note that this is not full path tree/forest. It is difficult

Re: Memory consumed by paths during partitionwise join planning

2023-12-07 Thread Ashutosh Bapat
On Thu, Dec 7, 2023 at 6:19 PM David Rowley wrote: > > On Fri, 1 Dec 2023 at 19:59, Ashutosh Bapat > wrote: > > I am fine to work on this further if the community thinks it is > > acceptable. It seems from your point of view the worth of this work is > > as follows > > a. memory savings - not wor

Re: Memory consumed by paths during partitionwise join planning

2023-12-07 Thread David Rowley
On Fri, 1 Dec 2023 at 19:59, Ashutosh Bapat wrote: > I am fine to work on this further if the community thinks it is > acceptable. It seems from your point of view the worth of this work is > as follows > a. memory savings - not worth it > b. getting rid of !IsA(old_path, IndexPath) - worth it > c

Re: Memory consumed by paths during partitionwise join planning

2023-11-30 Thread Ashutosh Bapat
On Wed, Nov 29, 2023 at 1:10 AM David Rowley wrote: > > On Fri, 28 Jul 2023 at 02:06, Ashutosh Bapat > wrote: > > Table 1: Join between unpartitioned tables > > Number of tables | without patch | with patch | % reduction | > > being joined ||| | > > --

Re: Memory consumed by paths during partitionwise join planning

2023-11-28 Thread David Rowley
On Fri, 28 Jul 2023 at 02:06, Ashutosh Bapat wrote: > Table 1: Join between unpartitioned tables > Number of tables | without patch | with patch | % reduction | > being joined ||| | > -- >