Re: Should HashSetOp go away

2025-11-02 Thread Tom Lane
David Rowley writes: > On Mon, 3 Nov 2025 at 05:26, Tom Lane wrote: >> v3 attached. > I reviewed the changes. Looks good to me. Thanks again! I'll get that pushed shortly. regards, tom lane

Re: Should HashSetOp go away

2025-11-02 Thread David Rowley
On Mon, 3 Nov 2025 at 05:26, Tom Lane wrote: > v3 attached. I reviewed the changes. Looks good to me. David

Re: Should HashSetOp go away

2025-11-02 Thread Tom Lane
David Rowley writes: > I had a look at the v2 patches. Mostly quibbles, but #4 seems like an > oversight. Thanks for reviewing! > 1) For the following: > +tuples_space = nentries * (MAXALIGN(SizeofMinimalTupleHeader) + > + MAXALIGN(tupleWidth) + > +

Re: Should HashSetOp go away

2025-11-02 Thread David Rowley
On Sat, 1 Nov 2025 at 08:22, Tom Lane wrote: > I wrote: > > Here's a pair of patches to try to do better. The first one > > is concerned with getting more realistic size estimates for > > TupleHashTables in the planner. The second is some mop-up > > that's been pending for a long time in the sam

Re: Should HashSetOp go away

2025-10-31 Thread Tom Lane
I wrote: > Here's a pair of patches to try to do better. The first one > is concerned with getting more realistic size estimates for > TupleHashTables in the planner. The second is some mop-up > that's been pending for a long time in the same area, namely > getting rid of "long int" field types i

Re: Should HashSetOp go away

2025-10-31 Thread Tom Lane
I wrote: > * create_setop_path's required-space estimate of entrysize * numGroups > was rather lame before commit 5dfc19814, and it's even more so > afterwards. It's basically only accounting for the tuples themselves, > and not either the hashtable overhead or the SetOpStatePerGroupData > counter

Re: Should HashSetOp go away

2025-10-26 Thread Tom Lane
David Rowley writes: > If Hash Joins did support IS NOT DISTINCT FROM clauses, then at least > the non-ALL cases could be done with Hash Semi Join and Hash Anti Join > for INTERSECT and EXCEPT, respectively, followed by a HashAgg. I doubt > it would be any faster for the general case, but at least

Re: Should HashSetOp go away

2025-10-26 Thread David Rowley
On Mon, 27 Oct 2025 at 07:00, Tom Lane wrote: > > Jeff Janes writes: > > I was thinking of ways to improve the memory usage (or at least its > > estimation) but decided maybe it would be better if HashSetOp went away > > entirely. As far as I can tell HashSetOp has nothing to recommend it other

Re: Should HashSetOp go away

2025-10-26 Thread Tom Lane
I wrote: > Jeff Janes writes: >> I noticed some changes in this code v18, so wanted to revisit the issue. >> Under commit 27627929528e, it looks like it got 25% more memory efficient, >> but it thinks it got 40% more efficient, so the memory use got better but >> the estimation actually got worse.

Re: Should HashSetOp go away

2025-10-26 Thread Tom Lane
Jeff Janes writes: > I noticed some changes in this code v18, so wanted to revisit the issue. > Under commit 27627929528e, it looks like it got 25% more memory efficient, > but it thinks it got 40% more efficient, so the memory use got better but > the estimation actually got worse. Hmm, so why n

Should HashSetOp go away

2025-10-26 Thread Jeff Janes
Many years ago I ran into some problems doing maintenance tasks checking for short identifiers which existed in one table but not another. It would choose HashSetOp, which was memory inefficient, and it was also unaware of how memory inefficient it was, leading it to blow well past work_mem, by ma