Emre Hasegeli e...@hasegeli.com writes:
[ inet-selfuncs-v14.patch ]
After further reflection I concluded that the best way to deal with the
O(N^2) runtime problem for the join selectivity function was to set a
limit on the number of statistics values we'd consider, as was discussed
awhile back
New version of the patch attached with the optimization to break the
loop before looking at all of the histogram values. I can reduce
join selectivity estimation runtime by reducing the values of the
left hand side or both of the sides, if there is interest.
Even if the above aspects of the
On Wed, Dec 3, 2014 at 6:14 AM, Emre Hasegeli e...@hasegeli.com wrote:
Actually, there's a second large problem with this patch: blindly
iterating through all combinations of MCV and histogram entries makes the
runtime O(N^2) in the statistics target. I made up some test data (by
scanning my
I spent a fair chunk of the weekend hacking on this patch to make
it more understandable and fix up a lot of what seemed to me pretty
clear arithmetic errors in the upper layers of the patch. However,
I couldn't quite convince myself to commit it, because the business
around estimation for
Actually, there's a second large problem with this patch: blindly
iterating through all combinations of MCV and histogram entries makes the
runtime O(N^2) in the statistics target. I made up some test data (by
scanning my mail logs) and observed the following planning times, as
reported by
I wrote:
I spent a fair chunk of the weekend hacking on this patch to make
it more understandable and fix up a lot of what seemed to me pretty
clear arithmetic errors in the upper layers of the patch. However,
I couldn't quite convince myself to commit it, because the business
around
Emre Hasegeli e...@hasegeli.com writes:
Thanks. Overall, my impression of this patch is that it works very
well. But damned if I understood *how* it works :-). There's a lot
of statistics involved, and it's not easy to see why something is
multiplied by something else. I'm adding comments as I
Thanks. Overall, my impression of this patch is that it works very
well. But damned if I understood *how* it works :-). There's a lot
of statistics involved, and it's not easy to see why something is
multiplied by something else. I'm adding comments as I read through
it.
Thank you for
On 09/07/2014 07:09 PM, Emre Hasegeli wrote:
I updated the patch to cover semi and anti joins with eqjoinsel_semi().
I think it is better than returning a constant.
What you did there is utterly unacceptable from a modularity standpoint;
and considering that the values will be nowhere near
New version with semi join estimation function attached.
I have performed the following initial review:
- Patch format. -- submitted as unified, but not sure it makes it any
easier to read than context format.
- Apply to current master (77e65bf). -- success (though, I do get
Stripping
I updated the patch to cover semi and anti joins with eqjoinsel_semi().
I think it is better than returning a constant.
What you did there is utterly unacceptable from a modularity standpoint;
and considering that the values will be nowhere near right, the argument
that it's better
* Isn't X Y equivalent to network_scan_first(X) Y AND
network_scan_last(X) Y? Or at least close enough for selectivity
estimation purposes? Pardon my ignorance - I'm not too familiar with the
inet datatype - but how about just calling scalarineqsel for both bounds?
Actually, X Y is
Heikki Linnakangas hlinnakan...@vmware.com writes:
* inet_mcv_join_selec() is O(n^2) where n is the number of entries in
the MCV lists. With the max statistics target of 1, a worst case
query on my laptop took about 15 seconds to plan. Maybe that's
acceptable, but you went through
What you did there is utterly unacceptable from a modularity standpoint;
and considering that the values will be nowhere near right, the argument
that it's better than returning a constant seems pretty weak. I think
you should just take that out again.
I will try to come up with a better,
Emre Hasegeli e...@hasegeli.com writes:
I updated the patch to cover semi and anti joins with eqjoinsel_semi().
I think it is better than returning a constant.
What you did there is utterly unacceptable from a modularity standpoint;
and considering that the values will be nowhere near right,
Heikki Linnakangas hlinnakan...@vmware.com writes:
* inet_mcv_join_selec() is O(n^2) where n is the number of entries in
the MCV lists. With the max statistics target of 1, a worst case
query on my laptop took about 15 seconds to plan. Maybe that's
acceptable, but you went through some
On 08/26/2014 12:44 PM, Emre Hasegeli wrote:
I agree with you that we can support other join type and anti join later,
If others don’t have any objection in doing other parts later I will mark as Ready
For Committer.
I updated the patch to cover semi and anti joins with eqjoinsel_semi().
I
I agree with you that we can support other join type and anti join later,
If others don’t have any objection in doing other parts later I will mark as
Ready For Committer.
I updated the patch to cover semi and anti joins with eqjoinsel_semi().
I think it is better than returning a constant.
On 12 July 2014 23:25, Emre Hasegeli Wrote,
I have one last comment, after clarifying this I can move it to
ready for committer.
1. In networkjoinsel, For avoiding the case of huge statistics, only
some of the values from mcv and histograms are used (calculated using
SQRT).
-- But in my
I have one last comment, after clarifying this I can move it to ready for
committer.
1. In networkjoinsel, For avoiding the case of huge statistics, only some of
the values from mcv and histograms are used (calculated using SQRT).
-- But in my opinion, if histograms and mcv both are exist
On 06 July 2014 20:33, Emre Hasegeli Wrote,
Apart from that there is one spell check you can correct
-- in inet_his_inclusion_selec comments histogram boundies -
histogram boundaries :)
I fixed it. New version attached. The debug log statements are also
removed.
I have done with
Thank you for looking at it.
In inet_his_inclusion_selec function,
When the constant matches only the right side of the bucket, and if it’s a
last bucket then it's never considered as partial match candidate.
In my opinion, if it's not a last bucket then for next bucket it will become
On, 15 May 2014 14:04 Emre Hasegeli Wrote,
* matching first MCV to second MCV
* searching first MCV in the second histogram
* searching second MCV in the first histogram
* searching boundaries of the first histogram in the second histogram
Comparing the lists with each other slows down
I wanted to check the patch last time and found a bug effecting
MVC vs MVC part of the join selectivity. Fixed version attached.
Emre Hasegeli e...@hasegeli.com:
Comparing the lists with each other slows down the function when
statistics set to higher values. To avoid this problem I only use
New version of the selectivity estimation patch attached. I am adding
it to CommitFest 2014-06. Previous version of it reviewed by
Andreas Karlson on the previous CommitFest with the GiST support patch.
The new version includes join selectivity estimation.
Join selectivity is calculated in 4
25 matches
Mail list logo
