Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-06-17 Thread Tomas Vondra
Hi, On 05/13/15 23:07, Jeff Janes wrote: With the warning it is very hard to correlate the discrepancy you do see with which column is causing it, as the warnings don't include table or column names (Assuming of course that you run it on a substantial database--if you just run it on a few toy

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-05-15 Thread Josh Berkus
On 05/15/2015 11:30 AM, Robert Haas wrote: Once we enter beta (or even feature freeze), it's too late to whack around the algorithm heavily. We're pretty much committed to releasing and supporting whatever we have got at that point. I guess we could revert it if it doesn't work out, but

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-05-15 Thread Robert Haas
On Wed, May 13, 2015 at 5:07 PM, Jeff Janes jeff.ja...@gmail.com wrote: With the warning it is very hard to correlate the discrepancy you do see with which column is causing it, as the warnings don't include table or column names (Assuming of course that you run it on a substantial

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-05-15 Thread Robert Haas
On Fri, May 15, 2015 at 3:35 PM, Josh Berkus j...@agliodbs.com wrote: On 05/15/2015 11:30 AM, Robert Haas wrote: Once we enter beta (or even feature freeze), it's too late to whack around the algorithm heavily. We're pretty much committed to releasing and supporting whatever we have got at

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-05-15 Thread Josh Berkus
On 05/15/2015 12:58 PM, Robert Haas wrote: On Fri, May 15, 2015 at 3:35 PM, Josh Berkus j...@agliodbs.com wrote: On 05/15/2015 11:30 AM, Robert Haas wrote: Once we enter beta (or even feature freeze), it's too late to whack around the algorithm heavily. We're pretty much committed to

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-05-13 Thread Jeff Janes
On Thu, Apr 30, 2015 at 6:20 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: Hi, On 04/30/15 22:57, Robert Haas wrote: On Tue, Mar 31, 2015 at 3:02 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: attached is v4 of the patch implementing adaptive ndistinct estimator. So, I

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-05-01 Thread Robert Haas
On Thu, Apr 30, 2015 at 9:20 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: I agree that this is not ready for 9.5 - it was meant as an experiment (hence printing the estimate in a WARNING, to make it easier to compare the value to the current estimator). Without that it'd be much more

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-04-30 Thread Robert Haas
On Tue, Mar 31, 2015 at 3:02 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: attached is v4 of the patch implementing adaptive ndistinct estimator. So, I took a look at this today. It's interesting work, but it looks more like a research project than something we can commit to 9.5. As far

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-04-30 Thread Tomas Vondra
Hi, On 04/30/15 22:57, Robert Haas wrote: On Tue, Mar 31, 2015 at 3:02 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: attached is v4 of the patch implementing adaptive ndistinct estimator. So, I took a look at this today. It's interesting work, but it looks more like a research project

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-04-30 Thread Tomas Vondra
On 05/01/15 00:18, Robert Haas wrote: On Thu, Apr 30, 2015 at 5:31 PM, Heikki Linnakangas hlinn...@iki.fi wrote: You can override the ndistinct estimate with ALTER TABLE. I think that's enough for an escape hatch. I'm not saying that isn't nice to have, but I don't think it really helps

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-04-30 Thread Robert Haas
On Thu, Apr 30, 2015 at 5:31 PM, Heikki Linnakangas hlinn...@iki.fi wrote: On 04/30/2015 01:57 PM, Robert Haas wrote: 2. There should be a compatibility GUC to restore the old behavior. The new behavior should be the default, because if we're not confident that the new behavior will be better

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-04-30 Thread Heikki Linnakangas
On 04/30/2015 01:57 PM, Robert Haas wrote: 2. There should be a compatibility GUC to restore the old behavior. The new behavior should be the default, because if we're not confident that the new behavior will be better for most people, we have no business installing it in the first place (plus

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-04-19 Thread Jeff Janes
On Tue, Apr 14, 2015 at 11:45 PM, Jeff Janes jeff.ja...@gmail.com wrote: On Tue, Mar 31, 2015 at 12:02 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: Hi all, attached is v4 of the patch implementing adaptive ndistinct estimator. Hi Tomas, I have a case here where the adaptive

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-04-15 Thread Jeff Janes
On Tue, Mar 31, 2015 at 12:02 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: Hi all, attached is v4 of the patch implementing adaptive ndistinct estimator. Hi Tomas, I have a case here where the adaptive algorithm underestimates ndistinct by a factor of 7 while the default estimator

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-04-04 Thread Tomas Vondra
Hi, On 04/03/15 15:46, Greg Stark wrote: The simple workaround for this was adding a fallback to GEE when f[1] or f[2] is 0. GEE is another estimator described in the paper, behaving much better in those cases. For completeness, what's the downside in just always using GEE? That's a good

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-04-03 Thread Greg Stark
The simple workaround for this was adding a fallback to GEE when f[1] or f[2] is 0. GEE is another estimator described in the paper, behaving much better in those cases. For completeness, what's the downside in just always using GEE?

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-03-31 Thread Tomas Vondra
Hi all, attached is v4 of the patch implementing adaptive ndistinct estimator. I've been looking into the strange estimates, mentioned on 2014/12/07: values currentadaptive -- 106 99 107 1068 6449190