Re: [HACKERS] Incorrect assumptions with low LIMITs

2012-03-19 Thread Jeff Davis
On Sat, 2012-03-17 at 12:48 +, Simon Riggs wrote: The problems are as I described them (1) no account made for sparsity, and other factors leading to an overestimate of rows (N) (2) inappropriate assumption of the effect of LIMIT m, which causes a costly SeqScan to appear better than

Re: [HACKERS] Incorrect assumptions with low LIMITs

2012-03-19 Thread Daniel Farina
On Mon, Mar 19, 2012 at 6:19 PM, Jeff Davis pg...@j-davis.com wrote: On Sat, 2012-03-17 at 12:48 +, Simon Riggs wrote: The problems are as I described them (1) no account made for sparsity, and other factors leading to an overestimate of rows (N) (2) inappropriate assumption of the

Re: [HACKERS] Incorrect assumptions with low LIMITs

2012-03-17 Thread Simon Riggs
On Fri, Mar 16, 2012 at 9:11 PM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: 2. We assume that if values do exist that they have rows uniformly distributed across the whole table like rungs on a ladder. Well, yeah.  That's sometimes wrong, but not always.  In

Re: [HACKERS] Incorrect assumptions with low LIMITs

2012-03-17 Thread Greg Stark
On Sat, Mar 17, 2012 at 9:34 AM, Simon Riggs si...@2ndquadrant.com wrote: My wish was to register this as both a common and significant bug, It has definitely come up here before many times. However at root the problem is part of the general class of not understanding how two different columns

Re: [HACKERS] Incorrect assumptions with low LIMITs

2012-03-17 Thread Simon Riggs
On Sat, Mar 17, 2012 at 11:33 AM, Greg Stark st...@mit.edu wrote: On Sat, Mar 17, 2012 at 9:34 AM, Simon Riggs si...@2ndquadrant.com wrote: My wish was to register this as both a common and significant bug, It has definitely come up here before many times. However at root the problem is part

Re: [HACKERS] Incorrect assumptions with low LIMITs

2012-03-16 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: 2. We assume that if values do exist that they have rows uniformly distributed across the whole table like rungs on a ladder. Well, yeah. That's sometimes wrong, but not always. In the absence of evidence to the contrary, I think it's a better

Re: [HACKERS] Incorrect assumptions with low LIMITs

2012-03-16 Thread Jeff Davis
On Fri, 2012-03-16 at 18:25 +, Simon Riggs wrote: Any time we apply a LIMIT clause to a plan with a SeqScan or unqualified IndexScan, we shouldn't assume the scan will do less than say 10% of the table. It might, but its an unsafe assumption because as the selectivity decreases so does the

Re: [HACKERS] Incorrect assumptions with low LIMITs

2012-03-16 Thread Simon Riggs
On Fri, Mar 16, 2012 at 9:39 PM, Jeff Davis pg...@j-davis.com wrote: On Fri, 2012-03-16 at 18:25 +, Simon Riggs wrote: Any time we apply a LIMIT clause to a plan with a SeqScan or unqualified IndexScan, we shouldn't assume the scan will do less than say 10% of the table. It might, but its