Re: [HACKERS] Multi-pass planner

2013-04-19 Thread Claudio Freire
On Fri, Apr 19, 2013 at 7:43 PM, Jeff Janes wrote: > On Fri, Apr 19, 2013 at 2:24 PM, Claudio Freire > wrote: >> >> >> Especially if there's some locality of occurrence, since analyze >> samples pages, not rows. > > > But it doesn't take all rows in each sampled page. It generally takes about >

Re: [HACKERS] Multi-pass planner

2013-04-19 Thread Jeff Janes
On Fri, Apr 19, 2013 at 2:24 PM, Claudio Freire wrote: > > Especially if there's some locality of occurrence, since analyze > samples pages, not rows. > But it doesn't take all rows in each sampled page. It generally takes about one row per page, specifically to avoid the problem you indicate. M

Re: [HACKERS] Multi-pass planner

2013-04-19 Thread Jeff Janes
On Thu, Apr 4, 2013 at 11:53 AM, Dimitri Fontaine wrote: > Robert Haas writes: > > for estimate_worstcase_fraction. So, when computing the cost of a > > path, we'd compute our current expected-case estimate, and also a > > worst-case estimate, and then compute the final cost as: > > There also w

Re: [HACKERS] Multi-pass planner

2013-04-19 Thread Claudio Freire
On Fri, Apr 19, 2013 at 6:19 PM, Jeff Janes wrote: > On Wed, Apr 3, 2013 at 6:40 PM, Greg Stark wrote: >> >> >> On Fri, Aug 21, 2009 at 6:54 PM, decibel wrote: >>> >>> Would it? Risk seems like it would just be something along the lines of >>> the high-end of our estimate. I don't think confiden

Re: [HACKERS] Multi-pass planner

2013-04-19 Thread Jeff Janes
On Wed, Apr 3, 2013 at 6:40 PM, Greg Stark wrote: > > On Fri, Aug 21, 2009 at 6:54 PM, decibel wrote: > >> Would it? Risk seems like it would just be something along the lines of >> the high-end of our estimate. I don't think confidence should be that hard >> either. IE: hard-coded guesses have

Re: [HACKERS] Multi-pass planner

2013-04-04 Thread Amit Kapila
On Friday, April 05, 2013 1:59 AM Robert Haas wrote: > On Thu, Apr 4, 2013 at 2:53 PM, Dimitri Fontaine > wrote: > > Robert Haas writes: > >> for estimate_worstcase_fraction. So, when computing the cost of a > >> path, we'd compute our current expected-case estimate, and also a > >> worst-case e

Re: [HACKERS] Multi-pass planner

2013-04-04 Thread Robert Haas
On Thu, Apr 4, 2013 at 2:53 PM, Dimitri Fontaine wrote: > Robert Haas writes: >> for estimate_worstcase_fraction. So, when computing the cost of a >> path, we'd compute our current expected-case estimate, and also a >> worst-case estimate, and then compute the final cost as: > > There also was t

Re: [HACKERS] Multi-pass planner

2013-04-04 Thread Dimitri Fontaine
Robert Haas writes: > for estimate_worstcase_fraction. So, when computing the cost of a > path, we'd compute our current expected-case estimate, and also a > worst-case estimate, and then compute the final cost as: There also was the idea for the executor to be able to handle alternate plans and

Re: [HACKERS] Multi-pass planner

2013-04-04 Thread Robert Haas
On Wed, Apr 3, 2013 at 9:40 PM, Greg Stark wrote: > I used to advocate a similar idea. But when questioned on list I tried to > work out the details and ran into some problem coming up with a concrete > plan. > > How do you compare a plan that you think has a 99% chance of running in 1ms > but a 1

Re: [HACKERS] Multi-pass planner

2013-04-03 Thread Greg Stark
On Fri, Aug 21, 2009 at 6:54 PM, decibel wrote: > Would it? Risk seems like it would just be something along the lines of > the high-end of our estimate. I don't think confidence should be that hard > either. IE: hard-coded guesses have a low confidence. Something pulled > right out of most_commo

Re: [HACKERS] Multi-pass planner

2009-08-21 Thread decibel
On Aug 20, 2009, at 11:18 PM, Josh Berkus wrote: I don't think it's a bad idea, I just think you have to set your expectations pretty low. If the estimates are bad there isn't really any plan that will be guaranteed to run quickly. Well, the way to do this is via a risk-confidence system. That

Re: [HACKERS] Multi-pass planner

2009-08-20 Thread Josh Berkus
> I don't think it's a bad idea, I just think you have to set your > expectations pretty low. If the estimates are bad there isn't really > any plan that will be guaranteed to run quickly. Well, the way to do this is via a risk-confidence system. That is, each operation has a level of risk assig

Re: [HACKERS] Multi-pass planner

2009-08-20 Thread Mischa Sandberg
But the concept was pretty simple and as described above. > -Original Message- > From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Greg Stark > Sent: Thursday, August 20, 2009 10:32 AM > To: Robert Haas > Cc: Kevin Gritt

Re: [HACKERS] Multi-pass planner

2009-08-20 Thread Kevin Grittner
Greg Stark wrote: > Say you're deciding between an index scan and a sequential scan. The > sequential scan has a total cost of 1000..1000 but the index scan > has an estimated total cost of 1..1. My proposal was to use RMS, which would effectively favor lower worst case behavior. Specific

Re: [HACKERS] Multi-pass planner

2009-08-20 Thread Greg Stark
On Thu, Aug 20, 2009 at 6:28 PM, Greg Stark wrote: > I don't think it's a bad idea, I just think you have to set your > expectations pretty low. If the estimates are bad there isn't really > any plan that will be guaranteed to run quickly. Actually this is usually Tom's point when this topic comes

Re: [HACKERS] Multi-pass planner

2009-08-20 Thread Greg Stark
On Thu, Aug 20, 2009 at 6:10 PM, Robert Haas wrote: > Maybe.  The problem is that we have mostly two cases: an estimate that > we think is pretty good based on reasonable statistics (but may be way > off if there are hidden correlations we don't know about), and a wild > guess.  Also, it doesn't te

Re: [HACKERS] Multi-pass planner

2009-08-20 Thread Robert Haas
On Thu, Aug 20, 2009 at 12:55 PM, Kevin Grittner wrote: > Robert Haas wrote: > >> I think one of the problems with the planner is that all decisions >> are made on the basis of cost.  Honestly, it works amazingly well in >> a wide variety of situations, but it can't handle things like "we >> might

Re: [HACKERS] Multi-pass planner

2009-08-20 Thread Kevin Grittner
Robert Haas wrote: > I think one of the problems with the planner is that all decisions > are made on the basis of cost. Honestly, it works amazingly well in > a wide variety of situations, but it can't handle things like "we > might as well materialize here, because it doesn't cost much and >

Re: [HACKERS] Multi-pass planner

2009-08-20 Thread Robert Haas
On Thu, Aug 20, 2009 at 11:15 AM, decibel wrote: > There have been a number of planner improvement ideas that have been thrown > out because of the overhead they would add to the planning process, > specifically for queries that would otherwise be quiet fast. Other databases > seem to have dealt wi

[HACKERS] Multi-pass planner

2009-08-20 Thread decibel
There have been a number of planner improvement ideas that have been thrown out because of the overhead they would add to the planning process, specifically for queries that would otherwise be quiet fast. Other databases seem to have dealt with this by creating plan caches (which might be w