Re: Bitmap scan is undercosted?

2017-12-03 Thread Vitaliy Garnashevich
On 02/12/2017 23:17, Jeff Janes wrote: Right, so there is a cpu costing problem (which could only be fixed by hacking postgresql and recompiling it), but it is much smaller of a problem than the IO cost not being accurate due to the high hit rate. Fixing the CPU costing problem is unlikely to m

Re: Bitmap scan is undercosted?

2017-12-03 Thread Vitaliy Garnashevich
On 03/12/2017 01:44, Tom Lane wrote: I think it'd be a serious error to screw around with your cost settings on the basis of a single case in which the rowcount estimates are so far off. It's really those estimates that are the problem AFAICS. The core issue in this example is that, the way the

Re: Bitmap scan is undercosted?

2017-12-03 Thread Vitaliy Garnashevich
On 03/12/2017 03:27, Jeff Janes wrote: Due to that, when I disable bitmapscans and seqscans, I start getting slow index scans on the wrong index, i2 rather than i1.  I don't know why he doesn't see that in his example. When I increase effective_cache_size to 1024MB, I start getting the plan wit

Re: Bitmap scan is undercosted? - boolean correlation

2017-12-03 Thread Jeff Janes
On Sat, Dec 2, 2017 at 8:04 PM, Justin Pryzby wrote: > On Sat, Dec 02, 2017 at 05:27:51PM -0800, Jeff Janes wrote: > > I think the non-extended stats code also has trouble with booleans. > > pg_stats gives me a correlation of 0.8 or higher for the flag column. > > It's not due to the boolean tho

Re: Bitmap scan is undercosted? - boolean correlation

2017-12-03 Thread Tom Lane
Jeff Janes writes: > On Sat, Dec 2, 2017 at 8:04 PM, Justin Pryzby wrote: >> It thinks there's somewhat-high correlation since it gets a list of x >> and y values (integer positions by logical and physical sort order) and >> 90% of the x list (logical value) are the same value ('t'), and the >> C

Re: Bitmap scan is undercosted? - boolean correlation

2017-12-03 Thread Jeff Janes
On Dec 3, 2017 15:31, "Tom Lane" wrote: Jeff Janes writes: > On Sat, Dec 2, 2017 at 8:04 PM, Justin Pryzby wrote: >> It thinks there's somewhat-high correlation since it gets a list of x >> and y values (integer positions by logical and physical sort order) and >> 90% of the x list (logical val

Re: Bitmap scan is undercosted?

2017-12-03 Thread Tom Lane
I wrote: > I tried creating multiple-column statistics using the v10 facility for > that: > regression=# create statistics s1 on num, flag from aaa; > CREATE STATISTICS > regression=# analyze aaa; > ANALYZE > but that changed the estimate not at all, which surprised me because > dependency statisti