Re: [HACKERS] bitmap indexes - performance

2010-07-22 Thread Simon Riggs
On Thu, 2010-07-01 at 16:30 +, Leonardo F wrote: To sum up: IMHO nor improvements in memory usage nor in startup time would be good reasons to switch to bitmap indexes... but bulk index creation time (10 minutes to index what it takes 60 minutes with btree... and maybe more if tables

Re: [HACKERS] bitmap indexes - performance

2010-07-07 Thread Josh Berkus
Are these improvements (index creation time, index size) worth enough to keep on working on this? I mean: given that bitmaps don't give any benefits in query times, but only benefits related to disk size and bulk index creation times, and will have horrible performance for

Re: [HACKERS] bitmap indexes - performance

2010-07-02 Thread Mark Kirkwood
On 02/07/10 13:31, Bruce Momjian wrote: Leonardo F wrote: I'm trying to find more docs that explain the improvements of bitmap indexes in other products... but most of what I've found talks about bitmapAND/OR which is something that is very cool, but that postgres already does even with

Re: [HACKERS] bitmap indexes - performance

2010-07-02 Thread Mark Kirkwood
On 02/07/10 20:30, Mark Kirkwood wrote: I recall that for (some/most? of) those low cardinality cases, (on disk) bitmap indexes would perform better too. I think the size saving alone is a huge win for serious data warehousing situations. On the other hand problems I recall are possibly

[HACKERS] bitmap indexes - performance

2010-07-01 Thread Leonardo F
Using as a starting point the old bitmap patch in: http://archives.postgresql.org/message-id/20081101000154.go27...@fune I re-applied and re-worked the patch to see what kind of improvements over btrees bitmaps actually provided. Using a 20M rows table of 10/100/1000 random values, I've found

Re: [HACKERS] bitmap indexes - performance

2010-07-01 Thread Bruce Momjian
Leonardo F wrote: Using as a starting point the old bitmap patch in: http://archives.postgresql.org/message-id/20081101000154.go27...@fune I re-applied and re-worked the patch to see what kind of improvements over btrees bitmaps actually provided. Using a 20M rows table of 10/100/1000

Re: [HACKERS] bitmap indexes - performance

2010-07-01 Thread Robert Haas
On Thu, Jul 1, 2010 at 9:23 AM, Leonardo F m_li...@yahoo.it wrote: Using as a starting point the old bitmap patch in: http://archives.postgresql.org/message-id/20081101000154.go27...@fune I re-applied and re-worked the patch to see what kind of improvements over btrees bitmaps actually

Re: [HACKERS] bitmap indexes - performance

2010-07-01 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Hmm... no performance improvement? That's not encouraging. The index being smaller ought to by itself provide some performance improvement if, say, the smaller index can fit in cache and the larger one can't. With a 6-15x size difference, that's

Re: [HACKERS] bitmap indexes - performance

2010-07-01 Thread Robert Haas
On Thu, Jul 1, 2010 at 11:21 AM, Tom Lane t...@sss.pgh.pa.us wrote: In particular, I recall some discussions about developing a streaming API whereby an index AM could return a bitmap page-by-page or so, rather than having to construct the whole thing in-memory before anything could happen.  

Re: [HACKERS] bitmap indexes - performance

2010-07-01 Thread Leonardo F
In principle a bitmap index scan should be significantly faster if the index can return the bitmap more or less natively rather than having to construct it. The problem I'm seeing is that even on a 20M rows table, doing a select * from t where c1=10 and c2=1 where c1 and c2 are low

Re: [HACKERS] bitmap indexes - performance

2010-07-01 Thread Bruce Momjian
Leonardo F wrote: I'm trying to find more docs that explain the improvements of bitmap indexes in other products... but most of what I've found talks about bitmapAND/OR which is something that is very cool, but that postgres already does even with btree indexes... or index creation