On Thu, 2010-07-01 at 16:30 +, Leonardo F wrote:
To sum up: IMHO nor improvements in memory usage nor
in startup time would be good reasons to switch to bitmap
indexes... but bulk index creation time (10 minutes to index
what it takes 60 minutes with btree... and maybe more if tables
Are these improvements (index creation time, index size) worth enough
to keep on working on this?
I mean: given that bitmaps don't give any benefits in query times, but
only benefits related to disk size and bulk index creation times, and
will have horrible performance for
On 02/07/10 13:31, Bruce Momjian wrote:
Leonardo F wrote:
I'm trying to find more docs that explain the improvements of
bitmap indexes in other products... but most of what I've found
talks about bitmapAND/OR which is something that is very
cool, but that postgres already does even with
On 02/07/10 20:30, Mark Kirkwood wrote:
I recall that for (some/most? of) those low cardinality cases, (on
disk) bitmap indexes would perform better too. I think the size saving
alone is a huge win for serious data warehousing situations. On the
other hand problems I recall are possibly
Using as a starting point the old bitmap patch in:
http://archives.postgresql.org/message-id/20081101000154.go27...@fune
I re-applied and re-worked the patch to see what kind of improvements over
btrees bitmaps actually provided.
Using a 20M rows table of 10/100/1000 random values, I've found
Leonardo F wrote:
Using as a starting point the old bitmap patch in:
http://archives.postgresql.org/message-id/20081101000154.go27...@fune
I re-applied and re-worked the patch to see what kind of improvements over
btrees bitmaps actually provided.
Using a 20M rows table of 10/100/1000
On Thu, Jul 1, 2010 at 9:23 AM, Leonardo F m_li...@yahoo.it wrote:
Using as a starting point the old bitmap patch in:
http://archives.postgresql.org/message-id/20081101000154.go27...@fune
I re-applied and re-worked the patch to see what kind of improvements over
btrees bitmaps actually
Robert Haas robertmh...@gmail.com writes:
Hmm... no performance improvement? That's not encouraging.
The index being smaller ought to by itself provide some performance
improvement if, say, the smaller index can fit in cache and the larger
one can't. With a 6-15x size difference, that's
On Thu, Jul 1, 2010 at 11:21 AM, Tom Lane t...@sss.pgh.pa.us wrote:
In particular, I recall some discussions about developing a streaming
API whereby an index AM could return a bitmap page-by-page or so,
rather than having to construct the whole thing in-memory before
anything could happen.
In
principle a bitmap index scan should be significantly faster if the
index can
return the bitmap more or less natively rather than having
to construct
it.
The problem I'm seeing is that even on a 20M rows table, doing a
select * from t where c1=10 and c2=1
where c1 and c2 are low
Leonardo F wrote:
I'm trying to find more docs that explain the improvements of
bitmap indexes in other products... but most of what I've found
talks about bitmapAND/OR which is something that is very
cool, but that postgres already does even with btree indexes...
or index creation
Simon Riggs [EMAIL PROTECTED] writes:
On Mon, 2008-11-03 at 23:28 +, Simon Riggs wrote:
On Mon, 2008-11-03 at 17:37 -0500, Greg Stark wrote:
There are a lot of comments in the code which imply that vacuuming is
not implemented but in fact from what I can see it is -- sort of. It
On Mon, Nov 03, 2008 at 04:53:28PM -0700, Vladimir Sitnikov wrote:
I wish to focus on the performance aspect of the patch, however, it turned
out there are major issues with functionality: the index stores wrong tids
inside :(
I really would love to fix that issue and have a chance to validate
On 2008-10-31, Gianni Ciolli [EMAIL PROTECTED] wrote:
following the useful feedback that we received from this list, we
would like to submit the patch for Bitmap Indexes for the november
CommitFest (joint work of me with Gabriele Bartolini, starting from
Gavin Sherry's patch).
I skimmed
On Mon, 2008-11-03 at 17:37 -0500, Greg Stark wrote:
Secondly the locking seems to be a bit overoptimistic. I'm pretty sure
you have to take an exclusive lock on an index page any time you make
any data modifications in index pages -- even if you're just setting a
bit and not moving any data
On Mon, 2008-11-03 at 23:28 +, Simon Riggs wrote:
On Mon, 2008-11-03 at 17:37 -0500, Greg Stark wrote:
Secondly the locking seems to be a bit overoptimistic. I'm pretty sure
you have to take an exclusive lock on an index page any time you make
any data modifications in index pages --
I looked at that aspect of the patch specifically a few weeks back while
checking for possible issues with Hot Standby. IIRC the patch is fairly
careful with locking and uses Exclusive locks extensively throughout. I
looked at both the theory and the implementation. Unless Gianni changed
On Mon, 2008-11-03 at 16:53 -0700, Vladimir Sitnikov wrote:
The major thing there is to get the modifications right. There is no
much sense in reviewing wrong code against locking issues.
I didn't say there were no other bugs, nor would I know, only that I had
reviewed the locking issues
BTW: is there a framework to test recovery related features?
The only idea I could take from the top of my head is to comment out all the
page writes and leave only WAL logging. Then crash database at random and
verify if the index still performs as expected.
Regards,
Vladimir Sitnikov
On Mon, 2008-11-03 at 23:28 +, Simon Riggs wrote:
On Mon, 2008-11-03 at 17:37 -0500, Greg Stark wrote:
There are a lot of comments in the code which imply that vacuuming is
not implemented but in fact from what I can see it is -- sort of. It
does rewrite the bitmap in bmbulkdelete
On Wed, 2008-10-22 at 00:00 +0100, Gregory Stark wrote:
Actually as I recall the immediate issue was that the patch was more complex
than necessary. In particular it reimplemented parts of the executor
internally rather than figuring out what api was necessary to integrate it
fully into the
On Tue, 2008-10-21 at 14:26 -0700, Josh Berkus wrote:
Gianni,
me and Gabriele Bartolini have been working on Bitmap Indexes (BMI) in
the last weeks, with advice and guidance from Simon Riggs. We feel
that we are about to approach the point where it is appropriate to ask
for feedback
Josh Berkus wrote:
The other major issue with the Bitmap index patch as it stood in 2007 was
that performance just wasn't that much faster than a btree, except for
specific corner cases. Otherwise, someone else would have been interested
enough to pick it up and finish it.
So performance
Simon Riggs [EMAIL PROTECTED] writes:
On Wed, 2008-10-22 at 00:00 +0100, Gregory Stark wrote:
When we last left our heros they were proposing ways to refactor the index
api
to allow index ams to stream results to the executor in bitmap form.
The indexAM API has now been changed, so that is
On Wed, 2008-10-22 at 08:31 -0400, Tom Lane wrote:
Simon Riggs [EMAIL PROTECTED] writes:
On Wed, 2008-10-22 at 00:00 +0100, Gregory Stark wrote:
When we last left our heros they were proposing ways to refactor the index
api
to allow index ams to stream results to the executor in bitmap
Simon Riggs [EMAIL PROTECTED] writes:
On Wed, 2008-10-22 at 08:31 -0400, Tom Lane wrote:
No, that was merely one component of the problem. The APIs for
tidbitmaps need revision too. You can't stream a bitmap yet.
Please explain further.
Which calls? Why do we need to stream them?
Well, I
On Wed, 2008-10-22 at 09:13 -0400, Tom Lane wrote:
Simon Riggs [EMAIL PROTECTED] writes:
On Wed, 2008-10-22 at 08:31 -0400, Tom Lane wrote:
No, that was merely one component of the problem. The APIs for
tidbitmaps need revision too. You can't stream a bitmap yet.
Please explain
Simon Riggs [EMAIL PROTECTED] writes:
I agree we could do the stream bitmap as well, but I'd suggest we should
save that until the rest of the patch has been checked out.
Well, that would be a reasonable implementation path, but it's got
nothing to do with the patch as presented. The concern
On Wed, 2008-10-22 at 10:00 -0400, Tom Lane wrote:
Simon Riggs [EMAIL PROTECTED] writes:
I agree we could do the stream bitmap as well, but I'd suggest we should
save that until the rest of the patch has been checked out.
Well, that would be a reasonable implementation path, but it's got
Hi everybody,
me and Gabriele Bartolini have been working on Bitmap Indexes (BMI) in
the last weeks, with advice and guidance from Simon Riggs. We feel
that we are about to approach the point where it is appropriate to ask
for feedback from this list.
Thank you,
Dr. Gianni Ciolli - 2ndQuadrant
Gianni,
me and Gabriele Bartolini have been working on Bitmap Indexes (BMI) in
the last weeks, with advice and guidance from Simon Riggs. We feel
that we are about to approach the point where it is appropriate to ask
for feedback from this list.
The other major issue with the Bitmap index
Josh Berkus [EMAIL PROTECTED] writes:
Gianni,
me and Gabriele Bartolini have been working on Bitmap Indexes (BMI) in
the last weeks, with advice and guidance from Simon Riggs. We feel
that we are about to approach the point where it is appropriate to ask
for feedback from this list.
The
Gregory Stark [EMAIL PROTECTED] writes:
Josh Berkus [EMAIL PROTECTED] writes:
The other major issue with the Bitmap index patch as it stood in 2007 was
that performance just wasn't that much faster than a btree, except for
specific corner cases. Otherwise, someone else would have been
Greg,
if you're still in bitmap indices you may take a look on our
contrib/intarray module.
Regards,
Oleg
On 13 Mar 2002, Greg Copeland wrote:
One of the reasons why I originally stated following the hackers list is
because I wanted to implement bitmap indexes. I
On Tue, 19 Mar 2002, Oleg Bartunov wrote:
Sorry to reply over you, Oleg.
On 13 Mar 2002, Greg Copeland wrote:
One of the reasons why I originally stated following the hackers list is
because I wanted to implement bitmap indexes. I found in the archives,
the follow link,
On Tue, 2002-03-19 at 15:30, Matthew Kirkwood wrote:
On Tue, 19 Mar 2002, Oleg Bartunov wrote:
Sorry to reply over you, Oleg.
On 13 Mar 2002, Greg Copeland wrote:
One of the reasons why I originally stated following the hackers list is
because I wanted to implement bitmap indexes.
36 matches
Mail list logo
