Re: docid set compression and boolean docid set operations

2008-09-16 Thread John Wang
Paul and Eks: Because this is being hosted on code.google.com, it requires proj. members to have gmail accts. Can you guys send me yours? Also, we are developing on the BR_DEV_1_0_4branch. Thanks -John On Mon, Sep 15, 2008 at

Re: docid set compression and boolean docid set operations

2008-09-13 Thread Paul Elschot
Op Saturday 13 September 2008 09:21:21 schreef Anmol Bhasin: > Hi, > > Michael : > > True, the table is a placeholder right now. I will run my performance > tests and update the table in the next day or two. > > Paul : > > Thanks for skimming over the code. As John mentioned in his email, > we c

Re: docid set compression and boolean docid set operations

2008-09-13 Thread eks dev
gt; > To: java-dev@lucene.apache.org > Sent: Saturday, 13 September, 2008 9:21:21 > Subject: Re: docid set compression and boolean docid set operations > > Hi, > > Michael : > > True, the table is a placeholder right now. I will run my performance > tests and update the

Re: docid set compression and boolean docid set operations

2008-09-13 Thread Anmol Bhasin
Hi, Michael : True, the table is a placeholder right now. I will run my performance tests and update the table in the next day or two. Paul : Thanks for skimming over the code. As John mentioned in his email, we currently use Kamikaze for in memory caching for document hits. The Kamikaze proj

Re: docid set compression and boolean docid set operations

2008-09-11 Thread John Wang
Hi guys: I will let the author, Anmol Bhasin to respond with details. In our use case, we are not making changes to the index because we do not want to diverge from the lucene code base. (thought it'd be great if we can enhance indexing structure with this) We load the docIdSets into memo

Re: docid set compression and boolean docid set operations

2008-09-11 Thread Paul Elschot
John, I've taken a first look at the code, and I have a few questions. Did I understand correctly that it is basically a two way conversion between an integer array and an (Open)BitSet representing a p4delta data structure? In that case it would still be necessary to extend the lucene index stru

Re: docid set compression and boolean docid set operations

2008-09-11 Thread Michael McCandless
Hi John, I would love to see this added to Lucene! Do you have actual performance numbers? (Looks like the table in that link below isn't "real"?). Mike John Wang wrote: Sorry, I meant lucene 2.4 -John On Wed, Sep 10, 2008 at 2:08 PM, John Wang <[EMAIL PROTECTED]> wrote: Hi guys:

Re: docid set compression and boolean docid set operations

2008-09-10 Thread John Wang
Sorry, I meant lucene 2.4 -John On Wed, Sep 10, 2008 at 2:08 PM, John Wang <[EMAIL PROTECTED]> wrote: > Hi guys: > > We have build this on top of the lucene 1.4. api/refactoring for docid > sets and docIdIterater. > > We've implemented the p4Delta compression algorithm presented at > w