Well, an early patch somewhere was able to run PFor on trunk, but the performance wasn't great because the trunk bulk-read API is a bottleneck (this is why the bulk postings branch was created).
Mike On Wed, Dec 22, 2010 at 9:45 PM, Li Li <fancye...@gmail.com> wrote: > I used the bulkpostings > branch(https://svn.apache.org/repos/asf/lucene/dev/branches/bulkpostings/lucene) > does trunk have PForDelta decoder/encoder ? > > 2010/12/23 Michael McCandless <luc...@mikemccandless.com>: >> Those are nice speedups! >> >> Did you use the 4.0 branch (ie trunk) or the bulkpostings branch for this >> test? >> >> Mike >> >> On Tue, Dec 21, 2010 at 9:59 PM, Li Li <fancye...@gmail.com> wrote: >>> great improvement! >>> I did a test in our data set. doc count is about 2M+ and index size >>> after optimization is about 13.3GB(including fdt) >>> it seems lucene4's index format is better than lucene2.9.3. and PFor >>> give good results. >>> Besides BlockEncoder for frq and pos. is there any other modification >>> for lucene 4? >>> >>> decoder \ avg time single word(ms) and >>> query(ms) or query(ms) >>> VINT in lucene 2.9 11.2 >>> 36.5 38.6 >>> VINT in lucene 4 branch 10.6 >>> 26.5 35.4 >>> PFor in lucene 4 branch 8.1 >>> 22.5 30.7 >>> 2010/12/21 Li Li <fancye...@gmail.com>: >>>>> OK we should have a look at that one still. We need to converge on a >>>>> good default codec for 4.0. Fortunately it's trivial to take any int >>>>> block encoder (fixed or variable block) and make a Lucene codec out of >>>>> it! >>>> >>>> I suggests you not to use this one, I fixed dozens of bugs but it >>>> still failed when with random tests. it's codes is hand coded rather >>>> than generated by program. But we may learn something from it. >>>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>> For additional commands, e-mail: dev-h...@lucene.apache.org >>> >>> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> For additional commands, e-mail: dev-h...@lucene.apache.org >> >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
