[
https://issues.apache.org/jira/browse/LUCENE-2091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12784270#action_12784270
]
Joaquin Perez-Iglesias edited comment on LUCENE-2091 at 12/1/09 4:01 PM:
-------------------------------------------------------------------------
Hi Otis, Robert and Yuval.
I developed this add-on for Lucene in 2008, for some experiments that I was
doing, and I would like to express my impressions about this.
In my experience and after reading lot of papers I have never found a case
where the Lucene-VSM implementation improves BM25 performance.
BM25 (with standard parameters) outperforms Lucene-VSM, moreover a room for
improvement exists if the parameters are fixed specifically for the collection.
I made publish some results with the Eurogov collection some time ago.
I can show you now some experiments with TREC Disk4&5 collection, these results
have been obtained with default parameters with the Robust track topics. As you
can see BM25 improves the Lucene-VSM ranking function.
MAP p...@5
VSM 0.2079 0.4096
BM25 0.2340 0.4578
This implementation is getting more popular and I know that some people is
using it on their research, thus it will be really nice if at some point it is
included in the core.
The only concerns that I have about it, are related with:
- Only simple boolean queries based on terms are supported (with operators
or, and, not). For instance it does not support PhraseQuery.
- IDF cannot be calculated at a document level (this is important for BM25F).
- Another issue is related with computing the document average length, but
this could be easily solved.
These issues are described in detail in the documentation that I made public in
my website.
Thanks to all for your interest and work.
Joaquin Perez-Iglesias
was (Author: joaquin):
Hi Otis, Robert and Yuval.
I developed this add-on for Lucene in 2008, for some experiments that I was
doing, and I would like to express my impressions about this.
In my experience and after reading lot of papers I have never found a case
where the Lucene-VSM implementation improves BM25 performance.
BM25 (with standard parameters) outperforms Lucene-VSM, moreover a room for
improvement exists if the parameters are fixed specifically for the collection.
I made publish some results with the Eurogov collection some time ago.
I can show you now some experiments with TREC Disk4&5 collection, these results
have been obtained with default parameters with the Robust track topics. As you
can see BM25 improves the Lucene-VSM ranking function.
MAP p...@5
VSM 0.2079 0.4096
BM25 0.2340 0.4578
This implementation is getting more popular and I know that some people is
using it on their research, thus it will be really nice if at some point it is
included in the core.
The only concerns that I have about it, are related with:
- Only simple boolean queries based on terms are supported (with operators
or, and, not). For instance it does not support PhraseQuery.
- IDF cannot be calculated at a document level (this is important for BM25F).
- Another issue is related with computing the document average length, but
this could be easily solved.
These issues are described in detail in the documentation that I made public in
my website.
Thanks to all for your interest and work.
Joaquin Perez-Iglesias
> Add BM25 Scoring to Lucene
> --------------------------
>
> Key: LUCENE-2091
> URL: https://issues.apache.org/jira/browse/LUCENE-2091
> Project: Lucene - Java
> Issue Type: New Feature
> Components: contrib/*
> Reporter: Yuval Feinstein
> Priority: Minor
> Fix For: 3.1
>
> Attachments: persianlucene.jpg
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> http://nlp.uned.es/~jperezi/Lucene-BM25/ describes an implementation of
> Okapi-BM25 scoring in the Lucene framework,
> as an alternative to the standard Lucene scoring (which is a version of mixed
> boolean/TFIDF).
> I have refactored this a bit, added unit tests and improved the runtime
> somewhat.
> I would like to contribute the code to Lucene under contrib.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
