I should add in my $0.02 on whether to just get rid of queryNorm() altogether:
-1 from me, even though it's confusing, because having that call there (somewhere, at least) allows you to actually do compare scores across queries if you do the extra work of properly normalizing the documents as well (at index time). And for people who actually do machine-learning training of their per-field query boosts, this is pretty critical. -jake On Fri, Nov 20, 2009 at 8:15 AM, Jake Mannix <jake.man...@gmail.com> wrote: > The fact Lucene Similarity is most decidely *not* cosine similarity, but > strongly resembles it with the queryNorm() in there, means that I personally > would certainly like to see this called out, at least in the documentation. > > As for performance, is the queryNorm() called ever in any loops? It's all > set up in the construction of the Weight, right? Which means that by the > time you're doing scoring, all the weighting factors are already factored > into one? What's the performance issue which would be saved here? > > -jake > > > On Fri, Nov 20, 2009 at 7:56 AM, Grant Ingersoll <gsing...@apache.org>wrote: > >> For a long time now, we've been telling people not to compare scores >> across queries, yet we maintain the queryNorm() code as an attempt to do >> this and the javadocs even promote it. I'm in the process of researching >> this some more (references welcomed), but wanted to hear what people think >> about it here. I haven't profiled it just yet, but it seems like a good >> chunk of wasted computation to me (loops, divisions and square roots). At a >> minimum, I think we might be able to refactor the callback mechanism for it >> just as we did for the collectors, such that we push of the actual >> calculation of the sum of squares into Similarity, instead of just doing >> 1/sqrt(sumSqs). That way, when people want to override queryNorm() to >> return 1, they are saving more than just the 1/sqrt calculation. I haven't >> tested it yet, but wanted to find out what others think. >> >> Thoughts? >> >> -Grant >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org >> For additional commands, e-mail: java-dev-h...@lucene.apache.org >> >> >
