On 9/20/06, Neville Burnell <[EMAIL PROTECTED]> wrote:
> Hi,
>
> I'm confused about managing field boosting ...
>
> I have set the :boost for the :name field in my docs to 10, via :boost
> => 10
>
> Then I performed a search for 'keith' over all fields via with
> *:(keith*), expecting a doc with Keith in the :name field to come out on
> top. But another doc with Keith mentioned in other fields (:comments,
> :address) scored higher.
>
> I viewed the explanation from the searcher, but it wasn't clear to me
> why the boost wasn't pushing the :name = Keith document to the top.
>
> Any help on understanding field boosting and explain would be great.
>
> Regards
>
> Neville
>
> PS, the two explains are:
>
> Doc1:
> 0.3352959 = product of:
> 8.047102 = sum of:
> 4.011141 = weight(comments:<keith|[EMAIL PROTECTED]|keithex> in
> 4697), product of:
> 0.5685414 =
> query_weight(comments:<keith|[EMAIL PROTECTED]|keithex>), product of:
> 28.22057 = idf(comments:<(keithex=1) + ([EMAIL PROTECTED]) +
> (keith=115) = 117>)
> 0.02014635 = query_norm
> 7.055143 = field_weight(comments:<keith|[EMAIL PROTECTED]|keithex>
> in 4697), product of:
> 1.0 = The sum of:
> 1.0 = tf(term_freq(comments:keithex)=1)^1.0
> 28.22057 = idf(comments:<(keithex=1) + ([EMAIL PROTECTED]) +
> (keith=115) = 117>)
> 0.25 = field_norm(field=comments, doc=4697)
> 4.03596 = weight(address:<keith|keithex> in 4697), product of:
> 0.4032613 = query_weight(address:<keith|keithex>), product of:
> 20.0166 = idf(address:<(keithex=1) + (keith=8) = 9>)
> 0.02014635 = query_norm
> 10.0083 = field_weight(address:<keith|keithex> in 4697), product
> of:
> 1.0 = The sum of:
> 1.0 = tf(term_freq(address:keithex)=1)^1.0
> 20.0166 = idf(address:<(keithex=1) + (keith=8) = 9>)
> 0.5 = field_norm(field=address, doc=4697)
> 0.04166667 = coord(2/48)
>
>
> Doc2:
> 0.2977623 = product of:
> 14.29259 = weight(name:<keith> in 31416), product of:
> 0.2028171 = query_weight(name:<keith>), product of:
> 10.06719 = idf(name:<(keith=3) = 3>)
> 0.02014635 = query_norm
> 70.47034 = field_weight(name:<keith> in 31416), product of:
> 1.0 = The sum of:
> 1.0 = tf(term_freq(name:keith)=1)^1.0
> 10.06719 = idf(name:<(keith=3) = 3>)
> 7.0 = field_norm(field=name, doc=31416)
> 0.02083333 = coord(1/48)
Hi Neville,
The field's boost value affects the field_norm value in the
Explanations above. Here is how it is calculated:
field_norm = field_info->boost * doc->boost * field->boost *
(1 / sqrt(field->num_terms)
So as you can see from the Explanations above, field_norm is 7.0 on
the boosted field which is more than 10 times the field_norms on the
other two fields (0.25, 0.5) so at least you can see the boost is
having an effect. The address field probably has a higher field_norm
value than the comments field because the comments field is longer
(see that last part of the field_norm equation). Note that the reason
the boost is 7.0 and not 10.0 is that the field_norm gets stored in a
single byte so there is quite a large loss of precision.
Having said all this, there does seem to be a problem with the
calculations. I don't think I've calculated the idf value correctly
for MultiTermQueries. I've rectified this in subversion so the next
version should give your results in an order that you'd expect.
For information on tf and idf, check out this page:
http://en.wikipedia.org/wiki/Tf-idf
Hope that helps. I'd love to give a better explanation of the scoring
but I don't have time right now.
Cheers,
Dave
_______________________________________________
Ferret-talk mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/ferret-talk
