doing some copying I came up with the following
boolean fsv =
req.getParams().getBool(ResponseBuilder.FIELD_SORT_VALUES,false);
if(fsv){
NamedList sortVals = (NamedList)
rsp.getValues().get("sort_values");
Sort sort = searcher.weightSort(rb.getSortSpec().getSort());
SortField[] sortFields = sort==null ? new
SortField[]{SortField.FIELD_SCORE} : sort.getSort();
for (SortField sortField: sortFields) {
String fieldname = sortField.getField();
ArrayList<Object> list = (ArrayList<Object>)
sortVals.get(fieldname);
for(int index = 0; index < removedDocs.length; index ++)
list.remove(removedDocs[index]);
}
}
this seems to have worked, need to do more testing but I don't
understand why it worked, what exactly is this doing?
On Fri, Feb 10, 2012 at 3:12 PM, Jamie Johnson <jej2...@gmail.com> wrote:
> I'd like to look at the pseudo fields you're talking about (don't
> really understand it right now), but need to get something working in
> the short term. How do I go about removing these from the sort
> values?
>
> On Fri, Feb 10, 2012 at 3:06 PM, Yonik Seeley
> <yo...@lucidimagination.com> wrote:
>> On Fri, Feb 10, 2012 at 2:48 PM, Jamie Johnson <jej2...@gmail.com> wrote:
>>> So looking at query component it appears to sort the entire doc list
>>> at the end of process, my component is defined after this query so the
>>> doclist that I get should be sorted, right? To me this should mean
>>> that I can remove items from this list and shift everything left as
>>> needed and it should work fine, but this isn't what appears to be
>>> happening. For queries that are not distributed I don't see this
>>> issue, only for distributed queries.
>>
>> The document lists from the shards are merged by looking at the sort values.
>> Those are looked up by position in a different part of the response
>> (generated by fsv=true).
>> If you just mess with the doclists, those sort values will no longer
>> "line up" (doc #5 won't correspond to fsv slot #5).
>>
>> Short solution: if you remove a doc, remove that slot from all of the
>> sort values
>>
>> Better solution: We have pseudo-fields now... we should add sort
>> values directly to the documents so this type of parallel structure is
>> no longer needed.
>>
>> -Yonik
>> lucidimagination.com
