> The nature of the field cache itself means that the first
> sort on a particular field can take a long, long time.
> Synchronization won't really help that much.
I think you may be misunderstanding my description (probably because it was
not clear enough :). The issue is not that the first search is going to take
a while as this is clearly unavoidable. The issue I'm seeing is that when
there are a number of concurrent searches that start executing before the
cache has been populated they *all* end up doing the very expensive
ScoreDocComparator generation rather than just one of them doing the
generation and the rest simply blocking until that one is done. The more
concurrent searches and the longer the generation takes the worse the effect
becomes.
> There are two ways around this...
> 2) warm searchers in the background before exposing them to
> live queries (the approach Solr takes).
This is basically how we are working around this issue, we don't actually
pre-warm the search results as we don't have a window in which to do this
but we do synchronize the FieldSortedHitQueue cache generation so it can
never gets executed more than once per index reader:
private final Set<String> primedSortFields = new HashSet<String>();
protected void primeCache(Sort sort) throws IOException {
// This synchronized block allows us to be sure that a given sort
field is only primed once
// per searcher rather than the multiple times Lucene may prime the
field if left to its
// own devices (something we *really* want to avoid for big
indexes).
synchronized (primedSortFields) {
SortField[] sortFields = sort.getSort();
for (int i = 0; i < sortFields.length; i++) {
SortField sortField = sortFields[i];
if (!primedSortFields.contains(sortField.getField())) {
primedSortFields.add(sortField.getField());
new FieldSortedHitQueue(getIndexReader(), new
SortField[] { sortField }, 0);
}
}
}
}
obviously this not ideal.
Thanks,
Oliver
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
