Thanks Jon I'll investigate your idea further.
It would be nice if, in future, the Lucene API could provide a searchAfter that takes a position (int).
Regards Jamie On 2014/06/03, 3:24 PM, Jon Stewart wrote:
With regards to pagination, is there a way for you to cache the IndexSearcher, Query, and TopDocs between user pagination requests (a lot of webapp frameworks have object caching mechanisms)? If so, you may have luck with code like this: void ensureTopDocs(final int rank) throws IOException { if (StartDocIndex > rank) { Docs = Searcher.search(SearchQuery, TOP_DOCS_WINDOW); StartDocIndex = 0; } int len = Docs.scoreDocs.length; while (StartDocIndex + len <= rank) { StartDocIndex += len; Docs = Searcher.searchAfter(Docs.scoreDocs[len - 1], SearchQuery, TOP_DOCS_WINDOW); len = Docs.scoreDocs.length; } } StartDocIndex is a member variable denoting the current rank of the first item in TopDocs ("Docs") window. I call this function before each Document retrieval. The common case--of the user looking at the first page of results or the user advancing to the next page--is quite fast. But it still supports random access, albeit not in constant time. OTOH, if your app is concurrent, most search queries will probably be returned very quickly so the odd query that wants to jump deep into the result set will have more of the server's resources available to it. Also, given the size of your result sets, you have to allocate a lot of memory upfront which will then get gc'd after some time. From query to query, you will have a decent amount of memory churn. This isn't free. My guess is using Lucene's linear (search() & searchAfter()) pagination will perform faster than your current approach just based upon not having to create such large arrays. I'm not the Lucene expert that Robert is, but this has worked alright for me.
--------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org