> Nobody will hit next 499 times, but a lot of our users skip to the last page > quite often. Maybe I should make *that* as hard as possible. Hmm
Right. I'd actually argue that providing a "last page" link in this situation is 1) useless to the user, I mean what's the point? Curiosity? If it really _must_ be supported, Toke's approach is sneaky and elegant. Sort in reverse order and give them the first page ;). 2) dangerous as you well know... > several orders of magnitude larger than what was tested > there, so I'm still a bit worried. I sympathize, but somebody has to be first ;). Besides, the current situation is untenable from what you're saying... Good luck! Erick On Tue, Dec 23, 2014 at 7:07 AM, Toke Eskildsen <t...@statsbiblioteket.dk> wrote: > Bram Van Dam [bram.van...@intix.eu] wrote: > > [Solr cursors] > >> Oh thanks, that's a pretty interesting read. The scale we're >> investigating is several orders of magnitude larger than what was tested >> there, so I'm still a bit worried. > > The beauty of the cursor is that it is has little to no overhead, relative to > a standard top-X sorted search. A standard search uses a sliding window over > the full result set, as does a cursor-search. Same amount of work. It is just > a question of limits for the window. > >> The largest index I currently have access to is >> about a billion documents in size. Paging there is a nightmare, but the >> Solr version is too old to support cursors so I'm afraid I can't offer >> any useful data. > > Non-cursor paging in Solr uses a sliding window sort with a heap that > contains all documents up to the paging number. A heap is a very fine thing > for sliding window sort, as long as it is small. But performance drops to > horrible levels when it gets large as it is extremely RAM-cache unfriendly. > >> Does anyone have any performance data on multi-billion-document indexes? > > Sorry, no. I could do a test on our 7 billion documents index, but it would > have to wait until the end of January. > >>Nobody will hit next 499 times, but a lot of our users skip to the last >> page quite often. Maybe I should make *that* as hard as possible. Hmm. > > Issue a search with sort in reverse order, then reverse the returned list of > documents? > > - Toke Eskildsen
