You could switch to DocValues, and it would probably be more efficient if you are only retrieving a single stored field but you have a lot of other ones in the index since stored fields are stored together and have to be decoded together. As far as visiting every segment on disk I'm not sure what you mean -- if you mean Lucene segments this isn't really germane since both DocValues and StoredFields are stored in segments and the segment visitation pattern is going to be the same for both. Finally re: caching I wouldn't recommend it but the approach you describe could be made to work. To be efficient about it you'd want to have a per-segment cache, but this gets into poking around in Lucene internals - you might as well just use its data structures; they should be pretty efficient and a cache wouldn't likely win you very much and just lead to trouble,
On Mon, Aug 5, 2024 at 12:08 PM Marc Davenport <madavenp...@cargurus.com.invalid> wrote: > > Hello, > Right now our implementation retrieves our UID for our records from the > topdocs by calling IndexSearcher.doc(docid, fieldToLoad) (Deprecated) with > the UID as the only field. I'm looking to replace this with the > appropriate call to IndexSearcher.storedFields(). This feels a little > inefficient since it could be visiting each segment on disk. Some of our > searches are for an abusive 10k results (I would love to change this, but > that's the system as it exists). If I only want a single field, is there > a better way to retrieve these? Should I be retrieving the numeric docvals > and advancing the iterator? I've been looking around the internet for > different strategies and I'm kind of muddled with older blog posts vs the > current state of the codebase. Can I cache the relationship between the > docId and the UID as long as I clear it whenever I commit? > Thank you, > Marc --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
