What you are describing is very similar to collapse (CollapsingQParsePlugin) which needs to keep all the group heads in memory.
Recently block collapse was introduced which means only one group head needs to be in memory at one time when all the documents in the same group are indexed in the same block. I'm not sure if block indexing can be used in your use case or not though as it might be a grouping related feature. Joel Bernstein http://joelsolr.blogspot.com/ On Mon, Jan 24, 2022 at 10:09 AM Dan Rosher <[email protected]> wrote: > Hi, > > We have a use case ( to calc commute time) to run an expensive filter that > sends filtered docs (along with their lat/lon) to a partner SaaS to calc > commute time. > > I was thinking of writing as a PostFilter, but need to first gather all > the docs (and scores) in the DelegatingCollector.collector after filtering, > to the downstream collector in the finish method. > > Is there a less memory hungry method perhaps (as I have to temp' cache the > docset and scores)? > > Many thanks, > Dan >
