[
https://issues.apache.org/jira/browse/SOLR-10117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15860835#comment-15860835
]
David Smiley commented on SOLR-10117:
-------------------------------------
A couple tangential issues worth addressing:
* {{QueryComponent.doPreFetch}} logic should be configurable or perhaps simply
never do it and make the highlighting component smart enough to ensure the
applicable docs are in the cache with fl + hl.fl + id.
* {{UnifiedSolrHighlighter}} loads only the IDs in a way that will likely have
bad cache performance.
* {{SolrIndexSearcher#doc(docId,StoredFieldVisitor)}} doesn't populate the
DocumentCache; it only reads from it if present. It's more work but it'd be
nice if it populated it as well with not only the "needsField(field)" == true
results but perhaps also by detecting "fl".
> Big docs and the DocumentCache; umbrella issue
> ----------------------------------------------
>
> Key: SOLR-10117
> URL: https://issues.apache.org/jira/browse/SOLR-10117
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: David Smiley
> Assignee: David Smiley
>
> This is an umbrella issue for improved handling of large documents (large
> stored fields), generally related to the DocumentCache or SolrIndexSearcher's
> doc() methods. Highlighting is affected as it's the primary consumer of this
> data. "Large" here is multi-megabyte, especially tens even hundreds of
> megabytes. We'd like to support such users without forcing them to choose
> between no DocumentCache (bad performance), or having one but hitting OOM due
> to massive Strings winding up in there. I've contemplated this for longer
> than I'd like to admit and it's a complicated issue with difference concerns
> to balance.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
