[
https://issues.apache.org/jira/browse/SOLR-10117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15867396#comment-15867396
]
David Smiley commented on SOLR-10117:
-------------------------------------
Another idea that I'm starting to like even more I think about it is to put
large fields into BinaryDocValues, with compression (either at DocValuesFormat
(codec) layer, or at Solr layer). For very large fields, I think column stored
(hence docValues) actually makes more sense than the stored field codec (row
stored). Then at the Solr layer we add docValues support to TextField (as
BinaryDocValues), and then also at the Solr layer enable
SolrIndexSearcher.doc() to see {{useDocValuesAsStored}} fields thus enabling
highlighting to see it. I wish I had thought of this earlier.
> Big docs and the DocumentCache; umbrella issue
> ----------------------------------------------
>
> Key: SOLR-10117
> URL: https://issues.apache.org/jira/browse/SOLR-10117
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: David Smiley
> Assignee: David Smiley
> Attachments: SOLR_10117_large_fields.patch
>
>
> This is an umbrella issue for improved handling of large documents (large
> stored fields), generally related to the DocumentCache or SolrIndexSearcher's
> doc() methods. Highlighting is affected as it's the primary consumer of this
> data. "Large" here is multi-megabyte, especially tens even hundreds of
> megabytes. We'd like to support such users without forcing them to choose
> between no DocumentCache (bad performance), or having one but hitting OOM due
> to massive Strings winding up in there. I've contemplated this for longer
> than I'd like to admit and it's a complicated issue with difference concerns
> to balance.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
