[
https://issues.apache.org/jira/browse/SOLR-4589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13606910#comment-13606910
]
Yonik Seeley commented on SOLR-4589:
------------------------------------
I was initially worried about LazyDocument.getField, but I now realize that
LazyDocument is not a "Document" (i.e. it is not what you will ever retrieve
from the document cache). It's more of a LazyFieldSource
I think there may still be an issue with LazyField.realValue though, in that it
may return a partially constructed object (unless realValue is guaranteed to be
effectively immutable). The easiest fix would be to make realValue volatile
(and readers will thus cross a read memory barrier).
{code}
// edge case: if someone asks this LazyDoc for more LazyFields
// after other LazyFields from the same LazyDoc have been
// actuallized, we need to force the doc to be re-fetched
// so the new LazyFields are also populated.
doc = null;
{code}
Does this happen with Solr?
> 4.x + enableLazyFieldLoading + large multivalued fields + varying fl =
> pathological CPU load & response time
> ------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-4589
> URL: https://issues.apache.org/jira/browse/SOLR-4589
> Project: Solr
> Issue Type: Bug
> Affects Versions: 4.0, 4.1, 4.2
> Reporter: Hoss Man
> Assignee: Hoss Man
> Fix For: 4.3, 5.0, 4.2.1
>
> Attachments: SOLR-4589.patch, SOLR-4589.patch, SOLR-4589.patch,
> test-just-queries.out__4.0.0_mmap_lazy_using36index.txt,
> test-just-queries.sh, test.out__3.6.1_mmap_lazy.txt,
> test.out__3.6.1_mmap_nolazy.txt, test.out__3.6.1_nio_lazy.txt,
> test.out__3.6.1_nio_nolazy.txt, test.out__4.0.0_mmap_lazy.txt,
> test.out__4.0.0_mmap_nolazy.txt, test.out__4.0.0_nio_lazy.txt,
> test.out__4.0.0_nio_nolazy.txt, test.out__4.2.0_mmap_lazy.txt,
> test.out__4.2.0_mmap_nolazy.txt, test.out__4.2.0_nio_lazy.txt,
> test.out__4.2.0_nio_nolazy.txt, test.sh
>
>
> Following up on a [user report of exterme CPU usage in
> 4.1|http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201302.mbox/%[email protected]%3E],
> I've discovered that the following combination of factors can result in
> extreme CPU usage and excessively HTTP response times...
> * Solr 4.x (tested 3.6.1, 4.0.0, and 4.2.0)
> * enableLazyFieldLoading == true (included in example solrconfig.xml)
> * documents with a large number of values in multivalued fields (eg: tested
> ~10-15K values)
> * multiple requests returning the same doc with different "fl" lists
> I haven't dug into the route cause yet, but the essential observations is: if
> lazyloading is used in 4.x, then once a document has been fetched with an
> initial fl list X, subsequent requests for that document using a differnet fl
> list Y can be many orders of magnitute slower (while pegging the CPU) -- even
> if those same requests using fl Y uncached (or w/o lazy laoding) would be
> extremely fast.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
