[
https://issues.apache.org/jira/browse/SOLR-4589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hoss Man updated SOLR-4589:
---------------------------
Attachment: SOLR-4589.patch
updated patch
in addition to adding more tests, i added logic so that when the underlying
document is fetched to populate the lazy fields, only the specific fields
needed will be loaded (so if a doc contains fields w, x, y, and z, and the
client code fetches field w and requests that fields y and z be lazy loaded,
then field x will never be read. and field w will not be fetched redundantly.
would really appreciate feedback, i'd like to get this into th 4.2.1 bug fix
release
> 4.x + enableLazyFieldLoading + large multivalued fields + varying fl =
> pathological CPU load & response time
> ------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-4589
> URL: https://issues.apache.org/jira/browse/SOLR-4589
> Project: Solr
> Issue Type: Bug
> Affects Versions: 4.0, 4.1, 4.2
> Reporter: Hoss Man
> Attachments: SOLR-4589.patch, SOLR-4589.patch,
> test-just-queries.out__4.0.0_mmap_lazy_using36index.txt,
> test-just-queries.sh, test.out__3.6.1_mmap_lazy.txt,
> test.out__3.6.1_mmap_nolazy.txt, test.out__3.6.1_nio_lazy.txt,
> test.out__3.6.1_nio_nolazy.txt, test.out__4.0.0_mmap_lazy.txt,
> test.out__4.0.0_mmap_nolazy.txt, test.out__4.0.0_nio_lazy.txt,
> test.out__4.0.0_nio_nolazy.txt, test.out__4.2.0_mmap_lazy.txt,
> test.out__4.2.0_mmap_nolazy.txt, test.out__4.2.0_nio_lazy.txt,
> test.out__4.2.0_nio_nolazy.txt, test.sh
>
>
> Following up on a [user report of exterme CPU usage in
> 4.1|http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201302.mbox/%[email protected]%3E],
> I've discovered that the following combination of factors can result in
> extreme CPU usage and excessively HTTP response times...
> * Solr 4.x (tested 3.6.1, 4.0.0, and 4.2.0)
> * enableLazyFieldLoading == true (included in example solrconfig.xml)
> * documents with a large number of values in multivalued fields (eg: tested
> ~10-15K values)
> * multiple requests returning the same doc with different "fl" lists
> I haven't dug into the route cause yet, but the essential observations is: if
> lazyloading is used in 4.x, then once a document has been fetched with an
> initial fl list X, subsequent requests for that document using a differnet fl
> list Y can be many orders of magnitute slower (while pegging the CPU) -- even
> if those same requests using fl Y uncached (or w/o lazy laoding) would be
> extremely fast.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
