David Smiley created LUCENE-6868:
------------------------------------
Summary: ParallelLeafReader.getTermVectors can indirectly load TVs
multiple times
Key: LUCENE-6868
URL: https://issues.apache.org/jira/browse/LUCENE-6868
Project: Lucene - Core
Issue Type: Improvement
Components: core/index, core/termvectors
Reporter: David Smiley
ParallelLeafReader has a getTermVectors(docId) implementation that loops over
each field it has in a loop and calls getTermVector(docId,fieldName). But the
implementation of that will load all term vectors for all fields in that
reader, yet ParallelLeafReader only wants one. The effect is an O(n^2) where
'n' is the number of fields, when we could get O(n) if we do it right. PLR
should call getTermVectors(docId) (not referring to a specific field) for each
of it's readers and then aggregate them.
This wouldn't be such a problem if our term vector API/Codec was improved to
not load all term vectors for all fields from disk at once.
Found via randomized-testing of IndexWriter auto-picking ParallelAtomicReader
along with a test I have that asserts TVs aren't fetched for a doc more than
once.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
