[
https://issues.apache.org/jira/browse/LUCENE-7419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427205#comment-15427205
]
Michael McCandless commented on LUCENE-7419:
--------------------------------------------
The performance hit caused by this is really surprising: see the nightly
Elasticsearch indexing benchmark:
https://benchmarks-old.elastic.co/index.html#indexing
That drop on 8/9/2016 was from the "simple" change to Elasticsearch to pass
binary terms ({{BytesRef}}) to Lucene instead of {{String}} terms via
{{StringField}}.
It dropped indexing performance for the {{fastsettings}} run from ~80 K
docs/sec to ~39 K docs/sec, but [~rcmuir]'s patch here gets the performance
back to ~80 K docs/sec.
> performance bug in tokenstream.end()
> ------------------------------------
>
> Key: LUCENE-7419
> URL: https://issues.apache.org/jira/browse/LUCENE-7419
> Project: Lucene - Core
> Issue Type: Bug
> Reporter: Robert Muir
> Attachments: LUCENE-7419.patch
>
>
> TokenStream.end() calls getAttribute(), which is pretty costly to do
> per-stream.
> It does its current hack, because in the ctor of TokenStream is "too early".
> Instead, we can just add a variant of clear(), called end() to AttributeImpl.
> For most attributes it defers to clear, but for PosIncAtt it can handle the
> special case.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
