[
https://issues.apache.org/jira/browse/LUCENE-2218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12801341#action_12801341
]
Steven Rowe edited comment on LUCENE-2218 at 1/17/10 5:17 PM:
--------------------------------------------------------------
The rewrite included some optimizations (e.g., no longer constructing n
StringBuilders for every position in the input stream), and the performance is
now modestly better - below is a comparison generated using the attached
benchmark patch:
JAVA:
java version "1.5.0_15"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_15-b04)
Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_15-b04, mixed mode)
OS:
cygwin
WinVistaService Pack 2
Service Pack 26060022202561
||Max Shingle
Size||Unigrams?||Unpatched||Patched||StandardAnalyzer||Improvement||
|2|no|4.92s|4.74s|2.19s|7.5%|
|2|yes|5.04s|4.90s|2.19s|5.6%|
|4|no|6.21s|5.82s|2.19s|11.2%|
|4|yes|6.41s|5.97s|2.19s|12.1%|
was (Author: steve_rowe):
The rewrite included some optimizations (e.g., no longer constructing n
StringBuilders for every position in the input stream), and the performance is
now modestly better - below is a comparison generated using the attached
benchmark patch:
JAVA:
java version "1.5.0_15"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_15-b04)
Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_15-b04, mixed mode)
OS:
cygwin
WinVistaService Pack 2
Service Pack 26060022202561
||Max Shingle
Size||Unigrams?||Unpatched||Patched||StandardAnalyzer||Improvement||
|2|no|4.92|4.74|2.19|7.5%|
|2|yes|5.04|4.90|2.19|5.6%|
|4|no|6.21|5.82|2.19|11.2%|
|4|yes|6.41|5.97|2.19|12.1%|
> ShingleFilter improvements
> --------------------------
>
> Key: LUCENE-2218
> URL: https://issues.apache.org/jira/browse/LUCENE-2218
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/analyzers
> Affects Versions: 3.0
> Reporter: Steven Rowe
> Priority: Minor
> Attachments: LUCENE-2218.benchmark.patch,
> LUCENE-2218.benchmark.patch, LUCENE-2218.patch
>
>
> ShingleFilter should allow configuration of minimum shingle size (in addition
> to maximum shingle size), so that it's possible to (e.g.) output only
> trigrams instead of bigrams mixed with trigrams. The token separator used in
> composing shingles should be configurable too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org
