[
https://issues.apache.org/jira/browse/CODEC-132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13232327#comment-13232327
]
Thomas Neidhart commented on CODEC-132:
---------------------------------------
I have tested it with junit-benchmarks and my results are as follows:
With HashSet:
{noformat}
BeiderMorseEncoderTest.testAllChars: [measured 10 out of 15 rounds, threads: 1
(sequential)]
round: 2.85 [+- 0.01], round.gc: 0.00 [+- 0.00], GC.calls: 1213, GC.time:
0.29, time.total: 43.66, time.warmup: 15.20, time.bench: 28.45
BeiderMorseEncoderTest.testAsciiEncodeNotEmpty1Letter: [measured 10 out of 15
rounds, threads: 1 (sequential)]
round: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 1, GC.time: 0.00,
time.total: 0.06, time.warmup: 0.02, time.bench: 0.04
BeiderMorseEncoderTest.testAsciiEncodeNotEmpty2Letters: [measured 10 out of 15
rounds, threads: 1 (sequential)]
round: 0.18 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 72, GC.time: 0.02,
time.total: 2.79, time.warmup: 0.98, time.bench: 1.82
BeiderMorseEncoderTest.testEncodeAtzNotEmpty: [measured 10 out of 15 rounds,
threads: 1 (sequential)]
round: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00,
time.total: 0.02, time.warmup: 0.01, time.bench: 0.01
BeiderMorseEncoderTest.testEncodeGna: [measured 10 out of 15 rounds, threads: 1
(sequential)]
round: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00,
time.total: 0.00, time.warmup: 0.00, time.bench: 0.00
BeiderMorseEncoderTest.testInvalidLangIllegalArgumentException: [measured 10
out of 15 rounds, threads: 1 (sequential)]
round: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00,
time.total: 0.00, time.warmup: 0.00, time.bench: 0.00
BeiderMorseEncoderTest.testInvalidLangIllegalStateException: [measured 10 out
of 15 rounds, threads: 1 (sequential)]
round: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00,
time.total: 0.00, time.warmup: 0.00, time.bench: 0.00
BeiderMorseEncoderTest.testInvalidLanguageIllegalArgumentException: [measured
10 out of 15 rounds, threads: 1 (sequential)]
round: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00,
time.total: 0.00, time.warmup: 0.00, time.bench: 0.00
BeiderMorseEncoderTest.testLongestEnglishSurname: [measured 10 out of 15
rounds, threads: 1 (sequential)]
round: 0.01 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 1, GC.time: 0.00,
time.total: 0.10, time.warmup: 0.04, time.bench: 0.07
BeiderMorseEncoderTest.testNegativeIndexForRuleMatchIndexOutOfBoundsException:
[measured 10 out of 15 rounds, threads: 1 (sequential)]
round: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00,
time.total: 0.00, time.warmup: 0.00, time.bench: 0.00
BeiderMorseEncoderTest.testOOM: [measured 10 out of 15 rounds, threads: 1
(sequential)]
round: 0.01 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 2, GC.time: 0.00,
time.total: 0.12, time.warmup: 0.04, time.bench: 0.08
BeiderMorseEncoderTest.testSetConcat: [measured 10 out of 15 rounds, threads: 1
(sequential)]
round: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00,
time.total: 0.00, time.warmup: 0.00, time.bench: 0.00
BeiderMorseEncoderTest.testSetNameTypeAsh: [measured 10 out of 15 rounds,
threads: 1 (sequential)]
round: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00,
time.total: 0.00, time.warmup: 0.00, time.bench: 0.00
BeiderMorseEncoderTest.testSetRuleTypeExact: [measured 10 out of 15 rounds,
threads: 1 (sequential)]
round: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00,
time.total: 0.00, time.warmup: 0.00, time.bench: 0.00
BeiderMorseEncoderTest.testSetRuleTypeToRulesIllegalArgumentException:
[measured 10 out of 15 rounds, threads: 1 (sequential)]
round: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00,
time.total: 0.00, time.warmup: 0.00, time.bench: 0.00
BeiderMorseEncoderTest.testSpeedCheck: [measured 10 out of 15 rounds, threads:
1 (sequential)]
round: 0.41 [+- 0.01], round.gc: 0.00 [+- 0.00], GC.calls: 91, GC.time: 0.04,
time.total: 6.13, time.warmup: 2.04, time.bench: 4.09
BeiderMorseEncoderTest.testSpeedCheck2: [measured 10 out of 15 rounds, threads:
1 (sequential)]
round: 0.28 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 48, GC.time: 0.01,
time.total: 4.14, time.warmup: 1.38, time.bench: 2.76
BeiderMorseEncoderTest.testSpeedCheck3: [measured 10 out of 15 rounds, threads:
1 (sequential)]
round: 0.54 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 140, GC.time: 0.05,
time.total: 8.09, time.warmup: 2.69, time.bench: 5.39
BeiderMorseEncoderTest.testEncodeEmpty: [measured 10 out of 15 rounds, threads:
1 (sequential)]
round: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00,
time.total: 0.00, time.warmup: 0.00, time.bench: 0.00
BeiderMorseEncoderTest.testEncodeNull: [measured 10 out of 15 rounds, threads:
1 (sequential)]
round: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00,
time.total: 0.00, time.warmup: 0.00, time.bench: 0.00
BeiderMorseEncoderTest.testEncodeWithInvalidObject: [measured 10 out of 15
rounds, threads: 1 (sequential)]
round: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00,
time.total: 0.00, time.warmup: 0.00, time.bench: 0.00
BeiderMorseEncoderTest.testLocaleIndependence: [measured 10 out of 15 rounds,
threads: 1 (sequential)]
round: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00,
time.total: 0.01, time.warmup: 0.00, time.bench: 0.00
{noformat}
With LinkedHashSet:
{noformat}
BeiderMorseEncoderTest.testAllChars: [measured 10 out of 15 rounds, threads: 1
(sequential)]
round: 2.87 [+- 0.01], round.gc: 0.00 [+- 0.00], GC.calls: 1194, GC.time:
0.29, time.total: 44.02, time.warmup: 15.35, time.bench: 28.67
BeiderMorseEncoderTest.testAsciiEncodeNotEmpty1Letter: [measured 10 out of 15
rounds, threads: 1 (sequential)]
round: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 1, GC.time: 0.00,
time.total: 0.06, time.warmup: 0.02, time.bench: 0.04
BeiderMorseEncoderTest.testAsciiEncodeNotEmpty2Letters: [measured 10 out of 15
rounds, threads: 1 (sequential)]
round: 0.18 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 66, GC.time: 0.02,
time.total: 2.85, time.warmup: 1.01, time.bench: 1.84
BeiderMorseEncoderTest.testEncodeAtzNotEmpty: [measured 10 out of 15 rounds,
threads: 1 (sequential)]
round: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 1, GC.time: 0.00,
time.total: 0.02, time.warmup: 0.01, time.bench: 0.01
BeiderMorseEncoderTest.testEncodeGna: [measured 10 out of 15 rounds, threads: 1
(sequential)]
round: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00,
time.total: 0.01, time.warmup: 0.00, time.bench: 0.00
BeiderMorseEncoderTest.testInvalidLangIllegalArgumentException: [measured 10
out of 15 rounds, threads: 1 (sequential)]
round: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00,
time.total: 0.00, time.warmup: 0.00, time.bench: 0.00
BeiderMorseEncoderTest.testInvalidLangIllegalStateException: [measured 10 out
of 15 rounds, threads: 1 (sequential)]
round: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00,
time.total: 0.00, time.warmup: 0.00, time.bench: 0.00
BeiderMorseEncoderTest.testInvalidLanguageIllegalArgumentException: [measured
10 out of 15 rounds, threads: 1 (sequential)]
round: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00,
time.total: 0.00, time.warmup: 0.00, time.bench: 0.00
BeiderMorseEncoderTest.testLongestEnglishSurname: [measured 10 out of 15
rounds, threads: 1 (sequential)]
round: 0.01 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 1, GC.time: 0.00,
time.total: 0.10, time.warmup: 0.03, time.bench: 0.07
BeiderMorseEncoderTest.testNegativeIndexForRuleMatchIndexOutOfBoundsException:
[measured 10 out of 15 rounds, threads: 1 (sequential)]
round: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00,
time.total: 0.00, time.warmup: 0.00, time.bench: 0.00
BeiderMorseEncoderTest.testOOM: [measured 10 out of 15 rounds, threads: 1
(sequential)]
round: 0.01 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 3, GC.time: 0.00,
time.total: 0.12, time.warmup: 0.04, time.bench: 0.08
BeiderMorseEncoderTest.testSetConcat: [measured 10 out of 15 rounds, threads: 1
(sequential)]
round: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00,
time.total: 0.00, time.warmup: 0.00, time.bench: 0.00
BeiderMorseEncoderTest.testSetNameTypeAsh: [measured 10 out of 15 rounds,
threads: 1 (sequential)]
round: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00,
time.total: 0.00, time.warmup: 0.00, time.bench: 0.00
BeiderMorseEncoderTest.testSetRuleTypeExact: [measured 10 out of 15 rounds,
threads: 1 (sequential)]
round: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00,
time.total: 0.00, time.warmup: 0.00, time.bench: 0.00
BeiderMorseEncoderTest.testSetRuleTypeToRulesIllegalArgumentException:
[measured 10 out of 15 rounds, threads: 1 (sequential)]
round: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00,
time.total: 0.00, time.warmup: 0.00, time.bench: 0.00
BeiderMorseEncoderTest.testSpeedCheck: [measured 10 out of 15 rounds, threads:
1 (sequential)]
round: 0.31 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 97, GC.time: 0.03,
time.total: 4.59, time.warmup: 1.54, time.bench: 3.06
BeiderMorseEncoderTest.testSpeedCheck2: [measured 10 out of 15 rounds, threads:
1 (sequential)]
round: 0.27 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 81, GC.time: 0.02,
time.total: 4.03, time.warmup: 1.34, time.bench: 2.69
BeiderMorseEncoderTest.testSpeedCheck3: [measured 10 out of 15 rounds, threads:
1 (sequential)]
round: 0.46 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 149, GC.time: 0.04,
time.total: 6.92, time.warmup: 2.31, time.bench: 4.61
BeiderMorseEncoderTest.testEncodeEmpty: [measured 10 out of 15 rounds, threads:
1 (sequential)]
round: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00,
time.total: 0.00, time.warmup: 0.00, time.bench: 0.00
BeiderMorseEncoderTest.testEncodeNull: [measured 10 out of 15 rounds, threads:
1 (sequential)]
round: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00,
time.total: 0.00, time.warmup: 0.00, time.bench: 0.00
BeiderMorseEncoderTest.testEncodeWithInvalidObject: [measured 10 out of 15
rounds, threads: 1 (sequential)]
round: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00,
time.total: 0.00, time.warmup: 0.00, time.bench: 0.00
BeiderMorseEncoderTest.testLocaleIndependence: [measured 10 out of 15 rounds,
threads: 1 (sequential)]
round: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00,
time.total: 0.01, time.warmup: 0.00, time.bench: 0.00
{noformat}
The speed tests are even faster for some reason, but in general I do not think
that the data structure does make much of a difference as the number of
phonemes is anyway now limited to 20 by default.
> BeiderMorseEncoder OOM issues
> -----------------------------
>
> Key: CODEC-132
> URL: https://issues.apache.org/jira/browse/CODEC-132
> Project: Commons Codec
> Issue Type: Bug
> Affects Versions: 1.6
> Reporter: Robert Muir
> Fix For: 1.7
>
> Attachments: CODEC-132.patch, CODEC-132_test.patch
>
>
> In Lucene/Solr, we integrated this encoder into the latest release.
> Our tests use a variety of random strings, and we have recent jenkins failures
> from some input streams (of length <= 10), using huge amounts of memory (e.g.
> > 64MB),
> resulting in OOM.
> I've created a test case (length is 30 here) that will OOM with -Xmx256M.
> I haven't dug into this much as to what's causing it, but I suspect there
> might be a bug
> revolving around certain punctuation characters: we didn't see this happening
> until
> we beefed up our random string generation to start producing "html-like"
> strings.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
