GitHub user KellenSunderland opened a pull request:
https://github.com/apache/incubator-joshua/pull/10
Performance improvements and code cleanup
In these commits we've done a lot of performance improvement work mostly
focused around short-lived object allocations. By our measurements this perf
improvements will speed up decoding by around 30% and significantly reduce the
amount of garbage collection that happens during decoding. In addition we also
cleaned up a few items that were highlighted by FindBugs
(http://findbugs.sourceforge.net/). We've also added some extra testing (to be
hooked up at some point in the future).
One not around versioning, the locks used below do require Java 8.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/KellenSunderland/incubator-joshua master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-joshua/pull/10.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #10
----
commit 7fd3cfcbbd12f07d0b4eb58ef02c39f4af7ef7e3
Author: Pavel Danchenko <[email protected]>
Date: 2015-12-14T09:34:16Z
Clean up some code smells detected by findbugs (our static code analyzer)
commit 39f59a8d7950f362cc52b2414dbd53efc130e404
Author: Pavel Danchenko <[email protected]>
Date: 2016-02-10T16:12:15Z
Optimized allocations with sub-array indexes
------------
last 10 minutes of recordings taken
Statistics:
Before
Total TLAB: 1,391.77 GB
Allocation rate: 2.32 GB/s
After
Total TLAB: 1,320.95 GB
Allocation rate: 2.20 GB/s
------------
Results
-10 GB/s allocation rate
java.utils.Arrays.copyOfRange -70GB allocations from
joshua.decoder.ff.lm.berkeley_lm.LMGrammarBerkeley.sentenceLogProbability
10K translation performance testing
Before: avg 5.29
After: avg 5.24
commit cb7001406da2f601dac51669d56648342c881b45
Author: Kellen Sunderland <[email protected]>
Date: 2016-01-27T10:55:13Z
Reduce concurrency on Vocab by using opt stamped rw locks
Drops concurrency in Vocab class from 7min14s to 0m0s during training run
of 50K
commit ef6d5686380fc0965182bb5432adfb35eccab193
Author: Kellen Sunderland <[email protected]>
Date: 2016-01-27T12:35:21Z
Removed nonTerminalIndices functionality as it is no longer in use
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
