Daniel Naber <[email protected]> wrote: > Hi, > > LT now makes use of multiple threads when checking text. Each of the > threads checks a part of the rules. The idea behind this is that all > rules are independent, thus it makes more sense to split the rules than > the text. > > According to my tests this can speed up checking by about 70% on modern > hardware. It does not speed up program start though. > > Technically this is done by a new class MultiThreadedJLanguageTool, > which extends JLanguageTool but uses as many threads as the computer has > processors. > > Let me know if there are any problems with this.
Nice feature. Thanks! I measured the speed up of LT before and after the multithreaded change on a sample large file: (a) 3min 28 sec (b) 2min 10 sec (a) is At commit 69e8e1cdd7a16ba8bb297f2705c50ef89b1e3b30, so before mulithread change. (b) is at commit baf783c12f82a772f6f1f2b4a8bf4db82bb38b8a so after multithread change. In this example, the speed up is +37.5%. Useful speed up, but at the same time I was hoping for more speed up given that I have 4 cores on my i5 laptop. While checking the sample file, 'top' reports that java process used "only" ~ 220% of the CPU in general, and not 400% as it would be in the ideal case. Does it happen because... - there is contention between threads? (wait on locks on shared resources?). - or because load is not balanced between threads? I used this command to measure speed with a large sample file of almost 2 million Esperanto sentences (~ 100MB in size): $ time java -jar languagetool-standalone/target/LanguageTool-2.3-SNAPSHOT/LanguageTool-2.3-SNAPSHOT/languagetool-commandline.jar \ -l eo -v < ~/sentences-epo.csv > /dev/null 2>&1 Regards Dominique ------------------------------------------------------------------------------ Introducing Performance Central, a new site from SourceForge and AppDynamics. Performance Central is your source for news, insights, analysis and resources for efficient Application Performance Management. Visit us today! http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk _______________________________________________ Languagetool-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/languagetool-devel
