Daniel Naber wrote:
> On 2013-08-19 22:12, Dominique Pellé wrote:
>
>> In this example, the speed up is +37.5%. Useful speed up, but at the
>> same time I was hoping for more speed up given that I have 4
>> cores on my i5 laptop. While checking the sample file, 'top' reports
>> that java process used "only" ~ 220% of the CPU in general, and
>> not 400% as it would be in the ideal case.
>
> I checked with gnome-system-monitor and it reports all 4 CPUs at about
> 100% (after the initial text analysis). visualvm shows all 4 threads as
> a green bar, so there does not seem to be contention.
>
> I also tried using a ExecutorCompletionService[1] and changed the size
> of the rules that each worker threads has to match. It doesn't really
> matter a lot, but using only 1 rule per callable causes too much
> overhead. Using 4 threads means that each callable will have to match
> 450 rules (I'm trying with German text) and that works fine.
>
> So I don't know how to further improve this, expect also running the
> text analysis in parallel. Maybe there's one very slow rule for
> Esperanto that slows down everything? We already have a --profile option
> to check that.
>
> Regards
> Daniel
OK, I'll investigate later why it's slow (no time now).
I just tried to use the --profile option and got an exception.
It happens in all languages when reading from stdin:
$ echo "This is a test." | \
java -jar
languagetool/languagetool-standalone/target/LanguageTool-2.3-SNAPSHOT/LanguageTool-2.3-SNAPSHOT/languagetool-commandline.jar
-l en-US --profile
Expected text language: English (US)
Working on STDIN...
Testing 935 rules
Rule ID Time Sentences Matches Sentences per sec.
COMMA_PARENTHESIS_WHITESPACE 102 1 0 9.8
DOUBLE_PUNCTUATION 11 1 0 90.9
Exception in thread "main" java.io.IOException: Stream closed
at java.io.BufferedInputStream.getBufIfOpen(BufferedInputStream.java:162)
at java.io.BufferedInputStream.read(BufferedInputStream.java:325)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:273)
at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283)
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177)
at java.io.InputStreamReader.read(InputStreamReader.java:184)
at java.io.BufferedReader.fill(BufferedReader.java:154)
at java.io.BufferedReader.readLine(BufferedReader.java:317)
at java.io.BufferedReader.readLine(BufferedReader.java:382)
at org.languagetool.commandline.Main.runOnFileLineByLine(Main.java:229)
at org.languagetool.commandline.Main.runOnFile(Main.java:165)
at org.languagetool.commandline.Main.main(Main.java:504)
Dominique
------------------------------------------------------------------------------
Introducing Performance Central, a new site from SourceForge and
AppDynamics. Performance Central is your source for news, insights,
analysis and resources for efficient Application Performance Management.
Visit us today!
http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk
_______________________________________________
Languagetool-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/languagetool-devel
