W dniu 2012-11-16 09:28, R.J. Baars pisze: >> Hi, >> >> the load test I did for our HTTPS service showed that we have kind of a >> performance problem for languages that have a lot of rules. Testing a >> random German text with 125 sentences takes 2 seconds on my machine at >> average. About half of that time is spent in rule matching, i.e. in >> PatternRuleMatcher.match(). >> >> What can be done about this? >> >> -More micro-optimization of the inner loop of pattern matching. I think >> there's not much potential in that, but I'd love to be proven wrong. >> >> -Make the checking process work in parallel to better use multiple cores. >> >> -Rewrite the pattern matching to use a finite state machine. I think this >> could improve performance a lot, if we create one state machine that >> includes all rules of a language. This state machine would then work on a >> per-sentence basis. >> > Instead af checking a text completely, check gradually; active sentence > first, then back and forth (client code)
LibreOffice does basically this. But it won't really speed things up. > Could rules be 'compiled' instead of interpreted? This is the finite state idea above, but compiling is not a trivial business. > Is there a significant difference between direct word matching, postag > matching and regexp matching? Would it help changing some postag-rules > into word-match rules for the most common words? Regexp matching is the slowest thing. There's no difference in speed between postag and word level matching. > > Could rules be ordered by biggest chance to hit first? Anyway, we would have to match them all. > > Could rules be structured in if-then-else-like structures? (Mutually > excluding trules e.g.) We can already do this by making some operations on POS tags in the disambiguator. > > Is there time to be won in searching the postags for words? No. This is the fastest part. Best, Marcin ------------------------------------------------------------------------------ Monitor your physical, virtual and cloud infrastructure from a single web console. Get in-depth insight into apps, servers, databases, vmware, SAP, cloud infrastructure, etc. Download 30-day Free Trial. Pricing starts from $795 for 25 servers or applications! http://p.sf.net/sfu/zoho_dev2dev_nov _______________________________________________ Languagetool-devel mailing list Languagetool-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/languagetool-devel
