Hi Dominique yes, that's the case I tried with # of callables = # of rules (see my previous email), the wait time went down quite a bit (as expected) but the overall processing time went up, I suspect because of split/merge overhead. But this depends heavily of type/number of rules, the text and cpu (e.g. if rule processing time is more unbabalanced that in Ukrainian case then increasing # of callables will help, otherwise the effect could be reverse) so we would have to try other languages with different number of callables to see what's the best approach. I know we have regular wikipedia checks for some languages - that could be a good benchmarking test.
Regards, Andriy 2016-01-28 8:47 GMT-05:00 Dominique Pellé <dominique.pe...@gmail.com>: > Andriy Rysin wrote: > > >> Then I realized that in the check method we split rules into callables >> and their count is # of cores available (in my case 8), as I have 347 >> rules this means each bucket is 43 rules and rules being not equal in >> complexity this could lead to quite unequal time for each thread. > > Hi Andriy > > Thanks for having a look at multi-thread performances. > I don't know the code as much as you do. But if we indeed > split the number of rules equally before processing them, then > it seems bad for balancing the work. > > Can't we instead have a queue with all rules to be processed? > When a thread is ready to do work, it picks the next rule to process > from the queue. So load would be well balanced, even if some rules > are 10x slower than others. With such a queue, a thread that picks > up an expensive rules would end up processing less rules than a > thread that picks up fast rules, keeping all CPUs busy, as much > as possible. > > Regards > Dominique > > ------------------------------------------------------------------------------ > Site24x7 APM Insight: Get Deep Visibility into Application Performance > APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month > Monitor end-to-end web transactions and take corrective actions now > Troubleshoot faster and improve end-user experience. Signup Now! > http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140 > _______________________________________________ > Languagetool-devel mailing list > Languagetool-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/languagetool-devel ------------------------------------------------------------------------------ Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140 _______________________________________________ Languagetool-devel mailing list Languagetool-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/languagetool-devel
