It would be good if you can check in your change and take charge of it. If you're waiting for us academics to fix it, you'll be waiting a long time. We rarely use the server, we don't know what the issues are and we won't know if we've really fixed it when we change it
Hieu Hoang Sent while bumping into things On 5 Aug 2015 4:15 pm, "Martin Baumgärtner" < [email protected]> wrote: > Hi Oren, > > we temporarily fixed this issue with the following quick hack for Abyss > server's constructor call: > > xmlrpc_c::serverAbyss myAbyssServer( > xmlrpc_c::serverAbyss::constrOpt() > .registryP(&myRegistry) > .portNumber(port) // TCP port on which to listen > .logFileName(logfile) > .allowOrigin("*") > .maxConn((unsigned int)numThreads*4) // *4 (performance issue, > inofficial quick hack) > ); > > I'm also looking forward to the official fix, i.e. a configurable value > for abyss connections ... > > Kind regards, > Martin > > > Am 04.08.2015 um 09:08 schrieb Oren: > > Hi Barry and Martin, > > Has this issue been fixed in the source code? Should I take thr current > master branch and compile it myself to avoid this issue? > > Thanks. > > On Friday, July 24, 2015, Barry Haddow <[email protected]> wrote: > >> Hi Martin >> >> So it looks like it was the abyss connection limit that was causing the >> problem? I'm not sure why this should be, either it should queue the jobs >> up or discard them. >> >> Probably Moses server should allow users to configure the number of abyss >> connections directly rather than tying it to the number of Moses threads. >> >> cheers - Barry >> >> On 24/07/15 14:17, Martin Baumgärtner wrote: >> >> Hi Barry, >> >> thanks for your quick reply! >> >> We're currently testing on SHA e53ad4085942872f1c4ce75cb99afe66137e1e17 >> (master, from 2015-07-23). This version includes the fix for mosesserver >> recently mentioned by Hieu in the performance thread. >> >> Following my first intuition, I ran the critical experiments after having >> modified mosesserver.cpp just by simply doubling the given --threads value, >> but only for abyss server: .maxConn((unsigned int)numThreads*2): >> >> 2.) >> server: --threads: 8 (i.e. abyss: 16) >> client: shoots 10 threads => about 11 seconds, server shows busy CPU >> workload => OK >> >> 5.) >> server: --threads: 16 (i.e. abyss: 32) >> client: shoots 20 threads => about 11 seconds, server shows busy CPU >> workload => OK >> >> Helps. :-) >> >> Best wishes, >> Martin >> >> Am 24.07.2015 um 13:26 schrieb Barry Haddow: >> >> Hi Martin >> >> Thanks for the detailed information. It's a bit strange since >> command-line Moses uses the same threadpool, and we always overload the >> threadpool since the entire test set is read in and queued. >> >> The server was refactored somewhat recently - which git revision are you >> using? >> >> In the case where Moses takes a long time, and cpu activity is low, it >> could be either waiting on IO, or waiting on locks. If the former, I don't >> know why it works fine for command-line Moses, and if the latter then it's >> odd how it eventually frees itself. >> >> Is it possible to run scenario 2, then attach a debugger whilst Moses is >> in the low-CPU phase to see what it is doing? (You can do this in gdb with >> "info threads") >> >> cheers - Barry >> >> On 24/07/15 12:07, Martin Baumgärtner wrote: >> >> Hi, >> >> followed your discussion about mosesserver performance issue with much >> interest so far. >> >> We're having similar behaviour in our perfomance tests with a current >> github master clone. Both, mosesserver and complete engine run from same >> local machine, i.e. no NFS. Machine is virtualized CentOS 7 using Hyper-V: >> >> > lscpu >> >> Architecture: x86_64 >> CPU op-mode(s): 32-bit, 64-bit >> Byte Order: Little Endian >> CPU(s): 8 >> On-line CPU(s) list: 0-7 >> Thread(s) per core: 1 >> Core(s) per socket: 8 >> Socket(s): 1 >> NUMA node(s): 1 >> Vendor ID: GenuineIntel >> CPU family: 6 >> Model: 30 >> Model name: Intel(R) Core(TM) i7 CPU 860 @ 2.80GHz >> Stepping: 5 >> CPU MHz: 2667.859 >> BogoMIPS: 5335.71 >> Hypervisor vendor: Microsoft >> Virtualization type: full >> L1d cache: 32K >> L1i cache: 32K >> L2 cache: 256K >> L3 cache: 8192K >> >> >> Following experiments using an engine with 75000 segments for TM/LM >> (--minphr-memory, --minlexr-memory): >> >> 1.) >> server: --threads: 8 >> client: shoots 8 threads => about 12 seconds, server shows full CPU >> workload => OK >> >> 2.) >> server: --threads: 8 >> client: shoots 10 threads => about 85 seconds, server shows mostly low >> activity, full CPU workload only near end of process => NOT OK >> >> 3.) >> server: --threads: 16 >> client: shoots 10 threads => about 12 seconds, server shows busy CPU >> workload => OK >> >> 4.) >> server: --threads: 16 >> client: shoots 16 threads => about 11 seconds, server shows busy CPU >> workload => OK >> >> 5.) >> server: --threads: 16 >> client: shoots 20 threads => about 40-60 seconds (depending), server >> shows mostly low activity, full CPU workload only near end of process => >> NOT OK >> >> >> We've seen a breakdown in performance always when the client threads >> exceed the number of threads given by the --threads param. >> >> Kind regards, >> Martin >> >> -- >> >> >> *STAR Group* <http://www.star-group.net> >> <http://www.star-group.net/> >> *Martin Baumgärtner* >> STAR Language Technology & Solutions GmbH Umberto-Nobile-Straße 19 | >> 71063 Sindelfingen | Germany Tel. +49 70 31-4 10 92-0 >> [email protected] >> Fax +49 70 31-4 10 92-70 www.star-group.net Geschäftsführer: Oliver Rau, >> Bernd Barth >> Handelsregister Stuttgart HRB 245654 | St.-Nr. 56098/11677 >> >> >> >> >> _______________________________________________ >> Moses-support mailing >> [email protected]http://mailman.mit.edu/mailman/listinfo/moses-support >> >> >> >> >> The University of Edinburgh is a charitable body, registered in >> Scotland, with registration number SC005336. >> >> >> -- >> >> >> *STAR Group* <http://www.star-group.net> >> <http://www.star-group.net/> >> *Martin Baumgärtner* >> STAR Language Technology & Solutions GmbH Umberto-Nobile-Straße 19 | >> 71063 Sindelfingen | Germany Tel. +49 70 31-4 10 92-0 >> [email protected] >> Fax +49 70 31-4 10 92-70 www.star-group.net Geschäftsführer: Oliver Rau, >> Bernd Barth >> Handelsregister Stuttgart HRB 245654 | St.-Nr. 56098/11677 >> >> >> >> >> > -- > > > *STAR Group* <http://www.star-group.net> > <http://www.star-group.net/> > *Martin Baumgärtner* > STAR Language Technology & Solutions GmbH Umberto-Nobile-Straße 19 | > 71063 Sindelfingen | Germany Tel. +49 70 31-4 10 92-0 > [email protected] > Fax +49 70 31-4 10 92-70 www.star-group.net Geschäftsführer: Oliver Rau, > Bernd Barth > Handelsregister Stuttgart HRB 245654 | St.-Nr. 56098/11677 > > > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > >
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
