Hi Barry,
thanks for your quick reply!
We're currently testing on SHA e53ad4085942872f1c4ce75cb99afe66137e1e17
(master, from 2015-07-23). This version includes the fix for mosesserver
recently mentioned by Hieu in the performance thread.
Following my first intuition, I ran the critical experiments after
having modified mosesserver.cpp just by simply doubling the given
--threads value, but only for abyss server: .maxConn((unsigned
int)numThreads*2):
2.)
server: --threads: 8 (i.e. abyss: 16)
client: shoots 10 threads => about 11 seconds, server shows busy CPU
workload => OK
5.)
server: --threads: 16 (i.e. abyss: 32)
client: shoots 20 threads => about 11 seconds, server shows busy CPU
workload => OK
Helps. :-)
Best wishes,
Martin
Am 24.07.2015 um 13:26 schrieb Barry Haddow:
Hi Martin
Thanks for the detailed information. It's a bit strange since
command-line Moses uses the same threadpool, and we always overload
the threadpool since the entire test set is read in and queued.
The server was refactored somewhat recently - which git revision are
you using?
In the case where Moses takes a long time, and cpu activity is low, it
could be either waiting on IO, or waiting on locks. If the former, I
don't know why it works fine for command-line Moses, and if the latter
then it's odd how it eventually frees itself.
Is it possible to run scenario 2, then attach a debugger whilst Moses
is in the low-CPU phase to see what it is doing? (You can do this in
gdb with "info threads")
cheers - Barry
On 24/07/15 12:07, Martin Baumgärtner wrote:
Hi,
followed your discussion about mosesserver performance issue with
much interest so far.
We're having similar behaviour in our perfomance tests with a current
github master clone. Both, mosesserver and complete engine run from
same local machine, i.e. no NFS. Machine is virtualized CentOS 7
using Hyper-V:
> lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 1
Core(s) per socket: 8
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 30
Model name: Intel(R) Core(TM) i7 CPU 860 @ 2.80GHz
Stepping: 5
CPU MHz: 2667.859
BogoMIPS: 5335.71
Hypervisor vendor: Microsoft
Virtualization type: full
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 8192K
Following experiments using an engine with 75000 segments for TM/LM
(--minphr-memory, --minlexr-memory):
1.)
server: --threads: 8
client: shoots 8 threads => about 12 seconds, server shows full CPU
workload => OK
2.)
server: --threads: 8
client: shoots 10 threads => about 85 seconds, server shows mostly
low activity, full CPU workload only near end of process => NOT OK
3.)
server: --threads: 16
client: shoots 10 threads => about 12 seconds, server shows busy CPU
workload => OK
4.)
server: --threads: 16
client: shoots 16 threads => about 11 seconds, server shows busy CPU
workload => OK
5.)
server: --threads: 16
client: shoots 20 threads => about 40-60 seconds (depending), server
shows mostly low activity, full CPU workload only near end of process
=> NOT OK
We've seen a breakdown in performance always when the client threads
exceed the number of threads given by the --threads param.
Kind regards,
Martin
--
*STAR Group* <http://www.star-group.net>
<http://www.star-group.net/>
*Martin Baumgärtner*
STAR Language Technology & Solutions GmbH
Umberto-Nobile-Straße 19 | 71063 Sindelfingen | Germany
Tel. +49 70 31-4 10 92-0 [email protected]
<mailto:[email protected]>
Fax +49 70 31-4 10 92-70 www.star-group.net
<http://www.star-group.net/>
Geschäftsführer: Oliver Rau, Bernd Barth
Handelsregister Stuttgart HRB 245654 | St.-Nr. 56098/11677
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
--
*STAR Group* <http://www.star-group.net>
<http://www.star-group.net/>
*Martin Baumgärtner*
STAR Language Technology & Solutions GmbH
Umberto-Nobile-Straße 19 | 71063 Sindelfingen | Germany
Tel. +49 70 31-4 10 92-0 [email protected]
<mailto:[email protected]>
Fax +49 70 31-4 10 92-70 www.star-group.net <http://www.star-group.net/>
Geschäftsführer: Oliver Rau, Bernd Barth
Handelsregister Stuttgart HRB 245654 | St.-Nr. 56098/11677
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
