Hi Barry,
Thanks for explanation – I was referring to “[Moses-support] mosesserver
parallelization
issue<https://www.mail-archive.com/[email protected]&q=subject:%22%5C%5BMoses%5C-support%5C%5D+mosesserver+parallelization+issue%22&o=newest>”
thread in April 2014.
Do you thing the thread pool would be also the solution for this issue
reported on 2.1.1 branch.?
Thanks,
Tomas
From: Barry Haddow [mailto:[email protected]]
Sent: Tuesday, May 5, 2015 9:27 PM
To: Hieu Hoang; moses-support
Subject: Re: [Moses-support] Fwd: Fwd: Server development
HI Tomas
There were some issues in v2 with the way that caching was done in the
binarised phrase table. It used a cache per thread, and mosesserver used a
thread per request, so caching was effectively broken in the server. Since last
Autumn, mosesserver uses a thread pool ... and the binarised phrase table is
gone now anyway,
cheers - Barry
On 05/05/15 18:27, Hieu Hoang wrote:
What limitations are you referring to?
---------- Forwarded message ----------
From: "Ulrich Germann"
<[email protected]<mailto:[email protected]>>
Date: 5 May 2015 19:49
Subject: [Moses-support] Fwd: Server development
To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Cc:
This response was meant to go to moses-support as well Tomas.
---------- Forwarded message ----------
From: Tomas Fulajtar <[email protected]<mailto:[email protected]>>
Date: Fri, Apr 3, 2015 at 5:03 PM
Subject: RE: [Moses-support] Server development
To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Hi Ulrich,
Thanks for the thorough explanation - the idea of merging the server code back
to moses is great.
Apart from this (and I know is is a huge workload), were there any changes in
the thread support? I know this part had some limitations – as discussed on
the forum.
Kind regards,
Tomas
From: Ulrich Germann
[mailto:[email protected]<mailto:[email protected]>]
Sent: Thursday, April 2, 2015 12:57 AM
To: Tomas Fulajtar
Subject: Re: [Moses-support] Server development
Hi Tomas,
the plan is to fold server capabilities into the main moses executable. In
fact, that has already happened (in the sense that you can run the main moses
executable in server mode), but functional equivalence with the old code has
not been tested.
There are currently no server tests included in the regression tests, so I left
the old code mostly intact (adjusting only for changes in the API of functions
called) for legacy reasons, but adding new functionality to mosesserver is
extremely strongly DIScouraged.
Supplying regression tests for server functionality, on the other hand, is
equally strongly ENcouraged. In a nutshell, what you get back from calling
mosesserver and moses --server should be identical.
The long-term plan is to offer through RPC calls (almost) everything that moses
offers in batch mode (i.e., send search and output parameters through json/RCP
calls and have them noticed and respected). Notice the "long-term" there.
So mosesserver is on its way out, and moses --server-port=<port> --server will
replace the old call to mosesserver.
Best regards - Uli
On Wed, Apr 1, 2015 at 9:48 AM, Tomas Fulajtar
<[email protected]<mailto:[email protected]>> wrote:
Dear all,
I have spotted there were numerous commits in the server side development -
could the developers share the news/goals with the forum? I think it might be
interesting for more users – especially those out of core team.
Thank you,
Tomáš Fulajtár | Researcher
T: +420-545-552-340<tel:%2B420-545-552-340>
[email protected]<mailto:[email protected]> |
moravia.com<http://www.moravia.com/> | Skype: tomasfulajtar
_______________________________________________
Moses-support mailing list
[email protected]<mailto:[email protected]>
http://mailman.mit.edu/mailman/listinfo/moses-support
--
Ulrich Germann
Senior Researcher
School of Informatics
University of Edinburgh
--
Ulrich Germann
Senior Researcher
School of Informatics
University of Edinburgh
_______________________________________________
Moses-support mailing list
[email protected]<mailto:[email protected]>
http://mailman.mit.edu/mailman/listinfo/moses-support
_______________________________________________
Moses-support mailing list
[email protected]<mailto:[email protected]>
http://mailman.mit.edu/mailman/listinfo/moses-support
--- Begin Message ---
the cache is in
moses/TranslationModel/PhraseDictionary.h line 160
It's 1 cache per thread
On 16 June 2014 17:48, Mike Ladwig <[email protected]> wrote:
> Barry Haddow writes:
>
> >The -threads switch is intended for command-line Moses, not for Moses
> >server, but I didn't realise that it had this negative effect on the
> >Moses server. Moses server uses the threading model provided by xmlrpc-c
> >which, afaik, creates a new thread for each new tcp/ip connection. The
> >-threads switch controls the thread pool for command-line Moses.
>
> >This threading model interacts badly with the new caching mechanism in
> >Moses server 2.1.1 so needs to be updated - there was an earlier thread
> >(sorry!) on this topic,
> >
> >cheers - Barry
>
> I just got bitten by this regression in 2.11. I'm not sure changing out
> the xmlrpc library will help, but a pool for translation worker threads
> would. I took a quick look at the moses-cmd TranslationTask code but didn't
> see the problematic cache. Could someone point me in the right direction?
>
> Regards,
> mike.
>
> _______________________________________________
> Moses-support mailing list
> [email protected]
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
--- End Message ---
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
