Hi, I am having an issue with MosesServer. I am using compact phrase and reordering table, and KENLM.
The problem is this (I'll explain with an example): - I have one file with 20 very short sentences. I split and tokenize them and send one XMLPRC request per sentence to MosesServer - If I create just one XMLRPC ServerProxy instance and I use it to send all the requests through it, all the sentences get translated in approx 2.5 sec. The problem is that the first sentence takes almost 2 seconds to get translated, while the other 19 are much faster - If I create one ServerProxy instance per request, the translation time rises to 30 sec (now every sentence takes almost 2 sec) I don't understand the reason of that delay for the first request. I have followed the source of this delay to the function: GetTargetPhraseCollectionLEGACY(const Phrase& src) in the file: ...TranslationModel/PhraseDictionary.cpp It seems that for the first request it's needed to look for something in the phrase table, while for subsequent requests it can be retrieved (most of the times) from a cache. But, as the sentences in my file are not related one to another in any way, the information on this cache can not be sentence-dependent, so why wouldn't it be possible for the cache to be preloaded with the information needed? I think that perhaps I have something misconfigured, because I have seen other people using the approach of creating one ServerProxy object for each XMLRPC request (which would facilitate things a lot for me), so I don't think they are experiencing this overhead. Perhaps using the compact formats can have something to do with it? Any help would be much appreciated. I paste below my moses.ini, if that helps: Thanks :) ### MOSES CONFIG FILE ### ################### # input factors [input-factors] 0 # mapping steps [mapping] 0 T 0 # translation tables: table type (hierarchical(0), textual (0), binary (1)), source-factors, target-factors, number of scores, file # OLD FORMAT is still handled for back-compatibility # OLD FORMAT translation tables: source-factors, target-factors, number of scores, file # OLD FORMAT a binary table type (1) is assumed [ttable-file] 12 0 0 5 /opt/moses-compiling/modelos/es-en/phrase-model/phrase-table # no generation models, no generation-file section # language models: type(srilm/irstlm), factors, order, file [lmodel-file] 8 0 5 /opt/moses-compiling/modelos/es-en/lm/13-19-03gen_intec_head8m_sb5LM.kenlm # limit on how many phrase translations e for each phrase f are loaded # 0 = all elements loaded [ttable-limit] 10 # distortion (reordering) files [distortion-file] 0-0 wbe-msd-bidirectional-fe-allff 6 /opt/moses-compiling/modelos/es-en/phrase-model/reordering-table # distortion (reordering) weight [weight-d] 0.097107 0.150373 -0.0551767 -0.0307787 0.114613 0.214587 0.0467398 # language model weights [weight-l] 0.0442748 # translation model weights [weight-t] 0.00370888 0.0425665 0.0719956 0.0202699 0.071147 # no generation models, no weight-generation section # word penalty [weight-w] 0.0366626 [distortion-limit] 6 [v] 0 -- Marcos Fernández López Project Technician Centro de Investigación en Tecnoloxías da Información (CITIUS) Universidade de Santiago de Compostela 15782 Santiago de Compostela, Spain e-mail: [email protected] Phone:(+34) 881 816444 _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
