Ahoj, you don't need to include extra headers for SALM, you just need to use g++ 4.1 or older to build it.
Parallelization would be nice, it's really really slow now. If you merge it it will be better than me pushing my changes, since I believe you know more what's going on (my version will almost surely break with parallel corpora, factors etc) Karel On Wed, Aug 22, 2012 at 9:29 AM, Aleš Tamchyna <[email protected]> wrote: > Hi, > > sigtest filter was added to EMS in branch damt_phrase, you can check it > out. There's also a script for its parallelization. > > I didn't merge this into master because the sigtest filter still depends > on SALM (which in turn doesn't compile on Linux unless extra header > files are included in the source code) -- but maybe it's worth merging > anyway? > > Best, > Ales > > Dne 22.8.2012 01:14, Karel Bílek napsal(a): >> OK. >> >> I will add support of sigtest-filter ( >> http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc19 ) which >> ships with moses to my EMS experiment.meta ; I will add this to moses >> on git as another pull request as I properly test it >> >> It will use the first case I mentioned because I don't use factors and >> I don't understand them as well :) >> >> BTW >> there is a slight error in here >> http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc19 >> Build Instructions >> >> you can't just build it with "make", since it will work only with g++ >> 4.1 and not newer, you have to edit the makefile >> also, it's not in "directory sigtest-filter", but in >> "contrib/sigtest-filter". >> >> Karel >> >> On Tue, Aug 21, 2012 at 10:31 PM, Philipp Koehn<[email protected]> wrote: >>> Hi, >>> >>> "ignore" implies that not doing this steps also means that all >>> preceding steps do not need to be carried out (unless required >>> by other steps), while "pass" implies only that this step requires >>> no processing and hence the output of the preceding step >>> can be used. >>> >>> -phi >>> >>> On Tue, Aug 21, 2012 at 1:23 PM, Karel Bílek<[email protected]> wrote: >>>> By the way, one more question. (I am new to EMS.) >>>> >>>> What is the difference between "ignore-if" and "pass-unless"? >>>> >>>> K >>>> >>>> On Tue, Aug 21, 2012 at 10:16 PM, Karel Bílek<[email protected]> wrote: >>>>> Thanks for tips! >>>>> >>>>> Yes, I do want to use the same pruned tables for tuning and testing. >>>>> Currently, I do have only one translation table. >>>>> >>>>> KB >>>>> >>>>> On Tue, Aug 21, 2012 at 10:13 PM, Philipp Koehn<[email protected]> >>>>> wrote: >>>>>> Hi, >>>>>> >>>>>> there are several places where you could do this, depending >>>>>> on what your goal is. >>>>>> >>>>>> If you want use the same pruned phrase table for tuning >>>>>> and testing, it makes sense to add such a step into the >>>>>> model training part. >>>>>> >>>>>> Currently, there is a phrase translation table step: >>>>>> >>>>>> build-ttable >>>>>> in: extracted-phrases lexical-translation-table domains >>>>>> out: phrase-translation-table >>>>>> >>>>>> which feeds into the configuration file creation step: >>>>>> >>>>>> create-config >>>>>> in: reordering-table phrase-translation-table generation-table >>>>>> sparse-lexical domains INTERPOLATED-LM:binlm LM:binlm >>>>>> out: config >>>>>> >>>>>> So, you could put something inbetween, but changing the >>>>>> input of the create-config step into: >>>>>> >>>>>> create-config >>>>>> in: reordering-table pruned-phrase-translation-table >>>>>> generation-table sparse-lexical domains INTERPOLATED-LM:binlm LM:binlm >>>>>> out: config >>>>>> >>>>>> and creating a step >>>>>> >>>>>> prune-phrase-table >>>>>> in: phrase-translation-table >>>>>> out: pruned-phrase-translation-table >>>>>> >>>>>> However, this may get a bit messy when you have multiple phrase >>>>>> translation >>>>>> tables (due to factored models, for instance), and lexicalized >>>>>> reordering models >>>>>> which have to have the same phrase entries as the phrase translation >>>>>> table. >>>>>> In that case, it would better to place the pruning step after the >>>>>> "create-config" >>>>>> >>>>>> prune-tables >>>>>> in: config >>>>>> out: pruned-config >>>>>> >>>>>> and then have the subsequent steps take pruned-config as input. >>>>>> >>>>>> -phi >>>>>> >>>>>> >>>>>> On Tue, Aug 21, 2012 at 12:55 PM, Karel Bílek<[email protected]> >>>>>> wrote: >>>>>>> Hello everyone, >>>>>>> >>>>>>> I have a question about EMS. >>>>>>> >>>>>>> I want to prune the phrase table right before TUNING step. I already >>>>>>> do have a script for that; but I don't know where to "plug" it into >>>>>>> config EMS file. >>>>>>> >>>>>>> I can do the pruning "by hand" and then write the address directly >>>>>>> into "phrase-translation-table" but that seems to be against the >>>>>>> spirit of EMS. >>>>>>> >>>>>>> Thanks, >>>>>>> Karel Bílek >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Moses-support mailing list >>>>>>> [email protected] >>>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
