Hi, sigtest filter was added to EMS in branch damt_phrase, you can check it out. There's also a script for its parallelization.
I didn't merge this into master because the sigtest filter still depends on SALM (which in turn doesn't compile on Linux unless extra header files are included in the source code) -- but maybe it's worth merging anyway? Best, Ales Dne 22.8.2012 01:14, Karel Bílek napsal(a): > OK. > > I will add support of sigtest-filter ( > http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc19 ) which > ships with moses to my EMS experiment.meta ; I will add this to moses > on git as another pull request as I properly test it > > It will use the first case I mentioned because I don't use factors and > I don't understand them as well :) > > BTW > there is a slight error in here > http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc19 > Build Instructions > > you can't just build it with "make", since it will work only with g++ > 4.1 and not newer, you have to edit the makefile > also, it's not in "directory sigtest-filter", but in "contrib/sigtest-filter". > > Karel > > On Tue, Aug 21, 2012 at 10:31 PM, Philipp Koehn<[email protected]> wrote: >> Hi, >> >> "ignore" implies that not doing this steps also means that all >> preceding steps do not need to be carried out (unless required >> by other steps), while "pass" implies only that this step requires >> no processing and hence the output of the preceding step >> can be used. >> >> -phi >> >> On Tue, Aug 21, 2012 at 1:23 PM, Karel Bílek<[email protected]> wrote: >>> By the way, one more question. (I am new to EMS.) >>> >>> What is the difference between "ignore-if" and "pass-unless"? >>> >>> K >>> >>> On Tue, Aug 21, 2012 at 10:16 PM, Karel Bílek<[email protected]> wrote: >>>> Thanks for tips! >>>> >>>> Yes, I do want to use the same pruned tables for tuning and testing. >>>> Currently, I do have only one translation table. >>>> >>>> KB >>>> >>>> On Tue, Aug 21, 2012 at 10:13 PM, Philipp Koehn<[email protected]> >>>> wrote: >>>>> Hi, >>>>> >>>>> there are several places where you could do this, depending >>>>> on what your goal is. >>>>> >>>>> If you want use the same pruned phrase table for tuning >>>>> and testing, it makes sense to add such a step into the >>>>> model training part. >>>>> >>>>> Currently, there is a phrase translation table step: >>>>> >>>>> build-ttable >>>>> in: extracted-phrases lexical-translation-table domains >>>>> out: phrase-translation-table >>>>> >>>>> which feeds into the configuration file creation step: >>>>> >>>>> create-config >>>>> in: reordering-table phrase-translation-table generation-table >>>>> sparse-lexical domains INTERPOLATED-LM:binlm LM:binlm >>>>> out: config >>>>> >>>>> So, you could put something inbetween, but changing the >>>>> input of the create-config step into: >>>>> >>>>> create-config >>>>> in: reordering-table pruned-phrase-translation-table >>>>> generation-table sparse-lexical domains INTERPOLATED-LM:binlm LM:binlm >>>>> out: config >>>>> >>>>> and creating a step >>>>> >>>>> prune-phrase-table >>>>> in: phrase-translation-table >>>>> out: pruned-phrase-translation-table >>>>> >>>>> However, this may get a bit messy when you have multiple phrase >>>>> translation >>>>> tables (due to factored models, for instance), and lexicalized reordering >>>>> models >>>>> which have to have the same phrase entries as the phrase translation >>>>> table. >>>>> In that case, it would better to place the pruning step after the >>>>> "create-config" >>>>> >>>>> prune-tables >>>>> in: config >>>>> out: pruned-config >>>>> >>>>> and then have the subsequent steps take pruned-config as input. >>>>> >>>>> -phi >>>>> >>>>> >>>>> On Tue, Aug 21, 2012 at 12:55 PM, Karel Bílek<[email protected]> wrote: >>>>>> Hello everyone, >>>>>> >>>>>> I have a question about EMS. >>>>>> >>>>>> I want to prune the phrase table right before TUNING step. I already >>>>>> do have a script for that; but I don't know where to "plug" it into >>>>>> config EMS file. >>>>>> >>>>>> I can do the pruning "by hand" and then write the address directly >>>>>> into "phrase-translation-table" but that seems to be against the >>>>>> spirit of EMS. >>>>>> >>>>>> Thanks, >>>>>> Karel Bílek >>>>>> >>>>>> _______________________________________________ >>>>>> Moses-support mailing list >>>>>> [email protected] >>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
