Hi Rasul, that makes sense: the relaxation script writes the output in a slightly different style with the unannotated sentence at the start of the line followed by the XML tree annotation, and the filtering script should be able to handle that. I don't think it will work with the unrelaxed input though.
Phil On 5 Aug 2012, at 02:24, [email protected] wrote: > Hi Philip, > > I don't touch the format. I just use the output of relaxation script for > filtering. I checked it for tree-to-string and tree-to-tree models. I > translated with filtered and original tables for both models and the results > were exactly the same. I was not so however when I used the plain input for > filtering (but relaxed one for translation). So, we could conclude that > filter-model-given-input.pl works with all tree-based models implemented in > Moses, but the input for filtering should be exactly the one used for > translation. > > Thanks and regards, > Rasul. > > Quoting Philip Williams <[email protected]>: > >> Hi Rasul, >> >> for the tree-to-string model, if you stripped the XML out of the input file >> before running filter-model-given-input.pl then I think that should have >> worked OK. The filtered table would contain some rules that could >> potentially have been filtered out (if the tree annotation had been >> available), but the script shouldn't have filtered out usable rules. >> >> Phil >> >> >> On 3 Aug 2012, at 13:19, [email protected] wrote: >> >>> Hi Philip, >>> >>> Thanks for the information. I tried filter-model-given-input.pl on >>> tree-to-string and it worked. In fact, I had been previously using it for >>> this model and it used to work. However, I needed to filter against a >>> SAMT-relaxed input and was not sure. So, I did that, translated the input >>> using the filtered table, and compared to a translation with original >>> table; those were the same, then I concluded that it can be used for >>> source-tree models as well. I hope I have not made a mistake, though! >>> >>> Do you have any idea on this, as you are the author of filter-rule-table.py? >>> >>> Best Regards, >>> Rasul. >>> >>> Quoting Philip Williams <[email protected]>: >>> >>>> Hi Rasul, >>>> >>>> the filter-rule-table.py script can filter string-to-tree or >>>> hierarchical rule tables. The filter-model-given-input.pl script (with >>>> the --Hierarchical option) calls the former script to perform filtering >>>> in addition to performing some extra steps that are required in the >>>> standard Moses MT pipeline (such as generating a new Moses config file). >>>> >>>> I'm assuming those are the two scripts you meant. As far as I know, >>>> there aren't any scripts for filtering tree-to-string or tree-to-tree >>>> tables yet... >>>> >>>> Phil >>>> >>>> >>>> On 2 Aug 2012, at 11:35, [email protected] wrote: >>>> >>>>> Hi all, >>>>> >>>>> I was wondering which script is used for filtering tree-based rule >>>>> tables, including hierarchical, tree-to-string, string-to-tree, and >>>>> tree-to-tree. There are two filtering scripts in training scripts >>>>> directory. >>>>> >>>>> Thanks in advance. >>>>> >>>>> Regards, >>>>> Rasul. >>>>> _______________________________________________ >>>>> Moses-support mailing list >>>>> [email protected] >>>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>>> >>>> >>> >> >> > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
