yep, i'm not sure if the filtering script has been tested with tree-to-string models so it may not work properly.
if you're trying to minimize memory usage, you could use the on-disk rule-table. That works with tree-to-string On 03/02/2012 18:04, [email protected] wrote: > Hi Hieu, > > Thanks for reply. I used the option for glue rules in training and > -Hierarchical for filtering inside mert, and it solved the problem. > > However, I faced some puzzling results: the BLEU score shown in > tuning/moses.ini was normal. But when I filtered the rule table > separately and passed it to moses-chart, the BLEU score was strangely > low. Therefore, I decided to test with original (non-filtered) rule > table, and it worked. What is surprising for me here is that both > decoding procedures, inside mert and final translation, used filtered > tables, but produced very different results. Although filtering for > each of them was based on different input (development and test sets), > these inputs were similar and randomly chosen from a single data set. > I have not yet been able to figure it out. > > Thanks again. > > Best Regards, > Rasul. > > Quoting Hieu Hoang <[email protected]>: > >> hi razul >> >> 1. yes, you need the glue rules. For tree-to-string models, they are the >> same as for hierachical models. If the script produces anything >> different, >> then it's probably wrong >> 2. Probably yes. -Hierarchical is for syntactic models generally but >> no-one >> has had time to change the name of the argument. >> >> Not a lot of people are investigating tree-to-string models so some >> of the >> scripts may work as intended. Double check your data at each step. If >> you >> manage to fix scripts to work with tree-to-string models, please >> recommit >> them to github >> >> >> On Fri, Feb 3, 2012 at 8:59 AM, <[email protected]> wrote: >> >>> Hi all, >>> >>> I'm training a syntax-based tree-to-string model. I appreciate your >>> answers to these questions: >>> >>> - Do I need to use --glue-grammar option when training as in >>> hierarchical models? >>> - Do I need to use -Hierarchical option in passing --filtercmd option >>> to mert? The name implies that it should be used for hierarchical >>> models. However, when dropping it, segmentation fault occurs. >>> >>> Thank you very much in advance. >>> >>> Best Regards, >>> Rasul. >>> _______________________________________________ >>> Moses-support mailing list >>> [email protected] >>> http://mailman.mit.edu/mailman/listinfo/moses-support >>> >> > > > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
