When was the last time you checked out Moses? As of fairly recently, BilingualDynSuffixArray reports 5 features back. I checked in a bug fix for computation of the lexical scores yesterday or so, so you should check out the latest version. The backward probability is computed differently, so you'll need to re-tune your weights. The code under the hood is going to change in the foreseeable future, as I am switching to memory-mapped suffix arrays for persistent storage instead of building the suffix array every time at load time, which is grossly inefficient.
As far as I'm aware of it, nothing has been done yet about dynamic distortion tables. - Uli On Mon, Jul 29, 2013 at 10:46 PM, Marcin Junczys-Dowmunt <[email protected] > wrote: > Trying to undesrtand some more of the code in BilingualDynSuffixArray. I > see that the dynamic suffix array provides only one translation > pobability and one lexical weight, this is for perfomance reasons? > Also, what about reordering, do you only use the distance weighted > model? I guess I could try to build a dynamic reordering model, too. > > W dniu 24.07.2013 15:36, Hieu Hoang pisze: > > Oh, the lucene database stores the aligned corpus, rather than the > > trained phrase table? Uli knows more about the suffix array. yep, > > there is no easy way! > > > > On 24 July 2013 14:32, Marcin Junczys-Dowmunt <[email protected] > > <mailto:[email protected]>> wrote: > > > > Thanks, I am mainly interested in reusing most of the score > > calculation methods in BilingualDynSuffixArray.* and was looking > > for an easy way to do that. Now that I have inspected the code for > > a while I see there is no easy way :) > > > > W dniu 24.07.2013 15:29, Hieu Hoang pisze: > >> yes, copy PhraseDictionaryDynSuffixArray, or indeed, your own > >> PhraseDictionaryCompact. > >> > >> there's some docs on adding feature functions. > >> http://www.statmt.org/moses/?n=Moses.FeatureFunctions > >> > >> it should be easier than a year ago when you added > >> PhraseDictionaryCompact > >> > >> On 24 July 2013 13:29, Marcin Junczys-Dowmunt <[email protected] > >> <mailto:[email protected]>> wrote: > >> > >> Hi list, > >> I am planning to integrate a dynamic phrase table based on a > >> lucene > >> index and I am wondering how to approach it. Basically it > >> should have > >> the same functionality as the Dynamic Suffix Array already > >> present in > >> Moses. Now I am wondering if for a first working version it > >> would be > >> enough just to hijack DynSuffixArray.h and implement all the > >> public > >> functions with CLucene. Any hidden pitfalls there? The index > >> would be > >> indexed by the source sentence and contain alignment data and > >> target > >> sentences as fields. > >> Best, > >> Marcin > >> _______________________________________________ > >> Moses-support mailing list > >> [email protected] <mailto:[email protected]> > >> http://mailman.mit.edu/mailman/listinfo/moses-support > >> > >> > >> > >> > >> -- > >> Hieu Hoang > >> Research Associate > >> University of Edinburgh > >> http://www.hoang.co.uk/hieu > >> > > > > > > > > > > -- > > Hieu Hoang > > Research Associate > > University of Edinburgh > > http://www.hoang.co.uk/hieu > > > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > > -- Ulrich Germann Research Associate School of Informatics University of Edinburgh
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
