Hi, we also have an experiment on truecasing, see Table 1 in http://www.statmt.org/wmt13/pdf/WMT08.pdf
What works best for us is relying on the casing as guessed by the lemmatizer. (Our lemmatizer recognizes names as separate lemmas and keeps the lemma upcased; which we then cast to the token in the sentence.) Moses recaser was the worst option, it was actually better to lowercase only the source side of the parallel data, i.e. have the main search also pick the casing. Cheers, O. ----- Original Message ----- > From: "Lane Schwartz" <dowob...@gmail.com> > To: "Philipp Koehn" <p...@jhu.edu> > Cc: moses-support@mit.edu > Sent: Wednesday, 20 May, 2015 20:50:41 > Subject: Re: [Moses-support] When to truecase > Got it. So then, how was casing handled in the "mbr/mp" column? Was all of > the data lowercased, then models trained, then recasing applied after > decoding? Or something else? > > On Wed, May 20, 2015 at 1:30 PM, Philipp Koehn <p...@jhu.edu> wrote: > >> Hi, >> >> no, the changes are made incrementally. >> >> So the recesed "baseline" is the previous "mbr/mp" column. >> >> -phi >> >> On Wed, May 20, 2015 at 2:01 PM, Lane Schwartz <dowob...@gmail.com> wrote: >> >>> Philipp, >>> >>> In Table 2 of the WMT 2009 paper, are the "baseline" and "truecased" >>> columns directly comparable? In other words, do the two columns indicate >>> identical conditions other than a single variable (how and/or when casing >>> was handled)? >>> >>> In the baseline condition, how and when was casing handled? >>> >>> Thanks, >>> Lane >>> >>> >>> On Wed, May 20, 2015 at 12:43 PM, Philipp Koehn <p...@jhu.edu> wrote: >>> >>>> Hi, >>>> >>>> see Section 2.2 in our WMT 2009 submission: >>>> http://www.statmt.org/wmt09/pdf/WMT-0929.pdf >>>> >>>> One practical reason to avoid recasing is the need >>>> for a second large cased language model. >>>> >>>> But there is of course also the practical issue with >>>> have a unique truecasing scheme for each data >>>> condition, handling of headlines, all-caps emphasis, >>>> etc. >>>> >>>> It would be worth to revisit this issue again under >>>> different data conditions / language pairs. Both >>>> options are readily available in EMS. >>>> >>>> Each of the two alternative methods could be >>>> improved as well. See for instance: >>>> http://www.aclweb.org/anthology/N06-1001 >>>> >>>> -phi >>>> >>>> -phi >>>> >>>> >>>> On Wed, May 20, 2015 at 12:31 PM, Lane Schwartz <dowob...@gmail.com> >>>> wrote: >>>> >>>>> Philipp (and others), >>>>> >>>>> I'm wondering what people's experience is regarding when truecasing is >>>>> applied. >>>>> >>>>> One option is to truecase the training data, then train your TM and LM >>>>> using that truecased data. Another option would be to lowercase the data, >>>>> train TM and LM on the lowercased data, and then perform truecasing after >>>>> decoding. >>>>> >>>>> I assume that the former gives better results, but the latter approach >>>>> has an advantage in terms of extensibility (namely if you get more data >>>>> and >>>>> update your truecase model, you don't have to re-train all of your TMs and >>>>> LMs). >>>>> >>>>> Does anyone have any insights they would care to share on this? >>>>> >>>>> Thanks, >>>>> Lane >>>>> >>>>> >>>>> _______________________________________________ >>>>> Moses-support mailing list >>>>> Moses-support@mit.edu >>>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>>>> >>>>> >>>> >>> >>> >>> -- >>> When a place gets crowded enough to require ID's, social collapse is not >>> far away. It is time to go elsewhere. The best thing about space travel >>> is that it made it possible to go elsewhere. >>> -- R.A. Heinlein, "Time Enough For Love" >>> >>> _______________________________________________ >>> Moses-support mailing list >>> Moses-support@mit.edu >>> http://mailman.mit.edu/mailman/listinfo/moses-support >>> >>> >> > > > -- > When a place gets crowded enough to require ID's, social collapse is not > far away. It is time to go elsewhere. The best thing about space travel > is that it made it possible to go elsewhere. > -- R.A. Heinlein, "Time Enough For Love" > > _______________________________________________ > Moses-support mailing list > Moses-support@mit.edu > http://mailman.mit.edu/mailman/listinfo/moses-support -- Ondrej Bojar (mailto:o...@cuni.cz / bo...@ufal.mff.cuni.cz) http://www.cuni.cz/~obo _______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
