Thanks! Regarding the LemmatizerME getting stuck. Should i open an issue?
Markus -----Original message----- > From:Rodrigo Agerri <rodrigo.age...@ehu.eus> > Sent: Wednesday 26th February 2020 13:47 > To: users@opennlp.apache.org > Subject: Re: LemmatizerTrainerME gets stuck with GaussianSmoothing enabled > > https://opennlp.apache.org/docs/1.8.0/manual/opennlp.html#tools.lemmatizer.training > > The example of training is with a perceptron. An example (with default > features) can be found here: > > https://github.com/apache/opennlp/blob/master/opennlp-tools/lang/ml/PerceptronTrainerParams.txt > > Best, > > R > > On Wed, 26 Feb 2020 at 11:02, Markus Jelsma <markus.jel...@openindex.io> > wrote: > > > > Hello Rodrigo, > > > > Although i am using the Java API mainly, i use the CLI tools for the > > training jobs. And although there is some sporadic mention of Perceptron in > > the manual, i have not been able to find any CLI tool that supports > > training a Perceptron type model. > > > > How do you do it? I use v1.9.2. > > > > Thanks, > > Markus > > > > -----Original message----- > > > From:Rodrigo Agerri <rage...@apache.org> > > > Sent: Friday 21st February 2020 16:42 > > > To: users@opennlp.apache.org > > > Subject: Re: LemmatizerTrainerME gets stuck with GaussianSmoothing enabled > > > > > > Hello, > > > > > > I do not know why is this happening, but while testing the lemmatizer > > > the best performance (word and sentence accuracy) was always obtained > > > using the Perceptron trainer (for a number of languages), so I would > > > recommend to train perceptron models (perhaps you have already tried > > > this, but just in case). > > > > > > Best, > > > > > > R > > > > > > > > > > > > > > > > > > > > > On Fri, 21 Feb 2020 at 16:07, Markus Jelsma <markus.jel...@openindex.io> > > > wrote: > > > > > > > > Hello, > > > > > > > > When GaussianSmoothing is enabled, the LemmatizerTrainerME gets stuck. > > > > The loglikelihood gets warped to NaN right after the first iteration, > > > > like so: > > > > Computing model parameters in 16 threads... > > > > Performing 10000 iterations. > > > > 1: ... loglikelihood=-1238383.3965208859 0.8032247485201534 > > > > 2: ... loglikelihood=NaN 0.8178054720724305 > > > > 3: ... loglikelihood=NaN 0.8032247485201534 > > > > 4: ... loglikelihood=NaN 0.8032247485201534 > > > > > > > > Is this known to happen? Did i stumble upon some bug? > > > > > > > > Many thanks, > > > > Markus > > >
