Jorn,
as I wrote i have created the ner model, but when i try to use it via
"opennlp TokenNameFinder" tool it can't locate my custom FeatureGenerator.
This is the output:
bash opennlp TokenNameFinder /home/damiano/custom-ner-model.bin
Loading Token Name Finder model ... done (5,948s)
Exception in thread "main"
opennlp.tools.util.ext.ExtensionNotLoadedException: Unable to find
implementation for opennlp.tools.util.featuregen.AdaptiveFeatureGenerator,
the class or service
com.damiano.parser.generator.SpanWindowFeatureGenerator could not be
located!
at
opennlp.tools.util.ext.ExtensionLoader.instantiateExtension(ExtensionLoader.java:122)
at
opennlp.tools.util.featuregen.GeneratorFactory$CustomFeatureGeneratorFactory.create(GeneratorFactory.java:582)
at
opennlp.tools.util.featuregen.GeneratorFactory.createGenerator(GeneratorFactory.java:661)
at
opennlp.tools.util.featuregen.GeneratorFactory$AggregatedFeatureGeneratorFactory.create(GeneratorFactory.java:129)
at
opennlp.tools.util.featuregen.GeneratorFactory.createGenerator(GeneratorFactory.java:661)
at
opennlp.tools.util.featuregen.GeneratorFactory$CachedFeatureGeneratorFactory.create(GeneratorFactory.java:171)
at
opennlp.tools.util.featuregen.GeneratorFactory.createGenerator(GeneratorFactory.java:661)
at
opennlp.tools.util.featuregen.GeneratorFactory$AggregatedFeatureGeneratorFactory.create(GeneratorFactory.java:129)
at
opennlp.tools.util.featuregen.GeneratorFactory.createGenerator(GeneratorFactory.java:661)
at
opennlp.tools.util.featuregen.GeneratorFactory.create(GeneratorFactory.java:711)
at
opennlp.tools.namefind.TokenNameFinderFactory.createFeatureGenerators(TokenNameFinderFactory.java:153)
at
opennlp.tools.namefind.TokenNameFinderFactory.createContextGenerator(TokenNameFinderFactory.java:118)
at opennlp.tools.namefind.NameFinderME.<init>(NameFinderME.java:90)
at
opennlp.tools.cmdline.namefind.TokenNameFinderTool.run(TokenNameFinderTool.java:59)
at opennlp.tools.cmdline.CLI.main(CLI.java:227)
Why does the custom generator is not included on the model?
This is the my xml descriptor:
<generators>
<cache>
<generators>
<window prevLength="6" nextLength="3">
<tokenclass/>
</window>
<window prevLength="6" nextLength="3">
<token/>
</window>
<definition/>
<prevmap/>
<bigram/>
<sentence begin="true" end="false"/>
<dictionary prefix="name" dict="names.dictionary"/>
<custom
class="com.damiano.parser.generator.SpanWindowFeatureGenerator"
leftTokens="6" rightTokens="3"/>
</generators>
</cache>
</generators>
Damiano
2016-10-28 14:00 GMT+02:00 Damiano Porta <damianopo...@gmail.com>:
> Pardon, my wrong, i forgot to change <dictionary prefix="city"
> dict="damiano"/> into <dictionary prefix="city"
> dict="damiano.dictionary"/> in my train.xml
>
> now it is working well! and the .bin has my dictionary too
>
> 2016-10-28 13:51 GMT+02:00 Damiano Porta <damianopo...@gmail.com>:
>
>> Jorn
>> i change the code as you told me, this exactly:
>> https://gist.github.com/anonymous/8877b09d441d2e64c181fa9b5a
>> de4550#file-test-java-L15
>>
>> but i get this error:
>>
>> opennlp.tools.namefind.TokenNameFinderModel$FeatureGeneratorCreationError:
>> opennlp.tools.util.InvalidFormatException: No dictionary resource for
>> key: damiano
>> at opennlp.tools.namefind.TokenNameFinderFactory.createFeatureG
>> enerators(TokenNameFinderFactory.java:176)
>> at opennlp.tools.namefind.TokenNameFinderFactory.createContextG
>> enerator(TokenNameFinderFactory.java:118)
>> at opennlp.tools.namefind.NameFinderME.train(NameFinderME.java:333)
>> at com.damiano.parser.trainer.NER.compileNER(NER.java:185)
>> at com.damiano.parser.trainer.NER.main(NER.java:155)
>>
>> Caused by: opennlp.tools.util.InvalidFormatException: No dictionary
>> resource for key: damiano
>> at opennlp.tools.util.featuregen.GeneratorFactory$DictionaryFea
>> tureGeneratorFactory.create(GeneratorFactory.java:251)
>> at opennlp.tools.util.featuregen.GeneratorFactory.createGenerat
>> or(GeneratorFactory.java:661)
>> at opennlp.tools.util.featuregen.GeneratorFactory$AggregatedFea
>> tureGeneratorFactory.create(GeneratorFactory.java:129)
>> at opennlp.tools.util.featuregen.GeneratorFactory.createGenerat
>> or(GeneratorFactory.java:661)
>> at opennlp.tools.util.featuregen.GeneratorFactory$CachedFeature
>> GeneratorFactory.create(GeneratorFactory.java:171)
>> at opennlp.tools.util.featuregen.GeneratorFactory.createGenerat
>> or(GeneratorFactory.java:661)
>> at opennlp.tools.util.featuregen.GeneratorFactory$AggregatedFea
>> tureGeneratorFactory.create(GeneratorFactory.java:129)
>> at opennlp.tools.util.featuregen.GeneratorFactory.createGenerat
>> or(GeneratorFactory.java:661)
>> at opennlp.tools.util.featuregen.GeneratorFactory.create(Genera
>> torFactory.java:711)
>> at opennlp.tools.namefind.TokenNameFinderFactory.createFeatureG
>> enerators(TokenNameFinderFactory.java:153)
>> ... 4 more
>>
>> 2016-10-28 12:55 GMT+02:00 Joern Kottmann <kottm...@gmail.com>:
>>
>>> Try to rename the dictionary key to xyz.dictionary then the serializer
>>> will
>>> be mapped correctly.
>>>
>>> Jörn
>>>
>>> On Thu, Oct 27, 2016 at 11:14 PM, Damiano Porta <damianopo...@gmail.com>
>>> wrote:
>>>
>>> > Jorn i add the Dictionary here:
>>> > https://gist.github.com/anonymous/bc822fb0520c4c42b75748bf4147da
>>> > 34#file-train-java-L15
>>> >
>>> > And unfortunately i only see this error:
>>> >
>>> > java.lang.IllegalStateException: Missing serializer for damiano
>>> > at opennlp.tools.util.model.BaseModel.serialize(BaseModel.java:610)
>>> >
>>> > I do not have other info.
>>> > Do i have to create a custom Serializer too?
>>> >
>>> >
>>> >
>>> >
>>> > 2016-10-27 22:04 GMT+02:00 Joern Kottmann <kottm...@gmail.com>:
>>> >
>>> > > On Thu, 2016-10-27 at 21:18 +0200, Joern Kottmann wrote:
>>> > > > On Tue, 2016-10-25 at 18:49 +0200, Damiano Porta wrote:
>>> > > > >
>>> > > > > i am getting a strange error during the compiling of a NER model.
>>> > > > > Basically, the end of the build output is:
>>> > > > >
>>> > > > > 98: ... loglikelihood=-13340.018762351776 0.999005934601099
>>> > > > > 99: ... loglikelihood=-13258.358751926637 0.9990120681028991
>>> > > > > 100: ... loglikelihood=-13178.039964721707 0.9990177634974279
>>> > > > > Exception in thread "main" java.lang.IllegalStateException:
>>> Missing
>>> > > > > serializer for *mydictionary*
>>> > > > > at opennlp.tools.util.model.BaseModel.serialize(BaseModel.java:
>>> 610)
>>> > > >
>>> > > >
>>> > > > Can you please post the full exception stack trace?
>>> > > >
>>> > >
>>> > >
>>> > > And what is the name of they key you used for the dictionary?
>>> > > The dictionary serializers are only mapped by extension.
>>> > >
>>> > > Jörn
>>> > >
>>> >
>>>
>>
>>
>
