Our default cutoff of 5 doesn't really work when
you just have 200 sentences of training data.
Jörn
On 12/7/11 5:46 PM, Vyacheslav Zholudev wrote:
Hi Em,
thanks for sharing your experience.
I can't give you any suggestions since I switched to another tool for
named-entity recognition (called CRF++). It implements conditional random
fields which allows to tag multiple types of entities, e.g. location and
organization. This particular tool is kinda low-level: you have to convert your
training data and a sentence to be tagged first to a specific format, but
overall this tool works really well for me even on a small number of annotated
examples (200 sentences for my particular use case)
Vyacheslav
On Dec 4, 2011, at 2:12 PM, Em wrote:
Hi Vyacheslav,
I started using OpenNLP in my free-time. As I promised, I share my
outcomes of some tests - although I did not hit the 2.500
tagged-sentences due to time-reasons.
Unfortunately I didn't had the time to backup my outcomes with numbers,
statistics or something like that (since I just played around with data
and OpenNLP).
However I saw a correlation between the length of the training-data and
the length of the data you want to tag with OpenNLP.
I took several sub-passages of articles (let's call them test-set) of
training data and tried to tag them.
The result of my test was: If your training data's average length was
"long" and you test it with short test-sets, precision and recall are
relatively bad.
For example I took 10 sentences of my training-data and tried out to tag
them, starting with using just one sentence and going to go up to 10.
Additionally I rewrote the sentences so that their sense would be the
same to a human, while looking completely different in comparison to the
original sentences. The results were the same.
If openNLP detected something in one-sentence-examples, it often was one
of 2-3 entities. Some sentences remained untouched although they
contained entities.
However, if I combined more than one sentence with eachother openNLP
became much more accurate (since they formed a more typical
document-length than the one-sentence-examples).
Training-data:
My training-data contained passages of wiki-articles. The average length
was several sentences per document. To me a document was everything that
belongs to a specific headline in a wikipedia-article, grouped by the
most specific headline.
So if a H3-tag contained several H4-tags, every content belonging to
that H4-tag formed a document.
As a consequence there were several longer documents, since h4-tags were
rarely used.
I decided to create a document for every most-specific headline, because
the wiki contained documents of every kind of length - some to fill
short books with 10 or more pages, others with just one sentences per
article. Choosing the most-specific headline to form a document made it
possible to have a good mix of document lengths.
I am glad to hear your feedback!
Hope this helps,
Em
Am 05.10.2011 23:50, schrieb Vyacheslav Zholudev:
Hi Em,
could you please share the outcome when you have some results. I would be
interested to hear them
Thanks,
Vyacheslav
On Oct 5, 2011, at 11:08 PM, Em wrote:
Thanks Jörn!
I'll experiment with this.
Regards,
Em
Am 05.10.2011 19:47, schrieb Jörn Kottmann:
On 10/3/11 10:30 AM, Em wrote:
What about document's length?
Just as an example: The production-data will contain documents with a
length of several pages as well as very short texts containing only a
few sentences.
I think about chunking the long documents into smaller ones (i.e. a page
of a longer document is splitted into an individual doc). Does this
makes sense?
I would first try to process a long document at once. If you encounter any
issues you could just call clearAdaptiveData before the end of the
document.
But as Olivier said, you might just want to include a couple of these in
your training
data.
Jörn
Best,
Vyacheslav
