On 10/08/2013 12:45 AM, Michael Schmitz wrote:
Hi Jorn, let me be more precise. Do you have a notion of how the
precision-recall curve (AUC) changes as a function of the number of
annotations? I'm curious how many annotations are needed for a model
with reasonable precision-recall AUC and reasonable performance
(memory and speed).
No, I don't, you need to write a class which trains and test many times with
different amounts of training data.
Maybe we should make this use case really easy and add some kind of
experimenter
support to our components. The experimenter could take a class which
provides configuration
for the trainer depending on the iteration, the results of one iteration
could be recorded in a csv or
text file which can then later be analyzed with tools like Excel, R, etc.
I often run the name finder with slightly modified feature generation to
find a setup which works
with my data, this could probably be automated quite a bit.
Jörn
