On 04/17/2012 05:30 PM, Jim - FooBar(); wrote:
I think the root of all our problems here is the fact that we're
trying to generalise something that was not intended for that purpose.
If you remember, before writing the AggregateNameFinder i had similar
code in the TokeNameFinderEvaluator class. After your suggestion i
moved it to the name-find package but i had strong objections for
doing so because the merging of results should be an evaluation issue
only. Now however we've dug ourselves a hole...now, we're saying to
the user "you can use the aggregate finder instead of the individual
ones but we will decide on some things for you!" ...see? it is no
longer an evaluation issue - it has become a separate name-finder that
people might use for annotating their corpus which can lead to strange
behaviour if we don't resolve nested tags. We tried to improve the
evaluation and we've ended up discussing hard-coded rules as to how
such a name-finder must behave...
The whole point of the evaluation is to measure how good a specific
name finder setup performs on some test data. The evaluator should not help
in anyway with producing these names, because that is the responsibility of
the name finder. That is one reason.
Another is that people want to run the name finders exactly the same way
as they did run in the evaluation later to produce names. Therefor they must
merge the names in the same manner as it was done during evaluation.
But if the logic is hard coded into the evaluator it must be duplicated.
For these reasons I think that the merging logic
does not belong into the evaluators.
A separate name finder is indeed a good place for it because it solves the
two points explained above and makes it easily interchangeable against a
different
implementation.
+1 from me to implement a simple baseline merger as well.
Jörn
