Alex, There seems to be a discrepancy between the scoring function in the documentation and in the code. The figure in the documentation has the window being limited by relative positions -3 and 0. The scorer.py file sets the left edge as -1. It also assigned a value of 0.88 to the center while the documentation figure has a flat value of 1 from far left to center.
Should I assume this part of the documentation obsolete and go with the code instead? thanks, Nick > On Apr 22, 2015, at 6:34 PM, Alex Lavin <[email protected]> wrote: > > Hi Nick, > Comparing raw scores vs the normalized scores is really a matter of > preference; the normalized scores, with a set range of 0-100, may be easier > to comprehend. > > Precision, recall, and their harmonic mean, F1 score, are helpful evaluation > functions for most machine learning algorithms. They don't, however, > incorporate time in the calculations, and are thus unsuitable for evaluating > the ability of an algorithm to perform on real-time, streaming data. A main > motivation for NAB was to design a scoring system which incorporates time and > the TP,TN,FN, and FP counts. I wouldn't call this problematic, but rather > desirable that NAB scoring doesn't mimic other evaluation functions. > > Cheers, > Alex > > Alexander Lavin > Software Engineer > Numenta
