I am talking about results from one run, from this loop. I saw 6 different results. I can't think of any likely reason that would cause it to return the same results. The only thing I can imagine is that something is resetting the RNG each time, since our tests are able to do that, but I don't see anything like that triggere dhere.
On Thu, Sep 29, 2011 at 10:56 AM, Lance Norskog <[email protected]> wrote: > Yes, separate runs from the command line or Eclipse vary by +/- a bit. It > is > only when I run this loop inside one program run that I see the behavior. > > On Thu, Sep 29, 2011 at 12:53 AM, Sean Owen <[email protected]> wrote: > > > I don't see this behavior. Each result is slightly different by about > +/-2% > > when I run it, as expected. > > > > On Thu, Sep 29, 2011 at 7:15 AM, Lance Norskog <[email protected]> > wrote: > > > > > If you run this version of GroupLensRecommenderEvaluatorRunner, an odd > > > thing > > > happens. The first evaluation will be number X, the second will be Y, > the > > > third will be Y, ..., the Nth will be Y. The same Y as the second and > the > > > third and the fourth. The RecommenderEvaluator implementation is > > > stochastic, > > > but this should be a different number each time. Anyone know how this > > could > > > happen? (You can load the DataModel outside the loop, the numbers are > the > > > same.) > > > > > > public static void main(String... args) throws IOException, > > > TasteException, > > > OptionException { > > > File ratingsFile = TasteOptionParser.getRatings(args); > > > long first = System.currentTimeMillis(); > > > long last = first; > > > for(int i = 0; i < 30; i++) { > > > DataModel model = ratingsFile == null ? new GroupLensDataModel() : > > new > > > GroupLensDataModel(ratingsFile); > > > RecommenderEvaluator evaluator = new > > > AverageAbsoluteDifferenceRecommenderEvaluator(); > > > double evaluation = evaluator.evaluate(new > > > GroupLensRecommenderBuilder(), > > > null, > > > model, > > > 0.9, > > > 0.3); > > > long now = System.currentTimeMillis(); > > > log.info("Result #{}: {}", i, String.valueOf(evaluation)); > > > log.info("\t{} ms", now - last); > > > last = System.currentTimeMillis(); > > > } > > > log.info("Total time: {} ms", last - first); > > > } > > > > > > > > > -- > > > Lance Norskog > > > [email protected] > > > > > > > > > -- > Lance Norskog > [email protected] >
