This usually happens when you don't have enough data to have pushed a batch through the learning algorithm and try to look at average performance. It could be that certain combinations of evolutionary generation length and batch size could cause the evolutionary algorithm to try to look at performance before sending any data into the system, but I thought that these things were tightly coupled so as to make that impossible.
On Fri, Jun 24, 2011 at 3:02 AM, XiaoboGu <guxiaobo1...@gmail.com> wrote: > Hi, > > After debugging the source , the NullPointerException is because of not > seting CrossFolderLearner with a auc > > > > > -----Original Message----- > > From: XiaoboGu [mailto:guxiaobo1...@gmail.com] > > Sent: Friday, June 24, 2011 4:55 PM > > To: 'Ted Dunning'; 'dev@mahout.apache.org' > > Subject: RE: FW: NullPointerException after getBest() during training a > > AdaptiveLogisticRegression. > > > > When I debug the patch, I found there are 3 threads suspended by line 126 > of > > AdaptiveLogisticRegression.java, > > > > I see there a inner Record buffer with length 500, the > NullPointerException was thrown when > > the inner buffer is full, is this a multi-threading related problem? > > > > Regards, > > > > Xiaobo Gu > > > > From: Ted Dunning [mailto:ted.dunn...@gmail.com] > > Sent: Monday, June 06, 2011 6:23 PM > > To: dev@mahout.apache.org > > Cc: XiaoboGu > > Subject: Re: FW: NullPointerException after getBest() during training a > > AdaptiveLogisticRegression. > > > > I am out of pocket for at least another week and will have a hard time > finding time to look at > > patches. > > > > I should be able to do so on the time scale of weeks, however. I am for > this patch in principle, > > but details matter a lot. > > On Sat, Jun 4, 2011 at 8:45 AM, Sean Owen <sro...@gmail.com> wrote: > > I don't know anything about this code; Ted does. Do you have a proposed > > patch that he could evaluate? > > > > On Sat, Jun 4, 2011 at 4:34 PM, XiaoboGu <guxiaobo1...@gmail.com> wrote: > > > > > Hi Sean, > > > Can you help with this? > > > > > > Regards, > > > > > > Xiaobo Gu > > > > > > > -----Original Message----- > > > > From: XiaoboGu [mailto:guxiaobo1...@gmail.com] > > > > Sent: Sunday, May 29, 2011 4:23 PM > > > > To: 'Ted Dunning' > > > > Cc: 'dev@mahout.apache.org' > > > > Subject: NullPointerException after getBest() during training a > > > AdaptiveLogisticRegression. > > > > > > > > Hi, > > > > > > > > The main process for MAHOUT-696 is as following, but it will > always > > > cause a > > > > NullPointerException after the first call to getBest, can we continue > > > training > > > > AdaptiveLogisticRegressions after using getBest() to score some new > lines > > > just as > > > > TrainLogistic does? > > > > > > > > > > > > > > > > double logPEstimate = 0; > > > > int k = 0; > > > > > > > > CsvRecordFactory csv = > lmp.getCsvRecordFactory(); > > > > model = lmp.createAdaptiveLogisticRegression(); > > > > State<Wrapper, CrossFoldLearner> best = null; > > > > CrossFoldLearner learner = null; > > > > > > > > for (int pass = 0; pass < passes; pass++) { > > > > BufferedReader in = open(inputFile); > > > > > > > > // read variable names > > > > csv.firstLine(in.readLine()); > > > > > > > > String line = in.readLine(); > > > > > > > > while (line != null) { > > > > // for each new line, get > target > > > and predictors > > > > Vector input = new > > > > RandomAccessSparseVector(lmp.getNumFeatures()); > > > > int targetValue = > > > csv.processLine(line, input); > > > > > > > > // update model > > > > model.train(targetValue, > input); > > > > > > > > k ++; > > > > > > > > if (scores && (k % > (skipscorenum + > > > 1) == 0) ) { > > > > > > > > best = model.getBest(); > > > > if (null != best) { > > > > learner = > > > best.getPayload().getLearner(); > > > > } > > > > if (learner != null) { > > > > // check performance > while > > > this is still news > > > > double logP = > > > learner.logLikelihood(targetValue, input); > > > > if > > > (!Double.isInfinite(logP)) { > > > > if (k < 20) { > > > > > > > logPEstimate = (k * logPEstimate + logP) > > > > > > > / (k + 1); > > > > } else { > > > > > > > logPEstimate = 0.95 * logPEstimate + 0.05 > > > > > > > * logP; > > > > } > > > > } > > > > double p = > > > learner.classifyScalar(input); > > > > > > > > > > > output.printf(Locale.ENGLISH, > > > > "%10d > %2d > > > %10.2f %2.4f %10.4f %10.4f\n", > > > > k, > > > targetValue, > > > > > > > learner.percentCorrect(), p, logP, > > > > > > > logPEstimate); > > > > }else{ > > > > > > > output.printf(Locale.ENGLISH, > > > > > > > "%10d %2d %s\n", k, targetValue, > > > > > > > "AdaptiveLogisticRegression is not ready for > > > > scoring ... "); > > > > } > > > > } > > > > > > > > > > > > line = in.readLine(); > > > > } > > > > in.close(); > > > > } > > > > > > > > > > > > > > > > > > > > 100 1 AdaptiveLogisticRegression is not ready for scoring ... > > > > 200 0 AdaptiveLogisticRegression is not ready for scoring ... > > > > 300 1 AdaptiveLogisticRegression is not ready for scoring ... > > > > 400 0 AdaptiveLogisticRegression is not ready for scoring ... > > > > 500 1 AdaptiveLogisticRegression is not ready for scoring ... > > > > Exception in thread "main" java.lang.IllegalStateException: > > > java.lang.NullPointerException > > > > at > > > > > > > > > > org.apache.mahout.classifier.sgd.AdaptiveLogisticRegression.trainWithBufferedExamples( > > > > AdaptiveLogisticRegression.java:144) > > > > at > > > > > > > > > > org.apache.mahout.classifier.sgd.AdaptiveLogisticRegression.train(AdaptiveLogisticRegress > > > > ion.java:117) > > > > at > > > > > > > > > > org.apache.mahout.classifier.sgd.AdaptiveLogisticRegression.train(AdaptiveLogisticRegress > > > > ion.java:103) > > > > at > > > > > > > > > > org.apache.mahout.classifier.sgd.TrainAdaptiveLogistic.main(TrainAdaptiveLogistic.java:7 > > > > 2) > > > > Caused by: java.lang.NullPointerException > > > > at > > > > org.apache.mahout.classifier.sgd.CrossFoldLearner.train(CrossFoldLearner.java:134) > > > > at > > > > > > > > > > org.apache.mahout.classifier.sgd.AdaptiveLogisticRegression$Wrapper.train(AdaptiveLogis > > > > ticRegression.java:411) > > > > at > > > > > > > > > > org.apache.mahout.classifier.sgd.AdaptiveLogisticRegression$1.apply(AdaptiveLogisticReg > > > > ression.java:128) > > > > at > > > > > > > > > > org.apache.mahout.classifier.sgd.AdaptiveLogisticRegression$1.apply(AdaptiveLogisticReg > > > > ression.java:1) > > > > at > > > > org.apache.mahout.ep.EvolutionaryProcess$1.call(EvolutionaryProcess.java:146) > > > > at > > > > org.apache.mahout.ep.EvolutionaryProcess$1.call(EvolutionaryProcess.java:1) > > > > at > > > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) > > > > at java.util.concurrent.FutureTask.run(FutureTask.java:138) > > > > at > > > > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > > > > at > > > > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > > > > at java.lang.Thread.run(Thread.java:662) > > > > > > > > >
