I don't know anything about this code; Ted does. Do you have a proposed patch that he could evaluate?
On Sat, Jun 4, 2011 at 4:34 PM, XiaoboGu <guxiaobo1...@gmail.com> wrote: > Hi Sean, > Can you help with this? > > Regards, > > Xiaobo Gu > > > -----Original Message----- > > From: XiaoboGu [mailto:guxiaobo1...@gmail.com] > > Sent: Sunday, May 29, 2011 4:23 PM > > To: 'Ted Dunning' > > Cc: 'dev@mahout.apache.org' > > Subject: NullPointerException after getBest() during training a > AdaptiveLogisticRegression. > > > > Hi, > > > > The main process for MAHOUT-696 is as following, but it will always > cause a > > NullPointerException after the first call to getBest, can we continue > training > > AdaptiveLogisticRegressions after using getBest() to score some new lines > just as > > TrainLogistic does? > > > > > > > > double logPEstimate = 0; > > int k = 0; > > > > CsvRecordFactory csv = lmp.getCsvRecordFactory(); > > model = lmp.createAdaptiveLogisticRegression(); > > State<Wrapper, CrossFoldLearner> best = null; > > CrossFoldLearner learner = null; > > > > for (int pass = 0; pass < passes; pass++) { > > BufferedReader in = open(inputFile); > > > > // read variable names > > csv.firstLine(in.readLine()); > > > > String line = in.readLine(); > > > > while (line != null) { > > // for each new line, get target > and predictors > > Vector input = new > > RandomAccessSparseVector(lmp.getNumFeatures()); > > int targetValue = > csv.processLine(line, input); > > > > // update model > > model.train(targetValue, input); > > > > k ++; > > > > if (scores && (k % (skipscorenum + > 1) == 0) ) { > > > > best = model.getBest(); > > if (null != best) { > > learner = > best.getPayload().getLearner(); > > } > > if (learner != null) { > > // check performance while > this is still news > > double logP = > learner.logLikelihood(targetValue, input); > > if > (!Double.isInfinite(logP)) { > > if (k < 20) { > > > logPEstimate = (k * logPEstimate + logP) > > > / (k + 1); > > } else { > > > logPEstimate = 0.95 * logPEstimate + 0.05 > > > * logP; > > } > > } > > double p = > learner.classifyScalar(input); > > > > > output.printf(Locale.ENGLISH, > > "%10d %2d > %10.2f %2.4f %10.4f %10.4f\n", > > k, > targetValue, > > > learner.percentCorrect(), p, logP, > > > logPEstimate); > > }else{ > > > output.printf(Locale.ENGLISH, > > > "%10d %2d %s\n", k, targetValue, > > > "AdaptiveLogisticRegression is not ready for > > scoring ... "); > > } > > } > > > > > > line = in.readLine(); > > } > > in.close(); > > } > > > > > > > > > > 100 1 AdaptiveLogisticRegression is not ready for scoring ... > > 200 0 AdaptiveLogisticRegression is not ready for scoring ... > > 300 1 AdaptiveLogisticRegression is not ready for scoring ... > > 400 0 AdaptiveLogisticRegression is not ready for scoring ... > > 500 1 AdaptiveLogisticRegression is not ready for scoring ... > > Exception in thread "main" java.lang.IllegalStateException: > java.lang.NullPointerException > > at > > > org.apache.mahout.classifier.sgd.AdaptiveLogisticRegression.trainWithBufferedExamples( > > AdaptiveLogisticRegression.java:144) > > at > > > org.apache.mahout.classifier.sgd.AdaptiveLogisticRegression.train(AdaptiveLogisticRegress > > ion.java:117) > > at > > > org.apache.mahout.classifier.sgd.AdaptiveLogisticRegression.train(AdaptiveLogisticRegress > > ion.java:103) > > at > > > org.apache.mahout.classifier.sgd.TrainAdaptiveLogistic.main(TrainAdaptiveLogistic.java:7 > > 2) > > Caused by: java.lang.NullPointerException > > at > org.apache.mahout.classifier.sgd.CrossFoldLearner.train(CrossFoldLearner.java:134) > > at > > > org.apache.mahout.classifier.sgd.AdaptiveLogisticRegression$Wrapper.train(AdaptiveLogis > > ticRegression.java:411) > > at > > > org.apache.mahout.classifier.sgd.AdaptiveLogisticRegression$1.apply(AdaptiveLogisticReg > > ression.java:128) > > at > > > org.apache.mahout.classifier.sgd.AdaptiveLogisticRegression$1.apply(AdaptiveLogisticReg > > ression.java:1) > > at > org.apache.mahout.ep.EvolutionaryProcess$1.call(EvolutionaryProcess.java:146) > > at > org.apache.mahout.ep.EvolutionaryProcess$1.call(EvolutionaryProcess.java:1) > > at > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) > > at java.util.concurrent.FutureTask.run(FutureTask.java:138) > > at > > > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > > at > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > > at java.lang.Thread.run(Thread.java:662) > >
