> -----Original Message----- > From: Ted Dunning [mailto:[email protected]] > Sent: Thursday, May 26, 2011 12:19 PM > To: [email protected] > Subject: Re: Are the OnlineLogisticRegression s of a CrossFolderLearner > object equal after > trainning? > > Xiaobo, > > Sorry to be slow answering you. > > In general, there is no reason to pick one OLR inside a CrossFolderLearner > over another. They all have seen 80% of the data. Some day, we might want > to produce a different kind of CFL that is not symmetrical, but I haven't > had a need for that yet. For instance, we might have one OLR that gets all > of the data for training and another that gets 80% for training and 20% for > evaluation. > > For now, any of the OLR's is as good as any other. > > For your second question, I think that you are asking "According to what > criterion is the best ...". > > Typically the choice is based on AUC for binary models and log-likelihood > for multinomial models. You could change that to be percent correct or any > other metric you might like. Grouped AUC is common, for instance.
Just to confirm, "Binary models" means the target only has two distinct values, and they must be 0 and 1. "Multinomial models" means the number n of distinct values the target is more than 2, and they should be encoded as 0, 1, 2,......, n-1, And AUC and log-likelihood are used for evaluating the performance of binary and multinomial models respectively, can't mix them up? > On Mon, May 23, 2011 at 8:23 AM, XiaoboGu <[email protected]> wrote: > > > Hi, > > The TrainNewsGroup.java just use OnlineLogisticRegression model = > > state.getModels().get(0); to get an OLR object to do the overall description > > of the AdaptiveLogisticRegression’s performance, there are two questions: > > 1. Are the OLR objects of the best CrosFolderLearner equal. > > 2. According to what cretirear, the best CrossFolderLearner object is > > chosen? > > > > Regards, > > > > Xiaobo Gu > > > >
