On Sat, May 28, 2011 at 5:45 PM, XiaoboGu <[email protected]> wrote:
> > ... > For now, any of the OLR's is as good as any other. > > > > For your second question, I think that you are asking "According to what > > criterion is the best ...". > > > > Typically the choice is based on AUC for binary models and log-likelihood > > for multinomial models. You could change that to be percent correct or > any > > other metric you might like. Grouped AUC is common, for instance. > > Just to confirm, > "Binary models" means the target only has two distinct values, and they > must be 0 and 1. > Yes. > "Multinomial models" means the number n of distinct values the target is > more than 2, and they should be encoded as 0, 1, 2,......, n-1, > Yes. > And AUC and log-likelihood are used for evaluating the performance of > binary and multinomial models respectively, can't mix them up? > No. Log-likelihood can be used for either. AUC normally is only used for binomial cases. There are generalizations of AUC, but we haven't implemented them. > > >
