On 07/10/2012 10:08 PM, Olivier Grisel wrote: > 2012/7/10 Andreas Mueller <[email protected]>: >> Hi Emanuel. >> Is there a reason not to train multinomial logistic regression >> (other than that it is not finished yet) ? >> I think it would be more straight-forward and any help >> on the multinomial logistic regression would be great >> (I'm very busy at the moment unfortunately). > One reason could be scalability w.r.t. the number of classes as for > One vs Rest, each class can be treated as a binary classification > problem and the weights trained independently (e.g. distributed on a > cluster) and only the final probability calibration weights (which are > much fewer) need access to the complete weight matrix (in readonly > mode) at once. >
If holding the weights in memory is the bottle-neck, you should probably not do one-vs-rest and are quite certainly in trouble in several respects. For parallelism, I'm not sure whether doing 1 vs rest in parallel would be better than doing some form of parallel SGD for multinomial logistic regression (say naively splitting the training set in n_classes parts and average the results). I agree, though, the straight-forward parallelism of OvR is neat. I find it just a bit weird to train with one loss and then evaluate with another. Cheers, Andy ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Scikit-learn-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
