I actually have two versions: one is based on gradient descent like the logistic regression on mllib. the other is based on Newtown iteration, it is not as fast as SGD, but we can get all the statistics from it like deviance, p-values and fisher info.
we can get confusion matrix in both versions the gradient descent version is just a modification of logistic regression with my own implementation. I did not use LabeledPoints class. On Mon, Jan 6, 2014 at 11:13 AM, Evan Sparks <[email protected]> wrote: > Hi Michael, > > What strategy are you using to train the multinomial classifier? > One-vs-all? I've got an optimized version of that method that I've been > meaning to clean up and commit for a while. In particular, rather than > shipping a (potentially very big) model with each map task, I ship it once > before each iteration with a broadcast variable. Perhaps we can compare > versions and incorporate some of my optimizations into your code? > > Thanks, > Evan > > > On Jan 6, 2014, at 10:57 AM, Michael Kun Yang <[email protected]> > wrote: > > > > Hi Spark-ers, > > > > I implemented a SGD version of multinomial logistic regression based on > > mllib's optimization package. If this classifier is in the future plan of > > mllib, I will be happy to contribute my code. > > > > Cheers >
