IF you really want to re-organize the multiplication it is fine by me.
On Tue, Jan 14, 2014 at 1:20 AM, Frank Scholten <[email protected]>wrote: > I see the update rule to the beta matrix is derived from pseudo code in the > innermost loop in 'Algorithm 1: Stochastic Gradient Descent' in > > http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.177.3514&rep=rep1&type=pdf > > In the paper the learning rate is put before the gradient of the error > function and the instance value multiplication is put before the gradient. > > Perhaps we can rearrange the code like this so it matches with the paper > and add a comment? The only difference then is the perTermAnnealingRate. > > // See 'Algorithm 1: SGD' in > > http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.177.3514&rep=rep1&type=pdf > double newValue = beta.getQuick(i, j) + learningRate * > perTermLearningRate(j) * instance.get(j) * gradientBase; > > Cheers, > > Frank > > > > > > > > On Mon, Jan 13, 2014 at 10:54 PM, Frank Scholten <[email protected] > >wrote: > > > Thanks guys, I have some reading to do :-) > > > > > > On Mon, Jan 13, 2014 at 10:45 PM, Ted Dunning <[email protected] > >wrote: > > > >> The reference is to the web site in general. > >> > >> If anything, this blog is closest: > >> > >> > http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.177.3514&rep=rep1&type=pdf > >> > >> > >> On Mon, Jan 13, 2014 at 1:14 PM, Suneel Marthi <[email protected] > >> >wrote: > >> > >> > I think this is the one. Yes, I don't see this paper referenced in the > >> > code sorry about that. > >> > http://leon.bottou.org/publications/pdf/compstat-2010.pdf > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > On Monday, January 13, 2014 3:51 PM, Frank Scholten < > >> > [email protected]> wrote: > >> > > >> > Do you know which paper it is? He has quite a few publications. I > don't > >> see > >> > any mention of one of his papers in the code. I only see > >> > > www.eecs.tufts.edu/~dsculley/papers/combined-ranking-and-regression.pdfin > >> > MixedGradient but this is something different. > >> > > >> > > >> > > >> > > >> > On Mon, Jan 13, 2014 at 1:27 PM, Suneel Marthi < > [email protected] > >> > >wrote: > >> > > >> > > Mahout's impl is based off of Leon Bottou's paper on this subject. > I > >> > > don't gave the link handy but it's referenced in the code or try > >> google > >> > > search > >> > > > >> > > Sent from my iPhone > >> > > > >> > > > On Jan 13, 2014, at 7:14 AM, Frank Scholten < > [email protected] > >> > > >> > > wrote: > >> > > > > >> > > > Hi, > >> > > > > >> > > > I followed the Coursera Machine Learning course quite a while ago > >> and I > >> > > am > >> > > > trying to find out how Mahout implements the Logistic Regression > >> cost > >> > > > function in the code surrounding AbstractOnlineLogisticRegression. > >> > > > > >> > > > I am looking at the train method in > AbstractOnlineLogisticRegression > >> > and > >> > > I > >> > > > see online gradient descent step where the beta matrix is updated > >> but > >> > to > >> > > me > >> > > > its unclear how matches with the cost function described at: > >> > > > http://www.holehouse.org/mlclass/06_Logistic_Regression.html > >> > > > > >> > > > Perhaps Mahout uses an optimized approach for that does not > directly > >> > map > >> > > > into the formula at that link? > >> > > > > >> > > > Cheers, > >> > > > > >> > > > Frank > >> > > > >> > > >> > > > > >
