Re: [Spark MLlib] about linear regression issue
Hi DB Tsai, Firstly I must show my deep appreciation towards your kind help. Did you just mean like this, currently there is no way for users to deal with constrains like all weights >= 0 in spark, though spark also has LBFGS ... Moreover, I did not know whether spark SVD will help some for that issue... But people do LBFGS-B for the constrains like all weights >= 0 , since LBFGS-B is already working. I should get familar to use breeze in the program then . Thank you, Zhiliang On Monday, November 2, 2015 11:12 AM, DB Tsai wrote: For the constrains like all weights >=0, people do LBFGS-B which is supported in our optimization library, Breeze. https://github.com/scalanlp/breeze/issues/323 However, in Spark's LiR, our implementation doesn't have constrain implementation. I do see this is useful given we're experimenting SLIM: Sparse Linear Methods for recommendation, http://www-users.cs.umn.edu/~xning/papers/Ning2011c.pdf which requires all the weights to be positive (Eq. 3) to represent positive relations between items. In summary, it's possible and not difficult to add this constrain to our current linear regression, but currently, there is no open source implementation in Spark. Sincerely, DB Tsai -- Web: https://www.dbtsai.com PGP Key ID: 0xAF08DF8D On Sun, Nov 1, 2015 at 9:22 AM, Zhiliang Zhu wrote: > Dear All, > > As for N dimension linear regression, while the labeled training points > number (or the rank of the labeled point space) is less than N, > then from perspective of math, the weight of the trained linear model may be > not unique. > > However, the output of model.weight() by spark may be with some wi < 0. My > issue is, is there some proper way only to get > some specific output weight with all wi >= 0 ... > > Yes, the above goes same with the issue about solving linear system of > equations, Aw = b, and r(A, b) = r(A) < columnNo(A), then w is > with infinite solutions, but here only needs one solution with all wi >= 0. > When there is only unique solution, both LR and SVD will work perfect. > > I will appreciate your all kind help very much~~ > Best Regards, > Zhiliang > > - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Re: [Spark MLlib] about linear regression issue
For the constrains like all weights >=0, people do LBFGS-B which is supported in our optimization library, Breeze. https://github.com/scalanlp/breeze/issues/323 However, in Spark's LiR, our implementation doesn't have constrain implementation. I do see this is useful given we're experimenting SLIM: Sparse Linear Methods for recommendation, http://www-users.cs.umn.edu/~xning/papers/Ning2011c.pdf which requires all the weights to be positive (Eq. 3) to represent positive relations between items. In summary, it's possible and not difficult to add this constrain to our current linear regression, but currently, there is no open source implementation in Spark. Sincerely, DB Tsai -- Web: https://www.dbtsai.com PGP Key ID: 0xAF08DF8D On Sun, Nov 1, 2015 at 9:22 AM, Zhiliang Zhu wrote: > Dear All, > > As for N dimension linear regression, while the labeled training points > number (or the rank of the labeled point space) is less than N, > then from perspective of math, the weight of the trained linear model may be > not unique. > > However, the output of model.weight() by spark may be with some wi < 0. My > issue is, is there some proper way only to get > some specific output weight with all wi >= 0 ... > > Yes, the above goes same with the issue about solving linear system of > equations, Aw = b, and r(A, b) = r(A) < columnNo(A), then w is > with infinite solutions, but here only needs one solution with all wi >= 0. > When there is only unique solution, both LR and SVD will work perfect. > > I will appreciate your all kind help very much~~ > Best Regards, > Zhiliang > > - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
[Spark MLlib] about linear regression issue
Dear All, As for N dimension linear regression, while the labeled training points number (or the rank of the labeled point space) is less than N, then from perspective of math, the weight of the trained linear model may be not unique. However, the output of model.weight() by spark may be with some wi < 0. My issue is, is there some proper way only to getsome specific output weight with all wi >= 0 ... Yes, the above goes same with the issue about solving linear system of equations, Aw = b, and r(A, b) = r(A) < columnNo(A), then w iswith infinite solutions, but here only needs one solution with all wi >= 0.When there is only unique solution, both LR and SVD will work perfect. I will appreciate your all kind help very much~~Best Regards,Zhiliang