yes.. but from the simple test I did, I felt L-BFGS is faster. Will confirm anyway.
On Sun, May 31, 2015 at 10:13 PM, Upul Bandara <[email protected]> wrote: > Actually, I'm thinking in terms of training time, even for large data sets > prediction accuracy of L-BFGS will outperform SGD. But its training time > would be considerably bigger than the training time of SGD. > On the other hand, SGD model gives a decent prediction accuracy in > relatively short period of training time. > > > On Sun, May 31, 2015 at 9:52 PM, Nirmal Fernando <[email protected]> wrote: > >> Thanks Upul. So, are you thinking along the lines of performance? Sure, >> I'll run a test. >> >> On Sun, May 31, 2015 at 9:50 PM, Upul Bandara <[email protected]> wrote: >> >>> If it is possible, I would like to have both. >>> >>> L-BFGS converges faster than SGD. But it goes through the entire data >>> set before moving from one iteration to the next. >>> Whereas, SGD uses a minit-batch of the training data set for calculating >>> and updating its gradient. >>> Hence, for large data sets SGD is more practical than L-BFGS. >>> >>> I think we can test this scenario by running these two algorithms >>> against a large data set (~ 1GB) >>> >>> Thanks, >>> Upul >>> >>> On Sun, May 31, 2015 at 8:02 PM, Nirmal Fernando <[email protected]> >>> wrote: >>> >>>> One other benefit of switching is, this API supports multi-class >>>> classification too. I've tested this API with Iris dataset. >>>> >>>> On Sun, May 31, 2015 at 7:33 PM, Nirmal Fernando <[email protected]> >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> Currently in ML, we use mini-batch gradient descent algorithm when >>>>> running logistic regression. But Spark-mllib recommends L-BFGS over >>>>> mini-batch gradient descent for faster convergence [1]. >>>>> >>>>> I tested both the implementation with the same dataset and gained an >>>>> improved accuracy in L-BFGS (80% vs 67% for SGD). >>>>> >>>>> Shall we switch? >>>>> >>>>> [1] >>>>> https://spark.apache.org/docs/latest/mllib-linear-methods.html#logistic-regression >>>>> >>>>> >>>>> -- >>>>> >>>>> Thanks & regards, >>>>> Nirmal >>>>> >>>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>>> Mobile: +94715779733 >>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> >>>> Thanks & regards, >>>> Nirmal >>>> >>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>> Mobile: +94715779733 >>>> Blog: http://nirmalfdo.blogspot.com/ >>>> >>>> >>>> >>> >>> >>> -- >>> Upul Bandara, >>> Associate Technical Lead, WSO2, Inc., >>> Mob: +94 715 468 345. >>> >> >> >> >> -- >> >> Thanks & regards, >> Nirmal >> >> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >> Mobile: +94715779733 >> Blog: http://nirmalfdo.blogspot.com/ >> >> >> > > > -- > Upul Bandara, > Associate Technical Lead, WSO2, Inc., > Mob: +94 715 468 345. > -- Thanks & regards, Nirmal Associate Technical Lead - Data Technologies Team, WSO2 Inc. Mobile: +94715779733 Blog: http://nirmalfdo.blogspot.com/
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
