Ok, let's have both, since both will be useful with different datasets. On Mon, Jun 1, 2015 at 12:21 AM, Maheshakya Wijewardena <[email protected] > wrote:
> I should agree with Upul about adding both if possible. Mini-batch adds > the question of determining the right size for batch size, but finding the > right batch size may greatly improve our results as well as time for > convergence. But still, it can depend heavily on the dataset. > > Have you tried with different datasets? Different in terms of size as well > as other statistical properties of features(such as standard deviation, > skewness, etc.)? > > On Sun, May 31, 2015 at 10:28 PM, Nirmal Fernando <[email protected]> wrote: > >> yes.. but from the simple test I did, I felt L-BFGS is faster. Will >> confirm anyway. >> >> On Sun, May 31, 2015 at 10:13 PM, Upul Bandara <[email protected]> wrote: >> >>> Actually, I'm thinking in terms of training time, even for large data >>> sets prediction accuracy of L-BFGS will outperform SGD. But its training >>> time would be considerably bigger than the training time of SGD. >>> On the other hand, SGD model gives a decent prediction accuracy in >>> relatively short period of training time. >>> >>> >>> On Sun, May 31, 2015 at 9:52 PM, Nirmal Fernando <[email protected]> >>> wrote: >>> >>>> Thanks Upul. So, are you thinking along the lines of performance? Sure, >>>> I'll run a test. >>>> >>>> On Sun, May 31, 2015 at 9:50 PM, Upul Bandara <[email protected]> wrote: >>>> >>>>> If it is possible, I would like to have both. >>>>> >>>>> L-BFGS converges faster than SGD. But it goes through the entire data >>>>> set before moving from one iteration to the next. >>>>> Whereas, SGD uses a minit-batch of the training data set for >>>>> calculating and updating its gradient. >>>>> Hence, for large data sets SGD is more practical than L-BFGS. >>>>> >>>>> I think we can test this scenario by running these two algorithms >>>>> against a large data set (~ 1GB) >>>>> >>>>> Thanks, >>>>> Upul >>>>> >>>>> On Sun, May 31, 2015 at 8:02 PM, Nirmal Fernando <[email protected]> >>>>> wrote: >>>>> >>>>>> One other benefit of switching is, this API supports multi-class >>>>>> classification too. I've tested this API with Iris dataset. >>>>>> >>>>>> On Sun, May 31, 2015 at 7:33 PM, Nirmal Fernando <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> Currently in ML, we use mini-batch gradient descent algorithm when >>>>>>> running logistic regression. But Spark-mllib recommends L-BFGS over >>>>>>> mini-batch gradient descent for faster convergence [1]. >>>>>>> >>>>>>> I tested both the implementation with the same dataset and gained an >>>>>>> improved accuracy in L-BFGS (80% vs 67% for SGD). >>>>>>> >>>>>>> Shall we switch? >>>>>>> >>>>>>> [1] >>>>>>> https://spark.apache.org/docs/latest/mllib-linear-methods.html#logistic-regression >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> Thanks & regards, >>>>>>> Nirmal >>>>>>> >>>>>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>>>>> Mobile: +94715779733 >>>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Thanks & regards, >>>>>> Nirmal >>>>>> >>>>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>>>> Mobile: +94715779733 >>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Upul Bandara, >>>>> Associate Technical Lead, WSO2, Inc., >>>>> Mob: +94 715 468 345. >>>>> >>>> >>>> >>>> >>>> -- >>>> >>>> Thanks & regards, >>>> Nirmal >>>> >>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>> Mobile: +94715779733 >>>> Blog: http://nirmalfdo.blogspot.com/ >>>> >>>> >>>> >>> >>> >>> -- >>> Upul Bandara, >>> Associate Technical Lead, WSO2, Inc., >>> Mob: +94 715 468 345. >>> >> >> >> >> -- >> >> Thanks & regards, >> Nirmal >> >> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >> Mobile: +94715779733 >> Blog: http://nirmalfdo.blogspot.com/ >> >> >> > > > -- > Pruthuvi Maheshakya Wijewardena > Software Engineer > WSO2 Lanka (Pvt) Ltd > Email: [email protected] > Mobile: +94711228855 > > > -- Thanks & regards, Nirmal Associate Technical Lead - Data Technologies Team, WSO2 Inc. Mobile: +94715779733 Blog: http://nirmalfdo.blogspot.com/
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
