Hi Mahesh, Any update on your progress?
Best regards. On Wed, May 4, 2016 at 8:35 PM, Maheshakya Wijewardena <[email protected]> wrote: > Hi Mahesh, > > is that "Put break points in train methods in Linear Regression class" >> means the spark/algorithms/ LinearRegrassion.java class in the >> org.wso2.carbon.ml.core? is that the correct file? > > > Yes, this is the correct place. > > You can refer to spark programming guide[1][2] as well as our ML code base > when you try those algorithms out. Please try to do rough implementations > of the streaming versions of linear regression, logistic regression and > k-means clustering as we have discussed in the proposal in plain Java. It's > better if you can create a git repo and share your code once you have made > some progress. > > Were you able debug and understand the flow of the ML siddhi extension? I > hope you haven't encountered more errors after switching the released > version of CEP. > > Is this Friday okay for you? Afternoon at 2:00 pm? > > Best regards. > > > Best regards. > > [1] http://spark.apache.org/docs/latest/programming-guide.html > [2] http://spark.apache.org/docs/latest/mllib-guide.html > > On Wed, May 4, 2016 at 1:07 PM, Mahesh Dananjaya < > [email protected]> wrote: > >> Hi Maheshakya, >> I have been looking into some algorithms related to stochastic gradient >> descent based algorithms.anything i should focus please let me know.Ans >> also i will be available for calling this week and next week.thank you. >> BR, >> Mahesh. >> >> On Tue, May 3, 2016 at 5:05 PM, Mahesh Dananjaya < >> [email protected]> wrote: >> >>> Hi Maheshakya, >>> thank you.that's good. i have been trying to fix that for couple of >>> days. please inform me when it will be fixed.now i have been testing the ML >>> algorithms and trying to identify the flow and the hierarchy. is that "Put >>> break points in train methods in Linear Regression class" means the >>> spark/algorithms/ LinearRegrassion.java class in the >>> org.wso2.carbon.ml.core? is that the correct file? >>> And also i am planning to write some programs to use apache spark mllib >>> algorithms. and i refer to [1] and some wso2 documentations to get some >>> idea about ML structure.thank you. >>> >>> BR, >>> Mahesh. >>> >>> [1]nirmalfdo.blogspot.com >>> >>> On Tue, May 3, 2016 at 4:36 PM, Maheshakya Wijewardena < >>> [email protected]> wrote: >>> >>>> Hi Mahesh, >>>> >>>> I have checked. It seems the issue you have encountered is cause only >>>> in the current development branch of the product-cep. It doesn't identify >>>> the ML siddhi extension as an extension. ML siddhi extension works fine in >>>> the latest release of CEP (4.1.0) [1]. >>>> Until we figure out the reason and come up with a solution, can you use >>>> the latest CEP release for your work. It's fine to use that since you >>>> haven't started actual development yet. >>>> >>>> Best regards. >>>> >>>> [1] http://wso2.com/products/complex-event-processor/ >>>> >>>> On Tue, May 3, 2016 at 3:19 PM, Maheshakya Wijewardena < >>>> [email protected]> wrote: >>>> >>>>> Hi Mahesh, >>>>> >>>>> >>>>>> Is is vital to use those local repo in my upcoming implementation? >>>>> >>>>> >>>>> Yes. The remote p2-repo contains the p2-repos of released versions. >>>>> What you have to develop on is the current master of the carbon-ml and >>>>> product-ml. You can try out with the modification I have suggested. In the >>>>> meantime, I'll verify whether the current repos are working as expected. >>>>> >>>>> And also i am trying to debug the carbon-ml org.wso2.carbon.ml.core by >>>>>> putting some break point in the spark/algorithms/Linear Regression >>>>> >>>>> >>>>> It's great that you have started looking at the implementation of >>>>> linear regression as well. Put break points in train methods in >>>>> LinearRegression class. This is being used when you run linear regression >>>>> from UI. >>>>> >>>>> I can see some comments left behind for streaming algo as well.thank >>>>>> you >>>>> >>>>> >>>>> You may be referring to the linear regression with SGD model. Here, >>>>> there's no retraining with streaming data involved. The SGD with >>>>> minibatches is used to train the model with the data set only once. >>>>> What you have to do is create a similar mechanism to involve streaming >>>>> data and retrain models. We will get to that part once you get comfortable >>>>> with siddhi extensions. >>>>> >>>>> BTW, is it possible for you to join a call on this Friday or in the >>>>> next week. We'll try to resolve your current issues and discuss further on >>>>> project. >>>>> >>>>> Best regards. >>>>> >>>>> On Tue, May 3, 2016 at 1:03 PM, Mahesh Dananjaya < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi maheshakya, >>>>>> Is it ok to go with p2 repo at >>>>>> http://product-dist.wso2.com/p2/carbon/releases/wilkes/features/ >>>>>> rather than the P2-repo at product-ml/modules/p2-profile/target/p2-repo >>>>>> in >>>>>> local repo.What is the impact?.Is is vital to use those local repo in my >>>>>> upcoming implementation?so i was trying to give remote p2 repo to cep >>>>>> built >>>>>> by source and trying to debug the cep ml extension and got the same error >>>>>> as yesterday. But pre-built product is working fine. Therefore now i am >>>>>> trying as you described in the last email. >>>>>> >>>>>> And also i am trying to debug the carbon-ml org.wso2.carbon.ml.core >>>>>> by putting some break point in the spark/algorithms/Linear Regression. I >>>>>> am >>>>>> trying to trigger it with product-ml project with data set. Does that >>>>>> Linear Regression also in UI is consuming that spark algorithms or is it >>>>>> in >>>>>> another place? I can see some comments left behind for streaming algo as >>>>>> well.thank you. >>>>>> BR, >>>>>> Mahesh. >>>>>> >>>>>> On Tue, May 3, 2016 at 9:35 AM, Maheshakya Wijewardena < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hi Mahesh, >>>>>>> >>>>>>> The earlier error you have mentioned may occur due to the >>>>>>> incompatible Siddhi versions in ML p2-repo and CEP, when you add the >>>>>>> p2-repo from prodcut-ml you built. >>>>>>> Current siddhi version in product-cep is 3.0.6-SNAPSHOT[1], but in >>>>>>> ML it's 3.0.2. >>>>>>> >>>>>>> Can you try changing the siddhi.version in carbon-ml/pom.xml to >>>>>>> 3.0.6-SNAPSHOT, build carbon-ml, then build product-ml again. After >>>>>>> this, >>>>>>> add p2-repo as a local repository again to fresh CEP pack and try it >>>>>>> out. >>>>>>> >>>>>>> Best regards. >>>>>>> >>>>>>> On Mon, May 2, 2016 at 7:02 PM, Mahesh Dananjaya < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hi Maheshakya, >>>>>>>> now i remotely debug the CEP extension for ML Prediction. What i >>>>>>>> did was, i have all the pre-build version of CEP and ML. Therefore i >>>>>>>> used >>>>>>>> that built CEP and did the same thing that i was doing with the source >>>>>>>> code. I think the only change i did was install those packages from >>>>>>>> remote >>>>>>>> p2 repo. This was work fine and i debugged the carbon-ml >>>>>>>> org.wso2.carbon.ml.siddhi.extension as described in the [1]. So now i >>>>>>>> have >>>>>>>> to try same thing with the build by source content.thank you. >>>>>>>> >>>>>>>> [1] >>>>>>>> https://docs.wso2.com/display/ML110/WSO2+CEP+Extension+for+ML+Predictions#WSO2CEPExtensionforMLPredictions-Siddhisyntaxfortheextension >>>>>>>> >>>>>>>> BR, >>>>>>>> Mahesh >>>>>>>> >>>>>>>> On Mon, Apr 25, 2016 at 5:49 PM, Maheshakya Wijewardena < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hi Mahesh, >>>>>>>>> >>>>>>>>> Congratulations and welcome to GSoC 2016. You did a great job in >>>>>>>>> preparing the proposal. Now it's time to dig deep and get started >>>>>>>>> with the >>>>>>>>> project. >>>>>>>>> >>>>>>>>> First of all you need to familiarize with the code base. We have >>>>>>>>> agreed to implement this with CEP event streams. We already have a CEP >>>>>>>>> extension for predictions [1][2]. Go through this implementation and >>>>>>>>> familiarize your self with that. You need to understand how: >>>>>>>>> >>>>>>>>> 1. Even streams are consumed >>>>>>>>> 2. predictions are made from individual event >>>>>>>>> 3. Results are sent back >>>>>>>>> >>>>>>>>> Get WSO2 ML and CEP sources (You may use latest released version >>>>>>>>> of CEP) and build the products. Get both carbon-ml[3] and >>>>>>>>> product-ml[4] >>>>>>>>> masters and create new branches for your work from masters. >>>>>>>>> >>>>>>>>> After you build the products, you may need to do remote >>>>>>>>> debugging[5] to understand the flow. So please follow an example of >>>>>>>>> real >>>>>>>>> time prediction with ML with debugging and get some idea. The >>>>>>>>> component you >>>>>>>>> need to debug is org.wso2.carbon.ml.siddhi.extension. >>>>>>>>> >>>>>>>>> Next tasks would be implementing online learning algorithms in >>>>>>>>> plain java with spark ml lib and integrating those to ML. We also >>>>>>>>> need to >>>>>>>>> come up with a proper and detailed architecture to employ those >>>>>>>>> algorithms >>>>>>>>> in ML. Getting familiar with the aforementioned sections would give >>>>>>>>> you >>>>>>>>> some insight on how this should be implemented. >>>>>>>>> >>>>>>>>> So please try to get a quick grasp then you can start the >>>>>>>>> implementation. Let us know if you have any questions or you get stuck >>>>>>>>> somewhere. >>>>>>>>> >>>>>>>>> Also, please always add WSO2 developer's list as well when you >>>>>>>>> communicate with us regarding the project so that you can get >>>>>>>>> opinions and >>>>>>>>> feedback from others as well. >>>>>>>>> >>>>>>>>> Best regards. >>>>>>>>> >>>>>>>>> [1] >>>>>>>>> https://docs.wso2.com/display/ML110/WSO2+CEP+Extension+for+ML+Predictions#WSO2CEPExtensionforMLPredictions-Siddhisyntaxfortheextension >>>>>>>>> >>>>>>>>> [2] >>>>>>>>> https://github.com/wso2/carbon-ml/tree/master/components/extensions/org.wso2.carbon.ml.siddhi.extension >>>>>>>>> >>>>>>>>> [3] https://github.com/wso2/carbon-ml >>>>>>>>> >>>>>>>>> [4] https://github.com/wso2/product-ml >>>>>>>>> >>>>>>>>> [5] https://dzone.com/articles/how-debug-wso2-carbon-kernel >>>>>>>>> >>>>>>>>> >>>>>>>>> On Mon, Apr 25, 2016 at 3:33 PM, Mahesh Dananjaya < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> thank you for accepting my GSOC 2016 proposal and i am looking >>>>>>>>>> forward for the further instruction and project continuation. thank >>>>>>>>>> you >>>>>>>>>> very much. >>>>>>>>>> regards, >>>>>>>>>> Mahesh. >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>> Pruthuvi Maheshakya Wijewardena >>>>>>>>> [email protected] >>>>>>>>> +94711228855 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Pruthuvi Maheshakya Wijewardena >>>>>>> [email protected] >>>>>>> +94711228855 >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Pruthuvi Maheshakya Wijewardena >>>>> [email protected] >>>>> +94711228855 >>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> Pruthuvi Maheshakya Wijewardena >>>> [email protected] >>>> +94711228855 >>>> >>>> >>>> >>> >> > > > -- > Pruthuvi Maheshakya Wijewardena > [email protected] > +94711228855 > > > -- Pruthuvi Maheshakya Wijewardena [email protected] +94711228855
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
