Hi Mahesh, Great work so far.
Regarding the queries: streamingkm(0, 2,2,20,1,0.95 salary, rbi, walks, strikeouts, errors) Can you give me the definitions of the first few entities in the order. Also in previous supervised cases (linear regression), what is the response variable, etc. I'll go through the code and give you a feedback. After this, we need to me this implementation into carbon-ml siddhi extension. Please also do a similar implementation for logistic regression as well because we need to have a streaming version for classification as well. Best regards. On Tue, Jun 7, 2016 at 5:50 PM, Mahesh Dananjaya <dananjayamah...@gmail.com> wrote: > Hi Maheshkya, > I have changed the siddhi query for our StreamingKMeansClustering by > adding Alpha into the picture which we can use to make data horizon (how > quickly a most recent data point becomes a part of the model) and data > obsolescence (how long does it take a past data point to become irrelevant > to the model)in the streaming clustering algorithms.i have added new > changes to repo [1] introducing StreamingKMeansClusteringModel and > StreamingKMeansCLustering classes to project.new siddhi query is as follows. > > from Stream8Input#streaming:streamingkm(0, 2,2,20,1,0.95 salary, rbi, > walks, strikeouts, errors) > > select * > insert into regResults; > > regrads, > Mahesh. > > [1] https://github.com/dananjayamahesh/GSOC2016/tree/master/gsoc > > On Mon, Jun 6, 2016 at 6:31 PM, Mahesh Dananjaya < > dananjayamah...@gmail.com> wrote: > >> Hi Maheshakya, >> As we have discussed the architecture of the project i have already >> developed a couple of essential components for our project. During last >> week i completed the writing cep siddhi extension for our streaming >> algorithms which are developed to learn incrementally with past >> experiences. I have written the siddhi extensions with StreamProcessor >> extension for StreamingLinearRegerssion and StreamingKMeansClustering with >> the relevant parameters to call it as siddhi query. On the other hand i did >> some research on developing Mini Batch KMeans clustering for our >> StreamingKMeansClustering. And also i added the moving window addition to >> usual batch processing. And currently i am working on the time based >> incremental re-trainign method for siddhi streams. On the >> StreamingClustering side i have already part of th >> StreamingKMeansClustering with the mini batch KMeans clustering. All the >> work i did were pushed to my repo in github [1]. you can find the >> development on gsoc/ directory. >> And also as the ml team and supun was asked, i have did some timing and >> performance analysis for our SGD (Stochastic Gradient Descent) algorithms >> for LinearRegression. Those results also add to my repo in [2]. Now i am >> developing the rest for our purpose and trying to looked into other >> researches on predictive analysis for online big data. Ans also doing some >> work related to mini batch KMEans Clustering. And also i have been working >> on the performance analysis, accuracy and basic comparison between mini >> batch algorithms and moving window algorithms for streaming and periodic >> re-training of ML model. thank you. >> BR, >> Mahesh. >> [1] https://github.com/dananjayamahesh/GSOC2016 >> [2] >> https://github.com/dananjayamahesh/GSOC2016/blob/master/spark-examples/first-example/output/lr_timing_10000.jpg >> >> >> On Sat, Jun 4, 2016 at 8:50 PM, Mahesh Dananjaya < >> dananjayamah...@gmail.com> wrote: >> >>> Hi Maheshkya, >>> If you want to run it please use following queries. >>> 1. StreamingLInearRegression >>> >>> from Stream4InputStream#streaming:streaminglr(0, 2, 0.95, salary, rbi, >>> walks, strikeouts, errors) >>> >>> select * >>> >>> insert into regResults; >>> >>> from Stream8Input#streaming:streamingkm(0, 2, 0.95,2,20, salary, rbi, >>> walks, strikeouts, errors) >>> >>> select * >>> insert into regResults; >>> >>> in both case the first parameter let you to decide which learning methos >>> you want, moving window, batch processing or time based model learning. >>> BR, >>> Mahesh. >>> >>> On Sat, Jun 4, 2016 at 8:45 PM, Mahesh Dananjaya < >>> dananjayamah...@gmail.com> wrote: >>> >>>> Hi Maheshkaya, >>>> I have added the moving window method and update the previos >>>> StreamingLinearRegression [1] which only performed batch processing with >>>> streaming data. and also i added the StreamingKMeansClustering [1] for our >>>> purposes and debugged them.thank you. >>>> regards, >>>> Mahesh. >>>> [1] >>>> https://github.com/dananjayamahesh/GSOC2016/tree/master/gsoc/siddhi/extension/streaming/src/main/java/org/gsoc/siddhi/extension/streaming >>>> >>>> On Sat, Jun 4, 2016 at 5:58 PM, Supun Sethunga <sup...@wso2.com> wrote: >>>> >>>>> Thanks Mahesh! The graphs look promising! :) >>>>> >>>>> So by looking at graph, LR with SGD can train a model within 60 secs >>>>> (6*10^10 nano sec), using about 900,000 data points . Means, this online >>>>> training can handle events/data points coming at rate of 15,000 per second >>>>> (or more) , if the batch size is set to 900,000 (or less) or window size >>>>> is >>>>> set to 60 secs (or less). This is great IMO! >>>>> >>>>> On Sat, Jun 4, 2016 at 10:51 AM, Mahesh Dananjaya < >>>>> dananjayamah...@gmail.com> wrote: >>>>> >>>>>> Hi Maheshakya, >>>>>> As you requested i can change other parameters as well such as >>>>>> feature size(p). Initially i did it with p=3;sure thing. Anyway you can >>>>>> see >>>>>> and run the code if you want. source is at [1]. the test timing is called >>>>>> with random data as you requested if you set args[0] to 1. And you can >>>>>> find >>>>>> the extension and streaming algorithms in gsoc/ directiry[2]. thank you. >>>>>> BR, >>>>>> Mahesh. >>>>>> [1] >>>>>> https://github.com/dananjayamahesh/GSOC2016/blob/master/spark-examples/first-example/src/main/java/org/sparkexample/StreamingLinearRegression.java >>>>>> [2] https://github.com/dananjayamahesh/GSOC2016/tree/master/gsoc >>>>>> >>>>>> On Sat, Jun 4, 2016 at 10:39 AM, Mahesh Dananjaya < >>>>>> dananjayamah...@gmail.com> wrote: >>>>>> >>>>>>> Hi supun, >>>>>>> Though i pushed it yesterday, there was some problems with the >>>>>>> network. now you can see them in the repo location [1].I added some >>>>>>> Matlab >>>>>>> plot you can see the patter there.you can use ml also. Ok sure thing. I >>>>>>> can >>>>>>> prepare a report or else blog if you want. files are as follows. The y >>>>>>> axis >>>>>>> is in ns and x axis is in batch size. And also i added two pplots as >>>>>>> jpegs[2], so you can easily compare. >>>>>>> lr_timing_1000.txt -> batch size incremented by 1000 >>>>>>> lr_timing_10000.txt -> batch size incremented by 10000 >>>>>>> lr_timing_power10.txt -> batch size incremented by power of 10 >>>>>>> >>>>>>> In here independent variable is only tha batch size.If you want i >>>>>>> can send you making other parameters such as step size, number of >>>>>>> iteration, feature vector size as independent variables. please let me >>>>>>> know >>>>>>> if you want further info. thank you. >>>>>>> regards, >>>>>>> Mahesh. >>>>>>> >>>>>>> >>>>>>> [1 >>>>>>> ]https://github.com/dananjayamahesh/GSOC2016/tree/master/spark-examples/first-example/output >>>>>>> [2] >>>>>>> https://github.com/dananjayamahesh/GSOC2016/blob/master/spark-examples/first-example/output/lr_timing_10000.jpg >>>>>>> >>>>>>> On Sat, Jun 4, 2016 at 9:58 AM, Supun Sethunga <sup...@wso2.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Mahesh, >>>>>>>> >>>>>>>> I have added those timing reports to my repo [1]. >>>>>>>> >>>>>>>> Whats the file name? :) >>>>>>>> >>>>>>>> Btw, can you compile simple doc (gdoc) with the above results, and >>>>>>>> bring everything to one place? That way it is easy to compare, and keep >>>>>>>> track. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Supun >>>>>>>> >>>>>>>> On Fri, Jun 3, 2016 at 7:23 PM, Mahesh Dananjaya < >>>>>>>> dananjayamah...@gmail.com> wrote: >>>>>>>> >>>>>>>>> Hi Maheshkya, >>>>>>>>> I have added those timing reports to my repo [1].please have a >>>>>>>>> look at. three files are there. one is using incremet as 1000 for >>>>>>>>> batch >>>>>>>>> sizes (lr_timing_1000). Otherone is using incremet by 10000 >>>>>>>>> (lr_timing_10000) upto 1 million in both scenarios.you can see the >>>>>>>>> reports >>>>>>>>> and figures in the location [2] in the repo. i also added the >>>>>>>>> streaminglinearregression classes in the repo gsoc folder.thank you. >>>>>>>>> regards, >>>>>>>>> Mahesh. >>>>>>>>> [1]https://github.com/dananjayamahesh/GSOC2016 >>>>>>>>> [2] >>>>>>>>> https://github.com/dananjayamahesh/GSOC2016/tree/master/spark-examples/first-example/output >>>>>>>>> >>>>>>>>> On Mon, May 30, 2016 at 9:24 AM, Maheshakya Wijewardena < >>>>>>>>> mahesha...@wso2.com> wrote: >>>>>>>>> >>>>>>>>>> Hi Mahesh, >>>>>>>>>> >>>>>>>>>> Thank you for the update. I will look into your implementation. >>>>>>>>>> >>>>>>>>>> And i will be able to send you the timing/performances analysis >>>>>>>>>>> report tomorrow for the SGD functions >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Great. Sent those asap so that we can proceed. >>>>>>>>>> >>>>>>>>>> Best regards. >>>>>>>>>> >>>>>>>>>> On Sun, May 29, 2016 at 6:56 PM, Mahesh Dananjaya < >>>>>>>>>> dananjayamah...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Hi maheshakay, >>>>>>>>>>> I have implemented the linear regression with cep siddhi event >>>>>>>>>>> stream with taking batch sizes as parameters from the cep. Now we >>>>>>>>>>> can >>>>>>>>>>> trying the moving window method to. Before that i think i should >>>>>>>>>>> get your >>>>>>>>>>> opinion on data structures to save the streaming data.please check >>>>>>>>>>> my repo >>>>>>>>>>> [1] /gsoc/ folder there you can find all new things i add.. there >>>>>>>>>>> in the >>>>>>>>>>> extension folder you can find those extension. And i will be able >>>>>>>>>>> to send >>>>>>>>>>> you the timing/performances analysis report tomorrow for the SGD >>>>>>>>>>> functions. >>>>>>>>>>> thank you. >>>>>>>>>>> regards, >>>>>>>>>>> Mahesh. >>>>>>>>>>> [1] https://github.com/dananjayamahesh/GSOC2016/tree/master/gsoc >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Fri, May 27, 2016 at 12:56 PM, Mahesh Dananjaya < >>>>>>>>>>> dananjayamah...@gmail.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi maheshkaya, >>>>>>>>>>>> i have written some siddhi extension and trying to develop a >>>>>>>>>>>> one for my one. In time series example in the [1], can you please >>>>>>>>>>>> explain >>>>>>>>>>>> me the input format and query lines in that example for my >>>>>>>>>>>> understanding. >>>>>>>>>>>> >>>>>>>>>>>> from baseballData#timeseries:regress(2, 10000, 0.95, salary, >>>>>>>>>>>> rbi, walks, strikeouts, errors) >>>>>>>>>>>> select * >>>>>>>>>>>> insert into regResults; >>>>>>>>>>>> >>>>>>>>>>>> i just want to knwo how i give a set of data into this >>>>>>>>>>>> extension and what is baseballData. Is it input stream as usual.or >>>>>>>>>>>> any data >>>>>>>>>>>> file?how can i find that data set to create dummy input stream like >>>>>>>>>>>> baseballData? >>>>>>>>>>>> >>>>>>>>>>>> thank you. >>>>>>>>>>>> regards, >>>>>>>>>>>> Mahesh. >>>>>>>>>>>> [1] >>>>>>>>>>>> https://docs.wso2.com/display/CEP400/Writing+a+Custom+Stream+Processor+Extension >>>>>>>>>>>> >>>>>>>>>>>> On Thu, May 26, 2016 at 2:58 PM, Mahesh Dananjaya < >>>>>>>>>>>> dananjayamah...@gmail.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi Maheshakya, >>>>>>>>>>>>> today i got the siddhi and debug the math extention. then did >>>>>>>>>>>>> some changes and check. Now i am trying to write same kind of >>>>>>>>>>>>> extension in >>>>>>>>>>>>> my code base. so i add dependencies and it was built fine. Now i >>>>>>>>>>>>> am trying >>>>>>>>>>>>> to debug my extension and i did the same thing as i did in >>>>>>>>>>>>> previous case. >>>>>>>>>>>>> Cep is sending data, bu my extension is not firing in relevant >>>>>>>>>>>>> break point. >>>>>>>>>>>>> 1. So how can i debug the siddhi extension in my new >>>>>>>>>>>>> extension.(you can see it in my example repoo) >>>>>>>>>>>>> >>>>>>>>>>>>> I think if i do it correctly we can built the extension for >>>>>>>>>>>>> our purpose. And i will send the relevant timing report of SGD >>>>>>>>>>>>> algorithms >>>>>>>>>>>>> very soon as supun was asking me. thank you. >>>>>>>>>>>>> regards, >>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>> >>>>>>>>>>>>> On Tue, May 24, 2016 at 11:07 AM, Maheshakya Wijewardena < >>>>>>>>>>>>> mahesha...@wso2.com> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Also note that there is a calculation interval in the siddhi >>>>>>>>>>>>>> time series regression function[1]. You maybe able get some >>>>>>>>>>>>>> insight for >>>>>>>>>>>>>> this from that as well. >>>>>>>>>>>>>> >>>>>>>>>>>>>> [1] https://docs.wso2.com/display/CEP400/Regression >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Tue, May 24, 2016 at 11:03 AM, Maheshakya Wijewardena < >>>>>>>>>>>>>> mahesha...@wso2.com> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Mahesh, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> As we discussed offline, we can use similar mechanism to >>>>>>>>>>>>>>> train linear regression models, logistic regression models and >>>>>>>>>>>>>>> k-means >>>>>>>>>>>>>>> clustering models. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> It is very interesting that i have found that somethings >>>>>>>>>>>>>>>> that can make use of our work. In the cep 4.0 documentation >>>>>>>>>>>>>>>> there is a >>>>>>>>>>>>>>>> Custom Stream Processor Extention program [1]. There is a >>>>>>>>>>>>>>>> example of >>>>>>>>>>>>>>>> LinearRegressionStreamProcessor [1]. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> As we have to train predictive models with Spark, you can >>>>>>>>>>>>>>> write wrappers around regression/clustering models of Spark. >>>>>>>>>>>>>>> Refer to >>>>>>>>>>>>>>> Siddhi time series regression source codes[1][2]. You can write >>>>>>>>>>>>>>> a streaming >>>>>>>>>>>>>>> linear regression class for ML in a similar fashion by wrapping >>>>>>>>>>>>>>> Spark mllib >>>>>>>>>>>>>>> implementations. You can use the methods "addEvent", >>>>>>>>>>>>>>> "removeEvent", etc. >>>>>>>>>>>>>>> (may have to be changed according to requirements) for the >>>>>>>>>>>>>>> similar purpose. >>>>>>>>>>>>>>> You can introduce >>>>>>>>>>>>>>> trainLinearRegression/LogisticRegression/Kmeans which >>>>>>>>>>>>>>> does a similar thing as in createLinearRegression in those time >>>>>>>>>>>>>>> series >>>>>>>>>>>>>>> functions. In the processData method you can use Spark mllib >>>>>>>>>>>>>>> classes to >>>>>>>>>>>>>>> actually train models and return the model weights, evaluation >>>>>>>>>>>>>>> metrics. So, >>>>>>>>>>>>>>> converting streams into RDDs and retrieving information from >>>>>>>>>>>>>>> the trained >>>>>>>>>>>>>>> models shall happen in this method. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> In the stream processor extension example, you can retrieve >>>>>>>>>>>>>>> those values then use them to train new models with new batches. >>>>>>>>>>>>>>> Weights/cluster centers maybe passed as initialization >>>>>>>>>>>>>>> parameters for the >>>>>>>>>>>>>>> wrappers. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Please note that we have to figure out the best siddhi >>>>>>>>>>>>>>> extension type for this process. In the siddhi query, we define >>>>>>>>>>>>>>> batch size, >>>>>>>>>>>>>>> type of algorithm and number of features (there can be more). >>>>>>>>>>>>>>> After batch >>>>>>>>>>>>>>> size number of events received, train a model and save >>>>>>>>>>>>>>> parameters, return >>>>>>>>>>>>>>> evaluation metric. With the next batch, retrain the model >>>>>>>>>>>>>>> initialized with >>>>>>>>>>>>>>> previously learned parameters. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> We also may need to test the same scenario with a moving >>>>>>>>>>>>>>> window, but I suspect that that approach may become so slow as >>>>>>>>>>>>>>> a model is >>>>>>>>>>>>>>> trained each time an event is received. So, we may have to >>>>>>>>>>>>>>> change the >>>>>>>>>>>>>>> number of slots the moving window moves at a time (eg: not one >>>>>>>>>>>>>>> by one, but >>>>>>>>>>>>>>> ten by ten). >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Once this is resolved, majority of the research part will be >>>>>>>>>>>>>>> finished and all we will be left to do is implementing wrappers >>>>>>>>>>>>>>> around the >>>>>>>>>>>>>>> 3 learning algorithms we consider. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Best regards. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>> https://github.com/wso2/siddhi/blob/master/modules/siddhi-extensions/timeseries/src/main/java/org/wso2/siddhi/extension/timeseries/linreg/RegressionCalculator.java >>>>>>>>>>>>>>> [2] >>>>>>>>>>>>>>> https://github.com/wso2/siddhi/blob/master/modules/siddhi-extensions/timeseries/src/main/java/org/wso2/siddhi/extension/timeseries/linreg/SimpleLinearRegressionCalculator.java >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Sat, May 21, 2016 at 2:55 PM, Mahesh Dananjaya < >>>>>>>>>>>>>>> dananjayamah...@gmail.com> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi Maheshkya, >>>>>>>>>>>>>>>> shall we use [1] for our work? i am checking the >>>>>>>>>>>>>>>> possibility. >>>>>>>>>>>>>>>> BR, >>>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>> https://docs.wso2.com/display/CEP400/Writing+a+Custom+Stream+Processor+Extension >>>>>>>>>>>>>>>> [2] >>>>>>>>>>>>>>>> https://docs.wso2.com/display/CEP400/Inbuilt+Windows#InbuiltWindows-lengthlength >>>>>>>>>>>>>>>> [3] >>>>>>>>>>>>>>>> https://docs.wso2.com/display/CEP400/Writing+a+Custom+Aggregate+Function >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Sat, May 21, 2016 at 2:44 PM, Mahesh Dananjaya < >>>>>>>>>>>>>>>> dananjayamah...@gmail.com> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Hi Maheshakya, >>>>>>>>>>>>>>>>> It is very interesting that i have found that somethings >>>>>>>>>>>>>>>>> that can make use of our work. In the cep 4.0 documentation >>>>>>>>>>>>>>>>> there is a >>>>>>>>>>>>>>>>> Custom Stream Processor Extention program [1]. There is a >>>>>>>>>>>>>>>>> example of >>>>>>>>>>>>>>>>> LinearRegressionStreamProcessor [1] and also i saw >>>>>>>>>>>>>>>>> private int batchSize = 1000000000; i am going through >>>>>>>>>>>>>>>>> this one. >>>>>>>>>>>>>>>>> Please check whether we can use. WIll there be any >>>>>>>>>>>>>>>>> compatibility or support issue? >>>>>>>>>>>>>>>>> regards, >>>>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>>> https://docs.wso2.com/display/CEP400/Writing+a+Custom+Stream+Processor+Extension >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Sat, May 21, 2016 at 11:52 AM, Mahesh Dananjaya < >>>>>>>>>>>>>>>>> dananjayamah...@gmail.com> wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Hi maheshakya, >>>>>>>>>>>>>>>>>> anyway how can test any siddhi extention after write it >>>>>>>>>>>>>>>>>> without integrating it to cep.can you please explain me the >>>>>>>>>>>>>>>>>> procedure. i am >>>>>>>>>>>>>>>>>> referring to [1] [2] [3] [4]. thank you. >>>>>>>>>>>>>>>>>> BR, >>>>>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>>>> https://docs.wso2.com/display/CEP310/Writing+Extensions+to+Siddhi >>>>>>>>>>>>>>>>>> [2] >>>>>>>>>>>>>>>>>> https://docs.wso2.com/display/CEP310/Writing+a+Custom+Function >>>>>>>>>>>>>>>>>> [3] >>>>>>>>>>>>>>>>>> https://docs.wso2.com/display/CEP310/Writing+a+Custom+Window >>>>>>>>>>>>>>>>>> [4] >>>>>>>>>>>>>>>>>> https://docs.wso2.com/display/CEP400/Writing+Extensions+to+Siddhi >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Thu, May 19, 2016 at 12:08 PM, Mahesh Dananjaya < >>>>>>>>>>>>>>>>>> dananjayamah...@gmail.com> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Hi Maheshakya, >>>>>>>>>>>>>>>>>>> thank you for the feedback. I have add data-sets into >>>>>>>>>>>>>>>>>>> repo. data-sets/lr. I am all right with next week.Now i am >>>>>>>>>>>>>>>>>>> writing some >>>>>>>>>>>>>>>>>>> examples to collect samples and build mini batches and run >>>>>>>>>>>>>>>>>>> the algorithms >>>>>>>>>>>>>>>>>>> on those mini-batches. thank you. will add those into repo >>>>>>>>>>>>>>>>>>> soon.I am still >>>>>>>>>>>>>>>>>>> working on that siddhi extention.i will let you know the >>>>>>>>>>>>>>>>>>> progress. >>>>>>>>>>>>>>>>>>> BR, >>>>>>>>>>>>>>>>>>> mahesh. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On Thu, May 19, 2016 at 11:10 AM, Maheshakya Wijewardena >>>>>>>>>>>>>>>>>>> <mahesha...@wso2.com> wrote: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Hi Mahesh, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I've look into your code sample of streaming linear >>>>>>>>>>>>>>>>>>>> regression. Looks good to me, apart from few issues in >>>>>>>>>>>>>>>>>>>> coding practices >>>>>>>>>>>>>>>>>>>> which we can improve when you're doing the implementations >>>>>>>>>>>>>>>>>>>> in carbon-ml and >>>>>>>>>>>>>>>>>>>> during the code reviews. You are using a set of files as >>>>>>>>>>>>>>>>>>>> mini-batches of >>>>>>>>>>>>>>>>>>>> data, right? Can you also send us the datasets you've been >>>>>>>>>>>>>>>>>>>> using. I'd like >>>>>>>>>>>>>>>>>>>> to run this. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> does that cep problem is now all right that we were >>>>>>>>>>>>>>>>>>>>> trying to fix. I am still using those pre-build versions. >>>>>>>>>>>>>>>>>>>>> If so i can merge >>>>>>>>>>>>>>>>>>>>> with the latest one. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I'll check this and let you know. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Can we arrange a meeting (preferably in WSO2 offices) >>>>>>>>>>>>>>>>>>>> in next week with ML team members as well. Coding period >>>>>>>>>>>>>>>>>>>> begins on next >>>>>>>>>>>>>>>>>>>> Monday, so it's better to get overall feedback from others >>>>>>>>>>>>>>>>>>>> and discuss more >>>>>>>>>>>>>>>>>>>> about the project. Let me know convenient time slots for >>>>>>>>>>>>>>>>>>>> you. I'll arrange >>>>>>>>>>>>>>>>>>>> a meeting with ML team. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Best regards. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On Wed, May 18, 2016 at 9:53 AM, Mahesh Dananjaya < >>>>>>>>>>>>>>>>>>>> dananjayamah...@gmail.com> wrote: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Hi Maheshakya, >>>>>>>>>>>>>>>>>>>>> Ok. I will check it.you have sent me those relevant >>>>>>>>>>>>>>>>>>>>> references and i am working on that thing.thank you. does >>>>>>>>>>>>>>>>>>>>> that cep problem >>>>>>>>>>>>>>>>>>>>> is now all right that we were trying to fix. I am still >>>>>>>>>>>>>>>>>>>>> using those >>>>>>>>>>>>>>>>>>>>> pre-build versions. If so i can merge with the latest >>>>>>>>>>>>>>>>>>>>> one.thanks. >>>>>>>>>>>>>>>>>>>>> BR, >>>>>>>>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On Wed, May 18, 2016 at 9:44 AM, Maheshakya >>>>>>>>>>>>>>>>>>>>> Wijewardena <mahesha...@wso2.com> wrote: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Hi Mahesh, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> You don't actually have to implement anything in >>>>>>>>>>>>>>>>>>>>>> spark streaming. Try to understand how streaming data is >>>>>>>>>>>>>>>>>>>>>> handled in and the >>>>>>>>>>>>>>>>>>>>>> specifics of the underlying algorithms in streaming. >>>>>>>>>>>>>>>>>>>>>> What we want to do is having the similar algorithms >>>>>>>>>>>>>>>>>>>>>> that support CEP event streams with siddhi. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Best regards. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> On Wed, May 18, 2016 at 9:38 AM, Mahesh Dananjaya < >>>>>>>>>>>>>>>>>>>>>> dananjayamah...@gmail.com> wrote: >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Hi Maheshakya, >>>>>>>>>>>>>>>>>>>>>>> Did you check the repo. I will add recent works >>>>>>>>>>>>>>>>>>>>>>> today.And also i was going through the Java docs >>>>>>>>>>>>>>>>>>>>>>> related to spark streaming >>>>>>>>>>>>>>>>>>>>>>> work. It is with that scala API. thank you. >>>>>>>>>>>>>>>>>>>>>>> regards, >>>>>>>>>>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> On Tue, May 17, 2016 at 10:11 AM, Mahesh Dananjaya < >>>>>>>>>>>>>>>>>>>>>>> dananjayamah...@gmail.com> wrote: >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Hi Maheshakya, >>>>>>>>>>>>>>>>>>>>>>>> I have gone through the Java Docs and run some of >>>>>>>>>>>>>>>>>>>>>>>> the Spark examples on spark shell which are paramount >>>>>>>>>>>>>>>>>>>>>>>> improtant for our >>>>>>>>>>>>>>>>>>>>>>>> work. Then i have been writing my codes to check the >>>>>>>>>>>>>>>>>>>>>>>> Linear regression, K >>>>>>>>>>>>>>>>>>>>>>>> means for streaming. please check my git repo [1]. I >>>>>>>>>>>>>>>>>>>>>>>> think now i have to >>>>>>>>>>>>>>>>>>>>>>>> ask on dev regarding the capturing event streams for >>>>>>>>>>>>>>>>>>>>>>>> our work. I will >>>>>>>>>>>>>>>>>>>>>>>> update the recent things on git. check the >>>>>>>>>>>>>>>>>>>>>>>> park-example directory for java. >>>>>>>>>>>>>>>>>>>>>>>> examples run on git shell is not included there. In my >>>>>>>>>>>>>>>>>>>>>>>> case i think i have >>>>>>>>>>>>>>>>>>>>>>>> to build mini batches from data streams that comes as >>>>>>>>>>>>>>>>>>>>>>>> individual samples. >>>>>>>>>>>>>>>>>>>>>>>> Now i am working on some coding to collect mini >>>>>>>>>>>>>>>>>>>>>>>> batches from data >>>>>>>>>>>>>>>>>>>>>>>> streams.thank you. >>>>>>>>>>>>>>>>>>>>>>>> regards, >>>>>>>>>>>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>>>>>>>>>>> [1]https://github.com/dananjayamahesh/GSOC2016 >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> On Tue, May 17, 2016 at 10:10 AM, Mahesh Dananjaya >>>>>>>>>>>>>>>>>>>>>>>> <dananjayamah...@gmail.com> wrote: >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Hi Maheshakya, >>>>>>>>>>>>>>>>>>>>>>>>> I have gone through the Java Docs and run some of >>>>>>>>>>>>>>>>>>>>>>>>> the Spark examples on spark shell which are paramount >>>>>>>>>>>>>>>>>>>>>>>>> improtant for our >>>>>>>>>>>>>>>>>>>>>>>>> work. Then i have been writing my codes to check the >>>>>>>>>>>>>>>>>>>>>>>>> Linear regression, K >>>>>>>>>>>>>>>>>>>>>>>>> means for streaming. please check my git repo [1]. I >>>>>>>>>>>>>>>>>>>>>>>>> think now i have to >>>>>>>>>>>>>>>>>>>>>>>>> ask on dev regarding the capturing event streams for >>>>>>>>>>>>>>>>>>>>>>>>> our work. I will >>>>>>>>>>>>>>>>>>>>>>>>> update the recent things on git. check the >>>>>>>>>>>>>>>>>>>>>>>>> park-example directory for java. >>>>>>>>>>>>>>>>>>>>>>>>> examples run on git shell is not included there. In >>>>>>>>>>>>>>>>>>>>>>>>> my case i think i have >>>>>>>>>>>>>>>>>>>>>>>>> to build mini batches from data streams that comes as >>>>>>>>>>>>>>>>>>>>>>>>> individual samples. >>>>>>>>>>>>>>>>>>>>>>>>> Now i am working on some coding to collect mini >>>>>>>>>>>>>>>>>>>>>>>>> batches from data >>>>>>>>>>>>>>>>>>>>>>>>> streams.thank you. >>>>>>>>>>>>>>>>>>>>>>>>> regards, >>>>>>>>>>>>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>>>>>>>>>>>> [1]https://github.com/dananjayamahesh/GSOC2016 >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> On Mon, May 16, 2016 at 1:19 PM, Mahesh Dananjaya >>>>>>>>>>>>>>>>>>>>>>>>> <dananjayamah...@gmail.com> wrote: >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Hi Maheshakya, >>>>>>>>>>>>>>>>>>>>>>>>>> thank you. i will update the repo today.thank >>>>>>>>>>>>>>>>>>>>>>>>>> you.i changed the carbon ml siddhi extention and see >>>>>>>>>>>>>>>>>>>>>>>>>> how the changes are >>>>>>>>>>>>>>>>>>>>>>>>>> effecting. i will update the progress as soon as >>>>>>>>>>>>>>>>>>>>>>>>>> possible.thank you. i had >>>>>>>>>>>>>>>>>>>>>>>>>> some problem in spark mllib dependency. i was fixing >>>>>>>>>>>>>>>>>>>>>>>>>> that. >>>>>>>>>>>>>>>>>>>>>>>>>> regards, >>>>>>>>>>>>>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>>>>>>>>>>>>> p.s: do i need to maintain a blog? >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, May 16, 2016 at 10:02 AM, Maheshakya >>>>>>>>>>>>>>>>>>>>>>>>>> Wijewardena <mahesha...@wso2.com> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Mahesh, >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry for replying late. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you for the update. I believe you have >>>>>>>>>>>>>>>>>>>>>>>>>>> done some implementations with with Spark MLLIb >>>>>>>>>>>>>>>>>>>>>>>>>>> algorithms in streaming >>>>>>>>>>>>>>>>>>>>>>>>>>> fashion as we have discussed. If so, can you please >>>>>>>>>>>>>>>>>>>>>>>>>>> share your code in a >>>>>>>>>>>>>>>>>>>>>>>>>>> Github repo. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Now i want to implements some machine learning >>>>>>>>>>>>>>>>>>>>>>>>>>>> algorithms with importing mllib and want to run >>>>>>>>>>>>>>>>>>>>>>>>>>>> within your code base >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> For the moment you can try out editing the same >>>>>>>>>>>>>>>>>>>>>>>>>>> class PredictStreamProcessor in the siddhi >>>>>>>>>>>>>>>>>>>>>>>>>>> extension in carbon-ml. Later we >>>>>>>>>>>>>>>>>>>>>>>>>>> will add this separately. You should be able to add >>>>>>>>>>>>>>>>>>>>>>>>>>> org.apache.spark.mllib. >>>>>>>>>>>>>>>>>>>>>>>>>>> classes to there. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> And i want to see how event streams are coming >>>>>>>>>>>>>>>>>>>>>>>>>>>> from cep. As i think it is not in a RDD format >>>>>>>>>>>>>>>>>>>>>>>>>>>> since it is arriving as the >>>>>>>>>>>>>>>>>>>>>>>>>>>> individual samples. I will send a email to dev >>>>>>>>>>>>>>>>>>>>>>>>>>>> asking about how to get the >>>>>>>>>>>>>>>>>>>>>>>>>>>> streams. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Please pay attention to length[1] and >>>>>>>>>>>>>>>>>>>>>>>>>>> lengthbatch[1] inbuilt windows in siddhi. What you >>>>>>>>>>>>>>>>>>>>>>>>>>> need to write are >>>>>>>>>>>>>>>>>>>>>>>>>>> functions similar to a custom aggregate function[2]. >>>>>>>>>>>>>>>>>>>>>>>>>>> When you send the email to dev list, explain >>>>>>>>>>>>>>>>>>>>>>>>>>> your requirement. You need to get a set of event >>>>>>>>>>>>>>>>>>>>>>>>>>> with from a stream with a >>>>>>>>>>>>>>>>>>>>>>>>>>> specified window size (number of events). Then >>>>>>>>>>>>>>>>>>>>>>>>>>> build a model within that >>>>>>>>>>>>>>>>>>>>>>>>>>> function. You also need to retain the data (learned >>>>>>>>>>>>>>>>>>>>>>>>>>> weights, cluster >>>>>>>>>>>>>>>>>>>>>>>>>>> centers, etc.) from the previous window to use in >>>>>>>>>>>>>>>>>>>>>>>>>>> the current window. Ask >>>>>>>>>>>>>>>>>>>>>>>>>>> what can be the most suitable option for this among >>>>>>>>>>>>>>>>>>>>>>>>>>> the set of siddhi >>>>>>>>>>>>>>>>>>>>>>>>>>> extensions given. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>>>>>>>>>>>>> https://docs.wso2.com/display/CEP400/Inbuilt+Windows#InbuiltWindows-lengthlength >>>>>>>>>>>>>>>>>>>>>>>>>>> [2] >>>>>>>>>>>>>>>>>>>>>>>>>>> https://docs.wso2.com/display/CEP400/Writing+a+Custom+Aggregate+Function >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, May 11, 2016 at 1:43 PM, Mahesh >>>>>>>>>>>>>>>>>>>>>>>>>>> Dananjaya <dananjayamah...@gmail.com> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> ---------- Forwarded message ---------- >>>>>>>>>>>>>>>>>>>>>>>>>>>> From: Mahesh Dananjaya < >>>>>>>>>>>>>>>>>>>>>>>>>>>> dananjayamah...@gmail.com> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Date: Wed, May 11, 2016 at 1:43 PM >>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re: [Dev] GSOC2016: [ML][CEP] >>>>>>>>>>>>>>>>>>>>>>>>>>>> Predictive analytic with online data for WSO2 >>>>>>>>>>>>>>>>>>>>>>>>>>>> Machine Learner >>>>>>>>>>>>>>>>>>>>>>>>>>>> To: Maheshakya Wijewardena <mahesha...@wso2.com >>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Maheshakya, >>>>>>>>>>>>>>>>>>>>>>>>>>>> sorry for not updating. I did what you wanted >>>>>>>>>>>>>>>>>>>>>>>>>>>> me to do. I checked the code base and train >>>>>>>>>>>>>>>>>>>>>>>>>>>> functions. I went through those >>>>>>>>>>>>>>>>>>>>>>>>>>>> java docs. I went through the carbon-ml current >>>>>>>>>>>>>>>>>>>>>>>>>>>> implementation of LG and >>>>>>>>>>>>>>>>>>>>>>>>>>>> K-Mean. And i had Apache Spark and i tried with >>>>>>>>>>>>>>>>>>>>>>>>>>>> several examples. Now i >>>>>>>>>>>>>>>>>>>>>>>>>>>> want to implements some machine learning >>>>>>>>>>>>>>>>>>>>>>>>>>>> algorithms with importing mllib >>>>>>>>>>>>>>>>>>>>>>>>>>>> and want to run within your code base. Can you >>>>>>>>>>>>>>>>>>>>>>>>>>>> help me with that. >>>>>>>>>>>>>>>>>>>>>>>>>>>> And i want to see how event streams are coming >>>>>>>>>>>>>>>>>>>>>>>>>>>> from cep. As i think it is not in a RDD format >>>>>>>>>>>>>>>>>>>>>>>>>>>> since it is arriving as the >>>>>>>>>>>>>>>>>>>>>>>>>>>> individual samples. I will send a email to dev >>>>>>>>>>>>>>>>>>>>>>>>>>>> asking about how to get the >>>>>>>>>>>>>>>>>>>>>>>>>>>> streams. I debugged many of those functions in the >>>>>>>>>>>>>>>>>>>>>>>>>>>> code base. So need >>>>>>>>>>>>>>>>>>>>>>>>>>>> further instructions to proceed.thank you. >>>>>>>>>>>>>>>>>>>>>>>>>>>> regards, >>>>>>>>>>>>>>>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, May 11, 2016 at 10:32 AM, Maheshakya >>>>>>>>>>>>>>>>>>>>>>>>>>>> Wijewardena <mahesha...@wso2.com> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Mahesh, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Any update on your progress? >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, May 4, 2016 at 8:35 PM, Maheshakya >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wijewardena <mahesha...@wso2.com> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Mahesh, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is that "Put break points in train methods in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Linear Regression class" means the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> spark/algorithms/ LinearRegrassion.java >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class in the org.wso2.carbon.ml.core? is that >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the correct file? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, this is the correct place. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can refer to spark programming >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> guide[1][2] as well as our ML code base when you >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> try those algorithms out. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please try to do rough implementations of the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> streaming versions of linear >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regression, logistic regression and k-means >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> clustering as we have discussed >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the proposal in plain Java. It's better if >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you can create a git repo and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share your code once you have made some progress. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Were you able debug and understand the flow >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the ML siddhi extension? I hope you haven't >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> encountered more errors >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after switching the released version of CEP. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is this Friday okay for you? Afternoon at >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2:00 pm? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://spark.apache.org/docs/latest/programming-guide.html >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://spark.apache.org/docs/latest/mllib-guide.html >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, May 4, 2016 at 1:07 PM, Mahesh >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dananjaya <dananjayamah...@gmail.com> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Maheshakya, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I have been looking into some algorithms >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> related to stochastic gradient descent based >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> algorithms.anything i should >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> focus please let me know.Ans also i will be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> available for calling this week >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and next week.thank you. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> BR, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, May 3, 2016 at 5:05 PM, Mahesh >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dananjaya <dananjayamah...@gmail.com> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Maheshakya, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thank you.that's good. i have been trying >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to fix that for couple of days. please inform >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> me when it will be fixed.now >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> i have been testing the ML algorithms and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> trying to identify the flow and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hierarchy. is that "Put break points in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> train methods in Linear >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Regression class" means the spark/algorithms/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LinearRegrassion.java class >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the org.wso2.carbon.ml.core? is that the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correct file? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> And also i am planning to write some >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> programs to use apache spark mllib algorithms. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and i refer to [1] and some >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wso2 documentations to get some idea about ML >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure.thank you. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> BR, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]nirmalfdo.blogspot.com >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, May 3, 2016 at 4:36 PM, Maheshakya >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wijewardena <mahesha...@wso2.com> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Mahesh, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I have checked. It seems the issue you >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have encountered is cause only in the current >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> development branch of the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> product-cep. It doesn't identify the ML >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> siddhi extension as an extension. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ML siddhi extension works fine in the latest >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release of CEP (4.1.0) [1]. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until we figure out the reason and come up >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a solution, can you use the latest CEP >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release for your work. It's >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fine to use that since you haven't started >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> actual development yet. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://wso2.com/products/complex-event-processor/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, May 3, 2016 at 3:19 PM, Maheshakya >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wijewardena <mahesha...@wso2.com> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Mahesh, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is is vital to use those local repo in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my upcoming implementation? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes. The remote p2-repo contains the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> p2-repos of released versions. What you have >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to develop on is the current >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master of the carbon-ml and product-ml. You >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can try out with the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> modification I have suggested. In the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> meantime, I'll verify whether the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> current repos are working as expected. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> And also i am trying to debug the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> carbon-ml org.wso2.carbon.ml.core by >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> putting some break point in the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> spark/algorithms/Linear Regression >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's great that you have started looking >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at the implementation of linear regression >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well. Put break points in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> train methods in LinearRegression class. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is being used when you run >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> linear regression from UI. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I can see some comments left behind for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> streaming algo as well.thank you >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You may be referring to the linear >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regression with SGD model. Here, there's no >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> retraining with streaming data >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> involved. The SGD with minibatches is used >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to train the model with the data >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> set only once. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What you have to do is create a similar >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mechanism to involve streaming data and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> retrain models. We will get to that >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part once you get comfortable with siddhi >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> extensions. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> BTW, is it possible for you to join a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> call on this Friday or in the next week. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We'll try to resolve your current >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues and discuss further on project. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, May 3, 2016 at 1:03 PM, Mahesh >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dananjaya <dananjayamah...@gmail.com> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi maheshakya, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is it ok to go with p2 repo at >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://product-dist.wso2.com/p2/carbon/releases/wilkes/features/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rather than the P2-repo at >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> product-ml/modules/p2-profile/target/p2-repo >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> local repo.What is the impact?.Is is vital >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to use those local repo in my >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> upcoming implementation?so i was trying to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> give remote p2 repo to cep built >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by source and trying to debug the cep ml >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> extension and got the same error >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as yesterday. But pre-built product is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> working fine. Therefore now i am >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> trying as you described in the last email. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> And also i am trying to debug the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> carbon-ml org.wso2.carbon.ml.core by >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> putting some break point in the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> spark/algorithms/Linear Regression. I am >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> trying to trigger it with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> product-ml project with data set. Does that >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Linear Regression also in UI is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consuming that spark algorithms or is it in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> another place? I can see some >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> comments left behind for streaming algo as >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> well.thank you. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> BR, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, May 3, 2016 at 9:35 AM, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Maheshakya Wijewardena < >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mahesha...@wso2.com> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Mahesh, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The earlier error you have mentioned >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> may occur due to the incompatible Siddhi >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> versions in ML p2-repo and CEP, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when you add the p2-repo from prodcut-ml >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you built. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Current siddhi version in product-cep >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is 3.0.6-SNAPSHOT[1], but in ML it's 3.0.2. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you try changing the siddhi.version >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in carbon-ml/pom.xml to 3.0.6-SNAPSHOT, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> build carbon-ml, then build >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> product-ml again. After this, add p2-repo >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as a local repository again to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fresh CEP pack and try it out. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, May 2, 2016 at 7:02 PM, Mahesh >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dananjaya <dananjayamah...@gmail.com> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Maheshakya, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> now i remotely debug the CEP extension >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for ML Prediction. What i did was, i have >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> all the pre-build version of CEP >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and ML. Therefore i used that built CEP >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and did the same thing that i was >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> doing with the source code. I think the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only change i did was install those >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> packages from remote p2 repo. This was >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work fine and i debugged the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> carbon-ml >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> org.wso2.carbon.ml.siddhi.extension as >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> described in the [1]. So >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> now i have to try same thing with the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> build by source content.thank you. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://docs.wso2.com/display/ML110/WSO2+CEP+Extension+for+ML+Predictions#WSO2CEPExtensionforMLPredictions-Siddhisyntaxfortheextension >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> BR, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mahesh >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Apr 25, 2016 at 5:49 PM, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Maheshakya Wijewardena < >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mahesha...@wso2.com> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Mahesh, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Congratulations and welcome to GSoC >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2016. You did a great job in preparing >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the proposal. Now it's time to dig >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deep and get started with the project. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> First of all you need to familiarize >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with the code base. We have agreed to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement this with CEP event >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> streams. We already have a CEP extension >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for predictions [1][2]. Go through >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this implementation and familiarize your >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> self with that. You need to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand how: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1. Even streams are consumed >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2. predictions are made from >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual event >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3. Results are sent back >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Get WSO2 ML and CEP sources (You may >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use latest released version of CEP) and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> build the products. Get both >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> carbon-ml[3] and product-ml[4] masters >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and create new branches for your >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work from masters. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> After you build the products, you may >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need to do remote debugging[5] to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand the flow. So please follow an >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> example of real time prediction with ML >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with debugging and get some idea. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The component you need to debug is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> org.wso2.carbon.ml.siddhi.extension. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Next tasks would be implementing >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> online learning algorithms in plain java >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with spark ml lib and integrating >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> those to ML. We also need to come up >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a proper and detailed >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> architecture to employ those algorithms >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in ML. Getting familiar with the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> aforementioned sections would give you >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> some insight on how this should be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implemented. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So please try to get a quick grasp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then you can start the implementation. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Let us know if you have any >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> questions or you get stuck somewhere. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, please always add WSO2 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> developer's list as well when you >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicate with us regarding the project >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so that you can get opinions and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feedback from others as well. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://docs.wso2.com/display/ML110/WSO2+CEP+Extension+for+ML+Predictions#WSO2CEPExtensionforMLPredictions-Siddhisyntaxfortheextension >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://github.com/wso2/carbon-ml/tree/master/components/extensions/org.wso2.carbon.ml.siddhi.extension >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3] https://github.com/wso2/carbon-ml >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [4] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://github.com/wso2/product-ml >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [5] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://dzone.com/articles/how-debug-wso2-carbon-kernel >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Apr 25, 2016 at 3:33 PM, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mahesh Dananjaya < >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dananjayamah...@gmail.com> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,thank you for accepting my GSOC >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2016 proposal and i am looking forward >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for the further instruction and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> project continuation. thank you very >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> much. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Pruthuvi Maheshakya >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wijewardenamahesha...@wso2.com >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +94711228855 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Pruthuvi Maheshakya >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wijewardenamahesha...@wso2.com >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +94711228855 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Pruthuvi Maheshakya Wijewardena >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mahesha...@wso2.com >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +94711228855 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Pruthuvi Maheshakya Wijewardena >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mahesha...@wso2.com >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +94711228855 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Pruthuvi Maheshakya Wijewardena >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mahesha...@wso2.com >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +94711228855 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Pruthuvi Maheshakya Wijewardena >>>>>>>>>>>>>>>>>>>>>>>>>>>>> mahesha...@wso2.com >>>>>>>>>>>>>>>>>>>>>>>>>>>>> +94711228855 >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>>>>>>>>>>>>>>>>> Dev mailing list >>>>>>>>>>>>>>>>>>>>>>>>>>>> Dev@wso2.org >>>>>>>>>>>>>>>>>>>>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>>>>>> Pruthuvi Maheshakya Wijewardena >>>>>>>>>>>>>>>>>>>>>>>>>>> mahesha...@wso2.com >>>>>>>>>>>>>>>>>>>>>>>>>>> +94711228855 >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>> Pruthuvi Maheshakya Wijewardena >>>>>>>>>>>>>>>>>>>>>> mahesha...@wso2.com >>>>>>>>>>>>>>>>>>>>>> +94711228855 >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>> Pruthuvi Maheshakya Wijewardena >>>>>>>>>>>>>>>>>>>> mahesha...@wso2.com >>>>>>>>>>>>>>>>>>>> +94711228855 >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> Pruthuvi Maheshakya Wijewardena >>>>>>>>>>>>>>> mahesha...@wso2.com >>>>>>>>>>>>>>> +94711228855 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> Pruthuvi Maheshakya Wijewardena >>>>>>>>>>>>>> mahesha...@wso2.com >>>>>>>>>>>>>> +94711228855 >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Pruthuvi Maheshakya Wijewardena >>>>>>>>>> mahesha...@wso2.com >>>>>>>>>> +94711228855 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> *Supun Sethunga* >>>>>>>> Senior Software Engineer >>>>>>>> WSO2, Inc. >>>>>>>> http://wso2.com/ >>>>>>>> lean | enterprise | middleware >>>>>>>> Mobile : +94 716546324 >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> *Supun Sethunga* >>>>> Senior Software Engineer >>>>> WSO2, Inc. >>>>> http://wso2.com/ >>>>> lean | enterprise | middleware >>>>> Mobile : +94 716546324 >>>>> >>>> >>>> >>> >> > -- Pruthuvi Maheshakya Wijewardena mahesha...@wso2.com +94711228855
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
