I am Ok to do both. On Fri, Sep 25, 2015 at 9:28 AM, Nirmal Fernando <[email protected]> wrote:
> Hi Srinath, > > Thanks for your thoughts. Please find my comments inline. > > On Fri, Sep 25, 2015 at 9:11 AM, Srinath Perera <[email protected]> wrote: > >> Hi Nirmal, >> >> However, ensemble would need test data to decide on the weight given to >> each model ( am I correct?) >> > > Yes (to be precise, I think the performance measurements of a model), we > could persist the performance measurements related to each of the model in > model itself. That way, it is available at the prediction time, if someone > intend to use. > > >> If that is the case, we have to do this in the ML, not CEP. >> >> I think also we need stats etc on accuracy. So we have to do this in ML >> anyway. >> > > Yes, we need to implement ensemble support in ML too, so that users can > easily select the models they wish and create an ensemble out of it and get > to see performance measurements of the ensemble in ML itself and use the > ensemble for prediction too. IMO having ensemble behaviour implemented at > CEP extension will also help at least as a short term solution. Wdyt? > > >> Thanks >> Srinath >> >> On Thu, Sep 24, 2015 at 10:37 PM, Maheshakya Wijewardena < >> [email protected]> wrote: >> >>> Indeed we should include bagging and boosting for linear models for >>> bias-variance trade-off. But these methods only concern a single learner at >>> a given moment and trains on the same dataset by sampling. >>> Our most imminent concern is to create ensemble of models created with >>> different datasets (different, but same features). >>> >>> It would be ideal if we can get a bagging mechanism for dataset versions. >>> >>> On Thu, Sep 24, 2015 at 10:28 PM, Upul Bandara <[email protected]> wrote: >>> >>>> Looks like a nice idea to try. >>>> >>>> But I have a little concern regarding how this is going affect the >>>> performance of stream processing especially when we have some expensive >>>> algorithms as base learners. >>>> >>>> As an alternative, we can try bagging with less expensive algorithms. >>>> >>>> >>>> >>>> On Thu, Sep 24, 2015 at 9:54 PM, Maheshakya Wijewardena < >>>> [email protected]> wrote: >>>> >>>>> There are constrained optimization techniques to determine the optimal >>>>> convex combinations of weights which we can implement, but at the moment >>>>> we >>>>> need to get the vanilla majority voting scheme implemented. >>>>> Moreover the weighting will be more important in numerical prediction. >>>>> >>>>> On Thu, Sep 24, 2015 at 9:48 PM, Nirmal Fernando <[email protected]> >>>>> wrote: >>>>> >>>>>> Thanks Supun! Initially I thought to have same weights, but excellent >>>>>> suggestion on accuracy based weights. >>>>>> >>>>>> On Thu, Sep 24, 2015 at 9:45 PM, Supun Sethunga <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> +1 for the idea, and looks very feasible! >>>>>>> >>>>>>> May be we need to decide on a voting criteria, if we already don't >>>>>>> have any, such as whether to assign similar weights to all the >>>>>>> classifiers, >>>>>>> or to assigns weights on their accuracy at the validation phase, etc.. >>>>>>> >>>>>>> Thanks, >>>>>>> Supun >>>>>>> >>>>>>> >>>>>>> On Thu, Sep 24, 2015 at 11:54 AM, Nirmal Fernando <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi All, >>>>>>>> >>>>>>>> In statistics and *machine learning*, *ensemble* methods use >>>>>>>> multiple *learning* algorithms to obtain better predictive >>>>>>>> performance that could be obtained from any of the constituent >>>>>>>> *learning* algorithms. >>>>>>>> >>>>>>>> We thought of implementing ensemble in CEP-ML extension. CEP-ML >>>>>>>> extension will be initialized using a list of ML model paths. When an >>>>>>>> event >>>>>>>> is received, CEP-ML extension will perform predictions using all the >>>>>>>> models >>>>>>>> and output the majority vote. >>>>>>>> >>>>>>>> We can implement the same, in ESB-ML extension. >>>>>>>> >>>>>>>> Thoughts are welcome! >>>>>>>> >>>>>>>> >>>>>>>> ---------- Forwarded message ---------- >>>>>>>> From: Manorama Perera <[email protected]> >>>>>>>> Date: Thu, May 14, 2015 at 3:35 PM >>>>>>>> Subject: CEP Extension for Machine Learner Predictions >>>>>>>> To: architecture <[email protected]> >>>>>>>> Cc: Nirmal Fernando <[email protected]>, Srinath Perera < >>>>>>>> [email protected]>, Supun Sethunga <[email protected]>, Upul Bandara < >>>>>>>> [email protected]>, Sriskandarajah Suhothayan <[email protected]>, >>>>>>>> Maheshakya Wijewardena <[email protected]> >>>>>>>> >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> We are in the process of implementing a CEP extension for Machine >>>>>>>> Learner Predictions. This extension allows the machine learning models >>>>>>>> generated by WSO2 ML to be used within CEP for predictions. >>>>>>>> >>>>>>>> To use this, following ML features need to be installed in CEP. >>>>>>>> >>>>>>>> - Machine Learner Core feature >>>>>>>> - Machine Learner Commons feature >>>>>>>> - Machine Learner Database Service feature >>>>>>>> >>>>>>>> This extension is implemented as a *StreamProcessor*. >>>>>>>> >>>>>>>> *The syntax :* >>>>>>>> >>>>>>>> There are two possible ways to use the extension. >>>>>>>> >>>>>>>> *<stream-name>#ml:predict(‘<path-to-ML-model>’) * >>>>>>>> >>>>>>>> *<stream-name>#ml:predict('<path-to-ML-model>', attribute 1, >>>>>>>> attribute 2, .......)* >>>>>>>> >>>>>>>> *path-to-MLModel* >>>>>>>> >>>>>>>> The storage location of the Machine learning model can be either >>>>>>>> registry or file system. >>>>>>>> >>>>>>>> If the model is stored in the registry, *path-to-ML-model* should >>>>>>>> have the prefix *registry:* >>>>>>>> If the model is stored in the file system, *path-to-ML-model* >>>>>>>> should have the prefix *file:* >>>>>>>> >>>>>>>> *attribute 1, attribute 2, ….* >>>>>>>> >>>>>>>> These are the attribute names of the stream. The values of these >>>>>>>> attributes are sent to the MLModel as feature input values. When the >>>>>>>> attribute names are not explicitly given, the extension will map the >>>>>>>> attribute names of the stream with the feature names of the ML model. >>>>>>>> >>>>>>>> The output events will contain the attribute* prediction* which >>>>>>>> holds the prediction result for that particular event. >>>>>>>> >>>>>>>> Thanks. >>>>>>>> >>>>>>>> -- >>>>>>>> Manorama Perera >>>>>>>> Software Engineer >>>>>>>> WSO2, Inc.; http://wso2.com/ >>>>>>>> Mobile : +94716436216 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> Thanks & regards, >>>>>>>> Nirmal >>>>>>>> >>>>>>>> Team Lead - WSO2 Machine Learner >>>>>>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>>>>>> Mobile: +94715779733 >>>>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> *Supun Sethunga* >>>>>>> Software Engineer >>>>>>> WSO2, Inc. >>>>>>> http://wso2.com/ >>>>>>> lean | enterprise | middleware >>>>>>> Mobile : +94 716546324 >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Thanks & regards, >>>>>> Nirmal >>>>>> >>>>>> Team Lead - WSO2 Machine Learner >>>>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>>>> Mobile: +94715779733 >>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Pruthuvi Maheshakya Wijewardena >>>>> Software Engineer >>>>> WSO2 : http://wso2.com/ >>>>> Email: [email protected] >>>>> Mobile: +94711228855 >>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> Upul Bandara, >>>> Associate Technical Lead, WSO2, Inc., >>>> Mob: +94 715 468 345. >>>> >>> >>> >>> >>> -- >>> Pruthuvi Maheshakya Wijewardena >>> Software Engineer >>> WSO2 : http://wso2.com/ >>> Email: [email protected] >>> Mobile: +94711228855 >>> >>> >>> >> >> >> -- >> ============================ >> Blog: http://srinathsview.blogspot.com twitter:@srinath_perera >> Site: http://people.apache.org/~hemapani/ >> Photos: http://www.flickr.com/photos/hemapani/ >> Phone: 0772360902 >> > > > > -- > > Thanks & regards, > Nirmal > > Team Lead - WSO2 Machine Learner > Associate Technical Lead - Data Technologies Team, WSO2 Inc. > Mobile: +94715779733 > Blog: http://nirmalfdo.blogspot.com/ > > > -- ============================ Blog: http://srinathsview.blogspot.com twitter:@srinath_perera Site: http://people.apache.org/~hemapani/ Photos: http://www.flickr.com/photos/hemapani/ Phone: 0772360902
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
