Hi Supun, Are you using UDAFs for the purpose? Can explain a bit how it's done?
On Wed, Jul 23, 2014 at 5:30 PM, Supun Sethunga <[email protected]> wrote: > Hi, > > The back-end service for the $subject has been implemented. For the time > being, it supports three algorithms (Naive Bayes, Logistic Regression and > Multilayer Perceptrons). The service accepts a JSON object having the input > data file path, output path where the created the model should be saved, > the algorithm to be used for modeling (one of the above three algos), and > the hyper-parameters and their values. > > Next goal would be to develop a carbon UI component, which allows a user > to call the above service, in a form of a wizard. The wizard would provide > a step-by-step guide to the user to create a ML model for any set of data. > > Appreciate any suggestions. > > Regards, > Supun > > > On Fri, Jul 4, 2014 at 7:39 AM, Srinath Perera <[email protected]> wrote: > >> Anjana and myself discussed. >> >> UDFs does not work. UDFs take an event as the input while mahout takes a >> complete dataset. >> >> May be we can use UDAF (A = Aggregate), but has to explore more. >> >> >> On Tue, Jul 1, 2014 at 10:03 AM, Anjana Fernando <[email protected]> wrote: >> >>> I see, sure, I was thinking of doing all the operations, including the >>> training operations using an UDF. Will come and meet you. >>> >>> Cheers, >>> Anjana. >>> >>> >>> On Tue, Jul 1, 2014 at 9:56 AM, Srinath Perera <[email protected]> wrote: >>> >>>> No, we need to get the data, preprocess them using hive, and send all >>>> the data (not 1-2 values, rather say 10 millions values) to training phase. >>>> Lets chat f2f a bit. >>>> >>>> --Srinath >>>> >>>> >>>> On Tue, Jul 1, 2014 at 6:24 AM, Anjana Fernando <[email protected]> >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> I was simply thinking, the UDF could directly mapped to some basic >>>>> Mahout operation it implements, and the input/output should be given as >>>>> parameters to the UDF, so probably, we can publish some input data >>>>> beforehand to Cassandra etc.. and give the location of that data to the >>>>> UDF, and the UDF will, as it is called, create the map/reduce jobs and >>>>> execute. >>>>> >>>>> Cheers, >>>>> Anjana. >>>>> >>>>> >>>>> On Tue, Jul 1, 2014 at 9:18 AM, Srinath Perera <[email protected]> >>>>> wrote: >>>>> >>>>>> +1 we wanted to explore that more. >>>>>> >>>>>> However, It is not a simple UDF as this is a stateful op where we >>>>>> feed lot of data and start a separate map reduce process. Anjana, do you >>>>>> have any thoughts on how it can be done? >>>>>> >>>>>> >>>>>> On Tue, Jul 1, 2014 at 5:37 AM, Anjana Fernando <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I'm just wondering if we have any way to integrate this to Hive >>>>>>> itself (UDF?), to get results of an ML algorithm run, to a result >>>>>>> there. A >>>>>>> similar scenario is possible in Shark/MLlib integration. >>>>>>> >>>>>>> Cheers, >>>>>>> Anjana. >>>>>>> >>>>>>> >>>>>>> On Mon, Jun 30, 2014 at 12:28 PM, Supun Sethunga <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> Im working on the $subject, and the objective is to apply Machine >>>>>>>> Learning algorithms on the data stored by WSO2 BAM. Apache Mahout will >>>>>>>> be >>>>>>>> used as the ML tool, for this purpose. >>>>>>>> >>>>>>>> As per the discussion I had With Srinath, the procedure for >>>>>>>> $subject would be: >>>>>>>> >>>>>>>> - Test a Machine Learning algorithm using Mahout libraries >>>>>>>> within Java. >>>>>>>> - Implement a RESTful service which provides the above >>>>>>>> functionality. >>>>>>>> - Since Mahout also uses Hadoop, the above service can send Map >>>>>>>> Reduce Jobs to the Hadoop built inside the BAM. >>>>>>>> - Deploy the service as a Carbon Component on WSO2 BAM. >>>>>>>> >>>>>>>> The first step is completed for now. >>>>>>>> Any feedback is highly appreciated. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Supun >>>>>>>> >>>>>>>> -- >>>>>>>> *Supun Sethunga* >>>>>>>> Software Engineer >>>>>>>> WSO2, Inc. >>>>>>>> lean | enterprise | middleware >>>>>>>> Mobile : +94 716546324 >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> *Anjana Fernando* >>>>>>> Senior Technical Lead >>>>>>> WSO2 Inc. | http://wso2.com >>>>>>> lean . enterprise . middleware >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> ============================ >>>>>> Director, Research, WSO2 Inc. >>>>>> Visiting Faculty, University of Moratuwa >>>>>> Member, Apache Software Foundation >>>>>> Research Scientist, Lanka Software Foundation >>>>>> Blog: http://srinathsview.blogspot.com twitter:@srinath_perera >>>>>> Site: http://people.apache.org/~hemapani/ >>>>>> Photos: http://www.flickr.com/photos/hemapani/ >>>>>> Phone: 0772360902 >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> *Anjana Fernando* >>>>> Senior Technical Lead >>>>> WSO2 Inc. | http://wso2.com >>>>> lean . enterprise . middleware >>>>> >>>> >>>> >>>> >>>> -- >>>> ============================ >>>> Director, Research, WSO2 Inc. >>>> Visiting Faculty, University of Moratuwa >>>> Member, Apache Software Foundation >>>> Research Scientist, Lanka Software Foundation >>>> Blog: http://srinathsview.blogspot.com twitter:@srinath_perera >>>> Site: http://people.apache.org/~hemapani/ >>>> Photos: http://www.flickr.com/photos/hemapani/ >>>> Phone: 0772360902 >>>> >>> >>> >>> >>> -- >>> *Anjana Fernando* >>> Senior Technical Lead >>> WSO2 Inc. | http://wso2.com >>> lean . enterprise . middleware >>> >> >> >> >> -- >> ============================ >> Director, Research, WSO2 Inc. >> Visiting Faculty, University of Moratuwa >> Member, Apache Software Foundation >> Research Scientist, Lanka Software Foundation >> Blog: http://srinathsview.blogspot.com twitter:@srinath_perera >> Site: http://people.apache.org/~hemapani/ >> Photos: http://www.flickr.com/photos/hemapani/ >> Phone: 0772360902 >> > > > > -- > *Supun Sethunga* > Software Engineer > WSO2, Inc. > lean | enterprise | middleware > Mobile : +94 716546324 > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- *Rajith Siriwardana* Software Engineer WSO2 Inc. ; http://wso2.com *lean. enterprise. middleware* ------------------------------------------------------ *http://people.apache.org/~siriwardana <http://people.apache.org/~siriwardana>*
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
