No, we need to get the data, preprocess them using hive, and send all the data (not 1-2 values, rather say 10 millions values) to training phase. Lets chat f2f a bit.
--Srinath On Tue, Jul 1, 2014 at 6:24 AM, Anjana Fernando <[email protected]> wrote: > Hi, > > I was simply thinking, the UDF could directly mapped to some basic Mahout > operation it implements, and the input/output should be given as parameters > to the UDF, so probably, we can publish some input data beforehand to > Cassandra etc.. and give the location of that data to the UDF, and the UDF > will, as it is called, create the map/reduce jobs and execute. > > Cheers, > Anjana. > > > On Tue, Jul 1, 2014 at 9:18 AM, Srinath Perera <[email protected]> wrote: > >> +1 we wanted to explore that more. >> >> However, It is not a simple UDF as this is a stateful op where we feed >> lot of data and start a separate map reduce process. Anjana, do you have >> any thoughts on how it can be done? >> >> >> On Tue, Jul 1, 2014 at 5:37 AM, Anjana Fernando <[email protected]> wrote: >> >>> Hi, >>> >>> I'm just wondering if we have any way to integrate this to Hive itself >>> (UDF?), to get results of an ML algorithm run, to a result there. A similar >>> scenario is possible in Shark/MLlib integration. >>> >>> Cheers, >>> Anjana. >>> >>> >>> On Mon, Jun 30, 2014 at 12:28 PM, Supun Sethunga <[email protected]> >>> wrote: >>> >>>> Hi, >>>> >>>> Im working on the $subject, and the objective is to apply Machine >>>> Learning algorithms on the data stored by WSO2 BAM. Apache Mahout will be >>>> used as the ML tool, for this purpose. >>>> >>>> As per the discussion I had With Srinath, the procedure for $subject >>>> would be: >>>> >>>> - Test a Machine Learning algorithm using Mahout libraries within >>>> Java. >>>> - Implement a RESTful service which provides the above >>>> functionality. >>>> - Since Mahout also uses Hadoop, the above service can send Map >>>> Reduce Jobs to the Hadoop built inside the BAM. >>>> - Deploy the service as a Carbon Component on WSO2 BAM. >>>> >>>> The first step is completed for now. >>>> Any feedback is highly appreciated. >>>> >>>> Thanks, >>>> Supun >>>> >>>> -- >>>> *Supun Sethunga* >>>> Software Engineer >>>> WSO2, Inc. >>>> lean | enterprise | middleware >>>> Mobile : +94 716546324 >>>> >>> >>> >>> >>> -- >>> *Anjana Fernando* >>> Senior Technical Lead >>> WSO2 Inc. | http://wso2.com >>> lean . enterprise . middleware >>> >> >> >> >> -- >> ============================ >> Director, Research, WSO2 Inc. >> Visiting Faculty, University of Moratuwa >> Member, Apache Software Foundation >> Research Scientist, Lanka Software Foundation >> Blog: http://srinathsview.blogspot.com twitter:@srinath_perera >> Site: http://people.apache.org/~hemapani/ >> Photos: http://www.flickr.com/photos/hemapani/ >> Phone: 0772360902 >> > > > > -- > *Anjana Fernando* > Senior Technical Lead > WSO2 Inc. | http://wso2.com > lean . enterprise . middleware > -- ============================ Director, Research, WSO2 Inc. Visiting Faculty, University of Moratuwa Member, Apache Software Foundation Research Scientist, Lanka Software Foundation Blog: http://srinathsview.blogspot.com twitter:@srinath_perera Site: http://people.apache.org/~hemapani/ Photos: http://www.flickr.com/photos/hemapani/ Phone: 0772360902
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
