I see, sure, I was thinking of doing all the operations, including the training operations using an UDF. Will come and meet you.
Cheers, Anjana. On Tue, Jul 1, 2014 at 9:56 AM, Srinath Perera <[email protected]> wrote: > No, we need to get the data, preprocess them using hive, and send all the > data (not 1-2 values, rather say 10 millions values) to training phase. > Lets chat f2f a bit. > > --Srinath > > > On Tue, Jul 1, 2014 at 6:24 AM, Anjana Fernando <[email protected]> wrote: > >> Hi, >> >> I was simply thinking, the UDF could directly mapped to some basic Mahout >> operation it implements, and the input/output should be given as parameters >> to the UDF, so probably, we can publish some input data beforehand to >> Cassandra etc.. and give the location of that data to the UDF, and the UDF >> will, as it is called, create the map/reduce jobs and execute. >> >> Cheers, >> Anjana. >> >> >> On Tue, Jul 1, 2014 at 9:18 AM, Srinath Perera <[email protected]> wrote: >> >>> +1 we wanted to explore that more. >>> >>> However, It is not a simple UDF as this is a stateful op where we feed >>> lot of data and start a separate map reduce process. Anjana, do you have >>> any thoughts on how it can be done? >>> >>> >>> On Tue, Jul 1, 2014 at 5:37 AM, Anjana Fernando <[email protected]> wrote: >>> >>>> Hi, >>>> >>>> I'm just wondering if we have any way to integrate this to Hive itself >>>> (UDF?), to get results of an ML algorithm run, to a result there. A similar >>>> scenario is possible in Shark/MLlib integration. >>>> >>>> Cheers, >>>> Anjana. >>>> >>>> >>>> On Mon, Jun 30, 2014 at 12:28 PM, Supun Sethunga <[email protected]> >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> Im working on the $subject, and the objective is to apply Machine >>>>> Learning algorithms on the data stored by WSO2 BAM. Apache Mahout will be >>>>> used as the ML tool, for this purpose. >>>>> >>>>> As per the discussion I had With Srinath, the procedure for $subject >>>>> would be: >>>>> >>>>> - Test a Machine Learning algorithm using Mahout libraries within >>>>> Java. >>>>> - Implement a RESTful service which provides the above >>>>> functionality. >>>>> - Since Mahout also uses Hadoop, the above service can send Map >>>>> Reduce Jobs to the Hadoop built inside the BAM. >>>>> - Deploy the service as a Carbon Component on WSO2 BAM. >>>>> >>>>> The first step is completed for now. >>>>> Any feedback is highly appreciated. >>>>> >>>>> Thanks, >>>>> Supun >>>>> >>>>> -- >>>>> *Supun Sethunga* >>>>> Software Engineer >>>>> WSO2, Inc. >>>>> lean | enterprise | middleware >>>>> Mobile : +94 716546324 >>>>> >>>> >>>> >>>> >>>> -- >>>> *Anjana Fernando* >>>> Senior Technical Lead >>>> WSO2 Inc. | http://wso2.com >>>> lean . enterprise . middleware >>>> >>> >>> >>> >>> -- >>> ============================ >>> Director, Research, WSO2 Inc. >>> Visiting Faculty, University of Moratuwa >>> Member, Apache Software Foundation >>> Research Scientist, Lanka Software Foundation >>> Blog: http://srinathsview.blogspot.com twitter:@srinath_perera >>> Site: http://people.apache.org/~hemapani/ >>> Photos: http://www.flickr.com/photos/hemapani/ >>> Phone: 0772360902 >>> >> >> >> >> -- >> *Anjana Fernando* >> Senior Technical Lead >> WSO2 Inc. | http://wso2.com >> lean . enterprise . middleware >> > > > > -- > ============================ > Director, Research, WSO2 Inc. > Visiting Faculty, University of Moratuwa > Member, Apache Software Foundation > Research Scientist, Lanka Software Foundation > Blog: http://srinathsview.blogspot.com twitter:@srinath_perera > Site: http://people.apache.org/~hemapani/ > Photos: http://www.flickr.com/photos/hemapani/ > Phone: 0772360902 > -- *Anjana Fernando* Senior Technical Lead WSO2 Inc. | http://wso2.com lean . enterprise . middleware
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
