Hi, The back-end service for the $subject has been implemented. For the time being, it supports three algorithms (Naive Bayes, Logistic Regression and Multilayer Perceptrons). The service accepts a JSON object having the input data file path, output path where the created the model should be saved, the algorithm to be used for modeling (one of the above three algos), and the hyper-parameters and their values.
Next goal would be to develop a carbon UI component, which allows a user to call the above service, in a form of a wizard. The wizard would provide a step-by-step guide to the user to create a ML model for any set of data. Appreciate any suggestions. Regards, Supun On Fri, Jul 4, 2014 at 7:39 AM, Srinath Perera <[email protected]> wrote: > Anjana and myself discussed. > > UDFs does not work. UDFs take an event as the input while mahout takes a > complete dataset. > > May be we can use UDAF (A = Aggregate), but has to explore more. > > > On Tue, Jul 1, 2014 at 10:03 AM, Anjana Fernando <[email protected]> wrote: > >> I see, sure, I was thinking of doing all the operations, including the >> training operations using an UDF. Will come and meet you. >> >> Cheers, >> Anjana. >> >> >> On Tue, Jul 1, 2014 at 9:56 AM, Srinath Perera <[email protected]> wrote: >> >>> No, we need to get the data, preprocess them using hive, and send all >>> the data (not 1-2 values, rather say 10 millions values) to training phase. >>> Lets chat f2f a bit. >>> >>> --Srinath >>> >>> >>> On Tue, Jul 1, 2014 at 6:24 AM, Anjana Fernando <[email protected]> wrote: >>> >>>> Hi, >>>> >>>> I was simply thinking, the UDF could directly mapped to some basic >>>> Mahout operation it implements, and the input/output should be given as >>>> parameters to the UDF, so probably, we can publish some input data >>>> beforehand to Cassandra etc.. and give the location of that data to the >>>> UDF, and the UDF will, as it is called, create the map/reduce jobs and >>>> execute. >>>> >>>> Cheers, >>>> Anjana. >>>> >>>> >>>> On Tue, Jul 1, 2014 at 9:18 AM, Srinath Perera <[email protected]> >>>> wrote: >>>> >>>>> +1 we wanted to explore that more. >>>>> >>>>> However, It is not a simple UDF as this is a stateful op where we feed >>>>> lot of data and start a separate map reduce process. Anjana, do you have >>>>> any thoughts on how it can be done? >>>>> >>>>> >>>>> On Tue, Jul 1, 2014 at 5:37 AM, Anjana Fernando <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I'm just wondering if we have any way to integrate this to Hive >>>>>> itself (UDF?), to get results of an ML algorithm run, to a result there. >>>>>> A >>>>>> similar scenario is possible in Shark/MLlib integration. >>>>>> >>>>>> Cheers, >>>>>> Anjana. >>>>>> >>>>>> >>>>>> On Mon, Jun 30, 2014 at 12:28 PM, Supun Sethunga <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> Im working on the $subject, and the objective is to apply Machine >>>>>>> Learning algorithms on the data stored by WSO2 BAM. Apache Mahout will >>>>>>> be >>>>>>> used as the ML tool, for this purpose. >>>>>>> >>>>>>> As per the discussion I had With Srinath, the procedure for $subject >>>>>>> would be: >>>>>>> >>>>>>> - Test a Machine Learning algorithm using Mahout libraries >>>>>>> within Java. >>>>>>> - Implement a RESTful service which provides the above >>>>>>> functionality. >>>>>>> - Since Mahout also uses Hadoop, the above service can send Map >>>>>>> Reduce Jobs to the Hadoop built inside the BAM. >>>>>>> - Deploy the service as a Carbon Component on WSO2 BAM. >>>>>>> >>>>>>> The first step is completed for now. >>>>>>> Any feedback is highly appreciated. >>>>>>> >>>>>>> Thanks, >>>>>>> Supun >>>>>>> >>>>>>> -- >>>>>>> *Supun Sethunga* >>>>>>> Software Engineer >>>>>>> WSO2, Inc. >>>>>>> lean | enterprise | middleware >>>>>>> Mobile : +94 716546324 >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Anjana Fernando* >>>>>> Senior Technical Lead >>>>>> WSO2 Inc. | http://wso2.com >>>>>> lean . enterprise . middleware >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> ============================ >>>>> Director, Research, WSO2 Inc. >>>>> Visiting Faculty, University of Moratuwa >>>>> Member, Apache Software Foundation >>>>> Research Scientist, Lanka Software Foundation >>>>> Blog: http://srinathsview.blogspot.com twitter:@srinath_perera >>>>> Site: http://people.apache.org/~hemapani/ >>>>> Photos: http://www.flickr.com/photos/hemapani/ >>>>> Phone: 0772360902 >>>>> >>>> >>>> >>>> >>>> -- >>>> *Anjana Fernando* >>>> Senior Technical Lead >>>> WSO2 Inc. | http://wso2.com >>>> lean . enterprise . middleware >>>> >>> >>> >>> >>> -- >>> ============================ >>> Director, Research, WSO2 Inc. >>> Visiting Faculty, University of Moratuwa >>> Member, Apache Software Foundation >>> Research Scientist, Lanka Software Foundation >>> Blog: http://srinathsview.blogspot.com twitter:@srinath_perera >>> Site: http://people.apache.org/~hemapani/ >>> Photos: http://www.flickr.com/photos/hemapani/ >>> Phone: 0772360902 >>> >> >> >> >> -- >> *Anjana Fernando* >> Senior Technical Lead >> WSO2 Inc. | http://wso2.com >> lean . enterprise . middleware >> > > > > -- > ============================ > Director, Research, WSO2 Inc. > Visiting Faculty, University of Moratuwa > Member, Apache Software Foundation > Research Scientist, Lanka Software Foundation > Blog: http://srinathsview.blogspot.com twitter:@srinath_perera > Site: http://people.apache.org/~hemapani/ > Photos: http://www.flickr.com/photos/hemapani/ > Phone: 0772360902 > -- *Supun Sethunga* Software Engineer WSO2, Inc. lean | enterprise | middleware Mobile : +94 716546324
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
