Sure. Ill send the document and the PR Thanks, Danula
On Fri, Sep 4, 2015 at 9:01 AM, Nirmal Fernando <[email protected]> wrote: > Also please send a PR to our repos. > > On Fri, Sep 4, 2015 at 8:59 AM, Nirmal Fernando <[email protected]> wrote: > >> Thanks Danula. >> >> ML team like to do the integration of this, since there're few things we >> need to clear up in ML side. >> >> Can you please come up with a clearly explained document on the project >> work carried out during the summer? >> >> On Wed, Sep 2, 2015 at 12:41 AM, Danula Eranjith <[email protected]> >> wrote: >> >>> Hi, >>> >>> I have made the changes you suggested at [1] and created a API to access >>> the wrangler service at [2]. >>> >>> Also added a new step in the wizard at [3] with the wrangler interface. >>> Please have a look. >>> >>> How can we create a sample of the dataset to be passed to wrangler? I >>> couldn't find any current implementation capable of this. >>> >>> Another concern is if we are adding the feature selection step after the >>> cleaning step, we need to reflect the changes done in step one at step two. >>> But since we do not apply transformations to RDD initially, we need to come >>> up an alternative approach. >>> >>> [1] - >>> https://github.com/danula/carbon-ml/tree/master/components/ml/org.wso2.carbon.ml.wrangler/src/main/java/org/wso2/carbon/ml/wrangler >>> >>> [2] - >>> https://github.com/danula/carbon-ml/blob/master/components/ml/org.wso2.carbon.ml.rest.api/src/main/java/org/wso2/carbon/ml/rest/api/WranglerApiV10.java >>> <https://github.com/danula/carbon-ml/blob/master/components/ml/org.wso2.carbon.ml.rest.api/src/main/java/org/wso2/carbon/ml/rest/api/WranglerApiV10.java> >>> >>> [3] - >>> https://github.com/danula/carbon-ml/blob/master/apps/ml/site/clean/clean.jag >>> >>> Thanks, >>> Danula >>> >>> >>> On Thu, Aug 27, 2015 at 9:41 AM, Danula Eranjith <[email protected]> >>> wrote: >>> >>>> Basically script exported from Wrangler tool has list of operations. >>>> Wrangler class parse that script and create WranglerOperation object >>>> for each operation with its parameters. >>>> Then when WranglerOperation.executeOperation() is invoked, it creates >>>> the respective SparkOperation object and then applies operations to the >>>> JavaRDD >>>> >>>> On Thu, Aug 27, 2015 at 9:35 AM, Nirmal Fernando <[email protected]> >>>> wrote: >>>> >>>>> What does WranglerOperation class do? >>>>> >>>>> On Thu, Aug 27, 2015 at 9:24 AM, Danula Eranjith <[email protected]> >>>>> wrote: >>>>> >>>>>> Currently Wrangler Operation is the class that holds details related >>>>>> to wrangler and SparkOperation contains the relevant Spark >>>>>> transformation. >>>>>> >>>>>> If we are changing SparkOperation as WranglerOperation, we need to >>>>>> rename the current WranglerOperation into something else. >>>>>> >>>>>> On Thu, Aug 27, 2015 at 9:18 AM, Nirmal Fernando <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> /s/SparkOpration/SparkOperation >>>>>>> >>>>>>> May be as Supun said, I too think we should call them as >>>>>>> 'WranglerOperation'. >>>>>>> >>>>>>> On Thu, Aug 27, 2015 at 7:02 AM, Nirmal Fernando <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Also, avoid static methods in transformations. >>>>>>>> >>>>>>>> On Thu, Aug 27, 2015 at 2:48 AM, Supun Sethunga <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi Danula, >>>>>>>>> >>>>>>>>> Few comments: >>>>>>>>> >>>>>>>>> - You might have to register the component in the OSGI >>>>>>>>> environment, to be able to call the services from a another >>>>>>>>> component. >>>>>>>>> Refer [1] on how to do this. >>>>>>>>> - Better to introduce an interface for WranglerOperation class. >>>>>>>>> - Add class level/ method level comments. >>>>>>>>> - Use logger in-place of System.out.println >>>>>>>>> >>>>>>>>> [1] >>>>>>>>> https://github.com/danula/carbon-ml/blob/master/components/ml/org.wso2.carbon.ml.database/src/main/java/org/wso2/carbon/ml/database/internal/ds/MLDatabaseServiceDS.java >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Supun >>>>>>>>> >>>>>>>>> On Wed, Aug 26, 2015 at 1:32 PM, Danula Eranjith < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> I added the component at [1] >>>>>>>>>> <https://github.com/danula/carbon-ml/tree/master/components/ml/org.wso2.carbon.ml.wrangler> >>>>>>>>>> Please have a look. >>>>>>>>>> >>>>>>>>>> [1] - >>>>>>>>>> https://github.com/danula/carbon-ml/tree/master/components/ml/org.wso2.carbon.ml.wrangler >>>>>>>>>> >>>>>>>>>> Danula >>>>>>>>>> >>>>>>>>>> On Tue, Aug 25, 2015 at 8:35 PM, Danula Eranjith < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Thanks Supun >>>>>>>>>>> >>>>>>>>>>> On Tue, Aug 25, 2015 at 7:25 PM, Supun Sethunga <[email protected] >>>>>>>>>>> > wrote: >>>>>>>>>>> >>>>>>>>>>>> You can integrate it to [1], by adding a new component >>>>>>>>>>>> "org.wso2.carbon.ml.wrangler". Each component is a carbon >>>>>>>>>>>> component. >>>>>>>>>>>> >>>>>>>>>>>> Please follow the naming conventions used in the other >>>>>>>>>>>> components, for package names and etc.. >>>>>>>>>>>> >>>>>>>>>>>> [1] https://github.com/wso2/carbon-ml/tree/master/components/ml >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Supun >>>>>>>>>>>> >>>>>>>>>>>> On Tue, Aug 25, 2015 at 7:33 AM, Danula Eranjith < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi all, >>>>>>>>>>>>> >>>>>>>>>>>>> Can you suggest where I should be ideally integrating these >>>>>>>>>>>>> files[1] >>>>>>>>>>>>> <https://github.com/danula/wso2-ml-wrangler-integration/tree/master/src> >>>>>>>>>>>>> in ML. >>>>>>>>>>>>> >>>>>>>>>>>>> [1] - >>>>>>>>>>>>> https://github.com/danula/wso2-ml-wrangler-integration/tree/master/src >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Danula >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> *Supun Sethunga* >>>>>>>>>>>> Software Engineer >>>>>>>>>>>> WSO2, Inc. >>>>>>>>>>>> http://wso2.com/ >>>>>>>>>>>> lean | enterprise | middleware >>>>>>>>>>>> Mobile : +94 716546324 >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> *Supun Sethunga* >>>>>>>>> Software Engineer >>>>>>>>> WSO2, Inc. >>>>>>>>> http://wso2.com/ >>>>>>>>> lean | enterprise | middleware >>>>>>>>> Mobile : +94 716546324 >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> Thanks & regards, >>>>>>>> Nirmal >>>>>>>> >>>>>>>> Team Lead - WSO2 Machine Learner >>>>>>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>>>>>> Mobile: +94715779733 >>>>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> Thanks & regards, >>>>>>> Nirmal >>>>>>> >>>>>>> Team Lead - WSO2 Machine Learner >>>>>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>>>>> Mobile: +94715779733 >>>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Thanks & regards, >>>>> Nirmal >>>>> >>>>> Team Lead - WSO2 Machine Learner >>>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>>> Mobile: +94715779733 >>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>> >>>>> >>>>> >>>> >>> >> >> >> -- >> >> Thanks & regards, >> Nirmal >> >> Team Lead - WSO2 Machine Learner >> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >> Mobile: +94715779733 >> Blog: http://nirmalfdo.blogspot.com/ >> >> >> > > > -- > > Thanks & regards, > Nirmal > > Team Lead - WSO2 Machine Learner > Associate Technical Lead - Data Technologies Team, WSO2 Inc. > Mobile: +94715779733 > Blog: http://nirmalfdo.blogspot.com/ > > >
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
