Also please send a PR to our repos. On Fri, Sep 4, 2015 at 8:59 AM, Nirmal Fernando <nir...@wso2.com> wrote:
> Thanks Danula. > > ML team like to do the integration of this, since there're few things we > need to clear up in ML side. > > Can you please come up with a clearly explained document on the project > work carried out during the summer? > > On Wed, Sep 2, 2015 at 12:41 AM, Danula Eranjith <hmdanu...@gmail.com> > wrote: > >> Hi, >> >> I have made the changes you suggested at [1] and created a API to access >> the wrangler service at [2]. >> >> Also added a new step in the wizard at [3] with the wrangler interface. >> Please have a look. >> >> How can we create a sample of the dataset to be passed to wrangler? I >> couldn't find any current implementation capable of this. >> >> Another concern is if we are adding the feature selection step after the >> cleaning step, we need to reflect the changes done in step one at step two. >> But since we do not apply transformations to RDD initially, we need to come >> up an alternative approach. >> >> [1] - >> https://github.com/danula/carbon-ml/tree/master/components/ml/org.wso2.carbon.ml.wrangler/src/main/java/org/wso2/carbon/ml/wrangler >> >> [2] - >> https://github.com/danula/carbon-ml/blob/master/components/ml/org.wso2.carbon.ml.rest.api/src/main/java/org/wso2/carbon/ml/rest/api/WranglerApiV10.java >> <https://github.com/danula/carbon-ml/blob/master/components/ml/org.wso2.carbon.ml.rest.api/src/main/java/org/wso2/carbon/ml/rest/api/WranglerApiV10.java> >> >> [3] - >> https://github.com/danula/carbon-ml/blob/master/apps/ml/site/clean/clean.jag >> >> Thanks, >> Danula >> >> >> On Thu, Aug 27, 2015 at 9:41 AM, Danula Eranjith <hmdanu...@gmail.com> >> wrote: >> >>> Basically script exported from Wrangler tool has list of operations. >>> Wrangler class parse that script and create WranglerOperation object for >>> each operation with its parameters. >>> Then when WranglerOperation.executeOperation() is invoked, it creates >>> the respective SparkOperation object and then applies operations to the >>> JavaRDD >>> >>> On Thu, Aug 27, 2015 at 9:35 AM, Nirmal Fernando <nir...@wso2.com> >>> wrote: >>> >>>> What does WranglerOperation class do? >>>> >>>> On Thu, Aug 27, 2015 at 9:24 AM, Danula Eranjith <hmdanu...@gmail.com> >>>> wrote: >>>> >>>>> Currently Wrangler Operation is the class that holds details related >>>>> to wrangler and SparkOperation contains the relevant Spark transformation. >>>>> >>>>> If we are changing SparkOperation as WranglerOperation, we need to >>>>> rename the current WranglerOperation into something else. >>>>> >>>>> On Thu, Aug 27, 2015 at 9:18 AM, Nirmal Fernando <nir...@wso2.com> >>>>> wrote: >>>>> >>>>>> /s/SparkOpration/SparkOperation >>>>>> >>>>>> May be as Supun said, I too think we should call them as >>>>>> 'WranglerOperation'. >>>>>> >>>>>> On Thu, Aug 27, 2015 at 7:02 AM, Nirmal Fernando <nir...@wso2.com> >>>>>> wrote: >>>>>> >>>>>>> Also, avoid static methods in transformations. >>>>>>> >>>>>>> On Thu, Aug 27, 2015 at 2:48 AM, Supun Sethunga <sup...@wso2.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Danula, >>>>>>>> >>>>>>>> Few comments: >>>>>>>> >>>>>>>> - You might have to register the component in the OSGI >>>>>>>> environment, to be able to call the services from a another >>>>>>>> component. >>>>>>>> Refer [1] on how to do this. >>>>>>>> - Better to introduce an interface for WranglerOperation class. >>>>>>>> - Add class level/ method level comments. >>>>>>>> - Use logger in-place of System.out.println >>>>>>>> >>>>>>>> [1] >>>>>>>> https://github.com/danula/carbon-ml/blob/master/components/ml/org.wso2.carbon.ml.database/src/main/java/org/wso2/carbon/ml/database/internal/ds/MLDatabaseServiceDS.java >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Supun >>>>>>>> >>>>>>>> On Wed, Aug 26, 2015 at 1:32 PM, Danula Eranjith < >>>>>>>> hmdanu...@gmail.com> wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I added the component at [1] >>>>>>>>> <https://github.com/danula/carbon-ml/tree/master/components/ml/org.wso2.carbon.ml.wrangler> >>>>>>>>> Please have a look. >>>>>>>>> >>>>>>>>> [1] - >>>>>>>>> https://github.com/danula/carbon-ml/tree/master/components/ml/org.wso2.carbon.ml.wrangler >>>>>>>>> >>>>>>>>> Danula >>>>>>>>> >>>>>>>>> On Tue, Aug 25, 2015 at 8:35 PM, Danula Eranjith < >>>>>>>>> hmdanu...@gmail.com> wrote: >>>>>>>>> >>>>>>>>>> Thanks Supun >>>>>>>>>> >>>>>>>>>> On Tue, Aug 25, 2015 at 7:25 PM, Supun Sethunga <sup...@wso2.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> You can integrate it to [1], by adding a new component >>>>>>>>>>> "org.wso2.carbon.ml.wrangler". Each component is a carbon component. >>>>>>>>>>> >>>>>>>>>>> Please follow the naming conventions used in the other >>>>>>>>>>> components, for package names and etc.. >>>>>>>>>>> >>>>>>>>>>> [1] https://github.com/wso2/carbon-ml/tree/master/components/ml >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Supun >>>>>>>>>>> >>>>>>>>>>> On Tue, Aug 25, 2015 at 7:33 AM, Danula Eranjith < >>>>>>>>>>> hmdanu...@gmail.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi all, >>>>>>>>>>>> >>>>>>>>>>>> Can you suggest where I should be ideally integrating these >>>>>>>>>>>> files[1] >>>>>>>>>>>> <https://github.com/danula/wso2-ml-wrangler-integration/tree/master/src> >>>>>>>>>>>> in ML. >>>>>>>>>>>> >>>>>>>>>>>> [1] - >>>>>>>>>>>> https://github.com/danula/wso2-ml-wrangler-integration/tree/master/src >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Danula >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> *Supun Sethunga* >>>>>>>>>>> Software Engineer >>>>>>>>>>> WSO2, Inc. >>>>>>>>>>> http://wso2.com/ >>>>>>>>>>> lean | enterprise | middleware >>>>>>>>>>> Mobile : +94 716546324 >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> *Supun Sethunga* >>>>>>>> Software Engineer >>>>>>>> WSO2, Inc. >>>>>>>> http://wso2.com/ >>>>>>>> lean | enterprise | middleware >>>>>>>> Mobile : +94 716546324 >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> Thanks & regards, >>>>>>> Nirmal >>>>>>> >>>>>>> Team Lead - WSO2 Machine Learner >>>>>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>>>>> Mobile: +94715779733 >>>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Thanks & regards, >>>>>> Nirmal >>>>>> >>>>>> Team Lead - WSO2 Machine Learner >>>>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>>>> Mobile: +94715779733 >>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>> >>>>>> >>>>>> >>>>> >>>> >>>> >>>> -- >>>> >>>> Thanks & regards, >>>> Nirmal >>>> >>>> Team Lead - WSO2 Machine Learner >>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>> Mobile: +94715779733 >>>> Blog: http://nirmalfdo.blogspot.com/ >>>> >>>> >>>> >>> >> > > > -- > > Thanks & regards, > Nirmal > > Team Lead - WSO2 Machine Learner > Associate Technical Lead - Data Technologies Team, WSO2 Inc. > Mobile: +94715779733 > Blog: http://nirmalfdo.blogspot.com/ > > > -- Thanks & regards, Nirmal Team Lead - WSO2 Machine Learner Associate Technical Lead - Data Technologies Team, WSO2 Inc. Mobile: +94715779733 Blog: http://nirmalfdo.blogspot.com/
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
