Hi Nirmal, Please find the document at [1] and I have already created the PR.
[1] - https://docs.google.com/document/d/172MavBl2TuBNHVoyEPuRIPwmSW5lBljHFSNxrq4gclQ/edit?usp=sharing Danula On Sat, Sep 5, 2015 at 12:25 PM, Danula Eranjith <[email protected]> wrote: > Sure. Ill send the document and the PR > > Thanks, > Danula > > On Fri, Sep 4, 2015 at 9:01 AM, Nirmal Fernando <[email protected]> wrote: > >> Also please send a PR to our repos. >> >> On Fri, Sep 4, 2015 at 8:59 AM, Nirmal Fernando <[email protected]> wrote: >> >>> Thanks Danula. >>> >>> ML team like to do the integration of this, since there're few things we >>> need to clear up in ML side. >>> >>> Can you please come up with a clearly explained document on the project >>> work carried out during the summer? >>> >>> On Wed, Sep 2, 2015 at 12:41 AM, Danula Eranjith <[email protected]> >>> wrote: >>> >>>> Hi, >>>> >>>> I have made the changes you suggested at [1] and created a API to >>>> access the wrangler service at [2]. >>>> >>>> Also added a new step in the wizard at [3] with the wrangler interface. >>>> Please have a look. >>>> >>>> How can we create a sample of the dataset to be passed to wrangler? I >>>> couldn't find any current implementation capable of this. >>>> >>>> Another concern is if we are adding the feature selection step after >>>> the cleaning step, we need to reflect the changes done in step one at step >>>> two. But since we do not apply transformations to RDD initially, we need to >>>> come up an alternative approach. >>>> >>>> [1] - >>>> https://github.com/danula/carbon-ml/tree/master/components/ml/org.wso2.carbon.ml.wrangler/src/main/java/org/wso2/carbon/ml/wrangler >>>> >>>> [2] - >>>> https://github.com/danula/carbon-ml/blob/master/components/ml/org.wso2.carbon.ml.rest.api/src/main/java/org/wso2/carbon/ml/rest/api/WranglerApiV10.java >>>> <https://github.com/danula/carbon-ml/blob/master/components/ml/org.wso2.carbon.ml.rest.api/src/main/java/org/wso2/carbon/ml/rest/api/WranglerApiV10.java> >>>> >>>> [3] - >>>> https://github.com/danula/carbon-ml/blob/master/apps/ml/site/clean/clean.jag >>>> >>>> Thanks, >>>> Danula >>>> >>>> >>>> On Thu, Aug 27, 2015 at 9:41 AM, Danula Eranjith <[email protected]> >>>> wrote: >>>> >>>>> Basically script exported from Wrangler tool has list of operations. >>>>> Wrangler class parse that script and create WranglerOperation object >>>>> for each operation with its parameters. >>>>> Then when WranglerOperation.executeOperation() is invoked, it creates >>>>> the respective SparkOperation object and then applies operations to the >>>>> JavaRDD >>>>> >>>>> On Thu, Aug 27, 2015 at 9:35 AM, Nirmal Fernando <[email protected]> >>>>> wrote: >>>>> >>>>>> What does WranglerOperation class do? >>>>>> >>>>>> On Thu, Aug 27, 2015 at 9:24 AM, Danula Eranjith <[email protected] >>>>>> > wrote: >>>>>> >>>>>>> Currently Wrangler Operation is the class that holds details related >>>>>>> to wrangler and SparkOperation contains the relevant Spark >>>>>>> transformation. >>>>>>> >>>>>>> If we are changing SparkOperation as WranglerOperation, we need to >>>>>>> rename the current WranglerOperation into something else. >>>>>>> >>>>>>> On Thu, Aug 27, 2015 at 9:18 AM, Nirmal Fernando <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> /s/SparkOpration/SparkOperation >>>>>>>> >>>>>>>> May be as Supun said, I too think we should call them as >>>>>>>> 'WranglerOperation'. >>>>>>>> >>>>>>>> On Thu, Aug 27, 2015 at 7:02 AM, Nirmal Fernando <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Also, avoid static methods in transformations. >>>>>>>>> >>>>>>>>> On Thu, Aug 27, 2015 at 2:48 AM, Supun Sethunga <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi Danula, >>>>>>>>>> >>>>>>>>>> Few comments: >>>>>>>>>> >>>>>>>>>> - You might have to register the component in the OSGI >>>>>>>>>> environment, to be able to call the services from a another >>>>>>>>>> component. >>>>>>>>>> Refer [1] on how to do this. >>>>>>>>>> - Better to introduce an interface for WranglerOperation >>>>>>>>>> class. >>>>>>>>>> - Add class level/ method level comments. >>>>>>>>>> - Use logger in-place of System.out.println >>>>>>>>>> >>>>>>>>>> [1] >>>>>>>>>> https://github.com/danula/carbon-ml/blob/master/components/ml/org.wso2.carbon.ml.database/src/main/java/org/wso2/carbon/ml/database/internal/ds/MLDatabaseServiceDS.java >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Supun >>>>>>>>>> >>>>>>>>>> On Wed, Aug 26, 2015 at 1:32 PM, Danula Eranjith < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> I added the component at [1] >>>>>>>>>>> <https://github.com/danula/carbon-ml/tree/master/components/ml/org.wso2.carbon.ml.wrangler> >>>>>>>>>>> Please have a look. >>>>>>>>>>> >>>>>>>>>>> [1] - >>>>>>>>>>> https://github.com/danula/carbon-ml/tree/master/components/ml/org.wso2.carbon.ml.wrangler >>>>>>>>>>> >>>>>>>>>>> Danula >>>>>>>>>>> >>>>>>>>>>> On Tue, Aug 25, 2015 at 8:35 PM, Danula Eranjith < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Thanks Supun >>>>>>>>>>>> >>>>>>>>>>>> On Tue, Aug 25, 2015 at 7:25 PM, Supun Sethunga < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> You can integrate it to [1], by adding a new component >>>>>>>>>>>>> "org.wso2.carbon.ml.wrangler". Each component is a carbon >>>>>>>>>>>>> component. >>>>>>>>>>>>> >>>>>>>>>>>>> Please follow the naming conventions used in the other >>>>>>>>>>>>> components, for package names and etc.. >>>>>>>>>>>>> >>>>>>>>>>>>> [1] >>>>>>>>>>>>> https://github.com/wso2/carbon-ml/tree/master/components/ml >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Supun >>>>>>>>>>>>> >>>>>>>>>>>>> On Tue, Aug 25, 2015 at 7:33 AM, Danula Eranjith < >>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Can you suggest where I should be ideally integrating these >>>>>>>>>>>>>> files[1] >>>>>>>>>>>>>> <https://github.com/danula/wso2-ml-wrangler-integration/tree/master/src> >>>>>>>>>>>>>> in ML. >>>>>>>>>>>>>> >>>>>>>>>>>>>> [1] - >>>>>>>>>>>>>> https://github.com/danula/wso2-ml-wrangler-integration/tree/master/src >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Danula >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> *Supun Sethunga* >>>>>>>>>>>>> Software Engineer >>>>>>>>>>>>> WSO2, Inc. >>>>>>>>>>>>> http://wso2.com/ >>>>>>>>>>>>> lean | enterprise | middleware >>>>>>>>>>>>> Mobile : +94 716546324 >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> *Supun Sethunga* >>>>>>>>>> Software Engineer >>>>>>>>>> WSO2, Inc. >>>>>>>>>> http://wso2.com/ >>>>>>>>>> lean | enterprise | middleware >>>>>>>>>> Mobile : +94 716546324 >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> >>>>>>>>> Thanks & regards, >>>>>>>>> Nirmal >>>>>>>>> >>>>>>>>> Team Lead - WSO2 Machine Learner >>>>>>>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>>>>>>> Mobile: +94715779733 >>>>>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> Thanks & regards, >>>>>>>> Nirmal >>>>>>>> >>>>>>>> Team Lead - WSO2 Machine Learner >>>>>>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>>>>>> Mobile: +94715779733 >>>>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Thanks & regards, >>>>>> Nirmal >>>>>> >>>>>> Team Lead - WSO2 Machine Learner >>>>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>>>> Mobile: +94715779733 >>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >>> >>> -- >>> >>> Thanks & regards, >>> Nirmal >>> >>> Team Lead - WSO2 Machine Learner >>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>> Mobile: +94715779733 >>> Blog: http://nirmalfdo.blogspot.com/ >>> >>> >>> >> >> >> -- >> >> Thanks & regards, >> Nirmal >> >> Team Lead - WSO2 Machine Learner >> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >> Mobile: +94715779733 >> Blog: http://nirmalfdo.blogspot.com/ >> >> >> >
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
