Hi, I checked the methods of apache spark. Some of them are related to wrangler script. I run spark as a maven project. But I am not able to run example code in java. Because it asking about the libraries. I found some simple example from stackoverflow . But I am not able to find the library of spark.apache. So can you please give me some idea about those libraries.
On Thu, Mar 12, 2015 at 11:35 AM, Supun Sethunga <sup...@wso2.com> wrote: > Hi Tharinda, > > Please find the comments inline. > > I think the major part of this project is create a back end of java code >> created using wrangler script. > > Yes. > > Wrangler scripts are mostly done by javascript. So I need to convert those >> into java. > > Yes. Further, java code should be Apache Spark Transformation. Please > refer my previous mail for details on Spark. > > I was not able to find some method. Other ways of doing this is used >> common methods like split,merge.. etc easily find on java. Rest of the >> methods are convert by manually. Is there any method to convert those >> scripts to java. > > You have to come up with a solution and an implementation for this. It is > the core part of the project :) > > Feel free to contact us if you need further clarifications. > > Regards, > Supun > > On Thu, Mar 12, 2015 at 1:04 PM, Tharinda Ehelepola <tharinda...@gmail.com > > wrote: > >> Hi, >> >> I think the major part of this project is create a back end of java code >> created using wrangler script. Wrangler scripts are mostly done by >> javascript. So I need to convert those into java. I was not able to find >> some method. Other ways of doing this is used common methods like >> split,merge.. etc easily find on java. Rest of the methods are convert by >> manually. Is there any method to convert those scripts to java. >> >> Thank you. >> Tharinda. >> >> On Wed, Mar 11, 2015 at 3:45 AM, Supun Sethunga <sup...@wso2.com> wrote: >> >>> Hi Tharinda, >>> >>> Ultimately, the java code created using the wrangler script (after >>> conversion), should be able to used as an Apache Spark [1] Transformations >>> [2]. (at the back-end) >>> >>> On a side note, please use the WSO2 dev mailing list for any project >>> related communication :) >>> >>> [1] https://spark.apache.org/ >>> [2] >>> http://spark.apache.org/docs/1.2.0/programming-guide.html#transformations >>> >>> Regards, >>> Supun >>> >>> On Wed, Mar 11, 2015 at 7:52 AM, Nirmal Fernando <nir...@wso2.com> >>> wrote: >>> >>>> Yes, that's exactly what we want ! >>>> >>>> On Wed, Mar 11, 2015 at 3:30 AM, Tharinda Ehelepola < >>>> tharinda...@gmail.com> wrote: >>>> >>>>> I checked the libraries on wrangler. I think that you mentioned can be >>>>> done using the libraries. What I think is get the libraries from wrangler >>>>> and create a back-end for the interface created using jaggery. Wrangler >>>>> script to java converter can be add to the back-end also. >>>>> >>>>> On Tue, Mar 10, 2015 at 4:10 AM, Nirmal Fernando <nir...@wso2.com> >>>>> wrote: >>>>> >>>>>> Remotely connecting to Wrangler (hosted service) is not a good >>>>>> option. What we expect is we take their library (it's Open source) and >>>>>> use >>>>>> the API to perform the transformations and cleaning. Could you please >>>>>> check >>>>>> the feasibility of that? >>>>>> >>>>>> On Tue, Mar 10, 2015 at 9:34 AM, Tharinda Ehelepola < >>>>>> tharinda...@gmail.com> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I drew what I understood the project to be. >>>>>>> >>>>>>> If one wants to do data cleaning and transformation he need to >>>>>>> connect to the wrangler directly. That is what I mean from the first two >>>>>>> blocks. If they have a customized UI you can do data cleaning and >>>>>>> transformation using ML also. They don't need to connect to the >>>>>>> wrangler. >>>>>>> Another thing i realized that is they don't need all the operations in >>>>>>> wrangler. They need some specific subset of operations only. What I mean >>>>>>> from the bottom block is the UI created using Jaggery interfacing >>>>>>> between >>>>>>> ML and the wrangler. >>>>>>> >>>>>>> On Tue, Mar 10, 2015 at 2:16 AM, Nirmal Fernando <nir...@wso2.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Tharindu, >>>>>>>> >>>>>>>> Thanks for the diagram. Do you mind explaining the diagram? >>>>>>>> >>>>>>>> On Tue, Mar 10, 2015 at 3:27 AM, Tharinda Ehelepola < >>>>>>>> tharinda...@gmail.com> wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> Here I attached a simple design of project development. Can you >>>>>>>>> give me some comments about this. >>>>>>>>> >>>>>>>>> Thank you. >>>>>>>>> >>>>>>>>> On Mon, Mar 9, 2015 at 2:16 AM, Nirmal Fernando <nir...@wso2.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Our ultimate goal is to provide data cleaning and transformation. >>>>>>>>>> Hence, Tharindu, you need to pick the things that is sufficient to >>>>>>>>>> perform >>>>>>>>>> those operations. >>>>>>>>>> >>>>>>>>>> What we prefer is a D3 based UI. >>>>>>>>>> >>>>>>>>>> On Sun, Mar 8, 2015 at 10:34 PM, Tharinda Ehelepola < >>>>>>>>>> tharinda...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> Wrangler already has a UI. I think that you excepting customized >>>>>>>>>>> UI for Wrangler using Jaggery. Which kind of conditions and options >>>>>>>>>>> do you >>>>>>>>>>> excepting from that UI. >>>>>>>>>>> >>>>>>>>>>> Thanks. >>>>>>>>>>> >>>>>>>>>>> On Sat, Mar 7, 2015 at 2:13 PM, Tharinda Ehelepola < >>>>>>>>>>> tharinda...@gmail.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> Thanks for your response. I have some idea about both Data >>>>>>>>>>>> Wrangler API and Jaggery. I'll try to bulid a UI. >>>>>>>>>>>> >>>>>>>>>>>> Thanks. >>>>>>>>>>>> >>>>>>>>>>>> On Sat, Mar 7, 2015 at 11:04 AM, Nirmal Fernando < >>>>>>>>>>>> nir...@wso2.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Sat, Mar 7, 2015 at 4:34 PM, Nirmal Fernando < >>>>>>>>>>>>> nir...@wso2.com> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Tharinda, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks for your interest on the proposal #17. Actually, it is >>>>>>>>>>>>>> not for WSO2 CEP but for WSO2 Machine Learner which is a new >>>>>>>>>>>>>> product that >>>>>>>>>>>>>> we are building these days. >>>>>>>>>>>>>> >>>>>>>>>>>>>> The main idea of this project is to integrate the Data >>>>>>>>>>>>>> Wrangler to WSO2 ML and get the data cleaning, transformation >>>>>>>>>>>>>> features into >>>>>>>>>>>>>> WSO2 ML. But as a start, you could play with Data Wrangler API >>>>>>>>>>>>>> and check >>>>>>>>>>>>>> how you could build a UI (preferably using HTML, Jaggery) that >>>>>>>>>>>>>> exposes Data >>>>>>>>>>>>>> Wrangler APIs. >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Sat, Mar 7, 2015 at 2:53 PM, Tharinda Ehelepola < >>>>>>>>>>>>>> tharinda...@gmail.com> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I am a student of university of Peradeniya, Faculty of >>>>>>>>>>>>>>> engineering, Department of computer engineering. I am >>>>>>>>>>>>>>> interested about your >>>>>>>>>>>>>>> projects. I have already seen the >>>>>>>>>>>>>>> "Proposal 17 : Data Wrangler extension for WSO2 Machine >>>>>>>>>>>>>>> Learner". Actually i am very interested about that project. I >>>>>>>>>>>>>>> have already >>>>>>>>>>>>>>> install the CEP. Now i really want to see some sample >>>>>>>>>>>>>>> extensions. I found >>>>>>>>>>>>>>> the link but Google drive not give the permission for me. Could >>>>>>>>>>>>>>> you please >>>>>>>>>>>>>>> give me some example extensions or approval for the Google >>>>>>>>>>>>>>> drive. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thank You. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>>>> Dev mailing list >>>>>>>>>>>>>>> Dev@wso2.org >>>>>>>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks & regards, >>>>>>>>>>>>>> Nirmal >>>>>>>>>>>>>> >>>>>>>>>>>>>> Senior Software Engineer- Platform Technologies Team, WSO2 >>>>>>>>>>>>>> Inc. >>>>>>>>>>>>>> Mobile: +94715779733 >>>>>>>>>>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks & regards, >>>>>>>>>>>>> Nirmal >>>>>>>>>>>>> >>>>>>>>>>>>> Senior Software Engineer- Platform Technologies Team, WSO2 Inc. >>>>>>>>>>>>> Mobile: +94715779733 >>>>>>>>>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> >>>>>>>>>> Thanks & regards, >>>>>>>>>> Nirmal >>>>>>>>>> >>>>>>>>>> Senior Software Engineer- Platform Technologies Team, WSO2 Inc. >>>>>>>>>> Mobile: +94715779733 >>>>>>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> Thanks & regards, >>>>>>>> Nirmal >>>>>>>> >>>>>>>> Senior Software Engineer- Platform Technologies Team, WSO2 Inc. >>>>>>>> Mobile: +94715779733 >>>>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Thanks & regards, >>>>>> Nirmal >>>>>> >>>>>> Senior Software Engineer- Platform Technologies Team, WSO2 Inc. >>>>>> Mobile: +94715779733 >>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>> >>>>>> >>>>>> >>>>> >>>> >>>> >>>> -- >>>> >>>> Thanks & regards, >>>> Nirmal >>>> >>>> Senior Software Engineer- Platform Technologies Team, WSO2 Inc. >>>> Mobile: +94715779733 >>>> Blog: http://nirmalfdo.blogspot.com/ >>>> >>>> >>>> >>> >>> >>> -- >>> *Supun Sethunga* >>> Software Engineer >>> WSO2, Inc. >>> lean | enterprise | middleware >>> Mobile : +94 716546324 >>> >> >> > > > -- > *Supun Sethunga* > Software Engineer > WSO2, Inc. > http://wso2.com/ > lean | enterprise | middleware > Mobile : +94 716546324 >
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
