Hi, You can find all the necessary libraries in the Maven Repository [1]. Please use the latest version of spark (v1.2.1).
[1] http://mvnrepository.com/artifact/org.apache.spark Regards, Supun On Fri, Mar 13, 2015 at 6:27 AM, Tharinda Ehelepola <[email protected]> wrote: > Hi, > > I checked the methods of apache spark. Some of them are related to > wrangler script. I run spark as a maven project. But I am not able to run > example code in java. Because it asking about the libraries. I found some > simple example from stackoverflow . But I am not able to find the library > of spark.apache. So can you please give me some idea about those > libraries. > > On Thu, Mar 12, 2015 at 11:35 AM, Supun Sethunga <[email protected]> wrote: > >> Hi Tharinda, >> >> Please find the comments inline. >> >> I think the major part of this project is create a back end of java code >>> created using wrangler script. >> >> Yes. >> >> Wrangler scripts are mostly done by javascript. So I need to convert >>> those into java. >> >> Yes. Further, java code should be Apache Spark Transformation. Please >> refer my previous mail for details on Spark. >> >> I was not able to find some method. Other ways of doing this is used >>> common methods like split,merge.. etc easily find on java. Rest of the >>> methods are convert by manually. Is there any method to convert those >>> scripts to java. >> >> You have to come up with a solution and an implementation for this. It is >> the core part of the project :) >> >> Feel free to contact us if you need further clarifications. >> >> Regards, >> Supun >> >> On Thu, Mar 12, 2015 at 1:04 PM, Tharinda Ehelepola < >> [email protected]> wrote: >> >>> Hi, >>> >>> I think the major part of this project is create a back end of java code >>> created using wrangler script. Wrangler scripts are mostly done by >>> javascript. So I need to convert those into java. I was not able to find >>> some method. Other ways of doing this is used common methods like >>> split,merge.. etc easily find on java. Rest of the methods are convert by >>> manually. Is there any method to convert those scripts to java. >>> >>> Thank you. >>> Tharinda. >>> >>> On Wed, Mar 11, 2015 at 3:45 AM, Supun Sethunga <[email protected]> wrote: >>> >>>> Hi Tharinda, >>>> >>>> Ultimately, the java code created using the wrangler script (after >>>> conversion), should be able to used as an Apache Spark [1] Transformations >>>> [2]. (at the back-end) >>>> >>>> On a side note, please use the WSO2 dev mailing list for any project >>>> related communication :) >>>> >>>> [1] https://spark.apache.org/ >>>> [2] >>>> http://spark.apache.org/docs/1.2.0/programming-guide.html#transformations >>>> >>>> Regards, >>>> Supun >>>> >>>> On Wed, Mar 11, 2015 at 7:52 AM, Nirmal Fernando <[email protected]> >>>> wrote: >>>> >>>>> Yes, that's exactly what we want ! >>>>> >>>>> On Wed, Mar 11, 2015 at 3:30 AM, Tharinda Ehelepola < >>>>> [email protected]> wrote: >>>>> >>>>>> I checked the libraries on wrangler. I think that you mentioned can >>>>>> be done using the libraries. What I think is get the libraries from >>>>>> wrangler and create a back-end for the interface created using jaggery. >>>>>> Wrangler script to java converter can be add to the back-end also. >>>>>> >>>>>> On Tue, Mar 10, 2015 at 4:10 AM, Nirmal Fernando <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Remotely connecting to Wrangler (hosted service) is not a good >>>>>>> option. What we expect is we take their library (it's Open source) and >>>>>>> use >>>>>>> the API to perform the transformations and cleaning. Could you please >>>>>>> check >>>>>>> the feasibility of that? >>>>>>> >>>>>>> On Tue, Mar 10, 2015 at 9:34 AM, Tharinda Ehelepola < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I drew what I understood the project to be. >>>>>>>> >>>>>>>> If one wants to do data cleaning and transformation he need to >>>>>>>> connect to the wrangler directly. That is what I mean from the first >>>>>>>> two >>>>>>>> blocks. If they have a customized UI you can do data cleaning and >>>>>>>> transformation using ML also. They don't need to connect to the >>>>>>>> wrangler. >>>>>>>> Another thing i realized that is they don't need all the operations in >>>>>>>> wrangler. They need some specific subset of operations only. What I >>>>>>>> mean >>>>>>>> from the bottom block is the UI created using Jaggery interfacing >>>>>>>> between >>>>>>>> ML and the wrangler. >>>>>>>> >>>>>>>> On Tue, Mar 10, 2015 at 2:16 AM, Nirmal Fernando <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi Tharindu, >>>>>>>>> >>>>>>>>> Thanks for the diagram. Do you mind explaining the diagram? >>>>>>>>> >>>>>>>>> On Tue, Mar 10, 2015 at 3:27 AM, Tharinda Ehelepola < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> Here I attached a simple design of project development. Can you >>>>>>>>>> give me some comments about this. >>>>>>>>>> >>>>>>>>>> Thank you. >>>>>>>>>> >>>>>>>>>> On Mon, Mar 9, 2015 at 2:16 AM, Nirmal Fernando <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Our ultimate goal is to provide data cleaning and >>>>>>>>>>> transformation. Hence, Tharindu, you need to pick the things that is >>>>>>>>>>> sufficient to perform those operations. >>>>>>>>>>> >>>>>>>>>>> What we prefer is a D3 based UI. >>>>>>>>>>> >>>>>>>>>>> On Sun, Mar 8, 2015 at 10:34 PM, Tharinda Ehelepola < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> Wrangler already has a UI. I think that you excepting customized >>>>>>>>>>>> UI for Wrangler using Jaggery. Which kind of conditions and >>>>>>>>>>>> options do you >>>>>>>>>>>> excepting from that UI. >>>>>>>>>>>> >>>>>>>>>>>> Thanks. >>>>>>>>>>>> >>>>>>>>>>>> On Sat, Mar 7, 2015 at 2:13 PM, Tharinda Ehelepola < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks for your response. I have some idea about both Data >>>>>>>>>>>>> Wrangler API and Jaggery. I'll try to bulid a UI. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks. >>>>>>>>>>>>> >>>>>>>>>>>>> On Sat, Mar 7, 2015 at 11:04 AM, Nirmal Fernando < >>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Sat, Mar 7, 2015 at 4:34 PM, Nirmal Fernando < >>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Tharinda, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks for your interest on the proposal #17. Actually, it >>>>>>>>>>>>>>> is not for WSO2 CEP but for WSO2 Machine Learner which is a new >>>>>>>>>>>>>>> product >>>>>>>>>>>>>>> that we are building these days. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The main idea of this project is to integrate the Data >>>>>>>>>>>>>>> Wrangler to WSO2 ML and get the data cleaning, transformation >>>>>>>>>>>>>>> features into >>>>>>>>>>>>>>> WSO2 ML. But as a start, you could play with Data Wrangler API >>>>>>>>>>>>>>> and check >>>>>>>>>>>>>>> how you could build a UI (preferably using HTML, Jaggery) that >>>>>>>>>>>>>>> exposes Data >>>>>>>>>>>>>>> Wrangler APIs. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Sat, Mar 7, 2015 at 2:53 PM, Tharinda Ehelepola < >>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I am a student of university of Peradeniya, Faculty of >>>>>>>>>>>>>>>> engineering, Department of computer engineering. I am >>>>>>>>>>>>>>>> interested about your >>>>>>>>>>>>>>>> projects. I have already seen the >>>>>>>>>>>>>>>> "Proposal 17 : Data Wrangler extension for WSO2 Machine >>>>>>>>>>>>>>>> Learner". Actually i am very interested about that project. I >>>>>>>>>>>>>>>> have already >>>>>>>>>>>>>>>> install the CEP. Now i really want to see some sample >>>>>>>>>>>>>>>> extensions. I found >>>>>>>>>>>>>>>> the link but Google drive not give the permission for me. >>>>>>>>>>>>>>>> Could you please >>>>>>>>>>>>>>>> give me some example extensions or approval for the Google >>>>>>>>>>>>>>>> drive. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thank You. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>>>>> Dev mailing list >>>>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks & regards, >>>>>>>>>>>>>>> Nirmal >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Senior Software Engineer- Platform Technologies Team, WSO2 >>>>>>>>>>>>>>> Inc. >>>>>>>>>>>>>>> Mobile: +94715779733 >>>>>>>>>>>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks & regards, >>>>>>>>>>>>>> Nirmal >>>>>>>>>>>>>> >>>>>>>>>>>>>> Senior Software Engineer- Platform Technologies Team, WSO2 >>>>>>>>>>>>>> Inc. >>>>>>>>>>>>>> Mobile: +94715779733 >>>>>>>>>>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> >>>>>>>>>>> Thanks & regards, >>>>>>>>>>> Nirmal >>>>>>>>>>> >>>>>>>>>>> Senior Software Engineer- Platform Technologies Team, WSO2 Inc. >>>>>>>>>>> Mobile: +94715779733 >>>>>>>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> >>>>>>>>> Thanks & regards, >>>>>>>>> Nirmal >>>>>>>>> >>>>>>>>> Senior Software Engineer- Platform Technologies Team, WSO2 Inc. >>>>>>>>> Mobile: +94715779733 >>>>>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> Thanks & regards, >>>>>>> Nirmal >>>>>>> >>>>>>> Senior Software Engineer- Platform Technologies Team, WSO2 Inc. >>>>>>> Mobile: +94715779733 >>>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Thanks & regards, >>>>> Nirmal >>>>> >>>>> Senior Software Engineer- Platform Technologies Team, WSO2 Inc. >>>>> Mobile: +94715779733 >>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> *Supun Sethunga* >>>> Software Engineer >>>> WSO2, Inc. >>>> lean | enterprise | middleware >>>> Mobile : +94 716546324 >>>> >>> >>> >> >> >> -- >> *Supun Sethunga* >> Software Engineer >> WSO2, Inc. >> http://wso2.com/ >> lean | enterprise | middleware >> Mobile : +94 716546324 >> > > -- *Supun Sethunga* Software Engineer WSO2, Inc. http://wso2.com/ lean | enterprise | middleware Mobile : +94 716546324
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
