Hi Supun, I just neeed a little help. I am in the process of integrating my samoa core functions and extension into carbon-ml's siddhi extension. For samoa i am using locally built samoa project to provide samoa 0.4.0 dependencies,since we dont have it in the maven repo or else where. But when i build carbon-ml by adding samoa dependencies, it seems to be maven search for remote location, not the local maven repo first. I am running maven with -U option. But still the problems occurs. Is there any specific thing in carbon-ml like settings to search remote before local one? I just need to give local maven repo for the dependency. My extension seperately working fine, so there is no problem wihat the local dependencies outside carbon-ml. So can you please help me with this.thank you. regards, Mahesh.
On Fri, Jul 22, 2016 at 3:17 PM, Srinath Perera <[email protected]> wrote: > Hi Mahesh, > > On Thu, Jul 21, 2016 at 2:10 PM, Mahesh Dananjaya < > [email protected]> wrote: > >> Hi All, >> I am onto connecting cep streams with samoa streams to data analysis >> using samoa framework. To connect samoa with cep siddhi event streams what >> i we can do is that try to convert cep streams into samoa streams or else >> writing wrpper for samoa for cep streasm to be used. In both cases i have >> to covert siddhi cep streasm into samoa streams. Samoa is using MOA to >> analyse data. Moo contains ML framework to analyse stream data. Samoa is >> wrapping MOA withsome of its classes. >> >> Samoa streams is based on MOA, Instance and InstanceStreams. Samoa see >> streams as a stream of instances [1]. So if we are going to convert cep >> events into samoa instances , it will take time. But if we have some >> similarity between cep siddhi streams and samoa streasm we can reduce the >> time. >> 1. What is the underlying infrastructure for cep siddhi streasm.? >> 2. Are there anything as Instances or InstanceStreams kind of >> implmentation underlying cep streams? >> 3. How can i get more underestanding on CEP siddhi streams. >> >> On the other hand i can use my cep siddhi extension and put those events >> into event queue and convert them into samoa instances and feed them into >> samoa streaming ml topologies. >> > I think this is OK. I assume this is much easier. Let's do this and check > the performance. > > >> There is another option. In Samoa what they are basically doing is that >> wrapping MOA ML framework and write some classes for build streaming ml >> topologies. So as the other option i can wrap samoa moa with my design and >> use moa ml framework directly. (No need for Samoa extension). I have >> building some topologies to streaming data analysis [2]. Main problem is >> that lack of documentation. Anyway i had go through their whole samoa >> design.thank you. >> > > If we use MOA directly, would we loose the distributed support in SAOMA. > Let's do a call when you can, so we can dsicuss this in detail. > > --Srinath > > > >> regards, >> Mahesh. >> >> [1] >> https://github.com/apache/incubator-samoa/blob/master/samoa-api/src/main/java/org/apache/samoa/streams/clustering/ClusteringStream.java >> [2] >> https://github.com/dananjayamahesh/GSOC2016/tree/master/gsoc/samoa/streaming/src/main/java/org/gsoc/samoa/streaming >> >> On Mon, Jul 18, 2016 at 11:40 AM, Mahesh Dananjaya < >> [email protected]> wrote: >> >>> Hi Maheshakya, >>> Samoa modules built as topologies that connect streams with the internal >>> processors. I have already written some examples to test the ML algorithms >>> and samoa analysis topologies. What we need to done is mostly developing a >>> wrapper around samoa topologies to connect their input and output streams >>> with our cep streams. So i am currently going through their stream >>> architecture to connect our streams with their streams. Couple of examples >>> exapaining samoa ml topologies and streaming can be found in my git hub >>> repo [1]. Samoa using MOA ml algorithms by wrapping them with their >>> classes. Initailly i am trying to develop a KMeansClustering analysis with >>> cep streams with samoa ml topologies. >>> And also i could not find a maven repo for samoa 0.4.0 incubating. So i >>> am currently using my local m2 repo's samoa 0..4.0 incubating for my >>> dependencies to work. The local one is built by original samoa source.thank >>> you. >>> >>> regards, >>> Mahesh. >>> >>> [1] >>> https://github.com/dananjayamahesh/GSOC2016/tree/master/gsoc/samoa/streaming/src/main/java/org/gsoc/samoa/streaming >>> >>> >>> On Mon, Jul 18, 2016 at 8:32 AM, Maheshakya Wijewardena < >>> [email protected]> wrote: >>> >>>> Hi Mahesh, >>>> >>>> Can you please share your samoa project? >>>> >>>> On Sun, Jul 17, 2016 at 11:19 AM, Mahesh Dananjaya < >>>> [email protected]> wrote: >>>> >>>>> >>>>> ---------- Forwarded message ---------- >>>>> From: Mahesh Dananjaya <[email protected]> >>>>> Date: Sun, Jul 17, 2016 at 11:18 AM >>>>> Subject: Re: GSOC2016: [ML][CEP] [SAMOA]Predictive analytic with >>>>> online data for WSO2 Machine Learner-Samoa Integration >>>>> To: Maheshakya Wijewardena <[email protected]> >>>>> >>>>> >>>>> Hi Maheshakaya, >>>>> just need a little help. In Samoa when we want to run a class what is >>>>> does it used this commands [1], >>>>> 1. bin/samoa storm target/SAMOA-Storm-0.0.1-SNAPSHOT.jar >>>>> "ClusteringEvaluation" >>>>> 2. bin/samoa storm target/SAMOA-Storm-0.0.1-SNAPSHOT.jar >>>>> "PrequentialEvaluation -d /tmp/dump.csv -i 1000000 -f 100000 -l >>>>> (classifiers.trees.VerticalHoeffdingTree -p 4) -s >>>>> (generators.RandomTreeGenerator -c 2 -o 10 -u 10)" >>>>> >>>>> what is does is call a class named LocalDoTask [4] and pass this >>>>> string as argument.After that that LocalDoTask call the relevent Tasks >>>>> such >>>>> as ClusteringEvaluation or PrequentialEvaluation. [2]. >>>>> >>>>> Now i have add samoa dependencies to my new maven project, where i >>>>> used original samoa source to write examples and test then earlier.Now i >>>>> want to push them into my new java project with samoa dependencies. I >>>>> added >>>>> dependency and it was built fine. Now i am calling my local DoTask.java >>>>> [3] >>>>> file as same as i did with samoa with, >>>>> java -cp target/streaming-1.0-SNAPSHOT.jar >>>>> org.gsoc.samoa.streaming.DoTask >>>>> "org.gsoc.samoa.streaming.ClusteringEvaluation" >>>>> But seems to be i am incorrect in some place. >>>>> Error: A JNI error has occurred, please check your installation and >>>>> try again >>>>> Exception in thread "main" java.lang.NoClassDefFoundError: >>>>> org/apache/samoa/topology/ComponentFactory >>>>> at java.lang.Class.getDeclaredMethods0(Native Method) >>>>> at java.lang.Class.privateGetDeclaredMethods(Class.java:2701) >>>>> at java.lang.Class.privateGetMethodRecursive(Class.java:3048) >>>>> at java.lang.Class.getMethod0(Class.java:3018) >>>>> at java.lang.Class.getMethod(Class.java:1784) >>>>> at >>>>> sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544) >>>>> at >>>>> sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526) >>>>> Caused by: java.lang.ClassNotFoundException: >>>>> org.apache.samoa.topology.ComponentFactory >>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:381) >>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424) >>>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) >>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>>>> ... 7 more >>>>> >>>>> >>>>> can i actually call the Task like this. >>>>> >>>>> BR, >>>>> Mahesh. >>>>> >>>>> [1] >>>>> https://samoa.incubator.apache.org/documentation/Prequential-Evaluation-Task.html >>>>> [2] >>>>> https://github.com/apache/incubator-samoa/blob/releases/0.4.0-incubating-RC0/samoa-api/src/main/java/org/apache/samoa/tasks/ClusteringEvaluation.java >>>>> [3] >>>>> https://github.com/dananjayamahesh/GSOC2016/tree/master/gsoc/samoa/streaming/src/main/java/org/gsoc/samoa/streaming >>>>> [4] >>>>> https://github.com/apache/incubator-samoa/tree/releases/0.4.0-incubating-RC0/samoa-local/src/main/java/org/apache/samoa >>>>> >>>>> >>>>> On Thu, Jul 14, 2016 at 3:47 PM, Mahesh Dananjaya < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi srinath, >>>>>> sure.i am working on it.thank you. >>>>>> regards, >>>>>> Mahesh. >>>>>> >>>>>> On Thu, Jul 14, 2016 at 11:12 AM, Srinath Perera <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi Mahesh, >>>>>>> >>>>>>> Let's focus on getting SAOMA work with CEP. It is OK to be limited >>>>>>> to few algorithms. >>>>>>> >>>>>>> --Srinath >>>>>>> >>>>>>> On Thu, Jul 14, 2016 at 10:49 AM, Mahesh Dananjaya < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hi Maheshakya, >>>>>>>> I think we can build new tasks [1] like the one in execution plan >>>>>>>> in cep with samoa. I will try to build a one. >>>>>>>> regards, >>>>>>>> Mahesh. >>>>>>>> [1] >>>>>>>> https://samoa.incubator.apache.org/documentation/Developing-New-Tasks-in-SAMOA.html >>>>>>>> >>>>>>>> >>>>>>>> On Thu, Jul 14, 2016 at 10:35 AM, Mahesh Dananjaya < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hi Maheshakya, >>>>>>>>> I am building and running samoa to see its functionality. In samoa >>>>>>>>> still we have limited supports in algorithms. Samoa supports only >>>>>>>>> classification and clustering with streams. It also use kind of >>>>>>>>> StreamProcessor, like the one we use in StreamProcessor extension. I >>>>>>>>> was >>>>>>>>> getting started with Samoa referring to this page [1]. Then i ran >>>>>>>>> couple of >>>>>>>>> examples to identified the flow. Samoa use hadoop framework instead >>>>>>>>> spark >>>>>>>>> for distribution. But i am using it in a local mode. When i see the >>>>>>>>> Samoa >>>>>>>>> core there is only limited algorithms. IMO if we are going to use >>>>>>>>> Samoa we >>>>>>>>> have to limit the functionality and algorithms [2]. When i go to >>>>>>>>> developer >>>>>>>>> corner in [3], it seems to be something like CEP extension that we are >>>>>>>>> using currenlty. SO in Samoa though the algorihtms are limited, they >>>>>>>>> have >>>>>>>>> implemented streaming support for them. Therefore if we integrate it >>>>>>>>> into >>>>>>>>> CEP we have to look for how to handle streams and algorithms in Samoa >>>>>>>>> side. >>>>>>>>> Is it good for your side to have both hadoop and spark running >>>>>>>>> background.thank you. >>>>>>>>> regards, >>>>>>>>> Mahesh. >>>>>>>>> >>>>>>>>> [1] https://samoa.incubator.apache.org/documentation/Home.html >>>>>>>>> [2] >>>>>>>>> https://samoa.incubator.apache.org/documentation/api/current/index.html >>>>>>>>> [3] >>>>>>>>> https://samoa.incubator.apache.org/documentation/SAMOA-Topology.html >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> ============================ >>>>>>> Srinath Perera, Ph.D. >>>>>>> http://people.apache.org/~hemapani/ >>>>>>> http://srinathsview.blogspot.com/ >>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Dev mailing list >>>>> [email protected] >>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>> >>>>> >>>> >>>> >>>> -- >>>> Pruthuvi Maheshakya Wijewardena >>>> >>>> [email protected] >>>> +94711228855 >>>> >>>> >>>> >>> >> > > > -- > ============================ > Srinath Perera, Ph.D. > http://people.apache.org/~hemapani/ > http://srinathsview.blogspot.com/ >
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
