Hi All, I am onto connecting cep streams with samoa streams to data analysis using samoa framework. To connect samoa with cep siddhi event streams what i we can do is that try to convert cep streams into samoa streams or else writing wrpper for samoa for cep streasm to be used. In both cases i have to covert siddhi cep streasm into samoa streams. Samoa is using MOA to analyse data. Moo contains ML framework to analyse stream data. Samoa is wrapping MOA withsome of its classes.
Samoa streams is based on MOA, Instance and InstanceStreams. Samoa see streams as a stream of instances [1]. So if we are going to convert cep events into samoa instances , it will take time. But if we have some similarity between cep siddhi streams and samoa streasm we can reduce the time. 1. What is the underlying infrastructure for cep siddhi streasm.? 2. Are there anything as Instances or InstanceStreams kind of implmentation underlying cep streams? 3. How can i get more underestanding on CEP siddhi streams. On the other hand i can use my cep siddhi extension and put those events into event queue and convert them into samoa instances and feed them into samoa streaming ml topologies. There is another option. In Samoa what they are basically doing is that wrapping MOA ML framework and write some classes for build streaming ml topologies. So as the other option i can wrap samoa moa with my design and use moa ml framework directly. (No need for Samoa extension). I have building some topologies to streaming data analysis [2]. Main problem is that lack of documentation. Anyway i had go through their whole samoa design.thank you. regards, Mahesh. [1] https://github.com/apache/incubator-samoa/blob/master/samoa-api/src/main/java/org/apache/samoa/streams/clustering/ClusteringStream.java [2] https://github.com/dananjayamahesh/GSOC2016/tree/master/gsoc/samoa/streaming/src/main/java/org/gsoc/samoa/streaming On Mon, Jul 18, 2016 at 11:40 AM, Mahesh Dananjaya < [email protected]> wrote: > Hi Maheshakya, > Samoa modules built as topologies that connect streams with the internal > processors. I have already written some examples to test the ML algorithms > and samoa analysis topologies. What we need to done is mostly developing a > wrapper around samoa topologies to connect their input and output streams > with our cep streams. So i am currently going through their stream > architecture to connect our streams with their streams. Couple of examples > exapaining samoa ml topologies and streaming can be found in my git hub > repo [1]. Samoa using MOA ml algorithms by wrapping them with their > classes. Initailly i am trying to develop a KMeansClustering analysis with > cep streams with samoa ml topologies. > And also i could not find a maven repo for samoa 0.4.0 incubating. So i am > currently using my local m2 repo's samoa 0..4.0 incubating for my > dependencies to work. The local one is built by original samoa source.thank > you. > > regards, > Mahesh. > > [1] > https://github.com/dananjayamahesh/GSOC2016/tree/master/gsoc/samoa/streaming/src/main/java/org/gsoc/samoa/streaming > > > On Mon, Jul 18, 2016 at 8:32 AM, Maheshakya Wijewardena < > [email protected]> wrote: > >> Hi Mahesh, >> >> Can you please share your samoa project? >> >> On Sun, Jul 17, 2016 at 11:19 AM, Mahesh Dananjaya < >> [email protected]> wrote: >> >>> >>> ---------- Forwarded message ---------- >>> From: Mahesh Dananjaya <[email protected]> >>> Date: Sun, Jul 17, 2016 at 11:18 AM >>> Subject: Re: GSOC2016: [ML][CEP] [SAMOA]Predictive analytic with online >>> data for WSO2 Machine Learner-Samoa Integration >>> To: Maheshakya Wijewardena <[email protected]> >>> >>> >>> Hi Maheshakaya, >>> just need a little help. In Samoa when we want to run a class what is >>> does it used this commands [1], >>> 1. bin/samoa storm target/SAMOA-Storm-0.0.1-SNAPSHOT.jar >>> "ClusteringEvaluation" >>> 2. bin/samoa storm target/SAMOA-Storm-0.0.1-SNAPSHOT.jar >>> "PrequentialEvaluation -d /tmp/dump.csv -i 1000000 -f 100000 -l >>> (classifiers.trees.VerticalHoeffdingTree -p 4) -s >>> (generators.RandomTreeGenerator -c 2 -o 10 -u 10)" >>> >>> what is does is call a class named LocalDoTask [4] and pass this string >>> as argument.After that that LocalDoTask call the relevent Tasks such as >>> ClusteringEvaluation or PrequentialEvaluation. [2]. >>> >>> Now i have add samoa dependencies to my new maven project, where i used >>> original samoa source to write examples and test then earlier.Now i want to >>> push them into my new java project with samoa dependencies. I added >>> dependency and it was built fine. Now i am calling my local DoTask.java [3] >>> file as same as i did with samoa with, >>> java -cp target/streaming-1.0-SNAPSHOT.jar >>> org.gsoc.samoa.streaming.DoTask >>> "org.gsoc.samoa.streaming.ClusteringEvaluation" >>> But seems to be i am incorrect in some place. >>> Error: A JNI error has occurred, please check your installation and try >>> again >>> Exception in thread "main" java.lang.NoClassDefFoundError: >>> org/apache/samoa/topology/ComponentFactory >>> at java.lang.Class.getDeclaredMethods0(Native Method) >>> at java.lang.Class.privateGetDeclaredMethods(Class.java:2701) >>> at java.lang.Class.privateGetMethodRecursive(Class.java:3048) >>> at java.lang.Class.getMethod0(Class.java:3018) >>> at java.lang.Class.getMethod(Class.java:1784) >>> at >>> sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544) >>> at >>> sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526) >>> Caused by: java.lang.ClassNotFoundException: >>> org.apache.samoa.topology.ComponentFactory >>> at java.net.URLClassLoader.findClass(URLClassLoader.java:381) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424) >>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>> ... 7 more >>> >>> >>> can i actually call the Task like this. >>> >>> BR, >>> Mahesh. >>> >>> [1] >>> https://samoa.incubator.apache.org/documentation/Prequential-Evaluation-Task.html >>> [2] >>> https://github.com/apache/incubator-samoa/blob/releases/0.4.0-incubating-RC0/samoa-api/src/main/java/org/apache/samoa/tasks/ClusteringEvaluation.java >>> [3] >>> https://github.com/dananjayamahesh/GSOC2016/tree/master/gsoc/samoa/streaming/src/main/java/org/gsoc/samoa/streaming >>> [4] >>> https://github.com/apache/incubator-samoa/tree/releases/0.4.0-incubating-RC0/samoa-local/src/main/java/org/apache/samoa >>> >>> >>> On Thu, Jul 14, 2016 at 3:47 PM, Mahesh Dananjaya < >>> [email protected]> wrote: >>> >>>> Hi srinath, >>>> sure.i am working on it.thank you. >>>> regards, >>>> Mahesh. >>>> >>>> On Thu, Jul 14, 2016 at 11:12 AM, Srinath Perera <[email protected]> >>>> wrote: >>>> >>>>> Hi Mahesh, >>>>> >>>>> Let's focus on getting SAOMA work with CEP. It is OK to be limited to >>>>> few algorithms. >>>>> >>>>> --Srinath >>>>> >>>>> On Thu, Jul 14, 2016 at 10:49 AM, Mahesh Dananjaya < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi Maheshakya, >>>>>> I think we can build new tasks [1] like the one in execution plan in >>>>>> cep with samoa. I will try to build a one. >>>>>> regards, >>>>>> Mahesh. >>>>>> [1] >>>>>> https://samoa.incubator.apache.org/documentation/Developing-New-Tasks-in-SAMOA.html >>>>>> >>>>>> >>>>>> On Thu, Jul 14, 2016 at 10:35 AM, Mahesh Dananjaya < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hi Maheshakya, >>>>>>> I am building and running samoa to see its functionality. In samoa >>>>>>> still we have limited supports in algorithms. Samoa supports only >>>>>>> classification and clustering with streams. It also use kind of >>>>>>> StreamProcessor, like the one we use in StreamProcessor extension. I >>>>>>> was >>>>>>> getting started with Samoa referring to this page [1]. Then i ran >>>>>>> couple of >>>>>>> examples to identified the flow. Samoa use hadoop framework instead >>>>>>> spark >>>>>>> for distribution. But i am using it in a local mode. When i see the >>>>>>> Samoa >>>>>>> core there is only limited algorithms. IMO if we are going to use Samoa >>>>>>> we >>>>>>> have to limit the functionality and algorithms [2]. When i go to >>>>>>> developer >>>>>>> corner in [3], it seems to be something like CEP extension that we are >>>>>>> using currenlty. SO in Samoa though the algorihtms are limited, they >>>>>>> have >>>>>>> implemented streaming support for them. Therefore if we integrate it >>>>>>> into >>>>>>> CEP we have to look for how to handle streams and algorithms in Samoa >>>>>>> side. >>>>>>> Is it good for your side to have both hadoop and spark running >>>>>>> background.thank you. >>>>>>> regards, >>>>>>> Mahesh. >>>>>>> >>>>>>> [1] https://samoa.incubator.apache.org/documentation/Home.html >>>>>>> [2] >>>>>>> https://samoa.incubator.apache.org/documentation/api/current/index.html >>>>>>> [3] >>>>>>> https://samoa.incubator.apache.org/documentation/SAMOA-Topology.html >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> ============================ >>>>> Srinath Perera, Ph.D. >>>>> http://people.apache.org/~hemapani/ >>>>> http://srinathsview.blogspot.com/ >>>>> >>>> >>>> >>> >>> >>> _______________________________________________ >>> Dev mailing list >>> [email protected] >>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>> >>> >> >> >> -- >> Pruthuvi Maheshakya Wijewardena >> >> [email protected] >> +94711228855 >> >> >> >
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
