Hi Maheshakya, I have a small problem. As we did all the time when i build a samoa streaming extension for the cep by providing samoa dependencies, i put that jar into cep_home/repository/component/lib/ folder.The start cep and try to develop the execution plan by providing my extension. i am remotely debugging the samoa streaming extension that connected to our samoa predictive learning topology. While on the run i get this exception. java.lang.ClassNotFoundException: org.apache.samoa.topology.ComponentFactory cannot be found by streaming_1.0_SNAPSHOT_1.0.0.
But without cep my samoa core is running fine. It seems to be samoa is not recognizing in the cep side. Is this kind of a dependency problem. In my extension side i think nothing wrong. When i replace the streamProcessor extension with locally build cep event stream emulation java class all the things run fine. The probelm occurs only when this is running with cep. Do i need to add anything on the cep side to underestand samoa in cep side.can you please help me with this.thank you. regards, Mahesh. On Wed, Jul 27, 2016 at 3:59 PM, Mahesh Dananjaya <[email protected] > wrote: > Hi Srinath, > > "I think this is OK. I assume this is much easier. Let's do this and check > the performance", > I also think so. I am currently on this and have a progress in this.for > your question, > "If we use MOA directly, would we loose the distributed support in SAOMA. > Let's do a call when you can, so we can dsicuss this in detail." > I have to check for that.i think if we are using MOA, we can use > distributed clusters. As i wen through their documentation MOA itself > cannot support distribution. But samoa can. What samoa does is providing > streaming and clustering support by wrapping MOA algorithms. So i think we > dont need to go for that option,directly MOA. Because now we can handle > samoa building blocks.So we had 2 options for integrating it with cep > without exploiting samoa architecture which is highly modular,scalable and > flexible. > 1. Develop Samoa topologies with basic samoa building blocks which make > use of MOA algorithms. > 2. Creating New streaming options with samoa stream building blocks which > can feed cep siddhi events into samoa streams and get results samoa streams > to cep back. > > As 2nd option is easy and take reasonable time i am currently developing > some modules to integrate cep streams into samoa which can be easily > further extended to 1 option as well. So i had to modify stream and > entrance modules for that and i think is has good progress. > So now i can feed my custom input stream to samoa topologies. That means > i can easily integrate cep event stream into samoa instance stream. > currently i am verifying the streaming clustering algorithms and its > results with my custom input input streams which can be connected to samoa > instance streams. As i have already developed siddhi extension for > streaming, i can use them to feed my custom input streams now. As the > initial step i am go with the streaming clustering algorithms. Those are in > my GSOC github repo [1]. > clustering - Streaming Clustering Support with samoa and CEP > streaming - Streaming extension for samoa for cep evet streams > > i am currenlty working on the verification of results with some of our > custom streams and then we will just have to integrate it with my > extensions, which are already developed for cep as my first part.thank you. > regards, > Mahesh. > > [1] > https://github.com/dananjayamahesh/GSOC2016/tree/master/gsoc/samoa/streaming/src/main/java/org/gsoc/samoa/streaming > > On Fri, Jul 22, 2016 at 3:17 PM, Srinath Perera <[email protected]> wrote: > >> Hi Mahesh, >> >> On Thu, Jul 21, 2016 at 2:10 PM, Mahesh Dananjaya < >> [email protected]> wrote: >> >>> Hi All, >>> I am onto connecting cep streams with samoa streams to data analysis >>> using samoa framework. To connect samoa with cep siddhi event streams what >>> i we can do is that try to convert cep streams into samoa streams or else >>> writing wrpper for samoa for cep streasm to be used. In both cases i have >>> to covert siddhi cep streasm into samoa streams. Samoa is using MOA to >>> analyse data. Moo contains ML framework to analyse stream data. Samoa is >>> wrapping MOA withsome of its classes. >>> >>> Samoa streams is based on MOA, Instance and InstanceStreams. Samoa see >>> streams as a stream of instances [1]. So if we are going to convert cep >>> events into samoa instances , it will take time. But if we have some >>> similarity between cep siddhi streams and samoa streasm we can reduce the >>> time. >>> 1. What is the underlying infrastructure for cep siddhi streasm.? >>> 2. Are there anything as Instances or InstanceStreams kind of >>> implmentation underlying cep streams? >>> 3. How can i get more underestanding on CEP siddhi streams. >>> >>> On the other hand i can use my cep siddhi extension and put those events >>> into event queue and convert them into samoa instances and feed them into >>> samoa streaming ml topologies. >>> >> I think this is OK. I assume this is much easier. Let's do this and check >> the performance. >> >> >>> There is another option. In Samoa what they are basically doing is that >>> wrapping MOA ML framework and write some classes for build streaming ml >>> topologies. So as the other option i can wrap samoa moa with my design and >>> use moa ml framework directly. (No need for Samoa extension). I have >>> building some topologies to streaming data analysis [2]. Main problem is >>> that lack of documentation. Anyway i had go through their whole samoa >>> design.thank you. >>> >> >> If we use MOA directly, would we loose the distributed support in SAOMA. >> Let's do a call when you can, so we can dsicuss this in detail. >> >> --Srinath >> >> >> >>> regards, >>> Mahesh. >>> >>> [1] >>> https://github.com/apache/incubator-samoa/blob/master/samoa-api/src/main/java/org/apache/samoa/streams/clustering/ClusteringStream.java >>> [2] >>> https://github.com/dananjayamahesh/GSOC2016/tree/master/gsoc/samoa/streaming/src/main/java/org/gsoc/samoa/streaming >>> >>> On Mon, Jul 18, 2016 at 11:40 AM, Mahesh Dananjaya < >>> [email protected]> wrote: >>> >>>> Hi Maheshakya, >>>> Samoa modules built as topologies that connect streams with the >>>> internal processors. I have already written some examples to test the ML >>>> algorithms and samoa analysis topologies. What we need to done is mostly >>>> developing a wrapper around samoa topologies to connect their input and >>>> output streams with our cep streams. So i am currently going through their >>>> stream architecture to connect our streams with their streams. Couple of >>>> examples exapaining samoa ml topologies and streaming can be found in my >>>> git hub repo [1]. Samoa using MOA ml algorithms by wrapping them with their >>>> classes. Initailly i am trying to develop a KMeansClustering analysis with >>>> cep streams with samoa ml topologies. >>>> And also i could not find a maven repo for samoa 0.4.0 incubating. So i >>>> am currently using my local m2 repo's samoa 0..4.0 incubating for my >>>> dependencies to work. The local one is built by original samoa source.thank >>>> you. >>>> >>>> regards, >>>> Mahesh. >>>> >>>> [1] >>>> https://github.com/dananjayamahesh/GSOC2016/tree/master/gsoc/samoa/streaming/src/main/java/org/gsoc/samoa/streaming >>>> >>>> >>>> On Mon, Jul 18, 2016 at 8:32 AM, Maheshakya Wijewardena < >>>> [email protected]> wrote: >>>> >>>>> Hi Mahesh, >>>>> >>>>> Can you please share your samoa project? >>>>> >>>>> On Sun, Jul 17, 2016 at 11:19 AM, Mahesh Dananjaya < >>>>> [email protected]> wrote: >>>>> >>>>>> >>>>>> ---------- Forwarded message ---------- >>>>>> From: Mahesh Dananjaya <[email protected]> >>>>>> Date: Sun, Jul 17, 2016 at 11:18 AM >>>>>> Subject: Re: GSOC2016: [ML][CEP] [SAMOA]Predictive analytic with >>>>>> online data for WSO2 Machine Learner-Samoa Integration >>>>>> To: Maheshakya Wijewardena <[email protected]> >>>>>> >>>>>> >>>>>> Hi Maheshakaya, >>>>>> just need a little help. In Samoa when we want to run a class what is >>>>>> does it used this commands [1], >>>>>> 1. bin/samoa storm target/SAMOA-Storm-0.0.1-SNAPSHOT.jar >>>>>> "ClusteringEvaluation" >>>>>> 2. bin/samoa storm target/SAMOA-Storm-0.0.1-SNAPSHOT.jar >>>>>> "PrequentialEvaluation -d /tmp/dump.csv -i 1000000 -f 100000 -l >>>>>> (classifiers.trees.VerticalHoeffdingTree -p 4) -s >>>>>> (generators.RandomTreeGenerator -c 2 -o 10 -u 10)" >>>>>> >>>>>> what is does is call a class named LocalDoTask [4] and pass this >>>>>> string as argument.After that that LocalDoTask call the relevent Tasks >>>>>> such >>>>>> as ClusteringEvaluation or PrequentialEvaluation. [2]. >>>>>> >>>>>> Now i have add samoa dependencies to my new maven project, where i >>>>>> used original samoa source to write examples and test then earlier.Now i >>>>>> want to push them into my new java project with samoa dependencies. I >>>>>> added >>>>>> dependency and it was built fine. Now i am calling my local DoTask.java >>>>>> [3] >>>>>> file as same as i did with samoa with, >>>>>> java -cp target/streaming-1.0-SNAPSHOT.jar >>>>>> org.gsoc.samoa.streaming.DoTask >>>>>> "org.gsoc.samoa.streaming.ClusteringEvaluation" >>>>>> But seems to be i am incorrect in some place. >>>>>> Error: A JNI error has occurred, please check your installation and >>>>>> try again >>>>>> Exception in thread "main" java.lang.NoClassDefFoundError: >>>>>> org/apache/samoa/topology/ComponentFactory >>>>>> at java.lang.Class.getDeclaredMethods0(Native Method) >>>>>> at java.lang.Class.privateGetDeclaredMethods(Class.java:2701) >>>>>> at java.lang.Class.privateGetMethodRecursive(Class.java:3048) >>>>>> at java.lang.Class.getMethod0(Class.java:3018) >>>>>> at java.lang.Class.getMethod(Class.java:1784) >>>>>> at >>>>>> sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544) >>>>>> at >>>>>> sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526) >>>>>> Caused by: java.lang.ClassNotFoundException: >>>>>> org.apache.samoa.topology.ComponentFactory >>>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:381) >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424) >>>>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>>>>> ... 7 more >>>>>> >>>>>> >>>>>> can i actually call the Task like this. >>>>>> >>>>>> BR, >>>>>> Mahesh. >>>>>> >>>>>> [1] >>>>>> https://samoa.incubator.apache.org/documentation/Prequential-Evaluation-Task.html >>>>>> [2] >>>>>> https://github.com/apache/incubator-samoa/blob/releases/0.4.0-incubating-RC0/samoa-api/src/main/java/org/apache/samoa/tasks/ClusteringEvaluation.java >>>>>> [3] >>>>>> https://github.com/dananjayamahesh/GSOC2016/tree/master/gsoc/samoa/streaming/src/main/java/org/gsoc/samoa/streaming >>>>>> [4] >>>>>> https://github.com/apache/incubator-samoa/tree/releases/0.4.0-incubating-RC0/samoa-local/src/main/java/org/apache/samoa >>>>>> >>>>>> >>>>>> On Thu, Jul 14, 2016 at 3:47 PM, Mahesh Dananjaya < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hi srinath, >>>>>>> sure.i am working on it.thank you. >>>>>>> regards, >>>>>>> Mahesh. >>>>>>> >>>>>>> On Thu, Jul 14, 2016 at 11:12 AM, Srinath Perera <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Mahesh, >>>>>>>> >>>>>>>> Let's focus on getting SAOMA work with CEP. It is OK to be limited >>>>>>>> to few algorithms. >>>>>>>> >>>>>>>> --Srinath >>>>>>>> >>>>>>>> On Thu, Jul 14, 2016 at 10:49 AM, Mahesh Dananjaya < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hi Maheshakya, >>>>>>>>> I think we can build new tasks [1] like the one in execution plan >>>>>>>>> in cep with samoa. I will try to build a one. >>>>>>>>> regards, >>>>>>>>> Mahesh. >>>>>>>>> [1] >>>>>>>>> https://samoa.incubator.apache.org/documentation/Developing-New-Tasks-in-SAMOA.html >>>>>>>>> >>>>>>>>> >>>>>>>>> On Thu, Jul 14, 2016 at 10:35 AM, Mahesh Dananjaya < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi Maheshakya, >>>>>>>>>> I am building and running samoa to see its functionality. In >>>>>>>>>> samoa still we have limited supports in algorithms. Samoa supports >>>>>>>>>> only >>>>>>>>>> classification and clustering with streams. It also use kind of >>>>>>>>>> StreamProcessor, like the one we use in StreamProcessor extension. >>>>>>>>>> I was >>>>>>>>>> getting started with Samoa referring to this page [1]. Then i ran >>>>>>>>>> couple of >>>>>>>>>> examples to identified the flow. Samoa use hadoop framework instead >>>>>>>>>> spark >>>>>>>>>> for distribution. But i am using it in a local mode. When i see the >>>>>>>>>> Samoa >>>>>>>>>> core there is only limited algorithms. IMO if we are going to use >>>>>>>>>> Samoa we >>>>>>>>>> have to limit the functionality and algorithms [2]. When i go to >>>>>>>>>> developer >>>>>>>>>> corner in [3], it seems to be something like CEP extension that we >>>>>>>>>> are >>>>>>>>>> using currenlty. SO in Samoa though the algorihtms are limited, they >>>>>>>>>> have >>>>>>>>>> implemented streaming support for them. Therefore if we integrate it >>>>>>>>>> into >>>>>>>>>> CEP we have to look for how to handle streams and algorithms in >>>>>>>>>> Samoa side. >>>>>>>>>> Is it good for your side to have both hadoop and spark running >>>>>>>>>> background.thank you. >>>>>>>>>> regards, >>>>>>>>>> Mahesh. >>>>>>>>>> >>>>>>>>>> [1] https://samoa.incubator.apache.org/documentation/Home.html >>>>>>>>>> [2] >>>>>>>>>> https://samoa.incubator.apache.org/documentation/api/current/index.html >>>>>>>>>> [3] >>>>>>>>>> https://samoa.incubator.apache.org/documentation/SAMOA-Topology.html >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> ============================ >>>>>>>> Srinath Perera, Ph.D. >>>>>>>> http://people.apache.org/~hemapani/ >>>>>>>> http://srinathsview.blogspot.com/ >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Dev mailing list >>>>>> [email protected] >>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Pruthuvi Maheshakya Wijewardena >>>>> >>>>> [email protected] >>>>> +94711228855 >>>>> >>>>> >>>>> >>>> >>> >> >> >> -- >> ============================ >> Srinath Perera, Ph.D. >> http://people.apache.org/~hemapani/ >> http://srinathsview.blogspot.com/ >> > >
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
