Hi All, I am reviving this thread as now we have support to add remove streams though CEP UI (and BAM?) and via toolboxes, but we still define the event streams with the data publisher each time. (this thread discusses about stop doing that).
Typical CEP/ BAM usecase would go like following 1) first you identify and define streams 2) then you write queries 3) then you write/configure the agents and point to streams they should publish to. However, we redefine streams from agents every time we publish. This has several complications. 1) If an agent choose to define stream different from queries, everything get messed up. (problem was in samples in BAM). 2) If there are multiple event receivers, we need a way to sync them with db everytime we update a stream. I would like to propose we switch to a model where we write queries and streams and deploy them as a toolbox. Then, just mention the stream name in the event publisher to keep things simple. --Srinath On Sat, Jan 19, 2013 at 5:41 PM, Tharindu Mathew <[email protected]> wrote: > A store discover already supported event streams seems to be a good idea. > It would be good idea which we can implement through a UI or a store. > > But, what do you mean by defining event streams everywhere? > > Clients can define any stream they want, but it is only defined at the > stream definition store. If there is an error, it is shown in the stream > definition store and returned to the client (I hope it is returned at > least, otherwise it is a bug). > > This model was first proposed to get out of the eventing hell that we put > ourselves into. Right now, if you want to publish something, you just > define it and publish. You don't have to switch between multiple servers > just to define and publish some events, which was extremely annoying. > > > > > On Sat, Jan 19, 2013 at 2:28 PM, Amila Suriarachchi <[email protected]>wrote: > >> >> >> On Fri, Jan 18, 2013 at 7:54 PM, Sriskandarajah Suhothayan <[email protected] >> > wrote: >> >>> >>> >>> On Fri, Jan 18, 2013 at 6:46 PM, Sanjiva Weerawarana >>> <[email protected]>wrote: >>> >>>> +1 for the stream definition store idea (a new use of the Store). >>>> >>>> +1 >>> >>>> Suho can we change our client API to take the streamID as a param? That >>>> way a user can look it up in the store and use it directly. >>>> >>>> Currently we do have server side APIs to get StreamDefintons from >>> StreamId or StreamName & Version. >>> >> >> I think we need to separate out the event definition from runtime event >> publishing. >> >> If we look at how brokers used in CEP, first users need to define broker >> at the broker Manager and use the broker id at the CEP bucket level. For >> event streams we can have default set of streams (which BAM publishers and >> other default Agents use) and let users to define at run time. At the event >> publishing side they can use the stream id. >> >> thanks, >> Amila. >> >> >>> >>> Sanjiva. >>>> >>>> >>>> On Fri, Jan 18, 2013 at 6:33 PM, Sriskandarajah Suhothayan < >>>> [email protected]> wrote: >>>> >>>>> >>>>> >>>>> On Fri, Jan 18, 2013 at 4:23 PM, Srinath Perera <[email protected]>wrote: >>>>> >>>>>> Hi Suho, >>>>>> >>>>>> Yes, users can define them. But they do not always have to, and we >>>>>> should recommend that users go and do that once rather than doing it >>>>>> every >>>>>> time. >>>>>> >>>>> Yes this is possible even now. First time when we define stream we >>>>> get a streamId and then we can simply reuse that (even after restart). >>>>> The issue is, our clients are not capable to store the streamId >>>>> returned by the server, and therefore to get the already defined streamId >>>>> corresponding to the stream definition we are redefining the stream. >>>>> This is because our streamId is not simply regeneratable like >>>>> <streamName>-<version> but we are using <streamName>-<version>-UUID. >>>>> >>>>> Suho >>>>> >>>>>> >>>>>> --Srinath >>>>>> >>>>>> >>>>>> On Thu, Jan 17, 2013 at 10:22 PM, Sriskandarajah Suhothayan < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> >>>>>>> >>>>>>> On Fri, Jan 18, 2013 at 2:00 AM, Srinath Perera <[email protected]>wrote: >>>>>>> >>>>>>>> Currently we handle CEP streams by defining it everywhere we publish >>>>>>>> events, and ignoring the duplicates. However, if content of events >>>>>>>> change, then we throw an exception. >>>>>>>> >>>>>>>> I have concerns that we are creating a data stream definition hell >>>>>>>> by >>>>>>>> defining it every where. Also, when something changes, you would >>>>>>>> have >>>>>>>> to change things everywhere. >>>>>>>> >>>>>>>> Maybe we should just put that via registry in one place >>>>>>>> >>>>>>>> e.g. Go to registry and add the definitions. When you publish, just >>>>>>>> use event stream name only, and do not redefine events. >>>>>>>> >>>>>>>> +1 to make registry based Stream Definition Store. >>>>>>> but we still need to allow the clients to define streams. >>>>>>> We have the ability to delete stream so I believe that is good >>>>>>> enough to. >>>>>>> The disadvantage in Cassandra based Stream Definition Store is that >>>>>>> it dont have a UI and hence we cannot tweak the stream Definition. >>>>>>> IMHO having UI for the Stream Definition Store will be a solution. >>>>>>> >>>>>>> Suho >>>>>>> >>>>>>> --Srinath >>>>>>>> >>>>>>>> -- >>>>>>>> ============================ >>>>>>>> Srinath Perera, Ph.D. >>>>>>>> Senior Software Architect, WSO2 Inc. >>>>>>>> Visiting Faculty, University of Moratuwa >>>>>>>> Member, Apache Software Foundation >>>>>>>> Research Scientist, Lanka Software Foundation >>>>>>>> Blog: http://srinathsview.blogspot.com/ >>>>>>>> Photos: http://www.flickr.com/photos/hemapani/ >>>>>>>> Phone: 0772360902 >>>>>>>> _______________________________________________ >>>>>>>> Architecture mailing list >>>>>>>> [email protected] >>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> *S. Suhothayan* >>>>>>> Software Engineer, >>>>>>> Data Technologies Team, >>>>>>> *WSO2, Inc. * >>>>>>> *http://wso2.com <http://wso2.com/> <http://wso2.com/>* >>>>>>> *lean.enterprise.middleware.* >>>>>>> >>>>>>> *email: **[email protected]* <[email protected]> >>>>>>> * cell: (+94) 779 756 757 blog: >>>>>>> **http://suhothayan.blogspot.com/*<http://suhothayan.blogspot.com/> >>>>>>> *twitter: >>>>>>> **http://twitter.com/suhothayan*<http://twitter.com/suhothayan> >>>>>>> * linked-in: **http://lk.linkedin.com/in/suhothayan >>>>>>> <http://lk.linkedin.com/in/suhothayan>* >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> ============================ >>>>>> Srinath Perera, Ph.D. >>>>>> http://www.cs.indiana.edu/~hperera/ >>>>>> http://srinathsview.blogspot.com/ >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> *S. Suhothayan* >>>>> Software Engineer, >>>>> Data Technologies Team, >>>>> *WSO2, Inc. * >>>>> *http://wso2.com <http://wso2.com/> <http://wso2.com/>* >>>>> *lean.enterprise.middleware.* >>>>> >>>>> *email: **[email protected]* <[email protected]> >>>>> * cell: (+94) 779 756 757 blog: >>>>> **http://suhothayan.blogspot.com/*<http://suhothayan.blogspot.com/> >>>>> *twitter: **http://twitter.com/suhothayan*<http://twitter.com/suhothayan> >>>>> * linked-in: **http://lk.linkedin.com/in/suhothayan >>>>> <http://lk.linkedin.com/in/suhothayan>* >>>>> >>>>> >>>>> _______________________________________________ >>>>> Architecture mailing list >>>>> [email protected] >>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>> >>>>> >>>> >>>> >>>> -- >>>> Sanjiva Weerawarana, Ph.D. >>>> Founder, Chairman & CEO; WSO2, Inc.; http://wso2.com/ >>>> email: [email protected]; phone: +94 11 763 9614; cell: +94 77 787 6880| +1 >>>> 650 265 8311 >>>> blog: http://sanjiva.weerawarana.org/ >>>> >>>> Lean . Enterprise . Middleware >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> [email protected] >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> >>> -- >>> >>> *S. Suhothayan* >>> Software Engineer, >>> Data Technologies Team, >>> *WSO2, Inc. * >>> *http://wso2.com <http://wso2.com/> <http://wso2.com/>* >>> *lean.enterprise.middleware.* >>> >>> *email: **[email protected]* <[email protected]> >>> * cell: (+94) 779 756 757 blog: >>> **http://suhothayan.blogspot.com/*<http://suhothayan.blogspot.com/> >>> *twitter: **http://twitter.com/suhothayan*<http://twitter.com/suhothayan> >>> * linked-in: **http://lk.linkedin.com/in/suhothayan >>> <http://lk.linkedin.com/in/suhothayan>* >>> >>> >>> _______________________________________________ >>> Architecture mailing list >>> [email protected] >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> *Amila Suriarachchi* >> >> Software Architect >> WSO2 Inc. ; http://wso2.com >> lean . enterprise . middleware >> >> phone : +94 71 3082805 >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > Regards, > > Tharindu Mathew > > Associate Technical Lead, WSO2 BAM > Member - Data Mgmt. Committee > > blog: http://tharindumathew.com/ > M: +94777759908 > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- ============================ Srinath Perera, Ph.D. http://people.apache.org/~hemapani/ http://srinathsview.blogspot.com/
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
