Hi Maninda, all, On 24 October 2014 12:39, Gihan Anuruddha <[email protected]> wrote:
> We need to maintain a single location to give publisher credential > details. At the moment we are maintaining same details in a different > location that is a bit inconvenient. As an example if we change BAM url > then we have to change BAM server profiles as well as mediation stat > publish UI as well. Also, this password should support secureVault. > +1 > > On Fri, Oct 24, 2014 at 12:25 PM, Maninda Edirisooriya <[email protected]> > wrote: > >> Hi all, >> >> Today morning we had a discussion where Anjana, Lahiru, Vijitha, Kasun >> and I participated. >> >> There we decided to develop a common Data Publisher / Data Sink axis2 >> artifact which is generic throughout the platform. This will enable any >> other carbon products also for finding data publishing connection / >> credential details when they want to publish data to CEP / BAM. >> > Are we going to couple with Axis2 for the data publisher/ data sink? Querying about this since you had mentioned 'axis2 artifacts'.. :-). Can't we make it generic to the platform without coupling with Axis2 since we are moving away from Axis2 in other areas of the platform? >> And specific to the ESB we are going to develop a mediator (it's name is >> not confirmed yet) which will be similar to the current BAM mediator which >> will refer to the above mentioned artifact for credentials and connection >> parameters. As also we should include the transport we are referring in >> that particular data sink / data publisher. For the first cut we can use >> only the Thrift and hope to extend for HTTP and JMS in future. >> >> In new implementation of the mediator, there will be no configuration for >> Stream configuration like the BAM mediator. Instead there will be inline >> configurations in the mediator XML which consists of >> 1. a stream ID (or the combination of Stream Name and Stream Version) >> 2. and mappings from message XPath to the stream field. (i.e. maps from >> message properties to the stream fields.) >> > +1 for mapping message to stream attributes as described above. >> For the convenience to the user, we can prompt the stream fields for >> mapping from the UI, (once stream ID is provided) if the stream is already >> there in the stream definition store. Anyway there (if want to get UI help) >> we have to use the existing streams that were already created from CEP or >> BAM. But still you can add a mapping for a new stream which can >> subsequently be added to the stream definition store from CEP or BAM. >> >> We have decided to go for a mediator instead of a connector as this has >> nothing to do for accessing an external API kind of operation but to >> publish data for our own products according to the connection information >> given by the internal Data Publisher / Data Sink artifact. >> >> WDYT about the cases when we use other transports like HTTP, JMS, MQTT >> etc. ? >> > Will we really be needing HTTP, JMS support when communicating within the platform across different products? Asking this since we already have similar functionality in many places (e.g. ESB, BAM/CEP input adaptors) and our general architectural approach is to get the events/messages into the platform from one product, then communicate within the platform using a single native format for efficiency/performance. WDYT? Thanks, Lasantha > >> Feedback is welcome. >> >> Thanks. >> >> >> *Maninda Edirisooriya* >> Senior Software Engineer >> >> *WSO2, Inc.*lean.enterprise.middleware. >> >> *Blog* : http://maninda.blogspot.com/ >> *E-mail* : [email protected] >> *Skype* : @manindae >> *Twitter* : @maninda >> >> On Mon, Oct 20, 2014 at 8:00 PM, Sriskandarajah Suhothayan <[email protected] >> > wrote: >> >>> >>> >>> On Mon, Oct 20, 2014 at 4:06 AM, Mohanadarshan Vivekanandalingam < >>> [email protected]> wrote: >>> >>>> Hi All, >>>> >>>> We had a discussion sometime back (last year) on this.. Please check >>>> the mail thread [1] for more info.. >>>> >>>> >>> Yes, I believe we all agree that we need to improve the current BAM >>> Mediator. >>> When I had a chat with kasun on this he suggested writing it as a >>> connector will be much cleaner. >>> >>> Where we can have the connection information and stream definition in >>> <init> and <defineStream> then then pass the data via <publish>. >>> >>> I believe its better to have a f2f meeting with kasun on this. >>> >>> Regards >>> Suho >>> >>> [1] [Architecture] Improving Data Publisher Mediator to replace BAM >>>> Mediator >>>> >>>> Thanks, >>>> Mohan >>>> >>>> >>>> On Mon, Oct 20, 2014 at 12:55 PM, Anjana Fernando <[email protected]> >>>> wrote: >>>> >>>>> Hi Sanjiva, >>>>> >>>>> On Sat, Oct 18, 2014 at 9:44 PM, Sanjiva Weerawarana <[email protected] >>>>> > wrote: >>>>> >>>>>> On Fri, Oct 17, 2014 at 6:47 PM, Lahiru Chandima <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> We (Vijitha<[email protected]> and myself) are going to develop an >>>>>>> ESB Connector which can publish events to BAM and CEP. >>>>>>> >>>>>> >>>>>> We're currently using the words "ESB Connector" to mean something >>>>>> that becomes a set of mediators to access (primarily) HTTP APIs. This is >>>>>> not that right?? If so lets not use the same words please. >>>>>> >>>>> >>>>> Yeah, I guess, this is simply a mediator, just like what we have now. >>>>> It was just Kasun has mentioned to Suho, to create this as an ESB >>>>> Connector, where we haven't actually gone into what it means to create a >>>>> connector. @Kasun, maybe you can shed some light on this, on what you >>>>> meant >>>>> earlier. >>>>> >>>>> >>>>>> >>>>>> Currently, BAM mediator is used for sending events from ESB to BAM. >>>>>>> BAM Mediator needs to be configured with server details and streams >>>>>>> need to >>>>>>> be defined prior to using them in sequences of ESB Synapse >>>>>>> configuration. >>>>>>> >>>>>>> With new ESB connector, it will be possible to add all >>>>>>> configurations needed to publish events to BAM or CEP in the ESB Synapse >>>>>>> configuration itself. This brings the configuration to a single place >>>>>>> unlike when using the BAM mediator. >>>>>>> >>>>>> >>>>>> Is this a real requirement? Do we expect different ESB configs >>>>>> (running on the same server) to want to send data to different BAM/CEP >>>>>> endpoints? I understand the multitenancy usecase - is it only for that? >>>>>> >>>>> >>>>> Actually, the idea here is, what we have now for defining the target >>>>> for events, "BAM Profiles", to replace that. Basically in a BAM Profile, >>>>> we >>>>> put all the information such as the event receiver details, and the stream >>>>> definition, and how the properties are mapped using properties from a >>>>> sequence. We want to clean this up by, simply having something equivalent >>>>> to a message store we have now, where we named is "data sink" for now, to >>>>> be the target for events, and the actual stream mapping will be done by >>>>> the >>>>> mediator itself, in the sequence. And the motivation for having this as an >>>>> artifact in the synapse configuration is, where from a place like >>>>> DevStudio >>>>> or any development environment, we can create these deployment artifacts >>>>> can directly deploy them, without going to the admin console and adding >>>>> them through the UI. >>>>> >>>>> >>>>>> >>>>>> *High level architecture - components.* >>>>>>> >>>>>>> 1) New Synapse construct will be introduced to configure details >>>>>>> related to the data sink (ie. CEP or BAM). >>>>>>> >>>>>>> Eg. >>>>>>> >>>>>>> <dataSink name=”bam_thrift_server”> >>>>>>> <url>tcp://localhost:7611<url> >>>>>>> <username>admin</username> >>>>>>> <password>admin</password> >>>>>>> </dataSink> >>>>>>> >>>>>> >>>>>> What do we call this now? The term "dataSink" doesn't work for me but >>>>>> that could be just me. >>>>>> >>>>> >>>>> Yeah, we had a meeting sometime back to discuss this, and that day >>>>> also, we couldn't decide on the name :) .. basically something related to >>>>> a >>>>> dataSink, any suggestions? :) .. >>>>> >>>>> >>>>>> >>>>>> 2) An ESB connector will be developed which can be used to publish >>>>>>> events to a data sink. Connector will use data-bridge API (Thrift) for >>>>>>> publishing events (It will be possible to introduce new transports other >>>>>>> than Thrift later). >>>>>>> >>>>>> >>>>>> Once again, what are the operations? Why do we want to write a >>>>>> connector instead of using / improving the current "BAM mediator"? >>>>>> >>>>> >>>>> So yeah, it will basically be improving/re-writing the current >>>>> mediator, with a name like "dataPublish" or something, without any "BAM" >>>>> part, and with the stream property mapping part added there itself, rather >>>>> than having it in another place. >>>>> >>>>> >>>>>> >>>>>> Thrift stream related data (stream name, version and stream >>>>>>> properties) will be configured in the ESB connector configuration (in >>>>>>> Synapse config), which is more logical compared to configuring it >>>>>>> separately as done in the BAM mediator. >>>>>>> >>>>>> >>>>>> Can you give a specific example to illustrate the difference please? >>>>>> >>>>> >>>>> So basically in the current BAM mediator, we do not give the direct >>>>> mapping to the stream properties there, we simply populate some message >>>>> context properties and call the mediator. And the mediator simply looks up >>>>> the information provided by the BAM Profile to do the actual mapping, >>>>> where >>>>> in our profile, we have given like Xpath expressions to say, what are the >>>>> values in a specific target stream event. >>>>> >>>>> Cheers, >>>>> Anjana. >>>>> >>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Sanjiva. >>>>>> -- >>>>>> Sanjiva Weerawarana, Ph.D. >>>>>> Founder, Chairman & CEO; WSO2, Inc.; http://wso2.com/ >>>>>> email: [email protected]; office: (+1 650 745 4499 | +94 11 214 >>>>>> 5345) x5700; cell: +94 77 787 6880 | +1 408 466 5099; voip: +1 650 >>>>>> 265 8311 >>>>>> blog: http://sanjiva.weerawarana.org/; twitter: @sanjiva >>>>>> Lean . Enterprise . Middleware >>>>>> >>>>>> _______________________________________________ >>>>>> Architecture mailing list >>>>>> [email protected] >>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> *Anjana Fernando* >>>>> Senior Technical Lead >>>>> WSO2 Inc. | http://wso2.com >>>>> lean . enterprise . middleware >>>>> >>>>> _______________________________________________ >>>>> Architecture mailing list >>>>> [email protected] >>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>> >>>>> >>>> >>>> >>>> -- >>>> *V. Mohanadarshan* >>>> *Software Engineer,* >>>> *Data Technologies Team,* >>>> *WSO2, Inc. http://wso2.com <http://wso2.com> * >>>> *lean.enterprise.middleware.* >>>> >>>> email: [email protected] >>>> phone:(+94) 771117673 >>>> >>> >>> >>> >>> -- >>> >>> *S. Suhothayan* >>> Technical Lead & Team Lead of WSO2 Complex Event Processor >>> *WSO2 Inc. *http://wso2.com >>> * <http://wso2.com/>* >>> lean . enterprise . middleware >>> >>> >>> *cell: (+94) 779 756 757 <%28%2B94%29%20779%20756%20757> | blog: >>> http://suhothayan.blogspot.com/ <http://suhothayan.blogspot.com/>twitter: >>> http://twitter.com/suhothayan <http://twitter.com/suhothayan> | linked-in: >>> http://lk.linkedin.com/in/suhothayan <http://lk.linkedin.com/in/suhothayan>* >>> >> >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > W.G. Gihan Anuruddha > Senior Software Engineer | WSO2, Inc. > M: +94772272595 > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- *Lasantha Fernando* Software Engineer - Data Technologies Team WSO2 Inc. http://wso2.com email: [email protected] mobile: (+94) 71 5247551
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
