Thanks Mohan! On Aug 25, 2016 5:30 PM, "Mohanadarshan Vivekanandalingam" <[email protected]> wrote:
> > > On Thu, Aug 25, 2016 at 11:33 AM, Malintha Amarasinghe <[email protected] > > wrote: > >> Hi, >> >> On Wed, Aug 24, 2016 at 11:02 AM, Mohanadarshan Vivekanandalingam < >> [email protected]> wrote: >> >>> HI Malintha/Nuwan, >>> >>> I have few queries and like to provide few clarifications on this.. >>> >>> Actually most of the points that you guys raised are discussed before >>> and changed based on some experiences.. >>> >>> - Defining stream from client side : We had this in old datapublisher >>> component and removed due to various complications.. As Srinath mentioned >>> this was done after many rounds of discussions.. >>> >>> - Having single configuration file to add DAS endpoint : In past, we had >>> two level of configs (one for adapter which deals with transport and one >>> for event formatting which deals with stream). That means we can use one >>> event adapter for multiple streams.. But, there were some complaints saying >>> it is not user friendly and should have one config to publish events.. >>> >>> And also, event publisher is also another deployable artifact like other >>> synapse files.. In APIM, we are configuring key manager url in multiple api >>> configs (revoke api and etc.. ), event publisher is also similar to that.. >>> >>> But, I do understand that your points are valid and think we have two >>> options.. >>> >>> - Have a global property in output-adapter-config.xml and use that in >>> Thrift event publisher (if that property exists).. >>> >> > @All, I have incorporated above fix by introducing a global default > property in master branch > > Thanks, > Mohan > > >> > >> - Allow to keep the DAS endpoint as a registry resource and use that.. >>> >>> Thanks for the response. It would be great to have above in order to >> mitigate the duplicated configuration issues. >> > > >> Thanks! >> Malintha >> >>> Thanks, >>> Mohan >>> >>> >>> On Wed, Aug 24, 2016 at 10:05 AM, Nuwan Dias <[email protected]> wrote: >>> >>>> Hi, >>>> >>>> I spoke with Srinath regarding this too. It would be ideal to have a >>>> global config which can override the per-stream configs so that we won't >>>> have to maintain the DAS configs in multiple places. In all cases we've >>>> come across so far we've only seen people use a single DAS (or cluster) and >>>> publish all streams to that DAS cluster. So at least as of now I don't see >>>> much value in having per stream config files. >>>> >>>> It would also be good if we can omit the necessity of having to define >>>> the streams on the client side (because they're already defined on the >>>> server). I see two main problems of having to define them on the clients. >>>> >>>> 1. When we have multiple event generators (a gateway cluster), the >>>> streams will have to be defined on all nodes. Do we use dep-sync to sync up >>>> the configs on the clients? Even if we do, I think its better to not depend >>>> on dep-sync for the new stuff we're developing because we're moving away >>>> from dep-sync soon. >>>> >>>> 2. Problems in upgrading to newer product versions. When upgrading the >>>> product version, if a schema change has occurred, currently we only have to >>>> worry about changing the schema on the server side. With the above >>>> limitation, we now have to worry about changing the clients too. >>>> >>>> Thanks, >>>> NuwanD. >>>> >>>> On Tue, Aug 23, 2016 at 10:16 PM, Malintha Amarasinghe < >>>> [email protected]> wrote: >>>> >>>>> Hi All, >>>>> >>>>> Currently APIM is using an internal APIM specific configuration >>>>> resides in api-manager.xml which include DAS server URL, username, >>>>> password >>>>> etc. Those configuration are used to instantiate >>>>> an org.wso2.carbon.databridge.agent.DataPublisher object, which is >>>>> then used to publish events directly to streams in DAS. >>>>> >>>>> As it is an APIM specific configuration, it is not reusable by other >>>>> non-APIM features. Also org.wso2.carbon.databridge.agent.DataPublisher is >>>>> intended to be used in non-carbon environments, for carbon environments, >>>>> the recommended way is to use the Event Stream OSGi Service. >>>>> >>>>> So we were trying to re-structure APIM data publishing code to use new >>>>> Event Stream OSGi service explained in [1]. >>>>> >>>>> 1. Create all the streams defined in DAS in APIM. >>>>> 2. Create an *event publisher per each stream *which takes data from >>>>> the stream and publish to DAS. >>>>> 3. In each event publisher we need to configure DAS specific >>>>> configuration. >>>>> >>>>> Ex: >>>>> <?xml version="1.0" encoding="UTF-8"?> >>>>> <eventPublisher name="test" statistics="disable" trace="disable" >>>>> xmlns="http://wso2.org/carbon/eventpublisher";> >>>>> <from streamName="streamTest" version="1.0.0"/> >>>>> <mapping customMapping="disable" type="wso2event"/> >>>>> <to eventAdapterType="wso2event"> >>>>> <property name="username">admin</property> >>>>> <property name="protocol">thrift</property> >>>>> <property name="publishingMode">non-blocking</property> >>>>> <property name="publishTimeout">0</property> >>>>> <property name="receiverURL">tcp://localhost:7611</property> >>>>> <property encrypted="true" name="password">XXXXXXXX</property> >>>>> </to> >>>>> </eventPublisher> >>>>> >>>>> But we are facing a small problem here; in APIM there are many event >>>>> streams being used. If there's a change happen to DAS configuration, we >>>>> need to change it in all the event publishers which makes it difficult to >>>>> maintain. As per the offline chat with DAS team, this is a current >>>>> limitation. >>>>> >>>>> Are we going to move forward with the existing implementation? >>>>> >>>>> Thanks, >>>>> Malintha >>>>> >>>>> [1] [Dev] Common configuration for publishing events from carbon >>>>> servers to DAS/CEP >>>>> -- >>>>> Malintha Amarasinghe >>>>> Software Engineer >>>>> *WSO2, Inc. - lean | enterprise | middleware* >>>>> http://wso2.com/ >>>>> >>>>> Mobile : +94 712383306 >>>>> >>>> >>>> >>>> >>>> -- >>>> Nuwan Dias >>>> >>>> Software Architect - WSO2, Inc. http://wso2.com >>>> email : [email protected] >>>> Phone : +94 777 775 729 >>>> >>>> _______________________________________________ >>>> Dev mailing list >>>> [email protected] >>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>> >>>> >>> >>> >>> -- >>> *V. Mohanadarshan* >>> *Associate Tech Lead,* >>> *Data Technologies Team,* >>> *WSO2, Inc. http://wso2.com <http://wso2.com> * >>> *lean.enterprise.middleware.* >>> >>> email: [email protected] >>> phone:(+94) 771117673 >>> >> >> >> >> -- >> Malintha Amarasinghe >> Software Engineer >> *WSO2, Inc. - lean | enterprise | middleware* >> http://wso2.com/ >> >> Mobile : +94 712383306 >> > > > > -- > *V. Mohanadarshan* > *Associate Tech Lead,* > *Data Technologies Team,* > *WSO2, Inc. http://wso2.com <http://wso2.com> * > *lean.enterprise.middleware.* > > email: [email protected] > phone:(+94) 771117673 >
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
