Hi, On Wed, Aug 24, 2016 at 11:02 AM, Mohanadarshan Vivekanandalingam < [email protected]> wrote:
> HI Malintha/Nuwan, > > I have few queries and like to provide few clarifications on this.. > > Actually most of the points that you guys raised are discussed before and > changed based on some experiences.. > > - Defining stream from client side : We had this in old datapublisher > component and removed due to various complications.. As Srinath mentioned > this was done after many rounds of discussions.. > > - Having single configuration file to add DAS endpoint : In past, we had > two level of configs (one for adapter which deals with transport and one > for event formatting which deals with stream). That means we can use one > event adapter for multiple streams.. But, there were some complaints saying > it is not user friendly and should have one config to publish events.. > > And also, event publisher is also another deployable artifact like other > synapse files.. In APIM, we are configuring key manager url in multiple api > configs (revoke api and etc.. ), event publisher is also similar to that.. > > But, I do understand that your points are valid and think we have two > options.. > > - Have a global property in output-adapter-config.xml and use that in > Thrift event publisher (if that property exists).. > - Allow to keep the DAS endpoint as a registry resource and use that.. > > Thanks for the response. It would be great to have above in order to mitigate the duplicated configuration issues. Thanks! Malintha > Thanks, > Mohan > > > On Wed, Aug 24, 2016 at 10:05 AM, Nuwan Dias <[email protected]> wrote: > >> Hi, >> >> I spoke with Srinath regarding this too. It would be ideal to have a >> global config which can override the per-stream configs so that we won't >> have to maintain the DAS configs in multiple places. In all cases we've >> come across so far we've only seen people use a single DAS (or cluster) and >> publish all streams to that DAS cluster. So at least as of now I don't see >> much value in having per stream config files. >> >> It would also be good if we can omit the necessity of having to define >> the streams on the client side (because they're already defined on the >> server). I see two main problems of having to define them on the clients. >> >> 1. When we have multiple event generators (a gateway cluster), the >> streams will have to be defined on all nodes. Do we use dep-sync to sync up >> the configs on the clients? Even if we do, I think its better to not depend >> on dep-sync for the new stuff we're developing because we're moving away >> from dep-sync soon. >> >> 2. Problems in upgrading to newer product versions. When upgrading the >> product version, if a schema change has occurred, currently we only have to >> worry about changing the schema on the server side. With the above >> limitation, we now have to worry about changing the clients too. >> >> Thanks, >> NuwanD. >> >> On Tue, Aug 23, 2016 at 10:16 PM, Malintha Amarasinghe < >> [email protected]> wrote: >> >>> Hi All, >>> >>> Currently APIM is using an internal APIM specific configuration resides >>> in api-manager.xml which include DAS server URL, username, password etc. >>> Those configuration are used to instantiate an org.wso2.carbon.databridge. >>> agent.DataPublisher object, which is then used to publish events >>> directly to streams in DAS. >>> >>> As it is an APIM specific configuration, it is not reusable by other >>> non-APIM features. Also org.wso2.carbon.databridge.agent.DataPublisher is >>> intended to be used in non-carbon environments, for carbon environments, >>> the recommended way is to use the Event Stream OSGi Service. >>> >>> So we were trying to re-structure APIM data publishing code to use new >>> Event Stream OSGi service explained in [1]. >>> >>> 1. Create all the streams defined in DAS in APIM. >>> 2. Create an *event publisher per each stream *which takes data from >>> the stream and publish to DAS. >>> 3. In each event publisher we need to configure DAS specific >>> configuration. >>> >>> Ex: >>> <?xml version="1.0" encoding="UTF-8"?> >>> <eventPublisher name="test" statistics="disable" trace="disable" xmlns=" >>> http://wso2.org/carbon/eventpublisher";> >>> <from streamName="streamTest" version="1.0.0"/> >>> <mapping customMapping="disable" type="wso2event"/> >>> <to eventAdapterType="wso2event"> >>> <property name="username">admin</property> >>> <property name="protocol">thrift</property> >>> <property name="publishingMode">non-blocking</property> >>> <property name="publishTimeout">0</property> >>> <property name="receiverURL">tcp://localhost:7611</property> >>> <property encrypted="true" name="password">XXXXXXXX</property> >>> </to> >>> </eventPublisher> >>> >>> But we are facing a small problem here; in APIM there are many event >>> streams being used. If there's a change happen to DAS configuration, we >>> need to change it in all the event publishers which makes it difficult to >>> maintain. As per the offline chat with DAS team, this is a current >>> limitation. >>> >>> Are we going to move forward with the existing implementation? >>> >>> Thanks, >>> Malintha >>> >>> [1] [Dev] Common configuration for publishing events from carbon servers >>> to DAS/CEP >>> -- >>> Malintha Amarasinghe >>> Software Engineer >>> *WSO2, Inc. - lean | enterprise | middleware* >>> http://wso2.com/ >>> >>> Mobile : +94 712383306 >>> >> >> >> >> -- >> Nuwan Dias >> >> Software Architect - WSO2, Inc. http://wso2.com >> email : [email protected] >> Phone : +94 777 775 729 >> >> _______________________________________________ >> Dev mailing list >> [email protected] >> http://wso2.org/cgi-bin/mailman/listinfo/dev >> >> > > > -- > *V. Mohanadarshan* > *Associate Tech Lead,* > *Data Technologies Team,* > *WSO2, Inc. http://wso2.com <http://wso2.com> * > *lean.enterprise.middleware.* > > email: [email protected] > phone:(+94) 771117673 > -- Malintha Amarasinghe Software Engineer *WSO2, Inc. - lean | enterprise | middleware* http://wso2.com/ Mobile : +94 712383306
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
