Hi Gokul, +1 for introducing a convention to propagate activity id field. But still we cannot introduce a tight convention because the propagation mechanism depends on the use case. Data publisher and/or the message flow channel should handle that complexity of activity ID propagation. In BAM side we have no control in that aspect other than supporting activity-ID and sorting-field field configurability
Thanks. *Maninda Edirisooriya* Senior Software Engineer *WSO2, Inc.*lean.enterprise.middleware. *Blog* : http://maninda.blogspot.com/ *E-mail* : [email protected] *Skype* : @manindae *Twitter* : @maninda On Wed, Nov 26, 2014 at 4:02 AM, Gokul Balakrishnan <[email protected]> wrote: > Hi Maninda, > > There's another point worth considering here: how the correlation field is > propagated across nodes. Currently we're using the transport headers (HTTP) > to carry the activity_id field. When we're making the activity monitoring > more generic, we should consider correlating across different transports > such as JMS. We had a discussion on this sometime back in the architecture@ > thread "Improving activity ID propagation in activity data publishing". > > Thanks, > Gokul. > > On 25 November 2014 at 06:05, Maninda Edirisooriya <[email protected]> > wrote: > >> >> On Tue, Nov 25, 2014 at 7:03 PM, Inosh Goonewardena <[email protected]> >> wrote: >> >>> Hi Maninda, >>> >>> On Tue, Nov 25, 2014 at 2:48 PM, Maninda Edirisooriya <[email protected]> >>> wrote: >>> >>>> Current implementation of BAM Activity Monitoring is based on a the >>>> activity ID field and sorting the timestamps. It has some limitations. >>>> >>>> 1. Only one activity monitoring scenario is possible per tenant >>>> >>> >>> Can you elaborate more on this. Do you mean that one activity can have >>> events from different tenants? >>> >> No. I mean activity monitoring should be possible with different >> correlation fields other than hard coded "activityID" field per tenant. >> >>> >>> >>>> 2. Activity ID the the hard coded parameter used for correlating among >>>> events >>>> >>> 3. Ordering mechanism is based only on sorting operation of timestamp >>>> field >>>> 4. Tightly bound with Cassandra custom indexes >>>> >>> >>> Yes. We have above known limitations in the current architecture >>> basically because indexing and searching is implemented based on Cassandra >>> custom index CFs which doesn't support advance sorting/searching and >>> grouping operations. >>> >>> >>>> >>>> With the introduction of custom data stores in BAM we should support >>>> activity monitoring for RDBMS indexes. There we can use SQL queries as >>>> tasks for updating indexes. This enables us to use multiple activity >>>> monitoring scenarios per each stream. For example all session ID, >>>> transaction ID and username can be considered as activities for different >>>> monitoring requirements at the same time. With the toolbox deployment we >>>> can define all the activity monitoring scenarios so that indexes are >>>> created for each field. And also we can make it possible to configure the >>>> sorting mechanism per each activity. The user can be able to select >>>> whatever field like timestamp or hop_number as the sorting field. This >>>> would enable user to introduce their own field to order the activity >>>> results. WDYT? >>>> >>> >>> Multiple activity monitoring scenarios per each stream is already >>> supported in current implementation. Sorting and pagination are the missing >>> pieces which we have design properly in our next major release. And we >>> should also make the activity correlation field configurable via the >>> toolbox. >>> >> We should have a way to specify different fields for monitoring different >> activities and see them in the Activity Monitoring dashboard. For example >> if you want to monitor the flow of session IDs among streams and suppose at >> the same time you want to see the transaction ID flow between ESB mediation >> flows. You need two correlation ID fields and two indexes and should be >> able to see each scenario separately in the activity dashboard. There the >> session ID may be sorted with the "timestamp" but transaction flow should >> be sorted based on a constant field set by the BAM mediator, >> "mediator_number". The later one can be used in many situations where the >> message flow time difference between the two interception points is lesser >> than a 1 millisecond. (for e.g. in the same sequence where no significant >> operations are done to the message) >> >>> >>> >>>> >>>> *Maninda Edirisooriya* >>>> Senior Software Engineer >>>> >>>> *WSO2, Inc.*lean.enterprise.middleware. >>>> >>>> *Blog* : http://maninda.blogspot.com/ >>>> *E-mail* : [email protected] >>>> *Skype* : @manindae >>>> *Twitter* : @maninda >>>> >>> >>> >>> >>> -- >>> Regards, >>> >>> Inosh Goonewardena >>> Associate Technical Lead- WSO2 Inc. >>> Mobile: +94779966317 >>> >> >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > *Balakrishnan Gokulakrishnan* > Software Engineer, > WSO2, Inc. http://wso2.com > Mob: +94 77 593 5789 | +1 650 272 9927 >
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
