Hi All, This is an open discussion about message broker analytics for the carbon 5 based message broker release and explain design of analytics story.
*Existing design* Message broker 3.0.0 and 3.1.0 has couple of stats publishing using carbon-metrics library. These stats can be found in Management console -> Monitor -> Messaging Metrics. Below are those stats. - Number of messages in inbound/outbound disruptor - Total publisher/subscribers/channels - Received and sent message rates - DB read/write time and rate These information are specific to node itself. If I explain it more, let's say there are two nodes in the cluster, user has to log into each node and look into details of that particular node. There is no aggregated view or navigation. *Propose design* Message broker analytic is going to handle as separate distribution (analytics-mb with necessary WSO2 DAS features) according to the platform story. Dash broad organized with different filters such as cluster, nodes, queues, topics and time range. Below general view going to be present in each filter level. Ex: total connections of cluster/ total connection of given node/ total connection of given queue or topic etc. - Total connections - Total messages - Message size distribution - Total publishers/subscribers - Enqueue messages - Dequeue messages - Total Message rejection/ Messages moved to DLC (error rates can be calculate) - Inbound/outbound throughput - Inbound/outbound latency - DB read/write time and rate Our plan is to trigger alerts based on these data with WSO2 CEP and ML in later phase. We are planing to use same carbon-metrics library to publish data to analytics-mb. DAS data publisher capability is added to carbon-metrics 2.0.0. But there is limitation identified in carbon-metrics 2.0.0. It can use to publish data with exposed API such as meter, counter, timer, histogram and gauges. These data always distinguished by the name field. public static Meter meter(String name, Level level, Level... levels) public static Counter counter(String name, Level level, Level... levels) public static Timer timer(String name, Level level) public static Histogram histogram(String name, Level level, Level... levels) public static <T> void gauge(String name, Level level, Gauge<T> gauge) But for us, it is essential to have more than one field to query different filters in DAS. For an example: query total connections of cluster/ query total connection of given node/ query total connections for given queue/topic. We should always need to pass node and queue/topic details when publishing data to DAS which make filtering more easier. We would like to suggest to do necessary changes in carbon-metric 2.0.0 implementation to cater for above requirement as well. Your thoughts are welcome. Cheers! -- Indika Sampath Senior Software Engineer WSO2 Inc. http://wso2.com Phone: +94 716 424 744 Blog: http://indikasampath.blogspot.com/
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
