Re: [DISCUSS] [RIP] Message Track Trace

[dongeforever]

+1.

Nice RIP, I would like to offer a help.

Gosling Von  于2018年11月6日周二 下午5:41写道：

> +1
>
> Very cool, It seems to the fifth RIP from the community.
>
> Best Regards,
> Von Gosling
>
> 在 2018年11月6日，下午1:52，huzongt...@cmss.chinamobile.com 写道：
>
> FYI
>
>
> *发件人：* huzongt...@cmss.chinamobile.com
> *发送时间：* 2018-11-06 13:38
> *收件人：* users-subscribe ; commits
> ; dev-help 
> *主题：* About the RIP for new Feature：Message Track Trace
> RIP-1: Message Track Trace
>
> *1、Status*
>
>- *Current State: Proposed*
>- *Authors: HuzongTang*
>- *Shepherds: Zhendong*
>- Mailing List Discussion: 
>- Pull Request: #PR_NUMBER
>- Released: 
>
> *2、Background & Motivation*
>   What do we need to do
>   Message track trace refers to the consumption processing of a
> message which is sent from the producer instance has arrived the broker,and
> then has been consumed by the consumer instance.In the whole process, the
> information of time, location and other related services(including Client
> and Broker) is aggregated into the complete link information of message
> communication.
>   In MQ system, the complete link of a message contains three roles:
> producer, broker server and consumer.In the process of message
> communicaton, each role adds relevant information to the track trace
> link.By aggregating these information,it can provide some powerful support
> to user.
>  About the Apache RocketMQ project,I will add the feature of message
> track trace which can help users query every complete link data of a
> message exactly.This funcation is important to RocketMQ,especially in
> some  financial application fields.
>
> *3、Goals*
>
>- What problem is this proposal designed to solve?
>
> Reliability and Availability is the most two important characters for
> each MQ system.Although RocketMQ does very well in the two fields,it need
> some other methods to make sure the complete link of a message is no
> problem.By some ways,we should view the complete link of a messag and
> find the root cause of failure in process messaging quickly.
> So,if RocketMQ support the feature of Message Track Trace,we can find
> and analyse the root cause of failure in process messaging easily.And we
> can query many parameter values,such as sending cost time,consumption
> cost time,store time in broker and so on.The architecture of Message Track
> Trace will be introduced in below chapter.
>
> *4、Non-Goals*
>
>- What problem is this proposal NOT designed to solve?
>
>   Every users who build application by using RocketMQ may lack of
> a efficient way to  find and analyse the root cause of failure in process
> messaging easily and quickly.The users may spend much more time and enery
> on operation of RocketMQ in production enviroment.
>
>- Are there any limits of this proposal?
>
>  In order to reduce the impact of message entity content storage
> in RocketMQ，we redefine a special broker which stores every message track
> trace.So,if user don't want to use this funcation,he can close it in client
> by setting a flag.
>
> *5、Changes*
> We need add some codes in client and broker component which include
> collecting producer and consumer instances' infomation and redefine a
> special broker node.Read below sections to get more details about the
> Message Track Trace for RocketMQ.
>
> *6、Architecture*
> We plan to design the function of message track trace including its
> store part and client instance collecting.The total architecture has two
> part below:
> *（1）Client instance collects the infomation of message track trace*
>
> The design of architecture above as follows:
> *（a）*."producer and consumer multi-thread module, and Blocking
> Queue"：here, in client,as a producer model,we can collect the infomation(such
> as,sending and consuming message cost time,broker store time,broker stored
> ip address and so on) and put this information into the Blocking Queue.And
> as s consumer model,use a thread called "MQ-AsyncArrayDispatcher" to take
> the message track trace infomation from Blocking Queue.Then this asynchronous
> thread( called "MQ-AsyncArrayDispatcher") pack the message track trace
> element as AsyncAppenderRequest task and submit to the thead pool.
> Last,the main execution process of  AsyncAppenderRequest task is
> sending the message track trace infomation which is collected above from
> client side to a special broker node which is redefined in below chapter.
> *（b）*.define a new pair hook which is implementation of 
> the“SendMessageHook/ConsumeMessageHook”,from
> this,we can collect message track trace data  before and after publishing
> and subscribing messages.
>
> *（2）Redefine a special broker node to store message track trace data*
>
> As shown in the above picture,we can define a specail broker service
> node which can store the message track trace data in a common RocketMQ
> cluster.Here,we can add a 

[DISCUSS] [RIP] Message Track Trace

[Gosling Von]

+1

Very cool, It seems to the fifth RIP from the community.

Best Regards,
Von Gosling

> 在 2018年11月6日，下午1:52，huzongt...@cmss.chinamobile.com 写道：
> 
> FYI
>  
> 发件人： huzongt...@cmss.chinamobile.com 
> 发送时间： 2018-11-06 13:38
> 收件人： users-subscribe ; commits 
> ; dev-help 
> 
> 主题： About the RIP for new Feature：Message Track Trace
> RIP-1: Message Track Trace
> 
> 1、Status
> Current State: Proposed
> Authors: HuzongTang
> Shepherds: Zhendong
> Mailing List Discussion: 
> Pull Request: #PR_NUMBER
> Released: 
> 2、Background & Motivation
>   What do we need to do
>   Message track trace refers to the consumption processing of a message 
> which is sent from the producer instance has arrived the broker,and then has 
> been consumed by the consumer instance.In the whole process, the information 
> of time, location and other related services(including Client and Broker) is 
> aggregated into the complete link information of message communication.
>   In MQ system, the complete link of a message contains three roles: 
> producer, broker server and consumer.In the process of message communicaton, 
> each role adds relevant information to the track trace link.By aggregating 
> these information,it can provide some powerful support to user.
>  About the Apache RocketMQ project,I will add the feature of message 
> track trace which can help users query every complete link data of a message 
> exactly.This funcation is important to RocketMQ,especially in some  financial 
> application fields.
> 
> 3、Goals
> What problem is this proposal designed to solve?
> Reliability and Availability is the most two important characters for 
> each MQ system.Although RocketMQ does very well in the two fields,it need 
> some other methods to make sure the complete link of a message is no 
> problem.By some ways,we should view the complete link of a messag and find 
> the root cause of failure in process messaging quickly.
> So,if RocketMQ support the feature of Message Track Trace,we can find and 
> analyse the root cause of failure in process messaging easily.And we can 
> query many parameter values,such as sending cost time,consumption cost 
> time,store time in broker and so on.The architecture of Message Track Trace 
> will be introduced in below chapter.
> 
> 4、Non-Goals
> What problem is this proposal NOT designed to solve?
>   Every users who build application by using RocketMQ may lack of a 
> efficient way to  find and analyse the root cause of failure in process 
> messaging easily and quickly.The users may spend much more time and enery on 
> operation of RocketMQ in production enviroment.
> Are there any limits of this proposal?  
>  In order to reduce the impact of message entity content storage in 
> RocketMQ，we redefine a special broker which stores every message track 
> trace.So,if user don't want to use this funcation,he can close it in client 
> by setting a flag.
> 
> 5、Changes
> We need add some codes in client and broker component which include 
> collecting producer and consumer instances' infomation and redefine a special 
> broker node.Read below sections to get more details about the Message Track 
> Trace for RocketMQ.
> 
> 6、Architecture
> We plan to design the function of message track trace including its store 
> part and client instance collecting.The total architecture has two part below:
> （1）Client instance collects the infomation of message track trace
> 
> The design of architecture above as follows:
> （a）."producer and consumer multi-thread module, and Blocking Queue"：here, 
> in client,as a producer model,we can collect the infomation(such as,sending 
> and consuming message cost time,broker store time,broker stored ip address 
> and so on) and put this information into the Blocking Queue.And as s consumer 
> model,use a thread called "MQ-AsyncArrayDispatcher" to take the message track 
> trace infomation from Blocking Queue.Then this asynchronous thread( called 
> "MQ-AsyncArrayDispatcher") pack the message track trace element as 
> AsyncAppenderRequest task and submit to the thead pool.
> Last,the main execution process of  AsyncAppenderRequest task is sending 
> the message track trace infomation which is collected above from client side 
> to a special broker node which is redefined in below chapter.
> （b）.define a new pair hook which is implementation of 
> the“SendMessageHook/ConsumeMessageHook”,from this,we can collect message 
> track trace data  before and after publishing and subscribing messages.
> 
> （2）Redefine a special broker node to store message track trace data
> 
> As shown in the above picture,we can define a specail broker service node 
> which can store the message track trace data in a