+1.
Nice RIP, I would like to offer a help.
Gosling Von <fengji...@gmail.com> 于2018年11月6日周二 下午5:41写道：
> +1
>
> Very cool, It seems to the fifth RIP from the community.
>
> Best Regards,
> Von Gosling
>
> 在 2018年11月6日，下午1:52，huzongt...@cmss.chinamobile.com 写道：
>
> FYI
>
>
> *发件人：* huzongt...@cmss.chinamobile.com
> *发送时间：* 2018-11-06 13:38
> *收件人：* users-subscribe <users-subscr...@rocketmq.apache.org>; commits
> <comm...@rocketmq.apache.org>; dev-help <dev-h...@rocketmq.apache.org>
> *主题：* About the RIP for new Feature：Message Track Trace
> RIP-1: Message Track Trace
>
> *1、Status*
>
> - * Current State: Proposed*
> - * Authors: HuzongTang*
> - * Shepherds: Zhendong*
> - Mailing List Discussion: <apache mailing list archive>
> - Pull Request: #PR_NUMBER
> - Released: <released_version>
>
> *2、Background & Motivation*
> What do we need to do
> Message track trace refers to the consumption processing of a
> message which is sent from the producer instance has arrived the broker,and
> then has been consumed by the consumer instance.In the whole process, the
> information of time, location and other related services(including Client
> and Broker) is aggregated into the complete link information of message
> communication.
> In MQ system, the complete link of a message contains three roles:
> producer, broker server and consumer.In the process of message
> communicaton, each role adds relevant information to the track trace
> link.By aggregating these information,it can provide some powerful support
> to user.
> About the Apache RocketMQ project,I will add the feature of message
> track trace which can help users query every complete link data of a
> message exactly.This funcation is important to RocketMQ,especially in
> some financial application fields.
>
> *3、Goals*
>
> - What problem is this proposal designed to solve?
>
> Reliability and Availability is the most two important characters for
> each MQ system.Although RocketMQ does very well in the two fields,it need
> some other methods to make sure the complete link of a message is no
> problem.By some ways,we should view the complete link of a messag and
> find the root cause of failure in process messaging quickly.
> So,if RocketMQ support the feature of Message Track Trace,we can find
> and analyse the root cause of failure in process messaging easily.And we
> can query many parameter values,such as sending cost time,consumption
> cost time,store time in broker and so on.The architecture of Message Track
> Trace will be introduced in below chapter.
>
> *4、Non-Goals*
>
> - What problem is this proposal NOT designed to solve?
>
> Every users who build application by using RocketMQ may lack of
> a efficient way to find and analyse the root cause of failure in process
> messaging easily and quickly.The users may spend much more time and enery
> on operation of RocketMQ in production enviroment.
>
> - Are there any limits of this proposal?
>
> In order to reduce the impact of message entity content storage
> in RocketMQ，we redefine a special broker which stores every message track
> trace.So,if user don't want to use this funcation,he can close it in client
> by setting a flag.
>
> *5、Changes*
> We need add some codes in client and broker component which include
> collecting producer and consumer instances' infomation and redefine a
> special broker node.Read below sections to get more details about the
> Message Track Trace for RocketMQ.
>
> *6、Architecture*
> We plan to design the function of message track trace including its
> store part and client instance collecting.The total architecture has two
> part below:
> * （1）Client instance collects the infomation of message track trace*
>
> The design of architecture above as follows:
> *（a）*."producer and consumer multi-thread module, and Blocking
> Queue"：here, in client,as a producer model,we can collect the infomation(such
> as,sending and consuming message cost time,broker store time,broker stored
> ip address and so on) and put this information into the Blocking Queue.And
> as s consumer model,use a thread called "MQ-AsyncArrayDispatcher" to take
> the message track trace infomation from Blocking Queue.Then this asynchronous
> thread( called "MQ-AsyncArrayDispatcher") pack the message track trace
> element as AsyncAppenderRequest task and submit to the thead pool.
> Last,the main execution process of AsyncAppenderRequest task is
> sending the message track trace infomation which is collected above from
> client side to a special broker node which is redefined in below chapter.
> *（b）*.define a new pair hook which is implementation of
> the“SendMessageHook/ConsumeMessageHook”,from
> this,we can collect message track trace data before and after publishing
> and subscribing messages.
>
> * （2）Redefine a special broker node to store message track trace data*
>
> As shown in the above picture,we can define a specail broker service
> node which can store the message track trace data in a common RocketMQ
> cluster.Here,we can add a flag(such as autoTraceBrokerEnable) in
> broker.properties file and use this variable to define Whether this
> broker is a specail node for storing message track trace data.
> *（a）.*autoTraceBrokerEnable is false.This indicates this broker is
> an ordinary node,then "Trace_Topic" will not be created in this node.And
> it will still follow the original processing flow without any change.
> *（b）.*autoTraceBrokerEnable is true.This indicates broker is an
> special node,which stores the message track trace data specailly.And The
> "Trace_Topic" is automatically created during the Broker's startup phase,this
> node automatically registers its owned set of topics in memory to
> nameserver(including Trace_Topic).Thus,in a RocketMQ cluster,there is
> only a specail broker node which stores message track trace datas.And
> clients(including publishing and subscribing messages) knows which broker
> node to send message trace data after they try fetch topic routing info
> from nameserver.
>
> *（3）How to solve the query message track trace data by original topic*
> For example,Topic which saves message track trace data is
> called "RMQ_SYS_TRACE_DATA_XXX" is instead of Topic of original
> data.But，message
> query(By messageId + topic,topic+ key or topic)
> which still uses original data on RocketMQ console side can't query the
> expected result for us.
> Here, we can fill in the keyset field of the message track trace data by
> using msgId (not offset MsgId) or Key of the original message content when
> send the message track trace data collected by the client, so the IndexFile
> index of the Broker end can be built according to the msgId or Key of the
> original message.
> And at the broker side,we can
> invokes the queryMessage() method of the storage part through the
> QueryMessageProcessor
> Business processor.
>
> * （4）Finishing querying messages track trace datas in console*
> Fisrtly,the console project is based on spring boot technology,so we
> can finish querying messages track track datas in this project.The basic
> design is using thread pool to send a RPC request to the redefined broker
> node to get the message track trace data.
>
> *7、Interface Design/Change*
> Here I just list part interface desgin and change in Client Side.And
> the chang parts of Broker and Console side leave out.
> * （1）The data transfering asynchronously interface is as followers: *
> public interface AsyncDispatcher {
>
> void start() throws MQClientException;
>
> boolean append(Object ctx);
>
> void flush() throws IOException;
>
> void shutdown();
> }
>
> （2）Defining the two Hooks which is implemented of ConsumeMessageHook
> and SendMessageHook are as followers:
> a.ClientSendMessageTraceHookImpl
> b.ClientConsumeMessageTraceHookImpl
>
> * （3）We define and write some codes for common data model which are as
> followers:*
> a.TraceBean
> b.TraceConstants
> c.TraceContext
> d.TraceDataEncoder
> e.TraceDispatcherType
> f.TraceTransferBean
> g.TuxeTraceType
>
> *8.Compatibility, Deprecation, and Migration Plan*
> （1）Are backward and forward compatibility taken into consideration?
> No issues;
> （2）Are there deprecated APIs?
> No issues;
> （3）How do we do migration?
> No issues;
>
>
> *9.Implementation Outline* We will implement the proposed changes by
> 1 phase. All of implementation will be finished in 1 phase.
>
> ------------------------------
> huzongt...@cmss.chinamobile.com
>
>
>
