[Adding missing images] Hi All.
We had an internal review on the retry mechanism design and Please find below for the points discussed during the review. - If we enable guaranteed delivery for the message producer, a separate failover message store and a message processor[1] have been created initially.Failover message store could be any store type which available in the ESB. - Messages which rollback due to communication problem or message store not available will direct to the failover message store. - Separate message processor initialize for the each failover message store and execute continuously for process the failover messages. We planned to use existing forwardingProcessor logic[2] for this purpose hence same retry mechanism will work. [1] [2] Thanks. On Thu, Jul 9, 2015 at 12:11 AM, Prabath Ariyarathna <[email protected]> wrote: > Hi All. > > We had an internal review on the retry mechanism design and Please find > below for the points discussed during the review. > > > - If we enable guaranteed delivery for the message producer, a > separate failover message store and a message processor[1] have been > created initially.Failover message store could be any store type which > available in the ESB. > > > > - Messages which rollback due to communication problem or message > store not available will direct to the failover message store. > > > > - Separate message processor initialize for the each failover message > store and execute continuously for process the failover messages. We > planned to use existing forwardingProcessor logic[2] for this purpose hence > same retry mechanism will work. > > > [1] > > > > > [2] > > > Thanks. > > > On Wed, Jul 8, 2015 at 11:30 AM, Shafreen Anfar <[email protected]> wrote: > >> Hi Prabath, >> >> On Wed, Jul 8, 2015 at 8:49 AM, Prabath Ariyarathna <[email protected]> >> wrote: >> >>> >>> >>> On Wed, Jul 8, 2015 at 11:15 AM, Malaka Silva <[email protected]> wrote: >>> >>>> IMO this should be in memory by default and should provide some other >>>> ways like JDBC if they require. WDYT? >>>> >>> >>> +1 We can start with the in-memory and later we can go for the other >>> permenent persistent options. >>> >> >> IMO, doing it in-memory will not be useful in production environments. >> The reason is when we store the message in-memory you have to store the >> whole message context. This will lead to a memory growth. IMO, Storing has >> to be done in some external entity. >> >> >>> >>>> On Wed, Jul 8, 2015 at 11:04 AM, Shafreen Anfar <[email protected]> >>>> wrote: >>>> >>>>> Hi Prabath, >>>>> >>>>> On Wed, Jul 8, 2015 at 8:24 AM, Prabath Ariyarathna < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi Malaka. >>>>>> >>>>>> Agreed on your thought. Then we need to store all the rollback >>>>>> messages and process separately by different thread same as message >>>>>> processor doing. It will change the behavior of store mediator if >>>>>> rollback >>>>>> has happened. >>>>>> >>>>> >>>>> Where are we going to store it ? Is it in an in-memory queue ? >>>>> >>>>> >>>>>> >>>>>> >>>>>> Thanks. >>>>>> >>>>>> On Wed, Jul 8, 2015 at 10:31 AM, Malaka Silva <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi Prabath, >>>>>>> >>>>>>> I don't think it's a good idea since it can cause threads to be >>>>>>> blocked when the broker is down? >>>>>>> >>>>>>> On Wed, Jul 8, 2015 at 10:25 AM, Prabath Ariyarathna < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hi Shafreen. >>>>>>>> >>>>>>>> We planned to use same thread for the retry since store mediator >>>>>>>> working in synchronous mode. >>>>>>>> >>>>>>>> >>>>>>>> Thanks. >>>>>>>> >>>>>>>> On Wed, Jul 8, 2015 at 9:19 AM, Shafreen Anfar <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi Prabath, >>>>>>>>> >>>>>>>>> >>>>>>>>> On Tue, Jul 7, 2015 at 6:21 PM, Prabath Ariyarathna < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi All, >>>>>>>>>> >>>>>>>>>> We are planning to implement guaranteed message delivery >>>>>>>>>> capability to the MSMP. We can divided current implementation into >>>>>>>>>> two >>>>>>>>>> basic sections based on process. >>>>>>>>>> >>>>>>>>>> 1. *Message Store:* Message store mainly responsible for >>>>>>>>>> producing messages to the message broker using message store >>>>>>>>>> mediator. >>>>>>>>>> 2. *Message process*: Message processor picks the messages >>>>>>>>>> from the message store and send to the specified endpoint. >>>>>>>>>> >>>>>>>>>> When we implement a guaranteed message delivery mechanism to the >>>>>>>>>> MSMP, we need to think about both of above scenarios. JMS >>>>>>>>>> specification >>>>>>>>>> provided us to two mechanisms(Acknowledgement, Transaction), >>>>>>>>>> which can be used to achieve guaranteed delivery. >>>>>>>>>> In our case message processor(consumer) side guaranteed delivery >>>>>>>>>> was already implemented using acknowledgements. Message processor >>>>>>>>>> ACK to >>>>>>>>>> the message store only if message was successfully processed. >>>>>>>>>> Message store removes the message once ACK received by the >>>>>>>>>> processor else >>>>>>>>>> redeliver the message, but for the message storing(producer) part, >>>>>>>>>> we don’t >>>>>>>>>> have any mechanism implemented to achieve this feature. Main >>>>>>>>>> intention of this implementation is to provide a way to handle >>>>>>>>>> guaranteed >>>>>>>>>> delivery for the message storing(producer) part. >>>>>>>>>> >>>>>>>>>> *Problem.* >>>>>>>>>> We don’t have mechanism to handle guaranteed delivery in message >>>>>>>>>> storing(producer) section. This issue can be occurred for following >>>>>>>>>> use >>>>>>>>>> cases. >>>>>>>>>> >>>>>>>>>> 1. Message was sent to the MB, but didn’t received to the MB >>>>>>>>>> side due to communication failure. >>>>>>>>>> 2. Message broker not available when try to store the message. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> *Solution.* >>>>>>>>>> We can choose either acknowledgement or transaction as a solution >>>>>>>>>> of guaranteed delivery implementation but since JMS 1.1 doesn't >>>>>>>>>> support for >>>>>>>>>> the producer acknowledgement, we decided to enable transaction >>>>>>>>>> support for >>>>>>>>>> the message storing section to provide guaranteed delivery. Other >>>>>>>>>> than the >>>>>>>>>> transactions enable in the message storing process, we planned to >>>>>>>>>> add retry >>>>>>>>>> mechanism to avoid scenario like message store is not available when >>>>>>>>>> we try >>>>>>>>>> to produce messages. User can define retry count and retry delay in >>>>>>>>>> the >>>>>>>>>> configuration of the store mediator. Message retry process is >>>>>>>>>> executing If >>>>>>>>>> message broker connection not available or transaction has already >>>>>>>>>> rollbacked. In addition to that features we are planning to add batch >>>>>>>>>> process facility to store mediator to improve performance, while >>>>>>>>>> enabling >>>>>>>>>> transactions as the future improvement. >>>>>>>>>> >>>>>>>>> >>>>>>>>> How do you really going to implement the retry feature ? Will the >>>>>>>>> retry happen using the same thread or will you be creating a separate >>>>>>>>> thread for it ? >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> -- >>>>>>>>>> >>>>>>>>>> *Prabath Ariyarathna* >>>>>>>>>> >>>>>>>>>> *Associate Technical Lead* >>>>>>>>>> >>>>>>>>>> *WSO2, Inc. * >>>>>>>>>> >>>>>>>>>> *lean . enterprise . middleware * >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> *Email: [email protected] <[email protected]>* >>>>>>>>>> >>>>>>>>>> *Blog: http://prabu-lk.blogspot.com >>>>>>>>>> <http://prabu-lk.blogspot.com>* >>>>>>>>>> >>>>>>>>>> *Flicker : https://www.flickr.com/photos/47759189@N08 >>>>>>>>>> <https://www.flickr.com/photos/47759189@N08>* >>>>>>>>>> >>>>>>>>>> *Mobile: +94 77 699 4730 * >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Regards, >>>>>>>>> *Shafreen* >>>>>>>>> Software Engineer >>>>>>>>> WSO2 Inc >>>>>>>>> Mobile : 077-556-395-1 >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> *Prabath Ariyarathna* >>>>>>>> >>>>>>>> *Associate Technical Lead* >>>>>>>> >>>>>>>> *WSO2, Inc. * >>>>>>>> >>>>>>>> *lean . enterprise . middleware * >>>>>>>> >>>>>>>> >>>>>>>> *Email: [email protected] <[email protected]>* >>>>>>>> >>>>>>>> *Blog: http://prabu-lk.blogspot.com <http://prabu-lk.blogspot.com>* >>>>>>>> >>>>>>>> *Flicker : https://www.flickr.com/photos/47759189@N08 >>>>>>>> <https://www.flickr.com/photos/47759189@N08>* >>>>>>>> >>>>>>>> *Mobile: +94 77 699 4730 * >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Architecture mailing list >>>>>>>> [email protected] >>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> Best Regards, >>>>>>> >>>>>>> Malaka Silva >>>>>>> Senior Tech Lead >>>>>>> M: +94 777 219 791 >>>>>>> Tel : 94 11 214 5345 >>>>>>> Fax :94 11 2145300 >>>>>>> Skype : malaka.sampath.silva >>>>>>> LinkedIn : http://www.linkedin.com/pub/malaka-silva/6/33/77 >>>>>>> Blog : http://mrmalakasilva.blogspot.com/ >>>>>>> >>>>>>> WSO2, Inc. >>>>>>> lean . enterprise . middleware >>>>>>> http://www.wso2.com/ >>>>>>> http://www.wso2.com/about/team/malaka-silva/ >>>>>>> <http://wso2.com/about/team/malaka-silva/> >>>>>>> >>>>>>> Save a tree -Conserve nature & Save the world for your future. Print >>>>>>> this email only if it is absolutely necessary. >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Architecture mailing list >>>>>>> [email protected] >>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> *Prabath Ariyarathna* >>>>>> >>>>>> *Associate Technical Lead* >>>>>> >>>>>> *WSO2, Inc. * >>>>>> >>>>>> *lean . enterprise . middleware * >>>>>> >>>>>> >>>>>> *Email: [email protected] <[email protected]>* >>>>>> >>>>>> *Blog: http://prabu-lk.blogspot.com <http://prabu-lk.blogspot.com>* >>>>>> >>>>>> *Flicker : https://www.flickr.com/photos/47759189@N08 >>>>>> <https://www.flickr.com/photos/47759189@N08>* >>>>>> >>>>>> *Mobile: +94 77 699 4730 * >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Architecture mailing list >>>>>> [email protected] >>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Regards, >>>>> *Shafreen* >>>>> Software Engineer >>>>> WSO2 Inc >>>>> Mobile : 077-556-395-1 >>>>> >>>> >>>> >>>> >>>> -- >>>> >>>> Best Regards, >>>> >>>> Malaka Silva >>>> Senior Tech Lead >>>> M: +94 777 219 791 >>>> Tel : 94 11 214 5345 >>>> Fax :94 11 2145300 >>>> Skype : malaka.sampath.silva >>>> LinkedIn : http://www.linkedin.com/pub/malaka-silva/6/33/77 >>>> Blog : http://mrmalakasilva.blogspot.com/ >>>> >>>> WSO2, Inc. >>>> lean . enterprise . middleware >>>> http://www.wso2.com/ >>>> http://www.wso2.com/about/team/malaka-silva/ >>>> <http://wso2.com/about/team/malaka-silva/> >>>> >>>> Save a tree -Conserve nature & Save the world for your future. Print >>>> this email only if it is absolutely necessary. >>>> >>> >>> >>> >>> -- >>> >>> *Prabath Ariyarathna* >>> >>> *Associate Technical Lead* >>> >>> *WSO2, Inc. * >>> >>> *lean . enterprise . middleware * >>> >>> >>> *Email: [email protected] <[email protected]>* >>> >>> *Blog: http://prabu-lk.blogspot.com <http://prabu-lk.blogspot.com>* >>> >>> *Flicker : https://www.flickr.com/photos/47759189@N08 >>> <https://www.flickr.com/photos/47759189@N08>* >>> >>> *Mobile: +94 77 699 4730 * >>> >>> >>> >>> >>> >>> >> >> >> -- >> Regards, >> *Shafreen* >> Software Engineer >> WSO2 Inc >> Mobile : 077-556-395-1 >> > > > > -- > > *Prabath Ariyarathna* > > *Associate Technical Lead* > > *WSO2, Inc. * > > *lean . enterprise . middleware * > > > *Email: [email protected] <[email protected]>* > > *Blog: http://prabu-lk.blogspot.com <http://prabu-lk.blogspot.com>* > > *Flicker : https://www.flickr.com/photos/47759189@N08 > <https://www.flickr.com/photos/47759189@N08>* > > *Mobile: +94 77 699 4730 * > > > > > > -- *Prabath Ariyarathna* *Associate Technical Lead* *WSO2, Inc. * *lean . enterprise . middleware * *Email: [email protected] <[email protected]>* *Blog: http://prabu-lk.blogspot.com <http://prabu-lk.blogspot.com>* *Flicker : https://www.flickr.com/photos/47759189@N08 <https://www.flickr.com/photos/47759189@N08>* *Mobile: +94 77 699 4730 *
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
