I just did a quick search about the Akka persistent[1], it looks like we could leverage it to implement the event source without WAL.
I agree we could use Akka cluster to provide the HA solution for the Alpha, but we need to do some POC for it. [1]https://doc.akka.io/docs/akka/current/persistence.html Willem Jiang Twitter: willemjiang Weibo: 姜宁willem On Fri, Jun 28, 2019 at 7:49 PM Zheng Feng <zh.f...@gmail.com> wrote: > > Thanks Zhang Lei, > > Zhang Lei <zhang_...@boco.com.cn> 于2019年6月28日周五 下午5:50写道： > > > Hi, All > > > > alpha-fsm has been pushed to the branch SCB-1321 > > > > Completed: > > 1. State machine design document[1] > > 2. State machine prototype > > 3. State machine test case > > 4. Receive saga events using the internal message bus > > > > Key emphasis of next stage in work: > > In order to carry out the feasibility verification as soon as possible, I > > will not consider the reliability issue for the time being. > > 1. Refactor Omega components, add SagaAbortedEvent, SagaTimeoutEvent, > > TxComponsitedEvent > > 2. Save compensation method parameters in Actor and trigger compensation > > in Actor > > 3. Do not use Kafka and only verify single node alpha, The Alpha server > > receives the saga event and puts it into the internal message bus. > > > It looks good to me ! > > > > > Planning: > > 1. Persist actor data to the database when it terminates > > > What are the actor data ? they are all the events ? > > > 2. Integration Kafka > > > So we can use the Kafka as a message broker and the invokings between the > Omega and the Alpha will become async ? > > > 3. Support WAL[2] recovery mode > > > Well, it looks interesting and does it support by the akka ? > > > 4. Verify Akka cluster reliability > > > > [1] > > https://github.com/apache/servicecomb-pack/tree/SCB-1321/alpha/alpha-fsm < > > https://github.com/apache/servicecomb-pack/tree/SCB-1321/alpha/alpha-fsm> > > [2] https://en.wikipedia.org/wiki/Write-ahead_logging < > > https://en.wikipedia.org/wiki/Write-ahead_logging> > > > > if you have other comments, please let us know. > > > Good luck ! > > > > > Thanks, > > Lei Zhang > > > > > 在 2019年6月27日，上午9:50，Willem Jiang <willem.ji...@gmail.com> 写道： > > > > > > We just leverage the message broker to make sure Alpha get the > > > transaction event from Omega. > > > In most cases Alpha don't need to talk back to Omega, we just need to > > > make sure all the transaction message are stored (Alpha can process it > > > later). > > > > > > If Omega cannot talk the message broker, Omega should abort the > > > transaction processing with transport exception. > > > > > > Willem Jiang > > > > > > Twitter: willemjiang > > > Weibo: 姜宁willem > > > > > > On Tue, Jun 25, 2019 at 8:42 AM Zhang Lei <zhang_...@boco.com.cn> wrote: > > >> > > >> Hi, Zhang jun > > >> > > >>> I just cared about the recovery scan thread design. > > >>> Kafka can ensure event message can be consumed by alpha exactly, but > > recovery need know all the participated transaction response to decide > > rollback or commit, so I think scan thread is also necessary. > > >> > > >> I am not sure, but I think Akka's persistence can solve this problem > > you care about. > > >> Of course, this ability needs to be verified > > >> > > >> Thanks, > > >> Zhang Lei > > >> > > >>> 在 2019年6月24日，上午10:46，赵俊 <zhaoju...@jd.com> 写道： > > >>> > > >>> Hi, Zhang Lei > > >>> > > >>>> A2 : I think we only need to ensure that the message can be reliably > > delivered to the state machine, The state machine is only a synchronous > > record state transition when the transaction is executed normally. At > > present, the compensation method based on table scan is also asynchronous. > > I am not sure if I have answered your question, or you can give me more > > information. > > >>> > > >>> If we have a mechanism that ensure main service can collect all the > > participated transaction response from alpha correctly before > > commit/rollback, it is OK. > > >>> > > >>>> Q2 : Also we should consider about recovery, it seems that recovery > > is as same as before based on database. > > >>>> A2 : I think the question you care about is how to recover when the > > alpha is down, this is a little different from the current version. > > >>>> 1. We can base on Kafka's reliability and control the offset of the > > topic, one message at a time > > >>>> 2. Of course, we can also do some extra design for it, such as > > logging the data log file locally after receiving the Kafka message. Resume > > the message by reading the data log file when the alpha machine restarts > > >>> > > >>> I just cared about the recovery scan thread design. > > >>> Kafka can ensure event message can be consumed by alpha exactly, but > > recovery need know all the participated transaction response to decide > > rollback or commit, so I think scan thread is also necessary. > > >>> > > >>> > > >>> > > >>>> On Jun 23, 2019, at 1:04 PM, Zhang Lei <zhang_...@boco.com.cn> wrote: > > >>>> > > >>>> Hi, Zhao Jun > > >>>> > > >>>> Thank you for your reply！ > > >>>> > > >>>> This design document does not elaborate on reliability aspects. > > >>>> > > >>>> My initial thought is this > > >>>> > > >>>> Q1 : It seems that omega should hold on after consuming the event > > message from Kafka instead of completing pushing message > > >>>> A2 : I think we only need to ensure that the message can be reliably > > delivered to the state machine, The state machine is only a synchronous > > record state transition when the transaction is executed normally. At > > present, the compensation method based on table scan is also asynchronous. > > I am not sure if I have answered your question, or you can give me more > > information. > > >>>> > > >>>> Q2 : Also we should consider about recovery, it seems that recovery > > is as same as before based on database. > > >>>> A2 : I think the question you care about is how to recover when the > > alpha is down, this is a little different from the current version. > > >>>> 1. We can base on Kafka's reliability and control the offset of the > > topic, one message at a time > > >>>> 2. Of course, we can also do some extra design for it, such as > > logging the data log file locally after receiving the Kafka message. Resume > > the message by reading the data log file when the alpha machine restarts > > >>>> > > >>>> > > >>>> Thanks, > > >>>> Lei Zhang > > >>>> > > >>>>> 在 2019年6月23日，上午7:08，zhaojun <zhaoju...@126.com> 写道： > > >>>>> > > >>>>> I have some questions about the design. > > >>>>> 1. It seems that omega should hold on after consuming the event > > message from Kafka instead of completing pushing message. > > >>>>> 2. Also we should consider about recovery, it seems that recovery is > > as same as before based on database. > > >>>>> > > >>>>> ------------------ > > >>>>> Zhao Jun > > >>>>> Apache Sharding-Sphere & ServiceComb > > >>>>> > > >>>>>> On Jun 21, 2019, at 6:41 PM, Zhang Lei <zhang_...@boco.com.cn> > > wrote: > > >>>>>> > > >>>>>> Hi, > > >>>>>> > > >>>>>> I have created the alpha-fsm module on branch SCB-1321 and > > submitted the design documentation, state machine prototype and test cases. > > >>>>>> If there is any problem, please let me know. > > >>>>>> > > >>>>>> Thanks, > > >>>>>> Lei Zhang > > >>>>>> > > >>>>>> [1] > > https://github.com/apache/servicecomb-pack/tree/SCB-1321/alpha/alpha-fsm < > > https://github.com/apache/servicecomb-pack/tree/SCB-1321/alpha/alpha-fsm> > > >>>>>> > > >>>>>>> 在 2019年6月20日，下午3:25，Zheng Feng <zh.f...@gmail.com> 写道： > > >>>>>>> > > >>>>>>> Yeah, I think Willem has create one [1] before and do you mind I > > assign > > >>>>>>> this issue to you ? > > >>>>>>> > > >>>>>>> [1] https://issues.apache.org/jira/browse/SCB-1258 > > >>>>>>> > > >>>>>>> Zhang Lei <zhang_...@boco.com.cn> 于2019年6月20日周四 下午2:34写道： > > >>>>>>> > > >>>>>>>> Hi, Zheng Feng > > >>>>>>>> > > >>>>>>>> Thanks for your advice, I will create a JIRA first and start with > > the > > >>>>>>>> design documentation. > > >>>>>>>> > > >>>>>>>> Lei Zhang > > >>>>>>>> > > >>>>>>>>> 在 2019年6月19日，下午8:09，Zheng Feng <zh.f...@gmail.com> 写道： > > >>>>>>>>> > > >>>>>>>>> Thanks a lot for sharing these information ! I think this state > > machine > > >>>>>>>>> could be very experimental so it would helpful to create an > > experimental > > >>>>>>>>> branch to add this module but not in the master branch. > > >>>>>>>>> > > >>>>>>>>> Zhang Lei <cool...@qq.com> 于2019年6月19日周三 下午5:42写道： > > >>>>>>>>> > > >>>>>>>>>> I have completed some of the design and prototype in my github. > > >>>>>>>>>> > > >>>>>>>>>> In the design document [1] my original idea was that a > > transaction > > >>>>>>>>>> consisted of a SagaActor and several TxActors, and later > > TxAcotr was > > >>>>>>>>>> removed to reduce implementation complexity. > > >>>>>>>>>> I haven't had time to modify the documentation yet, but the > > SagaActor > > >>>>>>>>>> state machine [2] is up to date. > > >>>>>>>>>> Here you can see the test cases of SagaActor [3] > > >>>>>>>>>> > > >>>>>>>>>> [1] > > >>>>>>>>>> > > >>>>>>>> > > https://github.com/coolbeevip/playground/tree/master/state_machine_demo/saga-akkafsm > > >>>>>>>>>> < > > >>>>>>>>>> > > >>>>>>>> > > https://github.com/coolbeevip/playground/tree/master/state_machine_demo/saga-akkafsm > > >>>>>>>>>>> > > >>>>>>>>>> [2] > > >>>>>>>>>> > > >>>>>>>> > > https://github.com/coolbeevip/playground/blob/master/state_machine_demo/saga-akkafsm/assets/saga_state_diagram.png > > >>>>>>>>>> < > > >>>>>>>>>> > > >>>>>>>> > > https://github.com/coolbeevip/playground/blob/master/state_machine_demo/saga-akkafsm/assets/saga_state_diagram.png > > >>>>>>>>>>> > > >>>>>>>>>> [3] > > >>>>>>>>>> > > >>>>>>>> > > https://github.com/coolbeevip/playground/blob/master/state_machine_demo/saga-akkafsm/src/test/java/coolbeevip/playgroud/statemachine/saga/SagaActorTest.java > > >>>>>>>>>> < > > >>>>>>>>>> > > >>>>>>>> > > https://github.com/coolbeevip/playground/blob/master/state_machine_demo/saga-akkafsm/src/test/java/coolbeevip/playgroud/statemachine/saga/SagaActorTest.java > > >>>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> Lei Zhang > > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>>> 在 2019年6月19日，下午2:34，zhaojun <zhaoju...@126.com> 写道： > > >>>>>>>>>>> > > >>>>>>>>>>> If we use AKKA, how can we design the actors, and how can we > > guarantee > > >>>>>>>>>> omega will receive the message synchronize. > > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>> > > >>>>>>>> > > >>>>>> > > >>>>> > > >>> > > >> > > > >
