Ok, I think we all agree we need some kind of transactional behaviour for DB and events (on, in other words, guarantee that events are not lost). Maybe I'm going too fast, but I would like to discuss which are the possible approaches and prioritize them. But, yes, we need to do that Thanks for the clarification.
On Fri, Aug 2, 2024 at 12:25 PM Enrique Gonzalez Martinez < elguard...@gmail.com> wrote: > Hi francisco, > > Please go back to the proposal, I am not setting any custom solutions > just mention in the risk assessment that there are two possible ways > to do this. one with transaction sync and another one with xa > resource. > debezium is ok but still requires transactions and that solution is > based on xa resource as you are using a database to queue the event > that will be sent through kafka. > > Hope it helps. This proposal is about adding transactions not about > how to deal with all possible cases that will come from that. The risk > assessment only identifies the problems and possible solutions. > > Cheers :) > > El vie, 2 ago 2024 a las 12:13, Francisco Javier Tirado Sarti > (<ftira...@redhat.com>) escribió: > > > > Sorry to be a PITA and somehow diminish the apparent overall enthusiasm > in > > most of the community, but should we not discuss what we are going to > > really do before voting? > > If for transactions we mean XA transaction, I have to vote -1 till we > > discuss other options. > > > > On Fri, Aug 2, 2024 at 12:05 PM Tibor Zimányi <tzima...@apache.org> > wrote: > > > > > +1 > > > > > > Dňa pi 2. 8. 2024, 12:02 Alex Porcelli <a...@porcelli.me> napísal(a): > > > > > > > +1 > > > > > > > > On Fri, Aug 2, 2024 at 4:09 AM Pere Fernandez (apache) < > > > > pefer...@apache.org> > > > > wrote: > > > > > > > > > +1 to transactions! > > > > > > > > > > On Fri, 2 Aug 2024 at 08:39, Enrique Gonzalez Martinez < > > > > > egonza...@apache.org> > > > > > wrote: > > > > > > > > > > > * Transactions* > > > > > > This document describes how to support transactions in the > domain of > > > > > > workflow engine and subsystems. > > > > > > > > > > > > The use cases for transactions in workflows is to enable > consistency > > > > > > during workflow executions. > > > > > > > > > > > > * Constraints * > > > > > > > > > > > > The constraints for this are related to different types of > > > transaction > > > > > > problems: > > > > > > > > > > > > Workflow transaction execution should be in one single > transaction > > > > > > (until idle elements are reached or there are no more elements to > > > > > > process) > > > > > > > > > > > > Process state should be consistent in storage in one single > > > > > > transaction. In the case of database multiple tables should be > > > written > > > > > > in an atomic transaction > > > > > > > > > > > > Reactive code should be removed as it does not behave properly > with > > > > > > transactions. > > > > > > > > > > > > Transactions Policy among workflow runtime and subsystems should > be > > > > > > consistent in terms of configuration (no subcomponent should > start a > > > > > > transaction if there is already one on the go, but they should > > > mandate > > > > > > to be in a transaction) > > > > > > > > > > > > Error handling should still produce an event that can be stored. > > > > > > > > > > > > Subsystems execution should be included during transactions > > > > > > > > > > > > Async execution will spawn its own transaction. > > > > > > > > > > > > * Architecture * > > > > > > > > > > > > The architecture of the solution impacts some areas: > > > > > > > > > > > > Components with reactive that are involved in transaction > refactor. > > > So > > > > > > far, the only subsystem using reactive code job service. > > > > > > > > > > > > Process Code generation should change in order to reflect the > > > > > > transactions of the workflow engine > > > > > > > > > > > > Error handling should be modified in a way the error is captured > > > > > > outside the transaction and handled in a different one to avoid > event > > > > > > loss. > > > > > > > > > > > > Exchange information among runtime and subsystems should be in a > way > > > > > > that those elements are involved in a transaction or they can be > > > > > > rolled back. At the moment the communication is being done with a > > > rest > > > > > > call that is not part of the transaction and cannot be rolled > back. > > > > > > > > > > > > Events produced within the transaction should be part of the > > > > > > transaction as well to avoid phantom events (events producing > during > > > > > > workflow execution that are sent at the end of the unit of work) > > > > > > > > > > > > * Risk Assessment * > > > > > > > > > > > > The risks identified for this work are the following: > > > > > > > > > > > > Error handling can be problematic depending where we set the > > > > > > boundaries of the transaction. There are two different > approaches: > > > > > > > > > > > > Boilerplate code for each task to start / commit / rollback the > > > > > > transaction and deail with error in the rest call tier itself > > > > > > > > > > > > Use the runtime environment to install error handling for doing > the > > > > > > operation. > > > > > > > > > > > > Exchange information among systems in a non-transactional way. > There > > > > > > are a couple of approaches > > > > > > > > > > > > Install every time a transaction sync listener whenever the rest > call > > > > > > is made against the subsystem and doing a compensation when it > fails > > > > > > > > > > > > Wrap the rest call in a XAResource that can be enlisted in the > > > > > transaction. > > > > > > > > > > > > The use of Kafka clients for stream that does not belong to the > > > > > > transactions > > > > > > > > > > > > Wrap with XAResource (Kafka client support transactions, but > does not > > > > > > offer XAresource) > > > > > > > > > > > > Install a transaction sync for each transaction. > > > > > > > > > > > > Performance impact with transactions. > > > > > > > > > > > > Different transaction methods in quarkus and spring boot > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > To unsubscribe, e-mail: dev-unsubscr...@kie.apache.org > > > > > > For additional commands, e-mail: dev-h...@kie.apache.org > > > > > > > > > > > > > > > > -- > Saludos, Enrique González Martínez :) > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@kie.apache.org > For additional commands, e-mail: dev-h...@kie.apache.org > >
