I agree for the need of transactions, and welcome the further conversation about how to accomplish it.
On 2024/08/02 11:08:10 Enrique Gonzalez Martinez wrote: > Spawn a new conversation for that. > > El vie, 2 ago 2024, 12:30, Francisco Javier Tirado Sarti < > ftira...@redhat.com> escribió: > > > Ok, I think we all agree we need some kind of transactional behaviour for > > DB and events (on, in other words, guarantee that events are not lost). > > Maybe I'm going too fast, but I would like to discuss which are the > > possible approaches and prioritize them. > > But, yes, we need to do that > > Thanks for the clarification. > > > > On Fri, Aug 2, 2024 at 12:25 PM Enrique Gonzalez Martinez < > > elguard...@gmail.com> wrote: > > > > > Hi francisco, > > > > > > Please go back to the proposal, I am not setting any custom solutions > > > just mention in the risk assessment that there are two possible ways > > > to do this. one with transaction sync and another one with xa > > > resource. > > > debezium is ok but still requires transactions and that solution is > > > based on xa resource as you are using a database to queue the event > > > that will be sent through kafka. > > > > > > Hope it helps. This proposal is about adding transactions not about > > > how to deal with all possible cases that will come from that. The risk > > > assessment only identifies the problems and possible solutions. > > > > > > Cheers :) > > > > > > El vie, 2 ago 2024 a las 12:13, Francisco Javier Tirado Sarti > > > (<ftira...@redhat.com>) escribió: > > > > > > > > Sorry to be a PITA and somehow diminish the apparent overall enthusiasm > > > in > > > > most of the community, but should we not discuss what we are going to > > > > really do before voting? > > > > If for transactions we mean XA transaction, I have to vote -1 till we > > > > discuss other options. > > > > > > > > On Fri, Aug 2, 2024 at 12:05 PM Tibor Zimányi <tzima...@apache.org> > > > wrote: > > > > > > > > > +1 > > > > > > > > > > Dňa pi 2. 8. 2024, 12:02 Alex Porcelli <a...@porcelli.me> > > napísal(a): > > > > > > > > > > > +1 > > > > > > > > > > > > On Fri, Aug 2, 2024 at 4:09 AM Pere Fernandez (apache) < > > > > > > pefer...@apache.org> > > > > > > wrote: > > > > > > > > > > > > > +1 to transactions! > > > > > > > > > > > > > > On Fri, 2 Aug 2024 at 08:39, Enrique Gonzalez Martinez < > > > > > > > egonza...@apache.org> > > > > > > > wrote: > > > > > > > > > > > > > > > * Transactions* > > > > > > > > This document describes how to support transactions in the > > > domain of > > > > > > > > workflow engine and subsystems. > > > > > > > > > > > > > > > > The use cases for transactions in workflows is to enable > > > consistency > > > > > > > > during workflow executions. > > > > > > > > > > > > > > > > * Constraints * > > > > > > > > > > > > > > > > The constraints for this are related to different types of > > > > > transaction > > > > > > > > problems: > > > > > > > > > > > > > > > > Workflow transaction execution should be in one single > > > transaction > > > > > > > > (until idle elements are reached or there are no more elements > > to > > > > > > > > process) > > > > > > > > > > > > > > > > Process state should be consistent in storage in one single > > > > > > > > transaction. In the case of database multiple tables should be > > > > > written > > > > > > > > in an atomic transaction > > > > > > > > > > > > > > > > Reactive code should be removed as it does not behave properly > > > with > > > > > > > > transactions. > > > > > > > > > > > > > > > > Transactions Policy among workflow runtime and subsystems > > should > > > be > > > > > > > > consistent in terms of configuration (no subcomponent should > > > start a > > > > > > > > transaction if there is already one on the go, but they should > > > > > mandate > > > > > > > > to be in a transaction) > > > > > > > > > > > > > > > > Error handling should still produce an event that can be > > stored. > > > > > > > > > > > > > > > > Subsystems execution should be included during transactions > > > > > > > > > > > > > > > > Async execution will spawn its own transaction. > > > > > > > > > > > > > > > > * Architecture * > > > > > > > > > > > > > > > > The architecture of the solution impacts some areas: > > > > > > > > > > > > > > > > Components with reactive that are involved in transaction > > > refactor. > > > > > So > > > > > > > > far, the only subsystem using reactive code job service. > > > > > > > > > > > > > > > > Process Code generation should change in order to reflect the > > > > > > > > transactions of the workflow engine > > > > > > > > > > > > > > > > Error handling should be modified in a way the error is > > captured > > > > > > > > outside the transaction and handled in a different one to avoid > > > event > > > > > > > > loss. > > > > > > > > > > > > > > > > Exchange information among runtime and subsystems should be in > > a > > > way > > > > > > > > that those elements are involved in a transaction or they can > > be > > > > > > > > rolled back. At the moment the communication is being done > > with a > > > > > rest > > > > > > > > call that is not part of the transaction and cannot be rolled > > > back. > > > > > > > > > > > > > > > > Events produced within the transaction should be part of the > > > > > > > > transaction as well to avoid phantom events (events producing > > > during > > > > > > > > workflow execution that are sent at the end of the unit of > > work) > > > > > > > > > > > > > > > > * Risk Assessment * > > > > > > > > > > > > > > > > The risks identified for this work are the following: > > > > > > > > > > > > > > > > Error handling can be problematic depending where we set the > > > > > > > > boundaries of the transaction. There are two different > > > approaches: > > > > > > > > > > > > > > > > Boilerplate code for each task to start / commit / rollback the > > > > > > > > transaction and deail with error in the rest call tier itself > > > > > > > > > > > > > > > > Use the runtime environment to install error handling for doing > > > the > > > > > > > > operation. > > > > > > > > > > > > > > > > Exchange information among systems in a non-transactional way. > > > There > > > > > > > > are a couple of approaches > > > > > > > > > > > > > > > > Install every time a transaction sync listener whenever the > > rest > > > call > > > > > > > > is made against the subsystem and doing a compensation when it > > > fails > > > > > > > > > > > > > > > > Wrap the rest call in a XAResource that can be enlisted in the > > > > > > > transaction. > > > > > > > > > > > > > > > > The use of Kafka clients for stream that does not belong to the > > > > > > > > transactions > > > > > > > > > > > > > > > > Wrap with XAResource (Kafka client support transactions, but > > > does not > > > > > > > > offer XAresource) > > > > > > > > > > > > > > > > Install a transaction sync for each transaction. > > > > > > > > > > > > > > > > Performance impact with transactions. > > > > > > > > > > > > > > > > Different transaction methods in quarkus and spring boot > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > To unsubscribe, e-mail: dev-unsubscr...@kie.apache.org > > > > > > > > For additional commands, e-mail: dev-h...@kie.apache.org > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > Saludos, Enrique González Martínez :) > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: dev-unsubscr...@kie.apache.org > > > For additional commands, e-mail: dev-h...@kie.apache.org > > > > > > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@kie.apache.org For additional commands, e-mail: dev-h...@kie.apache.org
