Hi all, This rationale applies in a deployment scenario where we have *runtimes *running in a pod and *data index* is running in another pod and quarkus http connector (based on Smallrye) is used for sending events from *runtimes *into *data index*
In that deployment scenarios, after process instance events were split with https://github.com/apache/incubator-kie-issues/issues/249, an http request to start a process over runtimes pod is followed by a variable number of http request to data index pod in order to send these multiple events (one per event). That affects system overall throughput, because *runtimes *eventually start queuing http requests since* data index* cannot handle them at the same speed. Apart from the obvious solution, to use a different communication mechanism to send events (kafka or the so called single write, which basically consist on runtimes instance updating data index tables within the same db transaction, which obviously implies that both runtimes and dataindex use the same db), I feel we should try to improve the HTTP communication scenario by grouping http request. Since EventPublisher already has the events of a UnitOfWork already grouped into a collection, but the Smallrye interface accepts one message per call; in order to group them, I foresee these two option: 1) Consolidate EventPublisher event collection into one single smallrye message 2) Change the http quarkus connector to group the messages that are being received one by one using a time heuristic approach (as old legacy JMS used to do) First approach is optimal from an application point of view but implies changing current consumers to be aware of the fact that a message can contain an array of DataEvent. Second approach is transparent to existing consumers (they still receive one message per event), but implies some delay before the http request is performed by the emitter. Wdyt?
