How computationally intensive is the responsibility of the Transaction Coordinator?
-- Devin G. Bost On Wed, Apr 21, 2021, 3:01 PM Sijie Guo <guosi...@gmail.com> wrote: > Enrico - I was suggesting not even making it a standalone. TC is a > component that really tied to Pulsar protocol and logic. We shouldn't do it > until it is really needed. > > - Sijie > > On Wed, Apr 21, 2021 at 1:21 PM Enrico Olivelli <eolive...@gmail.com> > wrote: > > > Il giorno mer 21 apr 2021 alle ore 21:57 Sijie Guo > > <guosi...@gmail.com> ha scritto: > > > > > > We build transaction components in modules. It provides the flexibility > > for > > > people to run those components independently. > > > > > > However, I don't think we should encourage this deployment at the > > > beginning. Because most of the users don't have deep knowledge about > how > > > the transaction works and have no idea how to manage them if you run > them > > > separately. Hence my recommendation is that we don't add such a > > deployment > > > option in the documentation and helm chart. It will just confuse users. > > > > I am fine with not documenting how to do it for the 2.8 release, the > > same for the official Apache Pulsar Helm Chart. > > > > but currently there is no way to deploy the Transaction Coordinator as > > a standalone module, > > if there is a way to launch an isolated TC please let me know > > otherwise I will at least create an issue on github and hopefully we > > can implement it for Pulsar 2.9.0, when transactions will be hopefully > > considered fully stable. > > > > Enrico > > > > > > > > > > > > > > > > - Sijie > > > > > > On Wed, Apr 21, 2021 at 8:59 AM Enrico Olivelli <eolive...@gmail.com> > > wrote: > > > > > > > Il giorno mer 21 apr 2021 alle ore 16:18 PengHui Li > > > > <codelipeng...@gmail.com> ha scritto: > > > > > > > > > > Hi Enrico, > > > > > > > > > > The current implementation of the transaction coordinator is run > with > > > > the broker, but the transaction coordinator > > > > > Is a separate module in Pulsar. This is also mentioned in PIP-31 > > > > https://github.com/apache/pulsar/wiki/PIP-31%3A-Transaction-Support > > > > > > > > Great. > > > > Is there any ongoing effort to add the ability to run the TC outside > > > > of the broker process ? > > > > > > > > Enrico > > > > > > > > > > > > > > Thanks, > > > > > Penghui > > > > > On Apr 21, 2021, 7:24 PM +0800, Enrico Olivelli < > eolive...@gmail.com > > >, > > > > wrote: > > > > > > Hello, > > > > > > Pulsar 2.8.0 is coming near to the release and we want > > transactions to > > > > > > be "production ready". > > > > > > > > > > > > If I understand correctly now we start the Transaction > Coordinator > > > > > > service inside the Broker. > > > > > > > > > > > > Does it make sense to have the option to run the TC as a separate > > > > > > service, as we do for the Functions Worker ? > > > > > > > > > > > > For instance in a big cluster, with many brokers, an user may > like > > to > > > > > > dedicate machines to the role of Transaction Coordinator and do > not > > > > > > serve all of the usual Broker activities. > > > > > > > > > > > > The same problem will also go into the Helm Chart, it would be > > better > > > > > > to have the ability to configure the transaction related services > > with > > > > > > specific options. > > > > > > > > > > > > What do you think ? > > > > > > > > > > > > > > > > > > Enrico > > > > > > >
