Minor updates could also be made after starting the vote. There's also a need to add the discussion thread link and voting thread link to the PIP document.
Btw, regarding future improvements, I left a comment about "transaction batching" on the PIP PR: https://github.com/apache/pulsar/pull/24704#issuecomment-3270421116 For high-scale use cases, it might be necessary to implement "transaction batching" so that unnecessary load isn't added to the broker (including the transaction coordinator) when enabling transactions. It might be useful to cover the "transaction batching" aspect directly in PIP-439 if you'd be ready to do that before proceeding to the vote. It's also possible to handle that later in another PIP. -Lari On 2025/09/09 12:07:20 Ramzan MAGOMADOW wrote: > Hi Lari! > > I've seen that you have already approved my PR, but I forgot to change the > title to "PIP-439: Adding Transaction Support to Pulsar Functions Through > Managed Transaction Wrapping" in the PIP-439.md. I'll add a comment about > making the "pulsar_function_txn_latency" configurable as well, as you > mentioned in your GitHub review. Finally, I'll squash the commits in my PR. > > Is it okay to start the PIP voting even while I do those things? > > -Ramzan > > > > Classification: GENERAL > > On 2025/09/09 11:47:49 Lari Hotari wrote: > > Great work, Ramzan! > > > > I think that we are ready to proceed to the PIP voting stage. Please > > initiate a new vote thread. > > > > -Lari > > > > On 2025/09/08 08:41:54 Ramzan MAGOMADOW wrote: > > > Hello Lari! > > > > > > Thank you for your valuable feedback and for merging the PR to support my > > > proposal! > > > > > > As you suggested, I will rename the feature to "managed transactions" , > > > which better reflects its nature, and will separate implementation > > > details into a dedicated PR later. The proposal will also maintain > > > consistency between transaction handling and the existing processing > > > model to properly support asynchronous functions as well, thanks for > > > pointing it out! > > > > > > I'll have the updated proposal ready soon. > > > > > > - Ramzan > > > > > > > > > > > > Classification: GENERAL > > > > > > On 2025/09/05 13:05:49 Lari Hotari wrote: > > > > Thank you Ramzan for driving this improvement to Pulsar! Good work! > > > > > > > > Naming Suggestion: Could we call this "managed transactions" for > > > > Pulsar Functions instead of "automatic transactions"? > > > > > > > > I provided a review with some suggestions: > > > > https://github.com/apache/pulsar/pull/24704 > > > > > > > > It could be easier to review code in a separate implementation PR, > > > > however, covering all implementation level details isn't necessary. > > > > > > > > It is useful to handle the main changes in the implementation and > > > > describe the implementation strategy. > > > > > > > > For example, I added a comment about keeping the transactional > > > > handling similar to current processing so that asynchronous functions > > > > would also be supported. > > > > > > > > A poorly documented feature of Pulsar Functions is that it supports > > > > asynchronous processing. This is really important in achieving higher > > > > performance since it allows having more than 1 outstanding message in > > > > processing. Async function support was added in > > > > https://github.com/apache/pulsar/pull/6684. By default there can be > > > > 1000 outstanding messages in processing and this can be controlled > > > > with the "maxPendingAsyncRequests" setting. Supporting transactions > > > > for asynchronous processing should also be part of the scope of this > > > > PIP so that the implementation doesn't unnecessarily divert between > > > > "managed transactions" and non-transactional processing. > > > > > > > > One interesting implementation detail is that this work requires the > > > > PIP-407 changes > > > > https://github.com/apache/pulsar/blob/master/pip/pip-407.md that just > > > > landed in the Pulsar master branch with > > > > https://github.com/apache/pulsar/pull/23942. > > > > > > > > Looking forward to seeing PIP-439 move forward! > > > > > > > > -Lari > > > > > > > > > > > > On Fri, 5 Sept 2025 at 11:53, Ramzan MAGOMADOW > > > > > > > > <ra...@rbinternational.com<mailto:ra...@rbinternational.com>>> wrote: > > > > > > > > > > Dear Pulsar Community! > > > > > > > > > > I would like to start a formal discussion on PIP-439. > > > > > > > > > > This proposal aims to enhance Pulsar Functions with transaction > > > > > capabilities, enabling atomic operations across multiple topics > > > > > without requiring code changes to function implementations. The > > > > > proposal builds on Pulsar's existing transaction system to provide > > > > > exactly-once processing semantics for Functions. > > > > > > > > > > Key aspects of the proposal include: > > > > > 1. Automatic transaction wrapping for Functions through > > > > > configuration settings > > > > > 2. Support for transactional publishing to multiple output topics > > > > > 3. Transactional acknowledgment of input messages > > > > > 4. Transaction timeout configuration > > > > > > > > > > This proposal continues the discussion started in the previous > > > > > thread: > > > > > https://lists.apache.org/thread/rll8qyovpd7t9v5yxth25qo44zksbgkn > > > > > > > > > > The complete proposal is available at: > > > > > https://github.com/apache/pulsar/pull/24704 > > > > > > > > > > I welcome your feedback, questions, and suggestions on this proposal. > > > > > Please share your thoughts on the design approach, implementation > > > > > details, or any considerations I may have overlooked. > > > > > > > > > > Best regards, > > > > > Ramzan Magomadow > > > > > This message and any attachment ("the Message") are confidential. If > > > > > you have received the Message in error, please notify the sender > > > > > immediately and delete the Message from your system, any use of the > > > > > Message is forbidden. Correspondence via e-mail is primarily for > > > > > information purposes. RBI neither makes nor accepts legally binding > > > > > statements via e-mail unless explicitly agreed otherwise. Information > > > > > pursuant to ? 14 Austrian Companies Code: Raiffeisen Bank > > > > > International AG; Registered Office: Am Stadtpark 9, 1030 Vienna, > > > > > Austria; Company Register Number: FN 122119m at the Commercial Court > > > > > of Vienna (Handelsgericht Wien). > > > > > > > > > > Classification: GENERAL > > > > > > > > > > Sent from Outlook for Mac > > > > > > This message and any attachment ("the Message") are confidential. If you > > > have received the Message in error, please notify the sender immediately > > > and delete the Message from your system, any use of the Message is > > > forbidden. Correspondence via e-mail is primarily for information > > > purposes. RBI neither makes nor accepts legally binding statements via > > > e-mail unless explicitly agreed otherwise. Information pursuant to ? 14 > > > Austrian Companies Code: Raiffeisen Bank International AG; Registered > > > Office: Am Stadtpark 9, 1030 Vienna, Austria; Company Register Number: FN > > > 122119m at the Commercial Court of Vienna (Handelsgericht Wien). > > > > > > > Sent from Outlook for Mac > > This message and any attachment ("the Message") are confidential. If you have > received the Message in error, please notify the sender immediately and > delete the Message from your system, any use of the Message is forbidden. > Correspondence via e-mail is primarily for information purposes. RBI neither > makes nor accepts legally binding statements via e-mail unless explicitly > agreed otherwise. Information pursuant to ? 14 Austrian Companies Code: > Raiffeisen Bank International AG; Registered Office: Am Stadtpark 9, 1030 > Vienna, Austria; Company Register Number: FN 122119m at the Commercial Court > of Vienna (Handelsgericht Wien). >
