Great work, Ramzan! I think that we are ready to proceed to the PIP voting stage. Please initiate a new vote thread.
-Lari On 2025/09/08 08:41:54 Ramzan MAGOMADOW wrote: > Hello Lari! > > Thank you for your valuable feedback and for merging the PR to support my > proposal! > > As you suggested, I will rename the feature to "managed transactions" , which > better reflects its nature, and will separate implementation details into a > dedicated PR later. The proposal will also maintain consistency between > transaction handling and the existing processing model to properly support > asynchronous functions as well, thanks for pointing it out! > > I'll have the updated proposal ready soon. > > - Ramzan > > > > Classification: GENERAL > > On 2025/09/05 13:05:49 Lari Hotari wrote: > > Thank you Ramzan for driving this improvement to Pulsar! Good work! > > > > Naming Suggestion: Could we call this "managed transactions" for > > Pulsar Functions instead of "automatic transactions"? > > > > I provided a review with some suggestions: > > https://github.com/apache/pulsar/pull/24704 > > > > It could be easier to review code in a separate implementation PR, > > however, covering all implementation level details isn't necessary. > > > > It is useful to handle the main changes in the implementation and > > describe the implementation strategy. > > > > For example, I added a comment about keeping the transactional > > handling similar to current processing so that asynchronous functions > > would also be supported. > > > > A poorly documented feature of Pulsar Functions is that it supports > > asynchronous processing. This is really important in achieving higher > > performance since it allows having more than 1 outstanding message in > > processing. Async function support was added in > > https://github.com/apache/pulsar/pull/6684. By default there can be > > 1000 outstanding messages in processing and this can be controlled > > with the "maxPendingAsyncRequests" setting. Supporting transactions > > for asynchronous processing should also be part of the scope of this > > PIP so that the implementation doesn't unnecessarily divert between > > "managed transactions" and non-transactional processing. > > > > One interesting implementation detail is that this work requires the > > PIP-407 changes > > https://github.com/apache/pulsar/blob/master/pip/pip-407.md that just > > landed in the Pulsar master branch with > > https://github.com/apache/pulsar/pull/23942. > > > > Looking forward to seeing PIP-439 move forward! > > > > -Lari > > > > > > On Fri, 5 Sept 2025 at 11:53, Ramzan MAGOMADOW > > > > <[email protected]<mailto:[email protected]>> wrote: > > > > > > Dear Pulsar Community! > > > > > > I would like to start a formal discussion on PIP-439. > > > > > > This proposal aims to enhance Pulsar Functions with transaction > > > capabilities, enabling atomic operations across multiple topics without > > > requiring code changes to function implementations. The proposal builds > > > on Pulsar's existing transaction system to provide exactly-once > > > processing semantics for Functions. > > > > > > Key aspects of the proposal include: > > > 1. Automatic transaction wrapping for Functions through configuration > > > settings > > > 2. Support for transactional publishing to multiple output topics > > > 3. Transactional acknowledgment of input messages > > > 4. Transaction timeout configuration > > > > > > This proposal continues the discussion started in the previous thread: > > > https://lists.apache.org/thread/rll8qyovpd7t9v5yxth25qo44zksbgkn > > > > > > The complete proposal is available at: > > > https://github.com/apache/pulsar/pull/24704 > > > > > > I welcome your feedback, questions, and suggestions on this proposal. > > > Please share your thoughts on the design approach, implementation > > > details, or any considerations I may have overlooked. > > > > > > Best regards, > > > Ramzan Magomadow > > > This message and any attachment ("the Message") are confidential. If you > > > have received the Message in error, please notify the sender immediately > > > and delete the Message from your system, any use of the Message is > > > forbidden. Correspondence via e-mail is primarily for information > > > purposes. RBI neither makes nor accepts legally binding statements via > > > e-mail unless explicitly agreed otherwise. Information pursuant to ? 14 > > > Austrian Companies Code: Raiffeisen Bank International AG; Registered > > > Office: Am Stadtpark 9, 1030 Vienna, Austria; Company Register Number: FN > > > 122119m at the Commercial Court of Vienna (Handelsgericht Wien). > > > > > > Classification: GENERAL > > > > Sent from Outlook for Mac > > This message and any attachment ("the Message") are confidential. If you have > received the Message in error, please notify the sender immediately and > delete the Message from your system, any use of the Message is forbidden. > Correspondence via e-mail is primarily for information purposes. RBI neither > makes nor accepts legally binding statements via e-mail unless explicitly > agreed otherwise. Information pursuant to ? 14 Austrian Companies Code: > Raiffeisen Bank International AG; Registered Office: Am Stadtpark 9, 1030 > Vienna, Austria; Company Register Number: FN 122119m at the Commercial Court > of Vienna (Handelsgericht Wien). >
