Re: [EXT] Re: [VOTE] KIP-1033: Add Kafka Streams exception handler for exceptions occurring during processing
Thanks Colin. Merged the PR to `trunk` and cherry-picked to `3.9` branch. -Matthias On 9/3/24 4:07 PM, Colin McCabe wrote: Hi Matthias and Sebastien, This sounds like a very small and low-risk PR for a KIP that is new in 3.9 (KIP-1033). I think we should be OK to cherry-pick it to 3.9, even though we're past code freeze. Please try to get it done this week, though :) best, Colin On Fri, Aug 30, 2024, at 19:03, Matthias J. Sax wrote: Guess it is a question for Colin, who is the RM for 3.9. -Matthias On 8/30/24 2:02 AM, Sebastien Viale wrote: Hi, All PRs related to KIP-1033 have been merged. However, after a last review from Matthias J. Sax and Bill Bejeck, we updated the KIP to include the timestamp in the ErrorHandlerContext interface. A new PR has been initialized. KAFKA-16448 Add timestamp to error handler context (https://github.com/apache/kafka/pull/17054) Please let us know if we need to work on this for 3.9 or for next release. thanks very much Sébastien De : Matthias J. Sax Envoyé : jeudi 1 août 2024 23:46 À : dev@kafka.apache.org Objet : [EXT] Re: [VOTE] KIP-1033: Add Kafka Streams exception handler for exceptions occurring during processing Warning External sender Do not click on any links or open any attachments unless you trust the sender and know the content is safe. Thanks Loïc! This email was screened for spam and malicious content but exercise caution anyway. On 8/1/24 6:19 AM, Loic Greffier wrote: Hi, Last PR that completed KIP-1033 has been merged. Thanks to all participants, for the guidance and the feedbacks! Just letting you know about some decisions and changes made through the different PRs that were not initially mentioned in the KIP: * The sourceRawKey and the sourceRawValue attributes present in the ErrorHandlerContext have been removed from this KIP. They will be part of KIP-1034 instead. * The number of processing exception handler instance has been fixed to one per StreamTask. * The processing exception handling mechanism has been extended to punctuations. * The handler crash management has been unified and improved. Exceptions thrown from a call to #handle are caught and fail the stream. This change covers 5 use-cases: * On processingExceptionHandler#handle to cover processing exceptions. * On processingExceptionHandler#handle to cover punctuation exceptions. * On productionExceptionHandler#handle to cover production exceptions. * On productionExceptionHandler#handleSerializationException to cover serialization exceptions. * On deserializationExceptionHandler#handle to cover deserialization exceptions. KIP-1033 has been updated with these changes. Loïc
Re: [EXT] Re: [VOTE] KIP-1033: Add Kafka Streams exception handler for exceptions occurring during processing
Hi Matthias and Sebastien, This sounds like a very small and low-risk PR for a KIP that is new in 3.9 (KIP-1033). I think we should be OK to cherry-pick it to 3.9, even though we're past code freeze. Please try to get it done this week, though :) best, Colin On Fri, Aug 30, 2024, at 19:03, Matthias J. Sax wrote: > Guess it is a question for Colin, who is the RM for 3.9. > > > > -Matthias > > > > On 8/30/24 2:02 AM, Sebastien Viale wrote: >> Hi, >> >> All PRs related to KIP-1033 have been merged. >> >> However, after a last review from Matthias J. Sax and Bill Bejeck, we >> updated the KIP to include the timestamp in the ErrorHandlerContext >> interface. >> >> A new PR has been initialized. >> KAFKA-16448 Add timestamp to error handler context >> (https://github.com/apache/kafka/pull/17054) >> >> Please let us know if we need to work on this for 3.9 or for next release. >> >> thanks very much >> >> Sébastien >> >> ____________________ >> De : Matthias J. Sax >> Envoyé : jeudi 1 août 2024 23:46 >> À : dev@kafka.apache.org >> Objet : [EXT] Re: [VOTE] KIP-1033: Add Kafka Streams exception handler for >> exceptions occurring during processing >> >> Warning External sender Do not click on any links or open any attachments >> unless you trust the sender and know the content is safe. >> >> Thanks Loïc! >> >> >> This email was screened for spam and malicious content but exercise caution >> anyway. >> >> >> On 8/1/24 6:19 AM, Loic Greffier wrote: >>> Hi, >>> >>> Last PR that completed KIP-1033 has been merged. >>> Thanks to all participants, for the guidance and the feedbacks! >>> >>> Just letting you know about some decisions and changes made through the >>> different PRs that were not initially mentioned in the KIP: >>> >>> * The sourceRawKey and the sourceRawValue attributes present in the >>> ErrorHandlerContext have been removed from this KIP. They will be part of >>> KIP-1034 instead. >>> * The number of processing exception handler instance has been fixed >>> to one per StreamTask. >>> * The processing exception handling mechanism has been extended to >>> punctuations. >>> * The handler crash management has been unified and improved. >>> Exceptions thrown from a call to #handle are caught and fail the stream. >>> This change covers 5 use-cases: >>>* On processingExceptionHandler#handle to cover processing >>> exceptions. >>>* On processingExceptionHandler#handle to cover punctuation >>> exceptions. >>>* On productionExceptionHandler#handle to cover production >>> exceptions. >>>* On productionExceptionHandler#handleSerializationException to >>> cover serialization exceptions. >>>* On deserializationExceptionHandler#handle to cover >>> deserialization exceptions. >>> >>> KIP-1033 has been updated with these changes. >>> >>> Loïc >>> >> >>
Re: [EXT] Re: [VOTE] KIP-1033: Add Kafka Streams exception handler for exceptions occurring during processing
Guess it is a question for Colin, who is the RM for 3.9. -Matthias On 8/30/24 2:02 AM, Sebastien Viale wrote: Hi, All PRs related to KIP-1033 have been merged. However, after a last review from Matthias J. Sax and Bill Bejeck, we updated the KIP to include the timestamp in the ErrorHandlerContext interface. A new PR has been initialized. KAFKA-16448 Add timestamp to error handler context (https://github.com/apache/kafka/pull/17054) Please let us know if we need to work on this for 3.9 or for next release. thanks very much Sébastien De : Matthias J. Sax Envoyé : jeudi 1 août 2024 23:46 À : dev@kafka.apache.org Objet : [EXT] Re: [VOTE] KIP-1033: Add Kafka Streams exception handler for exceptions occurring during processing Warning External sender Do not click on any links or open any attachments unless you trust the sender and know the content is safe. Thanks Loïc! This email was screened for spam and malicious content but exercise caution anyway. On 8/1/24 6:19 AM, Loic Greffier wrote: Hi, Last PR that completed KIP-1033 has been merged. Thanks to all participants, for the guidance and the feedbacks! Just letting you know about some decisions and changes made through the different PRs that were not initially mentioned in the KIP: * The sourceRawKey and the sourceRawValue attributes present in the ErrorHandlerContext have been removed from this KIP. They will be part of KIP-1034 instead. * The number of processing exception handler instance has been fixed to one per StreamTask. * The processing exception handling mechanism has been extended to punctuations. * The handler crash management has been unified and improved. Exceptions thrown from a call to #handle are caught and fail the stream. This change covers 5 use-cases: * On processingExceptionHandler#handle to cover processing exceptions. * On processingExceptionHandler#handle to cover punctuation exceptions. * On productionExceptionHandler#handle to cover production exceptions. * On productionExceptionHandler#handleSerializationException to cover serialization exceptions. * On deserializationExceptionHandler#handle to cover deserialization exceptions. KIP-1033 has been updated with these changes. Loïc
RE: [EXT] Re: [VOTE] KIP-1033: Add Kafka Streams exception handler for exceptions occurring during processing
Hi, All PRs related to KIP-1033 have been merged. However, after a last review from Matthias J. Sax and Bill Bejeck, we updated the KIP to include the timestamp in the ErrorHandlerContext interface. A new PR has been initialized. KAFKA-16448 Add timestamp to error handler context (https://github.com/apache/kafka/pull/17054) Please let us know if we need to work on this for 3.9 or for next release. thanks very much Sébastien De : Matthias J. Sax Envoyé : jeudi 1 août 2024 23:46 À : dev@kafka.apache.org Objet : [EXT] Re: [VOTE] KIP-1033: Add Kafka Streams exception handler for exceptions occurring during processing Warning External sender Do not click on any links or open any attachments unless you trust the sender and know the content is safe. Thanks Loïc! This email was screened for spam and malicious content but exercise caution anyway. On 8/1/24 6:19 AM, Loic Greffier wrote: > Hi, > > Last PR that completed KIP-1033 has been merged. > Thanks to all participants, for the guidance and the feedbacks! > > Just letting you know about some decisions and changes made through the > different PRs that were not initially mentioned in the KIP: > >* The sourceRawKey and the sourceRawValue attributes present in the > ErrorHandlerContext have been removed from this KIP. They will be part of > KIP-1034 instead. >* The number of processing exception handler instance has been fixed to > one per StreamTask. >* The processing exception handling mechanism has been extended to > punctuations. >* The handler crash management has been unified and improved. Exceptions > thrown from a call to #handle are caught and fail the stream. This change > covers 5 use-cases: > * On processingExceptionHandler#handle to cover processing exceptions. > * On processingExceptionHandler#handle to cover punctuation > exceptions. > * On productionExceptionHandler#handle to cover production exceptions. > * On productionExceptionHandler#handleSerializationException to cover > serialization exceptions. > * On deserializationExceptionHandler#handle to cover deserialization > exceptions. > > KIP-1033 has been updated with these changes. > > Loïc >
Re: [VOTE] KIP-1033: Add Kafka Streams exception handler for exceptions occurring during processing
Thanks Loïc! On 8/1/24 6:19 AM, Loic Greffier wrote: Hi, Last PR that completed KIP-1033 has been merged. Thanks to all participants, for the guidance and the feedbacks! Just letting you know about some decisions and changes made through the different PRs that were not initially mentioned in the KIP: * The sourceRawKey and the sourceRawValue attributes present in the ErrorHandlerContext have been removed from this KIP. They will be part of KIP-1034 instead. * The number of processing exception handler instance has been fixed to one per StreamTask. * The processing exception handling mechanism has been extended to punctuations. * The handler crash management has been unified and improved. Exceptions thrown from a call to #handle are caught and fail the stream. This change covers 5 use-cases: * On processingExceptionHandler#handle to cover processing exceptions. * On processingExceptionHandler#handle to cover punctuation exceptions. * On productionExceptionHandler#handle to cover production exceptions. * On productionExceptionHandler#handleSerializationException to cover serialization exceptions. * On deserializationExceptionHandler#handle to cover deserialization exceptions. KIP-1033 has been updated with these changes. Loïc
Re: [VOTE] KIP-1033: Add Kafka Streams exception handler for exceptions occurring during processing
Hi, Last PR that completed KIP-1033 has been merged. Thanks to all participants, for the guidance and the feedbacks! Just letting you know about some decisions and changes made through the different PRs that were not initially mentioned in the KIP: * The sourceRawKey and the sourceRawValue attributes present in the ErrorHandlerContext have been removed from this KIP. They will be part of KIP-1034 instead. * The number of processing exception handler instance has been fixed to one per StreamTask. * The processing exception handling mechanism has been extended to punctuations. * The handler crash management has been unified and improved. Exceptions thrown from a call to #handle are caught and fail the stream. This change covers 5 use-cases: * On processingExceptionHandler#handle to cover processing exceptions. * On processingExceptionHandler#handle to cover punctuation exceptions. * On productionExceptionHandler#handle to cover production exceptions. * On productionExceptionHandler#handleSerializationException to cover serialization exceptions. * On deserializationExceptionHandler#handle to cover deserialization exceptions. KIP-1033 has been updated with these changes. Loïc
[VOTE] KIP-1033: Add Kafka Streams exception handler for exceptions occurring during processing
Hi everyone, A first pull request has been opened: https://github.com/apache/kafka/pull/16090. As Bruno mentioned in the following discussion: https://github.com/apache/kafka/pull/16090#discussion_r1616264629, we bring a minor change to KIP-1033 by renaming both ProcessingExceptionHandler implementations to stick with the name of the interface: - "ProcessingLogAndContinueExceptionHandler" to "LogAndContinueProcessingExceptionHandler" - "ProcessingLogAndFailExceptionHandler" to "LogAndFailProcessingExceptionHandler" Regards, Loïc
Re: [VOTE] KIP-1033: Add Kafka Streams exception handler for exceptions occurring during processing
Thanks Lianet, I’ll conclude the vote with 4 binding votes from Matthias J. Sax, Sophie Blee-Goldman, Bill Bejeck and Bruno Cadona. Thank you everyone! On Tue, 14 May 2024 at 17:56, Lianet M. wrote: > > +1 (non-binding) > > Thanks for the KIP and updates! > > Lianet > > On Tue, May 14, 2024, 12:03 a.m. Matthias J. Sax wrote: > > > +1 (binding) > > > > On 5/13/24 5:54 PM, Sophie Blee-Goldman wrote: > > > Thanks for the KIP guys! > > > > > > +1 (binding) > > > > > > On Mon, May 13, 2024 at 6:02 AM Bill Bejeck wrote: > > > > > >> Thanks for the KIP, this will be a great addition! > > >> > > >> +1(binding) > > >> > > >> -Bill > > >> > > >> On Fri, May 3, 2024 at 4:48 AM Bruno Cadonna > > wrote: > > >> > > >>> Hi Damien, Sébastien, and Loïc, > > >>> > > >>> Thanks for the KIP! > > >>> > > >>> +1 (binding) > > >>> > > >>> Best, > > >>> Bruno > > >>> > > >>> > > >>> On 4/26/24 4:00 PM, Damien Gasparina wrote: > > Hi all, > > > > We would like to start a vote for KIP-1033: Add Kafka Streams > > exception handler for exceptions occurring during processing > > > > The KIP is available on > > > > >>> > > >> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1033%3A+Add+Kafka+Streams+exception+handler+for+exceptions+occurring+during+processing > > > > If you have any suggestions or feedback, feel free to participate to > > the discussion thread: > > https://lists.apache.org/thread/1nhhsrogmmv15o7mk9nj4kvkb5k2bx9s > > > > Best regards, > > Damien Sebastien and Loic > > >>> > > >> > > > > >
Re: [VOTE] KIP-1033: Add Kafka Streams exception handler for exceptions occurring during processing
+1 (non-binding) Thanks for the KIP and updates! Lianet On Tue, May 14, 2024, 12:03 a.m. Matthias J. Sax wrote: > +1 (binding) > > On 5/13/24 5:54 PM, Sophie Blee-Goldman wrote: > > Thanks for the KIP guys! > > > > +1 (binding) > > > > On Mon, May 13, 2024 at 6:02 AM Bill Bejeck wrote: > > > >> Thanks for the KIP, this will be a great addition! > >> > >> +1(binding) > >> > >> -Bill > >> > >> On Fri, May 3, 2024 at 4:48 AM Bruno Cadonna > wrote: > >> > >>> Hi Damien, Sébastien, and Loïc, > >>> > >>> Thanks for the KIP! > >>> > >>> +1 (binding) > >>> > >>> Best, > >>> Bruno > >>> > >>> > >>> On 4/26/24 4:00 PM, Damien Gasparina wrote: > Hi all, > > We would like to start a vote for KIP-1033: Add Kafka Streams > exception handler for exceptions occurring during processing > > The KIP is available on > > >>> > >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1033%3A+Add+Kafka+Streams+exception+handler+for+exceptions+occurring+during+processing > > If you have any suggestions or feedback, feel free to participate to > the discussion thread: > https://lists.apache.org/thread/1nhhsrogmmv15o7mk9nj4kvkb5k2bx9s > > Best regards, > Damien Sebastien and Loic > >>> > >> > > >
Re: [VOTE] KIP-1033: Add Kafka Streams exception handler for exceptions occurring during processing
+1 (binding) On 5/13/24 5:54 PM, Sophie Blee-Goldman wrote: Thanks for the KIP guys! +1 (binding) On Mon, May 13, 2024 at 6:02 AM Bill Bejeck wrote: Thanks for the KIP, this will be a great addition! +1(binding) -Bill On Fri, May 3, 2024 at 4:48 AM Bruno Cadonna wrote: Hi Damien, Sébastien, and Loïc, Thanks for the KIP! +1 (binding) Best, Bruno On 4/26/24 4:00 PM, Damien Gasparina wrote: Hi all, We would like to start a vote for KIP-1033: Add Kafka Streams exception handler for exceptions occurring during processing The KIP is available on https://cwiki.apache.org/confluence/display/KAFKA/KIP-1033%3A+Add+Kafka+Streams+exception+handler+for+exceptions+occurring+during+processing If you have any suggestions or feedback, feel free to participate to the discussion thread: https://lists.apache.org/thread/1nhhsrogmmv15o7mk9nj4kvkb5k2bx9s Best regards, Damien Sebastien and Loic
Re: [VOTE] KIP-1033: Add Kafka Streams exception handler for exceptions occurring during processing
Thanks for the KIP guys! +1 (binding) On Mon, May 13, 2024 at 6:02 AM Bill Bejeck wrote: > Thanks for the KIP, this will be a great addition! > > +1(binding) > > -Bill > > On Fri, May 3, 2024 at 4:48 AM Bruno Cadonna wrote: > > > Hi Damien, Sébastien, and Loïc, > > > > Thanks for the KIP! > > > > +1 (binding) > > > > Best, > > Bruno > > > > > > On 4/26/24 4:00 PM, Damien Gasparina wrote: > > > Hi all, > > > > > > We would like to start a vote for KIP-1033: Add Kafka Streams > > > exception handler for exceptions occurring during processing > > > > > > The KIP is available on > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1033%3A+Add+Kafka+Streams+exception+handler+for+exceptions+occurring+during+processing > > > > > > If you have any suggestions or feedback, feel free to participate to > > > the discussion thread: > > > https://lists.apache.org/thread/1nhhsrogmmv15o7mk9nj4kvkb5k2bx9s > > > > > > Best regards, > > > Damien Sebastien and Loic > > >
Re: [VOTE] KIP-1033: Add Kafka Streams exception handler for exceptions occurring during processing
Thanks for the KIP, this will be a great addition! +1(binding) -Bill On Fri, May 3, 2024 at 4:48 AM Bruno Cadonna wrote: > Hi Damien, Sébastien, and Loïc, > > Thanks for the KIP! > > +1 (binding) > > Best, > Bruno > > > On 4/26/24 4:00 PM, Damien Gasparina wrote: > > Hi all, > > > > We would like to start a vote for KIP-1033: Add Kafka Streams > > exception handler for exceptions occurring during processing > > > > The KIP is available on > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1033%3A+Add+Kafka+Streams+exception+handler+for+exceptions+occurring+during+processing > > > > If you have any suggestions or feedback, feel free to participate to > > the discussion thread: > > https://lists.apache.org/thread/1nhhsrogmmv15o7mk9nj4kvkb5k2bx9s > > > > Best regards, > > Damien Sebastien and Loic >
Re: [VOTE] KIP-1033: Add Kafka Streams exception handler for exceptions occurring during processing
Hi Damien, Sébastien, and Loïc, Thanks for the KIP! +1 (binding) Best, Bruno On 4/26/24 4:00 PM, Damien Gasparina wrote: Hi all, We would like to start a vote for KIP-1033: Add Kafka Streams exception handler for exceptions occurring during processing The KIP is available on https://cwiki.apache.org/confluence/display/KAFKA/KIP-1033%3A+Add+Kafka+Streams+exception+handler+for+exceptions+occurring+during+processing If you have any suggestions or feedback, feel free to participate to the discussion thread: https://lists.apache.org/thread/1nhhsrogmmv15o7mk9nj4kvkb5k2bx9s Best regards, Damien Sebastien and Loic
[VOTE] KIP-1033: Add Kafka Streams exception handler for exceptions occurring during processing
Hi all, We would like to start a vote for KIP-1033: Add Kafka Streams exception handler for exceptions occurring during processing The KIP is available on https://cwiki.apache.org/confluence/display/KAFKA/KIP-1033%3A+Add+Kafka+Streams+exception+handler+for+exceptions+occurring+during+processing If you have any suggestions or feedback, feel free to participate to the discussion thread: https://lists.apache.org/thread/1nhhsrogmmv15o7mk9nj4kvkb5k2bx9s Best regards, Damien Sebastien and Loic
