We are also catching the exception in serde and returning null and then filtering out null values downstream so as they are not included.
Thanks Sachin On Thu, Apr 13, 2017 at 9:13 PM, Mike Gould <mikeyg...@gmail.com> wrote: > Great to know I've not gone off in the wrong direction > Thanks > > On Thu, 13 Apr 2017 at 16:34, Matthias J. Sax <matth...@confluent.io> > wrote: > > > Mike, > > > > thanks for your feedback. You are absolutely right that Streams API does > > not have great support for this atm. And it's very valuable that you > > report this (you are not the first person). It helps us prioritizing :) > > > > For now, there is no better solution as the one you described in your > > email, but its on our roadmap to improve the API -- and its priority got > > just increase by your request. > > > > I am sorry, that I can't give you a better answer right now :( > > > > > > -Matthias > > > > > > On 4/13/17 8:16 AM, Mike Gould wrote: > > > Hi > > > Are there any better error handling options for Kafka streams in java. > > > > > > Any errors in the serdes will break the stream. The suggested > > > implementation is to use the byte[] serde and do the deserialisation > in a > > > map operation. However this isn't ideal either as there's no great way > > to > > > handle exceptions. > > > My current tactics are to use flatMap in place of map everywhere and > > return > > > empySet on error. Unfortunately this means the error has to be handled > > > directly in the function where it happened and can only be handled as a > > > side effect. > > > > > > It seems to me that this could be done better. Maybe the *Mapper > > interfaces > > > could allow specific checked exceptions. These could be handled by > > specific > > > downstream KStream.mapException() steps which might e.g. Put an error > > > response on another stream branch. > > > Alternatively could it be made easier to return something like an > Either > > > from the Mappers with a the addition of few extra mapError or mapLeft > > > mapRight methods on KStream? > > > > > > Unless there's a better error handling pattern which I've entirely > > missed? > > > > > > Thanks > > > MIkeG > > > > > > > -- > - MikeG > http://en.wikipedia.org/wiki/Common_misconceptions > <http://en.wikipedia.org/wiki/Special:Random> >
