Cool, thanks! I used the option value approach, worked well. On Thu, Dec 27, 2018, 03:49 Dominik Wosiński <wos...@gmail.com> wrote:
> Hey, > AFAIK, returning null from deserialize function in FlinkKafkaConsumer will > indeed filter the message out and it won't be further processed. > > Best Regards, > Dom. > > śr., 19 gru 2018 o 11:06 Dawid Wysakowicz <dwysakow...@apache.org> > napisał(a): > >> Hi, >> >> I'm afraid that there is no out-of-the box solution for this, but what >> you could do is to generate from KeyedDeserializationSchema some marker >> (Optional, null value...) based on the message key, that would allow you >> later to filter it out. So assuming the Optional solution the result of >> KeyedDeserializationSchema#deserialize could be Optional.empty() for >> invalid keys and Optional.of(deserializedValue) for valid keys. >> >> Best, >> >> Dawid >> On 18/12/2018 20:22, Hao Sun wrote: >> >> Hi, I am using 1.7 on K8S. >> >> I have a huge amount of data in kafka, but I only need a tiny portion of >> it. >> It is a keyed stream, the value in JSON encoded. I want to avoid >> deserialization of the value, since it is very expensive. Can I only filter >> based on the key? >> I know there is a KeyedDeserializationSchema, but can I use it to filter >> data? >> >> Hao Sun >> Team Lead >> 1019 Market St. 7F >> <https://maps.google.com/?q=1019+Market+St.+7F+%0D%0A+++++++++++++++++++++++San+Francisco,+CA+94103&entry=gmail&source=g> >> >> <https://maps.google.com/?q=1019+Market+St.+7F+%0D%0A+++++++++++++++++++++++San+Francisco,+CA+94103&entry=gmail&source=g> >> San Francisco, CA 94103 >> <https://maps.google.com/?q=1019+Market+St.+7F+%0D%0A+++++++++++++++++++++++San+Francisco,+CA+94103&entry=gmail&source=g> >> >>
