Hi Sachin, Check the last few comments I wrote in this thread:
https://lists.apache.org/thread/l71d1cqo9xv8rsw0gfjo19kb1pct2xj1 Regards, Alexis. On Wed, 20 Mar 2024, 18:51 Sachin Mittal, <sjmit...@gmail.com> wrote: > Hi, > I saw the post but I did not understand how I would configure these fields > to use those serializers. (I can change the set type to a list type for > now). > As per the docs I see that we can annotate fields with @TypeInfo > > But what I did not get is how using this annotation I can use ListSerializer > and *MapSerializer.* > > Thanks > Sachin > > > On Wed, Mar 20, 2024 at 10:47 PM Ken Krugler <kkrugler_li...@transpac.com> > wrote: > >> Flink doesn’t have built-in support for serializing Sets. >> >> See this (stale) issue about the same: >> https://issues.apache.org/jira/browse/FLINK-16729 >> >> You could create a custom serializer for sets, see >> https://stackoverflow.com/questions/59800851/flink-serialization-of-java-util-list-and-java-util-map >> and >> https://nightlies.apache.org/flink/flink-docs-stable/api/java/org/apache/flink/api/common/typeutils/base/ListSerializer.html >> for details on how this was done for a list, but it’s not trivial. >> >> Or as a hack, use a Map<Integer, Void> and the existing support for map >> serialization via >> https://nightlies.apache.org/flink/flink-docs-stable/api/java/org/apache/flink/api/common/typeutils/base/MapSerializer.html >> >> — Ken >> >> >> On Mar 20, 2024, at 10:04 AM, Sachin Mittal <sjmit...@gmail.com> wrote: >> >> Hi, >> I have a Pojo class like this >> >> public class A { >> >> public String str; >> >> public Set<Integer> aSet; >> >> public Map<String, Double> dMap; >> >> } >> >> However when I start the flink program I get this message: >> >> org.apache.flink.api.java.typeutils.TypeExtractor [] - Field A# >> dMap will be processed as GenericType. Please read the Flink >> documentation on "Data Types & Serialization" for details of the effect on >> performance and schema evolution. >> >> org.apache.flink.api.java.typeutils.TypeExtractor [] - Field A# >> aSet will be processed as GenericType. Please read the Flink >> documentation on "Data Types & Serialization" for details of the effect on >> performance and schema evolution. >> >> Also in my code I have added >> >> env.getConfig().disableGenericTypes(); >> >> So I don't understand when I use Maps and Sets of primitive types why is >> Flink not >> >> able to use PojoSerializer for these fields and even when I have disabled >> generics types. >> >> why I am getting message that it will be processed as GenericType? >> >> >> Any help in understanding what I need to do to ensure all the fields of my >> object are handled using PojoSerializer. >> >> >> Thanks >> >> Sachin >> >> >> >> -------------------------- >> Ken Krugler >> http://www.scaleunlimited.com >> Custom big data solutions >> Flink & Pinot >> >> >> >>
