Re: Fast serialization for Kotlin data classes
True, that's a valid concern you raised here, Alexis. Thanks for pointing that out. On Thu, Sep 16, 2021 at 1:58 PM Alexis Sarda-Espinosa < alexis.sarda-espin...@microfocus.com> wrote: > Someone please correct me if I’m wrong but, until FLINK-16686 [1] is > fixed, a class must be a POJO to be used in managed state with RocksDB, > right? That’s not to say that the approach with TypeInfoFactory won’t work, > just that even then it will mean none of the data classes can be used for > managed state. > > > > [1] https://issues.apache.org/jira/browse/FLINK-16686 > > > > Regards, > > Alexis. > > > > *From:* Matthias Pohl > *Sent:* Donnerstag, 16. September 2021 13:12 > *To:* Alex Cruise > *Cc:* Flink ML > *Subject:* Re: Fast serialization for Kotlin data classes > > > > Hi Alex, > > have you had a look at TypeInfoFactory? That might be the best way to come > up with a custom serialization mechanism. See the docs [1] for further > details. > > > > Best, > Matthias > > > > [1] > https://ci.apache.org/projects/flink/flink-docs-master/docs/dev/datastream/fault-tolerance/serialization/types_serialization/#defining-type-information-using-a-factory > > > > On Tue, Sep 14, 2021 at 8:33 PM Alex Cruise wrote: > > Hi there, > > > > I appreciate the fact that Flink has built-in support for making POJO and > Scala `case class` serialization faster, but in my project we use immutable > Kotlin `data class`es (analogous to Scala `case class`es) extensively, and > we'd really prefer not to make them POJOs, mostly for style/taste reasons > (e.g. need a default constructor and setters, both are anathema!) > > > > Does anyone know of a good way for us to keep using idiomatic, immutable > Kotlin data classes, but to get much faster serialization performance in > Flink? > > > > Thanks! > > > > -0xe1a > >
RE: Fast serialization for Kotlin data classes
Someone please correct me if I’m wrong but, until FLINK-16686 [1] is fixed, a class must be a POJO to be used in managed state with RocksDB, right? That’s not to say that the approach with TypeInfoFactory won’t work, just that even then it will mean none of the data classes can be used for managed state. [1] https://issues.apache.org/jira/browse/FLINK-16686 Regards, Alexis. From: Matthias Pohl Sent: Donnerstag, 16. September 2021 13:12 To: Alex Cruise Cc: Flink ML Subject: Re: Fast serialization for Kotlin data classes Hi Alex, have you had a look at TypeInfoFactory? That might be the best way to come up with a custom serialization mechanism. See the docs [1] for further details. Best, Matthias [1] https://ci.apache.org/projects/flink/flink-docs-master/docs/dev/datastream/fault-tolerance/serialization/types_serialization/#defining-type-information-using-a-factory On Tue, Sep 14, 2021 at 8:33 PM Alex Cruise mailto:a...@cluonflux.com>> wrote: Hi there, I appreciate the fact that Flink has built-in support for making POJO and Scala `case class` serialization faster, but in my project we use immutable Kotlin `data class`es (analogous to Scala `case class`es) extensively, and we'd really prefer not to make them POJOs, mostly for style/taste reasons (e.g. need a default constructor and setters, both are anathema!) Does anyone know of a good way for us to keep using idiomatic, immutable Kotlin data classes, but to get much faster serialization performance in Flink? Thanks! -0xe1a
Re: Fast serialization for Kotlin data classes
Hi Alex, have you had a look at TypeInfoFactory? That might be the best way to come up with a custom serialization mechanism. See the docs [1] for further details. Best, Matthias [1] https://ci.apache.org/projects/flink/flink-docs-master/docs/dev/datastream/fault-tolerance/serialization/types_serialization/#defining-type-information-using-a-factory On Tue, Sep 14, 2021 at 8:33 PM Alex Cruise wrote: > Hi there, > > I appreciate the fact that Flink has built-in support for making POJO and > Scala `case class` serialization faster, but in my project we use immutable > Kotlin `data class`es (analogous to Scala `case class`es) extensively, and > we'd really prefer not to make them POJOs, mostly for style/taste reasons > (e.g. need a default constructor and setters, both are anathema!) > > Does anyone know of a good way for us to keep using idiomatic, immutable > Kotlin data classes, but to get much faster serialization performance in > Flink? > > Thanks! > > -0xe1a >
Fast serialization for Kotlin data classes
Hi there, I appreciate the fact that Flink has built-in support for making POJO and Scala `case class` serialization faster, but in my project we use immutable Kotlin `data class`es (analogous to Scala `case class`es) extensively, and we'd really prefer not to make them POJOs, mostly for style/taste reasons (e.g. need a default constructor and setters, both are anathema!) Does anyone know of a good way for us to keep using idiomatic, immutable Kotlin data classes, but to get much faster serialization performance in Flink? Thanks! -0xe1a