Rui Il Mar 7 Lug 2020, 20:30 Rui Wang <[email protected]> ha scritto:
> Hi Community, > > In Apache Beam we are facing a use case where we need to keep RexNode in > our distributed primitives. Because of the nature of distributed computing, > Beam requires the usage of those primitives be serializable (thus those > primitives can be sent over the network to backend/workers for > further execution). > > In the Java world this requirement means to make RexNode implement the Java > Serializable interface. > > A workaround right now is to create a bunch of classes to "clone" RexNode > while making those classes implement the Serializable interface. > Did you evaluate to use some framework like Kryo that allows you to serialize Jon serializable classes? I think that in general Java serialisation is not efficient as it is too general purpose. It also brings in a few Security issues. Maybe an alternative idea is to add some serialisation ad-hoc mechanism in RexNode. We should also ensure that every RexNode will be able to be serialized and deserialized. Enrico > So what do you think of the idea that makes RexNode implement the > Serializable interface? > > > -Rui >
