Hi Rui,
AFAIK, RelNodes can be serialized to and deserialized from JSON format.
See test [1] as an example. If I understand it correct, RelNodes are
serialized along with enclosed RexNodes, so you can transfer them over
the network as plain strings.
[1]
https://github.com/apache/calcite/blob/f64cdcbb9f6535650f0227da19640e736496a9c3/core/src/test/java/org/apache/calcite/plan/RelWriterTest.java#L88
--
Roman Kondakov
On 07.07.2020 22:13, Enrico Olivelli wrote:
Rui
Il Mar 7 Lug 2020, 20:30 Rui Wang <[email protected]> ha scritto:
Hi Community,
In Apache Beam we are facing a use case where we need to keep RexNode in
our distributed primitives. Because of the nature of distributed computing,
Beam requires the usage of those primitives be serializable (thus those
primitives can be sent over the network to backend/workers for
further execution).
In the Java world this requirement means to make RexNode implement the Java
Serializable interface.
A workaround right now is to create a bunch of classes to "clone" RexNode
while making those classes implement the Serializable interface.
Did you evaluate to use some framework like Kryo that allows you to
serialize Jon serializable classes?
I think that in general Java serialisation is not efficient as it is too
general purpose.
It also brings in a few Security issues.
Maybe an alternative idea is to add some serialisation ad-hoc mechanism in
RexNode.
We should also ensure that every RexNode will be able to be serialized and
deserialized.
Enrico
So what do you think of the idea that makes RexNode implement the
Serializable interface?
-Rui
