alamb opened a new issue, #18270:
URL: https://github.com/apache/datafusion/issues/18270
### Is your feature request related to a problem or challenge?
https://docs.rs/datafusion-proto/latest/datafusion_proto/#version-compatibility
says:
> The serialized form are not guaranteed to be compatible across DataFusion
versions. A plan serialized with one version of DataFusion may not be able to
deserialized with a different version.
However, as these messages become more widely used (e.g. datafusion-proto is
now used to send `Expr`s over the FFI boundary, as mentioned by @timesaucer) so
having them be incompatible may cause bugs / unintended side effects
Also, we have some recent contributions such as the following from @tobixdev
and @paleolimbot that actually do attempt to maintain backwards compatibility
- https://github.com/apache/datafusion/pull/17986/files#r2442433612
Given the protobuf definitions aren't changing all that frequently anymore I
think it may be worth reconsidering our stance on compatibility
### Describe the solution you'd like
1. Consider if we should make a stronger statement about backwards
compatibility of protobuf serialized Exprs
### Describe alternatives you've considered
Maybe we can use substrait for this?
If we do choose to make a stronger statement, we should also consider how we
can test / verify it (aka perhaps check in stored protobuf exprs and ensure we
can continue to serialize them in subsequent releases)
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
