Hi all,
After some discussion with Steve, we'd like to propose and get feedback on an
alternative to representing expressions entirely with flatbuffers.
To give some context, we thought about how we'd construct flatbuffer
expressions in Java or another language if we went down that route. We realized
that it'd be possible, but not user friendly. An example is specifying an array
of int values in Java for an InExpression. In Java, we'd ideally have some
user-friendly class (i.e. arrow's IntVector) that then gets converted to the
appropriate flatbuffer representation. I think this is what Jacques was saying
about language support being too weak - it's possible for Java users to
construct a flatbuffer expression, but not easily without an additional
conversion layer for every language.
An alternative we're thinking about is to only represent enum values (i.e.
those defined in arrow::dataset::ExpressionType::type) in a flatbuffer schema,
and rely on the existing IPC format (used to serialize/deserialize cpp
expressions) to pass the struct array representation of an expression from for
example Java to C++. The one difference is in the struct array representation,
we use the enum values defined in our flatbuffer schema instead of existing cpp
enums. This approach requires us on the Java side (and languages other than
C++) to construct the struct array, but the benefit is minimal changes to the
C++ code (the main change being using our flatbuffer schema enum values).
On 2020/07/13 09:21:19, Antoine Pitrou <solip...@pitrou.net> wrote:
> On Sat, 11 Jul 2020 09:55:16 -0700
> Jacques Nadeau <jacq...@apache.org> wrote:
> >
> > I'm against extending use of flatbuf within Arrow. The language support is
> > too weak. Language support isn't just about having a binding for different
> > languages, it is about having a high-quality binding.
>
> Could you please expand on this? ("the language support is too weak")
>
> Thank you
>
> Antoine.
>
>
>
This e-mail and any attachments may contain information that is confidential
and proprietary and otherwise protected from disclosure. If you are not the
intended recipient of this e-mail, do not read, duplicate or redistribute it by
any means. Please immediately delete it and any attachments and notify the
sender that you have received it by mistake. Unintended recipients are
prohibited from taking action on the basis of information in this e-mail or any
attachments. The DRW Companies make no representations that this e-mail or any
attachments are free of computer viruses or other defects.
