Have you tried to use the existing expression representation provided by
Gandiva? What are the issues you've seen with it?
On Wed, Jul 22, 2020 at 10:24 AM Patrick Pai <p...@drwholdings.com> wrote:
> Hi all,
>
> After some discussion with Steve, we'd like to propose and get feedback on
> an alternative to representing expressions entirely with flatbuffers.
>
> To give some context, we thought about how we'd construct flatbuffer
> expressions in Java or another language if we went down that route. We
> realized that it'd be possible, but not user friendly. An example is
> specifying an array of int values in Java for an InExpression. In Java,
> we'd ideally have some user-friendly class (i.e. arrow's IntVector) that
> then gets converted to the appropriate flatbuffer representation. I think
> this is what Jacques was saying about language support being too weak -
> it's possible for Java users to construct a flatbuffer expression, but not
> easily without an additional conversion layer for every language.
>
> An alternative we're thinking about is to only represent enum values (i.e.
> those defined in arrow::dataset::ExpressionType::type) in a flatbuffer
> schema, and rely on the existing IPC format (used to serialize/deserialize
> cpp expressions) to pass the struct array representation of an expression
> from for example Java to C++. The one difference is in the struct array
> representation, we use the enum values defined in our flatbuffer schema
> instead of existing cpp enums. This approach requires us on the Java side
> (and languages other than C++) to construct the struct array, but the
> benefit is minimal changes to the C++ code (the main change being using our
> flatbuffer schema enum values).
>
>
> On 2020/07/13 09:21:19, Antoine Pitrou <solip...@pitrou.net> wrote:
> > On Sat, 11 Jul 2020 09:55:16 -0700
> > Jacques Nadeau <jacq...@apache.org> wrote:
> > >
> > > I'm against extending use of flatbuf within Arrow. The language
> support is
> > > too weak. Language support isn't just about having a binding for
> different
> > > languages, it is about having a high-quality binding.
> >
> > Could you please expand on this? ("the language support is too weak")
> >
> > Thank you
> >
> > Antoine.
> >
> >
> >
>
> This e-mail and any attachments may contain information that is
> confidential and proprietary and otherwise protected from disclosure. If
> you are not the intended recipient of this e-mail, do not read, duplicate
> or redistribute it by any means. Please immediately delete it and any
> attachments and notify the sender that you have received it by mistake.
> Unintended recipients are prohibited from taking action on the basis of
> information in this e-mail or any attachments. The DRW Companies make no
> representations that this e-mail or any attachments are free of computer
> viruses or other defects.
>
