GitHub user ryyyyyy1 added a comment to the discussion: [Flink] About SARG
Support in Flink and BaseVector-related Issues
For example, in q22, when dealing with expression lower(channel) in ('apple',
'google', 'facebook', 'baidu'). in Flink, it would be SARG['apple', 'google',
'facebook', 'baidu']. currently it will be wrapped in ConstantTypedExpr with
the format of string. when passing to velox, this string format cannot be
handled properly. In Velox, it seems that this kind of filter condition should
be in the format of BaseVector. but i noticed that in the definition of
ConstantTypedExpr, valueVector is reserved, which allow us to wrapp a
serializedVector in ConstantTypedExpr, which can be handled by velox. So my
idea is: when meets SARG['apple', 'google', 'facebook', 'baidu'], create a
BaseVector in planner stage by calling JNI api. . serialize it and return the
serializedVector string to flink and wrapped it into the ConstantTypedExpr in
java. During runtime, this serializedVector string will be deserialized and
handled properly.
GitHub link:
https://github.com/apache/incubator-gluten/discussions/10406#discussioncomment-14305491
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
