[
https://issues.apache.org/jira/browse/HIVE-24666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17274497#comment-17274497
]
Zhihua Deng commented on HIVE-24666:
------------------------------------
Correct me if wrong...
Before the patch, queries like
{code:java}
select count (distinct cint) from alltypesorc where cast(cint as boolean);{code}
the filter *_cast(cint as boolean)_* would be converted to
"_SelectColumnIsTrue(col 13:boolean)(children:
VectorUDFAdaptor(UDFToBoolean(cint)) -> 13:boolean)" ._ This is caused by the
vectored expressions of UDFToBoolean all in project mode, so an adapter will be
here.
For _cint1 or cint2_, SelectColumnIsTrue may not be needed, but this may make
things a bit difficult as we should take some actions when the expression *or*
cannot be converted to vectorized filter, we should put a cast to the children
and try again. For example:
{code:java}
select count (distinct cint) from alltypesorc where cfloat or cint; {code}
the filter _*cfloat or cint*_ would be converted into **
_"SelectColumnIsTrue(col 13:boolean)(children: VectorUDFAdaptor((cfloat or
cint)) -> 13:boolean)"_, a ClassCastException can be seen at runtime because
cfloat/cint is not the bool type, there may be some other cases that cannot be
converted to the vectorized filter of ** GenericUDFOPOr and result to such
problem.
For these cases, a simple useful method is that we insert a bool type
conversion when generating the expression, this can avoid the
ClassCastException at runtime, make such filters be converted to vectorized，and
also benefit the queries running on non vectorization mode.
> Vectorized UDFToBoolean may unable to filter rows if input is string
> --------------------------------------------------------------------
>
> Key: HIVE-24666
> URL: https://issues.apache.org/jira/browse/HIVE-24666
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
> Reporter: Zhihua Deng
> Assignee: Zhihua Deng
> Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-24666.2.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> If we use cast boolean in where conditions to filter rows, in vectorization
> execution the filter is unable to filter rows, step to reproduce:
> {code:java}
> create table vtb (key string, value string);
> insert into table vtb values('0', 'val0'), ('false', 'valfalse'),('off',
> 'valoff'),('no','valno'),('vk', 'valvk');
> select distinct value from vtb where cast(key as boolean); {code}
> It's seems we don't generate a SelectColumnIsTrue to filter the rows if the
> casted type is string:
>
> https://github.com/apache/hive/blob/ff6f3565e50148b7bcfbcf19b970379f2bd59290/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java#L2995-L2996
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
