frankli created SPARK-37051:
-------------------------------
Summary: The filter operator gets wrong results in ORC
char/varchar types
Key: SPARK-37051
URL: https://issues.apache.org/jira/browse/SPARK-37051
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 3.1.2
Environment: Spark 3.1.2
Scala 2.12 / Java 1.8
Reporter: frankli
When I try the following sample SQL on the TPCDS data, the filter operator
returns an empty row set (shown in web ui).
_select * from item where i_category = 'Music' limit 100;_
The table is in ORC format, and i_category is char(50) type.
I guest that the char(50) type will remains redundant blanks after the actual
word.
It will affect the boolean value of "x.equals(Y)", and results in wrong
results.
By the way, Spark's tests should add more cases on ORC format.
!image-2021-10-19-11-01-55-597.png|width=1085,height=499!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
