[
https://issues.apache.org/jira/browse/SPARK-33317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17225141#comment-17225141
]
Liu Neng commented on SPARK-33317:
----------------------------------
I run these sql on spark 3.0.0, condition 1 +(between ' 1000405134' and
'1000772585')+ find 6012 records, condition 2 ++(between '1000405134' and
'1000772585'++) find 2798 records.
I find that comparator in codegen is UTF8String
!image-2020-11-03-13-30-12-049.png!
" 1000405134" is smaller than "1000405134"
I think that it isn't an issue, because comparing value is String not Number.
I tried to analyze the parse tree, "1000405134" is a String literal.
> Spark Hive SQL returning empty dataframe
> ----------------------------------------
>
> Key: SPARK-33317
> URL: https://issues.apache.org/jira/browse/SPARK-33317
> Project: Spark
> Issue Type: Bug
> Components: Spark Core, Spark Shell
> Affects Versions: 2.4.6
> Reporter: Debadutta
> Priority: Major
> Attachments: farmers.csv, image-2020-11-03-13-30-12-049.png
>
>
> I am trying to run a sql query on a hive table using hive connector in spark
> but I am getting an empty dataframe. The query I am trying to run:-
> {{sparkSession.sql("select fmid from farmers where fmid between ' 1000405134'
> and '1000772585'")}}
> This is failing but if I remove the leading whitespaces it works.
> {{sparkSession.sql("select fmid from farmers where fmid between '1000405134'
> and '1000772585'")}}
> Currently, I am removing leading and trailing whitespaces as a workaround.
> But the same query with whitespaces works fine in hive console.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
