[
https://issues.apache.org/jira/browse/SPARK-10837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14981391#comment-14981391
]
Xiu Guo commented on SPARK-10837:
---------------------------------
Looks like this is a conscious rule in current code to promote left side to
string in this case:
{quote}
select * from testTable where createtime >= "2015-09-02 09:06:00.000";
{quote}
This is to accommodate the scenario like this:
{quote}
i.e. TimeStamp(2013-01-01 00:00 ...) < "2014" = true
{quote}
Promoting differently for equality and inequality cases does pose the problem
that '>=' is not a union of '=' and '>' any more.
Comparing with Hive's behavior, Hive promotes left side to string type in all
the three cases (=, > and >=). Thus return 0 rows in this case:
{quote}
select * from testTable where createtime = "2015-09-02 09:06:00.000";
{quote}
Which also does not feel completely right.
I am curious to know what is the expected behavior here.
> TimeStamp could not work on sparksql very well
> ----------------------------------------------
>
> Key: SPARK-10837
> URL: https://issues.apache.org/jira/browse/SPARK-10837
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.5.0
> Reporter: Yun Zhao
>
> create a file as follows：
> {quote}
> 2015-09-02 09:06:00.000
> 2015-09-02 09:06:00.001
> 2015-09-02 09:06:00.100
> 2015-09-02 09:06:01.000
> {quote}
> Then upload it to hdfs, for example,put it to /test/testTable.
> create table：
> {quote}
> CREATE EXTERNAL TABLE `testTable`(`createtime` timestamp) LOCATION
> '/test/testTable';
> {quote}
> process sqls：
> {quote}
> select * from testTable where createtime = "2015-09-02 09:06:00.000";
> select * from testTable where createtime > "2015-09-02 09:06:00.000";
> select * from testTable where createtime >= "2015-09-02 09:06:00.000";
> {quote}
> The set of ">=" is not union set of "=" and ">".
> but if process sqls as follows：
> {quote}
> select * from testTable where createtime = timestamp("2015-09-02
> 09:06:00.000");
> select * from testTable where createtime > timestamp("2015-09-02
> 09:06:00.000");
> select * from testTable where createtime >= timestamp("2015-09-02
> 09:06:00.000");
> {quote}
> There's no such former problem.
> User *explain extended* to find the difference of sqls:
> When uses "=","2015-09-02 09:06:00.000" is transfered to timestamp.
> When uses ">" or ">=",createtime is transfered to String.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
