GitHub user cloud-fan opened a pull request:
https://github.com/apache/spark/pull/21470
[SPARK-24443][SQL] comparison should accept structurally-equal types
## What changes were proposed in this pull request?
When comparing struct type values, it's a little hard to make the field
names same. Simple queries like
`SELECT (a, b) = (1, 2) FROM t` may fail because the field names do not
match.
In Postgres, when comparing struct type values, it will do safe type
coercion and ignore field name difference
```
# create table t(i smallint, j smallint);
CREATE TABLE
# select Row(i, j) = Row(1, 1) from t;
?column?
----------
t
(1 row)
# select Row(i, j) < Row(1, 1) from t;
?column?
----------
f
(1 row)
# select Row(i, j) = Row(j, i) from t;
?column?
----------
t
(1 row)
```
This PR follows Postgres and accept structurally-equal types in comparison
TODO:
* check Hive and Presto.
* think about array/map type. Postgres doesn't support type coercion for
elements in array/map when comparison.
## How was this patch tested?
new test cases.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cloud-fan/spark equal
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21470.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21470
----
commit 3555e0fd1a8003af0e6a7694aab8999698aab9c4
Author: Wenchen Fan <wenchen@...>
Date: 2018-05-31T18:00:42Z
comparison should accept structurally-equal types
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
