szehon-ho commented on PR #5113:
URL: https://github.com/apache/iceberg/pull/5113#issuecomment-1165055509
By the way, was playing around and was wondering, are comparison for struct
literal expected to be supported in Spark/Iceberg?
Today in Spark SQL:
select * from table where table.struct_field = struct(10)
> org.apache.spark.sql.AnalysisException: cannot resolve
'(table.struct_field = struct(10))' due to data type mismatch: differing types
in '(table.struct_field = struct(1))' (struct<nested:int> and
struct<col1:int>).; line 1 pos 39;
select * from table where table.struct_field in (struct(10))
```
java.lang.IllegalArgumentException: Cannot create expression literal from
org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema: [1]
at org.apache.iceberg.expressions.Literals.from(Literals.java:87)
at
org.apache.iceberg.expressions.UnboundPredicate.<init>(UnboundPredicate.java:40)
at org.apache.iceberg.expressions.Expressions.equal(Expressions.java:175)
at org.apache.iceberg.spark.SparkFilters.handleEqual(SparkFilters.java:239)
at org.apache.iceberg.spark.SparkFilters.convert(SparkFilters.java:152)
at
org.apache.iceberg.spark.source.SparkScanBuilder.pushFilters(SparkScanBuilder.java:106)
at
org.apache.spark.sql.execution.datasources.v2.PushDownUtils$.pushFilters(PushDownUtils.scala:69)
at
org.apache.spark.sql.execution.datasources.v2.V2ScanRelationPushDown$$anonfun$pushDownFilters$1.applyOrElse(V2ScanRelationPushDown.scala:60)
at
org.apache.spark.sql.execution.datasources.v2.V2ScanRelationPushDown$$anonfun$pushDownFilters$1.applyOrElse(V2ScanRelationPushDown.scala:47)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
