ConeyLiu commented on code in PR #8394:
URL: https://github.com/apache/iceberg/pull/8394#discussion_r1329951681
##########
spark/v3.4/spark/src/main/scala/org/apache/spark/sql/execution/datasources/SparkExpressionConverter.scala:
##########
@@ -36,7 +36,14 @@ object SparkExpressionConverter {
// Currently, it is a double conversion as we are converting Spark
expression to Spark predicate
// and then converting Spark predicate to Iceberg expression.
// But these two conversions already exist and well tested. So, we are
going with this approach.
-
SparkV2Filters.convert(DataSourceV2Strategy.translateFilterV2(sparkExpression).get)
+ DataSourceV2Strategy.translateFilterV2(sparkExpression) match {
+ case Some(filter) =>
+ val converted = SparkV2Filters.convert(filter)
+ assert(converted != null, s"Cannot convert Spark filter to Iceberg
expression: $filter")
Review Comment:
Updated
##########
spark/v3.4/spark/src/main/scala/org/apache/spark/sql/execution/datasources/SparkExpressionConverter.scala:
##########
@@ -36,7 +36,14 @@ object SparkExpressionConverter {
// Currently, it is a double conversion as we are converting Spark
expression to Spark predicate
// and then converting Spark predicate to Iceberg expression.
// But these two conversions already exist and well tested. So, we are
going with this approach.
-
SparkV2Filters.convert(DataSourceV2Strategy.translateFilterV2(sparkExpression).get)
+ DataSourceV2Strategy.translateFilterV2(sparkExpression) match {
+ case Some(filter) =>
+ val converted = SparkV2Filters.convert(filter)
+ assert(converted != null, s"Cannot convert Spark filter to Iceberg
expression: $filter")
+ converted
+ case _ =>
+ throw new IllegalArgumentException(s"Cannot translate Spark expression
to data source filter: $sparkExpression")
Review Comment:
Updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
