RussellSpitzer commented on pull request #3375:
URL: https://github.com/apache/iceberg/pull/3375#issuecomment-972115074
In the Datasource code in Spark3 it uses this little bit
``` val filters =
DataSourceStrategy.normalizeExprs(condition.toSeq, output)
.flatMap(splitConjunctivePredicates(_).map {
f => DataSourceStrategy.translateFilter(f, true).getOrElse(
throw
QueryCompilationErrors.cannotTranslateExpressionToSourceFilterError(f))
}).toArray
```
To convert to Source Filters, I think this is a little simpler than
reimplementing the Spark Expressions -> Iceberg Expressions directly. I just
figure this way, we will have less direct dependencies on Spark code.
This is protected but it may make sense for us to expose it in our
extensions for this purpose.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
