kbendick opened a new issue, #5349: URL: https://github.com/apache/iceberg/issues/5349
We need to implement Spark’s `FunctionCatalog` so that we can use the partition transformation functions in queries. This allows for using the partition transforms on non-partition columns in generated code. This is necessary in order to write Catalyst rules which will pass `bucket` So that storage partitioned joins (aka bucketed joins) can be implemented. See also: - [FunctionCatalog](https://spark.apache.org/docs/latest//api/java/index.html?org/apache/spark/sql/connector/catalog/FunctionCatalog.html) : https://spark.apache.org/docs/latest//api/java/index.html?org/apache/spark/sql/connector/catalog/FunctionCatalog.html) - [ScalarFunction](https://spark.apache.org/docs/3.2.0/api/java/org/apache/spark/sql/connector/catalog/functions/ScalarFunction.html) class, which has practical description of what is needed for codegeneration and the benefits of it. The functions we have that are likely highest priority: - truncate - bucket - zorder - date transformations -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
