jackye1995 commented on PR #4925: URL: https://github.com/apache/iceberg/pull/4925#issuecomment-1255328952
> Following this line of thought, we should probably also address the question of whether the SQL text should be validated in Iceberg/the API implementation, or validation is left to the enclosing engine. If it is validated in Iceberg/the API implementation, then Iceberg/implementation needs to understand the dialects as well. If it is left to the enclosing engine, there is the risk of passing an invalid view text. I would vote for leaving to the enclosing engine or translation tool to validate the correctness and semantic equivalence in the initial API implementation. Imagine in the most manual way, there is a data engineer manually adding 2 representations for Spark and Trino, and the engineer does the manual translation, and the translation might not be perfect but works for their use case. Then the engineer invokes this API to update the Iceberg view spec. We should make sure at least this use case can proceed without issue without Iceberg throwing unnecessary validation exceptions. We can always introduce features like `CoralValidatedSqlRepresentation` for people who would like to use certain tool, and plugin points in the Iceberg library, such that the integrations could live as plugin packages that don't even need to live in the Iceberg repository. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
