amogh-jahagirdar commented on PR #4925:
URL: https://github.com/apache/iceberg/pull/4925#issuecomment-1260144084
Thanks @wmoustafa for your thoughts and sorry for the late reply. So here
are my thoughts:
> Following this line of thought, we should probably also address the
question of whether the SQL text should be validated in Iceberg/the API
implementation, or validation is left to the enclosing engine. If it is
validated in Iceberg/the API implementation, then Iceberg/implementation needs
to understand the dialects as well. If it is left to the enclosing engine,
there is the risk of passing an invalid view text.
Yes, after some thought, my thinking is in the long run the Iceberg View
format should enforce correctness and semantic equality between the different
representations. However, I think for an initial implementation though it
suffices to just put it on the engine or translation layer to validate. Later
on, validation can be implemented within the View API.
>By the way, I believe adding new representations (e.g., dialects) of
semantically equivalent views (case 2 above) should not increment the view
version. Do you agree? It seems this might result in having to distinguish
between view representations that only add a dialect and preserve the semantics
and view representations that change the semantics.
So view versions in my mind should represent every single change to the
view, similar to there's a new table metadata version regardless of if there is
a new snapshot or not. Another argument is that view version should only
increment for semantic or schema changes in the view (which if i'm not mistaken
is what you're suggesting), but I think the mental model would be hard to grasp
here when people try and compare with Iceberg table specification.
Also for an initial implementation if we don't have any validation
controls, then I think it becomes necessary to increment on any new version
representations which are updated, so that people can rollback the view if it's
in a state they do not like.
I'm on board with having a separated updateViewRepresentations which has a
"merge" like semantic for updating view representations (either replacing
existing ones of the same type and dialect OR if non-existing, create the new
representation)
Thoughts @jzhuge @danielcweeks ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
