ebyhr commented on code in PR #11041:
URL: https://github.com/apache/iceberg/pull/11041#discussion_r4624793478


##########
format/view-spec.md:
##########
@@ -42,12 +42,28 @@ An atomic swap of one view metadata file for another 
provides the basis for maki
 
 Writers create view metadata files optimistically, assuming that the current 
metadata location will not be changed before the writer's commit. Once a writer 
has created an update, it commits by swapping the view's metadata file pointer 
from the base location to the new location.
 
+### Materialized Views
+
+Materialized views are a type of view with precomputed results from the view 
query stored as a table.
+When queried, engines may return the precomputed data for the materialized 
views, shifting the cost of query execution to the precomputation step.
+
+Iceberg materialized views are implemented as a combination of an Iceberg view 
and an underlying Iceberg table, the "storage-table", which stores the 
precomputed data.
+Materialized View metadata is a superset of View metadata with an additional 
pointer to the storage table. The storage table is an Iceberg table with 
additional materialized view refresh state metadata.
+This addition is backward compatible: a reader that is unaware of materialized 
views sees the unknown `storage-table` field, ignores it, and evaluates the 
view query directly.

Review Comment:
   One concern about the compatibility claim. @InvisibleProgrammer asked about 
bumping `format-version` in 
https://github.com/apache/iceberg/pull/11041#issuecomment-3709796824, and 
@stevenzwu replied in 
https://github.com/apache/iceberg/pull/11041#issuecomment-3873040325 that it 
isn't needed since there are no breaking changes. That's true for readers, but 
there's a problem on the writer side.
   
   An MV-unaware writer (e.g. an older Iceberg Java client running `ALTER VIEW 
... SET TBLPROPERTIES` against a non-REST catalog) would silently remove 
`storage-table`, since it drops unknown fields when rewriting the metadata. The 
MV would become a plain view and the storage table would be orphaned. I don't 
think we can prevent this with spec text, because old libraries can't tell an 
MV from a view.
   
   What do you think about revisiting the bump? Old engines would then fail on 
MVs instead of silently dropping them.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to