ebyhr commented on code in PR #11041: URL: https://github.com/apache/iceberg/pull/11041#discussion_r4636673970
########## format/view-spec.md: ########## @@ -42,12 +42,28 @@ An atomic swap of one view metadata file for another provides the basis for maki Writers create view metadata files optimistically, assuming that the current metadata location will not be changed before the writer's commit. Once a writer has created an update, it commits by swapping the view's metadata file pointer from the base location to the new location. +### Materialized Views + +Materialized views are a type of view with precomputed results from the view query stored as a table. +When queried, engines may return the precomputed data for the materialized views, shifting the cost of query execution to the precomputation step. + +Iceberg materialized views are implemented as a combination of an Iceberg view and an underlying Iceberg table, the "storage-table", which stores the precomputed data. +Materialized View metadata is a superset of View metadata with an additional pointer to the storage table. The storage table is an Iceberg table with additional materialized view refresh state metadata. +This addition is backward compatible: a reader that is unaware of materialized views sees the unknown `storage-table` field, ignores it, and evaluates the view query directly. Review Comment: Thanks, the `refs` precedent is a fair point, though I'd see it as a risk we accepted rather than a pattern to follow, and today far more engines and library versions read and write views. Silently turning an MV into a view isn't really graceful for MV-aware engines. `SHOW CREATE MATERIALIZED VIEW`, `REFRESH MATERIALIZED VIEW` and `DROP MATERIALIZED VIEW` would start failing, and users would have to fall back to `DROP VIEW`, which leaves the storage table orphaned. These failures would show up on upgraded engines, far from the old writer that caused them, while a format bump only makes old engines fail, with a clear error. I like the writer requirement as an addition, since it helps future implementations. I just wonder if it's enough on its own, given that already-released writers wouldn't be covered. Happy to go with whatever the group thinks is best, though. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
