findinpath commented on code in PR #11041: URL: https://github.com/apache/iceberg/pull/11041#discussion_r3967960238
########## format/view-spec.md: ########## @@ -160,7 +178,116 @@ Each entry in `version-log` is a struct with the following fields: | _required_ | `timestamp-ms` | Timestamp when the view's `current-version-id` was updated (ms from epoch) | | _required_ | `version-id` | ID that `current-version-id` was set to | -## Appendix A: An Example +#### Storage Table Identifier + +The table identifier for the storage table that stores the precomputed results. + +| Requirement | Field name | Description | +|-------------|----------------|-------------| +| _required_ | `namespace` | A list of strings for namespace levels | +| _required_ | `name` | A string specifying the name of the table | + +### Storage table metadata + +This section describes additional metadata for the storage table that supplements the regular table metadata and is required for materialized views. +The `refresh-state` property is set on the [snapshot summary](https://iceberg.apache.org/spec/#snapshots) property of a storage table snapshot to provide information about the state of the precomputed data. + +| Requirement | Field name | Description | +|-------------|-----------------|-------------| +| _optional_ | `refresh-state` | A [refresh state](#refresh-state) record stored as a JSON-encoded string | + +#### Freshness + +A materialized view is **fresh** when the storage table represents the result of the current view query. However, consumers may still decide to consume from a stale storage table based on their own policies. Review Comment: Iceberg snapshots internally contain the `schema-id` that was active at the time the snapshot was created. Therefore, to track schema evolution, engines do not need extra schema information injected into the refresh-state. During freshness evaluation, the query engine simply: - Looks at the source table snapshot-id recorded in the MV's refresh-state. - Checks the schema-id tied to that specific snapshot in the source table. - Compares it against the source table's current schema-id. If the schema IDs differ (e.g., a column was renamed or a default value was added), the engine immediately knows the MV is semantically stale or requires validation, even if the underlying data files haven't changed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
