gaborkaszab commented on PR #15045: URL: https://github.com/apache/iceberg/pull/15045#issuecomment-3748650668
An additional thought I had: Wouldn't requiring different ETag for different snapshot modes make it too complicated the ETag generation on server-side? Currently it's convenient to use the metadata location for instance as an input (e.g. hash the location) but with this requirement a new approach is needed. Potential options: 1) Append the snapshot-mode after the metadata location and hash them as a string. This is the cheapest option to detect early that the server can respond with a 304. However, this isn't future-proof in a sense that every time there is a new loadTableRequest parameter, the server have to change their logic to include that into the calculation. 2) Construct table metadata, convert it to a string format and hash it. This is future-proof, but seems expensive to calculate and also won't give the server the chance to return early with a 304, it has to go all the way (except the creds and configs part of the response). Because of this, I'm somewhat hesitant with this new requirement. Also there is no gain on the client-side either as any client-side cache has to include snapshot-mode into the cache key to avoid overwriting the table in the cache between REFS and ALL modes. (Imagine we have alternating requests, REFS, ALL, REFS, and we use TableIdentifier as key and ETag to decide if the table changed. We'd add table with REFS only first, then overwrite it with all snapshots, then again overwrite it back to refs only, etc.) So even though theoretically this makes sense and aligns well with the definition of ETags, I don't see the gain, and would introduce difficulties on the server-side. WDYT? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
