stefan-grafberger commented on code in PR #635: URL: https://github.com/apache/parquet-format/pull/635#discussion_r4470139634
########## LogicalTypes.md: ########## @@ -712,8 +713,9 @@ If set and non-zero, readers must seek to this offset to retrieve the referenced The byte length of the referenced data. Must be zero or a positive integer if set; a value of 0 indicates empty referenced data. `size` must be set whenever `offset` is set. -It may be omitted only for a whole-file external reference (`uri` set, `offset` not set), -in which case the range runs to the end of the referenced file. +It may be omitted for a whole-file external reference (`uri` set, `offset` not set), +in which case the range runs to the end of the referenced file. `size` may also be set +with `inline` and without `uri`, in which case it must equal the length of `inline`. Review Comment: Yes, good point, that applies when size is missing for some values. But I think the main benefit is that writers get the option to write size, and then a simple user query like `SELECT file.size FROM t` cheaply answers the size question for inline values too, without touching inline. Today that's not possible, since size isn't allowed with inline alone. An engine that wants to offer a function like `FILE_SIZE(file)` that always returns the size could rewrite it internally to `file.size` or `COALESCE(file.size, LENGTH(file.inline))`, depending on the null count of the size column. But that's optional; the spec change only makes cheap size queries possible for engines and writers that want them. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
