emkornfield commented on code in PR #461: URL: https://github.com/apache/parquet-format/pull/461#discussion_r1854914038
########## VariantEncoding.md: ########## @@ -39,13 +39,41 @@ Another motivation for the representation is that (aside from metadata) each nes For example, in a Variant containing an Array of Variant values, the representation of an inner Variant value, when paired with the metadata of the full variant, is itself a valid Variant. This document describes the Variant Binary Encoding scheme. -[VariantShredding.md](VariantShredding.md) describes the details of the Variant shredding scheme. +The [Variant Shredding specification](VariantShredding.md) describes the details of shredding Variant values as typed Parquet columns. Review Comment: Varia ```suggestion Variant fields can also be "shredded". Shredding refers to extracting some elements of the variant into separate columns for more efficient extraction/filter pushdown. The [Variant Shredding specification](VariantShredding.md) describes the details of shredding Variant values as typed Parquet columns. ``` I think this is the first place shredding is used and I think we should try to give readers a background on what is meant here? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
