I want to apologize for the extra spam from replying from email.. Ignore my previous comments..
The definition for struct validity is documented here.. https://arrow.apache.org/docs/format/Columnar.html#struct-validity But I think this statement below is logically flawed when a child array is also a struct.. Therefore, to know whether a particular child entry is valid, one must take the logical AND of the corresponding bits in the two validity bitmaps (the struct array’s and the child array’s). If you have multiple layers of nesting it will be problematic to check cascading nullability masks for child D + parent C, child C + parent B, child B + parent A, etc.. Even though it isn't stated, all child arrays (fields) must have the exact same length as the parent struct array which is also problematic if you are injecting NULLs into a non-nullable child array because the parent array's field for that array is nullable. This could break data validation checks on child arrays if you are adding dummy values to it. For example: if you have a child array that represents car models. One parent called array called SUVs may just want to take the child array entries where the car models are SUVs. A different parent called Personal Vehicles may also want to use the same child array entries, but it may also contain NULL entries for people who don't own a car. I think the right solution is to support uneven child arrays in a struct array using a positional index that is similar with how string arrays use an offset index. The inner child Cars struct could have a value of [model:Tesla, model:Ford, model:Jeep] But the outer parent Personal Vehicles could have [(name: Bob, model: Tesla), (name: Jane, model: Tesla), (name: Joe, model: Jeep), (name: Sue, None). To support this, you would need a positional index of [0, 0, 2, None] to support nested structs without having to inject None or duplicate values in the child struct to match the parent length. This keeps the child struct intact. The alternative if you want to even out the child struct is would be to inject Nones and duplicate values to match the parent length, but for this scenario I think you would automatically copy or change the child struct fields to nullable if the parent field is also nullable to eliminate the need to check both the parent and child nullability masks. This message may contain information that is confidential or privileged. If you are not the intended recipient, please advise the sender immediately and delete this message. See https://www.blackrock.com/corporate/compliance/email-disclaimers for further information. Please refer to https://www.blackrock.com/corporate/compliance/privacy-policy for more information about BlackRock’s Privacy Policy. For a list of BlackRock's office addresses worldwide, see https://www.blackrock.com/corporate/about-us/contacts-locations. © 2026 BlackRock, Inc. All rights reserved.
