vibhatha commented on issue #40935: URL: https://github.com/apache/arrow/issues/40935#issuecomment-2110016278
## Usage of `TypeLayout.getTypeBufferCount` 1. `VectorUnloader` → `appendNodes` : used to validate the field buffer size against the defined size in the layout 2. `ValidateVectorBufferVisitor` → `validateVectorComon` → (this method is used to validate most vector types. Where it uses the field`s buffer size against the layouts defined buffer count. 3. `StructVectorLoader` → `loadBuffers` used to get the number of buffers to give the initial capacity to the list 4. `StructVectorUnloader` → `appendNodes` : used to validate the field buffer size against the defined size in the layout 5. `VectorLoader` → `loadBuffers` : used to get the number of buffers to give the initial capacity to the list 6. `ValidateVectorBufferVisitor` → `validateVectorCommon` : used to validate the field buffer size against the defined size in the layout The value set for the visitor here is a fixed value (number of buffers), but for StringView, the number of buffers are 2 + K and we can determine the value of K only by getting the `vector.dataBuffers.size()` which is not accessible in the visitor here. I can think of three ways of solving this. ### Method 1 Check whether the number of buffers are `2 <=` such that setting the `getTypeBufferCount` to 2 and do the validation. And in each scenario where we do the validation. And when a list is formulated by iterating through buffers, we could do it until the iterator gives null and populate the list only if the type is `Utf8View` or `BinaryView`. ### Method 2 We state validation is not supported, but update the list until iterator gives null. ### Method 3 We write a special case for StringView such that we get `FieldVector` and cast it to `BaseVariableWidthViewVector` and get the `dataBuffers` and get the size. And then we can do the validation and initialize the list capacity. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
