keirsalterego commented on issue #6528: URL: https://github.com/apache/arrow-rs/issues/6528#issuecomment-3724793516
hey @alamb @tustvold I'd like to pick this up. I read through the thread and I get the safety concern regarding `DictionaryArray` and `unmasking null slots`. To avoid hitting any UB or reading garbage data, I’m planning to implement `with_nulls` using a subset invariant check. The logic would ensure that the new `NullBuffer` is a subset of the existing validity (i.e., we can turn valid slots into nulls, but we cannot turn null slots back into valid values). This satisfies the primary use case mentioned (combining existing nulls with a new filter) while guaranteeing we never typically unmask garbage or undefined data. Does that approach sound good to you? If so, I’ll start with the primitive arrays. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
