jorgecarleitao commented on a change in pull request #12599:
URL: https://github.com/apache/arrow/pull/12599#discussion_r824398902
##########
File path: docs/source/format/Columnar.rst
##########
@@ -208,17 +208,19 @@ right-to-left: ::
0 0 1 0 1 0 1 1
Arrays having a 0 null count may choose to not allocate the validity
-bitmap. Implementations may choose to always allocate one anyway as a
-matter of convenience, but this should be noted when memory is being
-shared.
+bitmap; how this is represented depends on the implementation (for
+example, a C++ implementation may represent such an "absent" validity
+bitmap using a NULL pointer). Implementations may choose to always allocate
+a validity bitmap anyway as a matter of convenience. Consumers of Arrow
+arrays should be ready to handle those two possibilities.
-Nested type arrays except for union types have their own validity bitmap and
-null count regardless of the null count and valid bits of their child arrays.
+Nested type arrays (except for union types as noted above) have their own
+top-level validity bitmap and null count, regardless of the null count and
+valid bits of their child arrays.
-Array slots which are null are not required to have a particular
-value; any "masked" memory can have any value and need not be zeroed,
-though implementations frequently choose to zero memory for null
-values.
+Array slots which are null are not required to have a particular value;
+any "masked" memory can have any value and need not be zeroed, though
Review comment:
In Rust overflowing panics (doesn't UB)
```rust
#![allow(arithmetic_overflow)] // the linter does not even compile the below
without this ^^
fn main() {
let a = i32::MAX + 1; // this panics
}
```
but we usually use either saturating or wrapping as we do not want the query
engine to panic if the user performs an operation that would panic in a
non-null slot.
fwiw `numpy` uses wrapping: `import numpy; a = numpy.array([2147483647],
dtype=numpy.int32); print(a + 1) // [-2147483648]`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
