emkornfield commented on a change in pull request #12599:
URL: https://github.com/apache/arrow/pull/12599#discussion_r824371738
##########
File path: docs/source/format/Columnar.rst
##########
@@ -208,17 +208,19 @@ right-to-left: ::
0 0 1 0 1 0 1 1
Arrays having a 0 null count may choose to not allocate the validity
-bitmap. Implementations may choose to always allocate one anyway as a
-matter of convenience, but this should be noted when memory is being
-shared.
+bitmap; how this is represented depends on the implementation (for
+example, a C++ implementation may represent such an "absent" validity
+bitmap using a NULL pointer). Implementations may choose to always allocate
+a validity bitmap anyway as a matter of convenience. Consumers of Arrow
+arrays should be ready to handle those two possibilities.
-Nested type arrays except for union types have their own validity bitmap and
-null count regardless of the null count and valid bits of their child arrays.
+Nested type arrays (except for union types as noted above) have their own
+top-level validity bitmap and null count, regardless of the null count and
+valid bits of their child arrays.
-Array slots which are null are not required to have a particular
-value; any "masked" memory can have any value and need not be zeroed,
-though implementations frequently choose to zero memory for null
-values.
+Array slots which are null are not required to have a particular value;
+any "masked" memory can have any value and need not be zeroed, though
Review comment:
This would be a change in specification but a minor one. I'd point out
that i think unless you specify zero initialized you still need to worry about
UB because some could choose MAX_INT. Where adding 1 for the signed values
would cause an overflow and UB. C++ amortized null values by finding chunks
of non null sequences 256 values at a time and dispatching to SIMD for these
cases
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
