TheNeuralBit commented on a change in pull request #12426:
URL: https://github.com/apache/beam/pull/12426#discussion_r463311197
##########
File path: model/pipeline/src/main/proto/beam_runner_api.proto
##########
@@ -855,10 +855,21 @@ message StandardCoders {
// BOOLEAN: beam:coder:bool:v1
// BYTES: beam:coder:bytes:v1
// ArrayType: beam:coder:iterable:v1 (always has a known length)
- // MapType: not yet a standard coder (BEAM-7996)
+ // MapType: not a standard coder, specification defined below.
// RowType: beam:coder:row:v1
// LogicalType: Uses the coder for its representation.
//
+ // The MapType is encoded by:
+ // - An INT32 representing the size of the map (N)
+ // - Followed by N interleaved keys and values, encoded with their
+ // corresponding coder.
+ //
+ // Nullable types in container types (ArrayType, MapType) are encoded by:
+ // - A one byte null indicator, 0x00 for null values, or 0x01 for present
+ // values.
+ // - For present values the null indicator is followed by the value
+ // encoded with it's corresponding coder.
+ //
Review comment:
This is specifically for nullable types that are elements of an Array or
keys/values of a Map. For rows we nulls in a separate bitmask:
https://github.com/apache/beam/blob/587dde57cbb2b0095a1fa04b59798d1b62c66f18/model/pipeline/src/main/proto/beam_runner_api.proto#L837-L843
Java schemas will already let you use nullable types for keys and values in
Maps. I doubt anyone is relying on it.. but there's a risk if we disallow it
now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
