Abacn commented on code in PR #22561:
URL: https://github.com/apache/beam/pull/22561#discussion_r941824718
##########
sdks/python/apache_beam/coders/row_coder.py:
##########
@@ -163,10 +165,15 @@ def _nonnull_coder_from_type(field_type):
_coder_from_type(field_type.map_type.key_type),
_coder_from_type(field_type.map_type.value_type))
elif type_info == "logical_type":
- # Special case for the Any logical type. Just use the default coder for an
- # unknown Python object.
if field_type.logical_type.urn == PYTHON_ANY_URN:
+ # Special case for the Any logical type. Just use the default coder for
an
+ # unknown Python object.
return typecoders.registry.get_coder(object)
+ elif field_type.logical_type.urn == DATETIME_URN:
+ # Special case for datetime logical type.
+ # DATETIME_URN explicitly uses TimestampCoder which deals with fix length
+ # 8-bytes big-endian-long instead of varint coder.
+ return TimestampCoder()
Review Comment:
Thanks for the instructions! Will complete the proto change.
`beam:logical_type:datetime:v1` is the millisecond precision timestamp backed
by a fixed length INT64 that encoded with BigEndianLong. Will complete the
proto change.
> can we do the byte flipping in a to_language_type and
to_representation_type implementation?
Considered this before. Under the current framework the value is already
decoded with VarInt coder before sent to to_language_type or
to_representation_type which is incorrect. All I need is a 8-byte fixed long
integer primitive which does not exist in portable primitives, and there is not
yet a consensus on reference coder
([context](https://docs.google.com/document/d/1uu9pJktzT_O3DxGd1-Q2op4nRk4HekIZbzi-0oTAips/edit?disco=AAAADKzJ_Q8)).
Therefore I ended up this solution to make things work while not introducing
new primitives or other fundamental changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
