damccorm commented on code in PR #36979:
URL: https://github.com/apache/beam/pull/36979#discussion_r2619981792
##########
sdks/python/apache_beam/coders/coder_impl.py:
##########
@@ -616,7 +619,15 @@ def decode_from_stream(self, stream, nested):
msg = cls()
msg.ParseFromString(stream.read_all(True))
return msg
- elif t == DATACLASS_TYPE or t == NAMED_TUPLE_TYPE:
+ elif t == DATACLASS_TYPE:
Review Comment:
As written, this will break any pipelines that try to do an inflight update
from an older Beam version to this version. This is because any stored data
(e.g. shuffled data) will still be encoded using the old encoding mechanism
##########
sdks/python/apache_beam/coders/coder_impl.py:
##########
@@ -616,7 +619,15 @@ def decode_from_stream(self, stream, nested):
msg = cls()
msg.ParseFromString(stream.read_all(True))
return msg
- elif t == DATACLASS_TYPE or t == NAMED_TUPLE_TYPE:
+ elif t == DATACLASS_TYPE:
Review Comment:
I think to handle this, we need to introduce a new type (similar to
`DATACLASS_TYPE`) to represent kw only dataclasses.
Potentially we could move all encoding to use this new type (and just leave
the decode path), but it is probably more efficient to leave it for the types
that can use it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
