Abacn commented on issue #33802:
URL: https://github.com/apache/beam/issues/33802#issuecomment-4490673397
As of upcoming Beam 2.74.0 one can use RowCoder for dataclass: #22085
```
@coders_registry.register_row
@dataclass.dataclass
class NeatOutputClass:
id: int
data: Any
```
The field of Any typehint uses pickle coder while fields of known types will
use more efficient coders. This should make it less inefficient than using
pickle coder for the whole dataclass
The problem here is that Beam doesn't have a portable Any type (Feature
request: #38251). Python's Any is kind of similar. It uses FastPrimitiveCoder,
which has only 1 byte extra cost for primitive types, while fallback to
PickleType for objects
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
