tvalentyn commented on code in PR #30146:
URL: https://github.com/apache/beam/pull/30146#discussion_r1475077319
##########
sdks/python/apache_beam/ml/transforms/handlers.py:
##########
@@ -84,38 +84,37 @@
tft_process_handler_output_type = typing.Union[beam.Row, Dict[str, np.ndarray]]
-# alternatie: Use a single class for both encoding and decoding and
-# use beam.Map() instead of DoFns?
-class _EncodeDict(beam.DoFn):
- """
- Encode a dictionary into bytes and pass it along with the original element
- using a temporary key.
-
- Internal use only. No backward compatibility guarantees.
- """
+class DataCoder:
def __init__(self, exclude_columns=None):
- self._exclude_columns = exclude_columns
+ """
+ Uses PickleCoder to encode/decode the dictonaries.
+ Args:
+ exclude_columns: list of columns to exclude from the encoding.
+ """
+ self.coder = PickleCoder()
Review Comment:
```
# Returns FastPrimitivesCoder, but also allows users to supply an
override (custom) coder.
self.coder = coders.registry.get_coder(Any)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
