Jean-Baptiste Onofré created BEAM-881:
-----------------------------------------
Summary: Provide a PTransform in IOs providing a "standard" Avro
IndexedRecord
Key: BEAM-881
URL: https://issues.apache.org/jira/browse/BEAM-881
Project: Beam
Issue Type: New Feature
Components: sdk-java-extensions
Reporter: Jean-Baptiste Onofré
Assignee: Jean-Baptiste Onofré
Now, each IO is using a different data format. For instance, the {{JmsIO.Read}}
provides a {{PCollection}} of {{JmsRecord}} (and {{JmsIO.Write}} expects also a
{{JmsRecord}}), {{KafkaIO.Read}} provides a {{PCollection}} of {{KafkaRecord}}.
It could appear a bit "complex" for users to manipulate such kind of data
format: some users may expect kind of standard format.
Without modifying the existing IO, we could add a {{PTransform}} (as part of
the IO) that an user can optionally use. This transform will convert the IO
data format (let say {{JmsRecord}} for instance) to a standard Avro
{{IndexedRecord}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
