[
https://issues.apache.org/jira/browse/BEAM-7738?focusedWorklogId=332137&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-332137
]
ASF GitHub Bot logged work on BEAM-7738:
----------------------------------------
Author: ASF GitHub Bot
Created on: 22/Oct/19 18:26
Start Date: 22/Oct/19 18:26
Worklog Time Spent: 10m
Work Description: TheNeuralBit commented on issue #9268: [BEAM-7738] Add
external transform support to PubsubIO
URL: https://github.com/apache/beam/pull/9268#issuecomment-545092832
> How close are we to having schema coders ready to use? IIUC, we should be
able to register a converter between Row -> PubsubMessage, right? What's the
mechanism for that? It would be great to put that to use here as soon as the
schema coders are ready.
@robertwb gave #9188 a LGTM, I just need to get CI passing and I think we
can merge it. I'll work on that now.
Are you thinking you'd use beam:coder:row:v1 as the interface for the
external transform, and the Java ExternalTransform implementations would handle
the conversion of Row to/from PubsubMessage? There's no trivial way to register
a converter between Row and PubsubMessage since the latter isn't structured,
but of course on the Java side we could have code to serialize the Row to a
variety of formats to put in the PubsubMessage payload: Avro, JSON, or the row
serialization format itself (although I'm not sure we'd want to encourage using
that outside of Beam), would be pretty simple to add. Maybe the format to use
could be part of the external transform payload.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 332137)
Time Spent: 7h (was: 6h 50m)
> Support PubSubIO to be configured externally for use with other SDKs
> --------------------------------------------------------------------
>
> Key: BEAM-7738
> URL: https://issues.apache.org/jira/browse/BEAM-7738
> Project: Beam
> Issue Type: New Feature
> Components: io-java-gcp, runner-flink, sdk-py-core
> Reporter: Chad Dombrova
> Assignee: Chad Dombrova
> Priority: Major
> Labels: portability
> Time Spent: 7h
> Remaining Estimate: 0h
>
> Now that KafkaIO is supported via the external transform API (BEAM-7029) we
> should add support for PubSub.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
