[
https://issues.apache.org/jira/browse/BEAM-10962?focusedWorklogId=492032&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-492032
]
ASF GitHub Bot logged work on BEAM-10962:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 28/Sep/20 15:33
Start Date: 28/Sep/20 15:33
Worklog Time Spent: 10m
Work Description: chamikaramj commented on pull request #12930:
URL: https://github.com/apache/beam/pull/12930#issuecomment-700093404
The usual pattern for sources is.
(1) A transform that reads from a given source config
(2) A "ReadAll" transform that reads a PCollection of configs.
Given that PubSub is a native transform for Dataflow though, we cannot
really implement (2).
I'm not really sure a composite that just wraps a Flatten adds much value
since pipeline authors can easily do that themselves (we can do that for every
other transform as well but that will just clutter the API in my opinion).
(2) above will be more useful and will enable new use-cases. But we cannot
really do that for Dataflow.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 492032)
Time Spent: 2h (was: 1h 50m)
> Multiple PubSub Source Reader
> ------------------------------
>
> Key: BEAM-10962
> URL: https://issues.apache.org/jira/browse/BEAM-10962
> Project: Beam
> Issue Type: New Feature
> Components: io-py-gcp
> Reporter: Inigo San Jose Visiers
> Priority: P2
> Time Spent: 2h
> Remaining Estimate: 0h
>
> A very common use case in Dataflow / Beam is reading from multiple PubSub
> topics/subscriptions and flatten them out.
> I would be nice to have a PTransform that does this without users having to
> built it themselves.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
