[
https://issues.apache.org/jira/browse/BEAM-4275?focusedWorklogId=134364&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-134364
]
ASF GitHub Bot logged work on BEAM-4275:
----------------------------------------
Author: ASF GitHub Bot
Created on: 14/Aug/18 00:33
Start Date: 14/Aug/18 00:33
Worklog Time Spent: 10m
Work Description: udim commented on issue #6217: [BEAM-4275] Implement
integration tests for PubSub on DirectRunner
URL: https://github.com/apache/beam/pull/6217#issuecomment-412712607
run python postcommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 134364)
Time Spent: 10m
Remaining Estimate: 0h
> Pubsub: add DirectRunner support for id_label and timestamp_attribute in
> Python SDK
> -----------------------------------------------------------------------------------
>
> Key: BEAM-4275
> URL: https://issues.apache.org/jira/browse/BEAM-4275
> Project: Beam
> Issue Type: Bug
> Components: runner-direct, sdk-py-core
> Reporter: Udi Meiri
> Assignee: Udi Meiri
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> At least for publishing (and maybe pulling) messages, non-Dataflow-based
> sources and sinks for Pub/Sub use the [public
> API|https://cloud.google.com/pubsub/docs/publisher] for Pub/Sub, which
> doesn't support id_label and timestamp_attribute settings.
> Publishing:
> id_label - add an attribute to each message with a unique value
> timestamp_attribute - add an attribute to each message with the publishing
> time as its value
> Pulling:
> id_label - use the value of this message attribute to deduplicate messages
> timestamp_attribute - use the value of this message attribute as the
> element's timestamp
>
> Implementation details: could probably create a pubsubio.py module, for reuse
> with other runners (i.e. implement Pub/Sub IO as PTransforms and not
> NativeSinks and Sources).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
