Jian Zheng created BEAM-10926:
---------------------------------
Summary: Specify the event time when consuming pubsub data.
Key: BEAM-10926
URL: https://issues.apache.org/jira/browse/BEAM-10926
Project: Beam
Issue Type: Improvement
Components: io-java-gcp
Reporter: Jian Zheng
I need to specify the event time when consuming pubsub data.
{code:java}
PCollection<PubsubMessage> pubsubMessages = pipeline.apply("Read Pub/Sub Data",
PubsubIO.readMessagesWithAttributes()
.withTimestampAttribute(options.getTimeAttribute())
.fromSubscription(options.getInputSubscription()));
{code}
The only way to do this is to use the
{color:#FF0000}withTimestampAttribute(){color} method.
But if I use a timestamp in some other format, such as a 19-bit nanosecond, or
if save the event time in the payload. The method won't work !
So I had to extend the PubsubClient class and override the PubsubClient.
extractTimestamp() method.
I'am hoping to provide a way to pass in some implementation class that would
allow to parse out timestamp from the current pubsub message.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
