Unless someone else knows I think timestamp, attributes, and payload are all you're gonna get.
I'm definitely not familiar with the implementation, but the timestamp part is here: https://github.com/apache/beam/blob/ea7940d88d9d5ddb09fabc5c8993d8395c59abce/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubClient.java#L101-L115 Good luck! <https://github.com/apache/beam/blob/ea7940d88d9d5ddb09fabc5c8993d8395c59abce/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubClient.java#L101-L115> [https://avatars3.githubusercontent.com/u/47359?v=4&s=400]<https://github.com/apache/beam/blob/ea7940d88d9d5ddb09fabc5c8993d8395c59abce/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubClient.java#L101-L115> apache/beam<https://github.com/apache/beam/blob/ea7940d88d9d5ddb09fabc5c8993d8395c59abce/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubClient.java#L101-L115> github.com beam - Mirror of Apache Beam ________________________________ From: Derek Hao Hu <[email protected]> Sent: 02 October 2017 18:01:10 To: [email protected] Subject: Re: How to get the PublishTime of PubsubMessages? Thanks Reinier! Is it easy to get the [MessageId](http://googleapis.github.io/googleapis/java/proto-google-cloud-pubsub-v1/0.1.19/apidocs/com/google/pubsub/v1/PubsubMessage.html#getMessageId--) of the PubsubMessages? PS: If you are familiar with this part of the Beam implementation, can you also point out where it gets the timestamp attribute? I'm trying to understand the implementation as well. :) Cheers, Derek On Sat, Sep 30, 2017 at 5:58 AM, Reinier Kip <[email protected]<mailto:[email protected]>> wrote: Hi Derek, The publish time is the element's timestamp, retrievable via DoFn.ProcessContext#timestamp, unless you've configured a timestamp attribute with PubsubIO.Read#withTimestampAttribute. As far as I know, payload, attributes and publish time is all you get. What other information from Google's PubsubMessage are you looking for? Reinier ________________________________ From: Derek Hao Hu <[email protected]<mailto:[email protected]>> Sent: 30 September 2017 03:18:13 To: [email protected]<mailto:[email protected]> Subject: How to get the PublishTime of PubsubMessages? Hi, I'm trying to use [PubsubIO](https://beam.apache.org/documentation/sdks/javadoc/2.1.0/org/apache/beam/sdk/io/gcp/pubsub/PubsubIO.html) to read [PubsubMessage](https://beam.apache.org/documentation/sdks/javadoc/2.1.0/org/apache/beam/sdk/io/gcp/pubsub/PubsubMessage.html). I've noticed that the PubsubMessage class in Beam only exposes Attribute and Payload but the [PubSubMessage in Google Cloud](http://googleapis.github.io/googleapis/java/proto-google-cloud-pubsub-v1/0.1.19/apidocs/com/google/pubsub/v1/PubsubMessage.html?is-external=true) exposes a lot more information like `getPublishTime()`. Is there a way for us to get the same amount information in Beam easily? -- Derek Hao Hu Software Engineer | Snapchat Snap Inc. -- Derek Hao Hu Software Engineer | Snapchat Snap Inc.
