[
https://issues.apache.org/jira/browse/BEAM-2629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16092077#comment-16092077
]
Nigel Kilmer commented on BEAM-2629:
------------------------------------
Looks like the cause of this issue is that PubsubIO.FormatPayloadUsingCoder
creates the PubsubMessage with an ImmutableMap.of() for the attributes.
PubsubJsonClient is unable to add the timestamp and id attributes to this
immutable map. See:
https://github.com/apache/beam/blob/6a7eeeb932ffb33c876c0d11d7f325c47606d8de/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubIO.java#L984
I think it makes more sense to fix PubsubJsonClient to create a mutable copy of
the attributes rather than trying to ensure that PubsubIO format functions
always create pubsub messages with mutable attribute maps. I've implemented the
former in my pull request.
> Adding timestamp and id attributes to pubsub messages with no other
> attributes causes an exception.
> ---------------------------------------------------------------------------------------------------
>
> Key: BEAM-2629
> URL: https://issues.apache.org/jira/browse/BEAM-2629
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-gcp
> Affects Versions: 2.2.0
> Reporter: Nigel Kilmer
> Assignee: Nigel Kilmer
>
> I'm trying to use PubsubIO.writeProtos along with a timestamp and id
> attribute. When PubsubIO tries to write the messages to my topic, it throws a
> PipelineExecutionException with this cause:
> Caused by: java.lang.UnsupportedOperationException
> at java.util.AbstractMap.put(AbstractMap.java:209)
> at
> org.apache.beam.sdk.io.gcp.pubsub.PubsubJsonClient.publish(PubsubJsonClient.java:148)
> at
> org.apache.beam.sdk.io.gcp.pubsub.PubsubUnboundedSink$WriterFn.publishBatch(PubsubUnboundedSink.java:249)
> at
> org.apache.beam.sdk.io.gcp.pubsub.PubsubUnboundedSink$WriterFn.processElement(PubsubUnboundedSink.java:285)
> Looks like the problem is that PubsubJsonClient attempts to add the timestamp
> and id attributes to a map which may be empty and immutable if no other
> attributes are specified on the message. I will submit a pull request with a
> fix for this bug shortly.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
