egalpin commented on issue #21162: URL: https://github.com/apache/beam/issues/21162#issuecomment-1370134940
Thanks for confirming. I haven't yet had time to trace/validate. One other option you could try depending on your level of comfort would be to modify the beam source and vendor in the pubsubIO module. I noted my steps for doing so, specifically with dataflow, previously: https://lists.apache.org/thread/c6929ms0bjxtcw9ho4tdb5y3t8wnwnfy. This is not a long-term solution, but rather would be a stopgap solution until the same change can be made in the official SDK as per https://github.com/apache/beam/issues/23525. Given that setting the coder on the PCollection appears to be insufficient, the priority of https://github.com/apache/beam/issues/23525 is increased in my opinion. You'd want to modify the PubsubCoderProviderRegistrar[1] such that the `PubsubMessageWithAttributesAndMessageIdAndOrderingKeyCoder` appears first in the list (or as the sole list element), then follow the vendoring steps mentioned. [1] https://github.com/apache/beam/blob/9a22bf68767b8fa27aa381d6ff84e381a21af5a4/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubCoderProviderRegistrar.java -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
