Hi,
I'm trying to deploy an Apache Beam application in a managed Apache
Flink Cluster (Kinesis Data Analytics
<https://aws.amazon.com/kinesis/data-analytics/>). The pipeline uses
thePubsubIO
<https://beam.apache.org/releases/javadoc/2.19.0/org/apache/beam/sdk/io/gcp/pubsub/PubsubIO.html>connector.
But I can't get the application to authenticate with Google Cloud,
because Kinesis Data Analytics does not allow to export environment
variables, so exportingGOOGLE_APPLICATION_CREDENTIALS
<https://cloud.google.com/docs/authentication/getting-started>environment
variable doesn't seem to be an option.
I tried to authenticate from code, as below.
|GoogleCredentials credential =
GoogleCredentials.fromStream(credentialJsonInputStream);
options.as(GcpOptions.class).setGcpCredential(credential); Pipeline
pipeline = Pipeline.create(options)|
|​ |
But that didn't work. I'm not sure whether theGcpOptions
<https://beam.apache.org/releases/javadoc/2.1.0/org/apache/beam/sdk/extensions/gcp/options/GcpOptions.html>interface
is only supposed to be used with Dataflow runner.
I'd really appreciate any insights on how to authenticate in this scenario.
Thanks and regards,
*Gayan Weerakutti*
<https://www.linkedin.com/in/reversiblean/>linkedin.com/in/gayanweerakutti
<https://www.linkedin.com/in/gayanweerakutti/>
