Re: [External Sender] Re: [Question] Apache Beam Pipeline AWS Credentials

Hi Sachin, Thanks for the response. I'm trying to figure out a way to set the AwsCredentialsProvider using PipelineOptions, not the arguments. Thanks though! Ramya On Wed, Jan 10, 2024 at 1:04 PM Sachin Mittal wrote: > Hi, > Check this class org.apache.beam.sdk.io.aws2.options.AwsOptions >

Re: [External Sender] Re: [Question] Apache Beam Pipeline AWS Credentials

Hi, Check this class org.apache.beam.sdk.io.aws2.options.AwsOptions It tells you how to set AwsCredentialsProvider used to configure AWS service clients. For example you can run beam application with following application properties: --awsCredentialsProvider={"@type":

Re: [External Sender] Re: [Question] Apache Beam Pipeline AWS Credentials

Hi Alexey, Thanks for the quick response! Apologies for using the dev list, I was unsure which list to be using! So I did try to create my own AwsCredentialsProvider class, but I'm running into a Serializable issue, as the error I'm getting is "Failed to serialize PipelineOptions". This is roughly

Re: [Question] Apache Beam Pipeline AWS Credentials

Hi Ramya, I don’t think there is a solution out-of-the-box for such case but I believe you can create your own AwsCredentialsProvider and implement the logic to fetch the credentials dynamically. PS: Please, use only user@beam.apache.org mailing list for such

[Question] Apache Beam Pipeline AWS Credentials

Hello, I am a developer trying to use Apache Beam in Java, and I am having an issue with my AWS credentials expiring before my pipeline is done executing. I'm limited by how I get my AWS credentials, as I set the credentials using the StaticCredentialsProvider class. I set the credentials (which