Hi,
Check this class org.apache.beam.sdk.io.aws2.options.AwsOptions
It tells you how to set AwsCredentialsProvider used to configure AWS
service clients.
For example you can run beam application with following application
properties:
--awsCredentialsProvider={ "@type": "StaticCredentialsProvider",
"awsAccessKeyId": "key_id_value", "awsSecretKey": "secret_value",
"sessionToken": "token_value" }
Thanks
Sachin
On Wed, Jan 10, 2024 at 11:30 PM Ramya Prasad via user <user@beam.apache.org>
wrote:
> Hi Alexey,
> Thanks for the quick response! Apologies for using the dev list, I was
> unsure which list to be using! So I did try to create my own
> AwsCredentialsProvider class, but I'm running into a Serializable issue, as
> the error I'm getting is "Failed to serialize PipelineOptions". This is
> roughly what my class looks like, any suggestions?
>
> import software.amazon.awssdk.auth.credentials.AwsCredentials;
> import software.amazon.awssdk.auth.credentials.AwsCredentialsProvider;
> import software.amazon.awssdk.auth.credentials.AwsSessionCredentials;
>
> import java.io.Serializable;
> import java.time.Instant;
>
> public class CustomAwsCredentialsProvider implements Serializable,
> AwsCredentialsProvider {
> private AwsSessionCredentials creds;
>
> public CustomAwsCredentialsProvider(AwsSessionCredentials creds) {
> this.creds = creds;
> }
>
> @Override
> public AwsCredentials resolveCredentials() {
>
> if (this.creds.expirationTime().isPresent()) {
> Instant exp = this.creds.expirationTime().get();
> if (exp.isBefore(Instant.now())) {
> <logic to fetch new credentials>
> }
> }
> return this.creds;
> }
> }
>
> Thanks and sincerely,
> Ramya
> On Wed, Jan 10, 2024 at 9:00 AM Alexey Romanenko <aromanenko....@gmail.com>
> wrote:
>
>> Hi Ramya,
>>
>> I don’t think there is a solution out-of-the-box for such case but I
>> believe you can create your own AwsCredentialsProvider and implement the
>> logic to fetch the credentials dynamically.
>>
>> PS: Please, use only user@beam.apache.org mailing list for such type of
>> questions.
>>
>> —
>> Alexey
>>
>> On 9 Jan 2024, at 19:39, Ramya Prasad via user <user@beam.apache.org>
>> wrote:
>>
>> Hello,
>>
>> I am a developer trying to use Apache Beam in Java, and I am having an
>> issue with my AWS credentials expiring before my pipeline is done
>> executing. I'm limited by how I get my AWS credentials, as I set the
>> credentials using the StaticCredentialsProvider class. I set the
>> credentials (which is an access key, secret access key, and session token)
>> in the PipelineOptions object before I create the pipeline. However, I
>> believe the PipelineOptions object cannot be modified during runtime. I'm
>> not sure how to refresh my credentials so the pipeline doesn't break during
>> execution.
>>
>> Some code for how I set my credentials for reference:
>>
>> Credentials awsCreds = <API call to retrieve my credentials>;
>> AwsCredentials credentials =
>> AwsSessionCredentials.create(awsCreds.getAccessKeyId(),
>> awsCreds.getSecretAccessKey(), awsCreds.getSessionToken());
>> AwsCredentialsProvider provider =
>> StaticCredentialsProvider.create(credentials);
>>
>> pipelineOptions.as(AwsOptions.class).setAwsCredentialsProvider(provider);
>>
>>
>>
>> Any help would be appreciated!
>>
>> Thanks and sincerely,
>> Ramya
>> ------------------------------
>>
>>
>> The information contained in this e-mail may be confidential and/or
>> proprietary to Capital One and/or its affiliates and may only be used
>> solely in performance of work or services for Capital One. The information
>> transmitted herewith is intended only for use by the individual or entity
>> to which it is addressed. If the reader of this message is not the intended
>> recipient, you are hereby notified that any review, retransmission,
>> dissemination, distribution, copying or other use of, or taking of any
>> action in reliance upon this information is strictly prohibited. If you
>> have received this communication in error, please contact the sender and
>> delete the material from your computer.
>>
>>
>>
>> ------------------------------
>
> The information contained in this e-mail may be confidential and/or
> proprietary to Capital One and/or its affiliates and may only be used
> solely in performance of work or services for Capital One. The information
> transmitted herewith is intended only for use by the individual or entity
> to which it is addressed. If the reader of this message is not the intended
> recipient, you are hereby notified that any review, retransmission,
> dissemination, distribution, copying or other use of, or taking of any
> action in reliance upon this information is strictly prohibited. If you
> have received this communication in error, please contact the sender and
> delete the material from your computer.
>
>
>
>
>
