Hello all: I have an application that requires accessing DynamoDB tables. Each worker establishes a connection with the database on its own.
I have added both `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` to both master's and workers `spark-env.sh` file. I have also run the file using `sh` to make sure the variables are exported. When the code runs, I always get the error: Caused by: com.amazonaws.AmazonClientException: Unable to load AWS credentials from any provider in the chain at com.amazonaws.auth.AWSCredentialsProviderChain.getCredentials(AWSCredentialsProviderChain.java:131) at com.amazonaws.http.AmazonHttpClient.getCredentialsFromContext(AmazonHttpClient.java:774) at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:800) at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:695) at com.amazonaws.http.AmazonHttpClient.doExecute(AmazonHttpClient.java:447) at com.amazonaws.http.AmazonHttpClient.executeWithTimer(AmazonHttpClient.java:409) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:358) at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.doInvoke(AmazonDynamoDBClient.java:2051) at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.invoke(AmazonDynamoDBClient.java:2021) at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.describeTable(AmazonDynamoDBClient.java:1299) at com.amazon.titan.diskstorage.dynamodb.DynamoDBDelegate.describeTable(DynamoDBDelegate.java:635) ... 27 more It seems that the AWS SDK has failed to load the credentials even though they're exported. When I do export command, it returns the credentials. What type of solution should I try? Thanks, Mohamed Taher Alrefaie.