rahil-c commented on code in PR #8441:
URL: https://github.com/apache/hudi/pull/8441#discussion_r1171942578
##########
hudi-aws/src/main/java/org/apache/hudi/aws/credentials/HoodieAWSCredentialsProviderFactory.java:
##########
@@ -30,16 +30,23 @@
* Factory class for Hoodie AWSCredentialsProvider.
*/
public class HoodieAWSCredentialsProviderFactory {
- public static AWSCredentialsProvider getAwsCredentialsProvider(Properties
props) {
+ public static AwsCredentialsProvider getAwsCredentialsProvider(Properties
props) {
return getAwsCredentialsProviderChain(props);
}
- private static AWSCredentialsProvider
getAwsCredentialsProviderChain(Properties props) {
- List<AWSCredentialsProvider> providers = new ArrayList<>();
- providers.add(new HoodieConfigAWSCredentialsProvider(props));
- providers.add(new DefaultAWSCredentialsProviderChain());
- AWSCredentialsProviderChain providerChain = new
AWSCredentialsProviderChain(providers);
- providerChain.setReuseLastProvider(true);
+ private static AwsCredentialsProvider
getAwsCredentialsProviderChain(Properties props) {
+ List<AwsCredentialsProvider> providers = new ArrayList<>();
+ HoodieConfigAWSCredentialsProvider hoodieConfigAWSCredentialsProvider =
new HoodieConfigAWSCredentialsProvider(props);
+ if (hoodieConfigAWSCredentialsProvider.resolveCredentials() != null) {
+ providers.add(hoodieConfigAWSCredentialsProvider);
Review Comment:
So `hoodieConfigAWSCredentialsProvider.resolveCredentials()` returns
`AwsCredentials awsCredentials`
I noticed that without this `if` block it will hit a `null `exception
```
Caused by: java.lang.NullPointerException: left must not be null.
at
org.apache.hudi.software.amazon.awssdk.utils.Validate.paramNotNull(Validate.java:156)
~[hudi-aws-bundle-0.14.0-SNAPSHOT.jar:0.14.0-SNAPSHOT]
at
org.apache.hudi.software.amazon.awssdk.utils.Pair.<init>(Pair.java:36)
~[hudi-aws-bundle-0.14.0-SNAPSHOT.jar:0.14.0-SNAPSHOT]
at org.apache.hudi.software.amazon.awssdk.utils.Pair.of(Pair.java:86)
~[hudi-aws-bundle-0.14.0-SNAPSHOT.jar:0.14.0-SNAPSHOT]
at
org.apache.hudi.software.amazon.awssdk.core.internal.util.MetricUtils.measureDuration(MetricUtils.java:52)
~[hudi-aws-bundle-0.14.0-SNAPSHOT.jar:0.14.0-SNAPSHOT]
at
org.apache.hudi.software.amazon.awssdk.awscore.internal.authcontext.AwsCredentialsAuthorizationStrategy.resolveCredentials(AwsCredentialsAuthorizationStrategy.java:100)
~[hudi-aws-bundle-0.14.0-SNAPSHOT.jar:0.14.0-SNAPSHOT]
at
org.apache.hudi.software.amazon.awssdk.awscore.internal.authcontext.AwsCredentialsAuthorizationStrategy.addCredentialsToExecutionAttributes(AwsCredentialsAuthorizationStrategy.java:77)
~[hudi-aws-bundle-0.14.0-SNAPSHOT.jar:0.14.0-SNAPSHOT]
at
org.apache.hudi.software.amazon.awssdk.awscore.internal.AwsExecutionContextBuilder.invokeInterceptorsAndCreateExecutionContext(AwsExecutionContextBuilder.java:120)
~[hudi-aws-bundle-0.14.0-SNAPSHOT.jar:0.14.0-SNAPSHOT]
at
org.apache.hudi.software.amazon.awssdk.awscore.client.handler.AwsAsyncClientHandler.invokeInterceptorsAndCreateExecutionContext(AwsAsyncClientHandler.java:65)
~[hudi-aws-bundle-0.14.0-SNAPSHOT.jar:0.14.0-SNAPSHOT]
at
org.apache.hudi.software.amazon.awssdk.core.internal.handler.BaseAsyncClientHandler.lambda$execute$1(BaseAsyncClientHandler.java:77)
~[hudi-aws-bundle-0.14.0-SNAPSHOT.jar:0.14.0-SNAPSHOT]
at
org.apache.hudi.software.amazon.awssdk.core.internal.handler.BaseAsyncClientHandler.measureApiCallSuccess(BaseAsyncClientHandler.java:291)
~[hudi-aws-bundle-0.14.0-SNAPSHOT.jar:0.14.0-SNAPSHOT]
at
org.apache.hudi.software.amazon.awssdk.core.internal.handler.BaseAsyncClientHandler.execute(BaseAsyncClientHandler.java:75)
~[hudi-aws-bundle-0.14.0-SNAPSHOT.jar:0.14.0-SNAPSHOT]
at
org.apache.hudi.software.amazon.awssdk.awscore.client.handler.AwsAsyncClientHandler.execute(AwsAsyncClientHandler.java:52)
~[hudi-aws-bundle-0.14.0-SNAPSHOT.jar:0.14.0-SNAPSHOT]
at
org.apache.hudi.software.amazon.awssdk.services.cloudwatch.DefaultCloudWatchAsyncClient.putMetricData(DefaultCloudWatchAsyncClient.java:3376)
~[hudi-aws-bundle-0.14.0-SNAPSHOT.jar:0.14.0-SNAPSHOT]
at
org.apache.hudi.aws.cloudwatch.CloudWatchReporter.report(CloudWatchReporter.java:230)
~[hudi-aws-bundle-0.14.0-SNAPSHOT.jar:0.14.0-SNAPSHOT]
... 103 more
```
because if the `AwsCredentials awsCredentials`; might not be initialized
from the `HoodieConfigAWSCredentialsProvider` class (this happens if someone
does not configure these properties
```
public HoodieConfigAWSCredentialsProvider(Properties props) {
String accessKey =
props.getProperty(HoodieAWSConfig.AWS_ACCESS_KEY.key());
String secretKey =
props.getProperty(HoodieAWSConfig.AWS_SECRET_KEY.key());
String sessionToken =
props.getProperty(HoodieAWSConfig.AWS_SESSION_TOKEN.key());
if (StringUtils.isNullOrEmpty(accessKey) ||
StringUtils.isNullOrEmpty(secretKey)) {
LOG.debug("AWS access key or secret key not found in the Hudi
configuration. "
+ "Use default AWS credentials");
} else {
this.awsCredentials = createCredentials(accessKey, secretKey,
sessionToken);
}
}
```
then the `awsCredential` object is null. And we hit an error, when it should
in fact roll to the default AWS credentials. So the if is to guard against that
by only adding to it the arrayList if it has been intialized.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
