steveloughran commented on code in PR #4698:
URL: https://github.com/apache/hadoop/pull/4698#discussion_r954774809
##########
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java:
##########
@@ -1619,6 +1622,25 @@ public <T> CompletableFuture<T> submit(final
CallableRaisingIOE<T> operation) {
}
}
+ /**
+ * Callbacks for WriteOperationHelper.
+ */
+ private final class WriteOperationHelperCallbacksImpl
Review Comment:
i like this. yes, it makes for a bigger patch, but it was needed anyway
##########
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/WriteOperationHelper.java:
##########
@@ -157,7 +162,8 @@ protected WriteOperationHelper(S3AFileSystem owner,
Configuration conf,
S3AStatisticsContext statisticsContext,
final AuditSpanSource auditSpanSource,
- final AuditSpan auditSpan) {
+ final AuditSpan auditSpan,
+ final WriteOperationHelperCallbacks writeOperationHelperCallbacks) {
Review Comment:
nit: needs a javadoc entry
##########
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/MockS3AFileSystem.java:
##########
@@ -176,7 +177,8 @@ public void initialize(URI name, Configuration originalConf)
conf,
new EmptyS3AStatisticsContext(),
noopAuditor(conf),
- AuditTestSupport.NOOP_SPAN);
+ AuditTestSupport.NOOP_SPAN,
+ new MinimalWriteOperationHelperCallbacks());
Review Comment:
this is why i like the callback interface/impl design
##########
hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/troubleshooting_s3a.md:
##########
@@ -1989,3 +1990,51 @@ com.amazonaws.SdkClientException: Unable to execute HTTP
request:
When this happens, try to set `fs.s3a.connection.request.timeout` to a larger
value or disable it
completely by setting it to `0`.
+
+## <a name="upgrade_warnings"></a> SDK Upgrade Warnings
+
+S3A will soon be upgraded to [AWS's Java SDK
V2](https://github.com/aws/aws-sdk-java-v2).
+For more information on the upgrade and what's changing, see
+[Upcoming upgrade to AWS Java SDK V2](./aws_sdk_upgrade.html).
+
+S3A logs the following warnings for things that will be changing in the
upgrade. To disable these
+logs, comment out `log4j.logger.org.apache.hadoop.fs.s3a.SDKV2Upgrade` in
log4j.properties.
+
+### <a name="ProviderReferenced"></a> `Directly referencing AWS SDK V1
credential provider`
+
+This will be logged when an AWS credential provider is referenced directly in
+`fs.s3a.aws.credentials.provider`.
+For example, `com.amazonaws.auth.AWSSessionCredentialsProvider`
+
+To stop this warning, remove any AWS credential providers from
`fs.s3a.aws.credentials.provider`.
+Instead, use S3A's credential providers.
+
+### <a name="ClientRequested"></a> `getAmazonS3ClientForTesting() will be
removed`
+
+This will be logged when `getAmazonS3ClientForTesting()` is called to get the
S3 Client. With V2,
+the S3 client will change from type `com.amazonaws.services.s3.AmazonS3` to
+`software.amazon.awssdk.services.s3.S3Client`, and so this method will be
removed.
+
+### <a name="DelegationTokenProvider"></a>
+### `Custom credential providers used in delegation tokens binding classes
will need to be updated`
+
+This will be logged when delegation tokens are used.
+Delegation tokens allow the use of custom binding classes which can implement
custom credential
+providers.
+These credential providers will currently be implementing
+`com.amazonaws.auth.AWSCredentialsProvider` and will need to be updated to
implement
+`software.amazon.awssdk.auth.credentials.AwsCredentialsProvider`.
+
+### <a name="CustomSignerUsed"></a>
+### `The signer interface has changed in AWS SDK V2, custom signers will need
to be updated`
+
+This will be logged when a custom signer is used.
+Custom signers will currently be implementing `com.amazonaws.auth.Signer` and
will need to be
+updated to implement `software.amazon.awssdk.core.signer.Signer`.
+
+### <a name="GetObjectMetadataCalled"></a>
Review Comment:
afraid you need to strip the trailing spaces. we all do this on the markdowns
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
