steveloughran commented on a change in pull request #1661: HADOOP-16484. S3A to
warn or fail if S3Guard is disabled
URL: https://github.com/apache/hadoop/pull/1661#discussion_r337619708
##########
File path:
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Constants.java
##########
@@ -639,6 +639,14 @@ private Constants() {
public static final String S3GUARD_METASTORE_DYNAMO
= "org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore";
+ /**
+ * The warn level if S3Guard is disabled.
+ */
+ public static final String S3GUARD_DISABLED_WARN_LEVEL
+ = "org.apache.hadoop.fs.s3a.s3guard.disabled_warn_level";
+ public static final String DEFAULT_S3GUARD_DISABLED_WARN_LEVEL =
Review comment:
I'm the one recommending inform; I am too fed up with trying to debug error
traces where the root cause is "s3guard was not enabled".
That's when they thought it was turned on but it wasn't, or when they knew
it wasn't turned on that didn't bother mentioning that fact.
Either way, our ability to diagnose intermittent consistency problems of the
kind which s3guard defends against is hampered when nobody knows when s3guard
is not enabled.
I want "inform" to be the default, because people working with AWS S3 need
to know that without a consistency layer some of that work is going to fail.
They either need to enable s3guard in which case they don't get the message, or
they explicitly turned the message off. Their choice. But we need to make clear
that S3 without s3guard is not safe to be used when working with any store
where you are chaining work.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
