[
https://issues.apache.org/jira/browse/HADOOP-16085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16793905#comment-16793905
]
Ben Roling commented on HADOOP-16085:
-------------------------------------
I should also note that the new patch implements a getObjectMetadata() based
check on ETag or versionId before starting the read for select(). This is not
a full guarantee that select() will read the correct version of the file or
generate an exception. It could read the wrong file version without exception
if:
* S3 eventual consistency manifests in a way where one read sees the new
version and the next read sees the old version. This seems unlikely but I
don't think there is any guarantee it couldn't happen.
* After reading the correct new version, another overwrite comes through
writing an even newer version. The select() sees the newest version. Perhaps
that's not problematic though.
In change.detection.mode=client we could do the same thing in rename(), making
a getObjectMetadata() request before the copy and checking that the response
has the expected ETag or versionId and if not throwing
RemoteFileChangedException. It would have the same risk I just mentioned.
That can be mentioned in the documentation. It will make rename() in this mode
more expensive though as it will require the extra getObjectMetadata() call per
file being copied.
> S3Guard: use object version or etags to protect against inconsistent read
> after replace/overwrite
> -------------------------------------------------------------------------------------------------
>
> Key: HADOOP-16085
> URL: https://issues.apache.org/jira/browse/HADOOP-16085
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.2.0
> Reporter: Ben Roling
> Priority: Major
> Attachments: HADOOP-16085-003.patch, HADOOP-16085_002.patch,
> HADOOP-16085_3.2.0_001.patch
>
>
> Currently S3Guard doesn't track S3 object versions. If a file is written in
> S3A with S3Guard and then subsequently overwritten, there is no protection
> against the next reader seeing the old version of the file instead of the new
> one.
> It seems like the S3Guard metadata could track the S3 object version. When a
> file is created or updated, the object version could be written to the
> S3Guard metadata. When a file is read, the read out of S3 could be performed
> by object version, ensuring the correct version is retrieved.
> I don't have a lot of direct experience with this yet, but this is my
> impression from looking through the code. My organization is looking to
> shift some datasets stored in HDFS over to S3 and is concerned about this
> potential issue as there are some cases in our codebase that would do an
> overwrite.
> I imagine this idea may have been considered before but I couldn't quite
> track down any JIRAs discussing it. If there is one, feel free to close this
> with a reference to it.
> Am I understanding things correctly? Is this idea feasible? Any feedback
> that could be provided would be appreciated. We may consider crafting a
> patch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
