[
https://issues.apache.org/jira/browse/HADOOP-13760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16007038#comment-16007038
]
Aaron Fabbri commented on HADOOP-13760:
---------------------------------------
Looked at this {{s3GetFileStatus()}} code a bit more last night
{code}
- return new S3AFileStatus(Tristate.FALSE, path, username);
+ boolean isEmpty = true;
{code}
If tombstones is null, you've changed returned result from non-empty to empty.
{code}
+ if (tombstones != null) {
+ for (String prefix : prefixes) {
+ Path absolute = qualify(new Path("/" + prefix));
+ if (!tombstones.contains(absolute)) {
+ isEmpty = false;
+ break;
+ }
+ }
+ if (isEmpty) {
+ for (S3ObjectSummary summary : summaries) {
+ Path absolute = qualify(new Path("/" + summary.getKey()));
+ if (!tombstones.contains(absolute)) {
+ isEmpty = false;
+ break;
+ }
+ }
+ }
+ }
+ return new S3AFileStatus(Tristate.fromBool(isEmpty), path, username);
{code}
Above this code, we check
{code}
if (!objects.getCommonPrefixes().isEmpty() ||
!objects.getObjectSummaries().isEmpty()) {
{code}
.. I'd probably replace those two isEmpty() calls with helper functions that
evaluate emptiness based also on the tombstones. That plus a good comment or
two would make this more readable IMO.
> S3Guard: add delete tracking
> ----------------------------
>
> Key: HADOOP-13760
> URL: https://issues.apache.org/jira/browse/HADOOP-13760
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Aaron Fabbri
> Assignee: Sean Mackrory
> Attachments: HADOOP-13760-HADOOP-13345.001.patch,
> HADOOP-13760-HADOOP-13345.002.patch, HADOOP-13760-HADOOP-13345.003.patch,
> HADOOP-13760-HADOOP-13345.004.patch, HADOOP-13760-HADOOP-13345.005.patch,
> HADOOP-13760-HADOOP-13345.006.patch
>
>
> Following the S3AFileSystem integration patch in HADOOP-13651, we need to add
> delete tracking.
> Current behavior on delete is to remove the metadata from the MetadataStore.
> To make deletes consistent, we need to add a {{isDeleted}} flag to
> {{PathMetadata}} and check it when returning results from functions like
> {{getFileStatus()}} and {{listStatus()}}. In HADOOP-13651, I added TODO
> comments in most of the places these new conditions are needed. The work
> does not look too bad.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
