[
https://issues.apache.org/jira/browse/HADOOP-13760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15981907#comment-15981907
]
Sean Mackrory commented on HADOOP-13760:
----------------------------------------
Thanks, [~fabbri]. I've eliminated the redundant try/catch and gotten rid of
the switch statement with fall-through. I also switched from integer constants
for the checkPathForDirectory return value to an enum. Didn't update the patch
yet, though.
Regarding innerRename, there's no refactoring - all the changes are because of
the change in what object I'm looping through. The whitespace and variable name
change incident to that makes it look like a huge change, but if you interleave
the changes line-by-line it looks much less invasive. I'll see if I can clean
it up at all, though.
There's a function call at the end of innerRename that cleans up any
unnecessary directory markers, which is why no tests are seeing the changes you
predict. There's still a concern if there are unnecessary round trips, though.
One thing I'm considering is to change the function back and simply add a
separate call that just gets deleted items and doesn't consider them for the
rename operation. My only concern with that is that when that becomes an
operation that's too big to return all the results at once, we'll need to make
a change (not sure if we can sort by key / length) so we don't have context
that's needed for an S3 object further back in the Metadatastore iterator. Make
sense? I'll think about some solutions here and look at the problem some more.
> S3Guard: add delete tracking
> ----------------------------
>
> Key: HADOOP-13760
> URL: https://issues.apache.org/jira/browse/HADOOP-13760
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Aaron Fabbri
> Assignee: Sean Mackrory
> Attachments: HADOOP-13760-HADOOP-13345.001.patch,
> HADOOP-13760-HADOOP-13345.002.patch
>
>
> Following the S3AFileSystem integration patch in HADOOP-13651, we need to add
> delete tracking.
> Current behavior on delete is to remove the metadata from the MetadataStore.
> To make deletes consistent, we need to add a {{isDeleted}} flag to
> {{PathMetadata}} and check it when returning results from functions like
> {{getFileStatus()}} and {{listStatus()}}. In HADOOP-13651, I added TODO
> comments in most of the places these new conditions are needed. The work
> does not look too bad.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
