[
https://issues.apache.org/jira/browse/HADOOP-13448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15469109#comment-15469109
]
Aaron Fabbri commented on HADOOP-13448:
---------------------------------------
{quote}Are you interested in doing another revision, or should I?{quote}
Welcome back. You can do next revision. I'm here to help, though, so shout if
you'd like me to do the follow up.
...
{quote}
This is nice, but I expect in practice we'll instantiate through
ReflectionUtils#newInstance and lose the type bounds unless we do an unsafe
cast.
{quote}
Ah, ok.. We can drop the type parameter and just use {{FileStatus}}, I guess.
{quote}
The presence of the DirListingMetadata#put method implies mutable usage. Should
this class provide thread safety, or is there an expectation that the caller
will enforce thread-safe access? Either way, it would be good to add a comment
about thread safety to the class-level JavaDocs.
{quote}
Good point. Intention is that put() is only called inside MetadataStore
implementations. We should probably enforce that somehow (package private?).
{quote}I suppose a higher-level decision for all of the above is mutable vs.
immutable objects. {quote}
I agree with your points here. Both mutable and immutable DirListingMetadata
ideas seem to have significant downsides, as you say, depending on application.
I feel like we can make DirListingMetadata expose a thread-safe API independent
of whether or not the contents can change during a reference's lifetime. The
lifetime behavior may be important to clients, however. This reminds me of the
GCS link you shared before: They [expose a
flag|https://github.com/GoogleCloudPlatform/bigdata-interop/blob/1447da82f2bded2ac8493b07797a5c2483b70497/gcsio/src/main/java/com/google/cloud/hadoop/gcsio/DirectoryListCache.java#L128]
that seems to inform clients about the lifetime of the data. We could do
something similar, with the LocalMetadataStore (in-memory) returning true, and
something like the DynamoDB MetadataStore returning false.
{quote}
Does DirListingMetadata need a method to return all children in the listing? In
your prototype on HADOOP-13345, there was CachedDirectory#getFileStatuses.
Similarly, does DirListingMetadata need mutator methods for isAuthoritative?
{quote}
Yes, and yes. We'll also need a remove() on DirListingMetadata (I added one
for the LocalMetadataStore I'm working on).
> S3Guard: Define MetadataStore interface.
> ----------------------------------------
>
> Key: HADOOP-13448
> URL: https://issues.apache.org/jira/browse/HADOOP-13448
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Chris Nauroth
> Assignee: Chris Nauroth
> Attachments: HADOOP-13448-HADOOP-13345.001.patch,
> HADOOP-13448-HADOOP-13345.002.patch
>
>
> Define the common interface for metadata store operations. This is the
> interface that any metadata back-end must implement in order to integrate
> with S3Guard.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
