[jira] [Updated] (HADOOP-13631) S3Guard: implement move() for LocalMetadataStore, add unit tests
[ https://issues.apache.org/jira/browse/HADOOP-13631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13631: -- Resolution: Fixed Fix Version/s: HADOOP-13345 Status: Resolved (was: Patch Available) > S3Guard: implement move() for LocalMetadataStore, add unit tests > > > Key: HADOOP-13631 > URL: https://issues.apache.org/jira/browse/HADOOP-13631 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Reporter: Aaron Fabbri >Assignee: Aaron Fabbri > Fix For: HADOOP-13345 > > Attachments: HADOOP-13631-HADOOP-13345.001.patch > > > Building on HADOOP-13573 and HADOOP-13452, implement move() in > LocalMetadataStore and associated MetadataStore unit tests. > (Making this a separate JIRA to break up work into decent-sized and > reviewable chunks.) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13631) S3Guard: implement move() for LocalMetadataStore, add unit tests
[
https://issues.apache.org/jira/browse/HADOOP-13631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HADOOP-13631:
---
Hadoop Flags: Reviewed
+1, assuming [~eddyxu] thinks his feedback has been resolved satisfactorily.
I think this {{move}} interface does make sense based on our current goals with
metadata backed by DynamoDB. Eddy has a good point that some implementations
might operate better by working with the source and destination path, like the
existing interface. I could imagine an implementation backed by a simplified
NameNode or maybe NFS (though I'm generally wary of NFS high availability
capabilities). When that comes up, maybe the interface could evolve to include
both the collections and the source and destination path, and the
implementation can decide which parameters to use.
I think we'll have freedom to evolve the {{MetadataStore}} interface. We're
not planning to make it public with strict backward compatibility guarantees,
so we can change it easily later.
> S3Guard: implement move() for LocalMetadataStore, add unit tests
>
>
> Key: HADOOP-13631
> URL: https://issues.apache.org/jira/browse/HADOOP-13631
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Reporter: Aaron Fabbri
>Assignee: Aaron Fabbri
> Attachments: HADOOP-13631-HADOOP-13345.001.patch
>
>
> Building on HADOOP-13573 and HADOOP-13452, implement move() in
> LocalMetadataStore and associated MetadataStore unit tests.
> (Making this a separate JIRA to break up work into decent-sized and
> reviewable chunks.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13631) S3Guard: implement move() for LocalMetadataStore, add unit tests
[ https://issues.apache.org/jira/browse/HADOOP-13631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13631: -- Status: Patch Available (was: In Progress) > S3Guard: implement move() for LocalMetadataStore, add unit tests > > > Key: HADOOP-13631 > URL: https://issues.apache.org/jira/browse/HADOOP-13631 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Reporter: Aaron Fabbri >Assignee: Aaron Fabbri > Attachments: HADOOP-13631-HADOOP-13345.001.patch > > > Building on HADOOP-13573 and HADOOP-13452, implement move() in > LocalMetadataStore and associated MetadataStore unit tests. > (Making this a separate JIRA to break up work into decent-sized and > reviewable chunks.) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13631) S3Guard: implement move() for LocalMetadataStore, add unit tests
[ https://issues.apache.org/jira/browse/HADOOP-13631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13631: -- Attachment: HADOOP-13631-HADOOP-13345.001.patch > S3Guard: implement move() for LocalMetadataStore, add unit tests > > > Key: HADOOP-13631 > URL: https://issues.apache.org/jira/browse/HADOOP-13631 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Reporter: Aaron Fabbri >Assignee: Aaron Fabbri > Attachments: HADOOP-13631-HADOOP-13345.001.patch > > > Building on HADOOP-13573 and HADOOP-13452, implement move() in > LocalMetadataStore and associated MetadataStore unit tests. > (Making this a separate JIRA to break up work into decent-sized and > reviewable chunks.) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13631) S3Guard: implement move() for LocalMetadataStore, add unit tests
[ https://issues.apache.org/jira/browse/HADOOP-13631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13631: -- Attachment: (was: HADOOP-13631-HADOOP-13345.001.patch) > S3Guard: implement move() for LocalMetadataStore, add unit tests > > > Key: HADOOP-13631 > URL: https://issues.apache.org/jira/browse/HADOOP-13631 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Reporter: Aaron Fabbri >Assignee: Aaron Fabbri > > Building on HADOOP-13573 and HADOOP-13452, implement move() in > LocalMetadataStore and associated MetadataStore unit tests. > (Making this a separate JIRA to break up work into decent-sized and > reviewable chunks.) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13631) S3Guard: implement move() for LocalMetadataStore, add unit tests
[
https://issues.apache.org/jira/browse/HADOOP-13631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron Fabbri updated HADOOP-13631:
--
Attachment: HADOOP-13631-HADOOP-13345.001.patch
Attaching initial patch. I will modify this based on our discussion on
HADOOP-13448 about the move() interface.
{quote}
HADOOP-13631 S3Guard: implement move() for LocalMetadataStore, add unit tests
This is an initial RFC patch. Based on conversation on HADOOP-13448 we may
simplify the move() interface I've proposed here.
Also, change DirListingMetadata#setAuthoritative() to take a boolean arg.
Setting this to false is a way to trigger clients to re-consult the backing
store if, for example, a user adds new files to a directory.
{quote}
I think we probably need another sub-jira for implementing delete tracking. I
did not implement that here as it affects other code and may require other
interface changes.
> S3Guard: implement move() for LocalMetadataStore, add unit tests
>
>
> Key: HADOOP-13631
> URL: https://issues.apache.org/jira/browse/HADOOP-13631
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Reporter: Aaron Fabbri
>Assignee: Aaron Fabbri
> Attachments: HADOOP-13631-HADOOP-13345.001.patch
>
>
> Building on HADOOP-13573 and HADOOP-13452, implement move() in
> LocalMetadataStore and associated MetadataStore unit tests.
> (Making this a separate JIRA to break up work into decent-sized and
> reviewable chunks.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
