[ https://issues.apache.org/jira/browse/HADOOP-15038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16257288#comment-16257288 ]
Steve Loughran commented on HADOOP-15038: ----------------------------------------- Good point about placement. To date I've been adding dependency-less cross-store operations into hadoop-common. But with things like my proposed HADOOP-14766 cloudup function push for us having a common hadoop-cloud-core module with * common code * cli entry point * common (abstract/real) tests unique to stores (e.g. metastore ones here) This is different from the pom only hadoop-cloud-components module which can pull in the various cloud bindings: this would come above it. > Abstract MetadataStore in S3Guard into a common module. > ------------------------------------------------------- > > Key: HADOOP-15038 > URL: https://issues.apache.org/jira/browse/HADOOP-15038 > Project: Hadoop Common > Issue Type: New Feature > Components: fs > Affects Versions: 3.0.0-beta1 > Reporter: Genmao Yu > > Open this JIRA to discuss if we should move {{MetadataStore}} in {{S3Guard}} > into a common module. > Based on this work, other filesystem or object store can implement their own > metastore for optimization (known issues like consistency problem and > metadata operation performance). [~ste...@apache.org] and other guys have > done many base and great works in {{S3Guard}}. It is very helpful to start > work. I did some perf test in HADOOP-14098, and started related work for > Aliyun OSS. Indeed there are still works to do for {{S3Guard}}, like > metadata cache inconsistent with S3 and so on. It also will be a problem for > other object store. However, we can do these works in parallel. > [~ste...@apache.org] [~fabbri] [~drankye] Any suggestion is appreciated. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org