+1 Great work guys. -Vinay
On Tue, 7 Aug 2018, 7:20 pm Anu Engineer, <aengin...@hortonworks.com> wrote: > +1, Sorry for the late vote. Thanks for the perseverance and seeing this > thru. > > --Anu > > > On 8/7/18, 1:04 AM, "郑锴(铁杰)" <zhengkai...@alibaba-inc.com> wrote: > > +1 for the work to be in. Thanks Uma and folks for the hard taking! > > When it's in, I'd suggest we use a more general name for the new > daemon service. It'd be good to evolve and support more self-running admin > functionalities incubated first there before doing it directly in NameNode. > > Regards, > Kai > ------------------------------------------------------------------ > 发件人：Uma Maheswara Rao G <hadoop....@gmail.com> > 发送时间：2018年8月1日(星期三) 14:38 > 收件人：hdfs-dev <hdfs-dev@hadoop.apache.org> > 主 题：[VOTE] Merge Storage Policy Satisfier (SPS) [HDFS-10285] feature > branch to trunk > > Hi All, > > > > From the positive responses from JIRA discussion and no objections > from > below DISCUSS thread [1], I am converting it to voting thread. > > > > Last couple of weeks we spent time on testing the feature and so far > it is > working fine. Surendra uploaded a test report at HDFS-10285: [2] > > > > In this phase, we provide to run SPS outside of Namenode only and as a > next phase we continue to discuss and work on to enable it as Internal > SPS > as explained below. We have got clean QA report on branch and if there > are > any static tool comments triggered later while running this thread, we > will > make sure to fix them before merge. We committed and continue to > improve > the code on trunk. Please refer to HDFS-10285 for discussion details. > > > > This has been a long effort and we're grateful for the support we've > received from the community. In particular, thanks to Andrew Wang, > Anoop > Sam John, Anu Engineer, Chris Douglas, Daryn Sharp, Du Jingcheng , Ewan > Higgs, Jing Zhao, Kai Zheng, Rakesh R, Ramkrishna , Surendra Singh > Lilhore > , Thomas Demoor, Uma Maheswara Rao G, Vinayakumar, Virajith, Wei Zhou, > Yuanbo Liu. Without these members effort, this feature might not have > reached to this state. > > > > To start with, here is my +1 > > It will end on 6th Aug. > > > > Regards, > > Uma > > [1] https://s.apache.org/bhyu > [2] https://s.apache.org/AXvL > > > On Wed, Jun 27, 2018 at 3:21 PM, Uma Maheswara Rao G < > hadoop....@gmail.com> > wrote: > > > Hi All, > > > > After long discussions(offline and on JIRA) on SPS, we came to a > > conclusion on JIRA(HDFS-10285) that, we will go ahead with External > SPS > > merge in first phase. In this phase process will not be running > inside > > Namenode. > > We will continue discussion on Internal SPS. Current code base > supports > > both internal and external option. We have review comments for > Internal > > which needs some additional works for analysis and testing etc. We > will > > move Internal SPS work to under HDFS-12226 (Follow-on work for SPS > in NN) > > We are working on cleanup task HDFS-13076 for the merge. . > > For more clarity on Internal and External SPS proposal thoughts, > please > > refer to JIRA HDFS-10285. > > > > If there are no objections with this, I will go ahead for voting > soon. > > > > Regards, > > Uma > > > > On Fri, Nov 17, 2017 at 3:16 PM, Uma Maheswara Rao G < > hadoop....@gmail.com > > > wrote: > > > >> Update: We worked on the review comments and additional JIRAs above > >> mentioned. > >> > >> >1. After the feedbacks from Andrew, Eddy, Xiao in JIRA reviews, we > >> planned to take up the support for recursive API support. > HDFS-12291< > >> https://issues.apache.org/jira/browse/HDFS-12291> > >> > >> We provided the recursive API support now. > >> > >> >2. Xattr optimizations HDFS-12225<https://issues.apac > >> he.org/jira/browse/HDFS-12225> > >> Improved this portion as well > >> > >> >3. Few other review comments already fixed and committed > HDFS-12214< > >> https://issues.apache.org/jira/browse/HDFS-12214> > >> Fixed the comments. > >> > >> We are continuing to test the feature and working so far well. Also > we > >> uploaded a combined patch and got the good QA report. > >> > >> If there are no further objections, we would like to go for merge > vote > >> tomorrow. Please by default this feature will be disabled. > >> > >> Regards, > >> Uma > >> > >> On Fri, Aug 18, 2017 at 11:27 PM, Gangumalla, Uma < > >> uma.ganguma...@intel.com> wrote: > >> > >>> Hi Andrew, > >>> > >>> >Great to hear. It'd be nice to define which use cases are met by > the > >>> current version of SPS, and which will be handled after the merge. > >>> After the discussions in JIRA, we planned to support recursive API > as > >>> well. The primary use cases we planned was for Hbase. Please check > next > >>> point for use case details. > >>> > >>> >A bit more detail in the design doc on how HBase would use this > feature > >>> would also be helpful. Is there an HBase JIRA already? > >>> Please find the usecase details at this comment in JIRA: > >>> https://issues.apache.org/jira/browse/HDFS-10285?focusedComm > >>> entId=16120227&page=com.atlassian.jira.plugin.system.issueta > >>> bpanels:comment-tabpanel#comment-16120227 > >>> > >>> >I also spent some more time with the design doc and posted a few > >>> questions on the JIRA. > >>> Thank you for the reviews. > >>> > >>> To summarize the discussions in JIRA: > >>> 1. After the feedbacks from Andrew, Eddy, Xiao in JIRA reviews, we > >>> planned to take up the support for recursive API support. > HDFS-12291< > >>> https://issues.apache.org/jira/browse/HDFS-12291> (Rakesh started > the > >>> work on it) > >>> 2. Xattr optimizations HDFS-12225<https://issues.apac > >>> he.org/jira/browse/HDFS-12225> (Patch available) > >>> 3. Few other review comments already fixed and committed > HDFS-12214< > >>> https://issues.apache.org/jira/browse/HDFS-12214> > >>> > >>> For tracking the follow-up tasks we filed JIRA HDFS-12226, they > should > >>> not be critical for merge. > >>> > >>> Regards, > >>> Uma > >>> > >>> From: Andrew Wang <andrew.w...@cloudera.com<mailto: > >>> andrew.w...@cloudera.com>> > >>> Date: Friday, July 28, 2017 at 11:33 AM > >>> To: Uma Gangumalla <uma.ganguma...@intel.com<mailto: > >>> uma.ganguma...@intel.com>> > >>> Cc: "hdfs-dev@hadoop.apache.org<mailto:hdfs-dev@hadoop.apache.org>" > < > >>> hdfs-dev@hadoop.apache.org<mailto:hdfs-dev@hadoop.apache.org>> > >>> Subject: Re: [DISCUSS] Merge Storage Policy Satisfier (SPS) > [HDFS-10285] > >>> feature branch to trunk > >>> > >>> Hi Uma, > >>> > >>> > If there are still plans to make changes that affect > compatibility > >>> (the hybrid RPC and bulk DN work mentioned sound like they would), > then we > >>> can cut branch-3 first, or wait to merge until after these tasks > are > >>> finished. > >>> [Uma] We don’t see that 2 items as high priority for the feature. > Users > >>> would be able to use the feature with current code base and API. > So, we > >>> would consider them after branch-3 only. That should be perfectly > fine IMO. > >>> The current API is very much useful for Hbase scenario. In Hbase > case, they > >>> will rename files under to different policy directory. They will > not set > >>> the policies always. So, when rename files under to different > policy > >>> directory, they can simply call satisfyStoragePolicy, they don’t > need any > >>> hybrid API. > >>> > >>> Great to hear. It'd be nice to define which usecases are met by the > >>> current version of SPS, and which will be handled after the merge. > >>> > >>> A bit more detail in the design doc on how HBase would use this > feature > >>> would also be helpful. Is there an HBase JIRA already? > >>> > >>> I also spent some more time with the design doc and posted a few > >>> questions on the JIRA. > >>> > >>> Best, > >>> Andrew > >>> > >> > >> > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org >
