+1, Sorry for the late vote. Thanks for the perseverance and seeing this thru.
--Anu
On 8/7/18, 1:04 AM, "郑锴(铁杰)" <zhengkai...@alibaba-inc.com> wrote:
+1 for the work to be in. Thanks Uma and folks for the hard taking!
When it's in, I'd suggest we use a more general name for the new daemon
service. It'd be good to evolve and support more self-running admin
functionalities incubated first there before doing it directly in NameNode.
Regards,
Kai
------------------------------------------------------------------
发件人：Uma Maheswara Rao G <hadoop....@gmail.com>
发送时间：2018年8月1日(星期三) 14:38
收件人：hdfs-dev <hdfs-dev@hadoop.apache.org>
主 题：[VOTE] Merge Storage Policy Satisfier (SPS) [HDFS-10285] feature branch
to trunk
Hi All,
From the positive responses from JIRA discussion and no objections from
below DISCUSS thread [1], I am converting it to voting thread.
Last couple of weeks we spent time on testing the feature and so far it is
working fine. Surendra uploaded a test report at HDFS-10285: [2]
In this phase, we provide to run SPS outside of Namenode only and as a
next phase we continue to discuss and work on to enable it as Internal SPS
as explained below. We have got clean QA report on branch and if there are
any static tool comments triggered later while running this thread, we will
make sure to fix them before merge. We committed and continue to improve
the code on trunk. Please refer to HDFS-10285 for discussion details.
This has been a long effort and we're grateful for the support we've
received from the community. In particular, thanks to Andrew Wang, Anoop
Sam John, Anu Engineer, Chris Douglas, Daryn Sharp, Du Jingcheng , Ewan
Higgs, Jing Zhao, Kai Zheng, Rakesh R, Ramkrishna , Surendra Singh Lilhore
, Thomas Demoor, Uma Maheswara Rao G, Vinayakumar, Virajith, Wei Zhou,
Yuanbo Liu. Without these members effort, this feature might not have
reached to this state.
To start with, here is my +1
It will end on 6th Aug.
Regards,
Uma
[1] https://s.apache.org/bhyu
[2] https://s.apache.org/AXvL
On Wed, Jun 27, 2018 at 3:21 PM, Uma Maheswara Rao G <hadoop....@gmail.com>
wrote:
> Hi All,
>
> After long discussions(offline and on JIRA) on SPS, we came to a
> conclusion on JIRA(HDFS-10285) that, we will go ahead with External SPS
> merge in first phase. In this phase process will not be running inside
> Namenode.
> We will continue discussion on Internal SPS. Current code base supports
> both internal and external option. We have review comments for Internal
> which needs some additional works for analysis and testing etc. We will
> move Internal SPS work to under HDFS-12226 (Follow-on work for SPS in NN)
> We are working on cleanup task HDFS-13076 for the merge. .
> For more clarity on Internal and External SPS proposal thoughts, please
> refer to JIRA HDFS-10285.
>
> If there are no objections with this, I will go ahead for voting soon.
>
> Regards,
> Uma
>
> On Fri, Nov 17, 2017 at 3:16 PM, Uma Maheswara Rao G <hadoop....@gmail.com
> > wrote:
>
>> Update: We worked on the review comments and additional JIRAs above
>> mentioned.
>>
>> >1. After the feedbacks from Andrew, Eddy, Xiao in JIRA reviews, we
>> planned to take up the support for recursive API support. HDFS-12291<
>> https://issues.apache.org/jira/browse/HDFS-12291>
>>
>> We provided the recursive API support now.
>>
>> >2. Xattr optimizations HDFS-12225<https://issues.apac
>> he.org/jira/browse/HDFS-12225>
>> Improved this portion as well
>>
>> >3. Few other review comments already fixed and committed HDFS-12214<
>> https://issues.apache.org/jira/browse/HDFS-12214>
>> Fixed the comments.
>>
>> We are continuing to test the feature and working so far well. Also we
>> uploaded a combined patch and got the good QA report.
>>
>> If there are no further objections, we would like to go for merge vote
>> tomorrow. Please by default this feature will be disabled.
>>
>> Regards,
>> Uma
>>
>> On Fri, Aug 18, 2017 at 11:27 PM, Gangumalla, Uma <
>> uma.ganguma...@intel.com> wrote:
>>
>>> Hi Andrew,
>>>
>>> >Great to hear. It'd be nice to define which use cases are met by the
>>> current version of SPS, and which will be handled after the merge.
>>> After the discussions in JIRA, we planned to support recursive API as
>>> well. The primary use cases we planned was for Hbase. Please check next
>>> point for use case details.
>>>
>>> >A bit more detail in the design doc on how HBase would use this feature
>>> would also be helpful. Is there an HBase JIRA already?
>>> Please find the usecase details at this comment in JIRA:
>>> https://issues.apache.org/jira/browse/HDFS-10285?focusedComm
>>> entId=16120227&page=com.atlassian.jira.plugin.system.issueta
>>> bpanels:comment-tabpanel#comment-16120227
>>>
>>> >I also spent some more time with the design doc and posted a few
>>> questions on the JIRA.
>>> Thank you for the reviews.
>>>
>>> To summarize the discussions in JIRA:
>>> 1. After the feedbacks from Andrew, Eddy, Xiao in JIRA reviews, we
>>> planned to take up the support for recursive API support. HDFS-12291<
>>> https://issues.apache.org/jira/browse/HDFS-12291> (Rakesh started the
>>> work on it)
>>> 2. Xattr optimizations HDFS-12225<https://issues.apac
>>> he.org/jira/browse/HDFS-12225> (Patch available)
>>> 3. Few other review comments already fixed and committed HDFS-12214<
>>> https://issues.apache.org/jira/browse/HDFS-12214>
>>>
>>> For tracking the follow-up tasks we filed JIRA HDFS-12226, they should
>>> not be critical for merge.
>>>
>>> Regards,
>>> Uma
>>>
>>> From: Andrew Wang <andrew.w...@cloudera.com<mailto:
>>> andrew.w...@cloudera.com>>
>>> Date: Friday, July 28, 2017 at 11:33 AM
>>> To: Uma Gangumalla <uma.ganguma...@intel.com<mailto:
>>> uma.ganguma...@intel.com>>
>>> Cc: "hdfs-dev@hadoop.apache.org<mailto:hdfs-dev@hadoop.apache.org>" <
>>> hdfs-dev@hadoop.apache.org<mailto:hdfs-dev@hadoop.apache.org>>
>>> Subject: Re: [DISCUSS] Merge Storage Policy Satisfier (SPS) [HDFS-10285]
>>> feature branch to trunk
>>>
>>> Hi Uma,
>>>
>>> > If there are still plans to make changes that affect compatibility
>>> (the hybrid RPC and bulk DN work mentioned sound like they would), then
we
>>> can cut branch-3 first, or wait to merge until after these tasks are
>>> finished.
>>> [Uma] We don’t see that 2 items as high priority for the feature. Users
>>> would be able to use the feature with current code base and API. So, we
>>> would consider them after branch-3 only. That should be perfectly fine
IMO.
>>> The current API is very much useful for Hbase scenario. In Hbase case,
they
>>> will rename files under to different policy directory. They will not set
>>> the policies always. So, when rename files under to different policy
>>> directory, they can simply call satisfyStoragePolicy, they don’t need
any
>>> hybrid API.
>>>
>>> Great to hear. It'd be nice to define which usecases are met by the
>>> current version of SPS, and which will be handled after the merge.
>>>
>>> A bit more detail in the design doc on how HBase would use this feature
>>> would also be helpful. Is there an HBase JIRA already?
>>>
>>> I also spent some more time with the design doc and posted a few
>>> questions on the JIRA.
>>>
>>> Best,
>>> Andrew
>>>
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
