[jira] [Commented] (HDFS-7674) Adding metrics for Erasure Coding

2015-05-13 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14541542#comment-14541542 ] Wei Zhou commented on HDFS-7674: Sure, thanks! Adding metrics for Erasure Coding

[jira] [Commented] (HDFS-7674) Adding metrics for Erasure Coding

2015-05-13 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14541541#comment-14541541 ] Wei Zhou commented on HDFS-7674: Sure, thanks! Adding metrics for Erasure Coding

[jira] [Updated] (HDFS-8562) HDFS Performance is impacted by FileInputStream Finalizer

2015-12-11 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-8562: --- Attachment: HDFS-8562.01.patch This is the patch mentioned in Kai's comment. This patch is for POC and

[jira] [Updated] (HDFS-9608) Disk IO imbalance in HDFS with heterogeneous storages

2016-01-03 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-9608: --- Attachment: HDFS-9608.01.patch This patch insure choosing volumes in RR mode for each storage type. Besides,

[jira] [Created] (HDFS-9608) Disk IO imbalance in HDFS with heterogeneous storages

2016-01-03 Thread Wei Zhou (JIRA)
Wei Zhou created HDFS-9608: -- Summary: Disk IO imbalance in HDFS with heterogeneous storages Key: HDFS-9608 URL: https://issues.apache.org/jira/browse/HDFS-9608 Project: Hadoop HDFS Issue Type: Bug

[jira] [Updated] (HDFS-9608) Disk IO imbalance in HDFS with heterogeneous storages

2016-01-06 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-9608: --- Description: Currently RoundRobinVolumeChoosingPolicy use a shared index to choose volumes in HDFS with

[jira] [Updated] (HDFS-9608) Disk IO imbalance in HDFS with heterogeneous storages

2016-01-06 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-9608: --- Description: Currently RoundRobinVolumeChoosingPolicy use a shared index to choose volumes in HDFS with

[jira] [Updated] (HDFS-9608) Disk IO imbalance in HDFS with heterogeneous storages

2016-01-06 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-9608: --- Attachment: HDFS-9608.02.patch Thanks Kai for the helpful suggestions! Modifications made to the previous

[jira] [Updated] (HDFS-9608) Disk IO imbalance in HDFS with heterogeneous storages

2016-01-15 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-9608: --- Attachment: HDFS-9608.03.patch Thanks for reviewing the patch. The patch is modified as suggested, thanks! >

[jira] [Updated] (HDFS-9608) Disk IO imbalance in HDFS with heterogeneous storages

2016-01-17 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-9608: --- Attachment: HDFS-9608.04.patch Sorry for the issue. Thank you for review! > Disk IO imbalance in HDFS with

[jira] [Updated] (HDFS-9608) Disk IO imbalance in HDFS with heterogeneous storages

2016-01-18 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-9608: --- Attachment: HDFS-9608.05.patch Fix some test failure issues. > Disk IO imbalance in HDFS with heterogeneous

[jira] [Updated] (HDFS-9608) Disk IO imbalance in HDFS with heterogeneous storages

2016-01-20 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-9608: --- Attachment: HDFS-9608.07.patch Thanks for review! Synchronize removed in this patch. Thanks! > Disk IO

[jira] [Updated] (HDFS-9608) Disk IO imbalance in HDFS with heterogeneous storages

2016-01-19 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-9608: --- Status: Patch Available (was: Open) Resubmit the patch to rerun the check! > Disk IO imbalance in HDFS with

[jira] [Updated] (HDFS-9608) Disk IO imbalance in HDFS with heterogeneous storages

2016-01-19 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-9608: --- Status: Open (was: Patch Available) > Disk IO imbalance in HDFS with heterogeneous storages >

[jira] [Updated] (HDFS-9608) Disk IO imbalance in HDFS with heterogeneous storages

2016-01-20 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-9608: --- Attachment: HDFS-9608.06.patch Thanks Vinayakumar for the suggestions! Code refine and add fine-grained locks

[jira] [Commented] (HDFS-9608) Disk IO imbalance in HDFS with heterogeneous storages

2016-01-20 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15108486#comment-15108486 ] Wei Zhou commented on HDFS-9608: Thanks for your work! Code refined as your suggestion. For question 2, I

[jira] [Updated] (HDFS-11239) [SPS]: Check Mover file ID lease also to determine whether Mover is running

2017-02-02 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11239: Attachment: HDFS-11239-HDFS-10285.04.patch Rebase the patch. > [SPS]: Check Mover file ID lease also to

[jira] [Commented] (HDFS-11239) [SPS]: Check Mover file ID lease also to determine whether Mover is running

2017-02-02 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851091#comment-15851091 ] Wei Zhou commented on HDFS-11239: - Some of test failures are caused by timeout mentioned above and the

[jira] [Commented] (HDFS-11239) [SPS]: Check Mover file ID lease also to determine whether Mover is running

2017-01-20 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832834#comment-15832834 ] Wei Zhou commented on HDFS-11239: - Hi, I'm on vacation from 1/21/2017 to 2/2/2017 and back in 2/3/2017.

[jira] [Commented] (HDFS-7343) HDFS smart storage management

2017-02-14 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867046#comment-15867046 ] Wei Zhou commented on HDFS-7343: After discussed with [~andrew.wang] and [~drankye], in order to make the

[jira] [Comment Edited] (HDFS-7343) HDFS smart storage management

2017-02-14 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867046#comment-15867046 ] Wei Zhou edited comment on HDFS-7343 at 2/15/17 1:59 AM: - After discussed with

[jira] [Updated] (HDFS-11239) [SPS]: Check Mover file ID lease also to determine whether Mover is running

2017-01-19 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11239: Attachment: HDFS-11239-HDFS-10285.03.patch Thanks [~umamaheswararao] for reviewing the patch and the

[jira] [Updated] (HDFS-11338) [SPS]: Fix timeout issue in unit tests caused by longger NN down time

2017-02-27 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11338: Attachment: HDFS-11338-HDFS-10285.01.patch Fix TestPersistentStoragePolicySatisfier timeout issue. >

[jira] [Updated] (HDFS-11338) [SPS]: Fix timeout issue in unit tests caused by longger NN down time

2017-02-27 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11338: Attachment: HDFS-11338-HDFS-10285.00.patch The initial patch. > [SPS]: Fix timeout issue in unit tests

[jira] [Updated] (HDFS-11338) [SPS]: Fix timeout issue in unit tests caused by longger NN down time

2017-02-27 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11338: Status: Patch Available (was: Open) > [SPS]: Fix timeout issue in unit tests caused by longger NN down

[jira] [Commented] (HDFS-10885) Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-09-21 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15509302#comment-15509302 ] Wei Zhou commented on HDFS-10885: - Yes, you are right. Thanks for your reminding! > Mover tool should not

[jira] [Created] (HDFS-10885) Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-09-21 Thread Wei Zhou (JIRA)
Wei Zhou created HDFS-10885: --- Summary: Mover tool should not be allowed to run when Storage Policy Satisfier is on Key: HDFS-10885 URL: https://issues.apache.org/jira/browse/HDFS-10885 Project: Hadoop HDFS

[jira] [Updated] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-09-21 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-10885: Summary: [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on (was: Mover

[jira] [Updated] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-11-08 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-10885: Attachment: HDFS-10885-HDFS-10285.06.patch Thanks [~rakeshr] and [~umamaheswararao] for the comments! The

[jira] [Updated] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-11-08 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-10885: Attachment: HDFS-10885-HDFS-10285.07.patch Fix issues reported by Jenkins. The following unit test failures

[jira] [Commented] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-11-06 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15643150#comment-15643150 ] Wei Zhou commented on HDFS-10885: - Further, there is a logical issue here, it's quite strange for SPS to

[jira] [Commented] (HDFS-7343) HDFS smart storage management

2016-10-18 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587704#comment-15587704 ] Wei Zhou commented on HDFS-7343: Thanks [~anu] for reviewing the design document and great comments! For

[jira] [Commented] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-10-19 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587938#comment-15587938 ] Wei Zhou commented on HDFS-10885: - The failed test

[jira] [Updated] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-10-18 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-10885: Attachment: HDFS-10885-HDFS-10285.04.patch This patch fixes the testing errors report by Jenkins. Thanks!

[jira] [Commented] (HDFS-7343) HDFS smart storage management

2016-10-18 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15585033#comment-15585033 ] Wei Zhou commented on HDFS-7343: Thanks all for reviewing and great comments. I'm thinking about this and

[jira] [Commented] (HDFS-7343) HDFS smart storage management

2016-10-20 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15590949#comment-15590949 ] Wei Zhou commented on HDFS-7343: Hi [~andrew.wang], thank you for reviewing the document and great

[jira] [Commented] (HDFS-7343) HDFS smart storage management

2016-10-20 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15592342#comment-15592342 ] Wei Zhou commented on HDFS-7343: Continue on comments from [~andrew.wang], {quote} Could you talk a little

[jira] [Commented] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-10-20 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15592413#comment-15592413 ] Wei Zhou commented on HDFS-10885: - Hi [~umamaheswararao], thank you very much for reviewing this patch.

[jira] [Commented] (HDFS-7343) HDFS smart storage management

2016-10-23 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15601016#comment-15601016 ] Wei Zhou commented on HDFS-7343: Thanks [~xiaochen] for reviewing and the great comments. I'm busy with

[jira] [Commented] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-10-24 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15601301#comment-15601301 ] Wei Zhou commented on HDFS-10885: - Ok, that's good! I'll update the patch while waiting for [~rakeshr]'s

[jira] [Updated] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-10-17 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-10885: Attachment: HDFS-10800-HDFS-10885-02.patch Reupload the patch to trigger Q Thanks! > [SPS]: Mover tool

[jira] [Updated] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-10-17 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-10885: Attachment: HDFS-10885-HDFS-10285.03.patch Patch renamed for branch HDFS-10285. Thanks! > [SPS]: Mover

[jira] [Commented] (HDFS-7343) HDFS smart storage management

2016-10-25 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15604535#comment-15604535 ] Wei Zhou commented on HDFS-7343: Thanks [~andrew.wang] for the replies! I totally agree with your comments

[jira] [Commented] (HDFS-7343) HDFS smart storage management

2016-10-24 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15602545#comment-15602545 ] Wei Zhou commented on HDFS-7343: Hi, [~andrew.wang], thanks for the replies! {quote} Do you also plan to

[jira] [Updated] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-11-28 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-10885: Attachment: HDFS-10885-HDFS-10285.08.patch Hi, [~rakeshr], the patch is updated. After discussed offline

[jira] [Updated] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-11-28 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-10885: Attachment: HDFS-10885-HDFS-10285.09.patch Thanks [~rakeshr] for the suggestions! The failures reported are

[jira] [Created] (HDFS-11186) [SPS]: Daemon thread of SPS starts only in Active NN

2016-11-28 Thread Wei Zhou (JIRA)
Wei Zhou created HDFS-11186: --- Summary: [SPS]: Daemon thread of SPS starts only in Active NN Key: HDFS-11186 URL: https://issues.apache.org/jira/browse/HDFS-11186 Project: Hadoop HDFS Issue Type:

[jira] [Commented] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-11-24 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15693623#comment-15693623 ] Wei Zhou commented on HDFS-10885: - Thanks [~rakeshr] for the suggestions! I'll update the patch. Thanks!

[jira] [Commented] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-11-21 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15685936#comment-15685936 ] Wei Zhou commented on HDFS-10885: - Hi, [~rakeshr], sorry for the late response sincerely! I was on travel

[jira] [Comment Edited] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-11-21 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15685936#comment-15685936 ] Wei Zhou edited comment on HDFS-10885 at 11/22/16 7:26 AM: --- Hi, [~rakeshr],

[jira] [Commented] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-11-01 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15625018#comment-15625018 ] Wei Zhou commented on HDFS-10885: - Thanks [~rakeshr] for these considerate suggestions! {quote} Do you

[jira] [Commented] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-11-03 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15633265#comment-15633265 ] Wei Zhou commented on HDFS-10885: - Thanks [~rakeshr] for the comments! {quote} Alternate approach is to

[jira] [Updated] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-10-30 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-10885: Attachment: HDFS-10885-HDFS-10285.05.patch As disscussed above, SPS adds an xAttr {{user.creator}} to id

[jira] [Updated] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-10-30 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-10885: Status: Patch Available (was: Open) > [SPS]: Mover tool should not be allowed to run when Storage Policy

[jira] [Updated] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-10-13 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-10885: Attachment: HDFS-10800-HDFS-10885-00.patch Add a configure option "dfs.namenode.sps.enabled" to control

[jira] [Updated] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-10-16 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-10885: Status: Patch Available (was: Open) > [SPS]: Mover tool should not be allowed to run when Storage Policy

[jira] [Updated] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-10-17 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-10885: Status: Open (was: Patch Available) > [SPS]: Mover tool should not be allowed to run when Storage Policy

[jira] [Updated] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-10-17 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-10885: Fix Version/s: HDFS-10285 Affects Version/s: HDFS-10285 Status: Patch Available

[jira] [Updated] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-10-17 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-10885: Attachment: HDFS-10800-HDFS-10885-01.patch Thanks [~rakeshr] for reviewing and the comments. For comment

[jira] [Updated] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-10-17 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-10885: Status: Open (was: Patch Available) > [SPS]: Mover tool should not be allowed to run when Storage Policy

[jira] [Updated] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-10-17 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-10885: Target Version/s: HDFS-10285 Status: Patch Available (was: Open) > [SPS]: Mover tool should

[jira] [Updated] (HDFS-7343) HDFS smart storage management

2016-10-14 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-7343: --- Attachment: HDFS-Smart-Storage-Management.pdf The initial design document attached. Please help in reviewing

[jira] [Commented] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-11-29 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704841#comment-15704841 ] Wei Zhou commented on HDFS-10885: - Thanks [~rakeshr] and [~umamaheswararao] for reviewing the path!

[jira] [Created] (HDFS-11239) [SPS]: Check Mover file ID lease also to determine whether Mover is running

2016-12-12 Thread Wei Zhou (JIRA)
Wei Zhou created HDFS-11239: --- Summary: [SPS]: Check Mover file ID lease also to determine whether Mover is running Key: HDFS-11239 URL: https://issues.apache.org/jira/browse/HDFS-11239 Project: Hadoop HDFS

[jira] [Commented] (HDFS-11123) [SPS] Make storage policy satisfier daemon work on/off dynamically

2016-12-12 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15742018#comment-15742018 ] Wei Zhou commented on HDFS-11123: - Hi [~umamaheswararao], thanks for the contribution! After looking at

[jira] [Commented] (HDFS-11186) [SPS]: Daemon thread of SPS should start only in Active NN

2017-01-11 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15819914#comment-15819914 ] Wei Zhou commented on HDFS-11186: - The unit test failures are not related to this patch. > [SPS]: Daemon

[jira] [Created] (HDFS-11338) [SPS]: Fix timeout issue in unit tests caused by longger NN down time

2017-01-11 Thread Wei Zhou (JIRA)
Wei Zhou created HDFS-11338: --- Summary: [SPS]: Fix timeout issue in unit tests caused by longger NN down time Key: HDFS-11338 URL: https://issues.apache.org/jira/browse/HDFS-11338 Project: Hadoop HDFS

[jira] [Commented] (HDFS-7343) HDFS smart storage management

2017-01-09 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15813506#comment-15813506 ] Wei Zhou commented on HDFS-7343: {quote} though I'd encourage you to think about how far we can get with a

[jira] [Updated] (HDFS-11186) [SPS]: Daemon thread of SPS should start only in Active NN

2017-01-10 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11186: Attachment: HDFS-11186-HDFS-10285.03.patch Thanks [~rakeshr] for reviewing the patch! {quote} Probably, we

[jira] [Updated] (HDFS-11170) Add create API in filesystem public class to support assign parameter through builder

2017-01-10 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11170: Attachment: HDFS-11170-01.patch Hi [~andrew.wang], thanks for reviewing the patch and the great

[jira] [Updated] (HDFS-11186) [SPS]: Daemon thread of SPS should start only in Active NN

2017-01-11 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11186: Attachment: HDFS-11186-HDFS-10285.04.patch Thanks [~rakeshr] for the suggestions. Patch updated. > [SPS]:

[jira] [Updated] (HDFS-11172) Support an erasure coding policy RS-DEFAULT-10-4-64k in HDFS

2016-12-05 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11172: Attachment: HDFS-11172-v2.patch Thanks [~andrew.wang] for reviewing! The patch is updated and other

[jira] [Commented] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-12-05 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15724086#comment-15724086 ] Wei Zhou commented on HDFS-10885: - The test case failures are unrelated to this patch. Thanks! > [SPS]:

[jira] [Updated] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-12-04 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-10885: Attachment: HDFS-10885-HDFS-10285-10.patch Thanks [~rakeshr] and [~umamaheswararao] for the suggestions!

[jira] [Updated] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-12-05 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-10885: Attachment: HDFS-10885-HDFS-10285-11.patch Solve check style issue reported. The test case failures are

[jira] [Updated] (HDFS-11172) Support an erasure coding policy RS-DEFAULT-10-4-64k in HDFS

2016-12-05 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11172: Status: Patch Available (was: Open) > Support an erasure coding policy RS-DEFAULT-10-4-64k in HDFS >

[jira] [Updated] (HDFS-11172) Support an erasure coding policy RS-DEFAULT-10-4-64k in HDFS

2016-12-05 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11172: Attachment: HDFS-11172-v1.patch Initial patch to add this erasure coding policy. Thanks! > Support an

[jira] [Updated] (HDFS-11170) Add create API in filesystem public class to support assign parameter through builder

2017-01-03 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11170: Status: Patch Available (was: Open) > Add create API in filesystem public class to support assign

[jira] [Updated] (HDFS-11170) Add create API in filesystem public class to support assign parameter through builder

2017-01-03 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11170: Attachment: HDFS-11170-00.patch This initial patch. Thanks [~Sammi] for the great help! > Add create API

[jira] [Updated] (HDFS-7343) HDFS smart storage management

2017-01-05 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-7343: --- Attachment: HDFS-Smart-Storage-Management-update.pdf Based on the discussion and feedbacks collected, we

[jira] [Updated] (HDFS-7343) HDFS smart storage management

2017-01-09 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-7343: --- Attachment: move.jpg > HDFS smart storage management > - > > Key:

[jira] [Commented] (HDFS-7343) HDFS smart storage management

2017-01-09 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15811374#comment-15811374 ] Wei Zhou commented on HDFS-7343: Thanks [~anu] for reviewing the design document and the great comments.

[jira] [Commented] (HDFS-7343) HDFS smart storage management

2017-01-10 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15814478#comment-15814478 ] Wei Zhou commented on HDFS-7343: Thanks [~anu] for these insightful questions! {quote} Would you be able to

[jira] [Updated] (HDFS-11239) [SPS]: Check Mover file ID lease also to determine whether Mover is running

2016-12-19 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11239: Status: Patch Available (was: Open) > [SPS]: Check Mover file ID lease also to determine whether Mover is

[jira] [Updated] (HDFS-11239) [SPS]: Check Mover file ID lease also to determine whether Mover is running

2016-12-19 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11239: Attachment: HDFS-11239-HDFS-10285.00.patch The initial patch for this JIRA. > [SPS]: Check Mover file ID

[jira] [Created] (HDFS-11264) [SPS]: Double checks to ensure that SPS/Mover are not running together

2016-12-19 Thread Wei Zhou (JIRA)
Wei Zhou created HDFS-11264: --- Summary: [SPS]: Double checks to ensure that SPS/Mover are not running together Key: HDFS-11264 URL: https://issues.apache.org/jira/browse/HDFS-11264 Project: Hadoop HDFS

[jira] [Updated] (HDFS-11239) [SPS]: Check Mover file ID lease also to determine whether Mover is running

2016-12-21 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11239: Attachment: HDFS-11239-HDFS-10285.01.patch Thanks [~umamaheswararao] for reviewing the patch! INode related

[jira] [Commented] (HDFS-11239) [SPS]: Check Mover file ID lease also to determine whether Mover is running

2016-12-20 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15763748#comment-15763748 ] Wei Zhou commented on HDFS-11239: - The failures are not related and can not be reproduced in local tests.

[jira] [Updated] (HDFS-11186) [SPS]: Daemon thread of SPS should start only in Active NN

2016-12-20 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11186: Attachment: HDFS-11186-HDFS-10285.01.patch Thanks [~rakeshr] for reviewing the patch! Patch updated

[jira] [Updated] (HDFS-11186) [SPS]: Daemon thread of SPS should start only in Active NN

2016-12-19 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11186: Status: Patch Available (was: Open) > [SPS]: Daemon thread of SPS should start only in Active NN >

[jira] [Updated] (HDFS-11186) [SPS]: Daemon thread of SPS should start only in Active NN

2016-12-19 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11186: Attachment: HDFS-11186-HDFS-10285.00.patch The initial patch to this JIRA. > [SPS]: Daemon thread of SPS

[jira] [Updated] (HDFS-11186) [SPS]: Daemon thread of SPS should start only in Active NN

2016-12-27 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11186: Status: Open (was: Patch Available) In order to trigger Jenkins to work. > [SPS]: Daemon thread of SPS

[jira] [Updated] (HDFS-11186) [SPS]: Daemon thread of SPS should start only in Active NN

2016-12-27 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11186: Status: Patch Available (was: Open) > [SPS]: Daemon thread of SPS should start only in Active NN >

[jira] [Commented] (HDFS-11186) [SPS]: Daemon thread of SPS should start only in Active NN

2016-12-27 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15781900#comment-15781900 ] Wei Zhou commented on HDFS-11186: - This is a good way to bypass the issue, but I had add codes to other

[jira] [Commented] (HDFS-11186) [SPS]: Daemon thread of SPS should start only in Active NN

2016-12-27 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15781840#comment-15781840 ] Wei Zhou commented on HDFS-11186: - Hi [~rakeshr], thanks for reviewing the patch! The reason is that it

[jira] [Commented] (HDFS-11186) [SPS]: Daemon thread of SPS should start only in Active NN

2016-12-27 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15781841#comment-15781841 ] Wei Zhou commented on HDFS-11186: - Hi [~rakeshr], thanks for reviewing the patch! The reason is that it

[jira] [Commented] (HDFS-11186) [SPS]: Daemon thread of SPS should start only in Active NN

2016-12-27 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15781867#comment-15781867 ] Wei Zhou commented on HDFS-11186: - Sorry, {{In order to trigger Jenkins to work}} this comment is not

[jira] [Issue Comment Deleted] (HDFS-11186) [SPS]: Daemon thread of SPS should start only in Active NN

2016-12-27 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11186: Comment: was deleted (was: Hi [~rakeshr], thanks for reviewing the patch! The reason is that it takes

[jira] [Commented] (HDFS-11186) [SPS]: Daemon thread of SPS should start only in Active NN

2016-12-26 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15779444#comment-15779444 ] Wei Zhou commented on HDFS-11186: - The tests failures are caused by out of memory and not relate to this

[jira] [Updated] (HDFS-11239) [SPS]: Check Mover file ID lease also to determine whether Mover is running

2016-12-26 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11239: Attachment: HDFS-11239-HDFS-10285.02.patch Thanks [~umamaheswararao] for reviewing the patch! New API added

  1   2   >