[jira] [Assigned] (HDFS-13762) Support non-volatile storage class memory(SCM) in HDFS cache directives

2019-01-07 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou reassigned HDFS-13762: --- Assignee: (was: Wei Zhou) > Support non-volatile storage class memory(SCM) in HDFS cache

[jira] [Commented] (HDFS-13762) Support non-volatile storage class memory(SCM) in HDFS cache directives

2019-01-07 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16736010#comment-16736010 ] Wei Zhou commented on HDFS-13762: - Hi [~rakeshr], thank you very much for reviewing the patch and the

[jira] [Assigned] (HDFS-13762) Support non-volatile storage class memory(SCM) in HDFS cache directives

2019-01-07 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou reassigned HDFS-13762: --- Assignee: Wei Zhou > Support non-volatile storage class memory(SCM) in HDFS cache directives >

[jira] [Commented] (HDFS-13762) Support non-volatile storage class memory(SCM) in HDFS cache directives

2018-12-17 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16723539#comment-16723539 ] Wei Zhou commented on HDFS-13762: - The test failures seems not related to this patch, thanks! > Support

[jira] [Updated] (HDFS-13762) Support non-volatile storage class memory(SCM) in HDFS cache directives

2018-12-16 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-13762: Attachment: HDFS-13762.007.patch > Support non-volatile storage class memory(SCM) in HDFS cache directives

[jira] [Updated] (HDFS-13762) Support non-volatile storage class memory(SCM) in HDFS cache directives

2018-12-13 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-13762: Attachment: HDFS-13762.006.patch > Support non-volatile storage class memory(SCM) in HDFS cache directives

[jira] [Updated] (HDFS-13762) Support non-volatile storage class memory(SCM) in HDFS cache directives

2018-12-13 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-13762: Attachment: HDFS-13762.006.patch > Support non-volatile storage class memory(SCM) in HDFS cache directives

[jira] [Updated] (HDFS-13762) Support non-volatile storage class memory(SCM) in HDFS cache directives

2018-12-13 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-13762: Attachment: (was: HDFS-13762.006.patch) > Support non-volatile storage class memory(SCM) in HDFS cache

[jira] [Updated] (HDFS-13762) Support non-volatile storage class memory(SCM) in HDFS cache directives

2018-11-29 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-13762: Attachment: HDFS-13762.005.patch > Support non-volatile storage class memory(SCM) in HDFS cache directives

[jira] [Updated] (HDFS-13762) Support non-volatile storage class memory(SCM) in HDFS cache directives

2018-11-28 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-13762: Status: Patch Available (was: Open) To trigger checks. > Support non-volatile storage class memory(SCM)

[jira] [Updated] (HDFS-13762) Support non-volatile storage class memory(SCM) in HDFS cache directives

2018-11-28 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-13762: Status: Open (was: Patch Available) > Support non-volatile storage class memory(SCM) in HDFS cache

[jira] [Updated] (HDFS-13762) Support non-volatile storage class memory(SCM) in HDFS cache directives

2018-11-28 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-13762: Attachment: HDFS-13762.004.patch > Support non-volatile storage class memory(SCM) in HDFS cache directives

[jira] [Updated] (HDFS-13762) Support non-volatile storage class memory(SCM) in HDFS cache directives

2018-11-25 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-13762: Attachment: HDFS-13762.003.patch > Support non-volatile storage class memory(SCM) in HDFS cache directives

[jira] [Updated] (HDFS-13762) Support non-volatile storage class memory(SCM) in HDFS cache directives

2018-11-19 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-13762: Attachment: HDFS-13762.002.patch > Support non-volatile storage class memory(SCM) in HDFS cache directives

[jira] [Updated] (HDFS-13762) Support non-volatile storage class memory(SCM) in HDFS cache directives

2018-11-14 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-13762: Attachment: (was: HDFS-13762.001.patch) > Support non-volatile storage class memory(SCM) in HDFS cache

[jira] [Updated] (HDFS-13762) Support non-volatile storage class memory(SCM) in HDFS cache directives

2018-11-14 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-13762: Attachment: HDFS-13762.001.patch Status: Patch Available (was: Open) > Support non-volatile

[jira] [Commented] (HDFS-13762) Support non-volatile storage class memory(SCM) in HDFS cache directives

2018-11-14 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687419#comment-16687419 ] Wei Zhou commented on HDFS-13762: - After discussed with [~Sammi] offline, I'll take over the

[jira] [Updated] (HDFS-13762) Support non-volatile storage class memory(SCM) in HDFS cache directives

2018-11-14 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-13762: Attachment: HDFS-13762.001.patch > Support non-volatile storage class memory(SCM) in HDFS cache directives

[jira] [Commented] (HDFS-9608) Disk IO imbalance in HDFS with heterogeneous storages

2018-02-12 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16360654#comment-16360654 ] Wei Zhou commented on HDFS-9608: Hi, I'm on vacation from 2/8/2018 and back on 2/22/2018. My email

[jira] [Updated] (HDFS-12258) ec -listPolicies should list all policies in system, no matter it's enabled or disabled

2017-08-29 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-12258: Attachment: HDFS-12258.07.patch Thanks [~rakeshr] for reviewing, doc updated based on the comments, thanks!

[jira] [Commented] (HDFS-12258) ec -listPolicies should list all policies in system, no matter it's enabled or disabled

2017-08-29 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16144892#comment-16144892 ] Wei Zhou commented on HDFS-12258: - Thanks [~rakeshr] for reviewing the patch! The test failures are not

[jira] [Updated] (HDFS-12258) ec -listPolicies should list all policies in system, no matter it's enabled or disabled

2017-08-28 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-12258: Attachment: HDFS-12258.06.patch Fix checkstyle issue, the test failures are not related to this patch.

[jira] [Updated] (HDFS-12258) ec -listPolicies should list all policies in system, no matter it's enabled or disabled

2017-08-27 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-12258: Attachment: HDFS-12258.05.patch Thanks [~rakeshr] for reviewing, patch updated according to comments,

[jira] [Updated] (HDFS-12303) Change default EC cell size to 1MB for better performance

2017-08-23 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-12303: Attachment: HDFS-12303.02.patch Thanks [~drankye] for review the patch. Those cases that has connection

[jira] [Updated] (HDFS-12303) Change default EC cell size to 1MB for better performance

2017-08-20 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-12303: Attachment: HDFS-12303.01.patch Thanks [~andrew.wang] for reviewing the patch and clarifying the JIRA

[jira] [Updated] (HDFS-12258) ec -listPolicies should list all policies in system, no matter it's enabled or disabled

2017-08-17 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-12258: Attachment: HDFS-12258.04.patch Thanks [~rakeshr] for review the patch and suggestion! Patch rebased with

[jira] [Commented] (HDFS-12303) Change default EC cell size to 1MB for better performance

2017-08-15 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128369#comment-16128369 ] Wei Zhou commented on HDFS-12303: - The junit test failures reported can not be reproduced in my local

[jira] [Updated] (HDFS-12303) Change default EC cell size to 1MB for better performance

2017-08-15 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-12303: Status: Open (was: Patch Available) > Change default EC cell size to 1MB for better performance >

[jira] [Updated] (HDFS-12303) Change default EC cell size to 1MB for better performance

2017-08-15 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-12303: Status: Patch Available (was: Open) > Change default EC cell size to 1MB for better performance >

[jira] [Updated] (HDFS-12258) ec -listPolicies should list all policies in system, no matter it's enabled or disabled

2017-08-15 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-12258: Attachment: HDFS-12258.03.patch Thanks [~rakeshr] for reviewing and the comments! For the 4th issue, my

[jira] [Updated] (HDFS-12303) Change default EC cell size to 1MB for better performance

2017-08-15 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-12303: Attachment: HDFS-12303.00.patch Initial patch. > Change default EC cell size to 1MB for better performance

[jira] [Updated] (HDFS-12303) Change default EC cell size to 1MB for better performance

2017-08-15 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-12303: Status: Patch Available (was: Open) > Change default EC cell size to 1MB for better performance >

[jira] [Commented] (HDFS-11814) Benchmark and tune for prefered default cell size

2017-08-14 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126831#comment-16126831 ] Wei Zhou commented on HDFS-11814: - Thanks [~andrew.wang] for reviewing and suggestions! HDFS-12303 is

[jira] [Created] (HDFS-12303) Change default EC cell size to 1MB for better performance

2017-08-14 Thread Wei Zhou (JIRA)
Wei Zhou created HDFS-12303: --- Summary: Change default EC cell size to 1MB for better performance Key: HDFS-12303 URL: https://issues.apache.org/jira/browse/HDFS-12303 Project: Hadoop HDFS Issue

[jira] [Commented] (HDFS-12258) ec -listPolicies should list all policies in system, no matter it's enabled or disabled

2017-08-14 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16125319#comment-16125319 ] Wei Zhou commented on HDFS-12258: - The findbugs warnings and test failure are not related to this patch,

[jira] [Commented] (HDFS-11814) Benchmark and tune for prefered default cell size

2017-08-14 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16125290#comment-16125290 ] Wei Zhou commented on HDFS-11814: - Hi [~andrew.wang], I set the {{nrFiles}} to 89 to simulate high

[jira] [Updated] (HDFS-11814) Benchmark and tune for prefered default cell size

2017-08-14 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11814: Attachment: RS-6-3-Concurrent.png > Benchmark and tune for prefered default cell size >

[jira] [Updated] (HDFS-12258) ec -listPolicies should list all policies in system, no matter it's enabled or disabled

2017-08-13 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-12258: Attachment: HDFS-12258.02.patch Update patch, fix code style and related tests issue reported. Thanks! >

[jira] [Updated] (HDFS-12258) ec -listPolicies should list all policies in system, no matter it's enabled or disabled

2017-08-11 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-12258: Attachment: HDFS-12258.01.patch Initial patch for this issue. An {{ErasureCodingPolicyState}} added into

[jira] [Updated] (HDFS-12258) ec -listPolicies should list all policies in system, no matter it's enabled or disabled

2017-08-11 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-12258: Status: Patch Available (was: Open) > ec -listPolicies should list all policies in system, no matter it's

[jira] [Commented] (HDFS-11814) Benchmark and tune for prefered default cell size

2017-08-08 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118175#comment-16118175 ] Wei Zhou commented on HDFS-11814: - Hi Andrew, thanks for your suggestion, I'll do such tests. Thanks! >

[jira] [Commented] (HDFS-11814) Benchmark and tune for prefered default cell size

2017-08-07 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16116318#comment-16116318 ] Wei Zhou commented on HDFS-11814: - >From the test cases above, there is no need to change the default cell

[jira] [Commented] (HDFS-11814) Benchmark and tune for prefered default cell size

2017-08-07 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16116214#comment-16116214 ] Wei Zhou commented on HDFS-11814: - I did some performance testing on EC IO performance with different EC

[jira] [Updated] (HDFS-11814) Benchmark and tune for prefered default cell size

2017-08-07 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11814: Attachment: RS-Read.png RS-Write.png > Benchmark and tune for prefered default cell size >

[jira] [Commented] (HDFS-7343) HDFS smart storage management

2017-04-07 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15960830#comment-15960830 ] Wei Zhou commented on HDFS-7343: Thanks [~rakeshr] for reviewing the design and thoughtful comments.

[jira] [Commented] (HDFS-7343) HDFS smart storage management

2017-04-01 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15952175#comment-15952175 ] Wei Zhou commented on HDFS-7343: Thanks for [~eddyxu] and [~drankye] for the discussion on SSM design. On

[jira] [Updated] (HDFS-7343) HDFS smart storage management

2017-04-01 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-7343: --- Attachment: access_count_tables.jpg tables_in_ssm.xlsx > HDFS smart storage management >

[jira] [Commented] (HDFS-7343) HDFS smart storage management

2017-03-15 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15925928#comment-15925928 ] Wei Zhou commented on HDFS-7343: Thanks [~andrew.wang] for these great questions. These questions are

[jira] [Updated] (HDFS-7343) HDFS smart storage management

2017-03-15 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-7343: --- Attachment: HDFSSmartStorageManagement-Phase1-20170315.pdf

[jira] [Commented] (HDFS-11170) Add create API in filesystem public class to support assign parameter through builder

2017-03-14 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923824#comment-15923824 ] Wei Zhou commented on HDFS-11170: - Thanks [~andrew.wang] for the reminder and sorry for the delay due to

[jira] [Updated] (HDFS-11338) [SPS]: Fix timeout issue in unit tests caused by longger NN down time

2017-02-27 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11338: Attachment: HDFS-11338-HDFS-10285.01.patch Fix TestPersistentStoragePolicySatisfier timeout issue. >

[jira] [Updated] (HDFS-11338) [SPS]: Fix timeout issue in unit tests caused by longger NN down time

2017-02-27 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11338: Status: Patch Available (was: Open) > [SPS]: Fix timeout issue in unit tests caused by longger NN down

[jira] [Updated] (HDFS-11338) [SPS]: Fix timeout issue in unit tests caused by longger NN down time

2017-02-27 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11338: Attachment: HDFS-11338-HDFS-10285.00.patch The initial patch. > [SPS]: Fix timeout issue in unit tests

[jira] [Comment Edited] (HDFS-7343) HDFS smart storage management

2017-02-14 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867046#comment-15867046 ] Wei Zhou edited comment on HDFS-7343 at 2/15/17 1:59 AM: - After discussed with

[jira] [Commented] (HDFS-7343) HDFS smart storage management

2017-02-14 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867046#comment-15867046 ] Wei Zhou commented on HDFS-7343: After discussed with [~andrew.wang] and [~drankye], in order to make the

[jira] [Commented] (HDFS-11239) [SPS]: Check Mover file ID lease also to determine whether Mover is running

2017-02-02 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851091#comment-15851091 ] Wei Zhou commented on HDFS-11239: - Some of test failures are caused by timeout mentioned above and the

[jira] [Updated] (HDFS-11239) [SPS]: Check Mover file ID lease also to determine whether Mover is running

2017-02-02 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11239: Attachment: HDFS-11239-HDFS-10285.04.patch Rebase the patch. > [SPS]: Check Mover file ID lease also to

[jira] [Commented] (HDFS-11239) [SPS]: Check Mover file ID lease also to determine whether Mover is running

2017-01-20 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832834#comment-15832834 ] Wei Zhou commented on HDFS-11239: - Hi, I'm on vacation from 1/21/2017 to 2/2/2017 and back in 2/3/2017.

[jira] [Updated] (HDFS-11239) [SPS]: Check Mover file ID lease also to determine whether Mover is running

2017-01-19 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11239: Attachment: HDFS-11239-HDFS-10285.03.patch Thanks [~umamaheswararao] for reviewing the patch and the

[jira] [Created] (HDFS-11338) [SPS]: Fix timeout issue in unit tests caused by longger NN down time

2017-01-11 Thread Wei Zhou (JIRA)
Wei Zhou created HDFS-11338: --- Summary: [SPS]: Fix timeout issue in unit tests caused by longger NN down time Key: HDFS-11338 URL: https://issues.apache.org/jira/browse/HDFS-11338 Project: Hadoop HDFS

[jira] [Commented] (HDFS-11186) [SPS]: Daemon thread of SPS should start only in Active NN

2017-01-11 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15819914#comment-15819914 ] Wei Zhou commented on HDFS-11186: - The unit test failures are not related to this patch. > [SPS]: Daemon

[jira] [Updated] (HDFS-11186) [SPS]: Daemon thread of SPS should start only in Active NN

2017-01-11 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11186: Attachment: HDFS-11186-HDFS-10285.04.patch Thanks [~rakeshr] for the suggestions. Patch updated. > [SPS]:

[jira] [Updated] (HDFS-11186) [SPS]: Daemon thread of SPS should start only in Active NN

2017-01-10 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11186: Attachment: HDFS-11186-HDFS-10285.03.patch Thanks [~rakeshr] for reviewing the patch! {quote} Probably, we

[jira] [Updated] (HDFS-11170) Add create API in filesystem public class to support assign parameter through builder

2017-01-10 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11170: Attachment: HDFS-11170-01.patch Hi [~andrew.wang], thanks for reviewing the patch and the great

[jira] [Commented] (HDFS-7343) HDFS smart storage management

2017-01-10 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15814478#comment-15814478 ] Wei Zhou commented on HDFS-7343: Thanks [~anu] for these insightful questions! {quote} Would you be able to

[jira] [Commented] (HDFS-7343) HDFS smart storage management

2017-01-09 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15813506#comment-15813506 ] Wei Zhou commented on HDFS-7343: {quote} though I'd encourage you to think about how far we can get with a

[jira] [Commented] (HDFS-7343) HDFS smart storage management

2017-01-09 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15811374#comment-15811374 ] Wei Zhou commented on HDFS-7343: Thanks [~anu] for reviewing the design document and the great comments.

[jira] [Updated] (HDFS-7343) HDFS smart storage management

2017-01-09 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-7343: --- Attachment: move.jpg > HDFS smart storage management > - > > Key:

[jira] [Updated] (HDFS-7343) HDFS smart storage management

2017-01-05 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-7343: --- Attachment: HDFS-Smart-Storage-Management-update.pdf Based on the discussion and feedbacks collected, we

[jira] [Updated] (HDFS-11170) Add create API in filesystem public class to support assign parameter through builder

2017-01-03 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11170: Status: Patch Available (was: Open) > Add create API in filesystem public class to support assign

[jira] [Updated] (HDFS-11170) Add create API in filesystem public class to support assign parameter through builder

2017-01-03 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11170: Attachment: HDFS-11170-00.patch This initial patch. Thanks [~Sammi] for the great help! > Add create API

[jira] [Commented] (HDFS-11186) [SPS]: Daemon thread of SPS should start only in Active NN

2016-12-27 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15781900#comment-15781900 ] Wei Zhou commented on HDFS-11186: - This is a good way to bypass the issue, but I had add codes to other

[jira] [Commented] (HDFS-11186) [SPS]: Daemon thread of SPS should start only in Active NN

2016-12-27 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15781867#comment-15781867 ] Wei Zhou commented on HDFS-11186: - Sorry, {{In order to trigger Jenkins to work}} this comment is not

[jira] [Issue Comment Deleted] (HDFS-11186) [SPS]: Daemon thread of SPS should start only in Active NN

2016-12-27 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11186: Comment: was deleted (was: Hi [~rakeshr], thanks for reviewing the patch! The reason is that it takes

[jira] [Commented] (HDFS-11186) [SPS]: Daemon thread of SPS should start only in Active NN

2016-12-27 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15781841#comment-15781841 ] Wei Zhou commented on HDFS-11186: - Hi [~rakeshr], thanks for reviewing the patch! The reason is that it

[jira] [Commented] (HDFS-11186) [SPS]: Daemon thread of SPS should start only in Active NN

2016-12-27 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15781840#comment-15781840 ] Wei Zhou commented on HDFS-11186: - Hi [~rakeshr], thanks for reviewing the patch! The reason is that it

[jira] [Updated] (HDFS-11186) [SPS]: Daemon thread of SPS should start only in Active NN

2016-12-27 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11186: Status: Open (was: Patch Available) In order to trigger Jenkins to work. > [SPS]: Daemon thread of SPS

[jira] [Updated] (HDFS-11186) [SPS]: Daemon thread of SPS should start only in Active NN

2016-12-27 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11186: Status: Patch Available (was: Open) > [SPS]: Daemon thread of SPS should start only in Active NN >

[jira] [Updated] (HDFS-11239) [SPS]: Check Mover file ID lease also to determine whether Mover is running

2016-12-26 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11239: Attachment: HDFS-11239-HDFS-10285.02.patch Thanks [~umamaheswararao] for reviewing the patch! New API added

[jira] [Commented] (HDFS-11186) [SPS]: Daemon thread of SPS should start only in Active NN

2016-12-26 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15779444#comment-15779444 ] Wei Zhou commented on HDFS-11186: - The tests failures are caused by out of memory and not relate to this

[jira] [Updated] (HDFS-11186) [SPS]: Daemon thread of SPS should start only in Active NN

2016-12-26 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11186: Attachment: HDFS-11186-HDFS-10285.02.patch Fix issues reported by Jenkins. Some of the testing failures are

[jira] [Created] (HDFS-11269) Exclude the parity blocks when caching an erasure coding file

2016-12-21 Thread Wei Zhou (JIRA)
Wei Zhou created HDFS-11269: --- Summary: Exclude the parity blocks when caching an erasure coding file Key: HDFS-11269 URL: https://issues.apache.org/jira/browse/HDFS-11269 Project: Hadoop HDFS

[jira] [Updated] (HDFS-11239) [SPS]: Check Mover file ID lease also to determine whether Mover is running

2016-12-21 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11239: Attachment: HDFS-11239-HDFS-10285.01.patch Thanks [~umamaheswararao] for reviewing the patch! INode related

[jira] [Commented] (HDFS-11239) [SPS]: Check Mover file ID lease also to determine whether Mover is running

2016-12-20 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15763748#comment-15763748 ] Wei Zhou commented on HDFS-11239: - The failures are not related and can not be reproduced in local tests.

[jira] [Updated] (HDFS-11186) [SPS]: Daemon thread of SPS should start only in Active NN

2016-12-20 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11186: Attachment: HDFS-11186-HDFS-10285.01.patch Thanks [~rakeshr] for reviewing the patch! Patch updated

[jira] [Updated] (HDFS-11239) [SPS]: Check Mover file ID lease also to determine whether Mover is running

2016-12-19 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11239: Attachment: HDFS-11239-HDFS-10285.00.patch The initial patch for this JIRA. > [SPS]: Check Mover file ID

[jira] [Updated] (HDFS-11239) [SPS]: Check Mover file ID lease also to determine whether Mover is running

2016-12-19 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11239: Status: Patch Available (was: Open) > [SPS]: Check Mover file ID lease also to determine whether Mover is

[jira] [Created] (HDFS-11264) [SPS]: Double checks to ensure that SPS/Mover are not running together

2016-12-19 Thread Wei Zhou (JIRA)
Wei Zhou created HDFS-11264: --- Summary: [SPS]: Double checks to ensure that SPS/Mover are not running together Key: HDFS-11264 URL: https://issues.apache.org/jira/browse/HDFS-11264 Project: Hadoop HDFS

[jira] [Updated] (HDFS-11186) [SPS]: Daemon thread of SPS should start only in Active NN

2016-12-19 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11186: Status: Patch Available (was: Open) > [SPS]: Daemon thread of SPS should start only in Active NN >

[jira] [Updated] (HDFS-11186) [SPS]: Daemon thread of SPS should start only in Active NN

2016-12-19 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11186: Attachment: HDFS-11186-HDFS-10285.00.patch The initial patch to this JIRA. > [SPS]: Daemon thread of SPS

[jira] [Created] (HDFS-11239) [SPS]: Check Mover file ID lease also to determine whether Mover is running

2016-12-12 Thread Wei Zhou (JIRA)
Wei Zhou created HDFS-11239: --- Summary: [SPS]: Check Mover file ID lease also to determine whether Mover is running Key: HDFS-11239 URL: https://issues.apache.org/jira/browse/HDFS-11239 Project: Hadoop HDFS

[jira] [Commented] (HDFS-11123) [SPS] Make storage policy satisfier daemon work on/off dynamically

2016-12-12 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15742018#comment-15742018 ] Wei Zhou commented on HDFS-11123: - Hi [~umamaheswararao], thanks for the contribution! After looking at

[jira] [Commented] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-12-05 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15724086#comment-15724086 ] Wei Zhou commented on HDFS-10885: - The test case failures are unrelated to this patch. Thanks! > [SPS]:

[jira] [Updated] (HDFS-11172) Support an erasure coding policy RS-DEFAULT-10-4-64k in HDFS

2016-12-05 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11172: Attachment: HDFS-11172-v2.patch Thanks [~andrew.wang] for reviewing! The patch is updated and other

[jira] [Updated] (HDFS-11172) Support an erasure coding policy RS-DEFAULT-10-4-64k in HDFS

2016-12-05 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11172: Status: Patch Available (was: Open) > Support an erasure coding policy RS-DEFAULT-10-4-64k in HDFS >

[jira] [Updated] (HDFS-11172) Support an erasure coding policy RS-DEFAULT-10-4-64k in HDFS

2016-12-05 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11172: Attachment: HDFS-11172-v1.patch Initial patch to add this erasure coding policy. Thanks! > Support an

[jira] [Updated] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-12-05 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-10885: Attachment: HDFS-10885-HDFS-10285-11.patch Solve check style issue reported. The test case failures are

[jira] [Updated] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-12-04 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-10885: Attachment: HDFS-10885-HDFS-10285-10.patch Thanks [~rakeshr] and [~umamaheswararao] for the suggestions!

[jira] [Commented] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-11-29 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704841#comment-15704841 ] Wei Zhou commented on HDFS-10885: - Thanks [~rakeshr] and [~umamaheswararao] for reviewing the path!

[jira] [Created] (HDFS-11186) [SPS]: Daemon thread of SPS starts only in Active NN

2016-11-28 Thread Wei Zhou (JIRA)
Wei Zhou created HDFS-11186: --- Summary: [SPS]: Daemon thread of SPS starts only in Active NN Key: HDFS-11186 URL: https://issues.apache.org/jira/browse/HDFS-11186 Project: Hadoop HDFS Issue Type:

  1   2   >