[jira] [Commented] (HDFS-17316) Compatibility Benchmark over HCFS Implementations

[ https://issues.apache.org/jira/browse/HDFS-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17819450#comment-17819450 ] Kai Zheng commented on HDFS-17316: -- >>I think it should be a hadoop one for more than hdfs OK, let me

[jira] (HDFS-17316) Compatibility Benchmark over HCFS Implementations

[ https://issues.apache.org/jira/browse/HDFS-17316 ] Kai Zheng deleted comment on HDFS-17316: -- was (Author: drankye): >>I think it should be a hadoop one for more than hdfs OK, let me try and do it. >>one thing with the contract tests is we need

[jira] [Commented] (HDFS-17316) Compatibility Benchmark over HCFS Implementations

[ https://issues.apache.org/jira/browse/HDFS-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17819447#comment-17819447 ] Kai Zheng commented on HDFS-17316: -- >>I think it should be a hadoop one for more than hdfs OK, let me

[jira] [Commented] (HDFS-17316) Compatibility Benchmark over HCFS Implementations

[ https://issues.apache.org/jira/browse/HDFS-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17819444#comment-17819444 ] Kai Zheng commented on HDFS-17316: -- >>I think it should be a hadoop one for more than hdfs OK, let me

[jira] (HDFS-17316) Compatibility Benchmark over HCFS Implementations

[ https://issues.apache.org/jira/browse/HDFS-17316 ] Kai Zheng deleted comment on HDFS-17316: -- was (Author: drankye): >>I think it should be a hadoop one for more than hdfs OK, let me try and do it. >>one thing with the contract tests is we need

[jira] [Commented] (HDFS-17316) Compatibility Benchmark over HCFS Implementations

[ https://issues.apache.org/jira/browse/HDFS-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17819149#comment-17819149 ] Kai Zheng commented on HDFS-17316: -- This work looks great, thanks [~han.liu]! Maybe this jira would

[jira] [Updated] (HDFS-13668) FSPermissionChecker may throws AIOOE when check inode permission

[ https://issues.apache.org/jira/browse/HDFS-13668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-13668: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.2.0 Status: Resolved

[jira] [Updated] (HDFS-13668) FSPermissionChecker may throws AIOOE when check inode permission

[ https://issues.apache.org/jira/browse/HDFS-13668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-13668: - Summary: FSPermissionChecker may throws AIOOE when check inode permission (was: FSPermissionChecker may

[jira] [Commented] (HDFS-13668) FSPermissionChecker may throws AIOOE when check if inode has permission

[ https://issues.apache.org/jira/browse/HDFS-13668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16574231#comment-16574231 ] Kai Zheng commented on HDFS-13668: -- This NPE looks like a good catch and fix. +1 from me. Thanks

[jira] [Commented] (HDFS-11467) Support ErasureCoding section in OIV XML/ReverseXML

[ https://issues.apache.org/jira/browse/HDFS-11467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16237451#comment-16237451 ] Kai Zheng commented on HDFS-11467: -- Thanks [~HuafengWang]! The latest patch LGTM and +1. > Support

[jira] [Commented] (HDFS-12682) ECAdmin -listPolicies will always show SystemErasureCodingPolicies state as DISABLED

[ https://issues.apache.org/jira/browse/HDFS-12682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16236898#comment-16236898 ] Kai Zheng commented on HDFS-12682: -- The new approach using {{ErasureCodingPolicyInfo}} looks great!! I

[jira] [Commented] (HDFS-12682) ECAdmin -listPolicies will always show SystemErasureCodingPolicies state as DISABLED

[ https://issues.apache.org/jira/browse/HDFS-12682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16224526#comment-16224526 ] Kai Zheng commented on HDFS-12682: -- Thanks for the work and discussions here. I thought the smell of this

[jira] [Updated] (HDFS-12448) Make sure user defined erasure coding policy ID will not overflow

[ https://issues.apache.org/jira/browse/HDFS-12448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-12448: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0 Status: Resolved

[jira] [Commented] (HDFS-12448) Make sure user defined erasure coding policy ID will not overflow

[ https://issues.apache.org/jira/browse/HDFS-12448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16212021#comment-16212021 ] Kai Zheng commented on HDFS-12448: -- +1 on the latest patch. > Make sure user defined erasure coding

[jira] [Commented] (HDFS-12613) Native EC coder should implement release() as idempotent function.

[ https://issues.apache.org/jira/browse/HDFS-12613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16206900#comment-16206900 ] Kai Zheng commented on HDFS-12613: -- I double checked the C codes and it looks unlikely to break on

[jira] [Commented] (HDFS-12036) Add audit log for some erasure coding operations

[ https://issues.apache.org/jira/browse/HDFS-12036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16206882#comment-16206882 ] Kai Zheng commented on HDFS-12036: -- Thanks [~kshukla] for the good catch!! bq. The commit has the wrong

[jira] [Commented] (HDFS-12613) Native EC coder should implement release() as idempotent function.

[ https://issues.apache.org/jira/browse/HDFS-12613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205430#comment-16205430 ] Kai Zheng commented on HDFS-12613: -- Thanks Eddy. The patch has the +1 to be in. But I just thought of

[jira] [Commented] (HDFS-12613) Native EC coder should implement release() as idempotent function.

[ https://issues.apache.org/jira/browse/HDFS-12613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16203099#comment-16203099 ] Kai Zheng commented on HDFS-12613: -- Thanks [~eddyxu] for the update! +1 pending on the Jenkins and the

[jira] [Updated] (HDFS-12635) Unnecessary exception declaration of the CellBuffers constructor

[ https://issues.apache.org/jira/browse/HDFS-12635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-12635: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0 Status: Resolved

[jira] [Commented] (HDFS-12635) Unnecessary exception declaration of the CellBuffers constructor

[ https://issues.apache.org/jira/browse/HDFS-12635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16199918#comment-16199918 ] Kai Zheng commented on HDFS-12635: -- Nice catch, Huafeng. +1. > Unnecessary exception declaration of the

[jira] [Commented] (HDFS-12613) Native EC coder should implement release() as idempotent function.

[ https://issues.apache.org/jira/browse/HDFS-12613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16199912#comment-16199912 ] Kai Zheng commented on HDFS-12613: -- Thanks for the excellent work, [~eddyxu]! A minor: note

[jira] [Commented] (HDFS-12606) JVM crashes when running NNBench on EC enabled.

[ https://issues.apache.org/jira/browse/HDFS-12606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16193998#comment-16193998 ] Kai Zheng commented on HDFS-12606: -- Thanks for the ping Eddy. By design we can have multiple coder

[jira] [Commented] (HDFS-12448) Make sure user defined erasure coding policy ID will not overflow

[ https://issues.apache.org/jira/browse/HDFS-12448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16174396#comment-16174396 ] Kai Zheng commented on HDFS-12448: -- The patch looks good. Could you also update the doc, to say that the

[jira] [Updated] (HDFS-12448) Make sure user defined erasure coding policy ID will not overflow

[ https://issues.apache.org/jira/browse/HDFS-12448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-12448: - Description: Current policy ID is of type "byte". 1~63 is reserved for built-in erasure coding policy.

[jira] [Updated] (HDFS-12448) Make sure user defined erasure coding policy ID will not overflow

[ https://issues.apache.org/jira/browse/HDFS-12448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-12448: - Environment: (was: Current policy ID is of type "byte". 1~63 is reserved for built-in erasure coding

[jira] [Commented] (HDFS-7337) Configurable and pluggable erasure codec and policy

[ https://issues.apache.org/jira/browse/HDFS-7337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16174354#comment-16174354 ] Kai Zheng commented on HDFS-7337: - Resolved this issue with RN added. The remain issues left here were

[jira] [Resolved] (HDFS-7337) Configurable and pluggable erasure codec and policy

[ https://issues.apache.org/jira/browse/HDFS-7337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng resolved HDFS-7337. - Resolution: Fixed Fix Version/s: 3.0.0-beta1 Release Note: This allows users to: * develop and

[jira] [Updated] (HDFS-7337) Configurable and pluggable erasure codec and policy

[ https://issues.apache.org/jira/browse/HDFS-7337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-7337: Summary: Configurable and pluggable erasure codec and policy (was: Configurable and pluggable Erasure Codec

[jira] [Updated] (HDFS-8020) Erasure Coding: constructing and mapping BlockGroup info to utilize and call ErasureCoder API

[ https://issues.apache.org/jira/browse/HDFS-8020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-8020: Issue Type: Task (was: Sub-task) Parent: (was: HDFS-7337) > Erasure Coding: constructing and

[jira] [Commented] (HDFS-7346) Erasure Coding: perform stripping erasure encoding work given block reader and writer

[ https://issues.apache.org/jira/browse/HDFS-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16174345#comment-16174345 ] Kai Zheng commented on HDFS-7346: - I closed this as it's kinds of obsolete now. > Erasure Coding: perform

[jira] [Resolved] (HDFS-7346) Erasure Coding: perform stripping erasure encoding work given block reader and writer

[ https://issues.apache.org/jira/browse/HDFS-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng resolved HDFS-7346. - Resolution: Later > Erasure Coding: perform stripping erasure encoding work given block reader > and

[jira] [Updated] (HDFS-7363) Pluggable algorithms to form block groups in erasure coding

[ https://issues.apache.org/jira/browse/HDFS-7363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-7363: Issue Type: Task (was: Sub-task) Parent: (was: HDFS-7337) > Pluggable algorithms to form block

[jira] [Commented] (HDFS-7337) Configurable and pluggable Erasure Codec and schema

[ https://issues.apache.org/jira/browse/HDFS-7337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16174342#comment-16174342 ] Kai Zheng commented on HDFS-7337: - Good idea, Andrew. Let's do that. > Configurable and pluggable Erasure

[jira] [Updated] (HDFS-12479) Some misuses of lock in DFSStripedOutputStream

[ https://issues.apache.org/jira/browse/HDFS-12479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-12479: - Component/s: erasure-coding > Some misuses of lock in DFSStripedOutputStream >

[jira] [Updated] (HDFS-12479) Some misuses of lock in DFSStripedOutputStream

[ https://issues.apache.org/jira/browse/HDFS-12479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-12479: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-beta1 Status:

[jira] [Commented] (HDFS-12479) Some misuses of lock in DFSStripedOutputStream

[ https://issues.apache.org/jira/browse/HDFS-12479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16171415#comment-16171415 ] Kai Zheng commented on HDFS-12479: -- Good catches, Huafeng! The patch LGTM and +1. > Some misuses of lock

[jira] [Commented] (HDFS-12395) Support erasure coding policy operations in namenode edit log

[ https://issues.apache.org/jira/browse/HDFS-12395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16169832#comment-16169832 ] Kai Zheng commented on HDFS-12395: -- Thanks Brahma for raising this. I just got HDFS-12460 in. [~Sammi]

[jira] [Updated] (HDFS-12460) Make addErasureCodingPolicy an idempotent operation

[ https://issues.apache.org/jira/browse/HDFS-12460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-12460: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-beta1 Status:

[jira] [Updated] (HDFS-12460) Make addErasureCodingPolicy an idempotent operation

[ https://issues.apache.org/jira/browse/HDFS-12460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-12460: - Summary: Make addErasureCodingPolicy an idempotent operation (was: make addErasureCodingPolicy an

[jira] [Updated] (HDFS-12447) Refactor addErasureCodingPolicy

[ https://issues.apache.org/jira/browse/HDFS-12447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-12447: - Labels: hdfs-ec-3.0-must-do (was: ) > Refactor addErasureCodingPolicy > ---

[jira] [Commented] (HDFS-12444) Reduce runtime of TestWriteReadStripedFile

[ https://issues.apache.org/jira/browse/HDFS-12444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16169657#comment-16169657 ] Kai Zheng commented on HDFS-12444: -- Ops, yes it was. Sorry my mistake. LGTM and +1. > Reduce runtime of

[jira] [Commented] (HDFS-12447) Refactor addErasureCodingPolicy

[ https://issues.apache.org/jira/browse/HDFS-12447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167786#comment-16167786 ] Kai Zheng commented on HDFS-12447: -- [~andrew.wang] could you give this a review? Hope to make this in as

[jira] [Updated] (HDFS-12447) Refactor addErasureCodingPolicy

[ https://issues.apache.org/jira/browse/HDFS-12447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-12447: - Attachment: HDFS-12447.003.patch Looks like the patch missed the new file

[jira] [Commented] (HDFS-12460) make addErasureCodingPolicy an idempotent operation

[ https://issues.apache.org/jira/browse/HDFS-12460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167729#comment-16167729 ] Kai Zheng commented on HDFS-12460: -- Thanks Sammi for this. The patch LGTM. +1 pending on Jenkins. > make

[jira] [Commented] (HDFS-12444) Reduce runtime of TestWriteReadStripedFile

[ https://issues.apache.org/jira/browse/HDFS-12444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167726#comment-16167726 ] Kai Zheng commented on HDFS-12444: -- Hi [~HuafengWang], I saw {{TODO: HDFS-8797}} was still there.

[jira] [Commented] (HDFS-12444) Reduce runtime of TestWriteReadStripedFile

[ https://issues.apache.org/jira/browse/HDFS-12444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167333#comment-16167333 ] Kai Zheng commented on HDFS-12444: -- Thanks [~HuafengWang] for the update. Could you help check the

[jira] [Commented] (HDFS-12444) Reduce runtime of TestWriteReadStripedFile

[ https://issues.apache.org/jira/browse/HDFS-12444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167238#comment-16167238 ] Kai Zheng commented on HDFS-12444: -- Triggered the build manually. > Reduce runtime of

[jira] [Updated] (HDFS-12395) Support erasure coding policy operations in namenode edit log

[ https://issues.apache.org/jira/browse/HDFS-12395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-12395: - Resolution: Fixed Hadoop Flags: Incompatible change,Reviewed Fix Version/s: 3.0.0-beta1

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist erasure coding policies in NameNode

[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167195#comment-16167195 ] Kai Zheng commented on HDFS-7859: - [~Sammi], would you help with the release notes? Thanks again! >

[jira] [Updated] (HDFS-7859) Erasure Coding: Persist erasure coding policies in NameNode

[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-7859: Resolution: Fixed Hadoop Flags: Incompatible change,Reviewed Fix Version/s: 3.0.0-beta1

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist erasure coding policies in NameNode

[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167182#comment-16167182 ] Kai Zheng commented on HDFS-7859: - The building looks good. Will commit it shortly. > Erasure Coding:

[jira] [Commented] (HDFS-12395) Support erasure coding policy operations in namenode edit log

[ https://issues.apache.org/jira/browse/HDFS-12395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167178#comment-16167178 ] Kai Zheng commented on HDFS-12395: -- Checked the building results and it looks good. The relevant failure

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist erasure coding policies in NameNode

[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167073#comment-16167073 ] Kai Zheng commented on HDFS-7859: - Manually triggered, see if it works. > Erasure Coding: Persist erasure

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist erasure coding policies in NameNode

[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167014#comment-16167014 ] Kai Zheng commented on HDFS-7859: - Some issues like this and HDFS-12395 don't build, not sure what's wrong.

[jira] [Updated] (HDFS-12447) Refactor addErasureCodingPolicy

[ https://issues.apache.org/jira/browse/HDFS-12447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-12447: - Description: As a follow on to handle some issues discussed in HDFS-12395, this is to majorly refactor

[jira] [Updated] (HDFS-12447) Refactor addErasureCodingPolicy

[ https://issues.apache.org/jira/browse/HDFS-12447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-12447: - Description: Refactor addErasureCodingPoliy API, change AddECPolicyResponse =>

[jira] [Updated] (HDFS-12447) Refactor addErasureCodingPolicy

[ https://issues.apache.org/jira/browse/HDFS-12447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-12447: - Environment: (was: Refactor addErasureCodingPoliy API, change AddECPolicyResponse =>

[jira] [Commented] (HDFS-12444) Reduce runtime of TestWriteReadStripedFile

[ https://issues.apache.org/jira/browse/HDFS-12444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165937#comment-16165937 ] Kai Zheng commented on HDFS-12444: -- It's a good idea. I guess we can do both: (6 + 3) -> (3 + 2); 4

[jira] [Commented] (HDFS-12395) Support erasure coding policy operations in namenode edit log

[ https://issues.apache.org/jira/browse/HDFS-12395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165931#comment-16165931 ] Kai Zheng commented on HDFS-12395: -- Thanks Sammi for the update. The latest patch LGTM. +1 pending on the

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist erasure coding policies in NameNode

[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165930#comment-16165930 ] Kai Zheng commented on HDFS-7859: - Thanks Sammi for the update. The latest patch LGTM. +1 pending on the

[jira] [Updated] (HDFS-12438) Rename dfs.datanode.ec.reconstruction.stripedblock.threads.size to dfs.datanode.ec.reconstruction.threads

[ https://issues.apache.org/jira/browse/HDFS-12438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-12438: - Resolution: Fixed Hadoop Flags: Incompatible change,Reviewed (was: Incompatible change) Fix

[jira] [Commented] (HDFS-12438) Rename dfs.datanode.ec.reconstruction.stripedblock.threads.size to dfs.datanode.ec.reconstruction.threads

[ https://issues.apache.org/jira/browse/HDFS-12438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165884#comment-16165884 ] Kai Zheng commented on HDFS-12438: -- Thanks Andrew for the update. +1 on LGTM. Will commit it shortly. >

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist erasure coding policies in NameNode

[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165602#comment-16165602 ] Kai Zheng commented on HDFS-7859: - Sammi, please rebase this. Applying to the latest trunk failed. >

[jira] [Updated] (HDFS-12414) Ensure to use CLI command to enable/disable erasure coding policy

[ https://issues.apache.org/jira/browse/HDFS-12414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-12414: - Release Note: dfs.namenode.ec.policies.enabled was removed in order to ensure there is only one approach

[jira] [Commented] (HDFS-12414) Ensure to use CLI command to enable/disable erasure coding policy

[ https://issues.apache.org/jira/browse/HDFS-12414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165594#comment-16165594 ] Kai Zheng commented on HDFS-12414: -- TO CLARIFY: the commits here appear to be done by Sammi Chen, but

[jira] [Updated] (HDFS-12414) Ensure to use CLI command to enable/disable erasure coding policy

[ https://issues.apache.org/jira/browse/HDFS-12414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-12414: - Resolution: Fixed Hadoop Flags: Incompatible change,Reviewed Fix Version/s: 3.0.0-beta1

[jira] [Updated] (HDFS-12414) Ensure to use CLI command to enable/disable erasure coding policy

[ https://issues.apache.org/jira/browse/HDFS-12414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-12414: - Attachment: HDFS-12414.006.patch Uploaded the revision checked in, with the minor unimport issue fixed.

[jira] [Commented] (HDFS-12414) Ensure to use CLI command to enable/disable erasure coding policy

[ https://issues.apache.org/jira/browse/HDFS-12414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165566#comment-16165566 ] Kai Zheng commented on HDFS-12414: -- The latest patch LGTM and +1. Will commit it shortly with the minor

[jira] [Commented] (HDFS-12395) Support erasure coding policy operations in namenode edit log

[ https://issues.apache.org/jira/browse/HDFS-12395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164545#comment-16164545 ] Kai Zheng commented on HDFS-12395: -- [~eddyxu], would you also take a look at this? Thanks. If no few more

[jira] [Commented] (HDFS-12395) Support erasure coding policy operations in namenode edit log

[ https://issues.apache.org/jira/browse/HDFS-12395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164541#comment-16164541 ] Kai Zheng commented on HDFS-12395: -- The latest patch looks pretty good and close. Only a minor: Is this

[jira] [Commented] (HDFS-12405) Clean up removed erasure coding policies from namenode

[ https://issues.apache.org/jira/browse/HDFS-12405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164362#comment-16164362 ] Kai Zheng commented on HDFS-12405: -- Hi Huafeng, My thought is, the removed policies don't have to be

[jira] [Commented] (HDFS-12414) Ensure to use CLI command to enable/disable erasure coding policy

[ https://issues.apache.org/jira/browse/HDFS-12414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164204#comment-16164204 ] Kai Zheng commented on HDFS-12414: -- Thanks Sammi for the work. It LGTM. Please rebase since HDFS-1

[jira] [Commented] (HDFS-12438) Rename dfs.datanode.ec.reconstruction.stripedblock.threads.size to dfs.datanode.ec.reconstruction.stripedblock.threads

[ https://issues.apache.org/jira/browse/HDFS-12438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164080#comment-16164080 ] Kai Zheng commented on HDFS-12438: -- Sorry for the late. Maybe we could change it further, ending with

[jira] [Assigned] (HDFS-12414) Ensure to use CLI command to enable/disable erasure coding policy

[ https://issues.apache.org/jira/browse/HDFS-12414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng reassigned HDFS-12414: Assignee: SammiChen > Ensure to use CLI command to enable/disable erasure coding policy >

[jira] [Assigned] (HDFS-12395) Support erasure coding policy operations in namenode edit log

[ https://issues.apache.org/jira/browse/HDFS-12395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng reassigned HDFS-12395: Assignee: SammiChen (was: Kai Zheng) > Support erasure coding policy operations in namenode edit

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist erasure coding policies in NameNode

[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16160631#comment-16160631 ] Kai Zheng commented on HDFS-7859: - Thanks Sammi for the update and great work. In addition to the off-line

[jira] [Assigned] (HDFS-12395) Support erasure coding policy operations in namenode edit log

[ https://issues.apache.org/jira/browse/HDFS-12395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng reassigned HDFS-12395: Assignee: Kai Zheng (was: SammiChen) > Support erasure coding policy operations in namenode edit

[jira] [Commented] (HDFS-12395) Support erasure coding policy operations in namenode edit log

[ https://issues.apache.org/jira/browse/HDFS-12395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16160615#comment-16160615 ] Kai Zheng commented on HDFS-12395: -- Thanks [~Sammi] for working on this. Having looked into the work and

[jira] [Commented] (HDFS-12413) Inotify should support erasure coding policy op as replica meta change

[ https://issues.apache.org/jira/browse/HDFS-12413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16160586#comment-16160586 ] Kai Zheng commented on HDFS-12413: -- Would be great if we could get this done before 3.0 GA. > Inotify

[jira] [Commented] (HDFS-12413) Inotify should support erasure coding policy op as replica meta change

[ https://issues.apache.org/jira/browse/HDFS-12413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16160583#comment-16160583 ] Kai Zheng commented on HDFS-12413: -- Ping [~HuafengWang], I know you're familiar with HDFS inotify

[jira] [Created] (HDFS-12413) Inotify should support erasure coding policy op as replica meta change

Kai Zheng created HDFS-12413: Summary: Inotify should support erasure coding policy op as replica meta change Key: HDFS-12413 URL: https://issues.apache.org/jira/browse/HDFS-12413 Project: Hadoop HDFS

[jira] [Commented] (HDFS-10701) TestDFSStripedOutputStreamWithFailure#testBlockTokenExpired occasionally fails

[ https://issues.apache.org/jira/browse/HDFS-10701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159815#comment-16159815 ] Kai Zheng commented on HDFS-10701: -- Thanks Eddy. Yes we will look after it and other failed tests as well

[jira] [Commented] (HDFS-12398) Use JUnit Paramaterized test suite in TestWriteReadStripedFile

[ https://issues.apache.org/jira/browse/HDFS-12398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159810#comment-16159810 ] Kai Zheng commented on HDFS-12398: -- Hi [~HuafengWang], Thanks for working on the tests. The idea seems

[jira] [Commented] (HDFS-12412) Remove ErasureCodingWorker.stripedReadPool

[ https://issues.apache.org/jira/browse/HDFS-12412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159682#comment-16159682 ] Kai Zheng commented on HDFS-12412: -- Thanks Eddy for the ping. The idea to remove the striped read pool

[jira] [Updated] (HDFS-12405) Clean up removed erasure coding policies from namenode

[ https://issues.apache.org/jira/browse/HDFS-12405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-12405: - Description: Currently, when an erasure coding policy is removed, it's been transited to "removed" state.

[jira] [Updated] (HDFS-12395) Support erasure coding policy operations in namenode edit log

[ https://issues.apache.org/jira/browse/HDFS-12395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-12395: - Summary: Support erasure coding policy operations in namenode edit log (was: Support erasure coding

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist erasure coding policies in NameNode

[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16157633#comment-16157633 ] Kai Zheng commented on HDFS-7859: - bq. Checking file / directory that is using this particular policy is a

[jira] [Updated] (HDFS-12399) Improve erasure coding codec framework adding more unit tests

[ https://issues.apache.org/jira/browse/HDFS-12399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-12399: - Summary: Improve erasure coding codec framework adding more unit tests (was: Improve erasure coding

[jira] [Updated] (HDFS-12402) Refactor ErasureCodingPolicyManager and related codes

[ https://issues.apache.org/jira/browse/HDFS-12402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-12402: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-beta1 Status:

[jira] [Updated] (HDFS-12402) Refactor ErasureCodingPolicyManager and related codes

[ https://issues.apache.org/jira/browse/HDFS-12402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-12402: - Summary: Refactor ErasureCodingPolicyManager and related codes (was: Refactor

[jira] [Commented] (HDFS-12402) Refactor ErasureCodingPolicyManager

[ https://issues.apache.org/jira/browse/HDFS-12402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16156883#comment-16156883 ] Kai Zheng commented on HDFS-12402: -- The patch LGTM and +1. Will commit it shortly. > Refactor

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist erasure coding policies in NameNode

[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16156877#comment-16156877 ] Kai Zheng commented on HDFS-7859: - Thanks Sammi for the update. The patch seems in much better form. 1. Do

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist erasure coding policies in NameNode

[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16156269#comment-16156269 ] Kai Zheng commented on HDFS-7859: - HDFS-12395 handles edit log related changes as part of this work. >

[jira] [Updated] (HDFS-12395) Support erasure coding policy operation changes in namenode edit log

[ https://issues.apache.org/jira/browse/HDFS-12395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-12395: - Summary: Support erasure coding policy operation changes in namenode edit log (was: Support add, remove,

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist erasure coding policies in NameNode

[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16156260#comment-16156260 ] Kai Zheng commented on HDFS-7859: - Unfortunately I still found lots of not-so-relevant changes here. The

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist erasure coding policies in NameNode

[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16155100#comment-16155100 ] Kai Zheng commented on HDFS-7859: - Thanks [~Sammi] for the update! I'm happy to see the patch is very

[jira] [Commented] (HDFS-12353) Modify Dfsuse percent of dfsadmin report inconsistent with Dfsuse percent of datanode reports.

[ https://issues.apache.org/jira/browse/HDFS-12353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154977#comment-16154977 ] Kai Zheng commented on HDFS-12353: -- Thanks Steven for the ping, but I guess [~eddyxu] may have some idea

[jira] [Commented] (HDFS-1068) Reduce NameNode GC by reusing HdfsFileStatus objects in RPC handlers

[ https://issues.apache.org/jira/browse/HDFS-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154887#comment-16154887 ] Kai Zheng commented on HDFS-1068: - IIRC, there're some optimization work already for some cloud connectors,

[jira] [Commented] (HDFS-1068) Reduce NameNode GC by reusing HdfsFileStatus objects in RPC handlers

[ https://issues.apache.org/jira/browse/HDFS-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154877#comment-16154877 ] Kai Zheng commented on HDFS-1068: - Hi [~zhz], The idea to optimize {{getFileInfo}} sounds great to me,

[jira] [Updated] (HDFS-12392) Writing striped file failed due to different cell size

[ https://issues.apache.org/jira/browse/HDFS-12392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-12392: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-beta1 Status:

  1   2   3   4   5   6   7   8   9   10   >