[jira] [Commented] (HDFS-16875) Erasure Coding: data access proxy to allow old clients to read EC data

[ https://issues.apache.org/jira/browse/HDFS-16875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17654209#comment-17654209 ] Jing Zhao commented on HDFS-16875: -- Posted the design doc for the EC access proxy. > Erasure Coding:

[jira] [Updated] (HDFS-16875) Erasure Coding: data access proxy to allow old clients to read EC data

[ https://issues.apache.org/jira/browse/HDFS-16875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-16875: - Attachment: Erasure Coding Access Proxy.pdf > Erasure Coding: data access proxy to allow old clients to

[jira] [Created] (HDFS-16875) Erasure Coding: data access proxy to allow old clients to read EC data

Jing Zhao created HDFS-16875: Summary: Erasure Coding: data access proxy to allow old clients to read EC data Key: HDFS-16875 URL: https://issues.apache.org/jira/browse/HDFS-16875 Project: Hadoop HDFS

[jira] [Updated] (HDFS-16874) Improve DataNode decommission for Erasure Coding

[ https://issues.apache.org/jira/browse/HDFS-16874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-16874: - Description: There are a couple of issues with the current DataNode decommission implementation when

[jira] [Updated] (HDFS-16874) Improve DataNode decommission for Erasure Coding

[ https://issues.apache.org/jira/browse/HDFS-16874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-16874: - Description: There are a couple of issues with the current DataNode decommission implementation when

[jira] [Created] (HDFS-16874) Improve DataNode decommission for Erasure Coding

Jing Zhao created HDFS-16874: Summary: Improve DataNode decommission for Erasure Coding Key: HDFS-16874 URL: https://issues.apache.org/jira/browse/HDFS-16874 Project: Hadoop HDFS Issue Type:

[jira] [Commented] (HDFS-16422) Fix thread safety of EC decoding during concurrent preads

[ https://issues.apache.org/jira/browse/HDFS-16422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17503858#comment-17503858 ] Jing Zhao commented on HDFS-16422: -- Looks like we already have r/w lock protection in

[jira] [Commented] (HDFS-16283) RBF: improve renewLease() to call only a specific NameNode rather than make fan-out calls

[ https://issues.apache.org/jira/browse/HDFS-16283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17436168#comment-17436168 ] Jing Zhao commented on HDFS-16283: -- Checking into the code, looks like besides the router side

[jira] [Commented] (HDFS-16268) Balancer stuck when moving striped blocks due to NPE

[ https://issues.apache.org/jira/browse/HDFS-16268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17428552#comment-17428552 ] Jing Zhao commented on HDFS-16268: -- I've committed the fix. Thanks for the contribution, [~LeonG]! >

[jira] [Resolved] (HDFS-16268) Balancer stuck when moving striped blocks due to NPE

[ https://issues.apache.org/jira/browse/HDFS-16268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-16268. -- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Balancer stuck when moving

[jira] [Resolved] (HDFS-10648) Expose Balancer metrics through Metrics2

[ https://issues.apache.org/jira/browse/HDFS-10648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-10648. -- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Resolved Thanks for the work,

[jira] [Resolved] (HDFS-16224) testBalancerWithObserverWithFailedNode times out

[ https://issues.apache.org/jira/browse/HDFS-16224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-16224. -- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed I've committed the fix.

[jira] [Commented] (HDFS-16133) Support refresh of IP addresses behind DNS for clients

[ https://issues.apache.org/jira/browse/HDFS-16133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384672#comment-17384672 ] Jing Zhao commented on HDFS-16133: -- Sure, done > Support refresh of IP addresses behind DNS for clients

[jira] [Resolved] (HDFS-15842) HDFS mover to emit metrics

[ https://issues.apache.org/jira/browse/HDFS-15842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-15842. -- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed +1. Thank you for the

[jira] [Resolved] (HDFS-15781) Add metrics for how blocks are moved in replaceBlock

[ https://issues.apache.org/jira/browse/HDFS-15781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-15781. -- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed I've committed the patch.

[jira] [Updated] (HDFS-15683) Allow configuring DISK/ARCHIVE capacity for individual volumes

[ https://issues.apache.org/jira/browse/HDFS-15683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-15683: - Release Note: Add a new configuration "dfs.datanode.same-disk-tiering.capacity-ratio.percentage" to

[jira] [Resolved] (HDFS-15683) Allow configuring DISK/ARCHIVE capacity for individual volumes

[ https://issues.apache.org/jira/browse/HDFS-15683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-15683. -- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Allow configuring

[jira] [Commented] (HDFS-15683) Allow configuring DISK/ARCHIVE capacity for individual volumes

[ https://issues.apache.org/jira/browse/HDFS-15683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17281326#comment-17281326 ] Jing Zhao commented on HDFS-15683: -- +1. I've committed the patch. Thank you for the contribution,

[jira] [Comment Edited] (HDFS-15683) Allow configuring DISK/ARCHIVE capacity for individual volumes

[ https://issues.apache.org/jira/browse/HDFS-15683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17281326#comment-17281326 ] Jing Zhao edited comment on HDFS-15683 at 2/8/21, 7:35 PM: --- +1. I've committed

[jira] [Resolved] (HDFS-15549) Use Hardlink to move replica between DISK and ARCHIVE storage if on same filesystem mount

[ https://issues.apache.org/jira/browse/HDFS-15549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-15549. -- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed +1. Thank you for the

[jira] [Updated] (HDFS-15549) Use Hardlink to move replica between DISK and ARCHIVE storage if on same filesystem mount

[ https://issues.apache.org/jira/browse/HDFS-15549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-15549: - Summary: Use Hardlink to move replica between DISK and ARCHIVE storage if on same filesystem mount

[jira] [Commented] (HDFS-15549) Improve DISK/ARCHIVE movement if they are on same filesystem

[ https://issues.apache.org/jira/browse/HDFS-15549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17261690#comment-17261690 ] Jing Zhao commented on HDFS-15549: -- Thank you for working on this, [~LeonG]! I went through the current

[jira] [Commented] (HDFS-14904) Add Option to let Balancer prefer highly utilized nodes in each iteration

[ https://issues.apache.org/jira/browse/HDFS-14904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242816#comment-17242816 ] Jing Zhao commented on HDFS-14904: -- +1. I've committed the change. Thank you for the contribution,

[jira] [Resolved] (HDFS-14904) Add Option to let Balancer prefer highly utilized nodes in each iteration

[ https://issues.apache.org/jira/browse/HDFS-14904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-14904. -- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Add Option to let Balancer

[jira] [Updated] (HDFS-14904) Add Option to let Balancer prefer highly utilized nodes in each iteration

[ https://issues.apache.org/jira/browse/HDFS-14904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-14904: - Summary: Add Option to let Balancer prefer highly utilized nodes in each iteration (was: Option to let

[jira] [Resolved] (HDFS-15548) Allow configuring DISK/ARCHIVE storage types on same device mount

[ https://issues.apache.org/jira/browse/HDFS-15548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-15548. -- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed I've committed the change.

[jira] [Commented] (HDFS-15548) Allow configuring DISK/ARCHIVE storage types on same device mount

[ https://issues.apache.org/jira/browse/HDFS-15548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17227065#comment-17227065 ] Jing Zhao commented on HDFS-15548: -- The current PR looks good to me. Do you also want to take another

[jira] [Comment Edited] (HDFS-11797) BlockManager#createLocatedBlocks() can throw ArrayIndexOutofBoundsException when corrupt replicas are inconsistent

[ https://issues.apache.org/jira/browse/HDFS-11797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16042052#comment-16042052 ] Jing Zhao edited comment on HDFS-11797 at 6/8/17 1:29 AM: -- I have not checked the

[jira] [Comment Edited] (HDFS-11797) BlockManager#createLocatedBlocks() can throw ArrayIndexOutofBoundsException when corrupt replicas are inconsistent

[ https://issues.apache.org/jira/browse/HDFS-11797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16042052#comment-16042052 ] Jing Zhao edited comment on HDFS-11797 at 6/8/17 1:29 AM: -- I have not checked the

[jira] [Commented] (HDFS-11797) BlockManager#createLocatedBlocks() can throw ArrayIndexOutofBoundsException when corrupt replicas are inconsistent

[ https://issues.apache.org/jira/browse/HDFS-11797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16042052#comment-16042052 ] Jing Zhao commented on HDFS-11797: -- I have not checked the details, but is it related to HDFS-11445? >

[jira] [Updated] (HDFS-11823) Extend TestDFSStripedIutputStream/TestDFSStripedOutputStream with a random EC policy

[ https://issues.apache.org/jira/browse/HDFS-11823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-11823: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha3 Status:

[jira] [Commented] (HDFS-11823) Extend TestDFSStripedIutputStream/TestDFSStripedOutputStream with a random EC policy

[ https://issues.apache.org/jira/browse/HDFS-11823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16023376#comment-16023376 ] Jing Zhao commented on HDFS-11823: -- Thanks for working on this, [~tasanuma0829]! The patch looks good to

[jira] [Commented] (HDFS-11445) FSCK shows overall health stauts as corrupt even one replica is corrupt

[ https://issues.apache.org/jira/browse/HDFS-11445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16016362#comment-16016362 ] Jing Zhao commented on HDFS-11445: -- Thanks for updating the patch, [~brahmareddy]. The patch looks good

[jira] [Commented] (HDFS-11445) FSCK shows overall health stauts as corrupt even one replica is corrupt

[ https://issues.apache.org/jira/browse/HDFS-11445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16013006#comment-16013006 ] Jing Zhao commented on HDFS-11445: -- Thanks for the patch, [~brahmareddy]. For the current patch, instead

[jira] [Commented] (HDFS-11448) JN log segment syncing should support HA upgrade

[ https://issues.apache.org/jira/browse/HDFS-11448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15997365#comment-15997365 ] Jing Zhao commented on HDFS-11448: -- +1 for the 03 patch. Thank you for the work [~hanishakoneru]! Thank

[jira] [Commented] (HDFS-11448) JN log segment syncing should support HA upgrade

[ https://issues.apache.org/jira/browse/HDFS-11448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995387#comment-15995387 ] Jing Zhao commented on HDFS-11448: -- Thanks for continuing working on this, [~hanishakoneru]. And thanks

[jira] [Updated] (HDFS-11395) RequestHedgingProxyProvider#RequestHedgingInvocationHandler hides the Exception thrown from NameNode

[ https://issues.apache.org/jira/browse/HDFS-11395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-11395: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.9.0 Status: Resolved

[jira] [Commented] (HDFS-11395) RequestHedgingProxyProvider#RequestHedgingInvocationHandler hides the Exception thrown from NameNode

[ https://issues.apache.org/jira/browse/HDFS-11395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15922957#comment-15922957 ] Jing Zhao commented on HDFS-11395: -- +1. I will commit the patch shortly. >

[jira] [Commented] (HDFS-11395) RequestHedgingProxyProvider#RequestHedgingInvocationHandler hides the Exception thrown from NameNode

[ https://issues.apache.org/jira/browse/HDFS-11395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15899906#comment-15899906 ] Jing Zhao commented on HDFS-11395: -- hmmm looks like the testing node has not got fixed yet... >

[jira] [Commented] (HDFS-11395) RequestHedgingProxyProvider#RequestHedgingInvocationHandler hides the Exception thrown from NameNode

[ https://issues.apache.org/jira/browse/HDFS-11395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15899900#comment-15899900 ] Jing Zhao commented on HDFS-11395: -- The 005 patch looks good to me. Let me trigger the Jenkins. >

[jira] [Commented] (HDFS-11445) FSCK shows overall health stauts as corrupt even one replica is corrupt

[ https://issues.apache.org/jira/browse/HDFS-11445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15898236#comment-15898236 ] Jing Zhao commented on HDFS-11445: -- Thanks for working on this, [~brahmareddy]! I think you just find a

[jira] [Commented] (HDFS-11395) RequestHedgingProxyProvider#RequestHedgingInvocationHandler hides the Exception thrown from NameNode

[ https://issues.apache.org/jira/browse/HDFS-11395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15897830#comment-15897830 ] Jing Zhao commented on HDFS-11395: -- The 004 patch looks good to me. Just some minors: # Here since we're

[jira] [Updated] (HDFS-11476) Fix NPE in FsDatasetImpl#checkAndUpdate

[ https://issues.apache.org/jira/browse/HDFS-11476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-11476: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.9.0 Target Version/s:

[jira] [Commented] (HDFS-11395) RequestHedgingProxyProvider#RequestHedgingInvocationHandler hides the Exception thrown from NameNode

[ https://issues.apache.org/jira/browse/HDFS-11395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894925#comment-15894925 ] Jing Zhao commented on HDFS-11395: -- Thanks for updating the patch, [~nandakumar131]. Some further

[jira] [Commented] (HDFS-11395) RequestHedgingProxyProvider#RequestHedgingInvocationHandler hides the Exception thrown from NameNode

[ https://issues.apache.org/jira/browse/HDFS-11395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893227#comment-15893227 ] Jing Zhao commented on HDFS-11395: -- bq. in case of non RemoteException from ExecutionException, what

[jira] [Comment Edited] (HDFS-11395) RequestHedgingProxyProvider#RequestHedgingInvocationHandler hides the Exception thrown from NameNode

[ https://issues.apache.org/jira/browse/HDFS-11395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890867#comment-15890867 ] Jing Zhao edited comment on HDFS-11395 at 3/1/17 7:27 PM: -- Thanks for working on

[jira] [Commented] (HDFS-11395) RequestHedgingProxyProvider#RequestHedgingInvocationHandler hides the Exception thrown from NameNode

[ https://issues.apache.org/jira/browse/HDFS-11395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890867#comment-15890867 ] Jing Zhao commented on HDFS-11395: -- Thanks for working on this, [~nandakumar131]. I agree we should not

[jira] [Updated] (HDFS-8498) Blocks can be committed with wrong size

[ https://issues.apache.org/jira/browse/HDFS-8498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8498: Attachment: HDFS-8498.branch-2.001.patch Thanks for the review, [~jojochuang]. Update the branch-2 patch to

[jira] [Updated] (HDFS-4025) QJM: Sychronize past log segments to JNs that missed them

[ https://issues.apache.org/jira/browse/HDFS-4025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-4025: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha3 Status: Resolved

[jira] [Commented] (HDFS-4025) QJM: Sychronize past log segments to JNs that missed them

[ https://issues.apache.org/jira/browse/HDFS-4025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879572#comment-15879572 ] Jing Zhao commented on HDFS-4025: - The latest patch looks good to me. +1. I will commit the patch shortly.

[jira] [Commented] (HDFS-11402) HDFS Snapshots should capture point-in-time copies of OPEN files

[ https://issues.apache.org/jira/browse/HDFS-11402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872794#comment-15872794 ] Jing Zhao commented on HDFS-11402: -- bq. We have the same problem with parallel readers when there is an

[jira] [Commented] (HDFS-11402) HDFS Snapshots should capture point-in-time copies of OPEN files

[ https://issues.apache.org/jira/browse/HDFS-11402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872739#comment-15872739 ] Jing Zhao commented on HDFS-11402: -- Thanks for working on this, [~manojg]. Yes the length of open files

[jira] [Reopened] (HDFS-8498) Blocks can be committed with wrong size

[ https://issues.apache.org/jira/browse/HDFS-8498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao reopened HDFS-8498: - Reopen for the branch-2 patch. > Blocks can be committed with wrong size >

[jira] [Updated] (HDFS-8498) Blocks can be committed with wrong size

[ https://issues.apache.org/jira/browse/HDFS-8498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8498: Attachment: HDFS-8498.branch-2.patch Upload the patch for branch-2 > Blocks can be committed with wrong

[jira] [Commented] (HDFS-8498) Blocks can be committed with wrong size

[ https://issues.apache.org/jira/browse/HDFS-8498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868412#comment-15868412 ] Jing Zhao commented on HDFS-8498: - [~jojochuang], currently I do not plan to backport this change to branch

[jira] [Commented] (HDFS-8498) Blocks can be committed with wrong size

[ https://issues.apache.org/jira/browse/HDFS-8498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868333#comment-15868333 ] Jing Zhao commented on HDFS-8498: - I've committed the patch into trunk. > Blocks can be committed with

[jira] [Updated] (HDFS-8498) Blocks can be committed with wrong size

[ https://issues.apache.org/jira/browse/HDFS-8498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8498: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha3 Status: Resolved

[jira] [Commented] (HDFS-8498) Blocks can be committed with wrong size

[ https://issues.apache.org/jira/browse/HDFS-8498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858371#comment-15858371 ] Jing Zhao commented on HDFS-8498: - Do you also want to take a look at the patch, [~vinayrpet]? > Blocks

[jira] [Commented] (HDFS-4025) QJM: Sychronize past log segments to JNs that missed them

[ https://issues.apache.org/jira/browse/HDFS-4025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15850633#comment-15850633 ] Jing Zhao commented on HDFS-4025: - The failed unit test should be unrelated and has been reported in

[jira] [Commented] (HDFS-4025) QJM: Sychronize past log segments to JNs that missed them

[ https://issues.apache.org/jira/browse/HDFS-4025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15849052#comment-15849052 ] Jing Zhao commented on HDFS-4025: - Thanks for the updating the patch, [~hanishakoneru]. The latest patch

[jira] [Commented] (HDFS-11370) Optimize NamenodeFsck#getReplicaInfo

[ https://issues.apache.org/jira/browse/HDFS-11370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15848812#comment-15848812 ] Jing Zhao commented on HDFS-11370: -- The latest patch looks good to me. And the failed test should be

[jira] [Updated] (HDFS-11370) Optimize NamenodeFsck#getReplicaInfo

[ https://issues.apache.org/jira/browse/HDFS-11370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-11370: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha3 Status:

[jira] [Commented] (HDFS-11370) Optimize NamenodeFsck#getReplicaInfo

[ https://issues.apache.org/jira/browse/HDFS-11370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15847847#comment-15847847 ] Jing Zhao commented on HDFS-11370: -- Thanks for the discussion, [~manojg] and [~tasanuma0829]. I agree

[jira] [Commented] (HDFS-11370) Optimize NamenodeFsck#getReplicaInfo

[ https://issues.apache.org/jira/browse/HDFS-11370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15840282#comment-15840282 ] Jing Zhao commented on HDFS-11370: -- Thanks for working on this, [~tasanuma0829]. The current patch looks

[jira] [Updated] (HDFS-11124) Report blockIds of internal blocks for EC files in Fsck

[ https://issues.apache.org/jira/browse/HDFS-11124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-11124: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: (was: 3.0.0-alpha2)

[jira] [Commented] (HDFS-11124) Report blockIds of internal blocks for EC files in Fsck

[ https://issues.apache.org/jira/browse/HDFS-11124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836616#comment-15836616 ] Jing Zhao commented on HDFS-11124: -- Thanks for updating the patch, [~tasanuma0829]! The patch looks

[jira] [Updated] (HDFS-8498) Blocks can be committed with wrong size

[ https://issues.apache.org/jira/browse/HDFS-8498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8498: Attachment: HDFS-8498.001.patch Update the patch to fix bug when block is initialized as null. Also slightly

[jira] [Updated] (HDFS-8498) Blocks can be committed with wrong size

[ https://issues.apache.org/jira/browse/HDFS-8498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8498: Assignee: Jing Zhao (was: Daryn Sharp) Status: Patch Available (was: Reopened) > Blocks can be

[jira] [Commented] (HDFS-4025) QJM: Sychronize past log segments to JNs that missed them

[ https://issues.apache.org/jira/browse/HDFS-4025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828903#comment-15828903 ] Jing Zhao commented on HDFS-4025: - Thanks for the update, [~hkoneru]. The current patch looks better.

[jira] [Updated] (HDFS-8498) Blocks can be committed with wrong size

[ https://issues.apache.org/jira/browse/HDFS-8498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8498: Attachment: HDFS-8498.000.patch Vinay's proposed solution looks good to me. For the implementation, instead

[jira] [Commented] (HDFS-4025) QJM: Sychronize past log segments to JNs that missed them

[ https://issues.apache.org/jira/browse/HDFS-4025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15819779#comment-15819779 ] Jing Zhao commented on HDFS-4025: - Thanks for updating the patch, [~hkoneru]. Some further comments: # We

[jira] [Updated] (HDFS-8498) Blocks can be committed with wrong size

[ https://issues.apache.org/jira/browse/HDFS-8498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8498: Target Version/s: 2.9.0, 3.0.0-alpha2 (was: 2.7.3) > Blocks can be committed with wrong size >

[jira] [Reopened] (HDFS-8498) Blocks can be committed with wrong size

[ https://issues.apache.org/jira/browse/HDFS-8498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao reopened HDFS-8498: - > Blocks can be committed with wrong size > --- > > Key:

[jira] [Commented] (HDFS-8498) Blocks can be committed with wrong size

[ https://issues.apache.org/jira/browse/HDFS-8498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816262#comment-15816262 ] Jing Zhao commented on HDFS-8498: - We also saw the same scenario as described by [~vinayrpet]. Maybe we can

[jira] [Updated] (HDFS-11273) Move TransferFsImage#doGetUrl function to a Util class

[ https://issues.apache.org/jira/browse/HDFS-11273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-11273: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2 Status:

[jira] [Commented] (HDFS-11273) Move TransferFsImage#doGetUrl function to a Util class

[ https://issues.apache.org/jira/browse/HDFS-11273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15799513#comment-15799513 ] Jing Zhao commented on HDFS-11273: -- Thanks for updating the patch, [~hkoneru]! The updated patch is

[jira] [Commented] (HDFS-11273) Move TransferFsImage#doGetUrl function to a Util class

[ https://issues.apache.org/jira/browse/HDFS-11273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15796829#comment-15796829 ] Jing Zhao commented on HDFS-11273: -- Thanks for the patch, [~hkoneru]! The patch looks good to me. Just

[jira] [Updated] (HDFS-11273) Move TransferFsImage#doGetUrl function to a Util class

[ https://issues.apache.org/jira/browse/HDFS-11273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-11273: - Issue Type: Improvement (was: Bug) > Move TransferFsImage#doGetUrl function to a Util class >

[jira] [Updated] (HDFS-11273) Move TransferFsImage#doGetUrl function to a Util class

[ https://issues.apache.org/jira/browse/HDFS-11273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-11273: - Component/s: (was: hdfs) > Move TransferFsImage#doGetUrl function to a Util class >

[jira] [Commented] (HDFS-4025) QJM: Sychronize past log segments to JNs that missed them

[ https://issues.apache.org/jira/browse/HDFS-4025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15655302#comment-15655302 ] Jing Zhao commented on HDFS-4025: - Thanks for the patch, [~hanishakoneru]! The patch looks good to me in

[jira] [Commented] (HDFS-11095) BlockManagerSafeMode should respect extension period default config value (30s)

[ https://issues.apache.org/jira/browse/HDFS-11095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630482#comment-15630482 ] Jing Zhao commented on HDFS-11095: -- +1 pending Jenkins. > BlockManagerSafeMode should respect extension

[jira] [Commented] (HDFS-11090) Leave safemode immediately if all blocks have reported in

[ https://issues.apache.org/jira/browse/HDFS-11090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15629948#comment-15629948 ] Jing Zhao commented on HDFS-11090: -- If 100% block threshold has been met, this means all the blocks have

[jira] [Updated] (HDFS-10827) When there are unrecoverable ec block groups, Namenode Web UI shows "There are X missing blocks." but doesn't show the block names.

[ https://issues.apache.org/jira/browse/HDFS-10827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-10827: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2 Status:

[jira] [Commented] (HDFS-10827) When there are unrecoverable ec block groups, Namenode Web UI shows "There are X missing blocks." but doesn't show the block names.

[ https://issues.apache.org/jira/browse/HDFS-10827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15569512#comment-15569512 ] Jing Zhao commented on HDFS-10827: -- [~tasanuma0829], I think you're right. We actually do not need to add

[jira] [Updated] (HDFS-10968) BlockManager#isInNewRack should consider decommissioning nodes

[ https://issues.apache.org/jira/browse/HDFS-10968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-10968: - Resolution: Fixed Fix Version/s: 3.0.0-alpha2 Status: Resolved (was: Patch Available)

[jira] [Commented] (HDFS-10759) Change fsimage bool isStriped from boolean to an enum

[ https://issues.apache.org/jira/browse/HDFS-10759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556534#comment-15556534 ] Jing Zhao commented on HDFS-10759: -- Yeah I also think the idea is good. But we need to guarantee the

[jira] [Commented] (HDFS-10827) When there are unrecoverable ec block groups, Namenode Web UI shows "There are X missing blocks." but doesn't show the block names.

[ https://issues.apache.org/jira/browse/HDFS-10827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1949#comment-1949 ] Jing Zhao commented on HDFS-10827: -- Thanks for working on this, [~tasanuma0829]! The patch looks good to

[jira] [Updated] (HDFS-10968) BlockManager#isNewRack should consider decommissioning nodes

[ https://issues.apache.org/jira/browse/HDFS-10968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-10968: - Status: Patch Available (was: Open) > BlockManager#isNewRack should consider decommissioning nodes >

[jira] [Updated] (HDFS-10968) BlockManager#isNewRack should consider decommissioning nodes

[ https://issues.apache.org/jira/browse/HDFS-10968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-10968: - Attachment: HDFS-10968.000.patch Upload a patch to fix the issue. The patch also includes a unit test

[jira] [Created] (HDFS-10968) BlockManager#isNewRack should consider decommissioning nodes

Jing Zhao created HDFS-10968: Summary: BlockManager#isNewRack should consider decommissioning nodes Key: HDFS-10968 URL: https://issues.apache.org/jira/browse/HDFS-10968 Project: Hadoop HDFS

[jira] [Updated] (HDFS-10826) Correctly report missing EC blocks in FSCK

[ https://issues.apache.org/jira/browse/HDFS-10826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-10826: - Summary: Correctly report missing EC blocks in FSCK (was: The result of fsck should be CRITICAL when

[jira] [Commented] (HDFS-10826) The result of fsck should be CRITICAL when there are unrecoverable ec block groups.

[ https://issues.apache.org/jira/browse/HDFS-10826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549491#comment-15549491 ] Jing Zhao commented on HDFS-10826: -- I've committed the patch into trunk. Thanks for the contribution,

[jira] [Commented] (HDFS-10826) The result of fsck should be CRITICAL when there are unrecoverable ec block groups.

[ https://issues.apache.org/jira/browse/HDFS-10826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15546996#comment-15546996 ] Jing Zhao commented on HDFS-10826: -- The latest patch looks good to me. To do further code refactoring in

[jira] [Commented] (HDFS-10797) Disk usage summary of snapshots causes renamed blocks to get counted twice

[ https://issues.apache.org/jira/browse/HDFS-10797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15533611#comment-15533611 ] Jing Zhao commented on HDFS-10797: -- Actually my proposal is like your .005 patch. The current semantic

[jira] [Commented] (HDFS-10797) Disk usage summary of snapshots causes renamed blocks to get counted twice

[ https://issues.apache.org/jira/browse/HDFS-10797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15531432#comment-15531432 ] Jing Zhao commented on HDFS-10797: -- [~mackrorysd], I agree it will be great to have a consistent and

[jira] [Commented] (HDFS-10897) Ozone: SCM: Add NodeManager

[ https://issues.apache.org/jira/browse/HDFS-10897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15531259#comment-15531259 ] Jing Zhao commented on HDFS-10897: -- Thanks for the reply, Anu. bq. The reason of breaking up these data

[jira] [Commented] (HDFS-10912) Ozone:SCM: Add safe mode support to NodeManager.

[ https://issues.apache.org/jira/browse/HDFS-10912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15530843#comment-15530843 ] Jing Zhao commented on HDFS-10912: -- Thanks for the work, [~anu]. For SCM, I think we may have two

[jira] [Comment Edited] (HDFS-10897) Ozone: SCM: Add NodeManager

[ https://issues.apache.org/jira/browse/HDFS-10897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15530818#comment-15530818 ] Jing Zhao edited comment on HDFS-10897 at 9/28/16 8:43 PM: --- Thanks for working

[jira] [Commented] (HDFS-10897) Ozone: SCM: Add NodeManager

[ https://issues.apache.org/jira/browse/HDFS-10897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15530818#comment-15530818 ] Jing Zhao commented on HDFS-10897: -- Thanks for working on this, [~anu]. The patch looks good to me

[jira] [Commented] (HDFS-10826) The result of fsck should be CRITICAL when there are unrecoverable ec block groups.

[ https://issues.apache.org/jira/browse/HDFS-10826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15530710#comment-15530710 ] Jing Zhao commented on HDFS-10826: -- # The failure of TestLeaseRecoveryStriped seems related. Could you

  1   2   3   4   5   6   7   8   9   10   >