[jira] [Comment Edited] (HDFS-15012) NN fails to parse Edit logs after applying HDFS-13101

2019-11-25 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982106#comment-16982106 ] Wei-Chiu Chuang edited comment on HDFS-15012 at 11/26/19 3:26 AM: --

[jira] [Commented] (HDFS-15012) NN fails to parse Edit logs after applying HDFS-13101

2019-11-25 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982106#comment-16982106 ] Wei-Chiu Chuang commented on HDFS-15012: We recently received several reports regarding corrupt

[jira] [Commented] (HDFS-14908) LeaseManager should check parent-child relationship when filter open files.

2019-11-25 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981911#comment-16981911 ] Wei-Chiu Chuang commented on HDFS-14908: Sounds good to me. Thanks doing the extra check >

[jira] [Commented] (HDFS-14519) NameQuota is not update after concat operation, so namequota is wrong

2019-11-22 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16980407#comment-16980407 ] Wei-Chiu Chuang commented on HDFS-14519: +1 > NameQuota is not update after concat operation, so

[jira] [Commented] (HDFS-14820) The default 8KB buffer of BlockReaderRemote#newBlockReader#BufferedOutputStream is too big

2019-11-22 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16980404#comment-16980404 ] Wei-Chiu Chuang commented on HDFS-14820: 003 patch looks good to me. I would suggest we add ad

[jira] [Commented] (HDFS-15005) Backport HDFS-12300 to branch-2

2019-11-22 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16980403#comment-16980403 ] Wei-Chiu Chuang commented on HDFS-15005: {code} private String

[jira] [Commented] (HDFS-13594) the lock of ShortCircuitCache is hold while close the ShortCircuitReplica

2019-11-21 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-13594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16979750#comment-16979750 ] Wei-Chiu Chuang commented on HDFS-13594: Is this still an issue after HDFS-14541 and HDFS-14535?

[jira] [Commented] (HDFS-14751) TestNameNodeMetadataConsistency#testGenerationStampInFuture fail in trunk

2019-11-21 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16979748#comment-16979748 ] Wei-Chiu Chuang commented on HDFS-14751: I would like to push forward HDFS-14476. [~leosun08]

[jira] [Comment Edited] (HDFS-14820) The default 8KB buffer of BlockReaderRemote#newBlockReader#BufferedOutputStream is too big

2019-11-21 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16979714#comment-16979714 ] Wei-Chiu Chuang edited comment on HDFS-14820 at 11/22/19 12:09 AM: --- In

[jira] [Commented] (HDFS-14820) The default 8KB buffer of BlockReaderRemote#newBlockReader#BufferedOutputStream is too big

2019-11-21 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16979714#comment-16979714 ] Wei-Chiu Chuang commented on HDFS-14820: In this case, the buffer size is used for

[jira] [Assigned] (HDFS-14978) In-place Erasure Coding Conversion

2019-11-21 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang reassigned HDFS-14978: -- Assignee: Aravindan Vijayan (was: Wei-Chiu Chuang) > In-place Erasure Coding

[jira] [Commented] (HDFS-14908) LeaseManager should check parent-child relationship when filter open files.

2019-11-21 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16979641#comment-16979641 ] Wei-Chiu Chuang commented on HDFS-14908: Sure. By no means do I intend to stall this patch from

[jira] [Commented] (HDFS-15000) Improve FsDatasetImpl to avoid IO operation in datasetLock

2019-11-21 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16979616#comment-16979616 ] Wei-Chiu Chuang commented on HDFS-15000: HDFS-9668 might also help. > Improve FsDatasetImpl to

[jira] [Commented] (HDFS-15000) Improve FsDatasetImpl to avoid IO operation in datasetLock

2019-11-21 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16979498#comment-16979498 ] Wei-Chiu Chuang commented on HDFS-15000: Great idea. It reminds me of HDFS-8496 where we

[jira] [Commented] (HDFS-14820) The default 8KB buffer of BlockReaderRemote#newBlockReader#BufferedOutputStream is too big

2019-11-20 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16978978#comment-16978978 ] Wei-Chiu Chuang commented on HDFS-14820: This one looks useful. [~leosun08] are you planning to

[jira] [Commented] (HDFS-14993) checkDiskError doesn't work during datanode startup

2019-11-20 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16978973#comment-16978973 ] Wei-Chiu Chuang commented on HDFS-14993: [~sodonnell] please take a look. Perhaps the logic was

[jira] [Updated] (HDFS-14563) Enhance interface about recommissioning/decommissioning

2019-11-20 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14563: --- Labels: decommission (was: ) > Enhance interface about recommissioning/decommissioning >

[jira] [Commented] (HDFS-14563) Enhance interface about recommissioning/decommissioning

2019-11-20 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16978781#comment-16978781 ] Wei-Chiu Chuang commented on HDFS-14563: I'm about to commit HDFS-14854 but I suspect that has

[jira] [Commented] (HDFS-14999) Avoid Potential Infinite Loop in DFSNetworkTopology

2019-11-20 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16978779#comment-16978779 ] Wei-Chiu Chuang commented on HDFS-14999: It might be related to HADOOP-15317. (not sure. the

[jira] [Commented] (HDFS-14908) LeaseManager should check parent-child relationship when filter open files.

2019-11-20 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16978649#comment-16978649 ] Wei-Chiu Chuang commented on HDFS-14908: {code} * Check if the given path is the child of parent

[jira] [Updated] (HDFS-14861) Reset LowRedundancyBlocks Iterator periodically

2019-11-19 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14861: --- Labels: decommission (was: ) > Reset LowRedundancyBlocks Iterator periodically >

[jira] [Commented] (HDFS-14861) Reset LowRedundancyBlocks Iterator periodically

2019-11-19 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16977899#comment-16977899 ] Wei-Chiu Chuang commented on HDFS-14861: Sorry for the long delay. Patch makes sense to me. I'm

[jira] [Updated] (HDFS-14996) RBF: GetFileStatus fails for directory with EC policy set in case of multiple destinations

2019-11-19 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14996: --- Component/s: rbf ec > RBF: GetFileStatus fails for directory with EC policy

[jira] [Updated] (HDFS-14952) Skip safemode if blockTotal is 0 in new NN

2019-11-19 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14952: --- Fix Version/s: 2.11.0 3.2.2 3.1.4 > Skip safemode if

[jira] [Updated] (HDFS-14952) Skip safemode if blockTotal is 0 in new NN

2019-11-19 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14952: --- Resolution: Fixed Status: Resolved (was: Patch Available) Thanks [~hexiaoqiao] for

[jira] [Updated] (HDFS-14952) Skip safemode if blockTotal is 0 in new NN

2019-11-19 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14952: --- Fix Version/s: 3.3.0 > Skip safemode if blockTotal is 0 in new NN >

[jira] [Commented] (HDFS-14894) Add balancer parameter to balance top used nodes

2019-11-19 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16977737#comment-16977737 ] Wei-Chiu Chuang commented on HDFS-14894: [~pifta] could you help review the PR? > Add balancer

[jira] [Resolved] (HDFS-12866) Recursive delete of a large directory or snapshot makes namenode unresponsive

2019-11-18 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-12866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-12866. Resolution: Duplicate Resolve as it duplicates HDFS-11225 > Recursive delete of a large

[jira] [Resolved] (HDFS-11225) NameNode crashed because deleteSnapshot held FSNamesystem lock too long

2019-11-18 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-11225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-11225. Fix Version/s: 3.1.0 Resolution: Fixed [~shashikant] completed the subtasks. I'm

[jira] [Updated] (HDFS-7784) load fsimage in parallel

2019-11-18 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-7784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-7784: -- Target Version/s: (was: ) Resolution: Duplicate Status: Resolved

[jira] [Commented] (HDFS-14952) Skip safemode if blockTotal is 0 in new NN

2019-11-18 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976911#comment-16976911 ] Wei-Chiu Chuang commented on HDFS-14952: +1 > Skip safemode if blockTotal is 0 in new NN >

[jira] [Commented] (HDFS-14969) Fix HDFS client unnecessary failover log printing

2019-11-18 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976810#comment-16976810 ] Wei-Chiu Chuang commented on HDFS-14969: i would love to see the client being more intelligent

[jira] [Updated] (HDFS-14969) Fix HDFS client unnecessary failover log printing

2019-11-18 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14969: --- Labels: multi-sbnn (was: ) > Fix HDFS client unnecessary failover log printing >

[jira] [Updated] (HDFS-14942) Change Log Level to debug in JournalNodeSyncer#syncWithJournalAtIndex

2019-11-18 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14942: --- Fix Version/s: 3.2.2 3.1.4 > Change Log Level to debug in

[jira] [Issue Comment Deleted] (HDFS-14942) Change Log Level to debug in JournalNodeSyncer#syncWithJournalAtIndex

2019-11-18 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14942: --- Comment: was deleted (was: I think we want this in 2.x not 3.x) > Change Log Level to debug

[jira] [Commented] (HDFS-14942) Change Log Level to debug in JournalNodeSyncer#syncWithJournalAtIndex

2019-11-18 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976766#comment-16976766 ] Wei-Chiu Chuang commented on HDFS-14942: I think we want this in 2.x not 3.x > Change Log Level

[jira] [Updated] (HDFS-14963) Add HDFS Client machine caching active namenode index mechanism.

2019-11-15 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14963: --- Labels: multi-sbnn (was: ) > Add HDFS Client machine caching active namenode index

[jira] [Commented] (HDFS-14802) The feature of protect directories should be used in RenameOp

2019-11-15 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16975390#comment-16975390 ] Wei-Chiu Chuang commented on HDFS-14802: Feel free to reopen and provide a patch for lower

[jira] [Updated] (HDFS-14802) The feature of protect directories should be used in RenameOp

2019-11-15 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14802: --- Fix Version/s: 3.3.0 > The feature of protect directories should be used in RenameOp >

[jira] [Updated] (HDFS-14802) The feature of protect directories should be used in RenameOp

2019-11-15 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14802: --- Resolution: Fixed Status: Resolved (was: Patch Available) Merged the PR in github.

[jira] [Updated] (HDFS-14882) Consider DataNode load when #getBlockLocation

2019-11-15 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14882: --- Fix Version/s: 3.3.0 Resolution: Fixed Status: Resolved (was: Patch

[jira] [Commented] (HDFS-14882) Consider DataNode load when #getBlockLocation

2019-11-15 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16975375#comment-16975375 ] Wei-Chiu Chuang commented on HDFS-14882: Failed tests doesn't appear related. Committing 010

[jira] [Updated] (HDFS-14884) Add sanity check that zone key equals feinfo key while setting Xattrs

2019-11-14 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14884: --- Fix Version/s: 2.10.1 Resolution: Fixed Status: Resolved (was: Patch

[jira] [Commented] (HDFS-14884) Add sanity check that zone key equals feinfo key while setting Xattrs

2019-11-14 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974787#comment-16974787 ] Wei-Chiu Chuang commented on HDFS-14884: +1 > Add sanity check that zone key equals feinfo key

[jira] [Commented] (HDFS-14884) Add sanity check that zone key equals feinfo key while setting Xattrs

2019-11-12 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972859#comment-16972859 ] Wei-Chiu Chuang commented on HDFS-14884: Sorry missed it. I'll review it for sure. > Add sanity

[jira] [Resolved] (HDFS-14959) [SBNN read] access time should be turned off

2019-11-12 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-14959. Resolution: Fixed Merged the PR to trunk and cherry pick the commit to branch-3.2 and

[jira] [Updated] (HDFS-14959) [SBNN read] access time should be turned off

2019-11-12 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14959: --- Fix Version/s: 3.2.2 3.1.4 3.3.0 > [SBNN read] access

[jira] [Commented] (HDFS-14981) BlockStateChange logging is exceedingly verbose

2019-11-12 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972640#comment-16972640 ] Wei-Chiu Chuang commented on HDFS-14981: I think this is done by HDFS-6860. > BlockStateChange

[jira] [Commented] (HDFS-14978) In-place Erasure Coding Conversion

2019-11-12 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972384#comment-16972384 ] Wei-Chiu Chuang commented on HDFS-14978: bq. What is the client behavior during the CAS operation

[jira] [Commented] (HDFS-14978) In-place Erasure Coding Conversion

2019-11-11 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16971812#comment-16971812 ] Wei-Chiu Chuang commented on HDFS-14978: Also, this is pretty much the same as HDFS-11347.

[jira] [Commented] (HDFS-14978) In-place Erasure Coding Conversion

2019-11-11 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16971811#comment-16971811 ] Wei-Chiu Chuang commented on HDFS-14978: Attach the design doc. To encourage participation and

[jira] [Updated] (HDFS-14978) In-place Erasure Coding Conversion

2019-11-11 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14978: --- Attachment: In-place Erasure Coding Conversion.pdf > In-place Erasure Coding Conversion >

[jira] [Created] (HDFS-14978) In-place Erasure Coding Conversion

2019-11-11 Thread Wei-Chiu Chuang (Jira)
Wei-Chiu Chuang created HDFS-14978: -- Summary: In-place Erasure Coding Conversion Key: HDFS-14978 URL: https://issues.apache.org/jira/browse/HDFS-14978 Project: Hadoop HDFS Issue Type: New

[jira] [Commented] (HDFS-14963) Add HDFS Client machine caching active namenode index mechanism.

2019-11-07 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16969469#comment-16969469 ] Wei-Chiu Chuang commented on HDFS-14963: Thanks [~xudongcao]. I do know that this has been a

[jira] [Commented] (HDFS-14963) Add DFS Client caching active namenode mechanism.

2019-11-06 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968884#comment-16968884 ] Wei-Chiu Chuang commented on HDFS-14963: Doesn't client already remember active NN? > Add DFS

[jira] [Comment Edited] (HDFS-14951) KMS Jetty server does not startup if trust store password is null

2019-11-06 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968723#comment-16968723 ] Wei-Chiu Chuang edited comment on HDFS-14951 at 11/6/19 9:25 PM: - Good

[jira] [Commented] (HDFS-14951) KMS Jetty server does not startup if trust store password is null

2019-11-06 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968723#comment-16968723 ] Wei-Chiu Chuang commented on HDFS-14951: Good catch [~smeng]. Can we also add that check for

[jira] [Commented] (HDFS-14870) [SBN read] Client Settings to ObserverReadProxyProvider read a file request, due to the file atime time needs to be modified, lead to request to be processed twice

2019-11-06 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968665#comment-16968665 ] Wei-Chiu Chuang commented on HDFS-14870: FWIW Ambari disables access time by default for HDP

[jira] [Commented] (HDFS-14959) [SBNN read] access time should be turned off

2019-11-06 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968599#comment-16968599 ] Wei-Chiu Chuang commented on HDFS-14959: [~csun] [~xkrogen] may i have your opinion? is it also

[jira] [Created] (HDFS-14959) [SBNN read] access time should be turned off

2019-11-06 Thread Wei-Chiu Chuang (Jira)
Wei-Chiu Chuang created HDFS-14959: -- Summary: [SBNN read] access time should be turned off Key: HDFS-14959 URL: https://issues.apache.org/jira/browse/HDFS-14959 Project: Hadoop HDFS Issue

[jira] [Commented] (HDFS-14953) [Dynamometer] Missing blocks gradually increase after NN starts

2019-11-06 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968497#comment-16968497 ] Wei-Chiu Chuang commented on HDFS-14953: Yep. How about porting my PR over here? > [Dynamometer]

[jira] [Commented] (HDFS-14929) Hadoop 2.9.1 rename functionality infrequent breaking

2019-11-05 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16967797#comment-16967797 ] Wei-Chiu Chuang commented on HDFS-14929: is this a duplicate of HDFS-14947? > Hadoop 2.9.1

[jira] [Commented] (HDFS-14947) infrequent data loss due to rename functionality breaking

2019-11-05 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16967794#comment-16967794 ] Wei-Chiu Chuang commented on HDFS-14947: Hey [~abhishek.sahani] thanks for the details. bq.

[jira] [Commented] (HDFS-14942) Change Log Level to debug in JournalNodeSyncer#syncWithJournalAtIndex

2019-11-04 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16966983#comment-16966983 ] Wei-Chiu Chuang commented on HDFS-14942: Ok from me. > Change Log Level to debug in

[jira] [Updated] (HDFS-14951) KMS Jetty server does not startup if trust store password is null

2019-11-04 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14951: --- Status: Patch Available (was: Open) > KMS Jetty server does not startup if trust store

[jira] [Updated] (HDFS-14945) Revise PacketResponder's log.

2019-11-04 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14945: --- Fix Version/s: 3.2.2 3.1.4 3.3.0 Resolution:

[jira] [Commented] (HDFS-14942) Change Log Level to warn in JournalNodeSyncer#syncWithJournalAtIndex

2019-11-03 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16966197#comment-16966197 ] Wei-Chiu Chuang commented on HDFS-14942: Thanks [~leosun08] bq. yes, this issue happens during

[jira] [Resolved] (HDFS-14925) rename operation should check nest snapshot

2019-11-01 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-14925. Fix Version/s: 3.2.2 3.1.4 Resolution: Fixed Thanks! merged the

[jira] [Updated] (HDFS-14944) ec admin such as : -enablePolicy should support multi federation namespace not only the default namespace in core-site.xml

2019-11-01 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14944: --- Component/s: federation > ec admin such as : -enablePolicy should support multi federation

[jira] [Updated] (HDFS-14944) ec admin such as : -enablePolicy should support multi federation namespace not only the default namespace in core-site.xml

2019-11-01 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14944: --- Component/s: ec > ec admin such as : -enablePolicy should support multi federation namespace

[jira] [Assigned] (HDFS-14925) rename operation should check nest snapshot

2019-11-01 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang reassigned HDFS-14925: -- Assignee: Junwang Zhao > rename operation should check nest snapshot >

[jira] [Updated] (HDFS-14925) rename operation should check nest snapshot

2019-11-01 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14925: --- Fix Version/s: 3.3.0 > rename operation should check nest snapshot >

[jira] [Comment Edited] (HDFS-14802) The feature of protect directories should be used in RenameOp

2019-11-01 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16965151#comment-16965151 ] Wei-Chiu Chuang edited comment on HDFS-14802 at 11/1/19 11:20 PM: --

[jira] [Commented] (HDFS-14802) The feature of protect directories should be used in RenameOp

2019-11-01 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16965151#comment-16965151 ] Wei-Chiu Chuang commented on HDFS-14802: Unrelated. It's unfortunate the configuration is listed

[jira] [Commented] (HDFS-14945) Revise PacketResponder's log.

2019-11-01 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16965146#comment-16965146 ] Wei-Chiu Chuang commented on HDFS-14945: +1 > Revise PacketResponder's log. >

[jira] [Commented] (HDFS-14947) infrequent data loss due to rename functionality breaking

2019-11-01 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16965143#comment-16965143 ] Wei-Chiu Chuang commented on HDFS-14947: I would be really really surprised if a file simply

[jira] [Commented] (HDFS-14942) Change Log Level to warn in JournalNodeSyncer#syncWithJournalAtIndex

2019-11-01 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16965069#comment-16965069 ] Wei-Chiu Chuang commented on HDFS-14942: I'm assuming this issue happens during rolling upgrade

[jira] [Assigned] (HDFS-14942) Change Log Level to warn in JournalNodeSyncer#syncWithJournalAtIndex

2019-11-01 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang reassigned HDFS-14942: -- Assignee: Lisheng Sun > Change Log Level to warn in

[jira] [Commented] (HDFS-14720) DataNode shouldn't report block as bad block if the block length is Long.MAX_VALUE.

2019-11-01 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16965002#comment-16965002 ] Wei-Chiu Chuang commented on HDFS-14720: i'm okay without a test since it's mostly a cosmetic

[jira] [Commented] (HDFS-14794) [SBN read] reportBadBlock is rejected by Observer.

2019-10-31 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16964455#comment-16964455 ] Wei-Chiu Chuang commented on HDFS-14794: reportBadBlock is initiated by client or DataNode when

[jira] [Comment Edited] (HDFS-14794) [SBN read] reportBadBlock is rejected by Observer.

2019-10-31 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16964455#comment-16964455 ] Wei-Chiu Chuang edited comment on HDFS-14794 at 10/31/19 11:41 PM: ---

[jira] [Commented] (HDFS-14720) DataNode shouldn't report block as bad block if the block length is Long.MAX_VALUE.

2019-10-31 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16964450#comment-16964450 ] Wei-Chiu Chuang commented on HDFS-14720: BTW I think the fix is correct. +1 from me. > DataNode

[jira] [Commented] (HDFS-14720) DataNode shouldn't report block as bad block if the block length is Long.MAX_VALUE.

2019-10-31 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16964443#comment-16964443 ] Wei-Chiu Chuang commented on HDFS-14720: I don't think this fix is relevant to HDFS-14794. This

[jira] [Commented] (HDFS-14948) Improve HttpFS Server

2019-10-31 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16964354#comment-16964354 ] Wei-Chiu Chuang commented on HDFS-14948: [~smeng] and other contributors added a number of REST

[jira] [Updated] (HDFS-14941) Potential editlog race condition can cause corrupted file

2019-10-30 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14941: --- Labels: ha (was: ) > Potential editlog race condition can cause corrupted file >

[jira] [Commented] (HDFS-14941) Potential editlog race condition can cause corrupted file

2019-10-30 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963432#comment-16963432 ] Wei-Chiu Chuang commented on HDFS-14941: Quick question -- is this a problem with HA, multiple

[jira] [Updated] (HDFS-14912) Set dfs.image.string-tables.expanded default to false in branch-2.7

2019-10-28 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14912: --- Fix Version/s: 2.7.8 Resolution: Fixed Status: Resolved (was: Patch

[jira] [Comment Edited] (HDFS-14912) Set dfs.image.string-tables.expanded default to false in branch-2.7

2019-10-25 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16960220#comment-16960220 ] Wei-Chiu Chuang edited comment on HDFS-14912 at 10/26/19 12:07 AM: --- +1

[jira] [Commented] (HDFS-14912) Set dfs.image.string-tables.expanded default to false in branch-2.7

2019-10-25 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16960220#comment-16960220 ] Wei-Chiu Chuang commented on HDFS-14912: +1 > Set dfs.image.string-tables.expanded default to

[jira] [Commented] (HDFS-14931) hdfs crypto commands limit column width

2019-10-25 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16960214#comment-16960214 ] Wei-Chiu Chuang commented on HDFS-14931: +1 LGTM > hdfs crypto commands limit column width >

[jira] [Updated] (HDFS-14923) Remove dead code from HealthMonitor

2019-10-25 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14923: --- Fix Version/s: 3.2.2 3.1.4 3.3.0 Resolution:

[jira] [Resolved] (HDFS-14308) DFSStripedInputStream curStripeBuf is not freed by unbuffer()

2019-10-25 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-14308. Fix Version/s: 3.2.2 3.1.4 3.3.0 Resolution:

[jira] [Commented] (HDFS-14923) Remove dead code from HealthMonitor

2019-10-24 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16959319#comment-16959319 ] Wei-Chiu Chuang commented on HDFS-14923: +1 > Remove dead code from HealthMonitor >

[jira] [Updated] (HDFS-14492) Snapshot memory leak

2019-10-24 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14492: --- Fix Version/s: 3.2.2 3.1.4 > Snapshot memory leak >

[jira] [Updated] (HDFS-14910) Rename Snapshot with Pre Descendants Fail With IllegalArgumentException.

2019-10-24 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14910: --- Fix Version/s: 3.2.2 3.1.4 > Rename Snapshot with Pre Descendants Fail

[jira] [Updated] (HDFS-14492) Snapshot memory leak

2019-10-24 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14492: --- Fix Version/s: (was: 3.1.4) 3.3.0 > Snapshot memory leak >

[jira] [Updated] (HDFS-14910) Rename Snapshot with Pre Descendants Fail With IllegalArgumentException.

2019-10-24 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14910: --- Fix Version/s: 3.3.0 Resolution: Fixed Status: Resolved (was: Patch

[jira] [Updated] (HDFS-14450) Erasure Coding: decommissioning datanodes cause replicate a large number of duplicate EC internal blocks

2019-10-24 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14450: --- Resolution: Duplicate Status: Resolved (was: Patch Available) Close this one as a

[jira] [Commented] (HDFS-14450) Erasure Coding: decommissioning datanodes cause replicate a large number of duplicate EC internal blocks

2019-10-23 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16958500#comment-16958500 ] Wei-Chiu Chuang commented on HDFS-14450: [~ferhui] [~marvelrock] [~ayushtkn] this looks quite

[jira] [Commented] (HDFS-14908) LeaseManager should check parent-child relationship when filter open files.

2019-10-23 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16958331#comment-16958331 ] Wei-Chiu Chuang commented on HDFS-14908: Thanks. Looks really impressive. I just skimmed through

<    4   5   6   7   8   9   10   11   12   13   >