[jira] [Updated] (HDFS-8937) Erasure coding: do not throw exception when setting replication factor on EC files

[ https://issues.apache.org/jira/browse/HDFS-8937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8937: Summary: Erasure coding: do not throw exception when setting replication factor on EC files (was: Fix the

[jira] [Commented] (HDFS-8999) Namenode need not wait for {{blockReceived}} for the last block before completing a file.

[ https://issues.apache.org/jira/browse/HDFS-8999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14727883#comment-14727883 ] Jing Zhao commented on HDFS-8999: - Thanks for the comments, [~shv]. So one of the motivations for this

[jira] [Updated] (HDFS-9003) ForkJoin thread pool leaks

[ https://issues.apache.org/jira/browse/HDFS-9003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-9003: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved (was:

[jira] [Commented] (HDFS-8999) Namenode need not wait for {{blockReceived}} for the last block before completing a file.

[ https://issues.apache.org/jira/browse/HDFS-8999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14726300#comment-14726300 ] Jing Zhao commented on HDFS-8999: - bq. if we are going to trust the client, do we even need IBRs? Yeah, I

[jira] [Commented] (HDFS-8287) DFSStripedOutputStream.writeChunk should not wait for writing parity

[ https://issues.apache.org/jira/browse/HDFS-8287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14726497#comment-14726497 ] Jing Zhao commented on HDFS-8287: - Thanks for working on this, Kai! I agree with Zhe's comments. Besides of

[jira] [Commented] (HDFS-8937) Fix the exception when set replication to Erasure Coding files

[ https://issues.apache.org/jira/browse/HDFS-8937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14726169#comment-14726169 ] Jing Zhao commented on HDFS-8937: - Thanks for working on this, [~demongaorui]! Here are the comments for

[jira] [Commented] (HDFS-8833) Erasure coding: store EC schema and cell size in INodeFile and eliminate notion of EC zones

[ https://issues.apache.org/jira/browse/HDFS-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14723837#comment-14723837 ] Jing Zhao commented on HDFS-8833: - Thanks for all the discussion! Currently I think it's fine to set an EC

[jira] [Commented] (HDFS-8980) Remove unnecessary block replacement in INodeFile

[ https://issues.apache.org/jira/browse/HDFS-8980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14723852#comment-14723852 ] Jing Zhao commented on HDFS-8980: - Sure. I will take a look at HDFS-8763. > Remove unnecessary block

[jira] [Updated] (HDFS-8980) Remove unnecessary block replacement in INodeFile

[ https://issues.apache.org/jira/browse/HDFS-8980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8980: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved (was:

[jira] [Commented] (HDFS-8982) Consolidate getFileReplication and getPreferredBlockReplication in INodeFile

[ https://issues.apache.org/jira/browse/HDFS-8982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14724094#comment-14724094 ] Jing Zhao commented on HDFS-8982: - Actually I think here it's better to use {{getFileReplication}} instead

[jira] [Comment Edited] (HDFS-8982) Consolidate getFileReplication and getPreferredBlockReplication in INodeFile

[ https://issues.apache.org/jira/browse/HDFS-8982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14724094#comment-14724094 ] Jing Zhao edited comment on HDFS-8982 at 8/31/15 9:44 PM: -- Actually I think for

[jira] [Commented] (HDFS-8936) Simplify Erasure Coding Zone DiskSpace quota exceeded exception error message

[ https://issues.apache.org/jira/browse/HDFS-8936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14724151#comment-14724151 ] Jing Zhao commented on HDFS-8936: - Maybe we can keep this jira open until we fix the known issues,

[jira] [Commented] (HDFS-8982) Consolidate getFileReplication and getPreferredBlockReplication in INodeFile

[ https://issues.apache.org/jira/browse/HDFS-8982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14724372#comment-14724372 ] Jing Zhao commented on HDFS-8982: - After HDFS-8823, the replication factor is recorded inside of BlockInfo,

[jira] [Commented] (HDFS-8763) After file closed, a race condition between IBR of 3rd replica of lastBlock and ReplicationMonitor

[ https://issues.apache.org/jira/browse/HDFS-8763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14724484#comment-14724484 ] Jing Zhao commented on HDFS-8763: - So one question I always have is whether we need to let NameNode wait

[jira] [Comment Edited] (HDFS-8763) After file closed, a race condition between IBR of 3rd replica of lastBlock and ReplicationMonitor

[ https://issues.apache.org/jira/browse/HDFS-8763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14724484#comment-14724484 ] Jing Zhao edited comment on HDFS-8763 at 9/1/15 12:40 AM: -- So one question I

[jira] [Created] (HDFS-8980) Remove unnecessary block replacement in INodeFile

Jing Zhao created HDFS-8980: --- Summary: Remove unnecessary block replacement in INodeFile Key: HDFS-8980 URL: https://issues.apache.org/jira/browse/HDFS-8980 Project: Hadoop HDFS Issue Type:

[jira] [Updated] (HDFS-8980) Remove unnecessary block replacement in INodeFile

[ https://issues.apache.org/jira/browse/HDFS-8980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8980: Attachment: HDFS-8980.000.patch Remove unnecessary block replacement in INodeFile

[jira] [Updated] (HDFS-8980) Remove unnecessary block replacement in INodeFile

[ https://issues.apache.org/jira/browse/HDFS-8980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8980: Status: Patch Available (was: Open) Remove unnecessary block replacement in INodeFile

[jira] [Commented] (HDFS-8937) Fix the exception when set replication to Erasure Coding files

[ https://issues.apache.org/jira/browse/HDFS-8937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14715319#comment-14715319 ] Jing Zhao commented on HDFS-8937: - The MR issue is also found by [~demongaorui]. To put a

[jira] [Commented] (HDFS-8937) Fix the exception when set replication to Erasure Coding files

[ https://issues.apache.org/jira/browse/HDFS-8937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14715294#comment-14715294 ] Jing Zhao commented on HDFS-8937: - I agree with Zhe that setting replication on a file with

[jira] [Commented] (HDFS-8838) Tolerate datanode failures in DFSStripedOutputStream when the data length is small

[ https://issues.apache.org/jira/browse/HDFS-8838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14715767#comment-14715767 ] Jing Zhao commented on HDFS-8838: - Hi [~walter.k.su], I've committed HDFS-8896 in

[jira] [Updated] (HDFS-8909) Erasure coding: update BlockInfoContiguousUC and BlockInfoStripedUC to use BlockUnderConstructionFeature

[ https://issues.apache.org/jira/browse/HDFS-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8909: Attachment: HDFS-8909-HDFS-7285.004.patch Thanks for the analysis of the test failure, Walter! Update the

[jira] [Updated] (HDFS-8896) DataNode object isn't GCed when shutdown, because it has GC root in ShutdownHookManager

[ https://issues.apache.org/jira/browse/HDFS-8896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8896: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved (was:

[jira] [Commented] (HDFS-8248) Store INodeId instead of the INodeFile object in BlockInfoContiguous

[ https://issues.apache.org/jira/browse/HDFS-8248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14715863#comment-14715863 ] Jing Zhao commented on HDFS-8248: - The latest patch looks good to me. +1. The failed test

[jira] [Updated] (HDFS-8928) Improvements for BlockUnderConstructionFeature: ReplicaUnderConstruction as a separate class and replicas as an array

[ https://issues.apache.org/jira/browse/HDFS-8928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8928: Attachment: HDFS-8928.001.patch Improvements for BlockUnderConstructionFeature: ReplicaUnderConstruction as

[jira] [Commented] (HDFS-8248) Store INodeId instead of the INodeFile object in BlockInfoContiguous

[ https://issues.apache.org/jira/browse/HDFS-8248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709775#comment-14709775 ] Jing Zhao commented on HDFS-8248: - The 004 patch looks good to me. Some minors: #

[jira] [Updated] (HDFS-8928) Improvements for BlockUnderConstructionFeature: ReplicaUnderConstruction as a separate class and replicas as an array

[ https://issues.apache.org/jira/browse/HDFS-8928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8928: Priority: Minor (was: Major) Improvements for BlockUnderConstructionFeature: ReplicaUnderConstruction as a

[jira] [Updated] (HDFS-8928) Improvements for BlockUnderConstructionFeature: ReplicaUnderConstruction as a separate class and replicas as an array

[ https://issues.apache.org/jira/browse/HDFS-8928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8928: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved (was:

[jira] [Updated] (HDFS-8909) Erasure coding: update BlockInfoContiguousUC and BlockInfoStripedUC to use BlockUnderConstructionFeature

[ https://issues.apache.org/jira/browse/HDFS-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8909: Attachment: HDFS-8909.003.patch Thanks for the review, Walter and Zhe. Update the patch to address Walter's

[jira] [Commented] (HDFS-4167) Add support for restoring/rolling back to a snapshot

[ https://issues.apache.org/jira/browse/HDFS-4167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14707824#comment-14707824 ] Jing Zhao commented on HDFS-4167: - Sure, please feel free to assign the jira to yourself.

[jira] [Updated] (HDFS-8838) Tolerate datanode failures in DFSStripedOutputStream when the data length is small

[ https://issues.apache.org/jira/browse/HDFS-8838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8838: Attachment: HDFS-8838-HDFS-7285-20150821.patch Upload a rebased patch for Nicholas. Tolerate datanode

[jira] [Created] (HDFS-8930) Block report lease may leak if the 2nd full block report comes when NN is still in safemode

Jing Zhao created HDFS-8930: --- Summary: Block report lease may leak if the 2nd full block report comes when NN is still in safemode Key: HDFS-8930 URL: https://issues.apache.org/jira/browse/HDFS-8930

[jira] [Commented] (HDFS-8930) Block report lease may leak if the 2nd full block report comes when NN is still in safemode

[ https://issues.apache.org/jira/browse/HDFS-8930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14705942#comment-14705942 ] Jing Zhao commented on HDFS-8930: - A simple fix can be removing the lease in this scenario:

[jira] [Updated] (HDFS-8809) HDFS fsck reports under construction blocks as CORRUPT

[ https://issues.apache.org/jira/browse/HDFS-8809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8809: Summary: HDFS fsck reports under construction blocks as CORRUPT (was: HDFS fsck reports HBase WALs files

[jira] [Commented] (HDFS-8909) Erasure coding: update BlockInfoContiguousUC and BlockInfoStripedUC to use BlockUnderConstructionFeature

[ https://issues.apache.org/jira/browse/HDFS-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14705937#comment-14705937 ] Jing Zhao commented on HDFS-8909: - I do not think it's necessary to do another ID check

[jira] [Updated] (HDFS-8809) HDFS fsck reports under construction blocks as CORRUPT

[ https://issues.apache.org/jira/browse/HDFS-8809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8809: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Target Version/s:

[jira] [Updated] (HDFS-8930) Block report lease may leak if the 2nd full block report comes when NN is still in safemode

[ https://issues.apache.org/jira/browse/HDFS-8930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8930: Affects Version/s: 2.8.0 Status: Patch Available (was: Open) Block report lease may leak if

[jira] [Updated] (HDFS-8930) Block report lease may leak if the 2nd full block report comes when NN is still in safemode

[ https://issues.apache.org/jira/browse/HDFS-8930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8930: Attachment: HDFS-8930.000.patch Block report lease may leak if the 2nd full block report comes when NN is

[jira] [Assigned] (HDFS-8930) Block report lease may leak if the 2nd full block report comes when NN is still in safemode

[ https://issues.apache.org/jira/browse/HDFS-8930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao reassigned HDFS-8930: --- Assignee: Jing Zhao Block report lease may leak if the 2nd full block report comes when NN is

[jira] [Commented] (HDFS-8823) Move replication factor into individual blocks

[ https://issues.apache.org/jira/browse/HDFS-8823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14705840#comment-14705840 ] Jing Zhao commented on HDFS-8823: - +1 for the latest patch. Move replication factor into

[jira] [Updated] (HDFS-8867) Enable optimized block reports

[ https://issues.apache.org/jira/browse/HDFS-8867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8867: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.7.2 Status: Resolved (was:

[jira] [Commented] (HDFS-8803) Move DfsClientConf to hdfs-client

[ https://issues.apache.org/jira/browse/HDFS-8803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703438#comment-14703438 ] Jing Zhao commented on HDFS-8803: - The test failure should be unrelated. +1. Move

[jira] [Updated] (HDFS-8809) HDFS fsck reports HBase WALs files (under construction) as CORRUPT (missing blocks) when HBase is running

[ https://issues.apache.org/jira/browse/HDFS-8809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8809: Attachment: HDFS-8809.000.patch There is another issue here, which I think exists before HDFS-8215. When

[jira] [Updated] (HDFS-8809) HDFS fsck reports HBase WALs files (under construction) as CORRUPT (missing blocks) when HBase is running

[ https://issues.apache.org/jira/browse/HDFS-8809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8809: Status: Patch Available (was: Open) HDFS fsck reports HBase WALs files (under construction) as CORRUPT

[jira] [Commented] (HDFS-8828) Utilize Snapshot diff report to build incremental copy list in distcp

[ https://issues.apache.org/jira/browse/HDFS-8828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703863#comment-14703863 ] Jing Zhao commented on HDFS-8828: - Thanks for updating the patch, Yufei! The latest patch

[jira] [Commented] (HDFS-8823) Move replication factor into individual blocks

[ https://issues.apache.org/jira/browse/HDFS-8823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704010#comment-14704010 ] Jing Zhao commented on HDFS-8823: - The latest patch looks good to me. One comment is that

[jira] [Updated] (HDFS-8909) Erasure coding: update BlockInfoContiguousUC and BlockInfoStripedUC to use BlockUnderConstructionFeature

[ https://issues.apache.org/jira/browse/HDFS-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8909: Attachment: HDFS-8909.001.patch Thanks for the review, Zhe! Update the patch to address your comments. bq.

[jira] [Commented] (HDFS-8917) Cleanup BlockInfoUnderConstruction from comments and tests

[ https://issues.apache.org/jira/browse/HDFS-8917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703819#comment-14703819 ] Jing Zhao commented on HDFS-8917: - +1. I will commit it shortly. Cleanup

[jira] [Updated] (HDFS-8917) Cleanup BlockInfoUnderConstruction from comments and tests

[ https://issues.apache.org/jira/browse/HDFS-8917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8917: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved (was:

[jira] [Comment Edited] (HDFS-8828) Utilize Snapshot diff report to build copy list in distcp

[ https://issues.apache.org/jira/browse/HDFS-8828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14701760#comment-14701760 ] Jing Zhao edited comment on HDFS-8828 at 8/18/15 6:42 PM: -- Thanks

[jira] [Commented] (HDFS-8828) Utilize Snapshot diff report to build copy list in distcp

[ https://issues.apache.org/jira/browse/HDFS-8828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14701760#comment-14701760 ] Jing Zhao commented on HDFS-8828: - Thanks for updating the patch, Yufei! Some further

[jira] [Comment Edited] (HDFS-8828) Utilize Snapshot diff report to build copy list in distcp

[ https://issues.apache.org/jira/browse/HDFS-8828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14701760#comment-14701760 ] Jing Zhao edited comment on HDFS-8828 at 8/18/15 6:54 PM: -- Thanks

[jira] [Commented] (HDFS-8867) Enable optimized block reports

[ https://issues.apache.org/jira/browse/HDFS-8867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14701964#comment-14701964 ] Jing Zhao commented on HDFS-8867: - Thanks for the fix, Daryn. +1 pending Jenkins. Enable

[jira] [Assigned] (HDFS-8809) HDFS fsck reports HBase WALs files (under construction) as CORRUPT (missing blocks) when HBase is running

[ https://issues.apache.org/jira/browse/HDFS-8809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao reassigned HDFS-8809: --- Assignee: Jing Zhao HDFS fsck reports HBase WALs files (under construction) as CORRUPT (missing

[jira] [Commented] (HDFS-8809) HDFS fsck reports HBase WALs files (under construction) as CORRUPT (missing blocks) when HBase is running

[ https://issues.apache.org/jira/browse/HDFS-8809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702257#comment-14702257 ] Jing Zhao commented on HDFS-8809: - Looks like this is an issue caused by HDFS-8215. Before,

[jira] [Commented] (HDFS-8909) Erasure coding: update BlockInfoContiguousUC and BlockInfoStripedUC to use BlockUnderConstructionFeature

[ https://issues.apache.org/jira/browse/HDFS-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14700589#comment-14700589 ] Jing Zhao commented on HDFS-8909: - Instead of keeping doing git rebase, maybe we should

[jira] [Commented] (HDFS-8909) Erasure coding: update BlockInfoContiguousUC and BlockInfoStripedUC to use BlockUnderConstructionFeature

[ https://issues.apache.org/jira/browse/HDFS-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14700481#comment-14700481 ] Jing Zhao commented on HDFS-8909: - I think maybe to save us time from now on we can use git

[jira] [Updated] (HDFS-8909) Erasure coding: update BlockInfoContiguousUC and BlockInfoStripedUC to use BlockUnderConstructionFeature

[ https://issues.apache.org/jira/browse/HDFS-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8909: Attachment: HDFS-8909.000.patch Patch against HDFS-7285-REBASE branch. Erasure coding: update

[jira] [Commented] (HDFS-8833) Erasure coding: store EC schema and cell size in INodeFile and eliminate notion of EC zones

[ https://issues.apache.org/jira/browse/HDFS-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14699910#comment-14699910 ] Jing Zhao commented on HDFS-8833: - Yes, a conversion tool can be helpful. Erasure coding:

[jira] [Commented] (HDFS-8828) Utilize Snapshot diff report to build copy list in distcp

[ https://issues.apache.org/jira/browse/HDFS-8828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14699950#comment-14699950 ] Jing Zhao commented on HDFS-8828: - Yes, we can try to use DistCpSync or a new class to

[jira] [Commented] (HDFS-8278) HDFS Balancer should consider remaining storage % when checking for under-utilized machines

[ https://issues.apache.org/jira/browse/HDFS-8278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14700036#comment-14700036 ] Jing Zhao commented on HDFS-8278: - +1 pending Jenkins. HDFS Balancer should consider

[jira] [Assigned] (HDFS-8909) Erasure coding: update BlockInfoContiguousUC and BlockInfoStripedUC to use BlockUnderConstructionFeature

[ https://issues.apache.org/jira/browse/HDFS-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao reassigned HDFS-8909: --- Assignee: Jing Zhao Erasure coding: update BlockInfoContiguousUC and BlockInfoStripedUC to use

[jira] [Commented] (HDFS-8909) Erasure coding: update BlockInfoContiguousUC and BlockInfoStripedUC to use BlockUnderConstructionFeature

[ https://issues.apache.org/jira/browse/HDFS-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14700042#comment-14700042 ] Jing Zhao commented on HDFS-8909: - Thanks for filing the jira, Zhe! I will try to merge

[jira] [Commented] (HDFS-8909) Erasure coding: update BlockInfoContiguousUC and BlockInfoStripedUC to use BlockUnderConstructionFeature

[ https://issues.apache.org/jira/browse/HDFS-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14700043#comment-14700043 ] Jing Zhao commented on HDFS-8909: - Just to confirm, should I merge changes into both

[jira] [Commented] (HDFS-8867) Enable optimized block reports

[ https://issues.apache.org/jira/browse/HDFS-8867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14700135#comment-14700135 ] Jing Zhao commented on HDFS-8867: - Any update here? I can also upload a patch to fix if

[jira] [Commented] (HDFS-8833) Erasure coding: store EC schema and cell size in INodeFile and eliminate notion of EC zones

[ https://issues.apache.org/jira/browse/HDFS-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14698978#comment-14698978 ] Jing Zhao commented on HDFS-8833: - Thanks for the patch, Zhe! bq. to allow

[jira] [Commented] (HDFS-8828) Utilize Snapshot diff report to build copy list in distcp

[ https://issues.apache.org/jira/browse/HDFS-8828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14698985#comment-14698985 ] Jing Zhao commented on HDFS-8828: - Thanks again for working on this, Yufei. The patch looks

[jira] [Commented] (HDFS-8801) Convert BlockInfoUnderConstruction as a feature

[ https://issues.apache.org/jira/browse/HDFS-8801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14697787#comment-14697787 ] Jing Zhao commented on HDFS-8801: - Actually converting BlockInfoUnderConstruction can bring

[jira] [Updated] (HDFS-8891) HDFS concat should keep srcs order

[ https://issues.apache.org/jira/browse/HDFS-8891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8891: Issue Type: Bug (was: Improvement) HDFS concat should keep srcs order --

[jira] [Commented] (HDFS-8891) HDFS concat should keep srcs order

[ https://issues.apache.org/jira/browse/HDFS-8891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14697814#comment-14697814 ] Jing Zhao commented on HDFS-8891: - +1. I will commit the patch shortly. HDFS concat

[jira] [Updated] (HDFS-8801) Convert BlockInfoUnderConstruction as a feature

[ https://issues.apache.org/jira/browse/HDFS-8801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8801: Attachment: HDFS-8801.000.patch Initial patch to demo the idea. Convert BlockInfoUnderConstruction as a

[jira] [Updated] (HDFS-8801) Convert BlockInfoUnderConstruction as a feature

[ https://issues.apache.org/jira/browse/HDFS-8801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8801: Affects Version/s: (was: 2.7.1) Status: Patch Available (was: Open) Convert

[jira] [Updated] (HDFS-8891) HDFS concat should keep srcs order

[ https://issues.apache.org/jira/browse/HDFS-8891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8891: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved (was:

[jira] [Assigned] (HDFS-8801) Convert BlockInfoUnderConstruction as a feature

[ https://issues.apache.org/jira/browse/HDFS-8801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao reassigned HDFS-8801: --- Assignee: Jing Zhao Convert BlockInfoUnderConstruction as a feature

[jira] [Commented] (HDFS-8891) HDFS concat should keep srcs order

[ https://issues.apache.org/jira/browse/HDFS-8891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14695634#comment-14695634 ] Jing Zhao commented on HDFS-8891: - Thanks for working on this, Yong! Agree we should keep

[jira] [Commented] (HDFS-8828) Utilize Snapshot diff report to build copy list in distcp

[ https://issues.apache.org/jira/browse/HDFS-8828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696023#comment-14696023 ] Jing Zhao commented on HDFS-8828: - Sure. I will review the patch. Thanks for the work,

[jira] [Commented] (HDFS-8805) Archival Storage: getStoragePolicy should not need superuser privilege

[ https://issues.apache.org/jira/browse/HDFS-8805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14682126#comment-14682126 ] Jing Zhao commented on HDFS-8805: - Thanks for updating the patch, [~brahmareddy]. +1 for

[jira] [Updated] (HDFS-8805) Archival Storage: getStoragePolicy should not need superuser privilege

[ https://issues.apache.org/jira/browse/HDFS-8805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8805: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: (was: 2.6.0) 2.8.0

[jira] [Commented] (HDFS-8833) Erasure coding: store EC schema and cell size in INodeFile and eliminate notion of EC zones

[ https://issues.apache.org/jira/browse/HDFS-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14682419#comment-14682419 ] Jing Zhao commented on HDFS-8833: - bq. I agree that a pure XAttr-based solution is simpler,

[jira] [Commented] (HDFS-8833) Erasure coding: store EC schema and cell size in INodeFile and eliminate notion of EC zones

[ https://issues.apache.org/jira/browse/HDFS-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14692549#comment-14692549 ] Jing Zhao commented on HDFS-8833: - Thanks for the summary, Zhe. The new proposal looks

[jira] [Commented] (HDFS-8888) Support the volume concepts in HDFS

[ https://issues.apache.org/jira/browse/HDFS-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14692590#comment-14692590 ] Jing Zhao commented on HDFS-: - +1 to have volume in HDFS. Also congrats for the jira

[jira] [Commented] (HDFS-8881) Erasure Coding: internal blocks got missed and got over-replicated at the same time

[ https://issues.apache.org/jira/browse/HDFS-8881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14680944#comment-14680944 ] Jing Zhao commented on HDFS-8881: - Thanks for the fix, Walter! The patch looks good to me

[jira] [Commented] (HDFS-7285) Erasure Coding Support inside HDFS

[ https://issues.apache.org/jira/browse/HDFS-7285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681006#comment-14681006 ] Jing Zhao commented on HDFS-7285: - Looks like the current situation is little bit confusing

[jira] [Updated] (HDFS-8827) Erasure Coding: Fix NPE when NameNode processes over-replicated striped blocks

[ https://issues.apache.org/jira/browse/HDFS-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8827: Summary: Erasure Coding: Fix NPE when NameNode processes over-replicated striped blocks (was: Erasure

[jira] [Commented] (HDFS-8827) Erasure Coding: Fix NPE when NameNode processes over-replicated striped blocks

[ https://issues.apache.org/jira/browse/HDFS-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662143#comment-14662143 ] Jing Zhao commented on HDFS-8827: - +1. I will commit it shortly. Erasure Coding: Fix NPE

[jira] [Issue Comment Deleted] (HDFS-8867) Enable optimized block reports

[ https://issues.apache.org/jira/browse/HDFS-8867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8867: Comment: was deleted (was: Cool, thanks!) Enable optimized block reports --

[jira] [Created] (HDFS-8877) Allow bypassing some minor exceptions while loading editlog

Jing Zhao created HDFS-8877: --- Summary: Allow bypassing some minor exceptions while loading editlog Key: HDFS-8877 URL: https://issues.apache.org/jira/browse/HDFS-8877 Project: Hadoop HDFS Issue

[jira] [Commented] (HDFS-8877) Allow bypassing some minor exceptions while loading editlog

[ https://issues.apache.org/jira/browse/HDFS-8877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662259#comment-14662259 ] Jing Zhao commented on HDFS-8877: - Maybe a simple way as the first step can be: 1) to

[jira] [Commented] (HDFS-8805) Archival Storage: getStoragePolicy should not need superuser privilege

[ https://issues.apache.org/jira/browse/HDFS-8805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662129#comment-14662129 ] Jing Zhao commented on HDFS-8805: - Thanks for updating the patch, [~brahmareddy]! The new

[jira] [Commented] (HDFS-8867) Enable optimized block reports

[ https://issues.apache.org/jira/browse/HDFS-8867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662169#comment-14662169 ] Jing Zhao commented on HDFS-8867: - Looks like the issue here is that

[jira] [Commented] (HDFS-8876) Make hard coded parameters used by balancer and other tools configurable

[ https://issues.apache.org/jira/browse/HDFS-8876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662177#comment-14662177 ] Jing Zhao commented on HDFS-8876: - Thanks for reporting the issue, Ming! Maybe we can move

[jira] [Commented] (HDFS-8867) Enable optimized block reports

[ https://issues.apache.org/jira/browse/HDFS-8867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662180#comment-14662180 ] Jing Zhao commented on HDFS-8867: - Cool, thanks! Enable optimized block reports

[jira] [Updated] (HDFS-8827) Erasure Coding: Fix NPE when NameNode processes over-replicated striped blocks

[ https://issues.apache.org/jira/browse/HDFS-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8827: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-7285 Status: Resolved

[jira] [Commented] (HDFS-8877) Allow bypassing some minor exceptions while loading editlog

[ https://issues.apache.org/jira/browse/HDFS-8877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662822#comment-14662822 ] Jing Zhao commented on HDFS-8877: - [~cnauroth] just pointed me to the existing recovery

[jira] [Resolved] (HDFS-8877) Allow bypassing some minor exceptions while loading editlog

[ https://issues.apache.org/jira/browse/HDFS-8877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8877. - Resolution: Not A Problem Allow bypassing some minor exceptions while loading editlog

[jira] [Commented] (HDFS-8827) Erasure Coding: When namenode processes over replicated striped block, NPE will be occur in ReplicationMonitor

[ https://issues.apache.org/jira/browse/HDFS-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660542#comment-14660542 ] Jing Zhao commented on HDFS-8827: - Thanks [~tfukudom] for further investigation and thanks

[jira] [Reopened] (HDFS-8623) Refactor NameNode handling of invalid, corrupt, and under-recovery blocks

[ https://issues.apache.org/jira/browse/HDFS-8623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao reopened HDFS-8623: - Refactor NameNode handling of invalid, corrupt, and under-recovery blocks

[jira] [Resolved] (HDFS-8623) Refactor NameNode handling of invalid, corrupt, and under-recovery blocks

[ https://issues.apache.org/jira/browse/HDFS-8623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8623. - Resolution: Later Fix Version/s: (was: 2.8.0) The patch was reverted from trunk and branch-2

[jira] [Updated] (HDFS-8499) Refactor BlockInfo class hierarchy with static helper class

[ https://issues.apache.org/jira/browse/HDFS-8499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8499: Resolution: Later Fix Version/s: (was: 2.8.0) Status: Resolved (was: Patch Available)

[jira] [Commented] (HDFS-8805) Archival Storage: getStoragePolicy should not need superuser privilege

[ https://issues.apache.org/jira/browse/HDFS-8805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660391#comment-14660391 ] Jing Zhao commented on HDFS-8805: - Hi [~brahmareddy], thanks for updating the patch! Looks

<    3   4   5   6   7   8   9   10   11   12   >