[jira] [Created] (HDFS-3370) HDFS hardlink

Hairong Kuang created HDFS-3370: --- Summary: HDFS hardlink Key: HDFS-3370 URL: https://issues.apache.org/jira/browse/HDFS-3370 Project: Hadoop HDFS Issue Type: New Feature Reporter:

[jira] [Updated] (HDFS-3370) HDFS hardlink

[ https://issues.apache.org/jira/browse/HDFS-3370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-3370: Description: We'd like to add a new feature hardlink to HDFS that allows harlinked files to share

[jira] [Commented] (HDFS-3370) HDFS hardlink

[ https://issues.apache.org/jira/browse/HDFS-3370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13268735#comment-13268735 ] Hairong Kuang commented on HDFS-3370: - Sajay, you are right. HDFS hardlink is only a

[jira] [Created] (HDFS-2362) More Improvements on NameNode Scalability

More Improvements on NameNode Scalability - Key: HDFS-2362 URL: https://issues.apache.org/jira/browse/HDFS-2362 Project: Hadoop HDFS Issue Type: Improvement Components: name-node

[jira] [Commented] (HDFS-1779) After NameNode restart , Clients can not read partial files even after client invokes Sync.

[ https://issues.apache.org/jira/browse/HDFS-1779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13098350#comment-13098350 ] Hairong Kuang commented on HDFS-1779: - Uma, thanks for addressing Todd's review

[jira] [Commented] (HDFS-1779) After NameNode restart , Clients can not read partial files even after client invokes Sync.

[ https://issues.apache.org/jira/browse/HDFS-1779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13098486#comment-13098486 ] Hairong Kuang commented on HDFS-1779: - With this patch, blockReceived does not get sent

[jira] [Updated] (HDFS-1779) After NameNode restart , Clients can not read partial files even after client invokes Sync.

[ https://issues.apache.org/jira/browse/HDFS-1779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1779: Attachment: bbwReportAppend.patch Here comes the patch: 1. It adds a new RPC

[jira] [Commented] (HDFS-1779) After NameNode restart , Clients can not read partial files even after client invokes Sync.

[ https://issues.apache.org/jira/browse/HDFS-1779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092632#comment-13092632 ] Hairong Kuang commented on HDFS-1779: - In our fb internal branch, we fixed the problem

[jira] [Commented] (HDFS-1779) After NameNode restart , Clients can not read partial files even after client invokes Sync.

[ https://issues.apache.org/jira/browse/HDFS-1779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092647#comment-13092647 ] Hairong Kuang commented on HDFS-1779: - Yes, when processing a bbw block report, we put

[jira] [Updated] (HDFS-395) DFS Scalability: Incremental block reports

[ https://issues.apache.org/jira/browse/HDFS-395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-395: --- Resolution: Fixed Status: Resolved (was: Patch Available) I've committed this. Thank Tomasz

[jira] [Commented] (HDFS-395) DFS Scalability: Incremental block reports

[ https://issues.apache.org/jira/browse/HDFS-395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13091546#comment-13091546 ] Hairong Kuang commented on HDFS-395: Sure, marked it as 0.23. DFS Scalability:

[jira] [Updated] (HDFS-395) DFS Scalability: Incremental block reports

[ https://issues.apache.org/jira/browse/HDFS-395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-395: --- Fix Version/s: 0.23.0 DFS Scalability: Incremental block reports

[jira] [Updated] (HDFS-395) DFS Scalability: Incremental block reports

[ https://issues.apache.org/jira/browse/HDFS-395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-395: --- Fix Version/s: (was: 0.23.0) 0.24.0 DFS Scalability: Incremental block reports

[jira] [Commented] (HDFS-395) DFS Scalability: Incremental block reports

[ https://issues.apache.org/jira/browse/HDFS-395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13091564#comment-13091564 ] Hairong Kuang commented on HDFS-395: My bad. I did not notice that 0.23 is branched.

[jira] [Commented] (HDFS-395) DFS Scalability: Incremental block reports

[ https://issues.apache.org/jira/browse/HDFS-395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13090422#comment-13090422 ] Hairong Kuang commented on HDFS-395: The patch looks good to me and the failed tests do

[jira] [Assigned] (HDFS-395) DFS Scalability: Incremental block reports

[ https://issues.apache.org/jira/browse/HDFS-395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang reassigned HDFS-395: -- Assignee: Tomasz Nykiel (was: dhruba borthakur) DFS Scalability: Incremental block reports

[jira] [Commented] (HDFS-395) DFS Scalability: Incremental block reports

[ https://issues.apache.org/jira/browse/HDFS-395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13065643#comment-13065643 ] Hairong Kuang commented on HDFS-395: NN first deletes it in the block map and adds it

[jira] [Commented] (HDFS-395) DFS Scalability: Incremental block reports

[ https://issues.apache.org/jira/browse/HDFS-395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13060086#comment-13060086 ] Hairong Kuang commented on HDFS-395: For the incremental block reports, we can always

[jira] [Commented] (HDFS-1948) Forward port 'hdfs-1520 lightweight namenode operation to trigger lease reccovery'

[ https://issues.apache.org/jira/browse/HDFS-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13044979#comment-13044979 ] Hairong Kuang commented on HDFS-1948: - Both patches look good! Stack, thanks for your

[jira] [Commented] (HDFS-1948) Forward port 'hdfs-1520 lightweight namenode operation to trigger lease reccovery'

[ https://issues.apache.org/jira/browse/HDFS-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13045067#comment-13045067 ] Hairong Kuang commented on HDFS-1948: - Yes, please update the patch for the trunk.

[jira] [Commented] (HDFS-1948) Forward port 'hdfs-1520 lightweight namenode operation to trigger lease reccovery'

[ https://issues.apache.org/jira/browse/HDFS-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13045279#comment-13045279 ] Hairong Kuang commented on HDFS-1948: - +1 lgtm. Thanks, Stack! Forward port

[jira] [Commented] (HDFS-1332) When unable to place replicas, BlockPlacementPolicy should log reasons nodes were excluded

[ https://issues.apache.org/jira/browse/HDFS-1332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13033448#comment-13033448 ] Hairong Kuang commented on HDFS-1332: - Constructing a HashMap to keep track of error

[jira] [Commented] (HDFS-1458) Improve checkpoint performance by avoiding unnecessary image downloads

[ https://issues.apache.org/jira/browse/HDFS-1458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13032501#comment-13032501 ] Hairong Kuang commented on HDFS-1458: - Todd, you need the fix to HDFS-1627. We already

[jira] [Updated] (HDFS-1627) Fix NullPointerException in Secondary NameNode

[ https://issues.apache.org/jira/browse/HDFS-1627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1627: Attachment: NPE_SNN1.patch A patch with a unit test. Fix NullPointerException in Secondary

[jira] [Updated] (HDFS-1627) Fix NullPointerException in Secondary NameNode

[ https://issues.apache.org/jira/browse/HDFS-1627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1627: Status: Open (was: Patch Available) Fix NullPointerException in Secondary NameNode

[jira] [Updated] (HDFS-1627) Fix NullPointerException in Secondary NameNode

[ https://issues.apache.org/jira/browse/HDFS-1627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1627: Status: Patch Available (was: Open) Fix NullPointerException in Secondary NameNode

[jira] [Updated] (HDFS-1627) Fix NullPointerException in Secondary NameNode

[ https://issues.apache.org/jira/browse/HDFS-1627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1627: Resolution: Fixed Hadoop Flags: [Reviewed] Status: Resolved (was: Patch Available)

[jira] [Resolved] (HDFS-1904) Secondary Namenode dies when a mkdir on a non-existent parent directory is run

[ https://issues.apache.org/jira/browse/HDFS-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang resolved HDFS-1904. - Resolution: Duplicate Secondary Namenode dies when a mkdir on a non-existent parent directory is

[jira] [Commented] (HDFS-1826) NameNode should save image to name directories in parallel during upgrade

[ https://issues.apache.org/jira/browse/HDFS-1826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13031067#comment-13031067 ] Hairong Kuang commented on HDFS-1826: - +1 NameNode should save image to name

[jira] [Updated] (HDFS-1826) NameNode should save image to name directories in parallel during upgrade

[ https://issues.apache.org/jira/browse/HDFS-1826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1826: Resolution: Fixed Release Note: I've committed this. Thanks, Matt! Hadoop Flags:

[jira] [Commented] (HDFS-78) Eliminate redundant searches in the namespace directory tree.

[ https://issues.apache.org/jira/browse/HDFS-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13028879#comment-13028879 ] Hairong Kuang commented on HDFS-78: --- In our HDFS cluster NameNode becomes very slow when

[jira] [Assigned] (HDFS-78) Eliminate redundant searches in the namespace directory tree.

[ https://issues.apache.org/jira/browse/HDFS-78?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang reassigned HDFS-78: - Assignee: Hairong Kuang Eliminate redundant searches in the namespace directory tree.

[jira] [Commented] (HDFS-101) DFS write pipeline : DFSClient sometimes does not detect second datanode failure

[ https://issues.apache.org/jira/browse/HDFS-101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13027104#comment-13027104 ] Hairong Kuang commented on HDFS-101: By simply looking at your client-side log, it seems

[jira] [Commented] (HDFS-1826) NameNode should save image to name directories in parallel during upgrade

[ https://issues.apache.org/jira/browse/HDFS-1826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13025950#comment-13025950 ] Hairong Kuang commented on HDFS-1826: - Matt, you are right. SaveCurrent already takes

[jira] [Commented] (HDFS-1826) NameNode should save image to name directories in parallel during upgrade

[ https://issues.apache.org/jira/browse/HDFS-1826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13025953#comment-13025953 ] Hairong Kuang commented on HDFS-1826: - +1. The patch looks good. I would prefer to have

[jira] [Updated] (HDFS-1767) Namenode should ignore non-initial block reports from datanodes when in safemode during startup

[ https://issues.apache.org/jira/browse/HDFS-1767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1767: Attachment: trunkLocalNameImage9.patch Re-submitting to trigger auto testing. Namenode should

[jira] [Updated] (HDFS-1767) Namenode should ignore non-initial block reports from datanodes when in safemode during startup

[ https://issues.apache.org/jira/browse/HDFS-1767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1767: Attachment: (was: trunkLocalNameImage9.patch) Namenode should ignore non-initial block reports

[jira] [Updated] (HDFS-1070) Speedup NameNode image loading and saving by storing local file names

[ https://issues.apache.org/jira/browse/HDFS-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1070: Attachment: trunkLocalNameImage9.patch re-submitting to trigger hudson. Speedup NameNode image

[jira] [Updated] (HDFS-1070) Speedup NameNode image loading and saving by storing local file names

[ https://issues.apache.org/jira/browse/HDFS-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1070: Resolution: Fixed Release Note: This changes the fsimage format to be root directory-1

[jira] [Commented] (HDFS-1070) Speedup NameNode image loading and saving by storing local file names

[ https://issues.apache.org/jira/browse/HDFS-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13019617#comment-13019617 ] Hairong Kuang commented on HDFS-1070: - The failed tests seem not be caused by this

[jira] [Updated] (HDFS-1172) Blocks in newly completed files are considered under-replicated too quickly

[ https://issues.apache.org/jira/browse/HDFS-1172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1172: Fix Version/s: 0.23.0 Status: Open (was: Patch Available) Blocks in newly completed

[jira] [Updated] (HDFS-1172) Blocks in newly completed files are considered under-replicated too quickly

[ https://issues.apache.org/jira/browse/HDFS-1172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1172: Status: Patch Available (was: Open) Blocks in newly completed files are considered

[jira] [Commented] (HDFS-1070) Speedup NameNode image loading and saving by storing local file names

[ https://issues.apache.org/jira/browse/HDFS-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13019680#comment-13019680 ] Hairong Kuang commented on HDFS-1070: - Patch 9 fixed the tests failed in build 347. The

[jira] [Updated] (HDFS-1070) Speedup NameNode image loading and saving by storing local file names

[ https://issues.apache.org/jira/browse/HDFS-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1070: Fix Version/s: 0.23.0 Status: Open (was: Patch Available) Speedup NameNode image

[jira] [Updated] (HDFS-1070) Speedup NameNode image loading and saving by storing local file names

[ https://issues.apache.org/jira/browse/HDFS-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1070: Status: Patch Available (was: Open) Speedup NameNode image loading and saving by storing local

[jira] [Commented] (HDFS-1658) A less expensive way to figure out directory size

[ https://issues.apache.org/jira/browse/HDFS-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13019687#comment-13019687 ] Hairong Kuang commented on HDFS-1658: - I want to discuss if we could pursue option #1.

[jira] [Updated] (HDFS-1070) Speedup NameNode image loading and saving by storing local file names

[ https://issues.apache.org/jira/browse/HDFS-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1070: Attachment: trunkLocalNameImage9.patch patch 9 fixed a bug that write full path of an empty

[jira] [Updated] (HDFS-1070) Speedup NameNode image loading and saving by storing local file names

[ https://issues.apache.org/jira/browse/HDFS-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1070: Hadoop Flags: [Reviewed] Status: Patch Available (was: Open) Speedup NameNode image

[jira] [Updated] (HDFS-1070) Speedup NameNode image loading and saving by storing local file names

[ https://issues.apache.org/jira/browse/HDFS-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1070: Status: Open (was: Patch Available) Speedup NameNode image loading and saving by storing local

[jira] [Updated] (HDFS-1070) Speedup NameNode image loading and saving by storing local file names

[ https://issues.apache.org/jira/browse/HDFS-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1070: Attachment: (was: trunkLocalNameImage9.patch) Speedup NameNode image loading and saving by

[jira] [Updated] (HDFS-1070) Speedup NameNode image loading and saving by storing local file names

[ https://issues.apache.org/jira/browse/HDFS-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1070: Attachment: trunkLocalNameImage9.patch Speedup NameNode image loading and saving by storing local

[jira] [Commented] (HDFS-1630) Checksum fsedits

[ https://issues.apache.org/jira/browse/HDFS-1630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13018424#comment-13018424 ] Hairong Kuang commented on HDFS-1630: - Why would checksum ever be null? When reading

[jira] [Updated] (HDFS-1630) Checksum fsedits

[ https://issues.apache.org/jira/browse/HDFS-1630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1630: Attachment: editsChecksum2.patch I updated the exception message that addresses the review comment.

[jira] [Updated] (HDFS-1630) Checksum fsedits

[ https://issues.apache.org/jira/browse/HDFS-1630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1630: Resolution: Fixed Fix Version/s: 0.23.0 Hadoop Flags: [Reviewed] Status:

[jira] [Updated] (HDFS-1070) Speedup NameNode image loading and saving by storing local file names

[ https://issues.apache.org/jira/browse/HDFS-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1070: Attachment: trunkLocalNameImage8.patch This patch fixes the failed oiv related unit tests.

[jira] [Updated] (HDFS-1070) Speedup NameNode image loading and saving by storing local file names

[ https://issues.apache.org/jira/browse/HDFS-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1070: Status: Open (was: Patch Available) Speedup NameNode image loading and saving by storing local

[jira] [Updated] (HDFS-1070) Speedup NameNode image loading and saving by storing local file names

[ https://issues.apache.org/jira/browse/HDFS-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1070: Status: Patch Available (was: Open) Speedup NameNode image loading and saving by storing local

[jira] [Created] (HDFS-1826) NameNode should save image to name directories in parallel during upgrade

NameNode should save image to name directories in parallel during upgrade - Key: HDFS-1826 URL: https://issues.apache.org/jira/browse/HDFS-1826 Project: Hadoop HDFS

[jira] [Updated] (HDFS-1070) Speedup NameNode image loading and saving by storing local file names

[ https://issues.apache.org/jira/browse/HDFS-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1070: Attachment: trunkLocalNameImage7.patch Thank Matt for reviewing the patch and performing the test.

[jira] [Updated] (HDFS-1630) Checksum fsedits

[ https://issues.apache.org/jira/browse/HDFS-1630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1630: Status: Open (was: Patch Available) Checksum fsedits Key:

[jira] [Updated] (HDFS-1630) Checksum fsedits

[ https://issues.apache.org/jira/browse/HDFS-1630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1630: Status: Patch Available (was: Open) Checksum fsedits Key:

[jira] [Updated] (HDFS-1630) Checksum fsedits

[ https://issues.apache.org/jira/browse/HDFS-1630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1630: Attachment: editsChecksum1.patch This fixed the failed test. Checksum fsedits

[jira] [Updated] (HDFS-1070) Speedup NameNode image loading and saving by storing local file names

[ https://issues.apache.org/jira/browse/HDFS-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1070: Attachment: trunkLocalNameImage7.patch Speedup NameNode image loading and saving by storing local

[jira] [Updated] (HDFS-1070) Speedup NameNode image loading and saving by storing local file names

[ https://issues.apache.org/jira/browse/HDFS-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1070: Status: Open (was: Patch Available) Speedup NameNode image loading and saving by storing local

[jira] [Updated] (HDFS-1070) Speedup NameNode image loading and saving by storing local file names

[ https://issues.apache.org/jira/browse/HDFS-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1070: Status: Patch Available (was: Open) Speedup NameNode image loading and saving by storing local

[jira] [Updated] (HDFS-1070) Speedup NameNode image loading and saving by storing local file names

[ https://issues.apache.org/jira/browse/HDFS-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1070: Attachment: (was: trunkLocalNameImage7.patch) Speedup NameNode image loading and saving by

[jira] [Updated] (HDFS-1070) Speedup NameNode image loading and saving by storing local file names

[ https://issues.apache.org/jira/browse/HDFS-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1070: Attachment: (was: trunkLocalNameImage7.patch) Speedup NameNode image loading and saving by

[jira] [Updated] (HDFS-1070) Speedup NameNode image loading and saving by storing local file names

[ https://issues.apache.org/jira/browse/HDFS-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1070: Attachment: trunkLocalNameImage7.patch Speedup NameNode image loading and saving by storing local

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13017598#comment-13017598 ] Hairong Kuang commented on HDFS-744: @Linyin, I am ok if we simply sync every packet to

[jira] [Updated] (HDFS-1630) Checksum fsedits

[ https://issues.apache.org/jira/browse/HDFS-1630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1630: Status: Patch Available (was: Open) Checksum fsedits Key:

[jira] [Updated] (HDFS-1070) Speedup NameNode image loading and saving by storing local file names

[ https://issues.apache.org/jira/browse/HDFS-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1070: Attachment: trunkLocalNameImage6.patch Here is a patch that store image in the modified format

[jira] [Commented] (HDFS-1800) Extend image checksumming to function with multiple fsimage files

[ https://issues.apache.org/jira/browse/HDFS-1800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13015086#comment-13015086 ] Hairong Kuang commented on HDFS-1800: - Could we put checksum as part of the image file

[jira] [Commented] (HDFS-1070) Speedup NameNode image loading and saving by storing local file names

[ https://issues.apache.org/jira/browse/HDFS-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13014862#comment-13014862 ] Hairong Kuang commented on HDFS-1070: - I agree that the old design has the potential

[jira] [Updated] (HDFS-1172) Blocks in newly completed files are considered under-replicated too quickly

[ https://issues.apache.org/jira/browse/HDFS-1172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1172: Attachment: replicateBlocksFUC.patch An initial patch for review. Will add a unit test and do more

[jira] [Updated] (HDFS-1172) Blocks in newly completed files are considered under-replicated too quickly

[ https://issues.apache.org/jira/browse/HDFS-1172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1172: Attachment: replicateBlocksFUC1.patch This patch 1. makes sure that blocks in a newly-closed file

[jira] [Commented] (HDFS-1172) Blocks in newly completed files are considered under-replicated too quickly

[ https://issues.apache.org/jira/browse/HDFS-1172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13013750#comment-13013750 ] Hairong Kuang commented on HDFS-1172: - Thank Matt for pointing out the additional

[jira] [Updated] (HDFS-1070) Speedup NameNode image loading and saving by storing local file names

[ https://issues.apache.org/jira/browse/HDFS-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1070: Status: Patch Available (was: Open) Speedup NameNode image loading and saving by storing local

[jira] [Commented] (HDFS-1780) reduce need to rewrite fsimage on statrtup

[ https://issues.apache.org/jira/browse/HDFS-1780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012712#comment-13012712 ] Hairong Kuang commented on HDFS-1780: - In our largest cluster with 22G uncompressed

[jira] [Commented] (HDFS-1767) Namenode should ignore non-initial block reports from datanodes when in safemode during startup

[ https://issues.apache.org/jira/browse/HDFS-1767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012742#comment-13012742 ] Hairong Kuang commented on HDFS-1767: - Matt, the committed patch cuts the non-initial

[jira] [Created] (HDFS-1796) Switch NameNode to use non-fair locks

Switch NameNode to use non-fair locks - Key: HDFS-1796 URL: https://issues.apache.org/jira/browse/HDFS-1796 Project: Hadoop HDFS Issue Type: Improvement Reporter: Hairong Kuang According to

[jira] [Updated] (HDFS-1796) Switch NameNode to use non-fair locks

[ https://issues.apache.org/jira/browse/HDFS-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1796: Attachment: non-fair-lock.patch This is a very straightforward patch. So no unit test is written.

[jira] [Updated] (HDFS-1541) Not marking datanodes dead When namenode in safemode

[ https://issues.apache.org/jira/browse/HDFS-1541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1541: Resolution: Fixed Hadoop Flags: [Reviewed] Status: Resolved (was: Patch Available)

[jira] [Commented] (HDFS-1172) Blocks in newly completed files are considered under-replicated too quickly

[ https://issues.apache.org/jira/browse/HDFS-1172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13009403#comment-13009403 ] Hairong Kuang commented on HDFS-1172: - I worked on a similar solution for our internal

[jira] [Commented] (HDFS-1172) Blocks in newly completed files are considered under-replicated too quickly

[ https://issues.apache.org/jira/browse/HDFS-1172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13009404#comment-13009404 ] Hairong Kuang commented on HDFS-1172: - A block under construction keeps track of the

[jira] Updated: (HDFS-1767) Delay second Block Reports until after cluster finishes startup, to improve startup times

[ https://issues.apache.org/jira/browse/HDFS-1767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1767: Description: Consider a large cluster that takes 40 minutes to start up. The datanodes compete to

[jira] Commented: (HDFS-1767) Delay second Block Reports until after cluster finishes startup, to improve startup times

[ https://issues.apache.org/jira/browse/HDFS-1767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13008523#comment-13008523 ] Hairong Kuang commented on HDFS-1767: - Great data and findings! This will definitely

[jira] Created: (HDFS-1764) add 'Time Since Delared Dead' to namenode dead data nodes web page

add 'Time Since Delared Dead' to namenode dead data nodes web page -- Key: HDFS-1764 URL: https://issues.apache.org/jira/browse/HDFS-1764 Project: Hadoop HDFS Issue Type:

[jira] Updated: (HDFS-1764) Add 'Time Since Delared Dead' to namenode dead datanodes web page

[ https://issues.apache.org/jira/browse/HDFS-1764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1764: Summary: Add 'Time Since Delared Dead' to namenode dead datanodes web page (was: add 'Time Since

[jira] Updated: (HDFS-1764) Add 'Time Since Delared Dead' to namenode dead datanodes web page

[ https://issues.apache.org/jira/browse/HDFS-1764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1764: Attachment: deadnodesUI.patch This is the patch for review. 1. Move startTime from DecommingStatus

[jira] Created: (HDFS-1765) Block Replication should respect under-replication block priority

Block Replication should respect under-replication block priority - Key: HDFS-1765 URL: https://issues.apache.org/jira/browse/HDFS-1765 Project: Hadoop HDFS Issue Type:

[jira] Created: (HDFS-1766) Datanode is marked dead, but datanode process is alive and verifying blocks

Datanode is marked dead, but datanode process is alive and verifying blocks --- Key: HDFS-1766 URL: https://issues.apache.org/jira/browse/HDFS-1766 Project: Hadoop HDFS

[jira] Updated: (HDFS-1766) Datanode is marked dead, but datanode process is alive and verifying blocks

[ https://issues.apache.org/jira/browse/HDFS-1766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1766: Attachment: killDN.patch This patch makes sure that all other threads are down when 1. DN

[jira] Commented: (HDFS-1541) Not marking datanodes dead When namenode in safemode

[ https://issues.apache.org/jira/browse/HDFS-1541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13008211#comment-13008211 ] Hairong Kuang commented on HDFS-1541: - When replication queue starts to populate (if we

[jira] Commented: (HDFS-1766) Datanode is marked dead, but datanode process is alive and verifying blocks

[ https://issues.apache.org/jira/browse/HDFS-1766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13008213#comment-13008213 ] Hairong Kuang commented on HDFS-1766: - Todd, you meant the part of code in

[jira] Updated: (HDFS-1541) Not marking datanodes dead When namenode in safemode

[ https://issues.apache.org/jira/browse/HDFS-1541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-1541: Attachment: deadnodescheck1.patch Not marking datanodes dead When namenode in safemode

[jira] Commented: (HDFS-1675) Transfer RBW between datanodes

[ https://issues.apache.org/jira/browse/HDFS-1675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13004307#comment-13004307 ] Hairong Kuang commented on HDFS-1675: - Hi Nicolas, sorry for my delay at reviewing this

[jira] Commented: (HDFS-1675) Transfer RBW between datanodes

[ https://issues.apache.org/jira/browse/HDFS-1675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13004377#comment-13004377 ] Hairong Kuang commented on HDFS-1675: - I feel that it is better to put RBW in tmp first

[jira] Commented: (HDFS-1658) A less expensive way to figure out directory size

[ https://issues.apache.org/jira/browse/HDFS-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1399#comment-1399 ] Hairong Kuang commented on HDFS-1658: - Adding parameter depth works. But still an

[jira] Created: (HDFS-1658) A less expensive way to figure out directory size

A less expensive way to figure out directory size - Key: HDFS-1658 URL: https://issues.apache.org/jira/browse/HDFS-1658 Project: Hadoop HDFS Issue Type: Improvement Reporter:

[jira] Commented: (HDFS-1658) A less expensive way to figure out directory size

[ https://issues.apache.org/jira/browse/HDFS-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12999665#comment-12999665 ] Hairong Kuang commented on HDFS-1658: - FileSystem.getContentSummary seems to traverse

  1   2   3   4   5   6   7   8   9   10   >