[ https://issues.apache.org/jira/browse/HDFS-3457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
J.Andreina updated HDFS-3457: ----------------------------- Description: Scenario: ========= Write an HDFS application with the following sequence of operations 1. Create file. 2. Append and sync file. 3. Delete file. 4. Create file. 5. Rename file. Run the application using 50 threads for 4 hours. Next Run the same application using 200 threads for the next 4 hours. Next Run the application using 50 threads for the next 4 hours. The Number of under-Replicated blocks mentioned in the UI and fsck report differs Fsck report for the mismatch in Number of under-Replicated blocks : =================================================================== Status: HEALTHY Total size: 5670200922 B Total dirs: 2 Total files: 2015 Total blocks (validated): 977 (avg. block size 5803685 B) Minimally replicated blocks: 977 (100.0 %) Over-replicated blocks: 0 (0.0 %) Under-replicated blocks: 94 (9.621289 %) Mis-replicated blocks: 0 (0.0 %) Default replication factor: 2 Average block replication: 1.9037871 Corrupt blocks: 0 Missing replicas: 94 (5.0537634 %) Number of data-nodes: 3 Number of racks: 1 FSCK ended at Mon Mar 19 11:14:41 IST 2012 in 94 milliseconds The filesystem under path '/' is HEALTHY Have attached UI screenshot for this issue was: Scenario: ========= Write an HDFS application with the following sequence of operations 1. Create file. 2. Append and sync file. 3. Delete file. 4. Create file. 5. Rename file. Run the application using 50 threads for 4 hours. Next Run the same application using 200 threads for the next 4 hours. Next Run the application using 50 threads for the next 4 hours. The Number of under-Replicated blocks and Total number of files mentioned in the UI and fsck report differs Fsck report for the mismatch in Number of under-Replicated blocks : =================================================================== Status: HEALTHY Total size: 5670200922 B Total dirs: 2 Total files: 2015 Total blocks (validated): 977 (avg. block size 5803685 B) Minimally replicated blocks: 977 (100.0 %) Over-replicated blocks: 0 (0.0 %) Under-replicated blocks: 94 (9.621289 %) Mis-replicated blocks: 0 (0.0 %) Default replication factor: 2 Average block replication: 1.9037871 Corrupt blocks: 0 Missing replicas: 94 (5.0537634 %) Number of data-nodes: 3 Number of racks: 1 FSCK ended at Mon Mar 19 11:14:41 IST 2012 in 94 milliseconds The filesystem under path '/' is HEALTHY Fsck report for the mismatch in Total number of Files : ======================================================= Status: HEALTHY Total size: 19418 B (Total open files size: 42729325 B) Total dirs: 2 Total files: 4226 (Files currently being written: 15) Total blocks (validated): 266 (avg. block size 73 B) (Total open file blocks (not validated): 5) Minimally replicated blocks: 266 (100.0 %) Over-replicated blocks: 0 (0.0 %) Under-replicated blocks: 0 (0.0 %) Mis-replicated blocks: 0 (0.0 %) Default replication factor: 2 Average block replication: 2.0 Corrupt blocks: 0 Missing replicas: 0 (0.0 %) Number of data-nodes: 3 Number of racks: 2 FSCK ended at Wed Apr 04 10:41:44 IST 2012 in 2302 milliseconds The filesystem under path '/' is HEALTHY Have attached UI screenshot for both the issues Summary: Number of UnderReplicated blocks displayed in UI and Fsck report is different (was: Number of UnderReplicated blocks and Number of Files in the cluster, displayed in UI and Fsck report is different) > Number of UnderReplicated blocks displayed in UI and Fsck report is different > ----------------------------------------------------------------------------- > > Key: HDFS-3457 > URL: https://issues.apache.org/jira/browse/HDFS-3457 > Project: Hadoop HDFS > Issue Type: Bug > Affects Versions: 2.0.0-alpha > Reporter: J.Andreina > Priority: Minor > Attachments: Mismatch in Number of Underreplicated blocks UI.jpg, > Mismatch in number of files UI.jpg > > > Scenario: > ========= > Write an HDFS application with the following sequence of operations > 1. Create file. > 2. Append and sync file. > 3. Delete file. > 4. Create file. > 5. Rename file. > Run the application using 50 threads for 4 hours. > Next Run the same application using 200 threads for the next 4 hours. > Next Run the application using 50 threads for the next 4 hours. > The Number of under-Replicated blocks mentioned in the UI and fsck report > differs > Fsck report for the mismatch in Number of under-Replicated blocks : > =================================================================== > Status: HEALTHY > Total size: 5670200922 B > Total dirs: 2 > Total files: 2015 > Total blocks (validated): 977 (avg. block size 5803685 B) > Minimally replicated blocks: 977 (100.0 %) > Over-replicated blocks: 0 (0.0 %) > Under-replicated blocks: 94 (9.621289 %) > Mis-replicated blocks: 0 (0.0 %) > Default replication factor: 2 > Average block replication: 1.9037871 > Corrupt blocks: 0 > Missing replicas: 94 (5.0537634 %) > Number of data-nodes: 3 > Number of racks: 1 > FSCK ended at Mon Mar 19 11:14:41 IST 2012 in 94 milliseconds > The filesystem under path '/' is HEALTHY > Have attached UI screenshot for this issue -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira