Stuti, You'll want to:
1. Set "2" for mapred.submit.replication in mapred-site.xml to prevent the job related staging files to be created with default replication factor of 10. 2. Learn about the "-setrep" option to toggle replication factors of a path. See http://hadoop.apache.org/common/docs/current/file_system_shell.html#setrep HTH. On Mon, Oct 17, 2011 at 11:51 AM, Stuti Awasthi <stutiawas...@hcl.com> wrote: > Hi , > > I have a mini Hadoop cluster of 3 nodes with DN and TT running on each node > and replication is set to 3.By mistake I tried to write few files which > requires replication of 10. When I checked the HDFS status with FSCK utility > it shows : > > /root/Hadoop/hadoop-root/mapred/system/job_201109201112_0001/job.jar 2352841 > bytes, 1 block(s): Under replicated blk_-9220069752024407718_1546. Target > Replicas is 10 but found 3 replica(s). > 0. blk_-9220069752024407718_1546 len=2352841 repl=3 [10.33.100.74:50010, > 10.33.64.235:50010, 10.33.100.117:50010] > > /root/Hadoop/hadoop-root/mapred/system/job_201109201112_0001/libjars <dir> > /root/Hadoop/hadoop-root/mapred/system/job_201109201112_0001/libjars/hadoop-0.20.2-core.jar > 2707856 bytes, 1 block(s): Under replicated blk_-4791859336845413240_1544. > Target Replicas is 10 but found 3 replica(s). > 0. blk_-4791859336845413240_1544 len=2707856 repl=3 [10.33.100.74:50010, > 10.33.64.235:50010, 10.33.100.117:50010] > .................................................... > Status: HEALTHY > Total size: 45239704 B > Total dirs: 101 > Total files: 88 (Files currently being written: 3) > Total blocks (validated): 88 (avg. block size 514087 B) (Total open > file blocks (not validated): 3) > Minimally replicated blocks: 88 (100.0 %) > Over-replicated blocks: 0 (0.0 %) > Under-replicated blocks: 20 (22.727272 %) > Mis-replicated blocks: 0 (0.0 %) > Default replication factor: 2 > Average block replication: 2.4318182 > Corrupt blocks: 0 > Missing replicas: 140 (65.42056 %) > Number of data-nodes: 3 > Number of racks: 1 > > I wanted to know that can I fix : > 1. Under-replicated > 2. Missing replicas > > Please Suggest > Thanks > Stuti > > ::DISCLAIMER:: > ----------------------------------------------------------------------------------------------------------------------- > > The contents of this e-mail and any attachment(s) are confidential and > intended for the named recipient(s) only. > It shall not attach any liability on the originator or HCL or its affiliates. > Any views or opinions presented in > this email are solely those of the author and may not necessarily reflect the > opinions of HCL or its affiliates. > Any form of reproduction, dissemination, copying, disclosure, modification, > distribution and / or publication of > this message without the prior written consent of the author of this e-mail > is strictly prohibited. If you have > received this email in error please delete it and notify the sender > immediately. Before opening any mail and > attachments please check them for viruses and defect. > > ----------------------------------------------------------------------------------------------------------------------- > -- Harsh J
