---------- Forwarded message ---------- From: air <[email protected]> Date: 2011/8/11 Subject: Re: hadoop submit job very slow To: CDH Users <[email protected]>
when I execute : hadoop fsck / , it shows Total size: 2246829645313 B (Total open files size: 1144403881 B) Total dirs: 22494 Total files: 47853 (Files currently being written: 7) Total blocks (validated): 58416 (avg. block size 38462572 B) (Total open file blocks (not validated): 22) Minimally replicated blocks: 58416 (100.0 %) Over-replicated blocks: 254 (0.43481237 %) Under-replicated blocks: 875 (1.4978772 %) Mis-replicated blocks: 0 (0.0 %) Default replication factor: 3 Average block replication: 2.6108258 Corrupt blocks: 0 *Missing replicas: 1760 (1.1539924 %)* Number of data-nodes: 8 Number of racks: 1 FSCK ended at Thu Aug 11 17:44:39 CST 2011 in 1877 milliseconds 2011/8/11 air <[email protected]> > I am using CDH3 update1 hadoop 0.20.2, the cluster is composed of 9 > nodes(1NN + 8 (DN and TT)) , I found that it's very slow to submit job to > the jobtracker (using hive or hadoop jar command) when there are dozens of > jobs are running on the cluster, when the cluster is idle, submitting job to > the cluster is fast. > > > but when I use CDH3 Beta2, there was no such problems , is there some > change in CDH3 update1 that cause the problem ? > > > thank you for your help. > > -- > Knowledge Mangement . > > -- Knowledge Mangement . -- Knowledge Mangement .
