Since you say you see blocks loosing when it's IO busy/bound, I very much that think the xceivers has been set to low value. Raise its value.
Thanks, Jahangir Mohammed. On Sat, Nov 26, 2011 at 2:21 AM, Jahangir Mohammed <[email protected]>wrote: > What is dfs.datanode.max.xceivers set? > > > > On Sat, Nov 26, 2011 at 2:17 AM, Harsh J <[email protected]> wrote: > >> Ah wait, my bad. Do not raise dfs.replication.min when using HBase - it >> can cause RSes to go down if min block #s > 1 were not completely >> guaranteed. As a result, close() on files fail to work and block out until >> the replicas are available to satisfy dfs.replication.min - and thereby >> cause things to timeout/fail. >> >> (Think the problem is to do with use of sync, but am not sure yet -- >> general writes work properly with that config, by retrying enough times to >> get locations). >> >> On 26-Nov-2011, at 12:14 PM, Gaojinchao wrote: >> >> > When HBase use HDFS system file. How do we set "dfs.replication.min"? >> > who can share relevant experience? >> > Currently on our environment, We use the default value: >> > dfs.replication :3 >> > dfs.replication.min: 1 >> > I found some block lost when the IO is very busy. >> > >> > >> > >> > >> >> >
