Thanks for your response. My case is complicated. It is not the issue of xceivers. The main reason of asynchronously I/O, When we write some data to an "async" mounted partition(that is default in os), such as "create", "mkdir", "rmdir", "rename" and "write". These operations need a period time to sync. If one machine crashes, will may have some files or some blocks lose. If setting "sync" affects all I/O to the file system (both meta data and file contents), performance is low. I agree with Harsh, If sets "dfs.replication.min" to 2. It may cause RSes to go down. So I want to know how set the " dfs.replication.min" in production. Who has experience, Whether by increasing the value to increase reliability?
Cheers. -----邮件原件----- 发件人: Jahangir Mohammed [mailto:[email protected]] 发送时间: 2011年11月26日 15:22 收件人: [email protected] 主题: Re: A question about dfs.replication.min setting. Since you say you see blocks loosing when it's IO busy/bound, I very much that think the xceivers has been set to low value. Raise its value. Thanks, Jahangir Mohammed. On Sat, Nov 26, 2011 at 2:21 AM, Jahangir Mohammed <[email protected]>wrote: > What is dfs.datanode.max.xceivers set? > > > > On Sat, Nov 26, 2011 at 2:17 AM, Harsh J <[email protected]> wrote: > >> Ah wait, my bad. Do not raise dfs.replication.min when using HBase - it >> can cause RSes to go down if min block #s > 1 were not completely >> guaranteed. As a result, close() on files fail to work and block out until >> the replicas are available to satisfy dfs.replication.min - and thereby >> cause things to timeout/fail. >> >> (Think the problem is to do with use of sync, but am not sure yet -- >> general writes work properly with that config, by retrying enough times to >> get locations). >> >> On 26-Nov-2011, at 12:14 PM, Gaojinchao wrote: >> >> > When HBase use HDFS system file. How do we set "dfs.replication.min"? >> > who can share relevant experience? >> > Currently on our environment, We use the default value: >> > dfs.replication :3 >> > dfs.replication.min: 1 >> > I found some block lost when the IO is very busy. >> > >> > >> > >> > >> >> >
