For background information on HDFS-Hbase performance, reviewing the umbrella ticket is a good place to start...
https://issues.apache.org/jira/browse/HDFS-1599 But if I am reading this correctly... > I write 100M data in Hbase,only 1M key/value,so all keys are in only *one* > Store,and only 2 HFiles in that Store ... all your data wound up in one Store, thus it's all on one DataNode. See the Hbase book on pre-splitting tables if you haven't done that already... http://hbase.apache.org/book.html#performance On 7/30/11 7:29 AM, "李宏展" <[email protected]> wrote: >which property in configuration affects the concerrency? > > >在 2011年7月30日 下午6:21,Ryan Rawson <[email protected]>写道: > >> What is the level of concurrency? I find that HDFS gets worse >> performing with more concurrent read threads. >> >> -ryan >> >> 2011/7/30 seven garfee <[email protected]>: >> > hi,all >> > I set up a cluster on 4 machine.(1 HMaster,4RegionServer). >> > Each Machine has a 16G mem, one 2T Sata disk,CentOS 5.3,XFS . >> > I write 100M data in Hbase,only 1M key/value,so all keys are in only >> *one* >> > Store,and only 2 HFiles in that Store.(I used Large Data Set,but >>random >> read >> > performance is bad,so i try 100M) >> > Then I test random read with existing keys with only 1 client. But I >>got >> a >> > Bad Performance,About >100ms latency per read. >> > >> > Why did I get so long latency? >> > >> > I dig into the metric log,and found that, >> > in metrics_rpc.log: >> > get_avg_time=146, get_num_ops=22 >> > get_avg_time=94, get_num_ops=127 >> > get_avg_time=113, get_num_ops=182 >> > ....... >> > >> > and in metrics_hbase.log: >> > fsReadLatency_avg_time=140, fsReadLatency_num_ops=31 >> > fsReadLatency_avg_time=93, fsReadLatency_num_ops=137 >> > fsReadLatency_avg_time=109, fsReadLatency_num_ops=188 >> > ....... >> > >> > Is that say the fsReadLatency causes the random get latency so long? >> > >> > But i didn't tuning hdfs almost,except some ips/ports and >>max.xcievers, >> > Should I tuning for hdfs?And How? >> > >>
