Investigate .META. slowdowns when more than 1 store files
---------------------------------------------------------
Key: HBASE-2175
URL: https://issues.apache.org/jira/browse/HBASE-2175
Project: Hadoop HBase
Issue Type: Bug
Reporter: Jean-Daniel Cryans
Fix For: 0.21.0
I'm currently testing Hadoop 0.21 with HBase trunk + HBASE-2066 by importing
our main data set. After some time, probably because of log rolls which force
flushes and a cluster restart, the .META. region begins to accumulate store
files. I'm refreshing the master web UI a lot to see our insert speed and saw
that 1) it was getting slower to refresh and 2) the import speed went down at
the same time.
Having already seen something like that previously with 0.20, I forced a major
compaction on .META. and immediately the refresh speed got 10 times better and
the import throughput went 2x (tasks went from 20 min to 10 min).
Why is scanning and doing random reads from the client that slow when .META.
has more than 1 store file? If it's a more fondamental speed issue, could we at
least force major compactions on .META. when it grows so that the rest of the
cluster doesn't get super slow? By the way, that operation takes less than 1
second since that region is so small.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
