Can I wear my user hat?
I have used the Region Historian on occasion as it is much easier than grepping
through master logs to find transitions. We also discard master logs after 7
days because they are large, especially when running with DEBUG from time to
time. Obviously we don't have that problem of bulk with the historian data, but
this still begs the question just how useful is such long term history? Well...
Tracking region history ties into the audit aspects of HBASE-1697.
I am also considering an experiment with collecting, correlating, and
visualizing what amounts to historian data among other things to see what kind
of things HBase can tell users about their data out of the box without imposing
too much overhead, e.g. how fast a table grows (or shrinks), region split
probabilities, key distribution trends, various information theoretic metrics.
These things could be useful for capacity planning and application tuning. It's
an interesting question to pose in general: What meaningful and useful things
can HBase tell users about their data? An aspect of this is the history of
where the data has been.
On the issue of historians in general, there are three jiras:
Region Historian, Closed, https://issues.apache.org/jira/browse/HBASE-533
Service Historian, Open, https://issues.apache.org/jira/browse/HBASE-773
Client Historian, Open, https://issues.apache.org/jira/browse/HBASE-1095
Is there any interest in these facilities and the types of operational and
capacity planning analyses they may support? Worth putting on the roadmap for
0.22?
Analysis of Service Historian data can identify sub par or failing nodes.
Accordingly they can be blacklisted.
Correlating and visualizing Client Historian data can potentially reveal a lot
about client access patterns. One could look for probabilistic motifs over
different time scales, for example. This would be useful for debugging, or to
system analysts, or for security officers.
Thoughts?
- Andy
________________________________
From: Jean-Daniel Cryans <[email protected]>
To: [email protected]
Sent: Thursday, September 17, 2009 5:38:08 PM
Subject: Are you using the Region Historian? Read this
Hi users,
The Region Historian (the page in the web UI that you get when you
click on a region name) has been in use since HBase 0.2.0 and it
caused more than its share of problems. Furthermore, we had to cripple
it in many ways to make some things work, the main issue being that
the historian is kept in .META. so operations on that catalog table
were sometimes blocked.
We are planning to disable it for 0.20.1 and 0.21.0 until we come up
with a better solution. Is anybody using it? If so, would losing the
historian be a big deal for you? Your input would be much appreciated.
Thx,
J-D
