Thanks for the info. We are wondering if using syslog to aggregate these
type of "log" files would be an a safer alternative.
On 7/26/11 8:01 PM, Michael Segel wrote:
On Tue, Jul 26, 2011 at 7:39 AM, Mark<[email protected]> wrote:
So my first question is, would HBase fit our use case? If not
can anyone offer some advice on what would/should be used?
You mean HBase as the sink for your log emitters?
The pattern I usually see is that there is intermediary, a flume or
scribe pushing the logs up into hdfs and then the log events are
hoisted up into hbase to field queries [1, 2]
I think one of the advantages in not writing directly to HBase is that when you
write to HBase you have table splits and compactions along with GC. Depending
on the rate of writes to logs, this could become problematic.
(Note YMMV)
Other than that, I'd say write directly to HBase.
