Hi, I am not very familiar with flume 0.9, but I am with 1.x. It sounds like you won't want to write to a file?
In Flume 1.x you can do this using say AvroSource -> MemoryChannel -> HBase Sink (or AsyncHBase sink). You'd then use the Flume client or log4j appender to send events to the AvroSource. Brock On Thu, Sep 6, 2012 at 4:49 PM, Babu, Prashanth <[email protected]> wrote: > Hi, > > Is it possible to use Flume to write to HBase thru InMemory? > > I have a steady ingested data of 40,000 events / second. > My requirement is to write to HBase thru Flume and all this in memory. > Basically I am aware that we can write to a log file which will be tailed by > Flume and the data is dumped into HBase. > In our requirement, instead of this, we want to directly write to HBase [and > if not possible, at least to HDFS]. > > I had a look at MemoryChannel, but it seems its very limited. > Might not suffice our requirement. > > Can someone please shed some light on this? > Did anyone try something like this before? > > TIA. > > My Environment: > Hadoop - 0.20.203 > Flume - 0.90.4 > HBase - 0.9.4 > > Regards, > Prashanth Babu. > > ______________________________________________________________________ > Disclaimer:This email and any attachments are sent in strictest confidence > for the sole use of the addressee and may contain legally privileged, > confidential, and proprietary data. If you are not the intended recipient, > please advise the sender by replying promptly to this email and then delete > and destroy this email and any attachments without any further use, copying > or forwarding -- Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
