Well, like I mentioned, I'm still new to Flume so maybe I'm totally of course here, but how about an implementation similar to HBaseSink ? replace the list of Puts with a list of Strings of insert statements ? execute them in batches ?
On Thu, Oct 10, 2013 at 7:32 PM, Steve Morin <[email protected]> wrote: > I would be interested in knowing your use-case too > > > On Thu, Oct 10, 2013 at 8:47 AM, Jarek Jarcec Cecho <[email protected]>wrote: > >> Hi Amit, >> to my best knowledge, there is no MySQL sink that would store data from >> Flume in MySQL. Would you mind describing your use case? I would be >> interesting to hear how the MySQL fits into the use case. Flume is heavy >> distributed system for collecting and injesting data, that can very easily >> overload single MySQL server, so I'm wondering if you are planning to have >> some sort of sharding or perhaps a mysql cluster to sustain the load? >> >> Jarcec >> >> On Thu, Oct 10, 2013 at 09:10:34AM +0200, Amit Sela wrote: >> > Hi all, >> > >> > I'm new to Flume and I would like to know if anyone used it to aggregate >> > logs and process them into MySQL (JDBC) inserts ? >> > >> > Thanks. >> > >
