Amit, That sounds like a reasonable idea. You should be careful about handling SQL related issues (like duplicate primary keys etc). You'd probably need to add a lot of the SQL related configuration and failure handling
Thanks, Hari On Saturday, October 12, 2013 at 8:48 AM, Amit Sela wrote: > Well, like I mentioned, I'm still new to Flume so maybe I'm totally of course > here, but how about an implementation similar to HBaseSink ? replace the list > of Puts with a list of Strings of insert statements ? execute them in batches > ? > > > On Thu, Oct 10, 2013 at 7:32 PM, Steve Morin <[email protected] > (mailto:[email protected])> wrote: > > I would be interested in knowing your use-case too > > > > > > On Thu, Oct 10, 2013 at 8:47 AM, Jarek Jarcec Cecho <[email protected] > > (mailto:[email protected])> wrote: > > > Hi Amit, > > > to my best knowledge, there is no MySQL sink that would store data from > > > Flume in MySQL. Would you mind describing your use case? I would be > > > interesting to hear how the MySQL fits into the use case. Flume is heavy > > > distributed system for collecting and injesting data, that can very > > > easily overload single MySQL server, so I'm wondering if you are planning > > > to have some sort of sharding or perhaps a mysql cluster to sustain the > > > load? > > > > > > Jarcec > > > > > > On Thu, Oct 10, 2013 at 09:10:34AM +0200, Amit Sela wrote: > > > > Hi all, > > > > > > > > I'm new to Flume and I would like to know if anyone used it to aggregate > > > > logs and process them into MySQL (JDBC) inserts ? > > > > > > > > Thanks. > > >
