Looks more like a config issue, can you verify the memory channel transaction capacity ? By default it is 100, and the error reports the same too.
Thanks, Rufus On Wed, Jun 17, 2015 at 2:55 PM, Quintana, Cesar (C) < [email protected]> wrote: > Hello, > > > > I have a Memory Channel and a File Sink to a local disk configured to take > events. I have 1525 transactions that have been duplicated 583 times in a > single hour. It may be more, but my file rolls every hour, so I only looked > at that one. I do know the problem continues happening until I restart > flume, whether it is the same set of events being duplicated or a different > one. > > > > I can’t find any documentation on what “Take list for MemoryTransaction, > capacity 100 full” means. > > Network issues aren’t playing a role here, since it’s a local disk. > > The Channel Capacity was recently increased from 1,000 to 10,000, but that > did nothing, and seeing how the real problem are these duplicates, I can > see why that had no tangible effect. > > Eventually, the Channel hits 10,000 events and stops accepting data from > the Source. > > Resources are nowhere close to being strained. > > It looks like I’m getting around 30 to 50 events per second when this > occurs. > > Help, please! L > > > > Memory Channel Capacity: 10,000 > > Memory Channel Transaction Capacity: 1,000 > > > > Rolling File Sink Batch Size: 1,000 > > > > The below error is the first error that is logged when this starts > happening. > > > > 16 Jun 2015 13:56:24,279 ERROR > [SinkRunner-PollingRunner-DefaultSinkProcessor] > (org.apache.flume.SinkRunner$PollingRunner.run:160) - Unable to deliver > event. Exception follows. > > org.apache.flume.EventDeliveryException: Failed to process transaction > > at > org.apache.flume.sink.RollingFileSink.process(RollingFileSink.java:218) > > at com.dynatrace.diagnostics.flume.RollingFileSink.process(Unknown > Source) > > at > org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68) > > at > org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147) > > at java.lang.Thread.run(Thread.java:701) > > Caused by: org.apache.flume.ChannelException: Take list for > MemoryTransaction, capacity 100 full, consider committing more frequently, > increasing capacity, or increasing thread count > > at > org.apache.flume.channel.MemoryChannel$MemoryTransaction.doTake(MemoryChannel.java:100) > > at > org.apache.flume.channel.BasicTransactionSemantics.take(BasicTransactionSemantics.java:113) > > at > org.apache.flume.channel.BasicChannelSemantics.take(BasicChannelSemantics.java:95) > > at > org.apache.flume.sink.RollingFileSink.process(RollingFileSink.java:191) > > > > Cesar Quintana >
