I had it at 10 originally and the capacity at 100. Yes, I am committing the transaction and closing it (and backoff) even if the Event is null.
Thanks, Vikram From: Hari Shreedharan [mailto:[email protected]] Sent: Tuesday, March 12, 2013 11:06 AM To: [email protected]<mailto:[email protected]> Subject: Re: Dropped events Your channel capacity is set to 1. Are you sure you really want that? If a null event is returned by the channel, then you should commit and close the transaction, optionally backoff (usually, this is when a batch was empty), and then try again. You should probably also use a higher transaction capacity. Hari -- Hari Shreedharan On Tuesday, March 12, 2013 at 11:03 AM, Vikram Kulkarni wrote: I have my custom Source and Sink that I have hooked with a memory channel but I am noticing that it is not very consistent. Even after sending many events to the Source the Sink’s event is still null. It works for about 1 out 4 events. I do see the events going to the Logger Sink so I know the Source is doing its job. However, for the custom Sink I simply get Event is null messages. I tried adjusting the channel capacity from 10 to 1 but no difference. Thanks. Here’s my conf file # flume-httpxmlhttp.conf: A single-node Flume with Http Source and Http sink configuration # Name the components on this agent agent1.sources = r1 agent1.channels = c1 # Describe/configure the source agent1.sources.r1.type = org.apache.flume.source.http.HTTPSource agent1.sources.r1.port = 5140 agent1.sources.r1.handler = main.java.org.apache.flume.source.http.XMLHandler agent1.sources.r1.handler.nickname = random props # Setup the sinks agent1.sinks = httpsink logsink # Describe the sink agent1.sinks.logsink.type = logger # Describe the sink agent1.sinks.httpsink.type = main.java.org.apache.flume.sink.http.HttpPostSink agent1.sinks.httpsink.serverAddress = <my http server URL> # Use a channel which buffers events in memory agent1.channels.c1.type = memory agent1.channels.c1.capacity = 1 agent1.channels.c1.transactionCapacity = 1 # Bind the source and sink to the channel agent1.sources.r1.channels = c1 agent1.sinks.logsink.channel = c1 agent1.sinks.httpsink.channel = c1
