Vic, Can you do a jmap to find get some clouse about what kind of objects are leaking? Also, you can use jhat to analyze what was in the heap.
My first guess is that tail may have a slow mem leak. Jon. On Mon, Aug 8, 2011 at 11:31 AM, Huang, Zijian(Victor) < [email protected]> wrote: > Hi, Guys: > > Flume agent seem to have memory leaks. I have one setup last week, and it > was doing nothing. Today, I it almost used up about 50% of memory in the > machine, and out putting the following error message. > > =========== > Exception in thread "SpawningLogicalNode agent_load_test" > java.lang.OutOfMemoryError: Direct buffer memory > at java.nio.Bits.reserveMemory(Bits.java:656) > at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:113) > at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:305) > at > com.cloudera.flume.handlers.text.TailSource$Cursor.<init>(TailSource.java:154) > at > com.cloudera.flume.handlers.text.TailSource.<init>(TailSource.java:125) > at > com.cloudera.flume.handlers.text.TailSource$1.build(TailSource.java:630) > at > com.cloudera.flume.conf.SourceFactoryImpl.getSource(SourceFactoryImpl.java:139) > at > com.cloudera.flume.conf.FlumeBuilder.buildEventSource(FlumeBuilder.java:416) > at > com.cloudera.flume.conf.FlumeBuilder.buildSource(FlumeBuilder.java:268) > at > com.cloudera.flume.agent.LogicalNode.loadConfig(LogicalNode.java:303) > at > com.cloudera.flume.agent.LogicalNode.checkConfig(LogicalNode.java:371) > at > com.cloudera.flume.agent.LivenessManager$1.run(LivenessManager.java:141) > ============= > > Thanks > > Vic > > -- // Jonathan Hsieh (shay) // Software Engineer, Cloudera // [email protected]
