Hi Isuru, I've tested the msf4j fileserver and formparam samples with the carbon-transport 2.1.1-SNAPSHOTS. The memory leak issue is now fixed.
Thanks Thusitha On Tue, Jun 14, 2016 at 2:29 PM, Thusitha Thilina Dayaratne < [email protected]> wrote: > Hi Isuru, > > Thank for the clarification. > > Best Regards > Thusitha > > On Tue, Jun 14, 2016 at 2:24 PM, Isuru Ranawaka <[email protected]> wrote: > >> Hi Thusitha, >> >> Need to release Netty ByteBuf after consuming it. Netty ByteBuf >> internally keep reference count and if it does not actually has a >> reference then GC runs and try to remove the object from heap but if >> Netty level reference count >= 1 at that time this Error will be thrown. >> we will check how to eliminate this error from transport level. >> >> thanks >> >> On Tue, Jun 14, 2016 at 12:35 PM, Thusitha Thilina Dayaratne < >> [email protected]> wrote: >> >>> Hi, >>> >>> When I'm trying to stream a large request (about 500mb file) I'm >>> getting below error. >>> >>> ERROR ResourceLeakDetector:171 - LEAK: ByteBuf.release() was not called >>>> before it's garbage-collected. See >>>> http://netty.io/wiki/reference-counted-objects.html for more >>>> information. >>> >>> >>> When I run with *-Dio.netty.leakDetectionLevel=advanced *I can see the >>> below stacktrace >>> >>>> Recent access records: 5 >>>> >>>> #5: >>>> >>>> io.netty.buffer.AdvancedLeakAwareByteBuf.nioBuffer(AdvancedLeakAwareByteBuf.java:667) >>>> >>>> org.wso2.carbon.transport.http.netty.NettyCarbonMessage.getMessageBody(NettyCarbonMessage.java:62) >>>> org.wso2.msf4j.Request.getMessageBody(Request.java:56) >>>> >>>> org.wso2.msf4j.io.MSF4JRequestInputStream.read(MSF4JRequestInputStream.java:41) >>>> java.io.InputStream.read(InputStream.java:179) >>>> >>>> org.wso2.msf4j.formparam.MultipartStream$ItemInputStream.makeAvailable(MultipartStream.java:814) >>>> >>>> org.wso2.msf4j.formparam.MultipartStream$ItemInputStream.read(MultipartStream.java:718) >>>> java.io.InputStream.read(InputStream.java:101) >>>> java.nio.file.Files.copy(Files.java:2908) >>>> java.nio.file.Files.copy(Files.java:3027) >>>> >>>> org.wso2.msf4j.example.FormService.simpleFormStreaming(FormService.java:79) >>>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>> >>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >>>> >>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>> java.lang.reflect.Method.invoke(Method.java:497) >>>> >>>> org.wso2.msf4j.internal.router.HttpMethodInfo.invoke(HttpMethodInfo.java:106) >>>> >>>> org.wso2.msf4j.internal.MSF4JMessageProcessor.dispatchMethod(MSF4JMessageProcessor.java:126) >>>> >>>> org.wso2.msf4j.internal.MSF4JMessageProcessor.receive(MSF4JMessageProcessor.java:73) >>>> >>>> org.wso2.carbon.transport.http.netty.listener.WorkerPoolDispatchingSourceHandler$1.run(WorkerPoolDispatchingSourceHandler.java:125) >>>> >>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >>>> >>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >>>> java.lang.Thread.run(Thread.java:745) >>>> #4: >>>> >>>> io.netty.buffer.AdvancedLeakAwareByteBuf.release(AdvancedLeakAwareByteBuf.java:45) >>>> >>>> io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:237) >>>> >>>> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308) >>>> >>>> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294) >>>> >>>> io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:846) >>>> >>>> io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131) >>>> >>>> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511) >>>> >>>> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) >>>> >>>> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) >>>> io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) >>>> >>>> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:110) >>>> >>>> io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137) >>>> java.lang.Thread.run(Thread.java:745) >>>> #3: >>>> >>>> io.netty.buffer.AdvancedLeakAwareByteBuf.retain(AdvancedLeakAwareByteBuf.java:709) >>>> >>>> io.netty.handler.codec.http.HttpObjectDecoder.decode(HttpObjectDecoder.java:294) >>>> >>>> io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:327) >>>> >>>> io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:230) >>>> >>>> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308) >>>> >>>> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294) >>>> >>>> io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:846) >>>> >>>> io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131) >>>> >>>> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511) >>>> >>>> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) >>>> >>>> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) >>>> io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) >>>> >>>> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:110) >>>> >>>> io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137) >>>> java.lang.Thread.run(Thread.java:745) >>>> #2: >>>> >>>> io.netty.buffer.AdvancedLeakAwareByteBuf.readSlice(AdvancedLeakAwareByteBuf.java:91) >>>> >>>> io.netty.handler.codec.http.HttpObjectDecoder.decode(HttpObjectDecoder.java:294) >>>> >>>> io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:327) >>>> >>>> io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:230) >>>> >>>> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308) >>>> >>>> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294) >>>> >>>> io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:846) >>>> >>>> io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131) >>>> >>>> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511) >>>> >>>> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) >>>> >>>> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) >>>> io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) >>>> >>>> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:110) >>>> >>>> io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137) >>>> java.lang.Thread.run(Thread.java:745) >>>> #1: >>>> >>>> io.netty.buffer.AdvancedLeakAwareByteBuf.writeBytes(AdvancedLeakAwareByteBuf.java:589) >>>> >>>> io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:242) >>>> >>>> io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:119) >>>> >>>> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511) >>>> >>>> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) >>>> >>>> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) >>>> io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) >>>> >>>> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:110) >>>> >>>> io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137) >>>> java.lang.Thread.run(Thread.java:745) >>>> Created at: >>>> >>>> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:55) >>>> >>>> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:155) >>>> >>>> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:146) >>>> >>>> io.netty.buffer.AbstractByteBufAllocator.ioBuffer(AbstractByteBufAllocator.java:107) >>>> >>>> io.netty.channel.AdaptiveRecvByteBufAllocator$HandleImpl.allocate(AdaptiveRecvByteBufAllocator.java:104) >>>> >>>> io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:117) >>>> >>>> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511) >>>> >>>> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) >>>> >>>> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) >>>> io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) >>>> >>>> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:110) >>>> >>>> io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137) >>>> java.lang.Thread.run(Thread.java:745) >>>> >>>> >>> My wrapper class for msf4j request is [1]. Did I missed something here? >>> >>> >>> [1] - >>> https://github.com/thusithathilina/msf4j/blob/master/core/src/main/java/org/wso2/msf4j/io/MSF4JRequestInputStream.java#L43 >>> >>> [2] - >>> https://github.com/wso2/carbon-transports/blob/v2.1.0/http/netty/components/org.wso2.carbon.transport.http.netty/src/main/java/org/wso2/carbon/transport/http/netty/NettyCarbonMessage.java#L62 >>> >>> Thanks >>> -- >>> Thusitha Dayaratne >>> Software Engineer >>> WSO2 Inc. - lean . enterprise . middleware | wso2.com >>> >>> Mobile +94712756809 >>> Blog alokayasoya.blogspot.com >>> About http://about.me/thusithathilina >>> >>> >> >> >> -- >> Best Regards >> Isuru Ranawaka >> M: +94714629880 >> Blog : http://isurur.blogspot.com/ >> > > > > -- > Thusitha Dayaratne > Software Engineer > WSO2 Inc. - lean . enterprise . middleware | wso2.com > > Mobile +94712756809 > Blog alokayasoya.blogspot.com > About http://about.me/thusithathilina > > -- Thusitha Dayaratne Software Engineer WSO2 Inc. - lean . enterprise . middleware | wso2.com Mobile +94712756809 Blog alokayasoya.blogspot.com About http://about.me/thusithathilina
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
