> > > I am using netty for data transfer for a project. This part of the project > is basically sending and receiving data packets. Each packet is basically > an object that encapsulates some metadata and actual file chunks. When I > try to transfer large files, say 5GB, somehow memory usage going up and > does not free after file has been transferred. For 5gb file, memory usage > starts form ~1gb and goes up to 15GB, and stays there until I quit the > program. >
>From a quick look your code, it appears that the data you transfer is on the Java heap - i.e. loaded from wherever you're getting it into conventional Java objects. So what you're seeing is pretty normal behavior from your JVM's garbage collector - memory used typically goes up until it hits the limit (which you can set with -Xmx) and stays there. That has nothing to do with Netty - yes, you may be copying data into an off-heap buffer, but it did not start out off-heap - so 5GB of something has to pass through the Java heap on its way out the door. If you want to test that theory, simply crank down the max heap size as far as you can and still have it work, using -Xmx when you launch the process, run your tests again and observe memory usage. If it is smaller by the difference between the previous heap size and the new one, then it indeed has nothing to do with allocating ByteBufs. If the data is file chunks, and you want to avoid dragging it onto the heap in the first place, see if you can find a way to use FileRegion. If you want to see how memory is being used, JConsole can give you some info, as can -XX:+PrintGC and -XX:+PrintGCDetails - that will also answer the question of whether the allocations are on- or off-heap. -Tim -- You received this message because you are subscribed to the Google Groups "Netty discussions" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/netty/a44b8a38-f9ec-4099-909e-ade593d70fba%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
