I am using netty for data transfer for a project. This part of the project
is basically sending and receiving data packets. Each packet is basically
an object that encapsulates some metadata and actual file chunks. When I
try to transfer large files, say 5GB, somehow memory usage going up and
does not free after file has been transferred. For 5gb file, memory usage
starts form ~1gb and goes up to 15GB, and stays there until I quit the
program.
This is my packet object:
public class BlockPacket {
private String requestId;
private String blockId;
private int packetSeqNo=0;
private byte[] data;
public BlockPacket(){
}
public String toString()
{
return "BlockPacket [requestId=" + requestId + ", blockId=" + blockId +
", packetSeqNo=" + packetSeqNo + "]";
}
public String getRequestId()
{
return requestId;
}
public void setRequestId(String requestId)
{
this.requestId = requestId;
}
public String getBlockId()
{
return blockId;
}
public void setBlockId(String blockId)
{
this.blockId = blockId;
}
public int getPacketSeqNo()
{
return packetSeqNo;
}
public void setPacketSeqNo(int no)
{
this.packetSeqNo = no;
}
public byte[] getData()
{
return data;
}
public void setData(byte[] data)
{
this.data = data;
}}
Here is channelRead at ServerHandler:
@Override
public void channelRead(ChannelHandlerContext ctx, Object obj)
{
if (obj instanceof BlockPacket)
{
BlockPacket packet = (BlockPacket) obj;
//Do something with packet
}else if (obj instanceof Block){
Block block = (Block) obj;
//Do something with block
}else{
//Do something else
}
}
Here is my custom custom encode and decode:
@Override
protected void encode(ChannelHandlerContext ctx, Object in, ByteBuf
out) throws Exception {
ByteArrayOutputStream outStream = new ByteArrayOutputStream();
Output output = new Output(outStream);
kryo.writeClassAndObject(output, in);
output.flush();
byte[] outArray = outStream.toByteArray();
out.writeInt(outArray.length);
out.writeBytes(outArray);
}
@Override
protected void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object>
out) throws Exception {
if (in.readableBytes() < 4)
return;
in.markReaderIndex();
boolean read = false;
if(len == 0){
read = true;
len = in.readInt();
}
if ((in.readableBytes() < len && read) || (in.readableBytes() < len+4
&& !read)) {
in.resetReaderIndex();
read = false;
return;
}
if (!read)
in.readInt();
byte[] buf = new byte[len];
in.readBytes(buf);
Input input = new Input(buf);
Object object = kryo.readClassAndObject(input);
out.add(object);
}
I went through Reference counted object
<http://netty.io/wiki/reference-counted-objects.html>,
but I could not solve my problem. Examples in this tutorial are
generally for ByteBuf, which has release() method. However, my Packet
object is a simple Java object. Any help is appreciated.
--
You received this message because you are subscribed to the Google Groups
"Netty discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/netty/ac8ba0f3-75ed-48c6-ad35-b6f757093191%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
