Rohini Palaniswamy created TEZ-3242:
---------------------------------------
Summary: Reduce bytearray copy with TezEvent Serialization and
deserialization
Key: TEZ-3242
URL: https://issues.apache.org/jira/browse/TEZ-3242
Project: Apache Tez
Issue Type: Improvement
Reporter: Rohini Palaniswamy
Assignee: Rohini Palaniswamy
Byte arrays are created for serializing protobuf messages and parsing them
which creates lot of garbage when we have lot of events.
{code}
java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:3236)
at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:118)
at
java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:153)
at java.io.DataOutputStream.write(DataOutputStream.java:107)
at java.io.FilterOutputStream.write(FilterOutputStream.java:97)
at
org.apache.tez.runtime.api.impl.TezEvent.serializeEvent(TezEvent.java:197)
at org.apache.tez.runtime.api.impl.TezEvent.write(TezEvent.java:268)
at
org.apache.tez.runtime.api.impl.TezHeartbeatResponse.write(TezHeartbeatResponse.java:95)
at
org.apache.hadoop.io.ObjectWritable.writeObject(ObjectWritable.java:202)
at
org.apache.hadoop.io.ObjectWritable.writeObject(ObjectWritable.java:128)
at org.apache.hadoop.io.ObjectWritable.write(ObjectWritable.java:82)
at org.apache.hadoop.ipc.Server.setupResponse(Server.java:2496)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
