Hi Tez devs,

The current design of 
VertexInitializedEvent<https://github.com/apache/tez/blob/3f2373e2b2ab3825ef50e9f19b8704265542a8b2/tez-dag/src/main/java/org/apache/tez/dag/history/events/VertexInitializedEvent.java>
 contains all of the InputDataInformationEvents for a particular vertex. We've 
had trouble scaling this for large vertices as Protobuf limits message sizes to 
64 MB. I'm wondering if anyone is working on a more scaleable solution, and if 
not, if you guys have any suggestions for how to decompose this event into 
smaller events so that Protobuf's message size limit is never an issue.

Thanks,
Eric

Reply via email to