Hi Jeff, Events are a control-flow mechanism used by Tez. They are a way for user-land code to communicate with either the framework or other parts of user-land code. In addition to this, events are also used by the framework to propagate information to other parts of the framework. All events that user-land code can generate are available as part of the tez.runtime.api ( such as DataMovement or InputFailedError events ).
A DataMovement event is used by an Output to send information about the data that it has generated to a downstream Input. The event structure itself contains only a byte array as what kind of information is being passed between a pair of Output/Input is implementation specific. All that the framework does is route the event to the correct downstream task based on the type of the edge that is between the 2 vertices. There are different types of events based on what functionality/use they provide. The payloads of these events is usually a byte array/buffer which is user-code specific and not needed to be interpreted by the framework. Given this, in most scenarios, a custom event should not be needed. If you have any particular use-case for which existing events are not sufficient, lets discuss. Thanks for bringing this topic up. Our documentation on events is quite minimal ( non-existent ). thanks — Hitesh On Jun 27, 2014, at 12:28 AM, Jianfeng (Jeff) Zhang <jzh...@hortonworks.com> wrote: > Hi all, > > I notice that class Event is used by both Input, Output, Processor and > VertexManagerPlugin. > But IMHO it's not clear for users on how to use them and user don't have > full control on the events. Most of the logic is in tez-dag ( such as > whether sending it to TezChild, it need, How to serialize/ deserialize > these events ) It's hard to add customized events, although it may be a > rare requirement . > > I am thinking that in the Event class, we should provide some methods that > allow user to specify these attributes ( may need other code change ). My > purpose is to help user understand how to use these event. What's your > thinking on this ? Thanks > > > > Best Regards, > Jeff Zhang > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You.
