Chris, I think you really can use the events via the YARN Timeline Server:
http://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/TimelineServer.html Many YARN applications (MR, Tez, Spark soon) will emit events to the Timeline Server. hth, Arun On Wed, Aug 20, 2014 at 1:48 AM, Tsuyoshi OZAWA <[email protected]> wrote: > Hi, > > How about using REST APIs to get the information instead of hacking > MapReduce itself if you're using Hadoop 2? > > http://hadoop.apache.org/docs/r2.4.1/hadoop-yarn/hadoop-yarn-site/WebServicesIntro.html > > Thanks, > - Tsuyoshi > > > On Tue, Aug 19, 2014 at 3:28 AM, Chris Hinshaw <[email protected]> > wrote: > > Goal: > > I want to create an internal event listener / bridge that can take the > > events from hadoop's event Dispatcher and forward those to an activeMQ > jms > > system. This would allow me to cleanly notify clients of their job > status. > > > > Investigations: > > > > I have dug into the Dispatcher (in MRAppMaster) and it looks like it > > dispatches the events that I wish to register and listen for. I have > looked > > at the JobHistoryHandler and have a good idea of how the events are > > processed and handled. > > > > I am looking for some advice on where to register hooks and or can I add > a > > service that will register for events with the Dispatcher. > > > > Ideally this would like this to work for a local job as well as a Yarn > > submitted job but if it only works with Yarn this would be fine for my > > cases. > > > > Any advice would be greatly appreciated on where to add these hooks or if > > they already exist. > > > > Sorry if this is the wrong forum. > > > > Thanks, > > Chris Hinshaw > > > > -- > - Tsuyoshi > -- -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
