MapReduce Application can be configured to notify the status on completion through mapreduce.job.end-notification.url. Need to write a webservice to collect the status and send email to users. Below has examples
https://hadoopi.wordpress.com/2013/09/18/hadoop-get-a-callback-on-mapreduce-job-completion/ https://amalgjose.com/2015/07/21/notification-on-completion-of-mapreduce-jobs/ Other applications Spark, TEZ does not have similar way. YARN also does not have a generic way to notify. Will suggest you to write a service which checks job status periodically using YARN REST Api and notify users with status and application logs. The interval should be higher else it will overload YARN. https://community.cloudera.com/t5/Support-Questions/Hi-I-have-a-query-in-Spark-How-to-set-up-email-notification/td-p/117924 On Thu, Dec 5, 2019 at 10:07 AM Manuel Sopena Ballesteros < manuel...@garvan.org.au> wrote: > Dear Hadoop community, > > > > I have a Hadoop cluster with yarn, spark and hdfs. Users don’t have access > to the yarn web interface and I would like to as what options I have to > notify users when a job fails and the error message? > > > > Is there anything I can setup for that? > > > > Thank you very much > > > > Manuel Sopena Ballesteros > > Big Data Engineer | Kinghorn Centre for Clinical Genomics > > [image: cid:image001.png@01D4C835.ED3C2230] <https://www.garvan.org.au/> > > > *a:* 384 Victoria Street, Darlinghurst NSW 2010 > *p:* +61 2 9355 5760 | +61 4 12 123 123 > *e:* manuel...@garvan.org.au > > Like us on Facebook <http://www.facebook.com/garvaninstitute> | Follow us > on Twitter <http://twitter.com/GarvanInstitute> and LinkedIn > <http://www.linkedin.com/company/garvan-institute-of-medical-research> > > > NOTICE > Please consider the environment before printing this email. This message > and any attachments are intended for the addressee named and may contain > legally privileged/confidential/copyright information. If you are not the > intended recipient, you should not read, use, disclose, copy or distribute > this communication. If you have received this message in error please > notify us at once by return email and then delete both messages. We accept > no liability for the distribution of viruses or similar in electronic > communications. This notice should not be removed. >
