Could it be because of re-try. Thanks
On Thu, 21 Nov 2019 at 3:35 AM, Jiang, Yi J (CWM-NR) <[email protected]> wrote: > Hello > > We are running into an issue. > > We have customized the SparkListener class, and added that to spark > context. But when the spark job is failed, we find the “onApplicationEnd” > function was triggered twice. > > Is that supposed to be triggered just once when the spark job is failed? > Because the application level of spark job is only launched once, how can > it be triggered twice when it is failed. > > Please let us know > > Thank you > > > > > > _______________________________________________________________________ > > This email is intended only for the use of the individual(s) to whom it is > addressed and may be privileged and confidential. > Unauthorised use or disclosure is prohibited. If you receive this e-mail > in error, please advise immediately and delete the original message. > This message may have been altered without your or our knowledge and the > sender does not accept any liability for any errors or omissions in the > message. > > Ce courriel est confidentiel et protégé. L'expéditeur ne renonce pas aux > droits et obligations qui s'y rapportent. > Toute diffusion, utilisation ou copie de ce message ou des renseignements > qu'il contient par une personne autre que le (les) destinataire(s) > désigné(s) est interdite. > Si vous recevez ce courriel par erreur, veuillez m'en aviser > immédiatement, par retour de courriel ou par un autre moyen. >
