This is how it is. It is the whole application level retry after 4 tasks attempts fail the whole application fails and then the application retry.
Thanks, Hemant On Thu, 21 Nov 2019 at 7:24 PM, Jiang, Yi J (CWM-NR) <[email protected]> wrote: > Hello, > > Thank you for replying, it is retry, but why retry is happened from whole > application level? > > To my understanding, the retry can be done in job level. > > Jacky > > > > > > *From:* hemant singh [mailto:[email protected]] > *Sent:* November 21, 19 3:12 AM > *To:* Jiang, Yi J (CWM-NR) <[email protected]> > *Cc:* Martin, Phil <[email protected]>; [email protected] > *Subject:* Re: Spark onApplicationEnd run multiple times during the > application failure > > > > Could it be because of re-try. > > > > Thanks > > > > On Thu, 21 Nov 2019 at 3:35 AM, Jiang, Yi J (CWM-NR) < > [email protected]> wrote: > > Hello > > We are running into an issue. > > We have customized the SparkListener class, and added that to spark > context. But when the spark job is failed, we find the “onApplicationEnd” > function was triggered twice. > > Is that supposed to be triggered just once when the spark job is failed? > Because the application level of spark job is only launched once, how can > it be triggered twice when it is failed. > > Please let us know > > Thank you > > > > > > _______________________________________________________________________ > > This email is intended only for the use of the individual(s) to whom it is > addressed and may be privileged and confidential. > Unauthorised use or disclosure is prohibited. If you receive this e-mail > in error, please advise immediately and delete the original message. > This message may have been altered without your or our knowledge and the > sender does not accept any liability for any errors or omissions in the > message. > > Ce courriel est confidentiel et protégé. L'expéditeur ne renonce pas aux > droits et obligations qui s'y rapportent. > Toute diffusion, utilisation ou copie de ce message ou des renseignements > qu'il contient par une personne autre que le (les) destinataire(s) > désigné(s) est interdite. > Si vous recevez ce courriel par erreur, veuillez m'en aviser > immédiatement, par retour de courriel ou par un autre moyen. > >
