wangshengjie123 commented on pull request #34834: URL: https://github.com/apache/spark/pull/34834#issuecomment-990535344
> Do you have more details on why this is happening ? Did all the initial 3 task failures happen for similar reason ? > > We seem to be trying to work around very undeterministic/flakey behavior here (task attempt 0-2 all failed, and task attempt 3 worked while 4 failed again). > > If the application is extremely flakey, other suitable config changes might help ? (increased memory, increased task re-try, etc). Thanks for reviewing this PR, @mridulm , I agree that the application is extremely flakey, some tasks oom 3 times, increase memory is helpful. Here some picture and log below:   Shall we just fix this logic leak for this undeterministic behavior? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
