houqp commented on pull request #10917: URL: https://github.com/apache/airflow/pull/10917#issuecomment-723802570
@ashb no worry, I wasn't aware `context['exception']` is being widely used since this feature is currently half broken. I will refactor the code to exchange crash exception info through temp local file. But I would prefer to not use pickle and just dump the text since I think p @turbaszek the reason we can't use traceback module here is because failure call site is now in an external monitoring process, which is an entirely different process compared to the one that runs the actual task. As a result, caller won't have access to the exception object in its memory. @kaxil @ashb now that we won't be introduce a breaking change, I think we don't need to drag beta1 release along for this fix, we can include it in beta2 if needed. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
