houqp commented on a change in pull request #10917:
URL: https://github.com/apache/airflow/pull/10917#discussion_r524942318
##########
File path: airflow/models/taskinstance.py
##########
@@ -106,6 +108,29 @@ def set_current_context(context: Context):
)
+def load_error_file(fd: IO[bytes]) -> Optional[Union[str, Exception]]:
+ """Load and return error from error file"""
+ fd.seek(0, 0)
+ data = fd.read()
+ if not data:
+ return None
+ try:
+ return pickle.loads(data)
+ except Exception: # pylint: disable=broad-except
+ return "Failed to load task run error"
+
+
+def set_error_file(error_file: str, error: Union[str, Exception]) -> None:
+ """Write error into error file by path"""
+ with open(error_file, "wb") as fd:
+ try:
+ pickle.dump(error, fd)
+ except Exception: # pylint: disable=broad-except
Review comment:
@ashb minor change in behavior here. if user raises an exception from
locally scoped class definition during task execution, we won't be able to
pickle it, so we will have to serialize it as string instead. however, this
should not be a breaking change since `context['exception']` has always been
carrying a union type of `str` and `Exception`, so user code should not be
expecting it to always be an `Exception` anyway.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
