potiuk commented on PR #30308: URL: https://github.com/apache/airflow/pull/30308#issuecomment-1502906924
Ok. I updated it now and I hope it will better explain why the split and how we are going to use it when we move to AIP-44. Note that it is not **strictly** needed now (as @uranusjr noticed) for the prepare/complete methods to be able to handle JobPydantic as input (and they don't get it) - this is in preparation for the final step of refactoring in #30325 and the follow-up steps to add @internal_api calls and make it works for AIP-44 case. Adding the JobPydantic option allowed me to see (see the comment in the current version) that I need to remove job_runner from job altogether (because currently I need to use "type: ignore [union-attr] to get rid of MyPy error. I applied keyword-only session and :meta private: for the internal methods (also prefixed them with `_`) and extracted out retrieving the Job when you have just JobPydantic (which we will use when the methods will be used for AIP-44 case. Finally I added mermaid diagram showing the current Job lifecycle and how it will look like when we implement AIP-44. This might help with reasoning why we needed to separate prepare/complete and why they should also take JobPydantic as input. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
