potiuk commented on PR #38531: URL: https://github.com/apache/airflow/pull/38531#issuecomment-2063711770
> @potiuk , Thank you for helping with this PR. We see that at this point modifying Airflow core has really high cost and the data migration brings additional risks for customers. If we come back to the original issue, as you mentioned, the problem is failing a test for a particular operator. And apparently, it can be solved without core modification, because the PythonVirtualenvOperator's logic (virtual environment preparation) has nothing to do with Airflow models serialization. Therefore I updated this PR and left only changes in the operator level. So the Airflow core remains unchanged. It should resolve the initial problem. Please take a look. And thank you once again! Yes, sorry for the confusion - I think indeed we should limit that change to only that - all the complexity that replacing core executor config might still remain. It might not help with some dependency issues (I.e. dill will still be a core depenency) - but it will give the users a way to handle their pickling for PVO / External Python Operator better. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
