derkuci commented on pull request #20473: URL: https://github.com/apache/airflow/pull/20473#issuecomment-1002333155
> I just wonder, why do you think it should be consistent? We do not have auto-discovery of types and format of the returned values in Airflow. Airlfow is task-based not data-based, so basicaly in case you want to pass data between tasks both producer and consumer has to agree on naming conventions and data format used. There is no "universal" discovery. Also even in the case you started with - the "pickling" is deprecated and actually it should be set "per installlation". So it could be that in different installations, different format is used, but it will be "constistent" for that installation. I do not see huge value in providing "consistent" output format really in this case. Got you. Our particular case indeed involves "installation" change: upgrading from an old version of airflow to the current version. * We recently upgraded from 1.9 to 2.2... `enbale_xcom_pickling` was True with the old version, and I hesitated whether I should keep the old value, or switch to "False" as suggested (safety, etc.). * "run_as_user" is broken (at least for us) in the new version. We had to switch some `PythonOperator` to `BashOperator` (use sudo in our script) or, occasionally, `SSHOperator`. While we are experimenting with all these changes, we had to deal with uncertainties in the XCom data type/formats. My patch is not general enough (as you kindly explained) but served us well during the upgrade process. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
