ashb commented on pull request #12327: URL: https://github.com/apache/airflow/pull/12327#issuecomment-726698981
> Which queries XComs and calls deserialize_value when recreating XCom object from database in init_on_load method. Definitely not something we want to do with 100s of big dataframes or other objects. Ahhh yeah, def not something we want. This wasn't clear to me from this PR. If I'm understanding the problem there are actually two issues we need to address: 1. Looking at the Xcom list page would request a bunch of files from GCS bucket (in your example) 2. Even if that was "cheap"/cached(/whatever), we don't want to display huge xcom payloads _anyway_. Have I summarised the problem correctly? How about something clearer in intent -- something like `get_xcom_summary`? From reading the PR/code it this PR it wasn't clear when this new feature would actually be used. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
