uranusjr commented on issue #46580: URL: https://github.com/apache/airflow/issues/46580#issuecomment-2687010299
When Airflow pushes an XCom for task expansion, the worker that creates the upstream XCom calculates how long the list is (in this case `[3, 6, 9]` so it’s 3), and write that number into a separate table TaskMap This allows the scheduler to simply reads that table when it needs to expand the task into TIs. The scheduler cannot read the XCom directly because it may contain huge data and cripple the process (also deserialisation can be unsafe depending on the XCom backend). I’m guessing maybe something along this pipeline is not working correctly, and Airflow is complaining about the TaskMap record is missing. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
