jscheffl commented on PR #46042: URL: https://github.com/apache/airflow/pull/46042#issuecomment-2614555785
Just adding comments as copy of the 1:1 conversation: I doubt that pushing an XCom from an external system is a good way to signal data to an existing workflow. Technically it would be working but this opens the door to "hacky integrations" in my view. XCom (as far as I have seen it) was made for inter-task communication and not as an interface to external applications. I don't know if there is a specific design pattern that need to be considered. Just my thought. I would understand if the API can be used to "patch" wrong/incorrect XCom values administratively for specific szenarios where upstream tasks can not be re-executed but values are incorrect. But the PR of this API is only adding XCom, will raise an error if the value is existing. So before merging I'd propose to have a bit of discussion between the other maintainers if we want to open XCom up to external applications to signal a processing status. If so then this integration pattern would require a bit of documentation as well. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
