hkc-8010 commented on issue #65011:
URL: https://github.com/apache/airflow/issues/65011#issuecomment-4527736095
It appears that at some point the dag_run table was wiped (some sort of
truncation) without cascading to xcom. Due to this old xcom rows survived as
orphans. The Airflow DB then reused the same auto-increment IDs for new runs,
so those new runs inherited stale xcom rows from months ago.
All API lookups filter by run_id so the stale rows are invisible — but they
still hold the primary key slot, causing every new write to 409.
We have verified that there were many such rows, and after deleting them
from the database (they were unreachable already), this issue should get
resolved without any change needing to be made to the GlueJobOperator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
