takunnithan commented on issue #8494:
URL: https://github.com/apache/airflow/issues/8494#issuecomment-646824149
Ran into the same issue with REST API, We were trying to trigger a dag via
experimental API (sometimes more than 30 DAG runs at a time).
Tried setting the `replace_microseconds` flag to `False` in the API, It
doesn't solve the issue in my case. It just reduces the frequency of the issue.
I have a solution/Hack in mind,
Does it make sense to add a bool param in the experimental API, say
`handle_duplicate_execution_date`. if set as true, it will handle the duplicate
execution_date issue by increasing the `execution_date` , a few microseconds.
For eg:
id | dag_id | execution_date
-----+------------------+-------------------------------
171 | TEST_DAG | 2020-06-19 12:56:06.171041+00
if we encounter a unique constraint issue, we will simply increase the
execution time by a few microseconds.
id | dag_id | execution_date
-----+------------------+-------------------------------
171 | TEST_DAG | 2020-06-19 12:56:06.171041+00
172 | TEST_DAG | 2020-06-19 12:56:06.171051+00
This can be done by adding `try... except` block here :
https://github.com/apache/airflow/blob/05ea88869be6a62d312598b5b01095e16f0a16f8/airflow/api/common/experimental/trigger_dag.py#L86
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
