turbaszek commented on pull request #9590:
URL: https://github.com/apache/airflow/pull/9590#issuecomment-665091094
How about using a hash of job config to create `job_id`?
`{dag_id}{task_id}{exec_date}{hash(job_config)}`
This will not solve everything but we will be able to handle:
- creation of new job
- reattach to the job which is running (with same config)
- create new job in case of rerun / backfill with new changed config
We can also consider using `try_number` somewhere. Until this change, we
were using random `job_id`, so the second point was not possible.
The uniqueness of `job_id` enforces a lot of limitation and I think user
should be aware of that and I'm not sure if we will be able to handle 100%
cases.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
