potiuk commented on issue #17897:
URL: https://github.com/apache/airflow/issues/17897#issuecomment-910796933
In the meantime. @dimon222 - is it possible that you try my proposed fix,
to see if it actually fixes it ? You'd need to modify the
airflow/dag_processing/manager.py line ~994 (not sure in your version)
```
file_paths_to_exclude = set(file_paths_in_progress).union(
file_paths_recently_processed, files_paths_at_run_limit
)
```
->
```
file_paths_to_exclude = set(file_paths_in_progress).union(
file_paths_recently_processed, files_paths_at_run_limit
).union(set(self._file_path_queue))
```
And yeah. I think - after deeper looking - that transaction isolation would
not explain it (it might explain phantom reads but not really phantom writes
that you experience.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
