ecerulm commented on issue #21087: URL: https://github.com/apache/airflow/issues/21087#issuecomment-1120994210
@snjypl, does not `allow_watch_bookmarks` prevent a "410 GONE", and does not prevent the current airflow code path to enter an infinite loop because of it. The current code base will enter an infinite loop if kubernetes python client returns an Exception when sending the last known resource version to airflow. Using BOOKMARK will certainly allow airflow to track the last known resource version more accurately, but it doesn't rule out the possibility of a 410 GONE at all. I did test with `kubernetes==23.3.0` and you still get a 410 when using `allow_watch_bookmarks=True` if the `resource_version` is expired (which can happen if there is let's say a 10 minutes network disconnnect) I think there are two issues here * watch with bookmarks, which is good to have and allow better tracking of the last resource_version but does not prevent 410 gone. * this issue which is about airflow entering an unrecoverable infinite loop if watch ever returns a 410 GONE. This is solved on #23521. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
