seanghaeli commented on issue #76640: URL: https://github.com/apache/airflow/issues/76640#issuecomment-6965274871
Once `upload()` reports the result, the triggerer's watcher log files (#76527) can use it in five places that guess today, in `airflow/utils/log/watcher_log.py` and `airflow/jobs/triggerer_job_runner.py`: - [`WatcherLogs.upload_rotated`](https://github.com/apache/airflow/blob/426dafc3e73028d5544345b847a64c0ef940842d/airflow-core/src/airflow/utils/log/watcher_log.py#L329): a rotated file is sent again once ([`WatcherLogs._upload_rotated_again`](https://github.com/apache/airflow/blob/426dafc3e73028d5544345b847a64c0ef940842d/airflow-core/src/airflow/utils/log/watcher_log.py#L320), same file) if its upload raised, except with OSS, or if S3, GCS, WASB or OSS left its lines. - [`WatcherLogs._upload_orphaned_trigger_log_dir`](https://github.com/apache/airflow/blob/426dafc3e73028d5544345b847a64c0ef940842d/airflow-core/src/airflow/utils/log/watcher_log.py#L450): the same rule for the files a dead or stopped triggerer left behind; those it judges failed stay for the next triggerer that starts on that disk. - [`WatcherLogs.upload_rotated`](https://github.com/apache/airflow/blob/426dafc3e73028d5544345b847a64c0ef940842d/airflow-core/src/airflow/utils/log/watcher_log.py#L329) again: the next rotation deletes the previous file after its upload and one retry, whether or not it was stored. - [`WatcherLoggingFactory.upload_to_remote`](https://github.com/apache/airflow/blob/426dafc3e73028d5544345b847a64c0ef940842d/airflow-core/src/airflow/jobs/triggerer_job_runner.py#L638) / [`WatcherLogs.upload_and_close`](https://github.com/apache/airflow/blob/426dafc3e73028d5544345b847a64c0ef940842d/airflow-core/src/airflow/utils/log/watcher_log.py#L372): the upload when a watcher leaves the triggerer, or when the triggerer stops, isn't retried. A file it failed to store keeps its lines, so they go with that watcher's next upload if it comes back to this triggerer, or else to the next triggerer that starts on that disk. - [`upload_watcher_log_file`](https://github.com/apache/airflow/blob/426dafc3e73028d5544345b847a64c0ef940842d/airflow-core/src/airflow/utils/log/watcher_log.py#L183): `OSSRemoteLogIO.upload` drops the result of `oss_write`, so for OSS it calls `oss_write` directly and empties the file only when the write succeeded. Once `upload()` reports success, replace that direct `oss_write` call with `upload()`'s result. These pick the store by handler class name ([`_EMPTIES_STORED_FILE_HANDLERS`](https://github.com/apache/airflow/blob/426dafc3e73028d5544345b847a64c0ef940842d/airflow-core/src/airflow/utils/log/watcher_log.py#L148), [`_LINES_LEFT_MEAN_FAILED_HANDLERS`](https://github.com/apache/airflow/blob/426dafc3e73028d5544345b847a64c0ef940842d/airflow-core/src/airflow/utils/log/watcher_log.py#L152) and the lists next to them). That has to stay as the fallback for providers that don't return a result yet. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
