jedcunningham commented on code in PR #50715:
URL: https://github.com/apache/airflow/pull/50715#discussion_r2127592177
##########
airflow-core/src/airflow/utils/log/log_reader.py:
##########
@@ -96,10 +99,17 @@ def read_log_stream(self, ti: TaskInstance, try_number: int
| None, metadata: di
TaskInstanceState.RUNNING,
TaskInstanceState.DEFERRED,
):
- if not logs[0]:
+ if logs:
+ empty_iterations = 0
+ else:
# we did not receive any logs in this loop
# sleeping to conserve resources / limit requests on
external services
time.sleep(self.STREAM_LOOP_SLEEP_SECONDS)
+ empty_iterations += 1
+ if empty_iterations >=
self.STREAM_LOOP_STOP_AFTER_EMPTY_ITERATIONS:
+ # we have not received any logs for a while, so we
stop the stream
+ yield "(Log stream stopped - End of log marker not
found, so logs may be incomplete)\n"
+ break
Review Comment:
Done.
##########
airflow-core/src/airflow/utils/log/log_reader.py:
##########
@@ -96,10 +99,17 @@ def read_log_stream(self, ti: TaskInstance, try_number: int
| None, metadata: di
TaskInstanceState.RUNNING,
TaskInstanceState.DEFERRED,
):
- if not logs[0]:
+ if logs:
+ empty_iterations = 0
+ else:
# we did not receive any logs in this loop
# sleeping to conserve resources / limit requests on
external services
time.sleep(self.STREAM_LOOP_SLEEP_SECONDS)
+ empty_iterations += 1
+ if empty_iterations >=
self.STREAM_LOOP_STOP_AFTER_EMPTY_ITERATIONS:
+ # we have not received any logs for a while, so we
stop the stream
+ yield "(Log stream stopped - End of log marker not
found, so logs may be incomplete)\n"
Review Comment:
We add a newline onto each other line, including the last one, so kept it
that way for consistency.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
