jason810496 opened a new issue, #75211:
URL: https://github.com/apache/airflow/issues/75211
Follow-up to #70671.
`FileTaskHandler._read` wraps reads of a running task in
`LogStreamAccumulator` to count lines for `log_pos`. The accumulator drains the
whole log, to memory and then to a temp file, before the first line is returned.
The NDJSON path of `GET
/dags/{dag_id}/dagRuns/{dag_run_id}/taskInstances/{task_id}/logs/{try_number}`
never uses that `log_pos`:
- `get_log` builds the `Airflow-Continuation-Token` header from the inbound
metadata before `read_log_stream` runs.
- `read_log_stream` pops any inbound `log_pos` before reading.
NDJSON is the UI's default `accept`, so a running task open in the UI drains
its whole log on every refresh.
### Proposal
Let the caller say whether it needs `log_pos`, instead of inferring it from
`end_of_log`. Only the JSON branch of `get_log` turns `log_pos` into a
continuation token, so only that branch asks for the count. The NDJSON path
then streams running task logs lazily, like finished ones.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
