jason810496 commented on issue #46657: URL: https://github.com/apache/airflow/issues/46657#issuecomment-2668229238
> I am not too concerned about the speed - I am much more concerned about memory. Hi @potiuk, The Pydantic model that @ashb mentioned here is used for parsing per line, not for returning the entire log. Therefore, there won't be a significant difference compared to the current read log behavior. I will resolve conflicts for my PR (#45079) once @ashb's change is merged. --- After further review, I found that there is a **breaking change in the `_read` method**. Currently, 5 providers implement the `_read` interface, so a compatible interface is needed—perhaps something along these lines: ( The providers implemented `_read` method are also listed in the reference ) https://github.com/jason810496/airflow/blob/refactor/webserver-oom-for-large-log-read/airflow/utils/log/file_task_handler.py#L300-L336 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
