stavdav143 commented on PR #35677:
URL: https://github.com/apache/airflow/pull/35677#issuecomment-1837588155
Hi @eladkal
I committed a unit test for logging.
There were small changes I needed to carry in the test file in order to
comply with the logging algorithm we use.
These were committed separately for better explanation:
1. Follow=true for logs will always result in tasks not ending. This is
standard and provided upstream by the Docker API.
Therefore in DockerSwarmOperator follow is always false
[041e4c](https://github.com/apache/airflow/pull/35677/commits/041e4c5cabd3670227081eae4bc1c234822fde31)
2. service_logs called multiple times as we continuously poll the Docker API
for logs. As we indicated in the previous commmit,
the docker client malfunctions when we try to get the logs with
follow=True. Therefore we make multiple calls to the API (every 2 seconds), to
fetch the new logs.
[c8063a](https://github.com/apache/airflow/pull/35677/commits/c8063aceddc131dc96becbc3e98dd10a8354f283)
3. service_logs called multiple times. In this test the tasks increase (6
instead of 5) as we check if the service has terminated (+1). As this assertion
makes less sense in a situation where we do multiple calls to the Docker API
(polling), we might think of removing it or replacing it with something more
suitable.
[af2428](https://github.com/apache/airflow/pull/35677/commits/af24289badded7c989d963161eedec509c29dde0)
---------
And finally with
[048ba1](https://github.com/apache/airflow/pull/35677/commits/048ba1ece25fcd02c5ccc752d63759c76b098e44)
we have the final commit marking the test case that validates logging in the
Docker Swarm Operator. We log two times a different message and we assert that
the two lines are given back in the logs in the expected sequence.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
