Bugra: Can you give more detail on how caplog gives instability, but mocking the log doesn’t? I can’t see what is actually different between them.
> On 4 Feb 2025, at 22:38, Buğra Öztürk <ozturkbugr...@gmail.com> wrote: > > Hey all, > > The consensus has been reached. > > Regards, > > On Sat, Feb 1, 2025 at 5:38 PM Buğra Öztürk <ozturkbugr...@gmail.com> wrote: > >> Hello everyone, >> >> There have been recurring discussions about minimizing the usage of caplog >> in unit tests. If logs are necessary for a test, we should mock them unless >> the test strictly requires actual log output. Otherwise, the caplog tests >> should be removed and logs should be mocked. >> >> During recent large migrations, changes and ongoing CI/CD efforts, caplog >> has proven to be flaky in unit tests. They frequently cause red pipelines >> in PRs and scheduled CI/CD runs. There have been local discussions on >> removing caplog from unit tests. To formalize this, I propose a lazy >> consensus to remove and prevent caplog usage in unit tests, ensuring logs >> are mocked when needed and disallowing caplog without mocking unless >> explicitly approved. >> >> *Why should caplog be avoided?* >> >> - Big maintenance effort on CI/CD >> - Instability >> >> >> *What should we do instead if we use caplog in a unit test?* >> >> - Mocking the Log >> >> *In the Scope of this consensus* >> >> - Remove caplog usage from unit tests if possible >> - Mock logs and remove caplog from unit tests if possible >> - Exceptional cases will be subject to maintainer approval >> - Prevent caplog to be included in unit tests without explicitly >> mocking the log >> >> *Action Items related to the above Scope:* >> >> - Scan and replace caplog tests with mocking where possible >> - Scan and remove caplog tests where possible >> - Include CI check to prevent adding additional caplog tests without >> Mocking and/or without approval from a maintainer to allow flexibility in >> some exceptional cases >> - Create documentation with examples for implementing tests using logs >> without caplog (e.g., using mocking or avoiding logs entirely) >> >> >> The lazy consensus will be reached on 2025-02-04, (unless someone surfaces >> an objection). >> -- >> Bugra Ozturk >> > > > -- > Bugra Ozturk --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org For additional commands, e-mail: dev-h...@airflow.apache.org
