potiuk commented on pull request #9973: URL: https://github.com/apache/airflow/pull/9973#issuecomment-848232690
OK. The Mysql error is gone, so the only one left is the Mssql mysteriously failing without logs. And I think I know the issue. When you just start the containers, mssql is ~750 MB. That's a lot: ``` CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS 4c55f0861f17 docker-compose_mssql_1 6.45% 744.5MiB / 6.472GiB 11.23% 282kB / 10.2kB 1.58GB / 147MB 154 ``` The `default` setting of mssql container is 750 MB which is far too much for the tests of ours. The docker containers for databases are optimized for more memory use as they are supposed to handle "typical" use cases, but our test are not very demanding. On the other hand the Github runners have limited memory and when we run "integration" tests there are more containers. I've fine-tuned both postgres and mysql so when they start, they are ~ 100MB (and usually does not exceed 500 MB when runnning): mysql: ``` CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS 217b80108323 docker-compose_mysql_1 0.27% 107.9MiB / 6.472GiB 1.63% 1.62kB / 452B 73.1MB / 430MB 28 ``` postgres: ``` CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS b7e5b4a3a6a0 docker-compose_postgres_1 0.08% 101.9MiB / 6.472GiB 1.54% 1.68kB / 224B 54.7MB / 261MB 6 ``` I tried to find some ways to limit the memory used by mssql linux container but I think it might be difficult: https://github.com/microsoft/mssql-docker/issues/517 However, what we might want to do is to disable the Integration tests for Mssql. Those are the tests that start more containers and all those containers might simply take too much memory. I will add a quick fixup to try this out. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
