pdavis156879 commented on issue #15965: URL: https://github.com/apache/airflow/issues/15965#issuecomment-845037447
> The problem is that you are using sqlite database. Sqlite works in different way than postgres/mysql. Instead of having a server where different containers connect to, sqlites writes local file where the database is kept. Unless you create a shared volume which is mounted in all the containers, the database initialization you do in in "init" container is gone at the moment the container finishes. In your case, when webserver or scheduler start - each of them have their own new, fresh, non-initialized sqlite database and it is not going to work, because all of them should share a single database. > > There are two ways you can approach it: > > 1. modify the docker compose.yaml to include such a shared volume, mount it in all containers. This requires a bit more intrisic knowledge of how docker/docker-compose works. Ideally this should be a named volume that can survive stopping/starting docker containers, so that the DB is more persistent. > 2. use postgres database - in the way that the original docker compose of ours uses it. It starts a postgres service which then all the containers connect to, so the init container will initialize the database and other containers will be able to us it.. > > Please let me know what worked for you and I will close the issue then. Hi @potiuk , Thank you so much for your assistance. Regarding your suggestions, I would prefer to go with number 2, since it's the most standard and requires less intervention - and less place to mess up. However, I am using Airflow's original image (slightly modified for testing, which includes a common network to debug possible communication issues between services), in fact, here it is: - [docker-compose.yaml](https://gist.github.com/pdavis156879/1671f422b2c34cbb4d4e70810be001f6) Which makes it even stranger for the message to appear. In fact, the alchemy connection and result backend are hardcoded with a postgres connection string, so the sqlite error becomes even odder. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
