Thanks to @daviddavis for making PRs resolving both of the issues. Several core devs ack'd it so I merged this change. If your travis jobs are failing due to sqlite tests, rebasing will resolve it for you.
Note from @jeremy's link to the isolation docs, it suggests that the WAL mode allows for multiple writer locks even with multiprocessing. I was testing using WAL mode yesterday, so the issue was still that the timeout wasn't working. If someone is able to get Pulp working perfectly with sqlite please reply on-list. On Wed, Apr 25, 2018 at 5:18 PM, Brian Bouterse <[email protected]> wrote: > > > On Wed, Apr 25, 2018 at 3:16 PM, Brian Bouterse <[email protected]> > wrote: > >> tldr: A bug in sqlite is preventing Pulp from running pulp-smash tests >> without error making sqlite a non-viable default for Pulp. We should switch >> back to Postgres as the default for Pulp and its docs, the dev environment, >> and remove sqlite from the Travis matrix. >> >> Q: Why isn't the timeout [0] being applied? >> A: because it's broken in sqlite's C code (the evidence suggests) >> >> Q: What units is the timeout in exactly? >> A: it's in seconds and our configuration of it was correct. In cPython's >> code it gets converted to milliseconds which is the right unit for SQLite's >> interface. >> >> Q: If this gets fixed in the future will we switch back? >> A: Once we enter RC or GA for core we cannot switch the default. So if >> this gets resolved "real soon" by the sqlite folks then yes, otherwise no. >> >> Q: How do you know it's broken? >> A: I set the timeout to 300 seconds, but the database locked error is >> still raised after the default of 5 seconds. I use the `PRAGMA >> busy_timeout;` to confirm that the sqlite knows the 300 second timeout has >> been set. I've analyzed the django [1] and Python C [2] code also to >> confirm that our setting is being handled properly all the way down to >> sqlite's C interface. I've raised the issue with the sqlite community and >> I'm waiting to hear back. >> >> Q: What are the next steps? >> A: We need issues to switch Pulp's settings and docs to Postgresql. >> Similarly for switching the devel environment. Also one to remove sqlite >> from Travis. We need these like real soon. >> > > I think these two issues should be groomed and added to the sprint. There > is outstanding that can't merge because of this issue causes Travis tests > to fail which blocks merging. > > https://pulp.plan.io/issues/3606 > https://pulp.plan.io/issues/3505 > > >> >> Q: What about the community's concerns? >> A: They can help contribute to the sqlite bug that is prevent Pulp from >> being meaningfully used on top of sqlite. They can still configure sqlite >> and use it just like any django supported db, we just don't test on top of >> it on Travis anymore. It is erroring almost all of our Travis jobs. >> >> In terms of the how we document PostgreSQL, I want to advocate avoiding >> documenting command by command installs of Postgres. I used to feel >> differently about this, but I think if users start on our docs and they >> don't work, we're probably not going to help them get their Postgresql >> going on some random distro. Soon the Ansible installer will solve this for >> our users anyway so I don't think it's a big problem. >> >> Ideas, concerns, help writing issues, and questions are all welcome! >> >> [0]: https://docs.djangoproject.com/en/1.11/ref/databases/#databa >> se-is-locked-errors >> [1]: https://github.com/django/django/blob/1.11.12/django/db/back >> ends/sqlite3/base.py#L177 >> [2]: https://github.com/python/cpython/blob/master/Modules/_sqlit >> e/connection.c#L181 >> >> -Brian >> >> >> >
_______________________________________________ Pulp-dev mailing list [email protected] https://www.redhat.com/mailman/listinfo/pulp-dev
