Hi,
On 05/11/2020 01.11, Mads Kiilerich wrote:
On 11/3/20 5:08 PM, Valentin Kleibel wrote:
Hi,
We are using kallithea with a postgres database on another server
connected via ssl.
It seems that celery in version 4 uses prefork in its worker pool
model now, which leads to issues with such a database connection:
I think you are right something is wrong.
First, the celery-run will not only take a config file as parameter and
read it. It will also initialize the WSGI app and database connection,
before calling into celery. Effectively making it pre-fork. I think you
are right that we only should initialize things after celery has forked
and started worker processes.
But also, it might be a bit odd that we have the celery-run entry point.
It would perhaps be better to somehow use "celery" as entry point and
let it call into Kallithea in each worker. But I haven't investigated
how feasible that would be.
Thank you for the confirmation.
I'd be happy to help out and create a patch or do other work provided
there is a clear understanding in which way we'd like to fix this.
As a workaround, can you use celery.worker_concurrency = 1 but launch
celery-run several times?
This needs quite a bit of changes and I don't want to invest time in
that. The patch I've sent works for us right now so we'll keep this
workaround.
It might actually qualify as a proper fix as this is what the sqalchemy
documentation recommends to do. [1]
Thanks,
Valentin
[1]
https://docs.sqlalchemy.org/en/13/core/pooling.html#using-connection-pools-with-multiprocessing-or-os-fork
_______________________________________________
kallithea-general mailing list
[email protected]
https://lists.sfconservancy.org/mailman/listinfo/kallithea-general
