I usually don't deploy under uWSGI, so I can't confirm. But what you are describing would make sense if uWSGI "workers" are subprocesses.
In that case, by the description of "lazy-apps" option I would guess that if that option is False, uWSGI will fork() with the app already loaded (and by consequence the multiprocessing.Queue already existing). As multiprocessing.Queue is in fact implemented over a pipe the file descriptor will be cloned in sub processes. This is probably causing some odd condition in multiprocessing.Queue which breaks it. Solution is to run things with lazy-apps or try to find a way to recreate the queue within the new processes. I saw uwsgi has a "close-on-exec" option to close all inherited file descriptors in forked subprocesses, maybe that will help? If you are lucky enough the multiprocessing.Queue might recreate a new pipe if the current one is closed (or might just crash :D didn't check in multiprocessing source code) It's all guessing here, but it's a reasonable guess. On Mon, Jan 9, 2017 at 10:44 PM, damien accorsi < [email protected]> wrote: > Hil all, > > Anyone is using tgext.asyncjob with uwsgi ? The behavior is very strange. > It's like uwsgi does not allow threads to share a common queue. > Note : asyncjobs were working through apache wsgi ; now that we work with > nginx/uwsgi, the asyncjob thread does not communicate with web threads > anymore... > > Damien > > > Le vendredi 6 janvier 2017 13:05:04 UTC+1, Bastien Sevajol a écrit : >> >> Hello, >> >> We have troubles with *tgext.asyncjob* and *uwsgi*. *tgext.asyncjob* >> worker didn't receive job when project is started with uwsgi as a service. >> To resume when it's working or not: >> >> (*Debian 8.5*) >> >> - Project started with gearbox: working >> - Project started with uwsgi cli command and following ini config: >> working >> - Project started with uwsgi service and following ini config: NOT >> working >> - Project started with uwsgi service and following ini config AND >> lazy-apps >> <http://uwsgi-docs.readthedocs.io/en/latest/Options.html#lazy-apps> >> option: working >> >> *Note: We can't use lazy-apps uwsgi option because breaking some parts of >> our project.* >> >> The uwsgi ini config is the following: >> >> [uwsgi] >> plugins = python3 >> chdir = /home/bastien/Projects/tracim/tracim >> home = /home/bastien/.virtualenvs/tracim_dev >> module = wsgi >> callable = application >> socket = /var/run/uwsgi/app/algoo.trac.im.local/socket >> enable-threads = true >> >> >> During our debugging, wa can affirm this: Code who request job correctly >> reach *tgext.asyncjob.queue.AsyncJobQueue#perform* (and make correctly >> the *queue.Queue#put*). But in >> *tgext.asyncjob.worker.AsyncWorkerThread#run*, following line: >> >> msg = self.queue.get() >> >> still waiting. It's like this queue is not the same in main program and >> asyncjob thread. >> >> Do you hav any idea about this bug ? >> >> Best Regards, >> Bastien. >> > -- > You received this message because you are subscribed to the Google Groups > "TurboGears" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/turbogears. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "TurboGears" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/turbogears. For more options, visit https://groups.google.com/d/optout.
