Am sleeping now for the night, but FWIW, in a another discussion recently we were using interprocess communication mechanisms, which rely on pickle, from the multiprocessing module and not having an issue.
https://docs.python.org/2/library/multiprocessing.html#managers Graham On 03/03/2015, at 10:58 PM, Graham Dumpleton <[email protected]> wrote: > To use socket.makefile, the socket must be in blocking mode (it can not have > a timeout). Is this the case? > > To what extent is the socket visible/usable from multiple request threads? If > it is shared, how are you protecting it from concurrent use? > > Although CherryPy WSGI server may also use multithreading, the dynamics of > how threads are used in mod_wsgi daemon mode could well be quite different. > So if is a multithreading issue, you may simply have been lucky and not > encountered it yet on CherryPy WSGI server. > > So just to rule it out, explain whether this one socket can be used from > different request threads and different times and if so how that usage is > protected. > > Graham > > On 03/03/2015, at 10:52 PM, Stefan Rink <[email protected]> wrote: > >> self.f is the result from calling socket.make_file >> I just don't know why this is working only with one thread under flask and >> mod_wsgi. On flask-dev-server and on CherryPy standalone it also works. And >> CherryPy is also multithreaded. >> >> Am Dienstag, 3. März 2015 12:36:34 UTC+1 schrieb Graham Dumpleton: >> What is the type of the 'self.f' object? >> >> You talk about having a 'socket file', but sockets don't have a proper file >> like object interface. >> >> If you were somehow trying to wrap a file like object interface around a >> socket, that could cause issues. This is because a read on a socket can >> return partial results so the error may be the result of trying to decode >> incomplete data. >> >> What you would need to ensure you are doing is read the full message from >> the socket first, wrap that in a StringIO object and then feed that to >> pickle.load(). >> >> Graham >> >> On 03/03/2015, at 10:23 PM, Stefan Rink <[email protected]> wrote: >> >>> Hello Graham, >>> >>> i get this error and sometimes pickle.load gives an empty string. >>> The other process dumps data into the socket file that I load >>> result = pickle.load(self.f) >>> File "/usr/lib/python2.7/pickle.py", line 1378, in load >>> return Unpickler(file).load() >>> File "/usr/lib/python2.7/pickle.py", line 858, in load >>> dispatch[key](self) >>> File "/usr/lib/python2.7/pickle.py", line 1138, in load_pop >>> del self.stack[-1] >>> IndexError: list assignment index out of range >>> >>> Sorry I cannot provide more but have no clue why this is going wrong. Thank >>> you for your time. >>> >>> >>> Am Dienstag, 3. März 2015 12:01:48 UTC+1 schrieb Graham Dumpleton: >>> Can you go back and explain and provide the actual details of the error >>> message you are getting. If I can see the actual error messages then it >>> might be more obvious. >>> >>> Graham >>> >>> On 03/03/2015, at 8:49 PM, Stefan Rink <[email protected]> wrote: >>> >>>> Hello Graham, >>>> >>>> thank you very much for your answer. >>>> >>>> Only standard Python types are pickled: dicts, bytestrings. It works only >>>> when using a single thread in the WSGIDaemonProcess directive! Is that >>>> helping? >>>> >>>> Thank you >>>> >>>> >>>> Am Dienstag, 3. März 2015 09:58:11 UTC+1 schrieb Graham Dumpleton: >>>> >>>> On 03/03/2015, at 7:10 PM, Stefan Rink <[email protected]> wrote: >>>> >>>> > Hello Graham, >>>> > >>>> > under mod_wsgi I get pickle errors when pickling over a socket to >>>> > another process (no Apache child). Can this be caused by mod_wsgi? >>>> > mod_wsgi is in Daemon Mode. >>>> >>>> The only restriction I really know of which is specific to mod_wsgi is >>>> that you cannot pickle types which are defined in the WSGI script file >>>> itself. You should ensure that any types are defined in separately >>>> importable modules on the Python module search path. Those same modules >>>> with the types in them should also be importable in the application the >>>> data is being sent to so they are available when unpicking the data. >>>> >>>> For more information about the destruction I talk about see: >>>> >>>> http://code.google.com/p/modwsgi/wiki/IssuesWithPickleModule >>>> >>>> Graham >>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google Groups >>>> "modwsgi" group. >>>> To unsubscribe from this group and stop receiving emails from it, send an >>>> email to [email protected]. >>>> To post to this group, send email to [email protected]. >>>> Visit this group at http://groups.google.com/group/modwsgi. >>>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "modwsgi" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to [email protected]. >>> To post to this group, send email to [email protected]. >>> Visit this group at http://groups.google.com/group/modwsgi. >>> For more options, visit https://groups.google.com/d/optout. >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "modwsgi" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To post to this group, send email to [email protected]. >> Visit this group at http://groups.google.com/group/modwsgi. >> For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/modwsgi. For more options, visit https://groups.google.com/d/optout.
