Bugs item #689895, was opened at 2003-02-20 00:39 Message generated for change (Comment added) made by sitbon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=689895&group_id=5470
Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: M.-A. Lemburg (lemburg) Assigned to: Nobody/Anonymous (nobody) Summary: Imports can deadlock Initial Comment: [Just copying some discussion details from python-dev into this bug report so that we can reopen it to Python 2.4] [MAL] >> I wonder whether a general lock such as the one used in import >> is such a good idea. Perhaps it should only lock the importing >> of a specific module, keeping the locks in a dictionary indexed by >> module name instead of a static C variable ?! I've been thinking along the same lines. We could make the import locking much finer-grained, and limit the blocking only to threads that are importing a module that is in the middle of being loaded by some other thread. But this is hard work, and I suggest that we put this off until Python 2.4 so we can do it right. >> Then again it's hard to know the real name of the module being >> searched before finding it... There could be a short-lived lock for that problem. >> I see a more general problem here: the lock prevent starting >> up threaded applications which use client-server logic between >> the threads. If the application's main thread starts a client >> thread as a result of an import which then tries to import >> other Python modules, you have a deadlock. (At least that's how >> I understand the current implementation.) Correct. >> Don't know about others, but I frequently use the idiom of >> placing the server's main code in a separate module and then >> have small startup script importing this module. This kind >> of setup is also advertised for CGI programs, so it may not >> be uncommon out there. We had this problem with Zope2 -- I don't know why the import lock didn't bite us before, but we decided to change the Zope startup code so that you have to import Zope first and then, separately, make a call to start it. You could do the same for your application, and I recommend that Mark does the same for his. --Guido van Rossum (home page: http://www.python.org/~guido/) ---------------------------------------------------------------------- Comment By: Phillip Sitbon (sitbon) Date: 2005-12-08 13:50 Message: Logged In: YES user_id=1303233 Has anyone looked further into this? I seem to be having the same problem with a simple multithreaded program that imports from within thread functions. I didn't have the problem (at least not too often, but still in some cases) with 2.4.1 but now 2.4.2 has me deadlocking on the imports every time. It's a frustrating problem! ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2003-02-20 00:40 Message: Logged In: YES user_id=38388 [EMAIL PROTECTED] >[MAL] > >>> > I wonder whether a general lock such as the one used in import >>> > is such a good idea. Perhaps it should only lock the importing >>> > of a specific module, keeping the locks in a dictionary indexed by >>> > module name instead of a static C variable ?! I am interested in this also. >> >> [Guido] >> I've been thinking along the same lines. We could make the import >> locking much finer-grained, and limit the blocking only to threads >> that are importing a module that is in the middle of being loaded by >> some other thread. >> >> But this is hard work, and I suggest that we put this off until Python >> 2.4 so we can do it right. I ran into this exact thing when trying to get wxPython windows (data plots) to co-exist peacefully on the screen in parallel with an interactive shell. The module is called gui_thread, and we use it in SciPy. http://www.scipy.org/site_content/tutorials/gui_thread The command line took the main thread, and a wxPython app was started in the background thread. The first import of wxPython had to occur in the background thread for wxPython's to be happy. Initially, I put a lock in the import of gui_thread which waited for the wxPython import to complete before allowing the gui_thread import to complete. If 'import gui_thread' is the first statement executed, this guarantees that wxPython is always imported first in the background thread. Unfortunately, it also causes deadlock because of the import lock. A discussion that occurred about this problem on the thread-sig with is summarized here: http://www.scipy.org/site_content/tutorials/import_thread_lock_discussio n It includes an alternative version of several import.c routines (compared to 1.5.2 I believe) that solved my problem as proposed by MAL above and passed all the regression tests at the time. I just plugged the code into the 2.3CVS, and it mainly works. But, there are some import related errors in the regression tests for: test_threaded_import test_loggingllbacks I could spend some time on these if there is interest in getting this in the current release. Also, this code would need review by someone that is an expert on the subtleties of the import code to make sure it is sound. Let me know, and I'll submit a patch with of the current code. eric ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=689895&group_id=5470 _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
