Re: [Python-Dev] Pythreads and BSD descendants
Martin v. Löwis wrote: > Cameron Laird schrieb: >> Folklore that I remember so unreliably I avoid trying to repeat it here >> held that Python threading had problems on BSD and allied Unixes. What's >> the status of this? > > The problem that people run into again and again is the stack size. The > BSDs allow for so little stack so that even the quite conservative > estimates of Python as to how many recursions you can have are > incorrect, and you get an interpreter crash rather than a RuntimeError > (as you should). There are 2 aspects to the thread stack size issue: - the stack size for the primary thread; - the stack size for created threads. I haven't done any investigating for FreeBSD 6.x and later, but I know that FreeBSD 4.x had a hard coded stack size of 1MB for the primary thread in a thread enabled application, which is what Martin's comment above particularly applies to. This affects code that doesn't use threads at all, and was particularly painful with REs prior to SRE being made non-recursive. If you build the interpreter without thread support, stack space is instead controlled by session limits which are usually generous (typically 64MB). I don't recall exactly FreeBSD's default stack size for threads created via pthread_create() but it is fairly small (32kB or 64kB comes to mind).Zope is one application known to be affected by this lack of stack size in created threads. At least the stack size for new threads can be adjusted at runtime, and a mechanism for doing this was added to Python 2.5. > Furthermore, every time we decrease the that number, the next system > release somehow manages to make the limit even smaller. This was > never properly analyzed; I suspect that the stack usage of Python > increases, either due to compiler changes or due to change to Python > itself. I have seen examples of stack consumption increasing with increasing gcc version number, sometimes depending on optimisation choices. Regards, Andrew. -- - Andrew I MacIntyre "These thoughts are mine alone..." E-mail: [EMAIL PROTECTED] (pref) | Snail: PO Box 370 [EMAIL PROTECTED] (alt) |Belconnen ACT 2616 Web:http://www.andymac.org/ |Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] x86_64 Interix - Advise needed on size of long
Hello all, I'm in need of an advise how to handle sizeof long in python. I wanted a x86_64 compile of python for Interix (that is NT POSIX subsystem with x86_64 Interix 6 SDK). My first attempt to build failed due to the makefile insisted on linking as shared libraries (works only in x86 with GNU ld). Tried autoreconf to get rid of libtool - no luck. Q1: Is the static build broken? Q2: Anyone have a useable Makefile.am? My second attempt was based on the VS2005 project and the previous Makefile. Not to tire you with details, but for this to work I need to explicit assign the sizeof long (replace all long types with explicit sized ones, int32_t, ssize_t etc). There are 2 choices: All longs to 64bit (LP64 model) or all to 32bit (LLP64 model). Since Interix use LP64 the first alternative would be logic, but considering compatibility with the Windows DLL, performance(?) and whatever, I choosed the latter. A choice which later would turn me into trouble. Here's how I am reasoning: x64 Windows DLL = LLP64 model => sizeof(long) = 4 x86_64 Interix = LP64 model => sizeof(long) = 8 So, since the Windows build works, basically all long types in the code are 32bit (or at least works if they are 32bit). 64bit dependent variables like pointers have already been taken care of. Right? While it sounds reasonable as long as one are consistent, it's actually quite difficult to get it right (and a lot of work). To be precise, would this be OK? long PyInt_AsLong(PyObject *); change to: int32_t PyInt_AsLong(PyObject *); or unsigned long PyOS_strtoul(char*, char**, int); to: uint32_t PyOS_strtoul(char*, char**, int); Thanks, Erik ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] x86_64 Interix - Advise needed on size of long
> My first attempt to build failed due to the makefile insisted on linking as > shared libraries (works only in x86 with GNU ld). Tried autoreconf to get > rid of libtool - no luck. > Q1: Is the static build broken? > Q2: Anyone have a useable Makefile.am? Are you sure you are talking about Python as released? It uses neither automake nor libtool (IMO, fortunately so). As for the static vs. shared libpython: On Unix, Python is typically built as a single executable (only linked shared with the system libraries). The challenge is then with extension modules, which are shared libraries. In particular, it is a challenge that those want to find symbols defined in the executable, without being linked with it. So you have three options: 1. If you use a sane binary format (such as ELF), symbol resolution considers symbols defined by the executable for use in shared libraries. This is necessary to support standard C, as you want to be able to redefined malloc(3) in the executable, and then all libraries should use your malloc implementation; it comes handy for Python's extensions. By this definition, Portable Executable (PE) is insane. 2. Don't use extension modules. Edit Modules/Setup to statically link all extension modules into the interpreter binary. 3. Arrange to make the interpreter a shared library (libpythonxy.so), then link all extension modules against it. > There are 2 choices: All longs to 64bit (LP64 model) or all to 32bit (LLP64 > model). Since Interix use LP64 the first alternative would be logic, but > considering compatibility with the Windows DLL, performance(?) and whatever, > I choosed the latter. A choice which later would turn me into trouble. I don't see the compatibility issue. You aren't going to use Win32 extensions in the Interix interpreter, are you? So why care about Win32? > Here's how I am reasoning: > > x64 Windows DLL = LLP64 model => sizeof(long) = 4 > x86_64 Interix = LP64 model => sizeof(long) = 8 I think we agree that the Windows model is insane, also. A good 64-bit platform has sizeof(long)==8. > So, since the Windows build works, basically all long types in the code are > 32bit (or at least works if they are 32bit). Right. However, LP64 also works with Python, and has been for many more years. > 64bit dependent variables like > pointers have already been taken care of. Right? While it sounds reasonable > as long as one are consistent, it's actually quite difficult to get it right > (and a lot of work). > > To be precise, would this be OK? > long PyInt_AsLong(PyObject *); > change to: > int32_t PyInt_AsLong(PyObject *); > or > unsigned long PyOS_strtoul(char*, char**, int); > to: > uint32_t PyOS_strtoul(char*, char**, int); OK in what sense? You making these changes locally? You can make whatever changes you please; this is free software. I can't see *why* you want to make all these changes, but if you so desire... This becoming part of Python? No way. It is intentional that PyInt_AsLong returns long (why else would the function be called this way?), and it is also intentional that the int type has its internal representation as a long. Likewise for strtoul: this function is defined to return long, for whatever definition long has on a platform. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Summary of Tracker Issues
ACTIVITY SUMMARY (07/29/07 - 08/05/07) Tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1276 open ( +0) / 11101 closed ( +0) / 12377 total ( +0) Average duration of open issues: 695 days. Median duration of open issues: 561 days. Open Issues Breakdown open 1274 ( +0) pending 2 ( +0) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
