OpenBSD Python whitespace oddness (was Re: [Zope] Memory Errors)
Tim Peters wrote:
(" " in HTML). It's surprising to me to see \x89-\x8d there,
though. It could be your system is set to use "an unusual" locale, or
it could be a bug in the platform C libraries. Try writing a little C
program to see what isspace() returns.
Bingo! Thanks for the hints.
You were correct, it was down to a mis-interpretation of the C99 and ISO
8859 standards. Looks like OpenBSD interprets it differently to
everything else ;)
The policy was changed 8 days ago:
http://www.openbsd.org/cgi-bin/cvsweb/src/lib/libc/gen/ctype_.c
"Correct ctype classifications of chars >= 0x80 wrt C99/POSIX and our
man page. ok espie@ deraadt@"
-Matt
--
Matt Hamilton [EMAIL PROTECTED]
Netsight Internet Solutions, Ltd.Business Vision on the Internet
http://www.netsight.co.uk +44 (0)117 9090901
Web Design | Zope/Plone Development & Consulting | Co-location | Hosting
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Memory Errors
[Matt Hamilton]
> ...
> We have been running Zope on OpenBSD/AMD64 3.6 for about a year now
> and it works pretty well. I have however recently discovered a python bug
> that I am trying to track down. I am unsure of the exact problem, but it
> affects the re and string libs:
>
> zeo1# uname -a
> OpenBSD zeo1.netsight.co.uk 3.6 conf#0 amd64
> zeo1# python
> Python 2.3.4 (#1, Nov 16 2004, 08:26:06)
> [GCC 3.3.2 (propolice)] on openbsd3
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import string
> >>> string.whitespace
> '\t\n\x0b\x0c\r \x89\x8a\x8b\x8c\x8d\xa0'
>
> on all other platforms I've tried string.whitespace stops after '\r'...
> the trailing chars cause problems in weird and wonderful places. I
> upgraded to python 2.3.5 and get the same result. Not tried on python
> 2.4 yet.
Python version won't matter: the value of string.whitespace is
entirely determined by your platform C and the locale in effect when
the C-coded portion of Python's string module (Modules/stropmodule.c)
is imported. As you can see from that module's initstrop() function,
string.whitespace consists exactly of the 8-bit characters (0-255) for
which the platform C's isspace() macro returns a true value. The
results you've seen on most systems:
'\t\n\x0b\x0c\r '
is what it must be in the "C" locale (the C standard defines this),
but if you're not in the "C" locale it could be anything.
It's not unusual (well, not for non-Americans ) to see \xa0 in
that list, because \xa0 is Latin-1's non-breaking space character
(" " in HTML). It's surprising to me to see \x89-\x8d there,
though. It could be your system is set to use "an unusual" locale, or
it could be a bug in the platform C libraries. Try writing a little C
program to see what isspace() returns.
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Memory Errors
Malthe Borch wrote: MemoryError What is going wrong here? I'm running an OpenBSD-system with the latest Python 2.3.5 and 512 MB of RAM. The stack size is 0x10, as opposed to 0x2 that previous *BSD-distributions of Python had as default. Malthe, What architecture are you running OpenBSD on? We have been running Zope on OpenBSD/AMD64 3.6 for about a year now and it works pretty well. I have however recently discovered a python bug that I am trying to track down. I am unsure of the exact problem, but it affects the re and string libs: zeo1# uname -a OpenBSD zeo1.netsight.co.uk 3.6 conf#0 amd64 zeo1# python Python 2.3.4 (#1, Nov 16 2004, 08:26:06) [GCC 3.3.2 (propolice)] on openbsd3 Type "help", "copyright", "credits" or "license" for more information. >>> import string >>> string.whitespace '\t\n\x0b\x0c\r \x89\x8a\x8b\x8c\x8d\xa0' on all other platforms I've tried string.whitespace stops after '\r'... the trailing chars cause problems in weird and wonderful places. I upgraded to python 2.3.5 and get the same result. Not tried on python 2.4 yet. Other than that, we've not had any memory issues. On OpenBSD a single process cannot grow over 1GB of process memory (it can get more than that via anon-mmap, but python doesn't support that). On OpenBSD 3.5 we notice that if we hit the 1GB barrier hard it would panic the kernel, but that was fixed in 3.6. How much memory is python using when you get the memory errors? Has it truely used up all the memory on the system? There are soft limits that are set via login.conf and ulimit/limits which may be too low for you. -Matt -- Matt Hamilton [EMAIL PROTECTED] Netsight Internet Solutions, Ltd.Business Vision on the Internet http://www.netsight.co.uk +44 (0)117 9090901 Web Design | Zope/Plone Development & Consulting | Co-location | Hosting ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Memory Errors
> File "/usr/local/lib/zope/lib/python/ZPublisher/HTTPResponse.py", line > 351, in setBody > zlib.DEF_MEM_LEVEL,0) > MemoryError: Can't allocate memory for compression object > zlib? Did your zope work fine before? Remember, about 2 weeks ago they found a vunorability in zlib, didn't they. Maybe there are some implications into that on openbsd. > > What is going wrong here? I'm running an OpenBSD-system with the latest > Python 2.3.5 and 512 MB of RAM. The stack size is 0x10, as opposed to > 0x2 that previous *BSD-distributions of Python had as default. > > Any clues are appreciated, > > \mb > > ___ > Zope maillist - Zope@zope.org > http://mail.zope.org/mailman/listinfo/zope > ** No cross posts or HTML encoding! ** > (Related lists - > http://mail.zope.org/mailman/listinfo/zope-announce > http://mail.zope.org/mailman/listinfo/zope-dev ) > -- Peter Bengtsson, work www.fry-it.com home www.peterbe.com hobby www.issuetrackerproduct.com ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Memory Errors
Hello ng, --- After a few hours, Zope becomes irresponsive and lashes out these kind of errors. Traceback (most recent call last): File "/usr/local/lib/zope/lib/python/ZPublisher/Publish.py", line 104, in publish response.setBody(result) File "/usr/local/lib/zope/lib/python/ZServer/HTTPResponse.py", line 256, in setBody HTTPResponse.setBody(self, body, title, is_error, **kw) File "/usr/local/lib/zope/lib/python/ZPublisher/HTTPResponse.py", line 351, in setBody zlib.DEF_MEM_LEVEL,0) MemoryError: Can't allocate memory for compression object Traceback (most recent call last): File "/usr/local/lib/zope/lib/python/ZPublisher/Publish.py", line 101, in publish request, bind=1) File "/usr/local/lib/zope/lib/python/ZPublisher/mapply.py", line 88, in mapply if debug is not None: return debug(object,args,context) File "/usr/local/lib/zope/lib/python/ZPublisher/Publish.py", line 39, in call_object result=apply(object,args) # Type s to step into published object. File "/usr/local/lib/zope/lib/python/Shared/DC/Scripts/Bindings.py", line 306, in __call__ return self._bindAndExec(args, kw, None) File "/usr/local/lib/zope/lib/python/Shared/DC/Scripts/Bindings.py", line 343, in _bindAndExec return self._exec(bound_data, args, kw) File "/usr/local/lib/zope/lib/python/App/special_dtml.py", line 175, in _exec try: result = render_blocks(self._v_blocks, ns) File "/usr/local/lib/python2.3/string.py", line 135, in join return sep.join(words) MemoryError What is going wrong here? I'm running an OpenBSD-system with the latest Python 2.3.5 and 512 MB of RAM. The stack size is 0x10, as opposed to 0x2 that previous *BSD-distributions of Python had as default. Any clues are appreciated, \mb ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
