[issue37790] subprocess.Popen() is extremely slow
Alexander Pyhalov added the comment: Even if I use import subprocess process = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True, universal_newlines=True) return process.stdout.read() difference in times are the same. -- ___ Python tracker <https://bugs.python.org/issue37790> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37790] subprocess.Popen() is extremely slow
Alexander Pyhalov added the comment: I couldn't reproduce entire test, as wsdiff script is rather large, but here's the simplified version. If Popen is used more often, difference is much more significant. # Using workaround $ python3.5 ~/tmp/1.py 1 10.780487537384033 # Without workaround $ python3.5 ~/tmp/1.py 13.347045660018921 # Using python2.7 $ python2.7 ~/tmp/1.py 9.83385586739 -- Added file: https://bugs.python.org/file48549/1.py ___ Python tracker <https://bugs.python.org/issue37790> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37790] subprocess.Popen() is extremely slow
Alexander Pyhalov added the comment: I've tried to rewrite subporcess.getstatusoutput() usage with subprocess.Popen() and switch to shell=False, it didn't help, so I doubti it getstatusoutput() overhead, it's Popen() issue. -- ___ Python tracker <https://bugs.python.org/issue37790> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37790] subprocess.Popen() is extremely slow
New submission from Alexander Pyhalov : We've moved illumos-gate wsdiff python tool from Python 2 to Python 3. The tool does the following - for each file from old and new proto area compares file attributes to find differences in binary otput (spawning elfdump, dump and other utilities). Performance has degraded in two times when commands.getstatusoutput() was replaced by subprocess.getstatusoutput(). os.popen() now is Popen() wrapper, so it also has poor performance. Even naive popen() implementation using os.system() and os.mkstemp() behaved much more efficiently (about two times faster e.g. 20 minuts vs 40 minutes for single tool pass). -- components: Library (Lib) messages: 349197 nosy: Alexander.Pyhalov priority: normal severity: normal status: open title: subprocess.Popen() is extremely slow versions: Python 3.5 ___ Python tracker <https://bugs.python.org/issue37790> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23895] python socket module fails to build on Solaris when -zignore is in LDFLAGS
Alexander Pyhalov added the comment: We have similar patch here in OpenIndiana. The patch is correct. -- nosy: +Alexander.Pyhalov ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23895 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23895] python socket module fails to build on Solaris when -zignore is in LDFLAGS
Alexander Pyhalov added the comment: BTW, we use the following version for Python 3.4 -- Added file: http://bugs.python.org/file39539/socket.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23895 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20049] string.lowercase and string.uppercase can contain garbage
New submission from Alexander Pyhalov:
When Python 2.6 (or 2.7) compiled with _XOPEN_SOURCE=600 on illumos
string.lowercase and string.uppercase contain garbage when UTF-8 locale is
used.
(OpenIndiana bug report - https://www.illumos.org/issues/4411 ).
The reason is that with UTF-8 locale islower()/isupper() and similar functions
are not expected to work with non-ascii symbols.
So, code like
n = 0;
for (c = 0; c 256; c++) {
if (islower(c))
buf[n++] = c;
}
is expected to fail, because it calls islower on illegal UTF-8 symbols (with
codes 128-255). It should be converted to something like
n = 0;
for (c = 0; c 256; c++) {
if (isascii(c) islower(c))
buf[n++] = c;
}
or to
n = 0;
for (c = 0; c 128; c++) {
if (islower(c))
buf[n++] = c;
}
Before doing this you should check if locale is UTF-8. However, almost all
non-C locales on illumos are UTF-8.
Example of incorrect behavior:
Python 2.6.9 (unknown, Nov 12 2013, 13:54:48)
[GCC 4.7.3] on sunos5
Type help, copyright, credits or license for more information.
import string
string.lowercase
'abcdefghijklmnopqrstuvwxyz\\xaa\\xb5\\xba\\xdf\\xe0\\xe1\\xe2\\xe3\\xe4\\xe5\\xe6\\xe7\\xe8\\xe9\\xea\\xeb\\xec\\xed\\xee\\xef\\xf0\\xf1\\xf2\\xf3\\xf4\\xf5\\xf6\\xf8\\xf9\\xfa\\xfb\\xfc\\xfd\\xfe\\xff'
string.uppercase
'ABCDEFGHIJKLMNOPQRSTUVWXYZ\\xc0\\xc1\\xc2\\xc3\\xc4\\xc5\\xc6\\xc7\\xc8\\xc9\\xca\\xcb\\xcc\\xcd\\xce\\xcf\\xd0\\xd1\\xd2\\xd3\\xd4\\xd5\\xd6\\xd8\\xd9\\xda\\xdb\\xdc\\xdd\\xde'
--
components: Unicode
messages: 206786
nosy: Alexander.Pyhalov, ezio.melotti, haypo
priority: normal
severity: normal
status: open
title: string.lowercase and string.uppercase can contain garbage
type: behavior
versions: Python 2.7
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20049
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20049] string.lowercase and string.uppercase can contain garbage
Alexander Pyhalov added the comment: Honestly, I don't understand locale-related things good enough. But I received this explanation when discussed similar issue in illumos developers mailing list. http://comments.gmane.org/gmane.os.illumos.devel/14193 2013/12/22 Antoine Pitrou rep...@bugs.python.org Antoine Pitrou added the comment: The reason is that with UTF-8 locale islower()/isupper() and similar functions are not expected to work with non-ascii symbols. Can you explain why? -- nosy: +pitrou ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20049 ___ -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20049 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20049] string.lowercase and string.uppercase can contain garbage
Alexander Pyhalov added the comment: I've discussed this once more. From islower man page: RETURN VALUES If the argument to any of the character handling macros is not in the domain of the function, the result is undefined. And (char)128-255 are not legal UTF-8 (at least what I see from wikipedia: http://en.wikipedia.org/wiki/UTF-8 ). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20049 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
