[issue23628] See if os.scandir() could help speed up importlib
Brett Cannon added the comment: The answer is no. -- resolution: - rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23628 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23622] Deprecate unrecognized backslash+letter escapes in re
Changes by Serhiy Storchaka storch...@gmail.com: -- title: Deprecate unrecognized backslash+letter escapes - Deprecate unrecognized backslash+letter escapes in re ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23622 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23645] Incorrect doc for __getslice__
Changes by Ezio Melotti ezio.melo...@gmail.com: -- stage: - patch review type: - enhancement ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23645 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23566] RFE: faulthandler.register() should support file descriptors
Wei Wu added the comment: The last approach I proposed requires some change in template code of check_xxx methods. To make it better, we can add a bool parameter to the check_xxx functions, True value indicating a fd test. If a filename is given at the same time, then a fd can get from that file. Otherwise the fd should be sys.stderr.fileno(). e.g. file = None fp = None if filename: fp = open(filename, wb) # Must use a different name to prevent the file from closing... file = fp if fd: if fp is not None: file = fp.fileno() else: file = sys.stderr.fileno() # file can be file-object, fd or None (use_the_file_to_function...) The fd-passing approach can co-exist with this one. However it will make template code more complex. So I suggest just use one approach to write these fd tests. I will work on a patch(based on tip) at this weekend. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23566 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23637] Warnings error with non-ascii chars.
Serhiy Storchaka added the comment: The problem is not only with the file.write(). If one of arguments is unicode (even if it doesn't contain non-ascii characters) and other argument is non-ascii string, we get this error. warnings.showwarning(u'', DeprecationWarning, 'filè.py', 10) Traceback (most recent call last): File stdin, line 1, in module File /home/serhiy/py/cpython-2.7/Lib/warnings.py, line 33, in _show_warning file.write(formatwarning(message, category, filename, lineno, line)) File /home/serhiy/py/cpython-2.7/Lib/warnings.py, line 42, in formatwarning s = %s:%s: %s: %s\n % (filename, lineno, category.__name__, message) UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 3: ordinal not in range(128) Non-ascii file names are rare, and unicode warnings are rare, that is why this bug was not fixed before. I think it is worth to fix. It is better to output modified warning (e.g. backslashescaped) than fail without clear diagnostic. -- nosy: +serhiy.storchaka resolution: not a bug - stage: resolved - status: closed - open ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23637 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23658] multiprocessing: string arg to SystemExit
Changes by Serhiy Storchaka storch...@gmail.com: -- nosy: +sbt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23658 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23622] Deprecate unrecognized backslash+letter escapes
Changes by Chris Rebert pyb...@rebertia.com: -- nosy: +cvrebert ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23622 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23654] infinite loop in faulthandler._stack_overflow
Matt Frank added the comment: Yes, this is currently only a problem with the Intel compiler. The writes to buffer[] are dead (provably won't be ever used) at the point that the recursive call occurs. Actually gcc and llvm can figure this out. Thus all the space allocated for the first call can be reused by every subsequent call. The analysis that icc is doing that gcc and clang are not yet doing has to do with the pointer to depth. I believe icc is able to determine that depth doesn't point into buffer where gcc and clang aren't sure that this is the case. (If you change stack_overflow() so that depth is passed in and returned by value then gcc also does the tail optimization.) volatile doesn't disable the optimization. I think in this case it is too easy to determine that buffer is on the stack, and both buffer and sp are dead by the time of the recursive call. It occurs to me that the right way to deal with this is with __attribute__ ((optimize (no-optimize-sibling-calls))) on the function where it matters (stack_overflow()). I'm working on a solution (need to test whether the compiler supports attributes and etc.) and will post it ASAP. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23654 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23614] Opaque error message on UTF-8 decoding to surrogates
Ezio Melotti added the comment: The Table 3-7 of http://www.unicode.org/versions/Unicode5.2.0/ch03.pdf (page 93 of the book, or 40 of the pdf) shows that if the start byte is ED the continuation byte must be in range 80..9F. This means that, in order to decode a sequence starting with ED, you need two more valid continuation bytes. Since the following byte (B4) is not in allowed range 80..9F and is thus an invalid continuation byte, the decoder doesn't know how to decode the byte in position 0 (i.e. ED). It is also true that this particular sequence, if allowed, would result in a surrogate. However, by looking at the first two bytes only, you don't have enough information to be sure about that (e.g. ED B4 00 begins doesn't decode to a surrogate, so Pike's error message is imprecise). If handling this special case doesn't require too much extra code, it would be ok with me to have something like: b\xed\xb4\x80.decode(utf-8) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xed in position 0: invalid continuation byte (possible start of a surrogate) -- type: - enhancement ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23614 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23637] Warnings error with non-ascii chars.
Ezio Melotti added the comment:
I think that the problem is actually with the file.write() in _show_warning().
If any of the arguments of formatwarning() are unicode, the result will be
unicode, and if file (default sys.stderr) is opened in binary mode, Python
will try to encode the unicode result with the ASCII codec and fail with a
UnicodeEncodeError:
warnings.showwarning(u'你好', DeprecationWarning, 'foo.py', 10)
foo.py:10: DeprecationWarning: 你好
with open('err.log', 'wb') as f:
... warnings.showwarning(u'你好', DeprecationWarning, 'foo.py', 10, file=f)
...
Traceback (most recent call last):
File stdin, line 2, in module
File /usr/lib/python2.7/warnings.py, line 30, in _show_warning
file.write(formatwarning(message, category, filename, lineno, line))
UnicodeEncodeError: 'ascii' codec can't encode characters in position 31-32:
ordinal not in range(128)
--
nosy: +ezio.melotti
resolution: - not a bug
stage: - resolved
status: open - closed
type: crash - behavior
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23637
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23623] Python 3.5 docs need to clarify how to set PATH, etc
Changes by Ezio Melotti ezio.melo...@gmail.com: -- nosy: +ezio.melotti ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23623 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23642] Interaction of ModuleSpec and C Extension Modules
Brett Cannon added the comment: The PEP is now live: https://www.python.org/dev/peps/pep-0489/ . If you want to participate in the discussion it will probably happen on the import-sig. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23642 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21518] Expose RegUnLoadKey in winreg
Martin v. Löwis added the comment: The naming of the function needs discussion. I think it should be UnLoadKey, as the API function behind it is RegUnLoadKey (not RegUnloadKey). It may be illogical(*) that the function is called that way in the API, but it would add confusion if Python called it differently. (*) It's UnlockFile that pairs LockFile, and UnmapViewOfFile that matches MapViewOfFile. OTOH, (undocumented) UTRegister is paired with UTUnRegister, and GlobalWire with GlobalUnWire. -- nosy: +loewis title: Expose RegUnloadKey in winreg - Expose RegUnLoadKey in winreg ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21518 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23658] multiprocessing: string arg to SystemExit
Davin Potts added the comment: Are you looking at the current code in the 2.7 branch or just at the code committed in https://hg.python.org/cpython/rev/da5b370f41a1 (done as part of issue13854)? The code in 2.7 has since been updated and appears to use an exitcode of 1 whenever a str is provided as the arg to SystemExit. Are you observing something different in practice? -- nosy: +davin ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23658 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23614] Opaque error message on UTF-8 decoding to surrogates
Chris Angelico added the comment: Nice document. Is that actually how Python's decoder checks things? Does the decoder have different definitions of valid continuation byte based on the lead byte? If that's the case... well, ten out of ten for complying with the spec, to be sure, but unfortunately it leads to some opaque error messages! I haven't looked into the code even a little bit, but would it be possible to have a specific error message attached to certain invalid continuation bytes? * E0 followed by 80..9F: non-shortest form * ED followed by A0..BF: surrogate * F4 followed by 90..BF: outside defined range If that's too hard, it'd at least be helpful to point out that the invalid continuation byte is not the same as the byte 0x?? in position ? - the rejection here is actually of the B4 that follows it. How does this look? UnicodeDecodeError: 'utf-8' codec can't decode byte 0xed in position 0: invalid continuation byte 0xb4 for this start byte (BTW, I think Pike's decoder just always emits two bytes, no matter what the actual errant stream (after all, there's no way to know how many bytes ought to have been one character, when there's an error in it). So it's incomplete, yes, but when you're dealing with wrong data, completeness isn't all that possible anyway.) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23614 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23645] Incorrect doc for __getslice__
Roundup Robot added the comment: New changeset 3b91d834160f by Benjamin Peterson in branch '2.7': the default is sys.maxsize not sys.maxint (closes #23645) https://hg.python.org/cpython/rev/3b91d834160f -- nosy: +python-dev resolution: - fixed stage: patch review - resolved status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23645 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2052] Allow changing difflib._file_template character encoding.
Berker Peksag added the comment: Here is an updated patch. Thanks for the review, Serhiy. I will open a new issue for the HTML 5 part of the patch. -- Added file: http://bugs.python.org/file38474/issue2052_v2.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2052 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23655] Memory corruption using pickle over pipe to subprocess
Serhiy Storchaka added the comment: Could you please try to minimize you data and try to reproduce an issue without using third-party modules if this is possible? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23655 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2211] Cookie.Morsel interface needs update
Changes by Demian Brecht demianbre...@gmail.com: Added file: http://bugs.python.org/file38476/issue2211_4.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2211 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23655] Memory corruption using pickle over pipe to subprocess
John Nagle added the comment: minimize you data - that's a big job here. Where are the tests for pickle? Is there one that talks to a subprocess over a pipe? Maybe I can adapt that. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23655 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23637] Warnings error with non-ascii chars.
Serhiy Storchaka added the comment: Here is a patch that tries to coerce non-ascii filename and line to unicode using appropriate encoding if it is needed and possible. If it is not possible, the warning just gets lost, as in the case of IO error. -- keywords: +patch stage: - patch review Added file: http://bugs.python.org/file38475/warnings_unicode.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23637 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23614] Opaque error message on UTF-8 decoding to surrogates
Ezio Melotti added the comment: Nice document. Is that actually how Python's decoder checks things? Yes, Python follows the Unicode standard. * E0 followed by 80..9F: non-shortest form * ED followed by A0..BF: surrogate * F4 followed by 90..BF: outside defined range If you get a decode error while using UTF-8, it means that you are trying to decode something that is not (valid) UTF-8. I can see 3 situations where this might happen: 1) the input is using a different encoding; 2) the input is corrupted; 3) the input is using an encoding similar to UTF-8 (e.g. CESU-8); In the first two cases additional information about continuation bytes are meaningless and misleading (there's no such thing as short form or surrogates in e.g. ASCII). In the third case (which is actually a special case of 1), mentioning surrogates and perhaps non-shortest form might be useful if the developer is intimately familiar with UTF-8 and Unicode since he might suspect that the input is actually CESU-8 or the text has been encoded by an outdated encoder that follows the RFC 2044 specs from 1996. How does this look? UnicodeDecodeError: 'utf-8' codec can't decode byte 0xed in position 0: invalid continuation byte 0xb4 for this start byte Something similar would be ok with me, assuming is easy to implement in the code. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23614 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23655] Memory corruption using pickle over pipe to subprocess
John Nagle added the comment: Or just use pickle._Pickler instead of pickle.Pickler and like (implementation detail!). Tried that. Changed my own code as follows: 25a26 71,72c72,73 self.reader = pickle.Unpickler(self.proc.stdout)# set up reader self.writer = pickle.Pickler(self.proc.stdin,kpickleprotocolversion) --- self.reader = pickle._Unpickler(self.proc.stdout)# set up reader self.writer = pickle._Pickler(self.proc.stdin,kpickleprotocolversion 125,126c126,127 self.reader = pickle.Unpickler(self.datain) # set up reader self.writer = pickle.Pickler(self.dataout,kpickleprotocolversion) --- self.reader = pickle._Unpickler(self.datain) # set up reader self.writer = pickle._Pickler(self.dataout,kpickleprotocolversion) Program runs after those changes. So it looks like CPickle has a serious memory corruption problem. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23655 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23643] Segmentation Fault with Large Simple Function
Ned Deily added the comment: (Closed by submitter.) -- nosy: +ned.deily resolution: - duplicate stage: - resolved superseder: - stack overflow evaluating eval(() * 3) ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23643 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23187] Segmentation fault, possibly asyncio related
Michael Goldish added the comment:
I caught another crash just now, this time in update_refs(). A stack trace is
attached below. I still think this is the same issue.
static void
update_refs(PyGC_Head *containers)
{
PyGC_Head *gc = containers-gc.gc_next;
67382D60 mov rdx,qword ptr [rcx]
for (; gc != containers; gc = gc-gc.gc_next) {
67382D63 cmp rdx,rcx
67382D66 je update_refs+28h (67382D88h)
67382D68 nop dword ptr [rax+rax]
assert(_PyGCHead_REFS(gc) == GC_REACHABLE);
_PyGCHead_SET_REFS(gc, Py_REFCNT(FROM_GC(gc)));
67382D70 and qword ptr [rdx+10h],1
Here rdx is 0, which means we're writing to 0x10, probably because the member
gc_refs is at offset 0x10 of struct PyGC_Head.gc. So I suppose
containers-gc.gc_next was NULL.
(In case this is helpful: somehow, possibly due to compiler optimizations,
Visual Studio claims that containers-gc.gc_next is 0x34. I'm not sure what to
make of this. It also claims that containers-gc.gc_prev-gc.gc_next is 0x3e,
and that containers-gc.gc_prev-gc.gc_prev-gc.gc_next is 0x3e, and so on...
gc_prev always seems fine and gc_next is always 0x3e, except for the first one
which is 0x34. I'm attaching a screenshot to make this clearer.)
Stack trace (Python 3.4.2, 64 bit, Windows):
python34.dll!update_refs(_gc_head * containers=0x676af8e0) Line 345
python34.dll!collect(int generation=-290088656, __int64 *
n_collected=0xf166e920, __int64 * n_uncollectable=0x,
int nofail=0) Line 969
python34.dll!collect_with_callback(int generation=-290088656) Line 1141
python34.dll!_PyObject_GC_Malloc(unsigned __int64 basicsize=4046951880) Line
1739
python34.dll!_PyObject_GC_New(_typeobject * tp=0x01c624f0) Line 1749
python34.dll!PyList_New(__int64 size=0) Line 159 + 0xc bytes
python34.dll!PyEval_EvalFrameEx(_frame * f=0xbab65b08, int
throwflag=-244913096) Line 2346
python34.dll!fast_function(_object * func=0x0003, _object * * *
pp_stack=0xf77684e0, int n=102445400, int na=1732453353, int nk=0)
Line 4332
python34.dll!call_function(_object * * * pp_stack=0xf166ec29, int
oparg=131) Line 4260
python34.dll!PyEval_EvalFrameEx(_frame * f=0x601cbd68, int
throwflag=-244912600) Line 2838
python34.dll!fast_function(_object * func=0x0003, _object * * *
pp_stack=0xf7768f28, int n=56017240, int na=1732453353, int nk=0) Line
4332
python34.dll!call_function(_object * * * pp_stack=0xf166ee19, int
oparg=131) Line 4260
python34.dll!PyEval_EvalFrameEx(_frame * f=0xc65ff8f8, int throwflag=0)
Line 2838
python34.dll!PyEval_EvalCodeEx(_object * _co=0x0002, _object *
globals=0x0002, _object * locals=0x, _object * *
args=0x0358d248, int argcount=2, _object * * kws=0x01c50060,
int kwcount=0, _object * * defs=0x, int defcount=0, _object *
kwdefs=0x, _object * closure=0x) Line 3585 +
0xa bytes
python34.dll!function_call(_object * func=0x0355f048, _object *
arg=0xf12f7688, _object * kw=0x) Line 638 + 0x45 bytes
python34.dll!PyObject_Call(_object * func=0xf12f7688, _object *
arg=0xf11c4d08, _object * kw=0xf4e9ba58) Line 2068
python34.dll!ext_do_call(_object * func=0x0355f048, _object * * *
pp_stack=0xf166f0d9, int flags=-200649216, int na=1, int nk=0) Line
4558 + 0xe bytes
python34.dll!PyEval_EvalFrameEx(_frame * f=0xf40a5458, int
throwflag=-244911400) Line 2879
python34.dll!fast_function(_object * func=0x0001, _object * * *
pp_stack=0xf1380f98, int n=45993224, int na=1732453353, int nk=0) Line
4332
python34.dll!call_function(_object * * * pp_stack=0xf166f2c9, int
oparg=131) Line 4260
python34.dll!PyEval_EvalFrameEx(_frame * f=0xd6922548, int throwflag=0)
Line 2838
python34.dll!PyEval_EvalCodeEx(_object * _co=0x0002, _object *
globals=0x0002, _object * locals=0x, _object * *
args=0x02334200, int argcount=2, _object * * kws=0x01c50060,
int kwcount=0, _object * * defs=0x, int defcount=0, _object *
kwdefs=0x, _object * closure=0x) Line 3585 +
0xa bytes
python34.dll!function_call(_object * func=0x02bdcbf8, _object *
arg=0xe41ef808, _object * kw=0x) Line 638 + 0x45 bytes
python34.dll!PyObject_Call(_object * func=0xe41ef808, _object *
arg=0xef4ad308, _object * kw=0xdeda7148) Line 2068
python34.dll!ext_do_call(_object * func=0x02bdcbf8, _object * * *
pp_stack=0xf166f589, int flags=-280305184, int na=0, int nk=0) Line
4558 + 0xe bytes
python34.dll!PyEval_EvalFrameEx(_frame * f=0xef4ae048, int
throwflag=-244910200) Line 2879
[issue23529] Limit decompressed data when reading from LZMAFile and BZ2File
Nikolaus Rath added the comment: On Mar 06 2015, Martin Panter rep...@bugs.python.org wrote: Still to do: Need to find a better home for the _DecompressReader and _BaseStream classes. Currently it lives in “lzma”, but apparently it is possible for any of the gzip, bz2, lzma modules to not be importable, so it would have to live elsewhere. Yes. Possible options are the io module, or a brand new internal module (e.g. Lib/_compression.py). Thoughts? I think a new internal module would be the right choice, but I don't know what needs to be done to properly add it to the build system. So for now I'd just put it in the io module and wait for a core committer to complain :-). Best, -Nikolaus -- GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F »Time flies like an arrow, fruit flies like a Banana.« -- title: Limit decompressed data when reading from LZMAFile, BZ2File, GzipFile - Limit decompressed data when reading from LZMAFile and BZ2File ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23529 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23661] Setting a exception side_effect on a mock from create_autospec does not work
Ignacio Rossi added the comment: The problem only affects autospecced functions. Apparently, the problem lies here (all code excerpts from Lib/unittest/mock.py): - When autospeccing functions, the Mock._mock_delegate field is populated at the end of _setup_func (and its the only place I found where the delegate is set): 197 def _setup_func(funcopy, mock): [...] 237 mock._mock_delegate = funcopy - Mock.side_effect is a property, and proxies the get/set to _mock_delegate when it exists, and on the way out does not detect the exception and tries to make an _IterMock out of it and everything explodes. 504 def __get_side_effect(self): 505 delegated = self._mock_delegate 506 if delegated is None: 507 return self._mock_side_effect 508 sf = delegated.side_effect 509 if sf is not None and not callable(sf) and not isinstance(sf, _MockIter): 510 sf = _MockIter(sf) I've attached a patch which adds a test for this use case, and a proposed fix. Hope it helps :) -- keywords: +patch Added file: http://bugs.python.org/file38479/autospec_exception.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23661 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23187] Segmentation fault, possibly asyncio related
Guido van Rossum added the comment:
So what do you make of this?
PyObject_GC_Malloc(unsigned __int64 basicsize=4046951880)
That's nearly 4 GB. I somehow doubt your app is actually trying to allocate
that much memory -- maybe the type object a few lines below in the stack is
overwritten, or maybe the compiler is lying about everything. Is there any
chance you can run an unoptimized build?
On Fri, Mar 13, 2015 at 7:27 PM, Michael Goldish rep...@bugs.python.org
wrote:
Michael Goldish added the comment:
I caught another crash just now, this time in update_refs(). A stack trace
is attached below. I still think this is the same issue.
static void
update_refs(PyGC_Head *containers)
{
PyGC_Head *gc = containers-gc.gc_next;
67382D60 mov rdx,qword ptr [rcx]
for (; gc != containers; gc = gc-gc.gc_next) {
67382D63 cmp rdx,rcx
67382D66 je update_refs+28h (67382D88h)
67382D68 nop dword ptr [rax+rax]
assert(_PyGCHead_REFS(gc) == GC_REACHABLE);
_PyGCHead_SET_REFS(gc, Py_REFCNT(FROM_GC(gc)));
67382D70 and qword ptr [rdx+10h],1
Here rdx is 0, which means we're writing to 0x10, probably because the
member gc_refs is at offset 0x10 of struct PyGC_Head.gc. So I suppose
containers-gc.gc_next was NULL.
(In case this is helpful: somehow, possibly due to compiler optimizations,
Visual Studio claims that containers-gc.gc_next is 0x34. I'm not sure what
to make of this. It also claims that containers-gc.gc_prev-gc.gc_next is
0x3e, and that containers-gc.gc_prev-gc.gc_prev-gc.gc_next is 0x3e, and
so on... gc_prev always seems fine and gc_next is always 0x3e, except for
the first one which is 0x34. I'm attaching a screenshot to make this
clearer.)
Stack trace (Python 3.4.2, 64 bit, Windows):
python34.dll!update_refs(_gc_head * containers=0x676af8e0) Line
345
python34.dll!collect(int generation=-290088656, __int64 *
n_collected=0xf166e920, __int64 *
n_uncollectable=0x, int nofail=0) Line 969
python34.dll!collect_with_callback(int generation=-290088656) Line 1141
python34.dll!_PyObject_GC_Malloc(unsigned __int64 basicsize=4046951880)
Line 1739
python34.dll!_PyObject_GC_New(_typeobject * tp=0x01c624f0) Line
1749
python34.dll!PyList_New(__int64 size=0) Line 159 + 0xc bytes
python34.dll!PyEval_EvalFrameEx(_frame * f=0xbab65b08, int
throwflag=-244913096) Line 2346
python34.dll!fast_function(_object * func=0x0003, _object * *
* pp_stack=0xf77684e0, int n=102445400, int na=1732453353, int
nk=0) Line 4332
python34.dll!call_function(_object * * * pp_stack=0xf166ec29, int
oparg=131) Line 4260
python34.dll!PyEval_EvalFrameEx(_frame * f=0x601cbd68, int
throwflag=-244912600) Line 2838
python34.dll!fast_function(_object * func=0x0003, _object * *
* pp_stack=0xf7768f28, int n=56017240, int na=1732453353, int
nk=0) Line 4332
python34.dll!call_function(_object * * * pp_stack=0xf166ee19, int
oparg=131) Line 4260
python34.dll!PyEval_EvalFrameEx(_frame * f=0xc65ff8f8, int
throwflag=0) Line 2838
python34.dll!PyEval_EvalCodeEx(_object * _co=0x0002, _object *
globals=0x0002, _object * locals=0x, _object *
* args=0x0358d248, int argcount=2, _object * *
kws=0x01c50060, int kwcount=0, _object * * defs=0x,
int defcount=0, _object * kwdefs=0x, _object *
closure=0x) Line 3585 + 0xa bytes
python34.dll!function_call(_object * func=0x0355f048, _object *
arg=0xf12f7688, _object * kw=0x) Line 638 + 0x45
bytes
python34.dll!PyObject_Call(_object * func=0xf12f7688, _object *
arg=0xf11c4d08, _object * kw=0xf4e9ba58) Line 2068
python34.dll!ext_do_call(_object * func=0x0355f048, _object * * *
pp_stack=0xf166f0d9, int flags=-200649216, int na=1, int nk=0)
Line 4558 + 0xe bytes
python34.dll!PyEval_EvalFrameEx(_frame * f=0xf40a5458, int
throwflag=-244911400) Line 2879
python34.dll!fast_function(_object * func=0x0001, _object * *
* pp_stack=0xf1380f98, int n=45993224, int na=1732453353, int
nk=0) Line 4332
python34.dll!call_function(_object * * * pp_stack=0xf166f2c9, int
oparg=131) Line 4260
python34.dll!PyEval_EvalFrameEx(_frame * f=0xd6922548, int
throwflag=0) Line 2838
python34.dll!PyEval_EvalCodeEx(_object * _co=0x0002, _object *
globals=0x0002, _object * locals=0x, _object *
* args=0x02334200, int argcount=2, _object * *
kws=0x01c50060, int kwcount=0, _object * * defs=0x,
int defcount=0, _object * kwdefs=0x, _object *
closure=0x) Line 3585 + 0xa bytes
python34.dll!function_call(_object * func=0x02bdcbf8,
[issue12641] Remove -mno-cygwin from distutils
Ned Deily added the comment: Michael, this issue is closed and the changes have long since been released. Comments here will likely be ignored. Please open a new issue describing the problem you are seeing and under what environment, with exact steps to reproduce it. -- nosy: +ned.deily ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12641 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23187] Segmentation fault, possibly asyncio related
Michael Goldish added the comment: I don't see a reason to assume the machine was running out of memory. After each crash the process was kept alive by one of those Windows crash dialogs - the process terminated unexpectedly or similar. I could see exactly how much memory was occupied by the process and it was never a significant percentage of the machine's total memory. It sometimes exceeded 2GB but I'm using a 64 bit build so this shouldn't be a problem. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23187 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23246] distutils fails to locate vcvarsall with Visual C++ Compiler for Python
Jason R. Coombs added the comment: The setuptools code is licensed under the ZPL and PSF licenses. If either of those licenses are insufficient, I'm happy to release my contribution in that file to the public domain. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23246 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23624] str.center inconsistent with format ^
Changes by Raymond Hettinger raymond.hettin...@gmail.com: -- assignee: - rhettinger ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23624 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23187] Segmentation fault, possibly asyncio related
Michael Goldish added the comment: Why do you consider that it's the same issue? Because it's a very similar stack trace, the crash itself is in subtype_dealloc(), it happens once every few days, asyncio is involved, and the issue is relatively new - I couldn't find another bug report with a similar description. I have core dumps if you're interested but am not sure where to upload them. You can try to use faulthandler to write the traceback of all Python threads where the crash occurred I've enabled faulthandler, the program crashed, and nothing was written to the file. For reference, I caused an artificial crash in the same Python program (using ctypes) and the stack traces *were* written. I verified this several times, both with the real crash and the artificial one, and the real crash always left the file empty. I guess that Visual Studio can load such minidump files. I've already looked at one of my dumps in Visual Studio but couldn't make sense of what was going on. The structures involved seemed to contain data that made no sense to me (e.g. some pointers had values like 0x10, which usually makes no sense for a pointer and seems to indicate memory corruption). Did you see some errors or warnings when running your application with asyncio in debug mode? No, but I'll try. I doubt the problem is in asyncio itself because it's mostly written in Python. This looks like a garbage collection issue. Can you maybe send me the source code of your application? I'll ask to see if that's permissible, but either way you will probably have a hard time reproducing the crash - the program is a server which maintains active TCP connections to more than 1000 client instances, writes large text files in background threads, pickles large objects... The frequency of the crash seems to be correlated to how busy the server is. Without a proper setup it'll be difficult to make it busy. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23187 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23639] Not documented special names
Terry J. Reedy added the comment: Idlelib.rpc uses the somewhat arbitrary *strings* '__methods__' and '__attributes__' as private signals in its protocol. They are not object or attribute names, special or otherwise, in the sense used here. The use of '__methods__' in idlelib.rpc (line 176) harkens back to pre 2.2 days when __methods__ was officially used in the type introspection api. That class attribute was a list of method names supported by the object.. See PEP 252. Its complement was __members__, a list of non-method attributes (other than itself, I presume). rpc uses '__attributes__' instead. -- nosy: +terry.reedy ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23639 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23556] Scope for raise without argument is different in Python 2 and 3
Changes by Ethan Furman et...@stoneleaf.us: -- nosy: +ethan.furman ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23556 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23187] Segmentation fault, possibly asyncio related
Guido van Rossum added the comment: Are we all in agreement that it's probably somehow running out of memory? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23187 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9679] unicode DNS names in urllib, urlopen
Changes by Demian Brecht demianbre...@gmail.com: -- stage: - patch review ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9679 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9679] unicode DNS names in urllib, urlopen
Changes by Berker Peksag berker.pek...@gmail.com: -- nosy: +berker.peksag versions: +Python 3.5 -Python 3.3, Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9679 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23661] Setting a exception side_effect on a mock from create_autospec does not work
New submission from Ignacio Rossi:
The following fails on python 3.4.2, 3.4.3 and 3.5.0a2 (downloaded from
python.org and compiled on Ubuntu 14.04).
The same happens when using mock.patch with autospec=True.
from unittest.mock import create_autospec
def function():
... pass
...
mock = create_autospec(function)
mock.side_effect = ValueError('MyError')
mock()
Traceback (most recent call last):
File stdin, line 1, in module
File string, line 3, in function
File /usr/local/lib/python3.5/unittest/mock.py, line 910, in __call__
return _mock_self._mock_call(*args, **kwargs)
File /usr/local/lib/python3.5/unittest/mock.py, line 963, in _mock_call
effect = self.side_effect
File /usr/local/lib/python3.5/unittest/mock.py, line 510, in
__get_side_effect
sf = _MockIter(sf)
File /usr/local/lib/python3.5/unittest/mock.py, line 351, in __init__
self.obj = iter(obj)
TypeError: 'ValueError' object is not iterable
But, on Python 3.3.5, 3.4.0, or when the mock is created via Mock(), for
instance, the exception is raised as expected:
[...]
mock()
Traceback (most recent call last):
File stdin, line 1, in module
File string, line 3, in function
File /usr/lib/python3.4/unittest/mock.py, line 885, in __call__
return _mock_self._mock_call(*args, **kwargs)
File /usr/lib/python3.4/unittest/mock.py, line 941, in _mock_call
raise effect
ValueError: MyError
--
components: Library (Lib)
messages: 238064
nosy: Ignacio Rossi
priority: normal
severity: normal
status: open
title: Setting a exception side_effect on a mock from create_autospec does not
work
type: behavior
versions: Python 3.4, Python 3.5
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23661
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23042] Python 2.7.9 ctypes module doesn't build on FreeBSD x86
Danya Alexeyevsky added the comment: Faced the same problem on FreeBSD. I confirm: applying the reversed patch fixed it. Thanks! -- nosy: +Danya.Alexeyevsky ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23042 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9679] unicode DNS names in urllib, urlopen
Demian Brecht added the comment: Here's a simple patch that adds functionality matching that in http.client to urllib.request. As pointed out by John, I see no reason why urllib and http.client shouldn't have consistent handling if IDNs independent of IRIs (although IRI encoding would be a nice addition as well). -- keywords: +patch Added file: http://bugs.python.org/file38477/issue9679.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9679 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12641] Remove -mno-cygwin from distutils
Michael Clerx added the comment: I'm seeing this bug in 2.7.9. The reason seems to be that the version detection doesn't work... This snippet: out = os.popen(gcc_exe + ' -dumpversion', 'r') out_string = out.read() returns an empty out_string, causing gcc_version = None '4' Maybe the '4' check could be restructured to see None as probably modern instead of probably very out of date ? -- nosy: +Michael.Clerx ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12641 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22852] urllib.parse wrongly strips empty #fragment
Changes by Demian Brecht demianbre...@gmail.com: -- stage: - patch review ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22852 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23051] multiprocessing.pool methods imap()[_unordered()] deadlock
Serhiy Storchaka added the comment: Thank you for your contribution Alon and Davin. -- resolution: - fixed stage: patch review - resolved status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23051 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23138] cookiejar parses cookie value as int with empty name-value pair and Expires
Roundup Robot added the comment: New changeset 44c1c0cbdc06 by Serhiy Storchaka in branch '2.7': Issue #23138: Fixed parsing cookies with absent keys or values in cookiejar. https://hg.python.org/cpython/rev/44c1c0cbdc06 New changeset c1abcbcfefab by Serhiy Storchaka in branch '3.4': Issue #23138: Fixed parsing cookies with absent keys or values in cookiejar. https://hg.python.org/cpython/rev/c1abcbcfefab New changeset 7cc7c794d1cb by Serhiy Storchaka in branch 'default': Issue #23138: Fixed parsing cookies with absent keys or values in cookiejar. https://hg.python.org/cpython/rev/7cc7c794d1cb -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23138 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2211] Cookie.Morsel interface needs update
Changes by Demian Brecht demianbre...@gmail.com: Added file: http://bugs.python.org/file38466/issue2211_3.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2211 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23655] Memory corruption using pickle over pipe to subprocess
Serhiy Storchaka added the comment: sys.modules['_pickle'] del sys.modules['pickle'] # if exists import pickle Or just use pickle._Pickler instead of pickle.Pickler and like (implementation detail!). -- nosy: +alexandre.vassalotti, pitrou, serhiy.storchaka stage: - test needed type: - behavior versions: +Python 3.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23655 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23655] Memory corruption using pickle over pipe to subprocess
New submission from John Nagle:
I'm porting a large, working system from Python 2 to Python 3, using six, so
the same code works with both. One part of the system works a lot like the
multiprocessing module, but predates it. It launches child processes with
Popen and talks to them using pickle over stdin/stdout as pipes. Works
fine under Python 2, and has been working in production for years.
Under Python 3, I'm getting errors that indicate memory corruption:
Fatal Python error: GC object already tracked
Current thread 0x1a14 (most recent call first):
File C:\python34\lib\site-packages\pymysql\connections.py, line 411
in description
File C:\python34\lib\site-packages\pymysql\connections.py, line 1248
in _get_descriptions
File C:\python34\lib\site-packages\pymysql\connections.py, line 1182
in _read_result_packet
File C:\python34\lib\site-packages\pymysql\connections.py, line 1132
in read
File C:\python34\lib\site-packages\pymysql\connections.py, line 929
in _read_query_result
File C:\python34\lib\site-packages\pymysql\connections.py, line 768
in query
File C:\python34\lib\site-packages\pymysql\cursors.py, line 282 in
_query
File C:\python34\lib\site-packages\pymysql\cursors.py, line 134 in
execute
File C:\projects\sitetruth\domaincacheitem.py, line 128 in select
File C:\projects\sitetruth\domaincache.py, line 30 in search
File C:\projects\sitetruth\ratesite.py, line 31 in ratedomain
File C:\projects\sitetruth\RatingProcess.py, line 68 in call
File C:\projects\sitetruth\subprocesscall.py, line 140 in docall
File C:\projects\sitetruth\subprocesscall.py, line 158 in run
File C:\projects\sitetruth\RatingProcess.py, line 89 in main
File C:\projects\sitetruth\RatingProcess.py, line 95 in module
That's clear memory corruption.
Also,
File C:\projects\sitetruth\InfoSiteRating.py, line 200, in scansite
if len(self.badbusinessinfo) 0 : # if bad stuff
NameError: name 'len' is not defined
There are others, but those two should be impossible to cause from Python
source.
I've done the obvious stuff - deleted all .pyc files and Python cache
directories. All my code is in Python. Every library module came in via pip,
into a clean Python 3.4.3 (32 bit) installation on Win7/x86-64.
Currently installed packages (via pip list)
beautifulsoup4 (4.3.2)
dnspython3 (1.12.0)
html5lib (0.999)
pip (6.0.8)
PyMySQL (0.6.6)
pyparsing (2.0.3)
setuptools (12.0.5)
six (1.9.0)
Nothing exotic there. The project has zero local C code; any C code came
from the Python installation or the above packages, most of which are pure
Python.
It all works fine with Python 2.7.9. Everything else in the program seems
to be working fine under both 2.7.9 and 3.4.3, until subprocesses are involved.
What's being pickled is very simple; no custom objects, although Exception
types are sometimes pickled if the subprocess raises an exception.
Pickler and Unpickler instances are being reused here. A message is pickled,
piped to the subprocess, unpickled, work is done, and a response comes back
later via the return pipe. A send looks like:
self.writer.dump(args) # send data
self.dataout.flush()# finish output
self.writer.clear_memo()# no memory from cycle to cycle
and a receive looks like:
result = self.reader.load() # read and return from child
self.reader.memo = {} # no memory from cycle to cycle
Those were the recommended way to reset pickle for new traffic years ago.
(You have to clear the receive side as well as the send side, or the dictionary
of saved objects grows forever.) My guess is that there's something about
reusing pickle instances that botches memory uses in CPython 3's C code
for cpickle. That should work, though; the multiprocessing module works
by sending pickled data over pipes.
The only code difference between Python 2 and 3 is that under Python 3 I have
to use sys.stdin.buffer and sys.stdout.buffer as arguments to Pickler and
Unpickler. Otherwise they complain that they're getting type str.
Unfortunately, I don't have an easy way to reproduce this bug yet.
Is there some way to force the use of the pure Python pickle module under
Python 3? That would help isolate the problem.
John Nagle
--
components: Library (Lib)
messages: 238009
nosy: nagle
priority: normal
severity: normal
status: open
title: Memory corruption using pickle over pipe to subprocess
versions: Python 3.4
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23655
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23138] cookiejar parses cookie value as int with empty name-value pair and Expires
Serhiy Storchaka added the comment:
As side effect the parsing is now twice faster.
$ ./python -m timeit -s from http.cookiejar import parse_ns_headers --
parse_ns_headers('foo=bar; Expires=Thu, 01 Jan 1970 00:00:10 GMT')
Before: 1000 loops, best of 3: 976 usec per loop
After: 1000 loops, best of 3: 537 usec per loop
--
resolution: - fixed
stage: commit review - resolved
status: open - closed
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23138
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23051] multiprocessing.pool methods imap()[_unordered()] deadlock
Roundup Robot added the comment: New changeset 525ccfcc55f7 by Serhiy Storchaka in branch '3.4': Issue #23051: multiprocessing.Pool methods imap() and imap_unordered() now https://hg.python.org/cpython/rev/525ccfcc55f7 New changeset 7891d084a9ad by Serhiy Storchaka in branch 'default': Issue #23051: multiprocessing.Pool methods imap() and imap_unordered() now https://hg.python.org/cpython/rev/7891d084a9ad New changeset 311d52878a65 by Serhiy Storchaka in branch '2.7': Issue #23051: multiprocessing.Pool methods imap() and imap_unordered() now https://hg.python.org/cpython/rev/311d52878a65 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23051 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23639] Not documented special names
Changes by Ian Lee ianlee1...@gmail.com: -- nosy: +IanLee1521 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23639 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17797] Visual C++ 11.0 reports fileno(stdin) == 0 for non-console program
Jean-Charles Lefebvre added the comment: Well, just in case, I've attached the patch to apply against 3.5.0a2. -- Added file: http://bugs.python.org/file38469/python-3.5.0a2-fdvalidation.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17797 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23645] Incorrect doc for __getslice__
Alex Shkop added the comment: This is true. In ceval.c default value for end index is PY_SSIZE_T_MAX. Whereas in sysmodule.c defined: SET_SYS_FROM_STRING(maxsize, PyInt_FromSsize_t(PY_SSIZE_T_MAX)); SET_SYS_FROM_STRING(maxint, PyInt_FromLong(PyInt_GetMax())); I added a patch that fixes documentation. But please note, that __getslice__ is deprecated since Python 2.0 and is no longer supported in Python 3. Use __getitem__ instead. -- keywords: +patch nosy: +ashkop Added file: http://bugs.python.org/file38468/issue23645.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23645 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23539] Content-length not set for HTTP methods expecting body when body is None
James Rutherford added the comment: Hi all, apologies for the spam, but I just wanted to confirm that no-one is waiting on anything from me... I'm happy to consolidate the final minor points make the patch against python3 if that would simplify things. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23539 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23605] Use the new os.scandir() function in os.walk()
STINNER Victor added the comment: I don't understand your benchmark. Do you mean that os.walk() is slower with fast_bottom-up.patch because islink() is called or because I replaced for entry in scandir(top): with entry = next(scandir_it)? Are you testing the top-bottom or bottom-up? Here is a variant of my patch with for entry in scandir(top):. I would prefer to avoid this variant with a boolean to not catch OSError on the recursive call to walk() if next() has similar performances. -- Added file: http://bugs.python.org/file38471/fast_bottom-up-2.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23605 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23656] shutil.copyfile (or built-in open) releases sqlite3's transaction lock
New submission from Hiroaki Miyajima:
When copying sqlite3's database file by shutil.copyfile while holding its
transaction lock, its lock is released unexpectedly.
Attached is a sample program showing this. Inside,
process-1 and process-2 just do following steps respectively:
(1) open a sqlite dbfile
(2) execute('begin EXCLUSIVE transaction') to acquire lock
(3) copy the dbfile
(4) sleep some seconds for demonstration
(4) rollback() to release lock
if shutil.copyfile() is used as (3), process-2 gets a lock
while process-1 must hold a lock.
It seems that this happens if database file is opened with built-in open(),
which is used in shutil.copyfile.
In addition,
- With os.open(), there seems no problem.
- With linux cp command, it works correct.
- Using multi-thread instead of multi-process, it works expectedly.
I think this is a bug, but does someone explain this behaviour?
Thank you.
Below is the execute example of the attached on python 2.7.9.
--
# python Test_sqlite3_lock.py
Use function shutil_copyfile at 0x7fe810b22500
process-1 : Transaction lock acquired
process-2 : Transaction lock acquired
process-1 : Transaction lock released
process-2 : Transaction lock released
Use function low_copyfile at 0x7fe810b22578
process-1 : Transaction lock acquired
process-1 : Transaction lock released
process-2 : Transaction lock acquired
process-2 : Transaction lock released
Use function command_copyfile at 0x7fe810b225f0
process-1 : Transaction lock acquired
process-1 : Transaction lock released
process-2 : Transaction lock acquired
process-2 : Transaction lock released
--
--
files: Test_sqlite3_lock.py
messages: 238013
nosy: h-miyajima
priority: normal
severity: normal
status: open
title: shutil.copyfile (or built-in open) releases sqlite3's transaction lock
type: behavior
versions: Python 2.7
Added file: http://bugs.python.org/file38467/Test_sqlite3_lock.py
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23656
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9679] unicode DNS names in urllib, urlopen
Changes by Demian Brecht demianbre...@gmail.com: -- nosy: +demian.brecht ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9679 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23566] RFE: faulthandler.register() should support file descriptors
STINNER Victor added the comment: I commited a fix to repair Windows buildbots. Or we could reuse the file created by filename in subprocess? I tried to pass a file descriptor from the parent to the child process, but this option is complex. It's possible to pass a handle with close_fds=False, but then I have to recreate a file descriptor with a function from msvcrt.open_fshandle(). This solution looks complex for a simple unit test. Your solution looks simple. Would you like to work on a patch? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23566 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13290] get vars for object with __slots__
Serhiy Storchaka added the comment: Oh, this was incorrect example. The correct one is: $ ./python -m timeit -s from http.cookiejar import parse_ns_headers -- parse_ns_headers(['foo=bar; path=/; version=1; Expires=Thu, 01 Jan 1970 00:00:10 GMT']) Before: 1 loops, best of 3: 177 usec per loop After: 1 loops, best of 3: 104 usec per loop -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13290 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23566] RFE: faulthandler.register() should support file descriptors
Roundup Robot added the comment: New changeset 211e29335e72 by Victor Stinner in branch 'default': Issue #23566: Skip fd tests of test_faulthandler on Windows https://hg.python.org/cpython/rev/211e29335e72 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23566 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13290] get vars for object with __slots__
Changes by Serhiy Storchaka storch...@gmail.com: -- Removed message: http://bugs.python.org/msg238021 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13290 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23645] Incorrect doc for __getslice__
Gaëtan de Menten added the comment: Thanks for creating the patch! PS: I know it is deprecated. But I am stuck on Python2 for that project and my class inherits from numpy ndarray, so I do not have a choice... -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23645 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23654] infinite loop in faulthandler._stack_overflow
STINNER Victor added the comment: When the faulthandler module is compiled at -O3 (the default for non-debug builds) with a compiler that does tailcall optimization ... test_faulthandler pass with GCC 4.9.2 (Fedora 21) when faulthandler.c is compiled in -O3 mode. I guess that such config doesn't support tailcall optimization. I guess that this issue is currently specific to Intel C compiler (ICC), right? ... with a compiler that does tailcall optimization the Modules/faulthandler.c:stack_overflow() function may become an infinite loop that does not expand the stack. Hum, I didn't expect that I would be so hard to write a reliable stack overflow :-) I modified recently the function in issue #23433 to a different issue with void*/intptr_t conversion. My function writes into a buffer allocated on the stack. How the compiler avoids to allocate 4 KB for each loop iteration? It doesn't matter if the function is recursive or not, it only matters if each iteration allocates 4 KB on the stack. I'm writing into the buffer allocated on the stack to ensure that the buffer is really allocated. Maybe the pattern is too reliable? Can you try to mark the buffer with the volatile keyword? (See attached patch.) This keyword is already used in faulthandler_read_null() and faulthandler_sigfpe() to workaround issues with compiler optimizations. (See attached patch.) -- Added file: http://bugs.python.org/file38470/volatile.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23654 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23566] RFE: faulthandler.register() should support file descriptors
Changes by Christoph Sieghart s...@0x2a.at: -- nosy: -sigi ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23566 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23138] cookiejar parses cookie value as int with empty name-value pair and Expires
Serhiy Storchaka added the comment: Oh, this was incorrect example. The correct one is: $ ./python -m timeit -s from http.cookiejar import parse_ns_headers -- parse_ns_headers(['foo=bar; path=/; version=1; Expires=Thu, 01 Jan 1970 00:00:10 GMT']) Before: 1 loops, best of 3: 177 usec per loop After: 1 loops, best of 3: 104 usec per loop -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23138 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23605] Use the new os.scandir() function in os.walk()
Serhiy Storchaka added the comment: You fast_bottom-up looks awesome, but much more correct. This is what I meant when warned that correct implementations with scandir() will be complex. Could you please add a test based on my example (i.e. converting symlinks to a directory during walking) and may be other (creating new directory and adding it to the dirs list)? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23605 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23539] Content-length not set for HTTP methods expecting body when body is None
R. David Murray added the comment: It would...I've started twice to do the commit and gotten interrupted both times. Having a patch I can just apply would help. If you would be willing to also generate a patch for python3, that would be a help as well. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23539 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23491] PEP 441 - Improving Python Zip Application Support
Brett Cannon added the comment: Thanks for the patch, Paul! I touched up the formatting to be more PEP 8 compliant and fixed the tests as they initially failed for me (TESTFN didn't exist prior to you trying to make a directory, so it just failed; switched to tempfile.TemporaryDirectory instead). -- nosy: +brett.cannon resolution: - fixed stage: patch review - resolved status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23491 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23657] Don't do isinstance checks in zipapp
New submission from Brett Cannon: As it stand, zipapp's code checks for str and then otherwise assumes an object is a file-like object. It might work out better to do some duck typing and simply check if an object has 'read' and 'readline' attributes then it's a file-like object and otherwise that it's a path. Doing this would then potentially make it easier to use pathlib.Path through the module rather than the os module. -- assignee: pmoore components: Library (Lib) messages: 238030 nosy: brett.cannon, pmoore priority: normal severity: normal stage: needs patch status: open title: Don't do isinstance checks in zipapp type: enhancement versions: Python 3.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23657 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23491] PEP 441 - Improving Python Zip Application Support
Roundup Robot added the comment: New changeset d1e9f337fea1 by Brett Cannon in branch 'default': Issue #23491: Implement PEP 441: Improving Python Zip Application Support https://hg.python.org/cpython/rev/d1e9f337fea1 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23491 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23657] Don't do isinstance checks in zipapp
Paul Moore added the comment: That sounds reasonable. I'll have a look at this. The code was originally based on a similar pattern in the zipfile module, so maybe zipfile should be changed in the same way? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23657 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23605] Use the new os.scandir() function in os.walk()
Ben Hoyt added the comment: I don't understand your benchmark. Do you mean that os.walk() is slower with fast_bottom-up.patch because islink() is called or because I replaced for entry in scandir(top): with entry = next(scandir_it)? No, sorry, I was making two separate comments: 1) the code's gotten quite a bit more complex (and if it needs to be that way for correctness, I'm okay with that), and 2) I'm surprised at how fast it still is. Are you testing the top-bottom or bottom-up? My benchmark.py calls os.walk() with topdown=True, which is the default. I was testing the Python 3.4 version of os.walk() via listdir against your fast_bottom-up.patch. I'm keen to look into this a bit further, but it won't be today. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23605 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23605] Use the new os.scandir() function in os.walk()
STINNER Victor added the comment: Are you testing the top-bottom or bottom-up? My benchmark.py calls os.walk() with topdown=True, which is the default. Is it worth to mention in the os.walk() doc that topdown=False can be faster? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23605 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23660] Turtle left/right inverted when using different coordinates orientation
New submission from Andre Roberge: The turtle module using screen coordinates such that the vertical coordinate increases vertically on the screen. This orientation is different from the traditional orientation for graphics programs. It is possible to set coordinates with the normal orientation using the setworldcoordinates function, e.g. from turtle import * setworldcoordinates(0, 400, 400, 0) With the above choice, the y coordinate increases as the turtle moves down on the screen. However, doing so inverts the role of left() and right(). I have attached a diff file which introduces a change needed so that left() and right() behave correctly with this different orientation. -- files: turtle_diff.txt messages: 238037 nosy: aroberge priority: normal severity: normal status: open title: Turtle left/right inverted when using different coordinates orientation type: behavior versions: Python 3.4 Added file: http://bugs.python.org/file38473/turtle_diff.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23660 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23657] Don't do isinstance checks in zipapp
Paul Moore added the comment: Looks good. Would it be worth adding tests for providing pathlib.Path objects, or is that overkill? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23657 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23491] PEP 441 - Improving Python Zip Application Support
Paul Moore added the comment: Thanks Brett! Apologies about the TESTFN issue - I changed it (from TemporaryDirectory, in fact) based on one of the review comments - it looks like I didn't test the change properly. My mistake, thanks for catching it. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23491 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23657] Don't do isinstance checks in zipapp
Brett Cannon added the comment: Here is a proposed patch that does what I was thinking. What do you think? As for updating zipfile, it's possible but slightly risky because of backwards-compatibility. Since this is all-new code there is no worry about breaking someone's pre-existing code because they were relying on str being treated in a special way instead of the file-like object case. -- keywords: +patch stage: needs patch - patch review Added file: http://bugs.python.org/file38472/duck_typed_zipapp.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23657 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23643] Segmentation Fault with Large Simple Function
Changes by Brett Cannon br...@python.org: -- nosy: +brett.cannon ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23643 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23658] multiprocessing: string arg to SystemExit
New submission from Dan Nawrocki: It seems that the fix for 13854 (http://bugs.python.org/issue13854) actually tried to solve 2 issues: 1. handle non-integer, non-string arg to SystemExit 2. use exit code of 0 when the arg was a string The change involved for #2 seems to go against the documentation for sys.exit: ...In particular, sys.exit(some error message) is a quick way to exit a program when an error occurs. Indeed, python 2.7.5 appears to follow this behavior: [me@localhost ~]$ python -c 'raise SystemExit(bye)'; echo $? bye 1 Shouldn't the return code from a subprocess when using multiprocessing match the return code when called w/o the multiprocessing module? -- components: Library (Lib) messages: 238034 nosy: Dan Nawrocki priority: normal severity: normal status: open title: multiprocessing: string arg to SystemExit type: behavior versions: Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23658 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23659] csv.register_dialect doc string
New submission from Vladimir Ulupov: I use PyCharm. This IDE checks the signature functions. When I used the extra arguments to the function register_dialect - IDE reported error. This is what has generated IDE for this function. http://dumpz.org/1347592/ Apparently error here https://hg.python.org/cpython/file/44c1c0cbdc06/Modules/_csv.c#l1547 https://hg.python.org/cpython/file/c1abcbcfefab/Modules/_csv.c#l1590 https://docs.python.org/2/library/csv.html?highlight=csv#csv.register_dialect https://docs.python.org/3/library/csv.html?highlight=csv#csv.register_dialect -- assignee: docs@python components: Documentation messages: 238035 nosy: Vladimir Ulupov, docs@python priority: normal severity: normal status: open title: csv.register_dialect doc string versions: Python 2.7, Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23659 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23659] csv.register_dialect doc string
Changes by Skip Montanaro s...@pobox.com: -- nosy: +skip.montanaro ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23659 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23659] csv.register_dialect doc string
R. David Murray added the comment: Could you clarify what bug in python you are reporting? -- nosy: +r.david.murray ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23659 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16318] FTP_TLS in ftplib not supporting prot_p storlines in FTP7.5
Nathan Cox added the comment: I have this issue intermittently with a server that I connect to, but the issue (at least for me) seems to be the minBytesPerSecond configuration triggering a remote disconnect of the connection. -- nosy: +Nathan Cox ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16318 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23539] Content-length not set for HTTP methods expecting body when body is None
James Rutherford added the comment: Ok I'll have a go at a consolidated python3 patch tomorrow. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23539 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
