[issue22719] os.path.isfile os.path.exists bug in while loop
Aaron added the comment: Python 3.3.0, Windows 7, both 64 bit. Has it been resolved with the newer version, then? On Mon, Nov 3, 2014 at 11:15 PM, Zachary Ware rep...@bugs.python.org wrote: Zachary Ware added the comment: Aaron, what version of Python are you using on what version of Windows? Also, 32 or 64 bit on both? I can't reproduce this with any Python 3.3.6 or newer on 64-bit Windows 8.1. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22719 ___ -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22719 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22719] os.path.isfile os.path.exists bug in while loop
Zachary Ware added the comment: I haven't built 3.3.0 again yet to try to reproduce with it, but there have been enough bug and security fixes in the more recent 3.3 releases that I'd strongly advise updating on general principle and seeing if this issue goes away. If not to 3.4.2, at least to 3.3.5 (the last 3.3 version to have a Windows installer). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22719 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22719] os.path.isfile os.path.exists bug in while loop
Zachary Ware added the comment: I have had a chance to build 3.3.0 and I was able to reproduce the bug with it, so it is in fact fixed in later versions. -- resolution: - out of date stage: - resolved status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22719 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22719] os.path.isfile os.path.exists bug in while loop
Zachary Ware added the comment: Aaron, what version of Python are you using on what version of Windows? Also, 32 or 64 bit on both? I can't reproduce this with any Python 3.3.6 or newer on 64-bit Windows 8.1. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22719 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22719] os.path.isfile os.path.exists bug in while loop
Changes by Aaron hosfor...@gmail.com: -- title: os.path.isfile os.path.exists but in while loop - os.path.isfile os.path.exists bug in while loop ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22719 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22719] os.path.isfile os.path.exists bug in while loop
R. David Murray added the comment: Interesting bug. The obvious difference between the two cases is that in the += version the address of the string pointing to the filepath doesn't change, whereas when you use a temp variable it does (there's an optimization in += that reuses the same memory location if possible). It looks like something is seeing that repeated addresses and returning the same result as the last time that address was passed, which is wrong. I don't see anything obvious in os module. Although I can't rule out a Python bug, since this works fine on unix I suspect this is a Windows CRT bug. -- nosy: +r.david.murray ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22719 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22719] os.path.isfile os.path.exists bug in while loop
Steve Dower added the comment: I wonder whether the same thing occurs if you're not appending a new extension each time? There could be some optimisation (from the dark old days of 8.3 filename) that compares baseExcel and .bak separately and assumes that the name is known. Last I looked at the code for stat() and isfile(), it was going directly to the Win32 API and not via the CRT. Though that may not have been the case in 3.3... -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22719 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22719] os.path.isfile os.path.exists bug in while loop
Serhiy Storchaka added the comment: Could we encode both paths to the unicode_internal encoding and check if results are equal? -- nosy: +serhiy.storchaka ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22719 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22719] os.path.isfile os.path.exists bug in while loop
R. David Murray added the comment: Looking at the code, it looks like it calls the win32 api directly if path-wide is true, which I'm guessing is the case unless you are using bytes paths in windows? It looks like the critical call, then, is CreateFileA (why A in a _w method I have no idea...so my reading of this code is suspect :) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22719 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22719] os.path.isfile os.path.exists bug in while loop
eryksun added the comment: What do you get for os.stat? bak_path = rC:\Users\EAARHOS\Desktop\Python Review\baseExcel.py print(os.stat(bak_path)) bak_path += '.bak' print(os.stat(bak_path)) bak_path += '.bak' print(os.stat(bak_path)) # This should raise FileNotFoundError -- nosy: +eryksun ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22719 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22719] os.path.isfile os.path.exists bug in while loop
Aaron added the comment: Interesting. It continues to reuse the last one's stats once the path is no longer valid. bak_path = rC:\Users\EAARHOS\Desktop\Python Review\baseExcel.py print(os.stat(bak_path)) nt.stat_result(st_mode=33206, st_ino=8162774324652726, st_dev=0, st_nlink=1, st_uid=0, st_gid=0, st_size=29874, st_atime=1413389016, st_mtime=1413389016, st_ctime=1413388655) bak_path += '.bak' print(os.stat(bak_path)) nt.stat_result(st_mode=33206, st_ino=42502721483352490, st_dev=0, st_nlink=1, st_uid=0, st_gid=0, st_size=2, st_atime=1413389088, st_mtime=1413389088, st_ctime=1413388654) bak_path += '.bak' print(os.stat(bak_path)) nt.stat_result(st_mode=33206, st_ino=42502721483352490, st_dev=0, st_nlink=1, st_uid=0, st_gid=0, st_size=2, st_atime=1413389088, st_mtime=1413389088, st_ctime=1413388654) bak_path += '.bak' print(os.stat(bak_path)) nt.stat_result(st_mode=33206, st_ino=42502721483352490, st_dev=0, st_nlink=1, st_uid=0, st_gid=0, st_size=2, st_atime=1413389088, st_mtime=1413389088, st_ctime=1413388654) bak_path += '.bak' print(os.stat(bak_path)) nt.stat_result(st_mode=33206, st_ino=42502721483352490, st_dev=0, st_nlink=1, st_uid=0, st_gid=0, st_size=2, st_atime=1413389088, st_mtime=1413389088, st_ctime=1413388654) On Fri, Oct 24, 2014 at 1:49 PM, eryksun rep...@bugs.python.org wrote: eryksun added the comment: What do you get for os.stat? bak_path = rC:\Users\EAARHOS\Desktop\Python Review\baseExcel.py print(os.stat(bak_path)) bak_path += '.bak' print(os.stat(bak_path)) bak_path += '.bak' print(os.stat(bak_path)) # This should raise FileNotFoundError -- nosy: +eryksun ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22719 ___ -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22719 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22719] os.path.isfile os.path.exists bug in while loop
Aaron added the comment: If I use a separate temp variable, the bug doesn't show, but if I use the same variable, even with + instead of +=, it still happens. bak_path = rC:\Users\EAARHOS\Desktop\Python Review\baseExcel.py print(os.stat(bak_path)) nt.stat_result(st_mode=33206, st_ino=8162774324652726, st_dev=0, st_nlink=1, st_uid=0, st_gid=0, st_size=29874, st_atime=1413389016, st_mtime=1413389016, st_ctime=1413388655) temp = bak_path + '.bak' bak_path = temp print(os.stat(bak_path)) nt.stat_result(st_mode=33206, st_ino=42502721483352490, st_dev=0, st_nlink=1, st_uid=0, st_gid=0, st_size=2, st_atime=1413389088, st_mtime=1413389088, st_ctime=1413388654) temp = bak_path + '.bak' bak_path = temp print(os.stat(bak_path)) Traceback (most recent call last): File interactive input, line 1, in module FileNotFoundError: [WinError 2] The system cannot find the file specified: 'C:\\Users\\EAARHOS\\Desktop\\Python Review\\baseExcel.py.bak.bak' bak_path = rC:\Users\EAARHOS\Desktop\Python Review\baseExcel.py bak_path = bak_path + '.bak' print(os.stat(bak_path)) nt.stat_result(st_mode=33206, st_ino=42502721483352490, st_dev=0, st_nlink=1, st_uid=0, st_gid=0, st_size=2, st_atime=1413389088, st_mtime=1413389088, st_ctime=1413388654) bak_path = bak_path + '.bak' print(os.stat(bak_path)) nt.stat_result(st_mode=33206, st_ino=42502721483352490, st_dev=0, st_nlink=1, st_uid=0, st_gid=0, st_size=2, st_atime=1413389088, st_mtime=1413389088, st_ctime=1413388654) bak_path = bak_path + '.bak' print(os.stat(bak_path)) nt.stat_result(st_mode=33206, st_ino=42502721483352490, st_dev=0, st_nlink=1, st_uid=0, st_gid=0, st_size=2, st_atime=1413389088, st_mtime=1413389088, st_ctime=1413388654) bak_path = bak_path + '.bak' print(os.stat(bak_path)) nt.stat_result(st_mode=33206, st_ino=42502721483352490, st_dev=0, st_nlink=1, st_uid=0, st_gid=0, st_size=2, st_atime=1413389088, st_mtime=1413389088, st_ctime=1413388654) On Fri, Oct 24, 2014 at 2:24 PM, Aaron rep...@bugs.python.org wrote: Aaron added the comment: Interesting. It continues to reuse the last one's stats once the path is no longer valid. bak_path = rC:\Users\EAARHOS\Desktop\Python Review\baseExcel.py print(os.stat(bak_path)) nt.stat_result(st_mode=33206, st_ino=8162774324652726, st_dev=0, st_nlink=1, st_uid=0, st_gid=0, st_size=29874, st_atime=1413389016, st_mtime=1413389016, st_ctime=1413388655) bak_path += '.bak' print(os.stat(bak_path)) nt.stat_result(st_mode=33206, st_ino=42502721483352490, st_dev=0, st_nlink=1, st_uid=0, st_gid=0, st_size=2, st_atime=1413389088, st_mtime=1413389088, st_ctime=1413388654) bak_path += '.bak' print(os.stat(bak_path)) nt.stat_result(st_mode=33206, st_ino=42502721483352490, st_dev=0, st_nlink=1, st_uid=0, st_gid=0, st_size=2, st_atime=1413389088, st_mtime=1413389088, st_ctime=1413388654) bak_path += '.bak' print(os.stat(bak_path)) nt.stat_result(st_mode=33206, st_ino=42502721483352490, st_dev=0, st_nlink=1, st_uid=0, st_gid=0, st_size=2, st_atime=1413389088, st_mtime=1413389088, st_ctime=1413388654) bak_path += '.bak' print(os.stat(bak_path)) nt.stat_result(st_mode=33206, st_ino=42502721483352490, st_dev=0, st_nlink=1, st_uid=0, st_gid=0, st_size=2, st_atime=1413389088, st_mtime=1413389088, st_ctime=1413388654) On Fri, Oct 24, 2014 at 1:49 PM, eryksun rep...@bugs.python.org wrote: eryksun added the comment: What do you get for os.stat? bak_path = rC:\Users\EAARHOS\Desktop\Python Review\baseExcel.py print(os.stat(bak_path)) bak_path += '.bak' print(os.stat(bak_path)) bak_path += '.bak' print(os.stat(bak_path)) # This should raise FileNotFoundError -- nosy: +eryksun ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22719 ___ -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22719 ___ -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22719 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22719] os.path.isfile os.path.exists bug in while loop
eryksun added the comment: When appending to a singly-referenced string, the interpreter tries to reallocate the string in place. This applies to both `s += 'text'` and `s = s + 'text'`. Storing to a temp variable is adding a 2nd reference, so a new string gets allocated instead. If the former is the case (i.e. the object id is the same after appending), use ctypes to check the string's cached wide-string (wchar_t *) representation: from ctypes import * pythonapi.PyUnicode_AsUnicode.argtypes = [py_object] pythonapi.PyUnicode_AsUnicode.restype = c_wchar_p print(pythonapi.PyUnicode_AsUnicode(bak_path)) The wstr cache should be cleared when the string is reallocated in place, so this is probably a dead end. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22719 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22719] os.path.isfile os.path.exists bug in while loop
eryksun added the comment: i.e. the object id is the same after appending Actually, that's wrong. bak_path is a compact string. So the whole object is realloc'd, and the base address (i.e. id) could change. Check PyUnicode_AsUnicode even if the id changes. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22719 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
