[issue35981] shutil make_archive create wrong file when base name contains dots at end
Bruno P. Kinoshita added the comment: Tested locally on Win10 pro, and both suggestions of steve.dower worked. I've put a PR (https://github.com/python/cpython/pull/17435) based on his second suggestion. The rationale being that that way, the format function (zip/bztar/etc) will get the absolute file name. I thought this was less risk then trusting that the format function would work with the relative paths in win/linux/etc. -- nosy: +kinow ___ Python tracker <https://bugs.python.org/issue35981> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35981] shutil make_archive create wrong file when base name contains dots at end
Change by Bruno P. Kinoshita : -- keywords: +patch, patch pull_requests: +16914, 16915 stage: -> patch review pull_request: https://github.com/python/cpython/pull/17435 ___ Python tracker <https://bugs.python.org/issue35981> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35981] shutil make_archive create wrong file when base name contains dots at end
Change by Bruno P. Kinoshita : -- keywords: +patch pull_requests: +16914 stage: -> patch review pull_request: https://github.com/python/cpython/pull/17435 ___ Python tracker <https://bugs.python.org/issue35981> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38868] Shutil cannot delete a folder that contains an .ini file
Bruno P. Kinoshita added the comment: I think eryksun is correct. Reproduced it locally. Setting to read-only initially did not raise the issue, but then as administrator I removed the inherited permissions and set it to read-only for my user, then it raised the exact same WinError 5 Permission Error. With the extra code to remove the readonly attribute, the directory was successfully deleted. >It does so as an accident of history, not because it's justified I think it would be easier for users if this attribute was not copied, especially as it looks like other attributes are not being copied (on Win). That would remove the need of this extra code when using Windows. -- nosy: +kinow ___ Python tracker <https://bugs.python.org/issue38868> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38720] Logging failure with timestamp messages
Bruno P. Kinoshita added the comment:
I think this needs a script to reproduce the problem.
Tested first with Anaconda Python 3.7.1 on Win10 Pro. Not able to reproduce the
issue. Smae with the latest version.
Using Python master and Win10Pro and timezone set to NZ (not that really
matter, but just in case).
Python 3.9.0a1+ (heads/master:2fe4c48917, Dec 2 2019, 14:12:32) [MSC
v.1916 64 bit (AMD64)] on win32
with the example from https://docs.python.org/3/library/logging.html:
import logging
FORMAT = '%(asctime)-15s %(clientip)s %(user)-8s %(message)s'
logging.basicConfig(format=FORMAT)
d = {'clientip': '192.168.0.1', 'user': 'fbloggs'}
logger = logging.getLogger('tcpserver')
logger.warning('Protocol problem: %s', 'connection reset', extra=d)
Gives:
2019-12-02 14:14:46,615 192.168.0.1 fbloggs Protocol problem: connection
reset
If I modify the example to use just %(asctime)s
import logging
FORMAT = '%(asctime)s'
logging.basicConfig(format=FORMAT)
d = {'clientip': '192.168.0.1', 'user': 'fbloggs'}
logger = logging.getLogger('tcpserver')
logger.warning('Protocol problem: %s', 'connection reset', extra=d)
Now gives:
2019-12-02 14:18:29,047
Everything looks OK, at least on my environment.
--
nosy: +kinow
___
Python tracker
<https://bugs.python.org/issue38720>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38688] Python 3.8 regression: endless loop in shutil.copytree
Change by Bruno P. Kinoshita : -- pull_requests: +16878 stage: commit review -> patch review pull_request: https://github.com/python/cpython/pull/17397 ___ Python tracker <https://bugs.python.org/issue38688> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38688] Python 3.8 regression: endless loop in shutil.copytree
Bruno P. Kinoshita added the comment: Done. Rebased on master too, and edited commit message & GH PR title. Thanks Giampaolo! -- ___ Python tracker <https://bugs.python.org/issue38688> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38688] Python 3.8 regression: endless loop in shutil.copytree
Bruno P. Kinoshita added the comment: I really liked that improvement, and didn't think it needed to be removed. That's why the PR reverts it partially. I think the os.stat improvements were in the other methods changed, and should not be changed in my PR - unless I changed it by accident. So with the current PR for this issue, or with your suggested patch, both should have similar performance I think. (I hadn't seen that script to measure performance, thanks.) I am happy if we revert partially, or if we build the fix on top of the current code consuming the iterator. Should I update the PR with your suggested fix then? -- ___ Python tracker <https://bugs.python.org/issue38688> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38688] Python 3.8 regression: endless loop in shutil.copytree
Bruno P. Kinoshita added the comment: Hi Giampaolo, I think it is more or less the same as the previous code, which was using os.list to return a list in memory. My first tentative fix was: def copytree(src, ...): entries = os.list(src) return _copytree(entries=entries, ...) But the previous PR also altered _copytree to use the return of os.scandir DirEntry's, so the change above results in AttributeError: 'str' object has no attribute 'name'. Would be better to avoid using iterator to populate a list, and also using the DirEntry in _copytree, and just stick with the previous code with (i.e. os.listdir() and a single copytree method as before)? Or if you think we should go with your suggestion, I'm good with it as well - especially as it'd be a much simpler PR :) Thanks Bruno -- ___ Python tracker <https://bugs.python.org/issue38688> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38688] Python 3.8 regression: endless loop in shutil.copytree
Bruno P. Kinoshita added the comment:
Hi,
I did a quick `git bisect` using the example provided, and looks like this
regression was added in the fix for bpo-33695, commit
19c46a4c96553b2a8390bf8a0e138f2b23e28ed6.
It looks to me that the iterator returned by with os.scandir(...) is including
the newly created dst directory (see the call for `os.makedirs(dst,
exist_ok=dirs_exist_ok)` in
https://github.com/python/cpython/blob/e27449da92b13730a5e11182f329d5da98a5e05b/Lib/shutil.py#L449).
This results in the function finding an extra directory, and repeating the
steps for this folder and its subfolder recursively. This only happens because
in the example in this issue, dst is a subdirectory of src.
The bpo-33695 commit had more changes, so I've reverted just this block of the
copytree as a tentative fix, and added a unit test:
https://github.com/python/cpython/pull/17098
--
Here's a simplified version of what's going on:
```
import os
import shutil
shutil.rmtree('/tmp/test/', True)
os.makedirs('/tmp/test')
with open('/tmp/test/foo', 'w+') as f:
f.write('foo')
# now we have /tmp/test/foo, let's simulate what happens in copytree on master
with os.scandir('/tmp/test') as entries:
# up to this point, /tmp/test/foo is the only entry
os.makedirs('/tmp/test/1/2/3/', exist_ok=True) # < when the iterator
starts below in `f in entries`, it will find 1 too
# now 1 will have been added too
for f in entries:
print(f)
```
--
nosy: +kinow
___
Python tracker
<https://bugs.python.org/issue38688>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38688] Python 3.8 regression: endless loop in shutil.copytree
Change by Bruno P. Kinoshita : -- keywords: +patch pull_requests: +16604 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/17098 ___ Python tracker <https://bugs.python.org/issue38688> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33695] Have shutil.copytree(), copy() and copystat() use cached scandir() stat()s
Change by Bruno P. Kinoshita : -- pull_requests: +16605 pull_request: https://github.com/python/cpython/pull/17098 ___ Python tracker <https://bugs.python.org/issue33695> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37647] Wrong lineno in traceback when formatting strings with % and multilines
Bruno P. Kinoshita added the comment: Hi, Thanks for the explanation. Could you elaborate a bit on this one: >In Python 3.8 the traceback points to the start of the subexpression that >raises an exception. Just wanted to understand why I get the desired line number when I call Z().__str__(), but when I call simply Z() is returns - as you clarified - the start of the subexpression. -- ___ Python tracker <https://bugs.python.org/issue37647> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37647] Wrong lineno in traceback when formatting strings with % and multilines
Bruno P. Kinoshita added the comment:
Hi Ammar, thanks for the quick reply and for the suggestion.
Tried on the latest version on master, and looks like it's indeed different,
though still looks like it is displaying the wrong line number.
```
kinow@localhost:~/Development/python/workspace/cpython$ python
Python 3.9.0a0 (heads/master:b530a4460b, Jul 22 2019, 18:38:44)
[GCC 7.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
kinow@localhost:~/Development/python/workspace/cpython$ python z.py
Traceback (most recent call last):
File "/home/kinow/Development/python/workspace/cpython/z.py", line 14, in
print(list(raise_generator()))
File "/home/kinow/Development/python/workspace/cpython/z.py", line 7, in
raise_generator
yield 'three values are: %s %s %s' % (
File "/home/kinow/Development/python/workspace/cpython/z.py", line 3, in
__str__
raise ValueError('Z error')
ValueError: Z error
```
It is now displaying the line number 7, where the yield statement begins,
instead of where it actually happened (line 9).
And once again, if I replace `Z()` by `Z().__str__()`, then it displays the
right line number.
```
kinow@localhost:~/Development/python/workspace/cpython$ python z.py
Traceback (most recent call last):
File "/home/kinow/Development/python/workspace/cpython/z.py", line 14, in
print(list(raise_generator()))
File "/home/kinow/Development/python/workspace/cpython/z.py", line 9, in
raise_generator
Z().__str__(), # traceback must point to this lineno 9
File "/home/kinow/Development/python/workspace/cpython/z.py", line 3, in
__str__
raise ValueError('Z error')
ValueError: Z error
```
--
___
Python tracker
<https://bugs.python.org/issue37647>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37647] Wrong lineno in traceback when formatting strings with % and multilines
New submission from Bruno P. Kinoshita :
Hi,
Tested on Python 3.7 on Ubuntu LTS. But I believe this affects previous
versions too, as found it from an issue posted in Jinja2 in 2013:
https://github.com/pallets/jinja/issues/276
This code simulates the issue where the traceback has the wrong line number
(without using any Jinja2 code):
```
class Z(object):
def __str__(self):
raise ValueError('Z error')
def raise_generator():
yield 'three values are: %s %s %s' % (
'primeiro',
Z(), # traceback must point to this lineno 9
'terceiro' # but points to this lineno 10 (__str__ only, __eq__ is OK)
)
print(list(raise_generator()))
```
The output:
```
Traceback (most recent call last):
File
"/home/kinow/Development/python/workspace/cylc-flow/cylc/flow/tests/Z.py", line
14, in
print(list(raise_generator()))
File
"/home/kinow/Development/python/workspace/cylc-flow/cylc/flow/tests/Z.py", line
10, in raise_generator
'terceiro' # but points to this lineno 10 (__str__ only, __eq__ is OK)
File
"/home/kinow/Development/python/workspace/cylc-flow/cylc/flow/tests/Z.py", line
3, in __str__
raise ValueError('Z error')
ValueError: Z error
```
Jinja uses something similar to the class Z to raise errors when the template
has undefined variables. The curious part is that if instead of simply
formatting the string with "%s" % (Z()) you use "%s" % (str(Z())) or if you
callZ().__str__(), then the traceback reports the correct line number.
Not sure if intentional, but would be nice if the traceback reported the
correct line number, and I think other applications could end up having the
same issue.
This is my first issue, so I apologize if I did not include enough information,
or if there is anything missing in this ticket. Let me know if I need to update
it with more information or formatting the code.
Would be happy to work on this if this is considered an easy-to-fix issue.
Thanks all for your work on Python,
Bruno P. Kinoshita
--
components: Interpreter Core
messages: 348277
nosy: kinow
priority: normal
severity: normal
status: open
title: Wrong lineno in traceback when formatting strings with % and multilines
type: behavior
versions: Python 3.7
___
Python tracker
<https://bugs.python.org/issue37647>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
