[issue35686] BufferError with memory.release()
Thomas Waldmann added the comment: https://github.com/borgbackup/borg/pull/4247/files this is the current code i have for this (it is not as simple there as in the minimal code samples i attached here). i meanwhile think i can not use a contextmanager there. do you think this is as good as it gets for this kind of code? if you all think this is expected behaviour for the python contextmanagers and can't be done better, this issue can be closed. -- ___ Python tracker <https://bugs.python.org/issue35686> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35686] BufferError with memory.release()
Thomas Waldmann added the comment: I see 2 issues here: 1. I would expect that the CM approach (b) behaves equivalent to try/finally (c), but it does not and even causes an exception. 2. The traceback given in the BufferError (in (b)) is misleading, it points to the "print", but should rather point to mmap.__exit__). -- ___ Python tracker <https://bugs.python.org/issue35686> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35686] BufferError with memory.release()
Thomas Waldmann added the comment: hmm, issue tracker does not make it easy to see which file was uploaded for which comment, so: a: original code, works, potentially problematic exception handling (memoryview not always being released) b: using the memoryview context manager, fails with BufferError c: using try/finally to make sure memoryview is released: works -- ___ Python tracker <https://bugs.python.org/issue35686> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35686] BufferError with memory.release()
Thomas Waldmann added the comment: with try/finally: works. -- Added file: https://bugs.python.org/file48033/issue35686c.py ___ Python tracker <https://bugs.python.org/issue35686> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35686] BufferError with memory.release()
Thomas Waldmann added the comment: with context manager, does not work. -- Added file: https://bugs.python.org/file48032/issue35686b.py ___ Python tracker <https://bugs.python.org/issue35686> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35686] BufferError with memory.release()
Thomas Waldmann added the comment: working, but potentially problematic because .release is not always called (e.g. in case of an exception). -- Added file: https://bugs.python.org/file48031/issue35686a.py ___ Python tracker <https://bugs.python.org/issue35686> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35686] memoryview contextmanager causing strange crash
New submission from Thomas Waldmann : See there: https://github.com/borgbackup/borg/pull/4247 I did the first changeset after seeing some strange exception popping up which it was handling another exception - which I assumed was related to memoryview.release not being called in the original code. So it was clear to me, that we should use the CM there. So I added that (first changeset) and that made the code always fail (see first travis-ci link). Then removed the CM again and replaced it with functionally equivalent try/finally (second changeset) - that worked. So, the question is whether there is some issue in CPython's memoryview contextmanager code that make it fail in such a strange way. -- messages: 333236 nosy: Thomas.Waldmann priority: normal severity: normal status: open title: memoryview contextmanager causing strange crash type: crash versions: Python 3.5, Python 3.6, Python 3.7 ___ Python tracker <https://bugs.python.org/issue35686> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34269] logging in 3.7 behaves different due to caching
Thomas Waldmann added the comment: I agree that we should not dig that deep into logging internals and clear that dict from borg code. But, seems like one has no public api to reinitialize logging to a like-fresh state, right? So maybe THAT is the real problem. Add some .reset() method to do that? BTW, removing that .clear() from our code made our tests work again. I am not sure why that was added in the first place as I could not find regressions after removing it. -- ___ Python tracker <https://bugs.python.org/issue34269> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34269] logging in 3.7 behaves different due to caching
Thomas Waldmann added the comment: https://github.com/borgbackup/borg/pull/3998 Looks like this fix to our test tearDown method makes the tests work again. If one looks at what this dirty/problematic hack was done, it might be because of a missing logging reset functionality? -- ___ Python tracker <https://bugs.python.org/issue34269> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34269] logging in 3.7 behaves different due to caching
Change by Thomas Waldmann : -- components: +Library (Lib) type: -> behavior ___ Python tracker <https://bugs.python.org/issue34269> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34269] logging in 3.7 behaves different due to caching
New submission from Thomas Waldmann : When running the borgbackup tests on 3.7.0, quite some tests will be broken because expected log output is not emitted. I traced this down up the point when I noticed that reverting the caching feature change of isEnabledFor method in Logger makes the tests work on 3.7. I reviewed the logging code and the caching change, but could not easily find what is broken. There is of course some suspicion that cache invalidation might be the issue (it is present, but maybe it does not cover all the cases). See there for more info: https://github.com/borgbackup/borg/issues/3805 So, I suggest to first revert the caching feature change so we get a fixed logging in 3.7.1 and then to retry to add caching. Note: I'm at EuroPython sprints today at the borgbackup table. -- messages: 322617 nosy: Thomas.Waldmann priority: normal severity: normal status: open title: logging in 3.7 behaves different due to caching versions: Python 3.7 ___ Python tracker <https://bugs.python.org/issue34269> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5004] socket.getfqdn() doesn't cope properly with purely DNS-based setups
Thomas Waldmann <thomas.j.waldm...@gmail.com> added the comment: Embarassing as always to stumble over some stuff and then find a 9y old ticket here, where it is discussed and even (almost) solved. Our ticket: https://github.com/borgbackup/borg/issues/3471 Fixed getfqdn we use now instead of socket.getfqdn(): https://github.com/borgbackup/borg/pull/3484/commits/9b0d0f3127289fc3bbe620e8adce8038ed594e9f#diff-4b53f84e19a3bb376bf2202371ed269aR188 Note: no "else: name = hostname" at the end, that is a bug in the patch attached to this ticket (hostname is undefined after applying the patch). -- nosy: +Thomas.Waldmann ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue5004> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28494] is_zipfile false positives
Thomas Waldmann added the comment: Well, if you have a better idea how to fix is_zipfile, go on. I even suggested an alternative, how about that? It is a miserable state when the is_zipfile function in the stdlib detects random crap as a zip file. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28494> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28494] is_zipfile false positives
Thomas Waldmann added the comment: Note: checking the first bytes of the file (PK..) might be another option. But this has the "problem" that a self-extracting zip starts with an executable that has different first bytes. So whether this is an option or not depends on whether is_zipfile() should return truish for self-extracting ZIP files. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28494> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28494] is_zipfile false positives
Thomas Waldmann added the comment: patch for py2.7 The EOCD structure is at EOF. It either does not contain a comment (this is what the existing code checks first) or it contains a comment of the length that is specified in the structure. The patch checks consistency specified length vs. real length (end of fixed part of structure up to EOF). If this does not match, it is likely not a zip file, but just a file that happens to have the magic 4 bytes somewhere in its last 64kB. -- keywords: +patch Added file: http://bugs.python.org/file45164/isz_fail_fix.diff ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28494> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28494] is_zipfile false positives
New submission from Thomas Waldmann: zipfile.is_zipfile has false positives way too easily. I just have seen it in practive when a MoinMoin wiki site with a lot of pdf attachments crashed with 500. This was caused by a valid PDF that just happened to contain PK\005\006 somewhere in the middle - this was enough to satisfy is_zipfile() and triggered further processing as a zipfile, which then crashed with IOError (which was not catched in our code, yet). I have looked into zipfile code: if the usual EOCD structure (with empty comment) is not at EOF, it is suspected that there might be a non-empty comment and ~64K before EOF are searched for the PK\005\006 magic. If it is somewhere there, it is assumed that the file is a zip, without any further validity check. Attached is a failure demo that works with at least 2.7 and 3.5. https://en.wikipedia.org/wiki/Zip_(file_format) -- components: Library (Lib) files: isz_fail.py messages: 279084 nosy: Thomas.Waldmann priority: normal severity: normal status: open title: is_zipfile false positives type: behavior versions: Python 2.7, Python 3.5 Added file: http://bugs.python.org/file45162/isz_fail.py ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28494> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26545] os.walk is limited by python's recursion limit
Thomas Waldmann added the comment: Note: similar issues can be seen in other stdlib recursive filesystem-related functions also. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26545> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26545] os.walk is limited by python's recursion limit
New submission from Thomas Waldmann: os.walk calls itself recursively and that limits the directory depth it can "walk" into. On Linux, one can create directory hierarchies deeper than that. For some more details see there: https://github.com/borgbackup/borg/issues/380 -- components: Library (Lib) messages: 261626 nosy: Thomas.Waldmann priority: normal severity: normal status: open title: os.walk is limited by python's recursion limit versions: Python 3.5 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26545> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26544] platform.libc_ver() returns incorrect version number
New submission from Thomas Waldmann: platform.libc_ver() is trivially broken as it uses string comparison internally to determine the maximum libc version number (which is obviously broken as "2.9" > "2.10"). -- components: Library (Lib) messages: 261624 nosy: Thomas.Waldmann priority: normal severity: normal status: open title: platform.libc_ver() returns incorrect version number type: behavior versions: Python 3.5 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26544> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15801] Weird string interpolation behaviour
Thomas Waldmann added the comment: gave 2.7.4rc1 a try and was seeing a failing unit test that does not fail with 2.7.3. see the attached file for some minimal code that succeeds on 2.7.3, but not on 2.7.4rc1. it seems to have to do with being a subclass of Exception, it doesn't happen for subclass of object. -- nosy: +Thomas.Waldmann Added file: http://bugs.python.org/file29558/py274_str_mod_bug.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15801 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
