[issue43805] multiprocessing.Queue hangs when process on other side dies
Thomas Kluyver added the comment: It's not my decision, so I can't really say. But the Queue API is pretty stable, and exists 3 times over in Python (the queue module for use with threads, in multiprocessing and in asyncio). So I'd guess that anyone wanting to add to that API would need to make a compelling case for why it's important, and be prepared for a lot of wrangling over API details (like method names and exceptions). If you want to push that idea, you could try the ideas board on the Python discourse forum: https://discuss.python.org/c/ideas/6 . You might also want to look at previous discussions about adding a Queue.close() method: issue29701 and issue40888. -- ___ Python tracker <https://bugs.python.org/issue43805> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43805] multiprocessing.Queue hangs when process on other side dies
Thomas Kluyver added the comment: I think this is expected. The queue itself doesn't know that one particular process is meant to put data into it. It just knows that there's no data to get, so .get() blocks as the docs say it should. This doesn't apply to issue22393, because the pool knows about its worker processes, so if one dies before completing a task, it can know something is wrong. You could add a method to 'half close' a queue, so it can only be used for receiving, but not sending. If you called this in the parent process after starting the child, then if the child died, the queue would know that nothing could ever put data into it, and .get() could error. The channels API in Trio allows this, and it's the same idea I've just described at the OS level in issue43806. -- nosy: +takluyver ___ Python tracker <https://bugs.python.org/issue43805> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43806] asyncio.StreamReader hangs when reading from pipe and other process exits unexpectedly
Thomas Kluyver added the comment: In the example script, I believe you need to close the write end of the pipe in the parent after forking: cpid = os.fork() if cpid == 0: # Write to pipe (child) else: # Parent os.close(ctx) # Read from pipe This is the same with synchronous code: os.read(prx, 1) also hangs. You only get EOF when nothing has the write end open any more. All the asyncio machinery doesn't really make any difference to this. For a similar reason, the code writing (the child, in this case) should close the read end of the pipe after forking. If the parent goes away but the child still has the read end open, then trying to write to the pipe can hang (if the buffer is already full). If the child has closed the read end, trying to write will give you a BrokenPipeError. -- nosy: +takluyver ___ Python tracker <https://bugs.python.org/issue43806> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37612] os.link(..., follow_symlinks=True) broken on Linux
Change by Thomas Kluyver : -- nosy: +takluyver nosy_count: 4.0 -> 5.0 pull_requests: +23755 pull_request: https://github.com/python/cpython/pull/24997 ___ Python tracker <https://bugs.python.org/issue37612> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24916] In sysconfig, don't rely on sys.version format
Thomas Kluyver added the comment: Serhiy, I think you fixed the part that was actually likely to cause problems a few years ago in issue #25985 & commit 885bdc4946890f4bb80557fab80c3874b2cc4d39 . Using sys.version[:3] to get a short version like 3.8 was what I wanted to fix. People are presumably still trying to change the other bits because there's still a FIXME comment. If we're happy with the current code, I think we can remove that comment and close the issue. -- ___ Python tracker <https://bugs.python.org/issue24916> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36695] Change (regression?) in v3.8.0a3 doctest output after capturing the stderr output from a raised warning
Thomas Kluyver added the comment: D'oh, yes. I missed that the failing example was displaying the captured string through displayhook. It makes sense now. Thanks for patiently explaining. :-) -- ___ Python tracker <https://bugs.python.org/issue36695> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36695] Change (regression?) in v3.8.0a3 doctest output after capturing the stderr output from a raised warning
Thomas Kluyver added the comment:
The 'single' option to compile() means it's run like at a REPL, calling
displayhook if it's an expression returning a value.
But warnings shouldn't go through the displayhook, as far as I know:
>>> from contextlib import redirect_stdout, redirect_stderr
>>> from io import StringIO
>>> sio = StringIO()
>>> with redirect_stderr(sio):
... exec(compile('import warnings; warnings.warn(""" \' " """)', 'dummyfile',
'single'))
...
>>> print(sio.getvalue())
__main__:1: UserWarning: ' "
--
___
Python tracker
<https://bugs.python.org/issue36695>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36695] Change (regression?) in v3.8.0a3 doctest output after capturing the stderr output from a raised warning
Thomas Kluyver added the comment:
I'm still a bit confused why it gets escaped - as far as I know, the escaping
only happens when you repr() a string, as the displayhook does automatically:
>>> a = """ a ' single and " double quote """
>>> a
' a \' single and " double quote '
>>> print(repr(a))
' a \' single and " double quote '
>>> print("%r" % a)
' a \' single and " double quote '
>>> print(a)
a ' single and " double quote
The warnings code doesn't appear to ever repr() the message. So I guess it's
some further bit of interaction with doctest. But unfortunately I don't have
time to dig through doctest to try and understand it.
--
___
Python tracker
<https://bugs.python.org/issue36695>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36695] Change (regression?) in v3.8.0a3 doctest output after capturing the stderr output from a raised warning
Thomas Kluyver added the comment: It's not obvious to me why that change to finding the source file related to the warning should affect the format of the warning message printed. It might be something that could be fixed in the warning module. But I don't understand where it's going wrong at present. -- ___ Python tracker <https://bugs.python.org/issue36695> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36691] SystemExit & sys.exit : Allow both exit status and message
New submission from Thomas Kluyver : The SystemExit exception, and consequently the sys.exit() function, take a single parameter which is either an integer exit status for the process, or a message to print to stderr before exiting - in which case the exit status is implicitly 1. In certain situations, it would be useful to pass both an exit status and a message. E.g. when argparse handles '--help', it wants to display a message and exit successfully (status 0). You may also use specific exit codes to indicate different kinds of failure. Printing the message separately before raising SystemExit is not an entirely satisfactory subsitute, because the message attached to the exception is only printed if it is unhandled. E.g. for testing code that may raise SystemExit, it's useful to have the message as part of the exception. I imagine that the trickiest bit of changing this would be ensuring as much backwards compatibility as possible. In particular, SystemExit exceptions have a 'code' attribute which can be either the exit status or the message. -- messages: 340607 nosy: takluyver priority: normal severity: normal status: open title: SystemExit & sys.exit : Allow both exit status and message type: enhancement ___ Python tracker <https://bugs.python.org/issue36691> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33944] Deprecate and remove pth files
Thomas Kluyver added the comment: As a lurker on this issue: I think a lot of energy is being expended arguing about what is and isn't legitimate use cases, when there's actually more stuff that people agree about than not. I think this issue should be broken down into two, neither of which will actually result in removing pth files: 1. Better ways to inspect and control the sys.path extension feature (as per Barry's message https://bugs.python.org/issue33944#msg337417 ). 2. Designing a replacement for the arbitrary-code-at-startup feature (or even several replacements to meet different needs), leading to its eventual deprecation. If you like the ability for packages to install interpreter-startup hooks, then pth files are an ugly way to do it. If you don't, then you want better ways to control it. So let's see what we can come up with. At least several important use cases (coverage and debugging) would probably work with an environment variable to specify startup code. The coverage hooks already check an environment variable themselves, so it's clearly a control mechanism that works. It's also familiar from things like LD_PRELOAD that environment variables can affect code in powerful ways. But the PYTHONSTARTUP variable is not suitable for this, because it only affects interactive shell sessions. So maybe one useful step would be to specify a new environment variable, maybe PYTHONPRELOAD, and figure out how it will interact with all the other options. Then we can re-evaluate the use cases Anthony described (https://bugs.python.org/issue33944#msg337406 ) and debate the need for other startup-code mechanisms to go along with that. -- ___ Python tracker <https://bugs.python.org/issue33944> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28418] Raise Deprecation warning for tokenize.generate_tokens
Thomas Kluyver added the comment: Yes, I think this can be closed now. -- ___ Python tracker <https://bugs.python.org/issue28418> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33944] Deprecate and remove pth files
Thomas Kluyver added the comment: I don't want to use the execution features of .pth files, just their original functionality of adding extra directories to sys.path. I'd be very happy to see the arbitrary code execution 'feature' of .pth files go away. Windows supports symlinks, but the last I heard was that creating them requires some obscure permission bit. It seems to be awkward enough that Windows users aren't happy with the "just use symlinks" approach, which was what I was originally trying. -- ___ Python tracker <https://bugs.python.org/issue33944> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33944] Deprecate and remove pth files
Thomas Kluyver added the comment: I'm generally in favour of getting rid of .pth files. But I did accept a PR adding support for them in Flit to act as a substitute for symlinks on Windows, to achieve something like a 'development install'. I'm not sure what the alternative is if they go away. -- nosy: +takluyver ___ Python tracker <https://bugs.python.org/issue33944> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33912] [EASY] test_warnings: test_exec_filename() fails when run with -Werror
Thomas Kluyver added the comment: Yes, this should be easy. I misunderstood how to use the catch_warnings context manager. I thought that catch_warnings itself set up the warnings filters you need. You actually need to do that with a separate call inside the with block, as shown here: https://docs.python.org/3/library/warnings.html#testing-warnings -- ___ Python tracker <https://bugs.python.org/issue33912> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12486] tokenize module should have a unicode API
Thomas Kluyver added the comment: Thanks Carol :-) -- ___ Python tracker <https://bugs.python.org/issue12486> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12486] tokenize module should have a unicode API
Thomas Kluyver added the comment: The tests on PR #6957 are passing now, if anyone has time to have a look. :-) -- ___ Python tracker <https://bugs.python.org/issue12486> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12486] tokenize module should have a unicode API
Thomas Kluyver <tho...@kluyver.me.uk> added the comment: Thanks - I had forgotten it, just fixed it now. -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue12486> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12486] tokenize module should have a unicode API
Thomas Kluyver <tho...@kluyver.me.uk> added the comment: I agree, it's not a good design, but it's what's already there; I just want to ensure that it won't be removed without a deprecation cycle. My PR makes no changes to behaviour, only to documentation and tests. This and issue 9969 have both been around for several years. A new tokenize API is clearly not at the top of anyone's priority list - and that's fine. I'd rather have *some* unicode API today than a promise of a nice unicode API in the future. And it doesn't preclude adding a better API later, it just means that the existing API would have to have a deprecation cycle. -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue12486> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12486] tokenize module should have a unicode API
Thomas Kluyver <tho...@kluyver.me.uk> added the comment: I wouldn't say it's a good name, but I think the advantage of documenting an existing name outweighs that. We can start (or continue) using generate_tokens() right away, whereas a new name presumably wouldn't be available until Python 3.8 comes out. And we usually don't require a new Python version until a couple of years after it is released. If we want to add better names or clearer APIs on top of this, great. But I don't want that discussion to hold up the simple step of committing to keep the existing API. -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue12486> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28418] Raise Deprecation warning for tokenize.generate_tokens
Thomas Kluyver <tho...@kluyver.me.uk> added the comment: I've opened a PR moving in the other direction (making this public rather than deprecating it): https://github.com/python/cpython/pull/6957 -- nosy: +takluyver ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue28418> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9969] tokenize: add support for tokenizing 'str' objects
Thomas Kluyver <tho...@kluyver.me.uk> added the comment: I've opened a PR for issue #12486, which would make the existing but undocumented 'generate_tokens' function public: https://github.com/python/cpython/pull/6957 I agree that it would be good to design a nicer API for this, but the perfect shouldn't be the enemy of the good. -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue9969> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12486] tokenize module should have a unicode API
Change by Thomas Kluyver <tho...@kluyver.me.uk>: -- pull_requests: +6616 ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue12486> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33375] warnings: get filename from frame.f_code.co_filename
Thomas Kluyver <tho...@kluyver.me.uk> added the comment: Thanks! No rush, I just thought I'd take the opportunity when you added yourself to the nosy list. -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue33375> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33375] warnings: get filename from frame.f_code.co_filename
Thomas Kluyver <tho...@kluyver.me.uk> added the comment: Hi Brett! If you get a moment, any review of the linked PR would be welcome. :-) https://github.com/python/cpython/pull/6622 -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue33375> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33375] warnings: get filename from frame.f_code.co_filename
Change by Thomas Kluyver <tho...@kluyver.me.uk>: -- keywords: +patch pull_requests: +6318 stage: -> patch review ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue33375> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1692664] warnings.py gets filename wrong for eval/exec
Thomas Kluyver <tho...@kluyver.me.uk> added the comment: Thanks Guido, the new issue is #33375. -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue1692664> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33375] warnings: get filename from frame.f_code.co_filename
New submission from Thomas Kluyver <tho...@kluyver.me.uk>:
The warnings module tries to find and show the line of code which is
responsible for a warning, for the same reasons that tracebacks show a line of
code from each stack frame. However, they work in quite different ways.
Native tracebacks, the traceback module and pdb all do something like this:
frame.f_code.co_filename
But warnings does something like this (paraphrasing C code in _warnings.c):
frame.f_globals.get('__file__', sys.argv[0])
This causes problems for interactive interpreters like IPython, because there
are multiple pieces of entered code which have to share the same global
namespace. E.g. https://github.com/ipython/ipython/issues/11080
This was raised a long time ago in #1692664. Back then, the answer was that
co_filename could be wrong if the path of a pyc file changed. However, that
issue was fixed in #1180193. And it seems that the co_filename approach must
largely work today, because tracebacks and pdb rely on it.
So I'm proposing to make warnings match how those other tools find filenames. I
think this should also be a minor simplification of the code.
--
components: Library (Lib)
messages: 315848
nosy: takluyver
priority: normal
severity: normal
status: open
title: warnings: get filename from frame.f_code.co_filename
___
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue33375>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1692664] warnings.py gets filename wrong for eval/exec
Thomas Kluyver <tho...@kluyver.me.uk> added the comment: Time for some bug archaeology! The reason given for not using frame.f_code.co_filename for warnings was that it can be wrong when the path of .pyc files changes. However, it looks like that was fixed in #1180193 (thanks for the pointer zseil), so I'd like this idea to be reconsidered. Python uses frame.f_code.co_filename for tracebacks and in pdb, so it's counterintuitive that the warnings module works differently. They are all trying to do the same thing: find and show the bit of code that you're interested in. This causes problems in IPython with warnings attributed to interactive code: warnings sees it all as part of the __main__ module, so we can't distinguish which input it's pointing to (it's usually obvious, but still...). We have integrated with linecache to make tracebacks work, and I think that if warnings used code.co_filename, it would also work as expected. -- nosy: +takluyver ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue1692664> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22102] Zipfile generates Zipfile error in zip with 0 total number of disk in Zip64 end of central directory locator
Thomas Kluyver <tho...@kluyver.me.uk> added the comment: If every Windows 7 computer is generating zipfiles which are invalid in this way, that would be a pretty strong argument for Python (and other tools) to accept it. But if that was the case, I would also expect that there would be many more issues about it. Are the files you're compressing large (multi-GB)? Python only uses the zip64 format when the files are too big for the older zip format; maybe Windows is doing the same. Even in that case, I'm still surprised that more people don't hit it. -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue22102> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22102] Zipfile generates Zipfile error in zip with 0 total number of disk in Zip64 end of central directory locator
Thomas Kluyver <tho...@kluyver.me.uk> added the comment:
I found source code for some other projects handling the same data. They all
seem to agree that it should be 1:
- Golang's zip reading code:
https://github.com/golang/go/blob/f7ac70a56604033e2b1abc921d3f0f6afc85a7b3/src/archive/zip/reader.go#L536-L538
- A C contrib file with zlib:
https://github.com/madler/zlib/blob/cacf7f1d4e3d44d871b605da3b647f07d718623f/contrib/minizip/zip.c#L620-L624
- Code from Info-ZIP, which is used by many Linux distros, is a bit less clear,
but it has an illuminating comment:
if ((G.ecrec.number_this_disk != 0x) &&
(G.ecrec.number_this_disk != ecloc64_total_disks - 1)) {
/* Note: For some unknown reason, the developers at PKWARE decided to
store the "zip64 total disks" value as a counter starting from 1,
whereas all other "split/span volume" related fields use 0-based
volume numbers. Sigh... */
So I think you've got an invalid zip file. If it's otherwise valid, there might
be a case for Python tolerating that particular mistake. But it's probably
better to fix whatever is making the incorrect zip file, because other tools
are also going to reject it.
--
___
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue22102>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22102] Zipfile generates Zipfile error in zip with 0 total number of disk in Zip64 end of central directory locator
Thomas Kluyver <tho...@kluyver.me.uk> added the comment: Do you know what tool created the zip file? I can't find anything in the spec (https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT ) to say whether 0 is a valid value for the number of disks. -- nosy: +takluyver ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue22102> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12486] tokenize module should have a unicode API
Thomas Kluyver <tho...@kluyver.me.uk> added the comment: > Why not just bless the existing generate_tokens() function as a public API We're actually using generate_tokens() from IPython - we wanted a way to tokenize unicode strings, and although it's undocumented, it's been there for a number of releases and does what we want. So +1 to promoting it to a public API. In fact, at the moment, IPython has its own copy of tokenize to fix one or two old issues. I'm trying to get rid of that and use the stdlib module again, which is how I came to notice that we're using an undocumented API. -- nosy: +takluyver ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue12486> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32699] pythonXY._pth : unclear how .pth files are handled
Thomas Kluyver <tho...@kluyver.me.uk> added the comment: Thanks Steve, that looks good. -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue32699> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32699] pythonXY._pth : unclear how .pth files are handled
Thomas Kluyver <tho...@kluyver.me.uk> added the comment: I'd reword the second sentence like this: > Note that .pth files (without leading underscore) will be processed only if > this file specifies ``import site``. The current wording sounds (to me) like a guarantee that .pth files will still be handled, despite the other things that this mechanism switches off. -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue32699> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32699] pythonXY._pth : unclear how .pth files are handled
New submission from Thomas Kluyver <tho...@kluyver.me.uk>: Nicholas Tollervey has been getting Pynsist to use the new pythonXY._pth file to assemble sys.path. However, I'm not clear on how this behaves with .pth files. The docs on this (https://docs.python.org/3/using/windows.html#finding-modules ) appear to contradict themselves: > site is not imported unless one line in the file specifies import site > ... > Note that .pth files (without leading underscore) will be processed normally > by the site module. I can see two possibilities: 1. .pth files are processed normally if and only if the ._pth file specifies 'import site'. If it doesn't, they are ignored. 2. If a .pth file is encountered, site is imported to process it, but with sys.flags.nosite set, so that site doesn't do the normal stuff on import. Some packages unfortunately seem to depend on .pth files getting processed, so if 1. is the case, Pynsist needs to ensure site is loaded. Thanks :-) -- messages: 310980 nosy: steve.dower, takluyver priority: normal severity: normal status: open title: pythonXY._pth : unclear how .pth files are handled ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue32699> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32247] shutil-copytree: Create dst folder only if it doesn't exist
Thomas Kluyver <tho...@kluyver.me.uk> added the comment: This is documented: > The destination directory, named by dst, must not already exist https://docs.python.org/3/library/shutil.html#shutil.copytree I guess that avoids complications that might arise from merging a directory. -- nosy: +takluyver ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue32247> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30639] inspect.getfile(obj) calls object repr on failure
Thomas Kluyver added the comment: OK, I've updated the PR to do that. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30639> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30639] inspect.getfile(obj) calls object repr on failure
Thomas Kluyver added the comment: Sure, I can do that if you want. The other thing I thought of was using object.__repr__, so the repr always shows like . -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30639> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30639] inspect.getfile(obj) calls object repr on failure
Changes by Thomas Kluyver <tho...@kluyver.me.uk>: -- pull_requests: +2186 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30639> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30639] inspect.getfile(obj) calls object repr on failure
Thomas Kluyver added the comment: Not exactly a bug, but I think that an improvement is possible. My plan is to delay computing the repr until the error is formatted, so that code which catches the TypeError can carry on safely. I think this is more relevant for inspect than for many other modules, because by design, we're often calling inspect functions on arbitrary objects which we can make few guarantees about. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30639> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30639] inspect.getfile(obj) calls object repr on failure
New submission from Thomas Kluyver: This came up in IPython & Jedi: several functions in inspect end up calling getfile(). If the object is something for which it can't find the source file, it throws an error, the message for which contains the object's repr. This is problematic for us because for some objects the repr may be expensive to calculate (e.g. data tables where the repr forms primary output, not just debugging information). In some cases, it could also throw another error. I plan to make a PR for this, but I'm opening an issue so I've got a bpo number. https://github.com/ipython/ipython/issues/10493 https://github.com/davidhalter/jedi/issues/919 -- components: Library (Lib) messages: 295782 nosy: takluyver priority: normal severity: normal status: open title: inspect.getfile(obj) calls object repr on failure type: behavior ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30639> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25532] infinite loop when running inspect.unwrap over unittest.mock.call
Thomas Kluyver added the comment: I could go either way on that. It's not hard to imagine it as a recursive algorithm, and using the recursion limit provides a simple configuration escape hatch if someone has a desperate need for something wrapped 3000 times for some strange reason. But it may also be somewhat surprising if someone sets a really high recursion limit and suddenly it behaves oddly. I've made the PR with getrecursionlimit() for now, but I'm happy to change it if people prefer it the other way. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25532> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25532] infinite loop when running inspect.unwrap over unittest.mock.call
Thomas Kluyver added the comment: This was just reported in IPython as well (https://github.com/ipython/ipython/issues/10578 ). I've prepared a pull request based on Nick's proposal: https://github.com/python/cpython/pull/1717 -- nosy: +takluyver ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25532> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25532] infinite loop when running inspect.unwrap over unittest.mock.call
Changes by Thomas Kluyver <tho...@kluyver.me.uk>: -- pull_requests: +1806 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25532> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29622] ast module doesn't support optional fields of Parser/Python.asdl
Changes by Thomas Kluyver <tho...@kluyver.me.uk>: -- nosy: +takluyver ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29622> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13285] signal module ignores external signal changes
Thomas Kluyver added the comment: I pitched both the opaque handle and the context manager to python-ideas, but didn't get any responses, positive or negative. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue13285> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13285] signal module ignores external signal changes
Thomas Kluyver added the comment: I'd like to make the case for a fix in the code again. Our use case is, I believe, the same as Vilya's. We want to temporarily set a signal handler from Python and then restore the previous handler. This is fairly straightforward for Python handler functions, and SIG_DFL and SIG_IGN, but it breaks if anything has set a C level signal handler. The opaque wrapper object is a solution that had occurred to me too. Another option would be a context manager implemented in C (I assume context managers can be written in C) which can set one or more signal handlers on entry, and restore them on exit. -- nosy: +takluyver ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue13285> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27050] Demote run() below the high level APIs in subprocess docs
Thomas Kluyver added the comment: I still feel that having one function with various options is easier to explain than three separate functions with awkward names and limited use cases (e.g. no capturing output without checking the exit code). The tweeter you replied to said he didn't like subprocess.call(). If you really think the trio is a better starting point, though, you're the one with the power to change the docs ;-) There's more awkwardness in the subprocess API; I suspect that what that tweeter wants is something built around an event loop - like Node - so you can handle output incrementally using events. That's not something that we can easily fix in subprocess, because we don't have a default event loop to attach subprocesses to. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27050> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28528] Pdb.checkline()
New submission from Thomas Kluyver: Pdb.checkline() does a hasattr() check to protect against self.curframe not existing. self.curframe can also be None (if self.forget() or self.reset() was called), but checkline() does not handle this. The attached patch treats self.curframe == None as equivalent to the attribute being absent. Background: http://bugs.python.org/issue9230 https://github.com/ipython/ipython/issues/10028 (Georg, I've nosy-listed you as I saw your name on a couple of similar issues; I hope you don't mind) -- components: Library (Lib) files: pdb-reset-checkline.patch keywords: patch messages: 279402 nosy: georg.brandl, takluyver priority: normal severity: normal status: open title: Pdb.checkline() type: behavior versions: Python 3.5, Python 3.6, Python 3.7 Added file: http://bugs.python.org/file45217/pdb-reset-checkline.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28528> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28384] hmac cannot be used with shake algorithms
Changes by Thomas Kluyver <tho...@kluyver.me.uk>: -- nosy: +takluyver ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28384> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23395] _thread.interrupt_main() errors if SIGINT handler in SIG_DFL, SIG_IGN
Thomas Kluyver added the comment: Does Py_DECREF(arglist); need to be called in all cases? I'm genuinely unsure, as I don't usually work on C code, but my guess would be that it does. I'm not sure whether review is now done on Github. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue23395> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23395] _thread.interrupt_main() errors if SIGINT handler in SIG_DFL, SIG_IGN
Thomas Kluyver added the comment: It's waiting for someone to make a patch - I'm no longer running into it, so it's not high on my priority list. Given that it's been over a year since I created this issue, it's probably not about to get fixed unless you've got some time to work on it. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue23395> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23395] _thread.interrupt_main() errors if SIGINT handler in SIG_DFL, SIG_IGN
Thomas Kluyver added the comment: As far as I know it has not been fixed. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue23395> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27113] sqlite3 connect parameter "check_same_thread" not documented
Thomas Kluyver added the comment: Congrats! :-) -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27113> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27113] sqlite3 connect parameter "check_same_thread" not documented
Thomas Kluyver added the comment: That comment contained far too much 'probably'. Time for me to sleep... -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27113> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27113] sqlite3 connect parameter "check_same_thread" not documented
Thomas Kluyver added the comment: Having watched the video of Dave's PyCon talk (thanks Dave), I think he's talking about using locking to control transactions, which presumably the sqlite bindings don't handle by themselves. But I don't *think* you need manual locking just to maintain database integrity. If that's the case, it's probably helpful to clarify it. I'd probably suggest that if you want to write from multiple threads and control transactions (i.e. more than just individual INSERTs fired from different threads), it's probably easier to use separate connections and let sqlite handle the synchronisation than to synchronise them yourself. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27113> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27113] sqlite3 connect parameter "check_same_thread" not documented
Changes by Thomas Kluyver <tho...@kluyver.me.uk>: -- nosy: +takluyver ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27113> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27050] Demote run() below the high level APIs in subprocess docs
Thomas Kluyver added the comment: I'm obviously biased, but I find the 'high level convenience API' less convenient than the run() function: there are three different functions for the same basic operation, they're not clearly named (check_output is nothing to do with checking output), and there are things that should be simple but they can only do awkwardly (i.e. capturing both output and the exit code). Once I can depend on Python >= 3.5, I hope to never use call/check_call/check_output again. Using run() might make code slightly longer, but I think it also makes it clearer. I accept that the trio can probably never be removed, but this is why I demoted them a long way down the docs. Unfortunately I won't be at PyCon this year to discuss this. -- nosy: +takluyver ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27050> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26039] More flexibility in zipfile write interface
Thomas Kluyver added the comment: The backported package now exists on PyPI as zipfile36. It's passing the tests on 3.4 - I haven't tested further back. I'm trying out Gitlab for hosting it: https://gitlab.com/takluyver/zipfile36 -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26039> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26039] More flexibility in zipfile write interface
Thomas Kluyver added the comment: Updated version of the patch with the detail moved to rst. -- Added file: http://bugs.python.org/file42855/zipfile-flex-bonus2.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26039> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26039] More flexibility in zipfile write interface
Thomas Kluyver added the comment: Patch attached for points 2 and 3 -- Added file: http://bugs.python.org/file42851/zipfile-flex-bonus.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26039> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26039] More flexibility in zipfile write interface
Thomas Kluyver added the comment:
1. For the cases like read/write handles conflicting, ValueError doesn't seem
quite right to me - I take ValueError to mean that you have passed an invalid
value to a function. RuntimeError feels like the closest fit for 'the current
state does not allow this operation'. It is rather broad, though.
If I was writing it anew, I would use ValueError for something like an invalid
mode parameter: zf.open('foo', mode='q') . That particular case was already in
the code, though, so I think backwards compatibility should take precedence.
2 & 3. Agreed, I'll prepare a patch
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26039>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26039] More flexibility in zipfile write interface
Thomas Kluyver added the comment: Thanks Serhiy! I am marking this as fixed. -- resolution: -> fixed status: open -> closed ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26039> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26039] More flexibility in zipfile write interface
Thomas Kluyver added the comment: Thanks Serhiy. I'm quite happy with that set of constraints. I had a look over your patch set, and just spotted one thing in writestr that I have overlooked all along. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26039> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26039] More flexibility in zipfile write interface
Thomas Kluyver added the comment: Ping? Once this is landed, I intend to make a backport package of it so I can start using it before Python 3.6 comes out. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26039> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26039] More flexibility in zipfile write interface
Thomas Kluyver added the comment: Any further review comments on this? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26039> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26039] More flexibility in zipfile write interface
Thomas Kluyver added the comment: zipfile-open-w8 removes the concurrent read-write check for non-seekable files. As Martin points out, reading currently requires seeking, and a streaming read API would look very different from the current API. -- Added file: http://bugs.python.org/file42598/zipfile-open-w8.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26039> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26039] More flexibility in zipfile write interface
Thomas Kluyver added the comment: zipfile-open-w7 adds a test that Martin requested -- Added file: http://bugs.python.org/file42597/zipfile-open-w7.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26039> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26039] More flexibility in zipfile write interface
Thomas Kluyver added the comment: It wasn't as bad as I thought (hopefully that doesn't mean I've missed something). zipfile-open-w6.patch allows interleaved reads and writes so long as the underlying file object is seekable. You still can't have multiple write handles open (it wouldn't know where to start the second file), or read & write handles on a non-seekable stream (I don't think a non-seekable read-write file ever makes sense, except perhaps for a pty, which is really two separate streams on one file descriptor). Tests are passing again. -- Added file: http://bugs.python.org/file42588/zipfile-open-w6.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26039> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26039] More flexibility in zipfile write interface
Thomas Kluyver added the comment: Martin, Serhiy: Am I right that it would be unacceptable to change the test, and I therefore need to find a way to allow writestr while a reading-mode handle is open? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26039> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26039] More flexibility in zipfile write interface
Thomas Kluyver added the comment: Sorry for the delay, this fell off my radar because emails from both the bug tracker and Rietveld tend to fall foul of my spam filters, so I have to go and check. I have implemented Serhiy's suggestions, but there turns out to be a test (test_write_after_read) that calls writestr while a reading handle is open, and it now fails. This is absolutely expected according to the design we discussed, but if there's a test, I guess that means we need to keep that functionality working? In principle, the only thing that's not possible is interleaving writes to multiple files within the zip - because we don't know where to start writing the second file. We should be able to have one writer and n readers going at once, but every time I start looking into that I get mired in complexity. Maybe (hopefully) there's some critical abstraction that hasn't occurred to me. -- Added file: http://bugs.python.org/file42375/zipfile-open-w5.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26039> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26039] More flexibility in zipfile interface
Thomas Kluyver added the comment: Serhiy, have you had a chance to look at what the zf.open(mode='w') patch does with the lock? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26039> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26039] More flexibility in zipfile interface
Thomas Kluyver added the comment: Oh, I see test_interleaved now, which does test overlapping reads of two files from the same zip file. Do you want that clarified in the docs - which don't currently mention the lock at all - or in a comment in the module? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26039> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26039] More flexibility in zipfile interface
Thomas Kluyver added the comment: My initial patch would have allowed passing a readable file-like object into zipfile. I was persuaded that allowing ZipFile.open() to return a writable object was a more intuitive and flexible API. Concurrent writes with zf.open(mode='w') should be impossible, because it only allows one open handle at a time. It still uses the lock inside _ZipWriteFile, so concurrent writes to a single handle should be serialised. I would not recommend anyone try to do concurrent access to a single ZipFile object from multiple threads or coroutines. It's quite stateful, there is no mention of concurrency in the docs, and no tests I can see that try concurrent access. The only thing that might be safe is reading from two files inside the zip file (which shouldn't be changed by this), but I wouldn't want to guarantee even that. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26039> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26039] More flexibility in zipfile interface
Thomas Kluyver added the comment: Ping! ;-) -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26039> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26039] More flexibility in zipfile interface
Thomas Kluyver added the comment: Hi Serhiy, any more comments on the zf.open() patch? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26039> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26039] More flexibility in zipfile interface
Thomas Kluyver added the comment: Thanks Serhiy! I'll keep an eye out for comments on the other patch. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26039> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26039] More flexibility in zipfile interface
Thomas Kluyver added the comment: Is there anything more I should be doing with either of these patches? I think I've incorporated all review comments I've seen. Thanks! -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26039> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22107] tempfile module misinterprets access denied error on Windows
Thomas Kluyver added the comment: This issue was closed, but I believe the original bug reported was not fixed: trying to create a temporary file in a directory where you don't have write permissions hangs for a long time before failing with a misleading FileExistsError, rather than failing immediately with PermissionError. I've just run into this on Python 3.5.1 while trying to use tempfile to check if a directory is writable - which I'm doing precisely because os.access() isn't useful on Windows! I find it hard to believe that there is no way to distinguish a failure because the name is already used for a subdirectory from a failure because we don't have permission to create a file. -- nosy: +takluyver ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue22107> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26039] More flexibility in zipfile interface
Thomas Kluyver added the comment: Updated version of the ZipInfo.from_file() patch attached. -- Added file: http://bugs.python.org/file41758/zipinfo-from-file5.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26039> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26039] More flexibility in zipfile interface
Thomas Kluyver added the comment: Here's a new version of the zf.open() patch following Martin's review (thanks Martin!). I agree that it feels a bit awkward having two completely different actions for zf.open(), but it is a familiar interface, and since the mode parameter is already there, it requires a minimum of new public API. But I'm happy to add a new method like open_write() or write_handle() if people prefer that. The comments on the other patch are minimal, I'll put a new version of that together as well. -- Added file: http://bugs.python.org/file41752/zipfile-open-w3.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26039> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26039] More flexibility in zipfile interface
Changes by Thomas Kluyver <tak...@gmail.com>: Added file: http://bugs.python.org/file41753/zipinfo-from-file3.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26039> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26039] More flexibility in zipfile interface
Thomas Kluyver added the comment: Thanks Serhiy for review comments. -- Added file: http://bugs.python.org/file41754/zipinfo-from-file4.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26039> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26039] More flexibility in zipfile interface
Thomas Kluyver added the comment: The '2' versions of the two different patches include some docs and tests for these new features. -- Added file: http://bugs.python.org/file41726/zipfile-open-w2.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26039> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26039] More flexibility in zipfile interface
Thomas Kluyver added the comment: Serhiy, any chance you'd have some time to review my patch(es)? Or is there someone else interested in zipfile I might interest? Thanks :-) -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26039> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26039] More flexibility in zipfile interface
Thomas Kluyver added the comment: Thanks! I will work on docs and tests. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26039> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26039] More flexibility in zipfile interface
Changes by Thomas Kluyver <tak...@gmail.com>: Added file: http://bugs.python.org/file41722/zipinfo-from-file2.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26039> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8604] Adding an atomic FS write API
Thomas Kluyver added the comment: For reference, we added our own atomic_writing() context manager in Jupyter/IPython, after someone lost work when save failed on a full disk. We initially went for the same approach - write to a temporary file, fsync, rename over the original file - but it caused issues with network filesystems and folders synced by Dropbox or similar services. As a result we switched to an approach that copies the old file to a temporary name, writes the new data directly to the old name, then removes the temporary copy on success, or renames it back to the old name on failure. This is certainly less rigorous, but it means that we're not replacing the file on every save, and it's less of a problem if copying file attributes with copystat() fails. I'm sure there are use cases for atomic writing where these problems won't come up, but I hope this is a useful data point, at least for documenting any atomic writing solution. -- nosy: +takluyver ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue8604> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26039] More flexibility in zipfile interface
Thomas Kluyver added the comment: zipinfo-from-file.patch has an orthogonal but related change: the code in ZipFile.write() to construct a ZipInfo object from a filesystem file is pulled out to a classmethod ZipInfo.from_file(). Together, these changes make it much easier to control how a file is written to a zip file, like this: zi = ZipInfo.from_file(blah) # ... manipulate zi... with open(blah, 'rb') as src, zf.open(zi, 'w') as dest: # copy of the read/write loop - maybe this should be # pulled out separately as well? If these changes make it in, I might put a backported copy of the module on PyPI so I can start using it without waiting for Python 3.6. -- Added file: http://bugs.python.org/file41637/zipinfo-from-file.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26039> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18595] zipfile: symlinks etc.
Changes by Thomas Kluyver <tak...@gmail.com>: -- nosy: +takluyver ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue18595> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26039] More flexibility in zipfile interface
Thomas Kluyver added the comment: Attached is a first go at a patch enabling zipfile.open(blah, mode='w') Files must be written sequentially, so you have to close one writing handle before opening another. If you try to open a second one before closing the first, it will raise RuntimeError. I considered doing something where it would write to temporary files and add them to the zip file when they were closed, but it seemed like a bad idea. You can almost certainly break this by reading from a zip file while there's an open writing handle. Resolving this is tricky because there's a disconnect in the requirements for reading and writing: writing allows for a non-seekable output stream, but reading assumes that you can seek freely. The simplest fix is to block reading while there is an open file handle. I don't think many people will need to read one file from a zip while writing another, anyway. I have used the lock, but I haven't thought carefully about thread safety, so that should still be checked carefully. -- Added file: http://bugs.python.org/file41626/zipfile-open-w.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26039> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23936] Wrong references to deprecated find_module instead of find_spec
Thomas Kluyver added the comment: Updated patch attached, responding to Martin's suggestions. I'm not quite sure what to do about the 'finder' glossary entry. 'Finder' now seems to be a more abstract classification covering two distinct kinds of thing: *meta path finders* and *path entry finders*. They have similar methods (find_spec, find_module), but with slightly different signatures. Is 'finder' a sufficiently meaningful concept to remain in the glossary, or should we take it out and be explicit about what type of finder is meant elsewhere in the docs? -- Added file: http://bugs.python.org/file41237/finders_and_specs2.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue23936> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23936] Wrong references to deprecated find_module instead of find_spec
Thomas Kluyver added the comment: That's basically what I was aiming for. Should the description be briefer and more generic? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue23936> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23936] Wrong references to deprecated find_module instead of find_spec
Thomas Kluyver added the comment: Thanks Brett :-) -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue23936> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23936] Wrong references to deprecated find_module instead of find_spec
Thomas Kluyver added the comment: Third revision of patch responding to Brett's suggestions. -- Added file: http://bugs.python.org/file41240/finders_and_specs3.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue23936> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23936] Wrong references to deprecated find_module instead of find_spec
Thomas Kluyver added the comment: I also ran into confusion with this while trying to write an import hook. Attached is my shot at a docs patch. I've done a slightly larger rewording, trying to make it easier to understand, and added more links between parts of the docs. I left mentions of the deprecated find_module and find_loader methods in the 'finder' glossary entry, but for sys.meta_path I have only described find_spec() - it does still fall back to the deprecated methods, but I thought that trying to explain that would make the description of meta_path harder to follow. -- nosy: +takluyver Added file: http://bugs.python.org/file41227/finders_and_specs.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue23936> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25078] Document InstallAllUsers installer parameter default 0
New submission from Thomas Kluyver: In testing the 3.5.0rc4 default installer, I found that the default for InstallAllUsers appears to be 0, whereas it's currently documented as 1. -- assignee: docs@python components: Documentation files: installallusers-default.patch keywords: patch messages: 250515 nosy: docs@python, steve.dower, takluyver priority: normal severity: normal status: open title: Document InstallAllUsers installer parameter default 0 versions: Python 3.5, Python 3.6 Added file: http://bugs.python.org/file40445/installallusers-default.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25078> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24916] In sysconfig, don't rely on sys.version format
New submission from Thomas Kluyver: sysconfig currently calculates various formats of the Python version number by chopping up the sys.version string. This has a FIXME by it in the code, because the the format of sys.version is not guaranteed. With this patch, the config variables 'py_version', 'py_version_short' and 'py_version_nodot' are instead generated from sys.version_info, which has a specified structure: https://docs.python.org/3/library/sys.html#sys.version_info One piece of information is lost by this change: after a pre-release, a + is added to the version string - e.g. '3.5.0b4+' means an unreleased version somewhere after 3.5.0b4. I can't find any structured representation of this information, so 'py_version' no longer contains it. I'm not sure whether it matters: I can't find anything using the 'py_version' config variable. -- components: Library (Lib) files: sysconfig-version-fixme.patch keywords: patch messages: 248989 nosy: takluyver priority: normal severity: normal status: open title: In sysconfig, don't rely on sys.version format Added file: http://bugs.python.org/file40227/sysconfig-version-fixme.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24916 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24609] shutil.copytree fails with symlinks to directories when symlink=False
New submission from Thomas Kluyver: shutil.copytree behaves differently with symlinks depending on the 'symlinks' parameter. If this is True, symlinks are replicated in the destination. If False, the contents of the targets are copied to the destination. With symlinks=False, it currently assumes that all symlinks are pointing to regular files. With a symlink to a directory, it tries to copy it using the file copy function, which fails with: [Errno 21] Is a directory: '/tmp/tmpouavxt1u/link_to_dir' The attached patch adds an isdir() check to use copytree instead in that case. A test is also added. -- components: Library (Lib) files: shutil_copytree_symlink_dir.patch keywords: patch messages: 246585 nosy: takluyver priority: normal severity: normal status: open title: shutil.copytree fails with symlinks to directories when symlink=False versions: Python 3.4, Python 3.5, Python 3.6 Added file: http://bugs.python.org/file39892/shutil_copytree_symlink_dir.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24609 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
