[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Change by Serhiy Storchaka : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed versions: +Python 3.11 -Python 3.6, Python 3.7, Python 3.8 ___ Python tracker

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Serhiy Storchaka added the comment: New changeset 4d2099f455229b10f88846dbe9fe6debbee55356 by Serhiy Storchaka in branch '3.9': [3.9] bpo-14156: Make argparse.FileType work correctly for binary file modes when argument is '-' (GH-13165) (GH-31979)

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Change by Serhiy Storchaka : -- pull_requests: +30069 pull_request: https://github.com/python/cpython/pull/31979 ___ Python tracker ___

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Change by Owen T. Heisler : -- nosy: -owenh ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

miss-islington added the comment: New changeset ee18df425209cfa4f394b57220177c168fc3a1da by Miss Islington (bot) in branch '3.10': bpo-14156: Make argparse.FileType work correctly for binary file modes when argument is '-' (GH-13165)

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Change by miss-islington : -- nosy: +miss-islington nosy_count: 17.0 -> 18.0 pull_requests: +29826 pull_request: https://github.com/python/cpython/pull/31706 ___ Python tracker

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Serhiy Storchaka added the comment: New changeset eafec26ae5327bb23b6dace2650b074c3327dfa0 by MojoVampire in branch 'main': bpo-14156: Make argparse.FileType work correctly for binary file modes when argument is '-' (GH-13165)

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

William Woodruff added the comment: Nosying myself; this affects 3.9 and 3.10 as well. -- nosy: +yossarian versions: +Python 3.10, Python 3.9 ___ Python tracker ___

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Change by Owen T. Heisler : -- nosy: +owenh ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Peter Wu added the comment: I just ran into this issue on Linux when piping a binary file to stdin resulted in a UnicodeDecodeError while trying to read a byte from the stream. Passing /dev/stdin is a workaround that does not require modifications to an application. As for the proposed PR

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Josh Rosenberg added the comment: I've created PR13165 to address this bug. It's 99% test updates; the actual code changes to argparse.py are trivial and should be equally trivial to review. The only functional change beyond Moritz's proposal is that I added support for having accepting '-'

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Change by Josh Rosenberg : -- pull_requests: +13082 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Josh Rosenberg added the comment: Serhiy: To be clear, Moritz's patch won't work if stdin/stdout are io.StringIO or the like either, since StringIO lacks a buffer attribute. Personally, I'm content to: 1. Ignore the closeability of the standard handles; that's not part of this bug, and

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Serhiy Storchaka added the comment: The solution with fileno() is clever, but as was mentioned before, it doesn't work if stdin or stdout are not real files, but something like StringIO. It is not that in common use of argparse for parsing arguments in scripts they are redefined, but

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Change by Serhiy Storchaka : -- versions: +Python 3.8 -Python 3.4, Python 3.5 ___ Python tracker ___ ___ Python-bugs-list mailing

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

paul j3 added the comment: It's been sometime since I looked at this issue. The main sticking point is passing unittests, and ensuring that there are no backward compatibility issues. But, since FileType is a standalone class, anyone could put a corrected version in their own workspace

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Change by hongweipeng : -- pull_requests: +8577 stage: test needed -> patch review ___ Python tracker ___ ___ Python-bugs-list

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Leo Singer added the comment: I just hit this bug. Would the proposed patch get any more attention if submitted as a pull request? -- nosy: +Leo Singer ___ Python tracker

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

R. David Murray added the comment: The biggest problem, as paul.j3 says, is to get someone from core to review the argparse issues. I am currently planning to make argparse one of my foci in a sprint we are doing at the beginning of September, so there is some hope Any reviews/testing

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

sedrubal added the comment: What is the problem with using the patch by moritz (https://bugs.python.org/issue14156#msg162342) and change the unit test to this: class TestFileTypeWB(TempDirMixin, ParserTestCase): ... successes = [ ..., ('-x - -', NS(x=sys.stdout.buffer,

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Changes by Antoine Pitrou : -- nosy: -pitrou ___ Python tracker ___ ___ Python-bugs-list

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Marcel H added the comment: I want to see this fixed in python3.x as well, please :) the patch should be the same -- nosy: +Marcel H2 versions: +Python 3.6, Python 3.7 ___ Python tracker

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Matteo Bertini added the comment: Bumped in this bug yesterday, sadly a script working (by chance) in Python2 doesn't work in Python3 because of this bug. -- nosy: +naufraghi ___ Python tracker

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Evan added the comment: This issue is relevant to Python 2 on Windows since you need to disable the EOL conversion if you're trying to receive binary data on stdin. See: http://stackoverflow.com/questions/2850893/reading-binary-data-from-stdin -- nosy: +evan_

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Changes by Matt Joiner : -- nosy: -anacrolix ___ Python tracker ___ ___

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Aviv Palivoda added the comment: As we talk here about stdin and stdout which we don't want to close I think this issue is irrelevant to python2. In any case the patch in issue #13824 cover that problem. I actually write a lot of small scripts and if I need to open the file in binary mode I

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

paul j3 added the comment: The problem with Python2.7 is that 'open' does not take 'closefd', or any of the other parameters that were added for Python3. open(name[, mode[, buffering]]) 'rb' may make a difference on Py2 on Windows, but I haven't done any work in the environment in a long

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Aviv Palivoda added the comment: Hi paul thanks for looking into this. First are you sure this is a bug in python 2? If so I will happily port this patch once it is reviewed. As for use cases you may look at issue #26488. Although the patch was rejected you can see that I first used

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

paul j3 added the comment: The problem with the argparse backlog is that the original author of the module is too busy with other things. And no one has stepped into his shoes. There are people with experience in apply patches, and people who know the argparse code well, but few, if any

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Matt Joiner added the comment: This is why I stopped contributing to Python. -- ___ Python tracker ___ ___

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Changes by paul j3 : -- priority: normal -> high ___ Python tracker ___ ___

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Aviv Palivoda added the comment: Pinging as mentioned in the devguide. -- ___ Python tracker ___ ___

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Aviv Palivoda added the comment: I fixed the tests to work with Steven patch. Also changed the patch to open sys.std{in,out} with closefd=False. I changed the 'io.StringIO' that we redirect the stdout, stderr to. Now the 'StdIOBuffer' return the real stdout,stderr when '-' is passed to

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Changes by Serhiy Storchaka storch...@gmail.com: -- stage: - test needed type: crash - behavior versions: -Python 3.2, Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14156 ___

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Eli Bendersky added the comment: The patch looks reasonable? Is the only remaining problem with crafting the test? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14156 ___

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Eli Bendersky added the comment: [sorry, the first question mark shouldn't be - the patch indeed looks reasonable to me] Steven - how about launching a subprocess for stdin tests to avoid weird issues? -- ___ Python tracker rep...@bugs.python.org

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Eli Bendersky added the comment: Nosy-ing myself since I just ran into it. Annoying issue that precludes from using argparse's builtin '-' recognition for reading binary data. I'll try to carve some time later to look at the patches. -- nosy: +eli.bendersky versions: +Python 3.3,

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Changes by Mark Grandi markgra...@gmail.com: -- nosy: +markgrandi ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14156 ___ ___ Python-bugs-list

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

paul j3 added the comment: There are a couple of complications to using 'fileno'. We probably don't want to close 'sys.stdin' or 'sys.stdout' (not even if they are redirected to other files?). That means using: open(sys.stdin.fileno(), ..., closefd=False) 'closefd', on the other hand,

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

paul j3 added the comment: A related issue http://bugs.python.org/issue13824 argparse.FileType opens a file and never closes it I proposed a FileContext class that returns a `partial(open, filename,...)' context object. The issues of how to deal with stdin/out are similar. --

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Changes by paul j3 ajipa...@gmail.com: -- nosy: +paul.j3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14156 ___ ___ Python-bugs-list mailing list

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Changes by Josh Rosenberg shadowranger+pyt...@gmail.com: -- nosy: +josh.rosenberg ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14156 ___ ___

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Steven Bethard steven.beth...@gmail.com added the comment: The fix looks right, but we definitely need a test. I tried to write one, but I'm not sure how to do this properly given how test_argparse redirects standard input and output (so that fileno() doesn't work anymore). I've attached my

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Moritz Klammler moritz.klamm...@gmail.com added the comment: I don't know how if this is the perfect solution but it keeps the program from crashing. 1154c1154,1157 return _sys.stdin --- if 'b' in self._mode: return _sys.stdin.buffer

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Matt Joiner anacro...@gmail.com added the comment: Steven, patch attached. I lost steam in the unittests with all the meta, suffice it that the names match the file descriptors of the stream sources. i.e. FileType('rb') would give a file with name=0, and so forth. My chosen method also allows

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Matt Joiner anacro...@gmail.com added the comment: Roger that. I'll start on a patch for this in a month or two if all goes well. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14156 ___

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

New submission from Matt Joiner anacro...@gmail.com: If an argument of '-' is handled by argparse.FileType, it defaults to sys.stdin. However a mode of 'rb' is ignored, the returned file object does not work with raw bytes. -- components: Library (Lib) messages: 154612 nosy: anacrolix

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Changes by Éric Araujo mer...@netwok.org: -- nosy: +bethard, pitrou ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14156 ___ ___ Python-bugs-list

[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'

Steven Bethard steven.beth...@gmail.com added the comment: Yes, the problem is in FileType.__call__ - the handling of '-' is pretty simplistic. Patches welcome. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14156