[issue26488] hashlib command line interface

Changes by Raymond Hettinger : -- resolution: -> rejected status: open -> closed ___ Python tracker ___

[issue26488] hashlib command line interface

Guido van Rossum added the comment: I prefer not to go down this road. The modules that do this where I use it are typically Python specific, e.g. pdb or timeit. In the past we sometimes had little main() functions in modules for testing but I think we have better ways to test modules these

[issue26488] hashlib command line interface

Raymond Hettinger added the comment: Guido, would you can to opine on this? Every now and then we get a request to make command-line utilities out of tools in the standard library. Whether we should or not depends on whether the standard library intends to be primarily a library for Python

[issue26488] hashlib command line interface

Changes by Raymond Hettinger : -- assignee: rhettinger -> ___ Python tracker ___

[issue26488] hashlib command line interface

Antonio Valentino added the comment: Hi all, please note that to stay compatible with the GNU md5sum utility you should print the file name prefixed by a "*" if you read the file in binary mode. Also when digests is checked, files to check should be opened in binary or text mode according the

[issue26488] hashlib command line interface

STINNER Victor added the comment: > There are so many existing tools that already do this I don't really see why Python needs to become yet another one. What is the value in doing this? Portability. You described UNIX. There is no such tool by default on Windows for example (as Avid said).

[issue26488] hashlib command line interface

Aviv Palivoda added the comment: The use case that made me think about this feature was when I was working on a Windows PC and needed to calculate an md5 of a file. I agree that in a unix environment there are existing tools but on windows you usually don't have them. --

[issue26488] hashlib command line interface

Gregory P. Smith added the comment: There are so many existing tools that already do this I don't really see why Python needs to become yet another one. What is the value in doing this? Just use the openssl command. "openssl sha256 myfile" Or any of the md5sum, sha1sum and other plethora

[issue26488] hashlib command line interface

Aviv Palivoda added the comment: Hi, is there anything more I need to do on this patch? If not do you think this can be added in 3.6? -- ___ Python tracker

[issue26488] hashlib command line interface

Aviv Palivoda added the comment: Adding new patch after CR changes. -- Added file: http://bugs.python.org/file42355/hashlib-script-mod-md5sum-style-5.patch ___ Python tracker

[issue26488] hashlib command line interface

Martin Panter added the comment: I left some replies to Rietveld comments (sending review emails seems buggy). For a chunk size, don’t worry too much about it. I would say keep it large enough to limit time spent executing Python code and syscalls, keep it small to avoid wasting high speed

[issue26488] hashlib command line interface

Aviv Palivoda added the comment: Publishing another patch after SilentGhost and Victor CR. I also changed the block size to 256 KB. If someone can remove the dependency on issue 14156 (I don't think I have permissions). -- Added file:

[issue26488] hashlib command line interface

STINNER Victor added the comment: > The blocksize should be fixed and large (perhaps 256kB). I used strace to check md5sum & sha1sum: they use read() syscalls of 32,768 bytes. stat().st_blksize is 4,096 bytes. I'm not sure that it matters so much to use large read. But I don't really care,

[issue26488] hashlib command line interface

SilentGhost added the comment: Left comments on Rietveld. -- ___ Python tracker ___ ___ Python-bugs-list

[issue26488] hashlib command line interface

Aviv Palivoda added the comment: I am adding a new patch with changes from Martin CR (Thanks for the review) and support in the "check" option. I also changed to examples in the Documentation to use sha256 instead of md5 as Christian asked. I left one example with sha1 so when someone read it

[issue26488] hashlib command line interface

Changes by Aviv Palivoda : Added file: http://bugs.python.org/file42337/hashlib-script-mod-md5sum-style-2.patch ___ Python tracker ___

[issue26488] hashlib command line interface

Aviv Palivoda added the comment: Thanks for the review SilentGhost. I am including the patch after the changes from your CR comments. -- ___ Python tracker

[issue26488] hashlib command line interface

Christian Heimes added the comment: Threading doesn't make much sense here. The runtime of hash computation is usually dominated by I/O performance. On a typical consumer computer SSDs have a sequential read performance of 200 to 500 MiB/sec. SHA-512 performance between 100 to 150 Mib/sec.

[issue26488] hashlib command line interface

SilentGhost added the comment: The mailing system is acting up, so just for the record, I've left comments on rietveld regarding md5sum-style patch. -- ___ Python tracker

[issue26488] hashlib command line interface

Aviv Palivoda added the comment: I am adding a new patch with a API compatible to GNU md5sum: $ python -m hashlib md5 /bin/sh $ d985d0ea551c1253c2305140c583d11f /bin/sh I will soon add the feature's requested by Victor: 1) The check option. 2) Running the hash calculation for different

[issue26488] hashlib command line interface

Raymond Hettinger added the comment: I concur. The blocksize should be fixed and large (perhaps 256kB). -- ___ Python tracker ___

[issue26488] hashlib command line interface

Christian Heimes added the comment: I see a potential performance issue here. The block size is a small value, usually a couple of kb. With such a small value, the runtime will be dominated by Python call overhead and syscalls. -- ___ Python

[issue26488] hashlib command line interface

Aviv Palivoda added the comment: Sorry on the late response for some reason i don't receive email notification from the tracker for the past few days. 1) Thanks for the review SilentGhost the patch attached include your CR suggestions. 2) Raymond I have fixed the problem with ctrl+D. I tried

[issue26488] hashlib command line interface

STINNER Victor added the comment: > I think we should add a dependency in issue 14156 for this issue. Maybe you can start using strings for filename parameters and open files manually, and later consider switching back to FileType once it's fixed? --

[issue26488] hashlib command line interface

STINNER Victor added the comment: About the compatibility with existing tools, I recall a discussion when the tarfile module got a CLI. First I expected a clone of the UNIX tar command, but it was decided to design a new *simpler* CLI. --- $

[issue26488] hashlib command line interface

Raymond Hettinger added the comment: Overall, I'm +1 on the idea and think the patch looks good. There are two issues to look at: 1) Neither the md5 or shasum command-line tools offer control over the blocksize. I suggest that option be dropped from the command-line API giving a nice

[issue26488] hashlib command line interface

SilentGhost added the comment: I've left some comment on rietveld yesterday, not sure if you got the e-mail. -- ___ Python tracker ___

[issue26488] hashlib command line interface

Raymond Hettinger added the comment: Here is a quick check versus other command-line tools: $ md5 xml_prolog.diff MD5 (xml_prolog.diff) = c30e1fe70651d5a472efe9598db70771 $ python3.6 -m hashlib md5 xml_prolog.diff c30e1fe70651d5a472efe9598db70771 $ shasum -a 1 xml_prolog.diff

[issue26488] hashlib command line interface

Changes by Raymond Hettinger : -- assignee: -> rhettinger nosy: +rhettinger ___ Python tracker ___

[issue26488] hashlib command line interface

Changes by SilentGhost : -- dependencies: +argparse.FileType for '-' doesn't work for a mode of 'rb' nosy: +SilentGhost ___ Python tracker

[issue26488] hashlib command line interface

Aviv Palivoda added the comment: I actually have noticed issue 13824 while working on this issue. The patch I uploaded to Issue 14156 actually fixes the problem in issue 13824 in addition to the problem with the stdin/stdout. I think we should add a dependency in issue 14156 for this issue.

[issue26488] hashlib command line interface

Martin Panter added the comment: FWIW I am not really comfortable with argparse.FileType; see Issue 13824. How does this patch perform when warnings are enabled, or if you specify a file, but then cause some other argparse error? -- nosy: +martin.panter

[issue26488] hashlib command line interface

Terry J. Reedy added the comment: This strikes me as a sensible addition. While I did not review the patch in detail, it seems cleanly written and is complete in having a new main() for the code, a new doc section, and a new unittest Test class. -- nosy: +terry.reedy stage: -> patch

[issue26488] hashlib command line interface

New submission from Aviv Palivoda: I am suggesting to add a command line interface to the hashlib module. A simple example of the api I suggest is: $ python -m hashlib md5 /bin/sh $ d985d0ea551c1253c2305140c583d11f A patch is included. -- components: Library (Lib) files: