Re: [FFmpeg-user] FFmpeg & hashing algorithm speed
Thanks for the infos! And "yes": My PHP/Python examples were not related to FFmpeg, but this discovery led me to become curious how FFmpeg does it - and therefore ask this question :) Will read the Giorgio-Thread and maybe return with reproducible (commandline + uncut console output) question scenarios. Thank you very much! Pb On 07/11/2019 00:21, Carl Eugen Hoyos wrote: Am Do., 7. Nov. 2019 um 00:09 Uhr schrieb Carl Eugen Hoyos : Am Mi., 6. Nov. 2019 um 23:40 Uhr schrieb Peter B. : Hashing for fixity and transfer-checking seems to be quite a bottleneck for archiving big video/film files for archiving. Therefore I was considering if using "simpler" algorithms (like CRC instead of SHA256) could speed up the process. I was surprised that e.g. MD5 vs CRC in Python as well as PHP seemed to be equally "fast". But this is not related to FFmpeg, no? So I was wondering: Does FFmpeg have/do any speed optimization for hashing functionality (framemd5, hash muxer, etc) in its code? Some of Giorgio's commits were meant to increase hash function speed: http://git.videolan.org/?p=ffmpeg.git;a=search;s=vazzana;st=author There is an email with a script to compare the implementation's speeds, I don't know when it was sent to the development mailing list though. This mail leads to a long thread: http://mplayerhq.hu/pipermail/ffmpeg-devel/2013-May/143854.html In the end a file tools/ffhash.c was created to compare hash implementation speeds. Carl Eugen ___ ffmpeg-user mailing list ffmpeg-user@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-user To unsubscribe, visit link above, or email ffmpeg-user-requ...@ffmpeg.org with subject "unsubscribe". ___ ffmpeg-user mailing list ffmpeg-user@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-user To unsubscribe, visit link above, or email ffmpeg-user-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-user] FFmpeg & hashing algorithm speed
Am Do., 7. Nov. 2019 um 00:09 Uhr schrieb Carl Eugen Hoyos : > > Am Mi., 6. Nov. 2019 um 23:40 Uhr schrieb Peter B. : > > > Hashing for fixity and transfer-checking seems to be quite a bottleneck > > for archiving big video/film files for archiving. > > > > Therefore I was considering if using "simpler" algorithms (like CRC > > instead of SHA256) could speed up the process. > > > I was surprised that e.g. MD5 vs CRC in Python as well as PHP > > seemed to be equally "fast". > > But this is not related to FFmpeg, no? > > > So I was wondering: > > Does FFmpeg have/do any speed optimization for hashing functionality > > (framemd5, hash muxer, etc) in its code? > > Some of Giorgio's commits were meant to increase hash function > speed: > http://git.videolan.org/?p=ffmpeg.git;a=search;s=vazzana;st=author > There is an email with a script to compare the implementation's speeds, > I don't know when it was sent to the development mailing list though. This mail leads to a long thread: http://mplayerhq.hu/pipermail/ffmpeg-devel/2013-May/143854.html In the end a file tools/ffhash.c was created to compare hash implementation speeds. Carl Eugen ___ ffmpeg-user mailing list ffmpeg-user@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-user To unsubscribe, visit link above, or email ffmpeg-user-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-user] FFmpeg & hashing algorithm speed
Carl Eugen Hoyos (12019-11-07): > There is an email with a script to compare the implementation's speeds, > I don't know when it was sent to the development mailing list though. If you mean tools/crypto_bench.c, it was not only sent to the list but applied. Regards, -- Nicolas George signature.asc Description: PGP signature ___ ffmpeg-user mailing list ffmpeg-user@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-user To unsubscribe, visit link above, or email ffmpeg-user-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-user] FFmpeg & hashing algorithm speed
Am Mi., 6. Nov. 2019 um 23:40 Uhr schrieb Peter B. : > Hashing for fixity and transfer-checking seems to be quite a bottleneck > for archiving big video/film files for archiving. > > Therefore I was considering if using "simpler" algorithms (like CRC > instead of SHA256) could speed up the process. > I was surprised that e.g. MD5 vs CRC in Python as well as PHP > seemed to be equally "fast". But this is not related to FFmpeg, no? > So I was wondering: > Does FFmpeg have/do any speed optimization for hashing functionality > (framemd5, hash muxer, etc) in its code? Some of Giorgio's commits were meant to increase hash function speed: http://git.videolan.org/?p=ffmpeg.git;a=search;s=vazzana;st=author There is an email with a script to compare the implementation's speeds, I don't know when it was sent to the development mailing list though. > I've noticed, for example that "-f hash" for MD5 uses only half, SHA256 > just a quarter of my CPU. Command line and complete, uncut console output missing. Carl Eugen ___ ffmpeg-user mailing list ffmpeg-user@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-user To unsubscribe, visit link above, or email ffmpeg-user-requ...@ffmpeg.org with subject "unsubscribe".
