On 2015-12-30 23:10, Bruce Evans wrote: > On Thu, 31 Dec 2015, Bruce Evans wrote: > >> >> wc /proc/0/* works. md5 works like wc using a hack to avoid calling this >> broken function. E.g., >> >> for i in $(ls /proc/0/*); do echo -n "$i: "; md5 <$i; done # gives >> >> /proc/0/cmdline: 3c5896b1ac441f4998f052e2126e8d20 >> /proc/0/ctl: d41d8cd98f00b204e9800998ecf8427e >> /proc/0/etype: 674441960ca1ba2de08ad4e50c9fde98 >> /proc/0/rlimit: 67d6ad67b412e1bceb7cb508a3492197 >> /proc/0/status: 3ccc3067b97c3232ea2dbcb64c458fd4 > > Further examples: > > md5 # on terminal input > > works correctly by not using MDXFileChunk(). > > md5 /dev/stdin # on the same terminal input > > produces the d41d8cd98f00b204e9800998ecf8427e garbage using > MDXFileChunk(). truss shows that lseek() is broken too -- MDXFileChunk() > tries it and it succeeds on the unseekable file /dev/stdin. Bugs in this > area are common. E.g., lseek() on named pipes was broken so that it > succeeded, by rearranging the plumbing use fileops more or less and not > attaching lseek right. > > cat | md5 # on the same terminal input > > works correctly by not using MDXFileChunk(). > > cat | md5 /dev/stdin # on the same terminal input > > doesn't work correctly, but it fails better than without the pipe. Now > a seek error occurs and is reported as "md5: /dev/stdin: Illegal seek" > (I can't see where it is reported). Then md5 exits. Then cat waits to > read input. Then cat fails to write output and is killed by SIGPIPE. > So md5 handled the seek error in a fail-safe though incorrect way. One > correct way is to fall back to the working code, but it is better to > just use that without an lseek check. > > It is a bug that [l]stat() on /dev/stdin sees the device file and not the > actual stdin file. md5 can't work around this except by not using stat(). > > Bruce >
It seems these problems also slow things down, a lot:
# time md5 /media/md5test/bigdata
MD5 (/media/md5test/bigdata) = 6afad0bf5d8318093e943229be05be67
4.310u 3.476s 0:07.79 99.8% 20+167k 0+0io 0pf+0w
# time env LD_PRELOAD=/usr/obj/media/svn/md5/head/tmp/lib/libmd.so
/usr/obj/media/svn/md5/head/sbin/md5/md5 /media/md5test/bigdata
MD5 (/media/md5test/bigdata) = 6afad0bf5d8318093e943229be05be67
4.133u 0.354s 0:04.49 99.7% 20+167k 1+0io 0pf+0w
(file is fully cached in ZFS ARC, dd reads it at 11GB/s)
Will investigate more tomorrow.
char *
MDXFile(const char *filename, char *buf)
{
unsigned char buffer[16*1024];
MDX_CTX ctx;
int f, i;
MDXInit(&ctx);
f = open(filename, O_RDONLY);
if (f < 0)
return 0;
i = 0;
while (1) {
i = read(f, buffer, sizeof(buffer));
if (i <= 0)
break;
MDXUpdate(&ctx, buffer, i);
}
close(f);
return (MDXEnd(&ctx, buf));
}
--
Allan Jude
signature.asc
Description: OpenPGP digital signature
