Am Wed, 08 Aug 2012 02:49:00 -0700 schrieb Walter Bright <[email protected]>:
> > It should accept an input range. But using an Output Range confuses > me. A hash function is a reduce algorithm - it accepts a sequence of > input values, and produces a single value. You should be able to > write code like: > > ubyte[] data; > ... > auto crc = data.crc32(); auto crc = crc32Of(data); auto crc = data.crc32Of(); //ufcs This doesn't wok with every InputRange and this needs to be fixed. That's a quite simple fix (max 10 lines of code, one new overload) and not a inherent problem of the API (see below for more). > > For example, the hash example given is: > > foreach (buffer; file.byChunk(4096 * 1024)) > hash.put(buffer); > auto result = hash.finish(); > > Instead it should be something like: > > auto result = file.byChunk(4096 * 1025).joiner.hash(); But it also says this: //As digests implement OutputRange, we could use std.algorithm.copy //Let's do it manually for now You can basically do this with a range interface in 1 line: ---- import std.algorithm : copy; auto result = copy(file.byChunk(4096 * 1024), hash).finish(); ---- or with ufcs: ---- auto result = file.byChunk(4096 * 1024).copy(hash).finish(); ---- OK, you have to initialize hash and you have to call finish. With a new overload for digest it's as simple as this: ---- auto result = file.byChunk(4096 * 1024).digest!CRC32(); auto result = file.byChunk(4096 * 1024).crc32Of(); //with alias ---- The digests are OutputRanges, you can write data to them. There's absolutely no need to make them InputRanges as they only produce 1 value, and the hash sum is produced at once, so there's no way to receive the result in a partial way. A digest is very similar to Appender and it's .data property in this regard. The put function could accept an InputRange, but I think there was a thread recently which said this is evil for OutputRanges as the same feature can be achieved with copy. There's also no big benefit in doing it that way. If your InputRange is really unbuffered you could avoid double buffering. But then you transfer data byte by byte which will be horribly slow. If your InputRange has an internal buffer copy should just copy from that internal buffer to the 64 byte buffer used inside the digest implementation. This double buffering could only be avoided if the put function accepted an InputRange and could supply a buffer for that InputRange so the InputRange could write directly into the 64 byte buffer. But there's nothing like that in phobos, so this is all speculation. (Also the internal buffer is only used for the first 64 bytes (or less) of the supplied data. The rest is processed without copying. It could probably be optimized so that there's absolutely no copying as long as the input buffer length is a multiple of 64) > > The magic is that any input range that produces bytes could be used, > and that byte producing input range can be hooked up to the input of > any reducing function. See above. Every InputRange with byte element type does work. You just have to use copy. > > The use of a member finish() is not what any other reduce algorithm > has, and so the interface is not a general component interface. It's a struct with state, not a simple reduce function so it needs that finish member. It works like that way in every other language (and this is not cause those languages don't have ranges; streams and iterators (as in C#) work exactly the same in this case). Let's take a real world example: You want to download a huge file with std.net.curl and hash it on the fly. Completely reading into a buffer is not possible (large file!). Now std.net.curl has a callback interface (which is forced on us by libcurl). How would you map that into an InputRange? (The byLine range in std.net.curl is eager, byLineAsync needs an additional thread). A newbie trying to do that will despair as it would work just fine in every other language, but D forces that InputRange interface. Implementing it as an OutputRange is much better. The described scenario works fine and hashing an InputRange also works fine - just use copy. OutputRange is much more universal for this usecase. However, I do agree digest!Hash, md5Of, sha1Of should have an additional overload which takes a InputRange. It would be implemented with copy and be a nice convenience function. > > I know the documentation on ranges in Phobos is incomplete and > confusing. Especially for copy, as the documentation doesn't indicate the line I posted could work in any way ;-)
