Re: [Haskell-cafe] ANNOUNCE: hs-cryptohash 0.4
On 02/07/10 22:41, Gregory Crosswhite wrote: On 7/2/10 5:16 AM, Vincent Hanquez wrote: It's necessary in my case since i receive chunks of data to be hashed from the network, and I don't want to carry a buffer of data (with potential security issues), until i can hash everything. As an aside, this kind of pattern where you are processing chunks of input and producing a summary output sounds like a good fit for the Iteratees style of I/O. Indeed. I haven't played with iteratee yet, but it seems fairly easy for the user to add a iteratee interface on top of update and finalize. basically it seems that Chunk = update and EOF = finalize. Also, if you want the intermediate chunks to go away, then don't forget to use strict evaluation when you update the context. :-) That is unfortunately one of the kinds of bugs that doesn't screw up the behaviour of the program in a way that is immediately obvious. yes, you're absolutely right. I did actually forget in an early version of some code i've got on top of cryptohash, and it took me a good while to find ;) -- Vincent ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] ANNOUNCE: hs-cryptohash 0.4
On Fri, Jul 02, 2010 at 11:01:17AM -0400, Brandon S Allbery KF8NH wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 7/2/10 07:23 , Vincent Hanquez wrote: I'm not sure exactly what the API would looks like, but I think basically you would enter/leave the state monad quite frequently in incremental mode, since the whole point of the incremental api is having this context built partially. for this reason, you end up with something really close to explicit state keeping, isn't it ? You can do this if you need to (execState) but it's better to wrap the whole thing in a State monad. Think of it this way: (flip runState) can be read as withContext: flip runState initContext $ do -- do stuff, invoking update as needed to build the state I think that if the user wants to do that, he would have to build the state monad himself. Fortunately it's a trivial exercise in this case. In my case i'm having a state monad that is not only related to the hashing, but contains more state, so it would not be pratical having to lift into the digest context state monad just to update, then got back a context, that i would put in the bigger state monad. Although now that I think about it, if we're just appending to the state, this should possibly be a Writer instead of a State; the interface is simpler. Or to try to be a little more concrete about it: I think that's exactly what i don't want to expose for the simple reason that it's just too easy to shoot self in foot with that. for example, you might be in a situation where the data that you need to hash is really big, and appending Gb of data is not going to fly. In the end, I'ld rather leave this policy to the user, so that a careful user, on controlled case, could implement a writer monad. -- Vincent ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] ANNOUNCE: hs-cryptohash 0.4
Hi, Am Freitag, den 02.07.2010, 12:55 +1000 schrieb Ivan Miljenovic: This can be resolved by putting the build-depends line in the if statement (and should maybe put up the top of the executable section to make it more obvious): the latter was not possible last time I checked. Greetings, Joachim -- Joachim nomeata Breitner mail: m...@joachim-breitner.de | ICQ# 74513189 | GPG-Key: 4743206C JID: nome...@joachim-breitner.de | http://www.joachim-breitner.de/ Debian Developer: nome...@debian.org signature.asc Description: This is a digitally signed message part ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] ANNOUNCE: hs-cryptohash 0.4
On 2 July 2010 16:57, Joachim Breitner m...@joachim-breitner.de wrote: Am Freitag, den 02.07.2010, 12:55 +1000 schrieb Ivan Miljenovic: This can be resolved by putting the build-depends line in the if statement (and should maybe put up the top of the executable section to make it more obvious): the latter was not possible last time I checked. Do you mean putting the if statement at the top of the executable section? Works for me: http://hackage.haskell.org/packages/archive/graphviz/2999.9.0.0/graphviz.cabal (also in other cases that I've looked at). -- Ivan Lazar Miljenovic ivan.miljeno...@gmail.com IvanMiljenovic.wordpress.com ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] ANNOUNCE: hs-cryptohash 0.4
Hi, Am Freitag, den 02.07.2010, 17:14 +1000 schrieb Ivan Miljenovic: On 2 July 2010 16:57, Joachim Breitner m...@joachim-breitner.de wrote: Am Freitag, den 02.07.2010, 12:55 +1000 schrieb Ivan Miljenovic: This can be resolved by putting the build-depends line in the if statement (and should maybe put up the top of the executable section to make it more obvious): the latter was not possible last time I checked. Do you mean putting the if statement at the top of the executable section? Works for me: http://hackage.haskell.org/packages/archive/graphviz/2999.9.0.0/graphviz.cabal (also in other cases that I've looked at). sorry, no, I meant wrapping the whole executable section in an if statement, which to seems the most natural way of making an executable conditional. Greetings, Joachim -- Joachim nomeata Breitner mail: m...@joachim-breitner.de | ICQ# 74513189 | GPG-Key: 4743206C JID: nome...@joachim-breitner.de | http://www.joachim-breitner.de/ Debian Developer: nome...@debian.org signature.asc Description: This is a digitally signed message part ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] ANNOUNCE: hs-cryptohash 0.4
On Fri, Jul 02, 2010 at 12:55:03PM +1000, Ivan Miljenovic wrote: On 1 July 2010 17:25, Vincent Hanquez t...@snarc.org wrote: The main reason for this library is the lack of incremental api exposed by current digest libraries, and filling the void about some missing digest algorithms; Also the speed comes as a nice bonus. Can you explain what you mean by incremental API? The incremental API is the init/update/finalize functions where you can call update as many time as you need, instead of a single function hash where you need to hash all your data in one-go. It's necessary in my case since i receive chunks of data to be hashed from the network, and I don't want to carry a buffer of data (with potential security issues), until i can hash everything. The few existing packages that exposes the incremental API, usually do it in the IO monad; cryptohash do it purely, creating a new context as it get updated. (this has a cost but remains fast globally with the C implementation) i.e. update : ctx - bytestring - IO () becomes: update : ctx - bytestring - ctx (I'm not going to comment on the API decisions as I've never used any hashing functionality in my code and I thus wouldn't know what is good or bad.) First of all, Hackage couldn't seem to build it: http://hackage.haskell.org/packages/archive/cryptohash/0.4/logs/failure/ghc-6.12 ok, as a disclaimer: this the first time i'm using cabal sdist and uploading anything to hackage :) i forgot to add the .h files as extras, hence the build failure. Secondly, at the moment even if you disable the testing flag, whilst the executable isn't built Cabal still pulls in HUnit as a dependency. This can be resolved by putting the build-depends line in the if statement (and should maybe put up the top of the executable section to make it more obvious): ok I renamed the flag, and put the build-depends in the conditional. i'll move the conditional at the beggining in the next iteration of the package but i forgot before re-uploading the package. Oh, and in case it isn't obvious to people, the package name on hackage is just cryptohash, not hs-cryptohash. yes, I though that, on hackage prefixing things with hs- would make it a bit redundant. There's now a 4.0.1 package uploaded on hackage, that should fix most of the problems. Thanks for the comments ! -- Vincent ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] ANNOUNCE: hs-cryptohash 0.4
Vincent Hanquez t...@snarc.org writes: On Fri, Jul 02, 2010 at 12:55:03PM +1000, Ivan Miljenovic wrote: On 1 July 2010 17:25, Vincent Hanquez t...@snarc.org wrote: The main reason for this library is the lack of incremental api exposed by current digest libraries, and filling the void about some missing digest algorithms; Also the speed comes as a nice bonus. Can you explain what you mean by incremental API? The incremental API is the init/update/finalize functions where you can call update as many time as you need, instead of a single function hash where you need to hash all your data in one-go. It's necessary in my case since i receive chunks of data to be hashed from the network, and I don't want to carry a buffer of data (with potential security issues), until i can hash everything. The few existing packages that exposes the incremental API, usually do it in the IO monad; cryptohash do it purely, creating a new context as it get updated. (this has a cost but remains fast globally with the C implementation) i.e. update : ctx - bytestring - IO () becomes: update : ctx - bytestring - ctx So you're using explicit state parsing? Any particular reason for not using the State monad or something like that? -- Ivan Lazar Miljenovic ivan.miljeno...@gmail.com IvanMiljenovic.wordpress.com ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] ANNOUNCE: hs-cryptohash 0.4
On Fri, Jul 02, 2010 at 08:10:19PM +1000, Ivan Lazar Miljenovic wrote: i.e. update : ctx - bytestring - IO () becomes: update : ctx - bytestring - ctx So you're using explicit state parsing? Any particular reason for not using the State monad or something like that? hm, I'm basically just hiding the C call (IO) to %hash%_update. This is up to the user to put a state monad around it if he wants to. I'm not sure exactly what the API would looks like, but I think basically you would enter/leave the state monad quite frequently in incremental mode, since the whole point of the incremental api is having this context built partially. for this reason, you end up with something really close to explicit state keeping, isn't it ? also doing a bunch of update calls in a row is very practical at the moment: foldl HASH.update ctx [list,of,bytestrings,to,update,ctx,with] I don't think a state monad would make it *that* more practical ? unless maybe for scattered calls to update. If you have a better idea of how it could looks like, let me know. Cheers, -- Vincent ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] ANNOUNCE: hs-cryptohash 0.4
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 7/2/10 07:23 , Vincent Hanquez wrote: I'm not sure exactly what the API would looks like, but I think basically you would enter/leave the state monad quite frequently in incremental mode, since the whole point of the incremental api is having this context built partially. for this reason, you end up with something really close to explicit state keeping, isn't it ? You can do this if you need to (execState) but it's better to wrap the whole thing in a State monad. Think of it this way: (flip runState) can be read as withContext: flip runState initContext $ do -- do stuff, invoking update as needed to build the state Although now that I think about it, if we're just appending to the state, this should possibly be a Writer instead of a State; the interface is simpler. Or to try to be a little more concrete about it: type HashContext = String -- build up a hash context incrementally, then hash it hashWithContext :: () - Hash hashWithContext = hash . execWriter -- add a chunk to the hash context addToHash :: String - Writer HashContext () addToHash = tell -- use it messageHash = hashWithContext $ do -- do something addToHash aLine -- doSomethingElse foldM addToHash [aListOfStuff] -- ... - -- brandon s. allbery [linux,solaris,freebsd,perl] allb...@kf8nh.com system administrator [openafs,heimdal,too many hats] allb...@ece.cmu.edu electrical and computer engineering, carnegie mellon university KF8NH -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.10 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkwt/z0ACgkQIn7hlCsL25Uz5wCeO9db+eqE/azqoEgJ3L3vQJzF XKEAn0I8rEzf0tQfylc3pGq4+E6V4M72 =C/BN -END PGP SIGNATURE- ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] ANNOUNCE: hs-cryptohash 0.4
On Fri, Jul 02, 2010 at 08:10:19PM +1000, Ivan Lazar Miljenovic wrote: The few existing packages that exposes the incremental API, usually do it in the IO monad; cryptohash do it purely, creating a new context as it get updated. (this has a cost but remains fast globally with the C implementation) i.e. update : ctx - bytestring - IO () becomes: update : ctx - bytestring - ctx So you're using explicit state parsing? Any particular reason for not using the State monad or something like that? Not using the state monad allows explicit sharing/storing of the context, which would be quite handy if you arn't hashing your whole input in one go. John -- John Meacham - ⑆repetae.net⑆john⑈ - http://notanumber.net/ ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] ANNOUNCE: hs-cryptohash 0.4
On 7/2/10 11:01 AM, Brandon S Allbery KF8NH wrote: Although now that I think about it, if we're just appending to the state, this should possibly be a Writer instead of a State; the interface is simpler. The problem with this approach is that the hash context isn't a monoid; you can absorb data into the context, but you can't combine two hash contexts to form a new one. Thus, the Writer monad won't work for this purpose. Cheers, Greg ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] ANNOUNCE: hs-cryptohash 0.4
On 7/2/10 5:16 AM, Vincent Hanquez wrote: It's necessary in my case since i receive chunks of data to be hashed from the network, and I don't want to carry a buffer of data (with potential security issues), until i can hash everything. As an aside, this kind of pattern where you are processing chunks of input and producing a summary output sounds like a good fit for the Iteratees style of I/O. Also, if you want the intermediate chunks to go away, then don't forget to use strict evaluation when you update the context. :-) That is unfortunately one of the kinds of bugs that doesn't screw up the behaviour of the program in a way that is immediately obvious. Cheers, Greg ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] ANNOUNCE: hs-cryptohash 0.4
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 7/2/10 17:24 , Gregory Crosswhite wrote: The problem with this approach is that the hash context isn't a monoid; you can absorb data into the context, but you can't combine two hash contexts to form a new one. Thus, the Writer monad won't work for this purpose. No? The context is simply a String, which is a monoid, and you can combine contexts quite meaningfully before you actually compute the final hash (in fact, that's the whole point!). - -- brandon s. allbery [linux,solaris,freebsd,perl] allb...@kf8nh.com system administrator [openafs,heimdal,too many hats] allb...@ece.cmu.edu electrical and computer engineering, carnegie mellon university KF8NH -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.10 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkwuiMcACgkQIn7hlCsL25XSbACfZ72nV39ozNV2yvoTO1gtpewK AMMAn2mKd0j0xQ7Ns728Jzoz7Xq8GqSo =ZKBD -END PGP SIGNATURE- ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] ANNOUNCE: hs-cryptohash 0.4
On Fri, Jul 02, 2010 at 08:48:07PM -0400, Brandon S Allbery KF8NH wrote: On 7/2/10 17:24 , Gregory Crosswhite wrote: The problem with this approach is that the hash context isn't a monoid; you can absorb data into the context, but you can't combine two hash contexts to form a new one. Thus, the Writer monad won't work for this purpose. No? The context is simply a String, which is a monoid, and you can combine contexts quite meaningfully before you actually compute the final hash (in fact, that's the whole point!). The context isn't a string. it is an intermediate state in the algorithm of the hash function. it is usually an opaque binary blob (represented by a ByteString) of a size that is on the order of the final hash (unrelated to the size of the input). John -- John Meacham - ⑆repetae.net⑆john⑈ - http://notanumber.net/ ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] ANNOUNCE: hs-cryptohash 0.4
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 7/2/10 20:51 , John Meacham wrote: On Fri, Jul 02, 2010 at 08:48:07PM -0400, Brandon S Allbery KF8NH wrote: On 7/2/10 17:24 , Gregory Crosswhite wrote: The problem with this approach is that the hash context isn't a monoid; you can absorb data into the context, but you can't combine two hash contexts to form a new one. Thus, the Writer monad won't work for this purpose. No? The context is simply a String, which is a monoid, and you can combine contexts quite meaningfully before you actually compute the final hash (in fact, that's the whole point!). The context isn't a string. it is an intermediate state in the algorithm of the hash function. it is usually an opaque binary blob (represented by a ByteString) of a size that is on the order of the final hash (unrelated to the size of the input). If you read the example code I posted, the point was how to turn the current monolithic hash function into a cumulative one by using Writer. As such, there's no opaque hash state inside the Writer. (see `hash . runWriter'). - -- brandon s. allbery [linux,solaris,freebsd,perl] allb...@kf8nh.com system administrator [openafs,heimdal,too many hats] allb...@ece.cmu.edu electrical and computer engineering, carnegie mellon university KF8NH -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.10 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkwujqcACgkQIn7hlCsL25W6UQCfXrwrxclyns5s6TODstkO4z0a 1DwAoLLHJpZ36iEGiydkFy+0GTQWFbFI =bTrg -END PGP SIGNATURE- ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] ANNOUNCE: hs-cryptohash 0.4
On 7/2/10 9:13 PM, Brandon S Allbery KF8NH wrote: If you read the example code I posted, the point was how to turn the current monolithic hash function into a cumulative one by using Writer. As such, there's no opaque hash state inside the Writer. (see `hash . runWriter'). You can do that, but the price that you pay is that you are keeping all of the chunks around until the very end, rather than digesting each one into context as it comes in and then throwing it away. Cheers, Greg ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] ANNOUNCE: hs-cryptohash 0.4
On 1 July 2010 17:25, Vincent Hanquez t...@snarc.org wrote: The main reason for this library is the lack of incremental api exposed by current digest libraries, and filling the void about some missing digest algorithms; Also the speed comes as a nice bonus. Can you explain what you mean by incremental API? I'm looking forward to hear any comments, (I'm not going to comment on the API decisions as I've never used any hashing functionality in my code and I thus wouldn't know what is good or bad.) First of all, Hackage couldn't seem to build it: http://hackage.haskell.org/packages/archive/cryptohash/0.4/logs/failure/ghc-6.12 Secondly, at the moment even if you disable the testing flag, whilst the executable isn't built Cabal still pulls in HUnit as a dependency. This can be resolved by putting the build-depends line in the if statement (and should maybe put up the top of the executable section to make it more obvious): if flag(unittest) Buildable: True Build-depends: base = 3 5, HUnit, bytestring else Buildable: False Furthermore, the flag `test' is more common for enabling test suites than `unittest'. Oh, and in case it isn't obvious to people, the package name on hackage is just cryptohash, not hs-cryptohash. -- Ivan Lazar Miljenovic ivan.miljeno...@gmail.com IvanMiljenovic.wordpress.com ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
