Just wanted to add the Apache commons codec has a base64
encoder/decoder. With a quick test I was able to encode a ~100MB file
in 2.3sec. Example code below:
In leiningen: [commons-codec "1.4"]
(require '[clojure.java.io :as io])
(import '[org.apache.commons.codec.binary Base64OutputStream])
(defn encode [src dest]
(with-open [in (io/input-stream src)
out (-> dest (io/output-stream) (Base64OutputStream.)))]
(io/copy in out :buffer-size 8192)))
(.length (io/file "/tmp/blob"))
=> 99667238
(time (encode "/tmp/blob" "/tmp/output"))
=> "Elapsed time: 2254.521259 msecs"
org.apache.commons.codec.binary.Base64 has helper methods as well.
Allen
On Thu, Oct 6, 2011 at 1:27 PM, Aaron Bedra <aaron.be...@gmail.com> wrote:
> This actually introduces an opportunity for a much larger set of utilities.
>
> clojure.data.crypto
>
> base64 is part of this idea anyways, and putting it in place along with
> nice wrappers around the messy java crypto bits I think could provide a
> significant win. I have had to do this several times now and it would
> be great to have in the future. The thing that always pushed me away was
> relying on sun.misc when all of the jdks don't have it.
>
> Cheers,
>
> Aaron Bedra
> --
> Clojure/core
> http://clojure.com
>
> On 10/06/2011 08:07 AM, Stuart Halloway wrote:
>>> I use Base64 encoding a lot and the slow implementation is hurting a
>>> lot. It's slower than Sun misc encoder/decoder
>>> and that one is very very slow. I was using Sun's implementation a bit
>>> and it took 80 seconds to encode a 56 MB file.
>>> Then I found this: http://migbase64.sourceforge.net/
>>> It loaded from a disk drive and encoded the same file in 0.3 seconds!
>>> Would it be possible to have Clojure contrib use an implementation
>>> like this and thus enable all Clojure developers to have lightning
>>> fast Base64 encoding/decoding? I know having everything implemented in
>>> clojure is a big thing around here but in case of Base64 encoding and
>>> regular expressions I think the language would benefit greatly by
>>> having implementations which are orders of magnitude faster than
>>> default implementations in java. Did I say regular expressions?
>>> http://www.tusker.org/regex/regex_benchmark.html
>>> Like Rich Hickey said: why should we reinvent file streams and sockets
>>> in each language?
>>> I think the same principle applies to Base64 and regular expressions,
>>> especially when in Clojure we have an opportunity to replace standard
>>> java implementations with ones that are orders of magnitude faster.
>>> What are your views on this?
>> I too would like a fast base64. There are a few different things we could do:
>>
>> 1. Vet the best Java one and link it from contrib.base64 (and possibly
>> deprecate the contrib as well).
>>
>> 2. Approve a base64 lib for inclusion by reference in contribs.
>>
>> 3. Write a fast clojure one (possibly a direct port of the project you link,
>> if it is the best).
>>
>> I have several Clojure projects that ship with no binary dependencies. This
>> is a maintenance virtue, which is why I think #3 is important.
>>
>> If somebody writes a good one I will fast track getting it into contrib.
>>
>> Stu
>>
>>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with your
> first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
