Hi, On Wednesday, March 6, 2013 2:53:26 PM UTC+1, Alan Busby wrote: > > > With the release of Clojure 1.5 and it's new reducers, I figured this > would be a good time to release a library to help with file IO for > reducers. As reducers can only operate in parallel over specific > collections, like vec, it requires some work to use them against files. > > Iota wraps a text files using Java NIO's mmap() so you can treat files > larger than memory like a vector. Clojure's normal collection functions > (first, last, nth, etc) work, while enabling 1.5's reducer's to operate in > parallel for fold. >
Thanks, I already love your lib ! However, in order to squeeze the last bits of performance, what I'd really need would to be directly access the raw data as byte arrays. The ideal API would provide my function with things looking and quaking like bytes arrays but doing the idx offset maths to directly access the maped data (read-only ), the next-to ideal would provide my function the ref to whole bytes array and give me the start and end idx for current line. (I'd like to port a Java code where the most significant performance boost was avoiding Strings creation when at all possible, which was possible by only peeking at most of the lines). Cheers, B. -- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
