On Wed, Jul 15, 2015 at 12:02 PM, Daniel Higginbotham
nonrecurs...@gmail.com wrote:
Surely reducers/map should be faster, but it doesn’t seem like I’ve done
something wrong? This is driving me crazy :)
It is much faster, and runs in parallel, if you use fold.
Try this;
(defn fold-into-vec
I wrote a library ( https://github.com/thebusby/iota ) to handle a very
similar issue, which I hope would be of some help to you here.
On Wed, May 6, 2015 at 12:18 PM, Sam Raker sam.ra...@gmail.com wrote:
I've got two really big CSV files that I need to compare. Stay tuned for
the
On Mon, Oct 27, 2014 at 7:10 AM, Brian Craft craft.br...@gmail.com wrote:
I found iota, which looks like a good solution for the read portion of the
problem. However I also need to process the data in the file. If I start
with an iota/vec and need to sort it, something like
(sort (iota/vec
On Mon, Oct 27, 2014 at 12:52 PM, Brian Craft craft.br...@gmail.com wrote:
Makes sense, but not an option for this application. What about something
similar to iota, backed with byte arrays, or something?
As Patrick pointed out, if you're working directly with byte array's you
might want to
On Tue, Sep 9, 2014 at 3:09 AM, Sean Corfield s...@corfield.org wrote:
I find that it is quite difficult to use clojure unless one knows Java,
which I believe to be a barrier to new comers.
I'm surprised every time I hear this. You can write a lot of Clojure
without having to do any interop
On Fri, Feb 21, 2014 at 8:51 AM, shlomivak...@gmail.com wrote:
One note about your SuperVecs idea though, it seems that using that
approach we could get a deeply nested tree to represent our a vector, and I
think this is the exact thing vectors are trying to avoid..
In general I think this
office here in Shibuya if
there is enough interest. If schedules work out Mr. Ambrose may be able to
give us a quick intro / Q+A on core.typed as well.
Anyone interested?
Thanks,
Alan Busby
On Thu, Oct 3, 2013 at 2:32 PM, Dave Della Costa ddellaco...@gmail.comwrote:
Speaking
On Thu, Oct 3, 2013 at 9:30 AM, Marek Kubica ma...@xivilization.net wrote:
On Tue, 1 Oct 2013 22:01:02 -0700 (PDT)
Nicolas Modrzyk hellon...@gmail.com wrote:
The whole text has also been written in English, so I guess there
could be a chance to put it out there.
But we have had no
for advertising the power of Clojure, it's likely
safe to say that it should be ignored if you're actually looking for
performance.
Hope this helps,
Alan Busby
--
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email
If you're working with large text files, tsv especially, you may encounter
Java's memory overhead for Strings as well.
I remember parsing a 1GB TSV file into EDN, and watching it consume 10GB of
RAM.
I'd recommend taking a look at the Iota library to see if it would be of
any help to you,
On Thu, Sep 5, 2013 at 2:35 PM, Dave Ray dave...@gmail.com wrote:
could you have a macro that rewrites code to use your ops?
(require '[clojure.core.matrix :as m])
(m/with-ops (+ ... (* ...) ...))
I wonder if there is value in a more general (with-ns clojure.core.matrix
(foo (bar ...)))
This isn't the solution you were looking for, but it should work;
1. Add simmetrics to your local maven repo;
$ mvn install:install-file
-Dfile=*simmetrics_jar_v1_6_2_d07_02_07.jar* -DgroupId=simmetrics
-DartifactId=simmetrics -Dversion=1.6.2 -Dpackaging=jar -DgeneratePom=true
2. Refer to it in
On Wed, Aug 28, 2013 at 12:18 PM, guns s...@sungpae.com wrote:
Oh, I was confused; I was thinking about sentinel values in user code.
Yes, I imagine a single private (Object.) would work just fine, with
very little overhead.
First, I'd hope that sentinel values would be handled by the
Here is something similar pulled from bagotricks 1.5.2, using Java's
linked blocking queue;
https://github.com/thebusby/bagotricks/blob/master/src/bagotricks.clj#L204-L238
It uses fold instead of reduce to run in parallel, so has a slightly
different use case than above.
On Mon, Aug 26, 2013 at
On Mon, Aug 26, 2013 at 1:37 PM, Timothy Baldridge tbaldri...@gmail.comwrote:
Since reducers use fork-join pools,
Reducers use multiple threads and fork-join pools when called with fold
on vectors (anything else?), not reduce.
By making the single producer thread of the reducer block on
On Wed, Jun 19, 2013 at 4:03 PM, Tassilo Horn t...@gnu.org wrote:
I might be wrong, but I think reducers are only faster in situations
where you bash many filters/maps/mapcats/etc on top of each other.
Or use fold on a large vector with a multi-core machine.
You might try fold-into-vec to
On Sat, Apr 27, 2013 at 7:35 PM, Stanislav Yurin jusk...@gmail.com wrote:
Actually, what I was trying to do, is to prototype multithreaded i/o
operation via reducers. And then use fold to regulate number of concurrent
operations.
But now something tells me I am doing not very clever thing.
On Sat, Apr 27, 2013 at 10:01 PM, Stanislav Yurin jusk...@gmail.com wrote:
By the way, fold function has [n combinef reducef coll] implementation,
where n is number of elements collection is folded by. 512 is just the
default.
Yep I misspoke there, but it is still one of the tricks to be
Hi All,
You may want to take a look at Prismatic's graph library, it does what
you've described above in a slightly different way.
Link: https://github.com/Prismatic/plumbing
On Sun, Apr 28, 2013 at 1:04 AM, Ben Wolfson wolf...@gmail.com wrote:
It's not too hard, though, to write a plet form
Some additional pointers here (this is a little old though);
http://www.thebusby.com/2012/07/tips-tricks-with-clojure-reducers.html
On Fri, Apr 26, 2013 at 11:51 PM, László Török ltoro...@gmail.com wrote:
Hi,
Not sure what you are trying to do, but xxx is a lazy seq, thus it can
only be
On Mon, Mar 25, 2013 at 12:24 PM, Rich Morin r...@cfcl.com wrote:
I can wait a bit for the editing;
clean results are more important than saving a month or so.
I wouldn't say anything if it was only a month, it's actually closer to 3-7
months after the conference though.
Clojure West 2012
On Sat, Mar 23, 2013 at 2:48 PM, Alex Miller a...@puredanger.com wrote:
The benefit to attendees and non-attendees is that the videos exist at all
- without the InfoQ deal, the cost of recording, editing, and hosting
videos is literally the difference between whether the conference is in the
On Tue, Mar 12, 2013 at 11:00 PM, Paul Butcher p...@paulbutcher.com wrote:
On 12 Mar 2013, at 13:49, Adam Clements adam.cleme...@gmail.com wrote:
How would feeding a line-seq into this compare to iota? And how would that
compare to a version of iota tweaked to work in a slightly less eager
to allow for
(- (iota/vec filename.tsv)
(r/filter identity)
...
Hope this helps,
Alan
On Sat, Mar 9, 2013 at 9:41 AM, bernardH un.compte.pour.tes...@gmail.comwrote:
Hi Alan,
On Friday, March 8, 2013 4:02:18 PM UTC+1, Alan Busby wrote:
Hi Bernard,
I'd certainly like
On Mon, Mar 11, 2013 at 9:40 AM, Paul Butcher p...@paulbutcher.com wrote:
I'm currently working on code that processes XML generated by
clojure.data.xml/parse, and would love to do so in parallel. I can't
immediately see any reason why it wouldn't be possible to create a version
of CollFold
, Alan Busby wrote:
With the release of Clojure 1.5 and it's new reducers, I figured this
would be a good time to release a library to help with file IO for
reducers. As reducers can only operate in parallel over specific
collections, like vec, it requires some work to use them against files
is available on Clojars here: https://clojars.org/iota
Hope it helps,
Alan Busby
--
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please
On Tue, Jan 8, 2013 at 12:18 AM, Wolodja Wentland babi...@gmail.com wrote:
On Thu, Nov 01, 2012 at 22:34 +0900, Alan Busby wrote:
On Thu, Nov 1, 2012 at 8:27 PM, Wolodja Wentland babi...@gmail.com
wrote:
Oh, fold-into-map and fold-into-map-with would be wonderful and I
tried
than having to be written as special
cases.
Andy
On Dec 27, 2012, at 8:54 PM, Alan Busby wrote:
I'm confused why we'd need to give up O(1) just to support something like
reduce-kv on subvectors.
Isn't the implementation of subvector just a wrapper around the original
vector along
I'm confused why we'd need to give up O(1) just to support something like
reduce-kv on subvectors.
Isn't the implementation of subvector just a wrapper around the original
vector along with a start and end value?
Current source here;
On Mon, Oct 29, 2012 at 10:00 PM, Wolodja Wentland babi...@gmail.com wrote:
I find this behaviour quite unfortunate because I now have to explicitly test
for nil? and ensure consistent behaviour. This inconsistency violates the
principle of least-surprise and I am not sure if the current
On Thu, Nov 1, 2012 at 8:27 PM, Wolodja Wentland babi...@gmail.com wrote:
It seems to me as if we are currently figuring out which (boilerplate?)
functions are missing in reducers.clj and that we will have a nice and
well-integrated library at the end.
To be fair, it's in beta and it's open
On Tue, Aug 28, 2012 at 10:29 PM, Stuart Sierra
the.stuart.sie...@gmail.com wrote:
SSH in iTerm 2 from an OS X machine to a Linux server. $TERM is
xterm-256color at both ends. We use this for pair-programming, so X and
tramp are not helpful.
To support what Tim said, after killing an afternoon
On Thu, Aug 30, 2012 at 1:13 AM, Phil Hagelberg p...@hagelb.org wrote:
Alan Busby thebu...@thebusby.com writes:
To support what Tim said, after killing an afternoon I got iTerm2 from
OSX to play nice with an Emacs in gnu screen on a remote Linux host.
All keys and combos were working
On Fri, Dec 24, 2010 at 7:10 PM, Devrim Baris Acar devrimba...@gmail.comwrote:
I guess it would be a better guess if you could includethe cpu type/speed
for a rough reference...
It was on an Intel Xeon E5410 (2.33GHz), though like others have already
said there are a number of factors that
On Sat, Dec 25, 2010 at 11:28 AM, David Nolen dnolen.li...@gmail.comwrote:
On Fri, Dec 24, 2010 at 8:19 PM, David Nolen dnolen.li...@gmail.comwrote:
On OS X at least the following program shows identical performance to the
JVM using 64 bit integers, ~2000 nanoseconds on my machine. So Clojure
Hi All,
On Fri, Dec 24, 2010 at 4:32 PM, Meikel Brandmeyer m...@kotka.de wrote:
Most interesting is also the relation between the different versions on the
given machine. Just the numbers of one algorithm aren't really comparable, I
guess. (different machine, different load, different phase
On Tue, Apr 13, 2010 at 11:54 AM, Douglas Philips d...@mac.com wrote:
On 2010 Apr 12, at 10:48 PM, Glen Rubin wrote:
I am working with a collection of integer sequences ...(e.g.
coll:
((3 7 3 5 9 2 0 8 4 0 1 2323 4 11...) (243 4 664 478 3948...) (6 3 7 4
3335 2 4 5 7 6...)...)
I want to
I'd just like to second the request for selling a CD with Clojure 1.0 on it.
No support, no additional features; just a CD with the Clojure jar file or
something.
I'd even go a step further and have multiple versions that would be
identical except for the disc label,
Gold, $1000
Silver, $500
On Wed, Aug 26, 2009 at 5:43 AM, npowell nathan.pow...@gmail.com wrote:
I mean, I didn't think the article was terribly in depth, but a real,
evenhanded comparison would be enlightening.
Reducing it further, I'd be interested just to hear more about the contrast
of static typing versus
Make them watch the following video.
http://ocaml.janestreet.com/?q=node/61
Although the video isn't about Clojure, I think most of the points regarding
ML are true of Clojure as well.
On Thu, Jun 25, 2009 at 3:50 PM, Timothy Pratley
timothyprat...@gmail.comwrote:
1. How do you get
41 matches
Mail list logo
