Thanks everyone. There's a lot here--I'm still digesting it all. It's
clear that a lot may depend on the underlying collections, and that, among
other things, foldmap is worth considering for benchmarking in
performance-critical code, as is mapping a vector across multiple sequences
to combin
Just for the reference, this is the code with neanderthal, so we can get
the difference when running with primitives. No native code has been used
here, just pure Clojure.
(def sv1 (sv v1))
(def sv2 (sv v2))
(defn side-effect-4 ^double [^double a ^double b] 1.0)
(with-progress-reporting (quick
Francis,
The times you got also heavily depend on the actual side-effect function,
which in this case is much faster when called with one arg, instead of with
varargs, that fluokitten need here.
If we give fluokitten a function that does not create a sequence for
multiple arguments, it is much
That's why I still think that in this particular case, there is no new
sequence creation (by fluokitten). yes, it does call first/next but those
do not require (significant) new memory or copying. They reuse the memory
of the underlying vectors, if I understood that well. Or there is something
else
sequence taps into the underlying TransfomerIterator which applies a
transducer while iterating across multiple iterators at the same time.
sequence wraps that into a chunked sequence, but you could tap directly
into TransformerIterator/createMulti without using the outer sequence.
Nothing in
Just a general note of caution. range is a highly specialized (and
optimized) impl for both reducing and seq traversal. Also note that seqs on
vecs are about the next-most optimized seq (because they also don't
allocate or cache - rather they just pull from the vector nodes as the
backing store
BTW I noticed that sequence got hotter and eventually became the fastest,
but it took many more runs:
(time (dorun (sequence (map side-effect) col1 col2)))
"Elapsed time: 31.321698 msecs"
=> nil
(time (dorun (sequence (map side-effect) col1 col2)))
"Elapsed time: 15.492247 msecs"
=> nil
(time (do
Well, you pay a cost whenever you use seqs instead of reduce, so it/s
strange to think of doseq as "as fast as possible". If it/s argument is a
true collection, its IReduce is usually faster than its seq. If it is a
sequence already, its IReduce is usually just using seqs anyway.
Let's get some
Francis is correct, and I want to make a point about something he said. If
you are mapping over two collections at once, you will either be using Java
iterators, or you will be creating lazy sequences when you walk over the
input collections. Sure, how many lazy sequences are create here and there
Now I am on the REPL, and the solution is straightforward:
(foldmap op nil println [1 2 3] [4 5 6])
gives:
1 4
2 5
3 6
nil
The first function is a folding function. In this case we can use op, a
monoid operation. Since nil is also a monoid, everything will be folded to
nil. The second part is
Thanks very much Francis.
So it sounds as if, if my data would already be in a vector, list or
sequence (maybe a lazy sequence), doseq will process that structure as fast
as possible, but there are other structures that might have faster internal
reduce operations.
--
You received this mess
> Francis, thanks for separating out the different types of intermediate
> collections. I'm not entirely clear on type 1. It sounds like those are
> just collections that are already there before processing them. Or are you
> saying that Clojure has to convert them to a seq as such? Why wou
Thanks everyone--I'm very much appreciating the discussion, though I'm not
sure I follow every point.
Dragan, thank you very much for suggesting (and writing) foldmap. Very
nice. I certainly don't mind using a library, though I still think there
ought to be something like what I've describe
There are a few typos and defn is missing from fb in my last message, but I
hope it is still readable. Sorry, I am typing this on a mobile device while
watching the last episode of The Man in the High Castle :) Also, I am
talking about the code I wrote years ago from the top of my mind without
A couple of things:
1. How fold/foldmap and any other function works, depends on the actual
type. For example, if you look
at
https://github.com/uncomplicate/neanderthal/blob/master/src/clojure/uncomplicate/neanderthal/impl/fluokitten.clj#L396
you can see that there no intermediate allocations
Well, hold on, it is creating seqs for its input collection (allocation
item 1 in my list), but it is still better than map because it is not
making an intermediate lazy-seqs of output (item 4).
(I must amend my earlier message, (map f coll1 col2) only pays 1 and 4)
On Friday, September 23, 20
Yeah, I have to call you out on this one Dragan. I ran the following code:
(ns fold-test
(:require [uncomplicate.fluokitten.core :refer [foldmap]]))
(defn fa [& args]
(println "fa " args)
1)
(defn fb [& args]
(println "fb " args)
1)
(defn test-fold []
(foldmap fa nil fb [1 2 3] [4 5 6]))
(test
There are a few intermediate collections here:
1. The source coll may produce a seq object. How costly this is depends
on the type of coll and the quality of its iterator/ireduce/seq
implementations.
2. You may need to collect multiple source colls into a tuple-like thing
to pro
fluokitten does not have anything to do with haskell, or how haskell does
those things. It is completely tailored to clojure. Regarding marsi0's
task, now I see that, actually, foldmap is what should be used, and not
fold. so, it's something like (foldmap dummy-nil-accumulating-fn nil
side-effe
How is this done, I searched the source, but there was so much indirection
I can't find it. I'm familiar with how Haskell does rfold vs lfold, but one
of those does create allocations, they may be in the form of closures, but
they are allocations none the less. So does fold use iterators or somethi
fluokitten's fold is MUCH better than (map f a b) because it does NOT
create intermediate collections. just use (fold f a b) and it would fold
everything into one thing (in this case nil). If f is a function with side
effects, it will invoke them. No intermediate collection is created AND the
f
But that would print 1 to 32 items depending on a ton of things. No the
correct way to write code like that is
(doseq [s coll]
(println s))
If you need to iterate over multiple items then use (map f a b).
Also I would suggest benchmarking things, sure (map f coll) is slower than
a transducer,
I've run into this when I wanted to print each item in a seq. Since println
returns nil, I just did (first (keep println coll)). --Needed the first
here because keep returns a lazy seq.
On Friday, September 23, 2016 at 12:02:09 AM UTC-4, Mars0i wrote:
>
> This is almost the same as an issue I ra
How is fluokitten's fold any better than using seqs like (map f a b) would?
Both create intermediate collections.
On Fri, Sep 23, 2016 at 11:40 AM, Dragan Djuric wrote:
> If you do not insist on vanilla clojure, but can use a library, fold from
> fluokitten might enable you to do this. It is sim
If you do not insist on vanilla clojure, but can use a library, fold from
fluokitten might enable you to do this. It is similar to reduce, but
accepts multiple arguments. Give it a vararg folding function that prints
what you need and ignores the first parameter, and you'd get what you asked
fo
On Friday, September 23, 2016 at 11:11:07 AM UTC-5, Alan Thompson wrote:
>
> Huh. I was also unaware of the run! function.
>
> I suppose you could always write it like this:
>
> (def x (vec (range 3)))
> (def y (vec (reverse x)))
>
> (run!
> (fn [[x y]] (println x y))
>
> (map vector x y))
>
Huh. I was also unaware of the run! function.
I suppose you could always write it like this:
(def x (vec (range 3)))
(def y (vec (reverse x)))
(run!
(fn [[x y]] (println x y))
(map vector x y))
> lein run
0 2
1 1
2 0
although the plain old for loop with dotimes looks simpler:
(doti
On Friday, September 23, 2016 at 1:47:45 AM UTC-5, Francis Avila wrote:
>
> It's not crystal clear to me what you are after, either from this post or
> the one you link to. I think you want a map that does not produce
> intermediate collections and accepts multiple colls as input at a time?
Ye
28 matches
Mail list logo
