Re: What's up with IMeta?

[Michał Marczyk]

About realizing the head of lazy seqs in with-meta:

On 5 November 2017 at 01:57, Didier  wrote:

> > That said, metadata and its relationship to an object is immutable - an
>>> object with different metadata is a different object. One consequence of
>>> this is that applying metadata to a lazy sequence will realize the head of
>>> the sequence so that both objects can share the same sequence.
>>
>>
On 5 November 2017 at 05:34, James Reeves  wrote:

> On 5 November 2017 at 00:57, Didier  wrote:
>
>> And then I'm confused as to why that would cause lazy-seq to realize
>> their head? Can't two lazy-seq share the same head?
>>
>
> This I'm not too certain about. It may just be an implementation detail.
>

They can, and indeed the point of realizing the lazy seq in .withMeta /
with-meta is precisely so that the input and output of with-meta share the
same head.

For example, in this situation:

(def xs (create-some-lazy-seq …))
(def ys (with-meta xs {:foo 1}))

the expectation is that the lazy seqs xs and ys will wrap a single
underlying sequence.

This is actually quite important – consider a lazy sequence that fully
consumes some resource at each step or that causes some other side effect
at each step, or perhaps simply one that is computationally expensive to
realize; one would expect the above to work just fine in such a case
regardless of how xs and ys are consumed, with any necessary computation,
resource consumption, side effects etc. occurring once per element, and all
the elements shared across xs and ys.

In other words, if a consumer comes along and asks for the first element of
xs, and then subsequently another consumer (or indeed the same one) asks
for the first element of ys, the latter request should be satisfied without
any further resource consumption or computation (beyond looking up a
pointer). And similarly if ys is realized first and the second request is
for an element of xs.

Now, if xs' head has not been realized, (with-meta xs {:foo 1}) can
conceivably operate in one of two ways:

1. it can copy over xs' thunk (the nullary function embedded in the lazy
seq object xs that is called – and expected to return a sequence or nil –
when xs needs to be realized) into a new lazy seq object ys;

2. it can realize the head of xs and call .withMeta on the result.

The first approach, however, breaks the promise that realizing the head of
xs will have the effect of realizing the head of ys, as neither xs nor ys
would subsequently have any way of knowing whether the other's head has
been realized when they finally need to realize their own heads, and so the
second one to be realized would be forced to recompute the sequence.

Thus the implementation takes the second approach.

Cheers,
Michał

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: varying realization of a lazy-seq of strings?

[Michał Marczyk]

That's right. This happens is because collection classes' toString
implementations currently delegate to RT.printString, which in turn is
affected by the value of *print-readably*.

Filed https://dev.clojure.org/jira/browse/CLJ-2248 with a fix.

Cheers,
Michał


On 18 September 2017 at 03:08, Justin Smith  wrote:

> my simplified reproduction of the issue:
>
> +user=> (let [mk-str (fn [] (lazy-seq [(str ["ZiZi"])]))
>   a (mk-str)
>   b (mk-str)]
>   (print-str a)
>   (pr-str b)
>   [a b])
> [("[ZiZi]") ("[\"ZiZi\"]")]
>
> isn't *print-readably* the difference between pr-str and print-str?
>
> On Sun, Sep 17, 2017 at 5:59 PM Didier  wrote:
>
>> Lazy sequences cache their values after the first time they are
>> evaluated. Since print alters the output of str by binding *print-readably*
>> to false, and also forces the sequence to realize itself, the values in
>> your sequence are now cached the the result of str without
>> *print-readably*. In subsequent calls to the sequence, you get the cache
>> results, so even if you change the setting of *print-readably* it doesn't
>> do anything.
>>
>> Either make sure that when you first access elements of the sequence, the
>> bindings are the way you want str to be configured, or have the bindings
>> set inside the lazy-seq.
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clojure@googlegroups.com
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to clojure+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Help ship Clojure 1.9!

[Michał Marczyk]

I've run into a behaviour change that was actually already present in
alpha20 – with the CLJ-99 patch in place, {min,max}-key now return the
first argument with the minimum/maximum key, whereas previously they
returned the last such argument.

The new behaviour seems like the more natural one, but this is a breaking
change, so I filed https://dev.clojure.org/jira/browse/CLJ-2247 to track
this (with a patch that takes the "default" approach of restoring
established behaviour).

Cheers,
Michał


On 3 October 2017 at 21:11, Beau Fabry  wrote:

> We've been using 1.9 in a small app for a while with no issues. After
> upgrading schema to the latest version (with the PR above) I've also
> successfully run our larger codebase with 1.9.
>
> On Tuesday, October 3, 2017 at 4:41:14 AM UTC-7, stuart@gmail.com
> wrote:
>>
>> Hi Mark,
>>
>> I think this approach totally makes sense, and the alpha naming exists to
>> inform this kind of decision-making.
>>
>> For libraries where the use of spec does not have to be user-facing, I am
>> putting specs in separate (Clojure) namespaces, and loading them in such a
>> way that they can coexist with non (or maybe different) spec environments.
>> But that is extra work for sure.
>>
>> Stu
>>
>> On Mon, Oct 2, 2017 at 3:35 PM, Mark Engelberg 
>> wrote:
>>
>>> On Mon, Oct 2, 2017 at 7:55 AM, Stuart Halloway 
>>> wrote:
>>>
 Hi David,

 Spec will be in alpha for a while. That is part of the point of it
 being a separate library. Can you say more about what problems this is
 causing?

 Stu


>>> As a library maintainer, I am forced to upgrade and release my library
>>> any time something I depend upon makes a breaking change.  I don't get paid
>>> for maintaining open source libraries, it's something I do in my spare
>>> time, so I prefer to do it on my own schedule.  When an underlying library
>>> makes a breaking change, I get dozens of urgent requests from people who
>>> need me to cut a new release ASAP, and by Murphy's Law, that often happens
>>> when I have very little time to do it.  It's a nuisance.
>>>
>>> Clojure is pretty good about not making breaking changes, but it happens
>>> from time to time.  Clojurescript is less good about not making breaking
>>> changes, and therefore, maintaining Clojurescript libraries is more of a
>>> headache.  On the plus side, Clojurescript users seem to care very little
>>> about backwards compatibility (most keep up with the latest version), so
>>> sometimes it is easier to make a change to keep up with a change in
>>> Clojurescript than one in Clojure, where I am expected to not only support
>>> the latest breaking change, but also the last several releases.
>>>
>>> Anything that is labeled as "alpha" is waving a big red flag that there
>>> could be breaking changes at any time with little warning.  For my
>>> libraries which depend on spec, there's no way I'm going to bring them out
>>> of alpha status until spec comes out of alpha status.  If I make an
>>> official release of something that depends on spec, then I'm going to be on
>>> the hook to rapidly cut a new release every time spec changes, which could
>>> be at any time.  I don't want that hassle.  I don't want to make a promise
>>> to the community to maintain a stable product if the thing I depend upon
>>> has not made a similar promise.  When spec reaches a point where the API
>>> will not be changing, or rather, when we know that new changes will only be
>>> additive, I can begin to trust that it won't be a huge maintenance headache
>>> to release something based on spec.
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To post to this group, send email to clo...@googlegroups.com
>>> Note that posts from new members are moderated - please be patient with
>>> your first post.
>>> To unsubscribe from this group, send email to
>>> clojure+u...@googlegroups.com
>>> For more options, visit this group at
>>> http://groups.google.com/group/clojure?hl=en
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to clojure+u...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop 

Re: Help me understand what part of this code is slow, and how to make it faster?

[Michał Marczyk]

PS. Results for the original input on my box. Going by the the timings
posted above, yours is rather beefier, so this is probably faster than the
current F# version.

(c/quick-bench (nth-shell 2000 (point. 0 0)))
Evaluation count : 6 in 6 samples of 1 calls.
 Execution time mean : 2.956519 sec
Execution time std-deviation : 22.423970 ms
   Execution time lower quantile : 2.930938 sec ( 2.5%)
   Execution time upper quantile : 2.978283 sec (97.5%)
   Overhead used : 21.423788 ns


On 22 November 2016 at 05:21, Michał Marczyk <michal.marc...@gmail.com>
wrote:

> Some further optimizations for a factor of ~2.3 speed-up over nth5 as copy
> & pasted from upthread (6.713742 ms → 2.897630 ms) in
>
>   (c/quick-bench (nth-shell 100 (point. 0 0)))
>
> (1) java.util.HashSet has a ctor that takes initial capacity of the set as
> an int. Passing in (* 4 (.size s1)) when constructing s0 – (HashSet. (* 4
> (.size s1)) – gives me a fair speed-up on its own.
>
> (2) Also, (java.util.HashSet. [p]) is a reflective call – replacing that
> with (doto (java.util.HashSet.) (.add p)) seems to result in a tiny, but
> measurable speed-up as well (to 5.245931 ms).
>
> (3) Using an iterator results in a good speed-up, particularly in a
> size-based loop (counting down from (.size current-set) rather than calling
> (.hasNext current-iterator)).
>
> (4) Finally, inlining all the visiting and switching to direct ctor calls
> also helped.
>
> Final code:
>
> (deftype point [^long i ^long j]
>   Object
>   (equals [this that]
> (and (= (.i this) (.i ^point that))
>  (= (.j this) (.j ^point that
>   (hashCode [this]
> (+ (.i this) (* 4000 (.j this)
>
> (defn nth-shell [^long n p]
>   (loop [n  n
>  s1 (doto (HashSet.) (.add p))
>  s2 (HashSet.)]
> (if (zero? n)
>   s1
>   (let [s0 (HashSet. (* 4 (.size s1)))
> i1 (.iterator s1)]
> (dotimes [_ (.size s1)]
>   (let [^point p (.next i1)
> i ^long (.i p)
> j ^long (.j p)
> p1 (point. (dec i) j)
> p2 (point. (inc i) j)
> p3 (point. i (dec j))
> p4 (point. i (inc j))]
> (if-not (or (.contains s1 p1) (.contains s2 p1))
>   (.add s0 p1))
> (if-not (or (.contains s1 p2) (.contains s2 p2))
>   (.add s0 p2))
> (if-not (or (.contains s1 p3) (.contains s2 p3))
>   (.add s0 p3))
> (if-not (or (.contains s1 p4) (.contains s2 p4))
>   (.add s0 p4
> (recur (dec n) s0 s1)
>
> Also, to check that this still outputs the same neighbours the original
> impl (nth*) does:
>
> (defn persistent-shell [shell]
>   (persistent!
> (reduce (fn [out ^point p]
>   (conj! out [(.-i p) (.-j p)]))
>   (transient #{})
>   shell)))
>
> (= (sort (persistent-shell (nth-shell 500 (point. 0 0
>(sort (nth* 500 [0 0])))
>
> Cheers,
> Michał
>
>
> On 22 November 2016 at 02:03, Didier <didi...@gmail.com> wrote:
>
>> I tried it with the safe equals, and it is slightly slower, but still
>> faster then all others at 4.5ms. The non safe equals gives me 4s. Though
>> this is now within my error margin. If ire-run quick-bench, I sometime get
>> a mean equal for each, so I don't think the instance check adds that much
>> overhead if any at all.
>>
>> @miner: Doesn't using the flag (set! *unchecked-math* :warn-on-boxed)
>> gives me unchecked math automatically? I was under the impression that +,
>> -, /, * etc. would all now perform in an equal way to unchecked-add, etc.
>> If not, what is the difference?
>>
>> @Andy: I tried with the "1.9.0-alpha14" version, and Records were still
>> just as slow as with "1.8.0". Maybe I'm using them wrong.
>>
>> On Tuesday, 15 November 2016 19:39:43 UTC-8, Didier wrote:
>>
>>> Hey all,
>>>
>>> I came upon a benchmark of F#, Rust and OCaml, where F# performs much
>>> faster then the other two. I decided for fun to try and port it to Clojure
>>> to see how Clojure does. Benchmark link: https://github.com/c-cube/hash
>>> set_benchs
>>>
>>> This is my code for it: https://gist.github.com/didibu
>>> s/1fd4c00b69d927745fbce3dcd7ca461a
>>>
>>> (ns hash-set-bench
>>>   "A Benchmark I modified to Clojure from:
>>>https://github.com/c-cube/hashset_benchs;)
>>>
>>> (defn iterNeighbors [f [i j]]
>>>   (f [(dec i) j])
>>>   (f [(inc i) j])
>>>   (f [

Re: Help me understand what part of this code is slow, and how to make it faster?

[Michał Marczyk]

Some further optimizations for a factor of ~2.3 speed-up over nth5 as copy
& pasted from upthread (6.713742 ms → 2.897630 ms) in

  (c/quick-bench (nth-shell 100 (point. 0 0)))

(1) java.util.HashSet has a ctor that takes initial capacity of the set as
an int. Passing in (* 4 (.size s1)) when constructing s0 – (HashSet. (* 4
(.size s1)) – gives me a fair speed-up on its own.

(2) Also, (java.util.HashSet. [p]) is a reflective call – replacing that
with (doto (java.util.HashSet.) (.add p)) seems to result in a tiny, but
measurable speed-up as well (to 5.245931 ms).

(3) Using an iterator results in a good speed-up, particularly in a
size-based loop (counting down from (.size current-set) rather than calling
(.hasNext current-iterator)).

(4) Finally, inlining all the visiting and switching to direct ctor calls
also helped.

Final code:

(deftype point [^long i ^long j]
  Object
  (equals [this that]
(and (= (.i this) (.i ^point that))
 (= (.j this) (.j ^point that
  (hashCode [this]
(+ (.i this) (* 4000 (.j this)

(defn nth-shell [^long n p]
  (loop [n  n
 s1 (doto (HashSet.) (.add p))
 s2 (HashSet.)]
(if (zero? n)
  s1
  (let [s0 (HashSet. (* 4 (.size s1)))
i1 (.iterator s1)]
(dotimes [_ (.size s1)]
  (let [^point p (.next i1)
i ^long (.i p)
j ^long (.j p)
p1 (point. (dec i) j)
p2 (point. (inc i) j)
p3 (point. i (dec j))
p4 (point. i (inc j))]
(if-not (or (.contains s1 p1) (.contains s2 p1))
  (.add s0 p1))
(if-not (or (.contains s1 p2) (.contains s2 p2))
  (.add s0 p2))
(if-not (or (.contains s1 p3) (.contains s2 p3))
  (.add s0 p3))
(if-not (or (.contains s1 p4) (.contains s2 p4))
  (.add s0 p4
(recur (dec n) s0 s1)

Also, to check that this still outputs the same neighbours the original
impl (nth*) does:

(defn persistent-shell [shell]
  (persistent!
(reduce (fn [out ^point p]
  (conj! out [(.-i p) (.-j p)]))
  (transient #{})
  shell)))

(= (sort (persistent-shell (nth-shell 500 (point. 0 0
   (sort (nth* 500 [0 0])))

Cheers,
Michał


On 22 November 2016 at 02:03, Didier  wrote:

> I tried it with the safe equals, and it is slightly slower, but still
> faster then all others at 4.5ms. The non safe equals gives me 4s. Though
> this is now within my error margin. If ire-run quick-bench, I sometime get
> a mean equal for each, so I don't think the instance check adds that much
> overhead if any at all.
>
> @miner: Doesn't using the flag (set! *unchecked-math* :warn-on-boxed)
> gives me unchecked math automatically? I was under the impression that +,
> -, /, * etc. would all now perform in an equal way to unchecked-add, etc.
> If not, what is the difference?
>
> @Andy: I tried with the "1.9.0-alpha14" version, and Records were still
> just as slow as with "1.8.0". Maybe I'm using them wrong.
>
> On Tuesday, 15 November 2016 19:39:43 UTC-8, Didier wrote:
>
>> Hey all,
>>
>> I came upon a benchmark of F#, Rust and OCaml, where F# performs much
>> faster then the other two. I decided for fun to try and port it to Clojure
>> to see how Clojure does. Benchmark link: https://github.com/c-cube/hash
>> set_benchs
>>
>> This is my code for it: https://gist.github.com/didibu
>> s/1fd4c00b69d927745fbce3dcd7ca461a
>>
>> (ns hash-set-bench
>>   "A Benchmark I modified to Clojure from:
>>https://github.com/c-cube/hashset_benchs;)
>>
>> (defn iterNeighbors [f [i j]]
>>   (f [(dec i) j])
>>   (f [(inc i) j])
>>   (f [i (dec j)])
>>   (f [i (inc j)]))
>>
>> (defn nth* [n p]
>>   (loop [n n s1 #{p} s2 #{}]
>> (if (= n 0)
>>   s1
>>   (let [s0 (atom #{})]
>> (letfn [(add [p]
>>  (when (not (or (contains? s1 p) (contains? s2 p)))
>>(reset! s0 (conj @s0 p]
>>(doseq [p s1] (iterNeighbors add p))
>>(recur (dec n) @s0 s1))
>>
>> #_(printf "result is %d" (count (time (nth* 2000 [0 0]
>>
>> And here's the F# code: https://github.com/c-cube/hash
>> set_benchs/blob/master/neighbors2.fsx
>>
>> Currently, this takes about 30s in Clojure, while it only takes around 3s
>> for OCaml, Rust and F#.
>>
>> From what I see, the differences between my code and theirs are:
>>
>>- Lack of a Point struct, I'm just using a vector.
>>- They use a mutable set, I don't.
>>- They overrode Hashing for their point struct, as well as equality.
>>I rely on Clojure's default hashing, and vector equality.
>>
>> I'm not sure if any of these things should really impact performance that
>> much though. And what I could do in Clojure if I wanted to improve it.
>>
>>
>> Any Help?
>>
>>
>> Thanks.
>>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post 

[ANN] maxiphobe 0.0.1 – Persistent Meldable Priority Queues

[Michał Marczyk]

Hi,

I am pleased to announce the initial release of maxiphobe, a meldable
priority queue library based on Okasaki's maxiphobic heaps (see
Okasaki, "Alternatives to Two Classic Data Structures"). Maxiphobic
heaps are a strikingly simple purely functional approach to
implementing priority queues that offers very reasonable performance
given the available operations.

  https://github.com/michalmarczyk/maxiphobe

  https://clojars.org/maxiphobe

Maxiphobe priority queues are, in effect, persistent lists that
maintain a non-decreasing order of elements, as determined either by
Clojure's default comparator or a custom comparator passed in by the
user.

Additionally, instances using the same comparator can be "melded" into
a single priority queue containing all the elements of the inputs.

  (require '[maxiphobe.core :as pq])

  (pq/pq [0 -1 3 4 2])
  ;= (-1 0 2 3 4)

  (pq/pqueue 0 -1 3 4 2)
  ;= (-1 0 2 3 4)

  (pq/pq-by > [0 -1 3 4 2])
  ;= (4 3 2 0 -1)

  (pq/pqueue-by > 0 -1 3 4 2)
  ;= (4 3 2 0 -1)

  (peek (pq/pqueue-by > 0 -1 3 4 2))   ; or first
  ;= 4

  (pop (pq/pqueue-by > 0 -1 3 4 2)); or next
  ;= (3 2 0 -1)

  (pq/meld (pq/pqueue -1 3 2 8) (pq/pqueue 0 -3 2 4))
  ;= (-3 -1 0 2 2 3 4 8)

To include maxiphobe in your project:

  [maxiphobe "0.0.1"]

  
maxiphobe
maxiphobe
0.0.1
  

  compile "maxiphobe:maxiphobe:0.0.1"

See below for some benchmark results.

Cheers,
Michał


Benchmarks
==

1. Compare basic operations to sorted-set-as-PQ and to regular (FIFO)
   peristent queues to get a high-level picture of maxiphobe
   performance. FIFO queue operations are included to provide a point
   of reference – they are, of course, completely different
   semantically.

(let [pq (pq/pq (range 1000))
  s  (apply sorted-set (range 1000))
  q  (into clojure.lang.PersistentQueue/EMPTY (range 1000))]
  (assert (= (seq s) pq q))
  (println "peek")
  (c/quick-bench (peek pq))
  (c/quick-bench (first s))
  (c/quick-bench (peek q))
  (println "pop")
  (c/quick-bench (pop pq))
  (c/quick-bench (disj s (first s)))
  (c/quick-bench (pop q))
  (let [pq (nth (iterate pop pq) 500)
s  (apply sorted-set pq)
q  (nth (iterate pop q) 500)]
(assert (= (seq s) pq q))
(println "pop2")
(c/quick-bench (pop pq))
(c/quick-bench (disj s (first s)))
(c/quick-bench (pop q

peek

;;; maxiphobe PQ:
;;; (peek pq)

Evaluation count : 32624694 in 6 samples of 5437449 calls.
 Execution time mean : -2.422742 ns
Execution time std-deviation : 0.630313 ns
   Execution time lower quantile : -2.934682 ns ( 2.5%)
   Execution time upper quantile : -1.681600 ns (97.5%)
   Overhead used : 21.406378 ns

;;; sorted set:
;;; (first s)

Evaluation count : 1944576 in 6 samples of 324096 calls.
 Execution time mean : 3.262937 µs
Execution time std-deviation : 440.634979 ns
   Execution time lower quantile : 2.728010 µs ( 2.5%)
   Execution time upper quantile : 3.753706 µs (97.5%)
   Overhead used : 21.406378 ns

;;; queue
;;; (peek q)

Evaluation count : 22712976 in 6 samples of 3785496 calls.
 Execution time mean : 7.395679 ns
Execution time std-deviation : 0.615558 ns
   Execution time lower quantile : 6.766551 ns ( 2.5%)
   Execution time upper quantile : 8.050620 ns (97.5%)
   Overhead used : 21.406378 ns

pop

;;; maxiphobe PQ:
;;; (pop pq)

Evaluation count : 1107840 in 6 samples of 184640 calls.
 Execution time mean : 2.143321 µs
Execution time std-deviation : 1.064519 µs
   Execution time lower quantile : 362.028152 ns ( 2.5%)
   Execution time upper quantile : 3.175097 µs (97.5%)
   Overhead used : 21.406378 ns

;;; sorted set:
;;; (disj s (first s))

Evaluation count : 508608 in 6 samples of 84768 calls.
 Execution time mean : 10.498305 µs
Execution time std-deviation : 2.658746 µs
   Execution time lower quantile : 6.260925 µs ( 2.5%)
   Execution time upper quantile : 12.859465 µs (97.5%)
   Overhead used : 21.406378 ns

;;; queue:
;;; (pop q)

Evaluation count : 8807040 in 6 samples of 1467840 calls.
 Execution time mean : 730.736239 ns
Execution time std-deviation : 122.924195 ns
   Execution time lower quantile : 561.778923 ns ( 2.5%)
   Execution time upper quantile : 839.133234 ns (97.5%)
   Overhead used : 21.406378 ns

pop2

;;; maxiphobe PQ:
;;; (pop pq)

Evaluation count : 1816404 in 6 samples of 302734 calls.
 Execution time mean : 2.874078 µs
Execution time std-deviation : 310.794245 ns
   Execution time lower quantile : 2.614476 µs ( 2.5%)
   Execution time upper quantile : 3.233927 µs (97.5%)
   Overhead used : 21.406378 ns

;;; sorted set:
;;; (disj s (first s))

Evaluation count : 554400 in 6 samples of 92400 calls.
 Execution time mean : 11.149426 µs
Execution time std-deviation : 1.675888 µs
   Execution time lower 

[ANN] psq.clj 0.0.2 – Persistent Priority Search Queues

[Michał Marczyk]

Hi,

I am pleased to announce the 0.0.2 release of psq.clj, a Clojure
priority search queue library based on Ralf Hinze's priority search
pennants (see Ralf Hinze, "A Simple Implementation Technique for
Priority Search Queues"):

  https://github.com/michalmarczyk/psq.clj

  https://clojars.org/psq.clj

Priority search queues simultaneously behave like sorted maps with
respect to their keys and like priority queues with respect to their
entries, with the priority queue ordering based on the values, or
priorities, attached to the keys. Additionally they support several
operations that blend the sorted map and priority queue aspects
together, such as "peek in key range" (retrieve a minimum-priority
entry – NB. ties on priorities are supported – within the given key
range) and optimized subseq-like traversals of key ranges that only
return those entries whose priorities fall below a certain threshold
(this is much more efficient than subseq + filter on a regular sorted
map).

psq.clj priority search queues support the full data.avl abstract data
type (with the exception of the transient API which I am not sure
would bring a significant return on the additional memory cost in this
context). The full public API is as follows:

  ;;; there is a single public namespace:
  (require '[psq.clj :as psq])


  ;;; 1. factory functions

  ;; the psq.clj counterpart to clojure.core/sorted-map:
  (psq/psqueue key priority …)
  ;= {key priority …}

  ;; a version of the above that takes keys+priorities in a seqable:
  (psq/psqueue* [key priority …])

  ;; a factory that accepts a map or seqable of map entries:
  (psq/psq {key priority …})
  (psq/psq [[key priority] …])

  ;; versions of the above that take *two* custom comparators: the
  ;; first one determines the PSQ's key order, the second one is used
  ;; for priorities:
  (psq/psqueue-by > > key priority …) ; reverse numeric order on keys
  ; and priorities

  psq/psqueue-by*, psq/psq-by ; like psq/psqueue*, psq/psq, but with
  ; custom comparators


  ;;; 2. regular sorted map API; NB. (r)(sub)seq use key order

  (seq (psq/psqueue 0 10 1 9))
  ;= ([0 10] [1 9])

  ;; also supported: assoc, dissoc, conj, rseq, subseq, rsubseq


  ;;; 3. nearest neighbour lookups
  (psq/nearest (psq/psq {0 1 4 5 9 10}) > 3)
  ;= [4 5]


  ;;; 4. nth, rank in key order

  (nth (psq/psqueue 0 3 6 -3) 0)
  ;= [0 3]

  (psq/rank (psq/psqueue 0 3 6 -3) 6) ; returns long, -1 for not found
  ;= 1


  ;;; 5. priority queue API based on values/priorities

  (peek (psq/psqueue 0 3 1 -3))
  ;= [1 -3]

  (pop (psq/psqueue 0 3 1 -3))
  ;= {0 3}

  ;; seq over the PSQ in order of non-decreasing priorities
  (psq/priority-seq (psq/psqueue 0 3 1 -3))
  ;= ([1 -3] [0 3])


  ;;; 6. splits, subranges – with structural sharing in the common
  ;;;parts, but no holding on to keys outside the stated range for
  ;;;GC purposes

  ;; return a vector of
  ;;
  ;;   1. a fully independent PSQ comprising the entries of the
  ;;  input PSQ to the left of the given key,
  ;;
  ;;   2. the entry at the given key, or nil if not present,
  ;;
  ;;   3. a fully independent PSQ comprising the entries of the
  ;;  input PSQ to the right of the given key:
  (psq/split (psq/psqueue* (range 10)) 4)
  ;= [{0 1 2 3} [4 5] {6 7 8 9}]

  ;; like subseq, but returns an independent PSQ
  (psq/subrange (psq/psqueue* (range 10)) >= 4 < 8)
  ;= {4 5 6 7}


  ;;; 7. priority-bounded traversals:
  (psq/seq<= a-psq priority-upper-bound)

  ;; also supported: rseq<=, subseq<=, rsubseq<= (the latter two
  ;; with subseq/rsubseq-like arguments to specify key bounds) and
  ;; < variants of all of the above (seq< etc.)

All of the above operations can be performed in O(log n) time, with
the exception of peek, which takes constant time, and seq-like
operations, which are sensitive to the number of entries actually
produced. Priority-bounded traversals take O(r(log n - log r) + r)
time, where r is the number of entries actually returned.

In practice, psq.clj priority search queues are slower than Clojure's
built-in sorted maps for pure sorted-map operations, typically by a
factor of 2 to 4. In return, the PSQ operations are very performant;
subseq<= can be over three orders of magnitude faster than the
equivalent combination of subseq and filter applied to a regular
sorted map, subrange takes tens of microseconds when the size of the
input is on the order of hundreds of thousands, peek takes less than
10 ns etc. See end of this message for some benchmark results.

Mark Engelberg's excellent data.priority-map is another natural point
of comparison. In short, whereas data.priority-map is hash-map-like,
with extremely fast lookups and no particular ordering of keys,
psq.clj is sorted-map-like, with a much richer abstract data type at
the cost of noticeably slower lookups. assoc is often faster with
psq.clj, although there are edge cases where data.priority-map is

Re: clojure.spec - relationship between :args

[Michał Marczyk]

Only one quick bit of feedback – it's preferable to use test.check
facilities instead of (rand) because they're "repeatable": the seed emitted
in test.check output can be used to rerun the test with the same test.check
PRNG state (see the docstring on clojure.test.check/quick-check,
specifically the :seed option), but if you call out to other PRNGs, you
defeat that mechanism and you'll still get different results on different
runs. So here you might want to say (gen/elements [:head :tails]).

Cheers,
Michał


On 5 November 2016 at 11:59, dimitris  wrote:

> Hi Alex,
>
> I think I've figured it out. Here is the complete solution I've come up
> with in case you feel like providing feedback:
>
> (defn- extract-sensible-k-gen
>   "Gen override for `::extract-args`."  []
>   (gen/bind(spc/gen ::persistent-coll)
> #(let [r (rand)
>heads? (>= r 0.5)] ;; flip a coin   (gen/tuple 
> (gen/elements   (cond (empty? %) (repeat 2 :NOT-FOUND) ;; 
> nothing can be possibly found in an empty coll (map? %) (if heads?
> (keys %) ;; keys that will be found   
>  (repeat 2 :NOT-FOUND)) ;; keys that will not be found (set? 
> %) (if heads?
> % ;; elements(repeat 2 
> :NOT-FOUND))
>  (sequential? %) (let [c (count %)]
>(if heads? ;; index overrides  
>(if (> r 0.75)
>(range c) ;;valid indices  
>  (concat (range -1 (dec (- c)) -1) ;; invalid (negative) 
> indices   (range c (+ c 10
>  (if (> r 0.25) ;; predicate overrides
>(repeat 2 (constantly true))
>(repeat 2 (constantly false)
>  ))
>  (gen/return %)))
> ))
>
> ;==(spc/def
>  ::persistent-coll  (spc/or:map (spc/map-of any? any?)
> :vector (spc/coll-of any? :kind vector?)
> :set (spc/coll-of any? :kind set?)
> :list (spc/coll-of any? :kind list?)))
>
> (spc/def ::predicate  (spc/fspec :args (spc/cat :x any?)
>  :ret boolean?))
>
> (spc/def ::extract-args  (spc/cat :k (spc/or:predicate 
> ::predicate:key-or-index any?)
>:coll ::persistent-coll))
>
> (spc/fdef enc/extract
>
>   :args ::extract-args  :ret (spc/tuple any? coll?)
>
>   :fn (spc/or:some-found #(let [[e c] (:ret %)
>[coll-type arg-coll] (-> % :args :coll)]
>   (and (some? e)
>(> (count arg-coll)
>   (count c))
>#_(do (println "SOME-FOUND - cret=" c "argc=" 
> arg-coll \newline (-> % :args :k second))   true) 
>   ))
> :nil-found #(let [[e c] (:ret %)
>   [coll-type arg-coll] (-> % :args :coll)]
>  (and (nil? e)
>   (> (count arg-coll)
>  (count c))
>   #_(do (println "NIL-FOUND - cret=" c "argc=" 
> arg-coll \newline (-> % :args :k second))  true)  
> ))
> :not-found #(let [[e c] (:ret %)
>   [coll-type arg-coll] (-> % :args :coll)]
>  (and (nil? e)
>   (= arg-coll c)
>   #_(do (println "NOT-FOUND - cret=" c "argc=" 
> arg-coll \newline (-> % :args :k second))  true)  
> )))
>   )
>
>  and i call it like so:
>
> (-> (test/check `treajure.encore/extract {:gen {::extract-args 
> extract-sensible-k-gen}})
> test/summarize-results
>
> I must say, i was surprised to see that my humble 8G-ram laptop can barely
> deal with the default number of generative tests (1000), but at least it
> works :).
>
> Many thanks again, for redirecting me to Stu's video - it all made much
> more sense after digesting that.
>
> Regards,
>
> Dimitris
>
> On 04/11/16 00:14, dimitris wrote:
>
> HI Alex,
>
> Many thanks for your response, it was very helpful. I see your point about
> customizing the generator, and in fact the video in the link does something
> sort of similar to what I am trying to. So yeah I'll figure it out tomorrow
> :). Thanks again!
>
> Dimitris
>
> On 03/11/16 18:53, Alex Miller wrote:
>
>
>
> On Thursday, November 3, 2016 at 1:12:39 PM UTC-5, Jim foo.bar wrote:
>>
>> Hi everyone,
>>
>> I'm starting to get familiar with clojure.spec, and in my very first spec
>> I needed to specify relationship between the args themselves (similar to
>> how :fn specs allow for 

[ANN] data.avl 0.0.17 – tree balance bugfix, more generative tests

[Michał Marczyk]

Hi,

I am pleased to announce the 0.0.17 release of data.avl, a Clojure
Contrib library providing highly performant drop-in replacements for
Clojure(Script)'s built-in sorted maps and sets that support O(log n)
nth, rank-of, first-class submaps/subsets (like subseq, but preserving
collection type; fully independent from the original for GC purposes)
and splits by key and index.

This release addresses an issue whereby the balancing invariant would
not be obeyed in certain cases, leading to degraded performance. The
problem was uncovered by newly expanded generative tests for the
balancing invariant; these should prevent any regressions in the
future.

  [org.clojure/data.avl "0.0.17"]

  
org.clojure
data.avl
0.0.17
  

  org.clojure:data.avl:0.0.17

Cheers,
Michał

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] data.avl 0.0.16 – bugfix to one-sided subrange

[Michał Marczyk]

Hi,

data.avl 0.0.16 is now available with a bugfix to the "one-sided
subrange past end of collection" case of subrange partly fixed in
0.0.15.

  [org.clojure/data.avl "0.0.16"]

  
org.clojure
data.avl
0.0.16
  

  org.clojure:data.avl:0.0.16

Cheers,
Michał

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] data.avl 0.0.15 – BUGFIX to subrange

[Michał Marczyk]

Hi,

I am pleased to announce the 0.0.15 release of data.avl, a Clojure
Contrib library providing highly performant drop-in replacements for
Clojure(Script)'s built-in sorted maps and sets that support O(log n)
nth, rank-of, first-class submaps/subsets (like subseq, but preserving
collection type; fully independent from the original for GC purposes)
and splits by key and index.

  [org.clojure/data.avl "0.0.15"]

  
org.clojure
data.avl
0.0.15
  

  org.clojure:data.avl:0.0.15

This is a bugfix release addressing a few issues in subrange:

 1. subrange could sometimes misjudge whether its output collection
should be empty and produce corrupt return values.

 2. subrange incorrectly handled the single-test case where the
specified limit fell past the extreme key actually present in the
collection (e.g. (avl/subrange (avl/sorted-set 0 1 2) >= 3)).

There are now multiple generative tests guarding against regressions
on this front.

Additionally, Darrick Wiebe, who reported DAVL-8 fixed in the previous
release, provided generative tests that should guard against
regressions in split-key.

As a final tweak, the Clojure implementation of
clojure.core.protocols/CollReduce is now replaced by an implementation
of clojure.lang.IReduce. Thanks to Alex Miller for the suggestion!

Cheers,
Michał

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] data.avl 0.0.14 – BUGFIX to splits, faster map/set reductions

[Michał Marczyk]

All credit to Francis for the reduce patch! The perf win is really great.

Re: IReduce – thanks, indeed, I forgot about the special-casing in reduce.
I guess I'll tweak that for the next release.

Incidentally, I notice I left out the final result in the after-patch 1.5.1
benchmark run in the previous email – repeated below in full.

Cheers,
Michał


After patch, Clojure 1.5.1:

user> (let [m1 (apply sorted-map (range 1))
m2 (apply avl/sorted-map (range 1))]
(assert (= (reduce (fn [out kv] (+ out (key kv) (val kv)))
 0
 m1)
   (reduce (fn [out kv] (+ out (key kv) (val kv)))
 0
 m2)))
(c/quick-bench
  (reduce (fn [out kv] (+ out (key kv) (val kv)))
0
m1))
(c/quick-bench
  (reduce (fn [out kv] (+ out (key kv) (val kv)))
0
m2)))
Evaluation count : 990 in 6 samples of 165 calls.
 Execution time mean : 599.205620 µs
Execution time std-deviation : 30.389791 µs
   Execution time lower quantile : 579.132582 µs ( 2.5%)
   Execution time upper quantile : 650.985417 µs (97.5%)
   Overhead used : 1.320107 ns

Found 1 outliers in 6 samples (16.6667 %)
low-severe   1 (16.6667 %)
 Variance from outliers : 13.8889 % Variance is moderately inflated by outliers
Evaluation count : 3198 in 6 samples of 533 calls.
 Execution time mean : 191.906227 µs
Execution time std-deviation : 3.447964 µs
   Execution time lower quantile : 188.550574 µs ( 2.5%)
   Execution time upper quantile : 197.006475 µs (97.5%)
   Overhead used : 1.320107 ns
nil


On 23 August 2016 at 02:59, Alex Miller <a...@puredanger.com> wrote:

> Nice work! Esp on the reduce stuff - great to see that. Any reason you
> didn't go the IReduce path in Clojure too instead of CollReduce? Generally,
> I'd say that's preferred when you control the data structure.
>
>
> On Monday, August 22, 2016 at 7:43:51 PM UTC-5, Michał Marczyk wrote:
>>
>> Hi,
>>
>> I am pleased to announce the 0.0.14 release of data.avl, a Clojure
>> Contrib library providing highly performant drop-in replacements for
>> Clojure(Script)'s built-in sorted maps and sets that support O(log n)
>> nth, rank-of, first-class submaps/subsets (like subseq, but preserving
>> collection type; fully independent from the original for GC purposes)
>> and splits by key and index.
>>
>>   [org.clojure/data.avl "0.0.14"]
>>
>>   
>> org.clojure
>> data.avl
>> 0.0.14
>>   
>>
>>   org.clojure:data.avl:0.0.14
>>
>> Changes in this release:
>>
>>  1. http://dev.clojure.org/jira/browse/DAVL-8
>>
>> Fixed a bug in split-key that caused incorrect rank and size
>> information to be stored in some collections resulting from splits
>> using a key absent from the input collection. See the ticket for a
>> more detailed post mortem and links to fixing commits.
>>
>> Many thanks to Darrick Wiebe for the report and test.check
>> properties that exposed broken split-key return values!
>>
>>  2. http://dev.clojure.org/jira/browse/DAVL-7
>>
>> data.avl maps and sets now implement CollReduce (in Clojure) /
>> IReduce (in ClojureScript), leading to significantly improved
>> reduce performance on data.avl inputs. See end of this email for
>> some Criterium benchmarks.
>>
>> Many thanks to Francis Avila for providing the patch!
>>
>>  3. Seqs over data.avl collections can now be = to j.u.Lists.
>>
>>  4. data.avl collections now have their own print-dup implementations
>> that correctly round-trip in Clojure.
>>
>>  5. Public Vars in the clojure.data.avl namespace now carry :added
>> metadata.
>>
>> Cheers,
>> Michał
>>
>>
>> Reduce benchmarks:
>> ==
>>
>> Before patch:
>> -
>>
>> user> (let [m1 (apply sorted-map (range 1))
>> m2 (apply avl/sorted-map (range 1))]
>> (assert (= (reduce (fn [out kv] (+ out (key kv) (val kv)))
>>  0
>>  m1)
>>(reduce (fn [out kv] (+ out (key kv) (val kv)))
>>  0
>>  m2)))
>> (c/quick-bench
>>   (reduce (fn [out kv] (+ out (key kv) (val kv)))
>> 0
>> m1))
>> (c/quick-bench
>>   (reduce (fn [out kv] (+ out (key kv) (val kv)))
>> 0
>> m2)))
>> Evalu

[ANN] data.avl 0.0.14 – BUGFIX to splits, faster map/set reductions

[Michał Marczyk]

Hi,

I am pleased to announce the 0.0.14 release of data.avl, a Clojure
Contrib library providing highly performant drop-in replacements for
Clojure(Script)'s built-in sorted maps and sets that support O(log n)
nth, rank-of, first-class submaps/subsets (like subseq, but preserving
collection type; fully independent from the original for GC purposes)
and splits by key and index.

  [org.clojure/data.avl "0.0.14"]

  
org.clojure
data.avl
0.0.14
  

  org.clojure:data.avl:0.0.14

Changes in this release:

 1. http://dev.clojure.org/jira/browse/DAVL-8

Fixed a bug in split-key that caused incorrect rank and size
information to be stored in some collections resulting from splits
using a key absent from the input collection. See the ticket for a
more detailed post mortem and links to fixing commits.

Many thanks to Darrick Wiebe for the report and test.check
properties that exposed broken split-key return values!

 2. http://dev.clojure.org/jira/browse/DAVL-7

data.avl maps and sets now implement CollReduce (in Clojure) /
IReduce (in ClojureScript), leading to significantly improved
reduce performance on data.avl inputs. See end of this email for
some Criterium benchmarks.

Many thanks to Francis Avila for providing the patch!

 3. Seqs over data.avl collections can now be = to j.u.Lists.

 4. data.avl collections now have their own print-dup implementations
that correctly round-trip in Clojure.

 5. Public Vars in the clojure.data.avl namespace now carry :added
metadata.

Cheers,
Michał


Reduce benchmarks:
==

Before patch:
-

user> (let [m1 (apply sorted-map (range 1))
m2 (apply avl/sorted-map (range 1))]
(assert (= (reduce (fn [out kv] (+ out (key kv) (val kv)))
 0
 m1)
   (reduce (fn [out kv] (+ out (key kv) (val kv)))
 0
 m2)))
(c/quick-bench
  (reduce (fn [out kv] (+ out (key kv) (val kv)))
0
m1))
(c/quick-bench
  (reduce (fn [out kv] (+ out (key kv) (val kv)))
0
m2)))
Evaluation count : 120 in 6 samples of 20 calls.
 Execution time mean : 663.862017 µs
Execution time std-deviation : 9.483652 µs
   Execution time lower quantile : 654.277800 µs ( 2.5%)
   Execution time upper quantile : 677.885431 µs (97.5%)
   Overhead used : 21.423458 ns
Evaluation count : 468 in 6 samples of 78 calls.
 Execution time mean : 1.295972 ms
Execution time std-deviation : 74.233890 µs
   Execution time lower quantile : 1.255853 ms ( 2.5%)
   Execution time upper quantile : 1.420871 ms (97.5%)
   Overhead used : 21.423458 ns

Found 1 outliers in 6 samples (16.6667 %)
low-severe 1 (16.6667 %)
 Variance from outliers : 14.4619 % Variance is moderately inflated by
outliers
nil


After patch:


;;; Clojure 1.9-alpha10

user> (let [m1 (apply sorted-map (range 1))
m2 (apply avl/sorted-map (range 1))]
(assert (= (reduce (fn [out kv] (+ out (key kv) (val kv)))
 0
 m1)
   (reduce (fn [out kv] (+ out (key kv) (val kv)))
 0
 m2)))
(c/quick-bench
  (reduce (fn [out kv] (+ out (key kv) (val kv)))
0
m1))
(c/quick-bench
  (reduce (fn [out kv] (+ out (key kv) (val kv)))
0
m2)))
Evaluation count : 882 in 6 samples of 147 calls.
 Execution time mean : 687.923681 µs
Execution time std-deviation : 17.527428 µs
   Execution time lower quantile : 669.270395 µs ( 2.5%)
   Execution time upper quantile : 710.828484 µs (97.5%)
   Overhead used : 2.633678 ns
Evaluation count : 2940 in 6 samples of 490 calls.
 Execution time mean : 207.386184 µs
Execution time std-deviation : 7.420049 µs
   Execution time lower quantile : 202.829682 µs ( 2.5%)
   Execution time upper quantile : 219.880774 µs (97.5%)
   Overhead used : 2.633678 ns

Found 1 outliers in 6 samples (16.6667 %)
low-severe 1 (16.6667 %)
 Variance from outliers : 13.8889 % Variance is moderately inflated by
outliers
nil

;;; Clojure 1.5.1

user> (let [m1 (apply sorted-map (range 1))
m2 (apply avl/sorted-map (range 1))]
(assert (= (reduce (fn [out kv] (+ out (key kv) (val kv)))
 0
 m1)
   (reduce (fn [out kv] (+ out (key kv) (val kv)))
 0
 m2)))
(c/quick-bench
  (reduce (fn [out kv] (+ out (key kv) (val kv)))
0
m1))
(c/quick-bench
  (reduce (fn [out kv] (+ out (key kv) (val kv)))
0
m2)))
Evaluation count : 990 in 6 samples of 165 calls.
 Execution time 

Re: is reduce/reduced faster than loop/recur?

[Michał Marczyk]

And just to add a pointer in case anybody's interested in opting in to this
behaviour in a custom type, it's possible to do so by implementing
clojure.core.protocols/IKVReduce (data.avl does this) or
clojure.lang.IKVReduce (the iface implemented by built-in maps;
clojure.core provides implementation of clojure.core.protocols/IKVReduce
targeting clojure.lang.IKVReduce). Of course then it is the responsibility
of that custom implementation to implement reduce-kv's semantics correctly.

Cheers,
Michał


On 30 April 2016 at 02:18, Timothy Baldridge  wrote:

> Yes, and it happens for most collections. Vectors, maps, etc. There's even
> a fast path for reduce-kv on maps that doesn't create key value entry
> objects.
>
>
> Timothy
>
> On Fri, Apr 29, 2016 at 6:06 PM, Mark Engelberg 
> wrote:
>
>> So you're saying that this is an optimization that is automatically
>> called when you invoke Clojure's standard reduce function on something like
>> a vector?
>>
>>
>> On Fri, Apr 29, 2016 at 1:14 PM, Alex Miller  wrote:
>>
>>> The main internal protocol is really CollReduce for collections that can
>>> reduce themselves.  InternalReduce is for concrete seq implementations that
>>> can reduce themselves.
>>>
>>> For cases where you are creating new things, you can also plug in a
>>> little more easily by implementing the IReduceInit (reduce with an init
>>> value) or IReduce (extends IReduceInit for the case where an init value is
>>> not supplied) Java interfaces. I would generally prefer these if you are
>>> creating a new thing.
>>>
>>>
>>> On Friday, April 29, 2016 at 2:41:10 PM UTC-5, Camilo Roca wrote:

 puzzler,
 No, Clojure actually has quite a lot of protocols for reducing
 "things". But they are so many that I got lost in which does what and how,
 so I wanted a clarification on the subject.

 Alex miller, excellent answer already gave me some overview of the
 topic.

 Here is a link to Clojure's protocols for reduce:
 https://github.com/clojure/clojure/blob/master/src/clj/clojure/core/protocols.clj
 

 El viernes, 29 de abril de 2016, 21:17:42 (UTC+2), puzzler escribió:
>
> By "internal reduce", are you all talking about the Clojure reducers
> library, or something else?
>
> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To post to this group, send email to clojure@googlegroups.com
>>> Note that posts from new members are moderated - please be patient with
>>> your first post.
>>> To unsubscribe from this group, send email to
>>> clojure+unsubscr...@googlegroups.com
>>> For more options, visit this group at
>>> http://groups.google.com/group/clojure?hl=en
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to clojure+unsubscr...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clojure@googlegroups.com
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to clojure+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
> “One of the main causes of the fall of the Roman Empire was that–lacking
> zero–they had no way to indicate successful termination of their C
> programs.”
> (Robert Firth)
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the 

Re: Scripting with Clojure / "slow" boot time

[Michał Marczyk]

FYI, you may want to have a look at the following commits:

commit 71930b6b6537a796cdf13c4ffa7cf93eb53b6235
Author: Rich Hickey 
Date:   Mon Feb 28 17:55:14 2011 -0500

improve startup time via lazy defn loading

commit c5681382da775e898915b17f3ab18b49c65359ec
Author: Rich Hickey 
Date:   Tue Apr 19 07:41:04 2011 -0400

temporarily disable lazy fn loading

Cheers,
Michał


On 10 February 2016 at 15:20, Herwig Hochleitner 
wrote:

> 2016-02-10 13:06 GMT+01:00 Gary Verhaegen :
>
>> Please excuse the very naive question, but if the main problem seems to
>> be compilation of loaded namespaces, what about aot? Does it help at all?
>> If not, why?
>
>
> As far as I understand the problem, it does help a little bit, but much
> less than expected. The main slowdown is not in generating the byte code,
> but in (statically) constructing all the bits and pieces, that constitute a
> namespace: var objects, fn objects, metadata. This is done on every start,
> because JVM lacks any kind of data literals (AFAIK, even arrays are
> constructed piecemeal on the JVM bytecode stack)
>
>
>> As far as I know, the JVM already does lazy loading with Java classes
>> essentially in the way that Mike described.
>
>
> It does, but when require'ing a namespace, every class it comprises is
> loaded, because the vars + their contents must be reconstructed. This is
> where Mike's proposal comes in.
>
>
> BTW Mike: I think your proposal might just work, if we do that kind of
> lazy loading only for vars containing plain fn forms. It would have to be
> done very carefully indeed, since even a metadata expression can have
> side-effects.
> Makes me wonder if that kind of elaborate effect-tracking, couldn't be
> used more generally, to do closed-world tree-shaking on clojure and
> clojurescript in addition to lazyfying side-effect free code-paths, killing
> multiple birds with one stone.
>
> Clojurescript already has GClosure, I hear you say? Right, then why does
> the lower bound for meaningful CLJS programs seem to be at around 600K
> (minified, uncompressed)?
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] data.avl 0.0.13 – fast sorted collections with O(log n) slices and nth

[Michał Marczyk]

Hi,

I am pleased to announce the 0.0.13 release of data.avl, a Clojure
Contrib library providing highly performant drop-in replacements for
Clojure(Script)'s built-in sorted maps and sets with the following
headline features:

 0. performance often superior to the built-ins (owing to the smaller
height of AVL trees), see end of this email for some benchmarks

 1. logarithmic-time slicing and splits:

(avl/split-key 3 (avl/sorted-set 0 1 2 3 4 5))
;= [#{0 1 2} 3 #{4 5 6}]

(avl/split-at 2 (avl/sorted-set 0 1 2 3 4 5))
;= [#{0 1} #{2 3 4 5}]

(avl/subrange (avl/sorted-set 0 1 2 3 4 5) >= 2 < 5)
;= #{2 3 4}

All returned collections are independent from the originals for GC
purposes (no holding on to out-of-range keys present in the
original collections), all operations run in O(log n) time in
asymptotic terms, and actually fast in real-world terms.

 2. logarithmic-time access by rank and rank queries:

(nth (avl/sorted-set 0 1 2) 1)
;= 1

(avl/rank-of (avl/sorted-map-by > 0 0 1 1 2 2) 0)
;= 2

 3. nearest-neighbour lookups:

(avl/nearest (avl/sorted-set 0 1 2) < 1)
;= 0

The complete clojure.core sorted collections API is also supported, as
is the transient API. See the README for more detailed examples and
feature descriptions.

Fixed in this release:

 1. IMeta and IFn implementations now use correct method names in the
ClojureScript version.

 2. sorted-map/sorted-map-by now throw exceptions if no value is
provided for the final key (in other words, if an odd number of
arguments is passed in) – patch by Andy Fingerhut, thanks!

Changed in this release:

 1. Seqs over data.avl maps now use node objects themselves as map
entries. This is in line with the behaviour of Clojure's built-in
sorted maps. Previously seqs over data.avl maps used
freshly-allocated map entry objects; this had the benefit of not
holding on to subtrees in some scenarios, however not without some
performance and GC pressure cost to regular traversals. Note that
the new behaviour allows the user to rewrap key/value pairs as map
entries if they so choose in a separate map step.

 2. Because of 1., AVL nodes now implement vector and map entry
interfaces.

Cheers,
Michał


;;; A handful of benchmarks

; CIDER 0.9.1 (Java 1.8.0_45-internal, Clojure 1.8.0-RC2, nREPL 0.2.10)

user> (def ks (range 1))
#'user/ks
user> (def ksks (doall (interleave ks ks)))
#'user/ksks
user> (def m1 (apply sorted-map ksks))
#'user/m1
user> (def m2 (apply avl/sorted-map ksks))
#'user/m2
user> (.tree m1)
[4095 4095]
user> (.getTree m2)
[4095 4095]
user> (let []
(prn :> 0)
(c/quick-bench (get m1 0))
(c/quick-bench (get m2 0))
(prn :> 4095)
(c/quick-bench (get m1 4095))
(c/quick-bench (get m2 4095))
(prn :> )
(c/quick-bench (get m1 ))
(c/quick-bench (get m2 )))
:> 0
WARNING: Final GC required 9.525315094951035 % of runtime
WARNING: Final GC required 88.3127760569387 % of runtime
Evaluation count : 2280474 in 6 samples of 380079 calls.
 Execution time mean : 262.138936 ns
Execution time std-deviation : 56.938518 ns
   Execution time lower quantile : 237.779848 ns ( 2.5%)
   Execution time upper quantile : 360.756510 ns (97.5%)
   Overhead used : 20.503990 ns

Found 1 outliers in 6 samples (16.6667 %)
low-severe 1 (16.6667 %)
 Variance from outliers : 64.2134 % Variance is severely inflated by
outliers
WARNING: Final GC required 78.56747149813818 % of runtime
Evaluation count : 2280498 in 6 samples of 380083 calls.
 Execution time mean : 261.626921 ns
Execution time std-deviation : 42.454179 ns
   Execution time lower quantile : 241.444705 ns ( 2.5%)
   Execution time upper quantile : 335.120524 ns (97.5%)
   Overhead used : 20.503990 ns

Found 1 outliers in 6 samples (16.6667 %)
low-severe 1 (16.6667 %)
 Variance from outliers : 47.5275 % Variance is moderately inflated by
outliers
:> 4095
WARNING: Final GC required 168.0614206986717 % of runtime
Evaluation count : 1056 in 6 samples of 176 calls.
 Execution time mean : 25.939625 ns
Execution time std-deviation : 4.135726 ns
   Execution time lower quantile : 22.648015 ns ( 2.5%)
   Execution time upper quantile : 32.134865 ns (97.5%)
   Overhead used : 20.503990 ns
WARNING: Final GC required 293.8046791844393 % of runtime
Evaluation count : 1056 in 6 samples of 176 calls.
 Execution time mean : 24.609377 ns
Execution time std-deviation : 1.015680 ns
   Execution time lower quantile : 23.720800 ns ( 2.5%)
   Execution time upper quantile : 26.283825 ns (97.5%)
   Overhead used : 20.503990 ns

Found 1 outliers in 6 samples (16.6667 %)
low-severe 1 (16.6667 %)
 Variance from outliers : 13.8889 % Variance is moderately inflated by
outliers

Re: [ANN] data.avl 0.0.13 – fast sorted collections with O(log n) slices and nth

[Michał Marczyk]

Just noticed that I forgot to include dependency info and links… Here they
are:

  https://github.com/clojure/data.avl

  [org.clojure/data.avl "0.0.13"]

  
org.clojure
data.avl
0.0.13
  

  compile "org.clojure:data.avl:0.0.13"

Cheers,
Michał


On 9 December 2015 at 10:39, Michał Marczyk <michal.marc...@gmail.com>
wrote:

> Hi,
>
> I am pleased to announce the 0.0.13 release of data.avl, a Clojure
> Contrib library providing highly performant drop-in replacements for
> Clojure(Script)'s built-in sorted maps and sets with the following
> headline features:
>
>  0. performance often superior to the built-ins (owing to the smaller
> height of AVL trees), see end of this email for some benchmarks
>
>  1. logarithmic-time slicing and splits:
>
> (avl/split-key 3 (avl/sorted-set 0 1 2 3 4 5))
> ;= [#{0 1 2} 3 #{4 5 6}]
>
> (avl/split-at 2 (avl/sorted-set 0 1 2 3 4 5))
> ;= [#{0 1} #{2 3 4 5}]
>
> (avl/subrange (avl/sorted-set 0 1 2 3 4 5) >= 2 < 5)
> ;= #{2 3 4}
>
> All returned collections are independent from the originals for GC
> purposes (no holding on to out-of-range keys present in the
> original collections), all operations run in O(log n) time in
> asymptotic terms, and actually fast in real-world terms.
>
>  2. logarithmic-time access by rank and rank queries:
>
> (nth (avl/sorted-set 0 1 2) 1)
> ;= 1
>
> (avl/rank-of (avl/sorted-map-by > 0 0 1 1 2 2) 0)
> ;= 2
>
>  3. nearest-neighbour lookups:
>
> (avl/nearest (avl/sorted-set 0 1 2) < 1)
> ;= 0
>
> The complete clojure.core sorted collections API is also supported, as
> is the transient API. See the README for more detailed examples and
> feature descriptions.
>
> Fixed in this release:
>
>  1. IMeta and IFn implementations now use correct method names in the
> ClojureScript version.
>
>  2. sorted-map/sorted-map-by now throw exceptions if no value is
> provided for the final key (in other words, if an odd number of
> arguments is passed in) – patch by Andy Fingerhut, thanks!
>
> Changed in this release:
>
>  1. Seqs over data.avl maps now use node objects themselves as map
> entries. This is in line with the behaviour of Clojure's built-in
> sorted maps. Previously seqs over data.avl maps used
> freshly-allocated map entry objects; this had the benefit of not
> holding on to subtrees in some scenarios, however not without some
> performance and GC pressure cost to regular traversals. Note that
> the new behaviour allows the user to rewrap key/value pairs as map
> entries if they so choose in a separate map step.
>
>  2. Because of 1., AVL nodes now implement vector and map entry
> interfaces.
>
> Cheers,
> Michał
>
>
> ;;; A handful of benchmarks
>
> ; CIDER 0.9.1 (Java 1.8.0_45-internal, Clojure 1.8.0-RC2, nREPL 0.2.10)
>
> user> (def ks (range 1))
> #'user/ks
> user> (def ksks (doall (interleave ks ks)))
> #'user/ksks
> user> (def m1 (apply sorted-map ksks))
> #'user/m1
> user> (def m2 (apply avl/sorted-map ksks))
> #'user/m2
> user> (.tree m1)
> [4095 4095]
> user> (.getTree m2)
> [4095 4095]
> user> (let []
> (prn :>>>>>>>>> 0)
> (c/quick-bench (get m1 0))
> (c/quick-bench (get m2 0))
> (prn :>>>>>>>>> 4095)
> (c/quick-bench (get m1 4095))
> (c/quick-bench (get m2 4095))
> (prn :>>>>>>>>> )
> (c/quick-bench (get m1 ))
> (c/quick-bench (get m2 )))
> :>>>>>>>>> 0
> WARNING: Final GC required 9.525315094951035 % of runtime
> WARNING: Final GC required 88.3127760569387 % of runtime
> Evaluation count : 2280474 in 6 samples of 380079 calls.
>  Execution time mean : 262.138936 ns
> Execution time std-deviation : 56.938518 ns
>Execution time lower quantile : 237.779848 ns ( 2.5%)
>Execution time upper quantile : 360.756510 ns (97.5%)
>Overhead used : 20.503990 ns
>
> Found 1 outliers in 6 samples (16.6667 %)
> low-severe 1 (16.6667 %)
>  Variance from outliers : 64.2134 % Variance is severely inflated by
> outliers
> WARNING: Final GC required 78.56747149813818 % of runtime
> Evaluation count : 2280498 in 6 samples of 380083 calls.
>  Execution time mean : 261.626921 ns
> Execution time std-deviation : 42.454179 ns
>Execution time lower quantile : 241.444705 ns ( 2.5%)
>Execution time upper quantile : 335.120524 ns (97.5%)
>Overhead used : 20.

Re: [ClojureScript] Re: [ANN] Clojure 1.7.0-beta1 released

[Michał Marczyk]

Just noticed that I sent my previous email to clojure-dev only – reposting
to all groups involved:

On 13 April 2015 at 16:25, Michał Marczyk michal.marc...@gmail.com wrote:
 On 13 April 2015 at 15:48, Alex Miller a...@puredanger.com wrote:
 To get the effect you want in this, using #_ *inside* the reader
conditional would work:

 #?(:cljs #_(def unrelated-1 nil))

 Actually this doesn't work because of the cond-like structure of #?
conditionals:

 user= (read-string {:read-cond :allow} #?(:clj #_foo) bar)
 RuntimeException read-cond requires an even number of forms.
 clojure.lang.Util.runtimeException (Util.java:221)

To this I would add that it is possible to say

Clojure 1.7.0-beta1
user= (read-string {:read-cond :allow} #?(#_#_:clj foo) bar)
bar

taking advantage of the nice stacking property of #_ (which follows from
the recursive nature of the reader in the same way that the original
surprising case does).

Cheers,
Michał


On 13 April 2015 at 21:38, whodidthis ton...@gmail.com wrote:



 On Monday, April 13, 2015 at 4:48:28 PM UTC+3, Alex Miller wrote:

 I think what you're seeing here makes sense.

 On Sunday, April 12, 2015 at 3:39:15 PM UTC-5, whodidthis wrote:

 Are there any thoughts on code like this:

 #_


 This says to ignore the next read form


 #?(:cljs (def unrelated-1 nil))


 This evaluates to *nothing*, ie nothing is read, so it is not ignored by
 the #_.


 #?(:cljs (def unrelated-2 nil))
 #?(:cljs (def unrelated-3 nil))


 These also read as *nothing*.


 #?(:clj (def n 10))


 This *is* read, but ignored per the prior #_

 #?(:clj (defn num [] n))
 ; compile on clj =RuntimeException: Unable to resolve symbol: n


 And then this makes sense.



 I guess it's fine if it continues to work that way but I can imagine it
 being a little surprising from time to time heh


 Conditional reading is definitely something to be careful about - I think
 in this case you are combining two types of conditional reading so be
 doubly careful. :)

 To get the effect you want in this, using #_ *inside* the reader
 conditional would work:

 #?(:cljs #_(def unrelated-1 nil))


 Sorry, back to this stuff again. I tried using discard inside but

 #?(:clj #_'whatever)

 just throws

 CompilerException java.lang.RuntimeException: read-cond starting on line
 32 requires an even number of forms

 when compiling on clojure.

 Would be nice to have a way to ignore reader conditional forms or the
 thingie things inside but there does not seem to be an easy way.

 --
 Note that posts from new members are moderated - please be patient with
 your first post.
 ---
 You received this message because you are subscribed to the Google Groups
 ClojureScript group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojurescript+unsubscr...@googlegroups.com.
 To post to this group, send email to clojurescr...@googlegroups.com.
 Visit this group at http://groups.google.com/group/clojurescript.


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Transducers: sequence versus eduction

[Michał Marczyk]

It may be worth noting that while the return value of range is wrapped in
lazy-seq and thus isn't itself a clojure.lang.IChunkedSeq, what you get
when you realize it is indeed chunked:

(contains? (ancestors (class (seq (range 128 clojure.lang.IChunkedSeq)
true

It doesn't implement c.l.IReduce, but clojure.core.protocols/InternalReduce
has an implementation for c.l.IChunkedSeq. At least transduce should be
able to take advantage of the InternalReduce implementation (via
CollReduce). transduce could be used for short-circuiting search with
(reduced …), so it might be a legitimate contender here.

Cheers,
Michał


On 2 April 2015 at 15:38, Tassilo Horn t...@gnu.org wrote:

 Alex Miller a...@puredanger.com writes:

 Hi Alex,

  If you're going to use expanding transformations and not realize all of
 the
  results then I think sequences are likely a better choice for you.

 Ok, I see.

  However, at least I had expected that in the case where all elements
  are realized the transducer version should have been faster than the
  traditional version which also needs to fully realize all
  intermediate lazy seqs.  Why is it still three times slower?
 
  I think my main suggestion here is that you are using a non-reducible
  source (range) throughout these timings, so transducers have no
  leverage on the input side. CLJ-1515 will make range reducible and
  should help a lot on this particular example.

 Well, even if I revamp the (admittedly contrieved) example to have
 reducible vectors as source and also intermediates

   (let [v (vec (range 0 1000))
 vs (zipmap (range 0 1000)
(for [i (range 0 1000)]
  (vec (range i 1000]
 (time (dorun (sequence (comp (mapcat (fn [i] (vs i)))
  (mapcat (fn [i] (vs i
v

 it still takes 18 seconds instead of 21 with lazy seqs produced by
 range, or just 7 seconds with normal lazy seq functions.

 In my real scenario, I think there's also no IReduces paths because the
 mapcat functions either return normal lazy seqs or Java Collections
 (which are not actually clojure collections).  But usually, the
 transformations are not so freaking expanding as the example above.  I
 benchmarked a bit, and there sometimes using transducers is faster and
 sometimes it is not.  So I've made than configurable (with normal lazy
 seqs as default) so users can benchmark and then decide, and I don't
 need to choose for them. :-)

 Oh, and actually *you* have made that possible by making me aware of

   (sequence (comp xform*) start-coll)

 is almost identical to

   (- start-coll xform*)

 that is, when my macro computes xforms as if they were meant for
 transducing, I can also use them traditionally with -.

 Until now, I've newer used - but before I had implemented the
 expansion for transducers, I used a for with gensyms for intermediates
 like:

   (for [G__1 start-coll
 G__2 (xform1 G__1)
 G__3 (xform2 G__2)]
 G__3)

 That's pretty much different to generate.  But since the xforms for
 transducers and - are the same, switching between lazy seq fns and
 transducers is just changing how start-coll and xforms are composed.
 Awesome!

  So my conclusion is that you cannot use transducers as a kind of
  drop-in replacement of traditional sequence manipulation functions.
  They pay off only when you can make very strong assumptions about the
  sizes and compututation costs of intermediate collections, and I
  think you cannot do that in general.  Or well, maybe you can when you
  program an application but you almost certainly cannot when you
  program a library and thus have no clue about how that's gonna be
  used by users.
 
  Transducers make different trade offs than sequences and there will
  always be cases where one or the other is a better choice.  I really
  appreciate this thread as highlighting some of the nuances.

 Yes, thanks a lot for your patience.  I appreciate that very much.

  Transducers break transformations into three parts - source iteration,
  composed transforms, and output collection.  In the case of reducible
  inputs, multiple transforms, and full realization, transducers can be
  much faster.  If not all of those are in play, then the results are
  more subtle.  One thing I've found in perf testing a lot of stuff is
  that chunked sequences continually surprise me at how fast they can
  be.

 Then maybe I should experiment with chunged seqs.

 Bye,
 Tassilo

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are 

Re: Update multiple values in vector at once

[Michał Marczyk]

(defn upd-vec [input-vector ids new-values]
  (reduce-kv #(assoc %1 %3 (new-values %2)) input-vector ids))

(upd-vec [0 0 0 0 0] [1 3] [1.44 1.45])
;= [0 1.44 0 1.45 0]


On 30 March 2015 at 20:05, Alexandr updates...@gmail.com wrote:

 Hello everybody,

 How can I update values in the vector given vector of id-s and new values?

 For example

 (defn upd-vec [input-vector ids new-values]
 
 )

 (upd-vec [0 0 0 0 0] [1 3] [1.44 1.45])

 Output: [0 1.44 0 1.45 0]  (1st and 3rd elements are replaced)

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Accessing branches in PersistentTreeMap / sorted-map / CLJ-1008

[Michał Marczyk]

PS. Well, I suppose you'd want to make sure that you're not getting a key
that lies past the right endpoint of the map when using the nearest-based
approach.


On 20 February 2015 at 16:39, Michał Marczyk michal.marc...@gmail.com
wrote:

 Do you mean you'd like to compress

   {0 :a 1 :a 2 :a 3 :a 4 :b 5 :b 6 :c 7 :c 8 :c 9 :c}

 to something like

   {[0 1 2] :a [4 5] :b [6 7 8 9] :c}

 while maintaining the ability to ask for the value at 0, 1, …, 9?

 If so, you could represent the above as

   {0 :a 2 :a 4 :b 5 :b 6 :c 9 :c}

 (notice no explicit entries for 1, 7, 8) and query for the value at 7,
 say, using

   (val (avl/nearest the-map = 7))
   ;= :c

 (avl/nearest can be implemented for built-in sorted maps using subseq and
 first, in fact the test suite for data.avl has an implementation like that
 that it compares avl/nearest against).

 If you need more operations – say, data.avl-style slice-{at,key} and
 subrange – those could be supported with some care (mostly around the
 possibility that a slice, say, removes an endpoint of an interval – you may
 need to add the slice boundary as a replacement in that case).

 Cheers,
 Michał


 On 20 February 2015 at 16:15, David James davidcja...@gmail.com wrote:

 Thanks for your comments. I see now that I should clarify: all I really
 need is public access to the left and right Java methods in PTM. (So,
 perhaps CLJ-1008 is asking for more than I really need.)

 It seems to me that since Clojure's RB-Tree implementation (i.e.
 sorted-map = PTM), it might as well expose a Java API for root node (which
 it does) and branches (which is does not, currently). Is there any downside
 to exposing the 'right' and 'left' Java methods as public?

 In the near term, I'll be using data.avl. I'm glad it exists! My use case
 involves a non-overlapping interval map to store time series data that
 doesn't change very often.

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.




-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Accessing branches in PersistentTreeMap / sorted-map / CLJ-1008

[Michał Marczyk]

Do you mean you'd like to compress

  {0 :a 1 :a 2 :a 3 :a 4 :b 5 :b 6 :c 7 :c 8 :c 9 :c}

to something like

  {[0 1 2] :a [4 5] :b [6 7 8 9] :c}

while maintaining the ability to ask for the value at 0, 1, …, 9?

If so, you could represent the above as

  {0 :a 2 :a 4 :b 5 :b 6 :c 9 :c}

(notice no explicit entries for 1, 7, 8) and query for the value at 7, say,
using

  (val (avl/nearest the-map = 7))
  ;= :c

(avl/nearest can be implemented for built-in sorted maps using subseq and
first, in fact the test suite for data.avl has an implementation like that
that it compares avl/nearest against).

If you need more operations – say, data.avl-style slice-{at,key} and
subrange – those could be supported with some care (mostly around the
possibility that a slice, say, removes an endpoint of an interval – you may
need to add the slice boundary as a replacement in that case).

Cheers,
Michał


On 20 February 2015 at 16:15, David James davidcja...@gmail.com wrote:

 Thanks for your comments. I see now that I should clarify: all I really
 need is public access to the left and right Java methods in PTM. (So,
 perhaps CLJ-1008 is asking for more than I really need.)

 It seems to me that since Clojure's RB-Tree implementation (i.e.
 sorted-map = PTM), it might as well expose a Java API for root node (which
 it does) and branches (which is does not, currently). Is there any downside
 to exposing the 'right' and 'left' Java methods as public?

 In the near term, I'll be using data.avl. I'm glad it exists! My use case
 involves a non-overlapping interval map to store time series data that
 doesn't change very often.

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Accessing branches in PersistentTreeMap / sorted-map / CLJ-1008

[Michał Marczyk]

You don't need left and right, you can literally use a sorted map / PTM
that looks like {0 :a 3 :a …} as your representation and build a wrapper to
query it appropriately using functions like avl/nearest, but implemented
using first/(r)(sub)seq.

Here's a prototype:

https://github.com/michalmarczyk/ticktickticktock

(require '[ticktickticktock.core :as ])

(/-map 0 :a 1 :a 2 :a 3 :a 4 :b 5 :b 6 :c 7 :c 8 :c 9 :c)
;= {0 :a, 3 :a, 4 :b, 5 :b, 6 :c, 9 :c}
;; the above is a wrapper around a sorted map

Some interactions:

user= (get (/-map 0 :a 1 :a 2 :a 3 :a 4 :b 5 :b 6 :c 7 :c 8 :c 9
:c) -1)
nil
user= (get (/-map 0 :a 1 :a 2 :a 3 :a 4 :b 5 :b 6 :c 7 :c 8 :c 9
:c) 0)
:a
user= (get (/-map 0 :a 1 :a 2 :a 3 :a 4 :b 5 :b 6 :c 7 :c 8 :c 9
:c) 1)
:a
user= (get (/-map 0 :a 1 :a 2 :a 3 :a 4 :b 5 :b 6 :c 7 :c 8 :c 9
:c) 8)
:c

NB. this really is a prototype – it supports only the what's strictly
necessary for the above demo to work – but it should be fairly
straightforward to extend it to support other desirable features. (For
example assoc elsewhere than beyond the rightmost key etc. dissoc of single
values doesn't really make sense here, but I suppose inserting a gap
would.) Also, I don't think it handles lookups in between intervals very
well… That's fixable as well. I guess the first thing to fix would be the
lack of a clear target API design. :-)

Cheers,
Michał


On 20 February 2015 at 20:09, David James davidcja...@gmail.com wrote:

 Yes, exactly, you read my mind.

 (I'd also like to do this a sorted-map / PersistentTreeMap (nudge, nudge)
 -- all that is missing would be public 'left' and 'right' accessors. I
 don't necessarily need rank functionality.)

 On Friday, February 20, 2015 at 10:39:26 AM UTC-5, Michał Marczyk wrote:

 Do you mean you'd like to compress

   {0 :a 1 :a 2 :a 3 :a 4 :b 5 :b 6 :c 7 :c 8 :c 9 :c}

 to something like

   {[0 1 2] :a [4 5] :b [6 7 8 9] :c}

 while maintaining the ability to ask for the value at 0, 1, …, 9?

 If so, you could represent the above as

   {0 :a 2 :a 4 :b 5 :b 6 :c 9 :c}

 (notice no explicit entries for 1, 7, 8) and query for the value at 7,
 say, using

   (val (avl/nearest the-map = 7))
   ;= :c

 (avl/nearest can be implemented for built-in sorted maps using subseq and
 first, in fact the test suite for data.avl has an implementation like that
 that it compares avl/nearest against).

 If you need more operations – say, data.avl-style slice-{at,key} and
 subrange – those could be supported with some care (mostly around the
 possibility that a slice, say, removes an endpoint of an interval – you may
 need to add the slice boundary as a replacement in that case).

 Cheers,
 Michał


 On 20 February 2015 at 16:15, David James david...@gmail.com wrote:

 Thanks for your comments. I see now that I should clarify: all I really
 need is public access to the left and right Java methods in PTM. (So,
 perhaps CLJ-1008 is asking for more than I really need.)

 It seems to me that since Clojure's RB-Tree implementation (i.e.
 sorted-map = PTM), it might as well expose a Java API for root node (which
 it does) and branches (which is does not, currently). Is there any downside
 to exposing the 'right' and 'left' Java methods as public?

 In the near term, I'll be using data.avl. I'm glad it exists! My use
 case involves a non-overlapping interval map to store time series data that
 doesn't change very often.

  --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Accessing branches in PersistentTreeMap / sorted-map / CLJ-1008

[Michał Marczyk]

 James david...@gmail.com wrote:

 Andy, to take advantage of the Red-Black Tree, I'm looking for public
 API access to the branches. (I'm not looking for a work-around.)

 More discussion on this at SO: http://stackoverflow.com/
 questions/1981859/finding-keys-closest-to-a-given-value-
 for-clojure-sorted-maps

 Thanks to both Michał Marczyk and Andy for commenting on CLJ-1008 (
 http://dev.clojure.org/jira/browse/CLJ-1008).

 On Thursday, February 19, 2015 at 1:02:53 PM UTC-5, Andy Fingerhut wrote:

 I haven't checked carefully, but from at least a quick look it appears
 that implementing the NavigableMap and NavigableSet interfaces could be
 done by using the existing subseq and rsubseq functions in clojure.core?

 It doesn't give you access to subtree nodes directly, but perhaps it is
 sufficient for your purposes?

 Andy

 On Wed, Feb 18, 2015 at 6:04 PM, David James david...@gmail.com
 wrote:

 Summary: I'd like to find a public API to work with the underlying
 tree of a sorted-map.

 For example:

 (def t (sorted-map 1 :a 2 :b 3 :c 4 :d 5 :e 6 :f))

 The underlying implementation of sorted-map uses a PersistentTreeMap,
 which can be accessed with `tree`:

 (.tree t) ;= [2 :b]

 I have not found a way to access the left and right branches, since
 calling `left` fails:

 (.left (.tree t))

 IllegalArgumentException Can't call public method of non-public class:
 public clojure.lang.PersistentTreeMap$Node clojure.lang.
 PersistentTreeMap$BlackBranch.left()  
 clojure.lang.Reflector.invokeMatchingMethod
 (Reflector.java:88)

 Perhaps it would be reasonable to make such calls possible, at least
 from the Java API (without reflection)?

 CLJ-1008 (http://dev.clojure.org/jira/browse/CLJ-1008) offers one
 possible way to support a public API. It was created in 2012. Perhaps it
 could use another look. Thoughts?

  --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com
 Note that posts from new members are moderated - please be patient
 with your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to clojure+u...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


  --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to clojure+u...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


  --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: :keys and :or destructuring where defaults refer to one another

[Michał Marczyk]

I created a ticket and posted a patch:

http://dev.clojure.org/jira/browse/CLJ-1613


On 12 December 2014 at 08:35, Michał Marczyk michal.marc...@gmail.com wrote:
 (let [foo 1
bar 2
{:keys [bar foo]
 :or {foo 3 bar (inc foo)}} {}]
   {:foo foo :bar bar})
 ;= {:foo 3, :bar 4}

 (let [foo 1
   bar 2
   {:keys [foo bar]
:or {foo 3 bar (inc foo)}} {}]
   {:foo foo :bar bar})
 ;= {:foo 3, :bar 2}

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: :keys and :or destructuring where defaults refer to one another

[Michał Marczyk]

Quoting Michael's Gist:

(let [{:keys [bar foo]
   :or {foo 1
bar (inc foo)}} {}]
  (assert (= foo 1))
  (assert (= bar 2)))

The main problem I see with this is that I'd expect the foo in :or to
refer to any foo in the enclosing scope. As things stand, whether it
does or not depends on the ordering of keys in :keys:

(let [foo 1
   bar 2
   {:keys [bar foo]
:or {foo 3 bar (inc foo)}} {}]
  {:foo foo :bar bar})
;= {:foo 3, :bar 4}

(let [foo 1
  bar 2
  {:keys [foo bar]
   :or {foo 3 bar (inc foo)}} {}]
  {:foo foo :bar bar})
;= {:foo 3, :bar 2}

Feels like a bug to me.

Cheers,
Michał


On 12 December 2014 at 06:44,  adrian.med...@mail.yu.edu wrote:
 Common Lisp has a really well thought approach to parameter lambda lists. If
 you feel strongly about resolving this ambiguity and enforcing consistency
 around sequential binding in the destructuring syntax, perhaps that would be
 a good place to root a design you can flesh out in Jira. Here's a resource
 you might find interesting as a starting point:
 http://www.lispworks.com/documentation/HyperSpec/Body/03_dad.htm

 On Thursday, December 11, 2014 11:55:36 PM UTC-5, Michael Blume wrote:

 Yep, I spent some time playing with the macro and the macroexpand. It
 looks like

 a) it only works if the dependent keys come *before* the keys they depend
 on (ie the opposite of how you'd order, say, defs)

 b) this ordering arises entirely from the seq ordering of
 PersistentArrayMap (keys are stuck into the map here
 https://github.com/clojure/clojure/blob/clojure-1.6.0/src/clj/clojure/core.clj#L4083
 and taken out again here
 https://github.com/clojure/clojure/blob/clojure-1.6.0/src/clj/clojure/core.clj#L4090)

 The latter makes it pretty clear that this is accidental behavior and, as
 you say, shouldn't be relied on -- in particular, if you have more than
 about 8 keys, you spill over to a PersistentHashMap and get everything in a
 random order and it almost certainly fails.

 The trouble is this behavior is already used by ring-middleware-format,
 which then fails to compile if clojure uses a different implementation for
 small maps.

 I'm wondering if given the brittleness of this behavior we should make
 sure it can't be used in future versions of clojure.

 On Thu Dec 11 2014 at 6:56:22 PM adrian...@mail.yu.edu wrote:

 Whenever you want to get insight in how a macro is rewriting your code,
 try evaluating your form quoted with macroexpand.

 Here's a gist with the macroexpansion each form.

 https://gist.github.com/aamedina/542b084d31d4e0c9a7a8

 Hopefully the expansion makes things clear!

 On Thursday, December 11, 2014 6:11:59 PM UTC-5, Michael Blume wrote:

 If I make my defaults on a :keys :or destructuring depend on the values
 of other keys, I *can* get a compile-time error, depending on what order I
 give the keys https://gist.github.com/MichaelBlume/4891dafdd31f0dcbc727

 Is this on-spec behavior? Should the former be allowed but not the
 latter? Should both be allowed? Should neither?

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+u...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, 

[ANN] ctries.clj 0.0.2 – concurrently modifiable, O(1)-snapshotable maps

[Michał Marczyk]

Hi,

I am pleased to announce the immediate availability of version 0.0.2
of ctries.clj, a Clojure implementation of the Ctrie data structure
introduced by Prokopec, Bronson, Bagwell and Odersky.

This release includes some new tests, both single and multithreaded,
exercising a number of features, most notably the
java.util.concurrent.ConcurrentMap methods. (This interface includes a
handful of atomic operations (see below) and is fully implemented by
ctries.clj maps. A Clojure API for this functionality will likely be
introduced in a future release.) The remaining changes were necessary
to make these new tests pass.

  [ctries.clj 0.0.2]

Cheers,
Michał


j.u.c.ConcurrentMap methods summary:

putIfAbsent(key, value) – atomically {
  if there is no mapping for key in the given map
add one with the given value
}

remove(key, value) – atomically {
  if key is mapped to value in the given map
remove it
}

replace(key, value) – atomically {
  if key is present in the given map
remap it to the given value
}

replace(key, value1, value2) – atomically {
  if key is mapped to value1 in the given map
remap it to value2
}

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: atoms, memoize, future-s and CAS

[Michał Marczyk]

On 8 December 2014 at 17:54, Andy L core.as...@gmail.com wrote:
 But I'd personally just use a delay rather than locking for this
 purpose.


 It is not that I like locking at all. However I still fail to see, how in a
 multithreaded context memoize/cache prevents executing a given function more
 than once (which I want to avoid at any cost here) since cache lookup and
 swap! does not seem to be atomic :
 https://github.com/clojure/core.cache/blob/master/src/main/clojure/clojure/core/cache.clj#L52

When you say

  (delay (foo)),

foo will be called at most once, regardless of how many times you
deref (@) / force the delay. (If you never force the delay, it will
not be called at all.) The way this is enforced is through making
deref a synchronized method on delays.

Cheers,
Michał

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: atoms, memoize, future-s and CAS

[Michał Marczyk]

Oh, and as for how to use it here, you could for example say

  (.putIfAbsent concurrent-hash-map :foo (delay (foo)))

Then the first thread to @(get concurrent-hash-map :foo (delay
:not-found)) (or similar) would actually compute the value.

With a map in an atom, you could swap! using a function like

  (fn [old-state]
(if (contains? old-state :foo)
  (assoc old-state :foo (delay (foo)))
  old-state))

I'd probably prefer a CHM for this purpose, though.

Michał


On 8 December 2014 at 21:33, Michał Marczyk michal.marc...@gmail.com wrote:
 On 8 December 2014 at 17:54, Andy L core.as...@gmail.com wrote:
 But I'd personally just use a delay rather than locking for this
 purpose.


 It is not that I like locking at all. However I still fail to see, how in a
 multithreaded context memoize/cache prevents executing a given function more
 than once (which I want to avoid at any cost here) since cache lookup and
 swap! does not seem to be atomic :
 https://github.com/clojure/core.cache/blob/master/src/main/clojure/clojure/core/cache.clj#L52

 When you say

   (delay (foo)),

 foo will be called at most once, regardless of how many times you
 deref (@) / force the delay. (If you never force the delay, it will
 not be called at all.) The way this is enforced is through making
 deref a synchronized method on delays.

 Cheers,
 Michał

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: atoms, memoize, future-s and CAS

[Michał Marczyk]

On 8 December 2014 at 21:46, Fluid Dynamics a2093...@trbvm.com wrote:
 [...]
 Which means it's locking or bust. You just get to either do the locking
 yourself or delegate :)

Sure, but isn't it nice when somebody else does your locking for you? :-)

Incidentally, there is a trade-off here between lockless reads and
cache-locking writes in the version with (locking …) and synchronized
reads (of delays) and somewhat concurrency-friendly writes in the
version with CHM.putIfAbsent and delays. So actually explicit (locking
…) might be preferable for certain workloads. Benchmarking required.

Michał


 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: clojure.edn won't accept clojure.java.io/reader? How to work around this and why isn't this documented anywhere?

[Michał Marczyk]

On 8 December 2014 at 21:17, Fluid Dynamics a2093...@trbvm.com wrote:
 On Monday, December 8, 2014 9:32:28 AM UTC-5, Las wrote:
 […]
 io/reader is not meant to be used solely as an input to edn/read.


 AFAICT, PushbackReader is substitutable anywhere a reader is expected, but
 apparently a plain unwrapped BufferedReader is not.

user= (line-seq (java.io.PushbackReader. (io/reader (io/file .bashrc
ClassCastException java.io.PushbackReader cannot be cast to
java.io.BufferedReader  clojure.core/line-seq (core.clj:2955)

It works with the plain unwrapped BufferedReader that io/reader returns.

Unfortunately PushbackReader and BufferedReader are both classes
rather than interfaces and they both have methods that the other class
does not. So, there isn't a single good choice for what a reader
function in Clojure on the JVM should return.

Cheers,
Michał



 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] ctries.clj, a Clojure implementation of Ctries

[Michał Marczyk]

Hi,

I am pleased to announce the initial release of ctries.clj, a port of
the original Scala implementation of this fascinating data structure
with a Clojure API that allows usage scenarios such as this:

  (def x (ct/concurrent-map :foo 1))
  ;; @ takes independently mutable snapshots;
  ;; persistent! takes immutable snapshots
  (def y @x)
  (def z @x)

  (assoc! x 1 1)
  (assoc! y 2 2)
  (assoc! z 3 3)

  [x y z]
  ;= [#Ctrie {1 1, :foo 1} #Ctrie {:foo 1, 2 2} #Ctrie {3 3, :foo 1}]

(NB. it is perfectly ok to modify Ctries in place even though I chose
to have them participate in the transient API rather than come up with
new function names for this initial release. Also note that the
resulting version graph is reminiscent of version graphs of
persistent data structures – there's an original an two independently
derived versions – but here the update operations actually perform the
updates in place and creating new versions is accomplished through
explicit calls to deref / @.)

Ctries are concurrently modifiable, lock-free (global progress
guaranteed), constant-time snapshotable, HAMT-like maps. They were
introduced in the following two-papers:

 * Prokopec, Bagwell, Odersky, *Cache-Aware Lock-Free Concurrent Hash
   Tries*, EPFL 2011
   (http://infoscience.epfl.ch/record/166908/files/ctries-techreport.pdf)

 * Prokopec, Bronson, Bagwell, Odersky, *Concurrent Tries with
   Efficient Non-Blocking Snapshots*, EPFL 2011
   (http://lampwww.epfl.ch/~prokopec/ctries-snapshot.pdf)

Their design bears some similarity to Clojure's transients in that
they prevent in-place modifications to a tree-based data structure
from affecting instances with which it shares structure by tracking
subtree ownership. The mechanism for accomplishing this in a
concurrent setting involves two versions of restricted double CAS,
of which one (GCAS) is an independently interesting contribution of
the second Ctries paper.

I had the opportunity to speak about this library and certain ideas
common to ctries and transients at Clojure eXchange 2014; the
presentation can be viewed here:

  Ephemeral-first data structures
  (https://skillsmatter.com/skillscasts/6028-ephemeral-first-data-structures).

This library should be considered experimental at this stage. Zach
Tellman's amazing collection-check (the frightful library that asks
you to call just one function in your test suite and then, when you
do, proceeds to summarily demolish your data structures) has grumpily
given it its stamp of approval for single-threaded use. A
multithreaded test suite is forthcoming.

If you'd like to take it for a spin, it is Apache licensed and
available here:

  https://github.com/michalmarczyk/ctries.clj

  https://clojars.org/ctries.clj

The version number for this initial release is 0.0.1.

Cheers,
Michał

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Using an atom for a caching map

[Michał Marczyk]

java.util.concurrent.ConcurrentHashMap has a putIfAbsent method that
could be used with delays to support this use case with a no
recomputation guarantee:

(def chm (java.util.concurrent.ConcurrentHashMap.))

;; this will work as expected whether chm has a value for :foo or not
(let [d (delay (+ 1 2))]
  ;; NB. the two @d expressions refer to different ds
  (if-let [d (.putIfAbsent chm :foo d)] @d @d))
;= 3

;; this will remove all entries
(.clear chm)

No point switching away from Atoms if occasional recomputation is ok
and they perform well enough, of course (and whether this would be an
improvement performance-wise in any given scenario would need to be
determined by benchmarking).


On 2 September 2014 00:54, Marcus Magnusson marx...@gmail.com wrote:
 Forget about sleep, there's work to be done! I realized that the drawback I
 mentioned with my solution, where concurrent calls to the memoized function
 with different sets of arguments would be handled one-by-one, could easily
 be resolved by having the cache inside synced-memoize store futures for
 calculating the value, rather than the value itself - some small changes and
 everything should hopefully work well (don't ask me about overhead, though!)


 (defn synced-memoize [f]
   (let [mem (atom {})
 handle-ch (chan)]
 (go-loop []
   (let [[args ret-ch] (! handle-ch)]
 (! ret-ch (if-let [e (find @mem args)]
  (val e)
  (let [ret (future (apply f args))]
(swap! mem assoc args ret)
ret)))
 (recur)))
 (fn [ args]
   (if-let [e (find @mem args)]
 (deref (val e))
 (let [ret (chan)]
   (!! handle-ch [args ret])
   (deref (!! ret)))

 (def lookup
   (let [calc-value* (synced-memoize calc-value)]
 (fn [cache key]
   (if (contains? @cache key)
 (@cache key)
 ((swap! cache assoc key (calc-value* key)) key)


 Den måndagen den 1:e september 2014 kl. 23:55:58 UTC+2 skrev Marcus
 Magnusson:

 Huh, I gave it some more thought - of course, the reason why memoize won't
 help us here is that there is no synchronization between simultaneous
 invocations with the same set of arguments. This is typically not a problem
 if the underlying function is fast, but in our case it would be neat with an
 alternative. I got the idea of writing a channel-based version of memoize,
 which will ensure that invoking the memoized function simultaneously with
 the same arguments will only call the underlying function once. Below is a
 quick implementation with one obvious drawback (and probably tons of bugs
 and points of improvement!), namely that if the underlying function is
 costly, and we invoke the memoized function with a bunch of different
 non-cached set of arguments, then calculating the values will be done
 one-by-one. This could be fixed for example by having handle-ch delegate to
 another channel, where we have one channel per set of arguments - I'll leave
 that for someone else, or for when I've had some sleep and realized what a
 bad idea it was...  :) Note that I haven't worked much with core.async, and
 that there is probably a much more straightforward solution (for example the
 one given by Thomas Heller), so please let me know of any issues with this:


 (defn synced-memoize [f]
   (let [mem (atom {})
 handle-ch (chan)]
 (go-loop []
   (let [[args ret-ch] (! handle-ch)]
 (! ret-ch (if-let [e (find @mem args)]
  (val e)
  (let [ret (apply f args)]
(swap! mem assoc args ret)
ret)))
 (recur)))
 (fn [ args]
   (if-let [e (find @mem args)]
 (val e)
 (let [ret (chan)]
   (!! handle-ch [args ret])
   (!! ret))

 (def lookup
   (let [calc-value* (synced-memoize calc-value)]
 (fn [cache key]
   (if (contains? @cache key)
 (@cache key)
 (swap! cache assoc key (calc-value* key))



 Den måndagen den 1:e september 2014 kl. 22:35:56 UTC+2 skrev Marcus
 Magnusson:

 I reckon if that worked, there would be no need for memoize anyway, but I
 don't think swap! will allow for it. I'm far from an expert on swap! or
 atoms, but several swap!s may be run simultaneously on a single atom (and
 swap! may re-run the swapping function if the atom has been changed since
 the swapping function was invoked). In other words, if two swap!s are
 invoked at about the same time for the same key (which is not currently in
 the cache), both invocations may be given the same value of the cache at
 that moment, which will lead to the value being re-calculated in both
 invocations - and even if calc-value is memoized, the same thing may occur
 in the memoized function.

 As I said, I'm far from an expert, so I wrote a small test that shows
 that calc-value may indeed be called more than once (core.async here is
 simply to ensure 

Re: Benchmarking structural sharing

[Michał Marczyk]

It seems hard to answer in a completely generic fashion. If there's a
certain collection of vectors which you'd like to share structure, it
may be possible to put a number on the degree of sharing achieved by
examining the internals of those vectors. That wouldn't address the
issue of intermediate allocations internal to your algorithms, but
that might be fine.

As for a possible approach to the above, I've just added a function
called count-nodes to clojure.core.rrb-vector.debug (the JVM version
only) which counts all vector nodes used by a collection of vectors
(PV / gvec / RRB). An example from my REPL:

(let [vs (mapv (comp vec range) [2048 2048 2048])]
  [(apply dv/count-nodes (apply fv/catvec vs) vs)
   (apply dv/count-nodes (reduce into [] vs) vs)])
;= [203 396]

I might tweak the way it works a little (for example, it could take a
collection rather than varargs and count internal arrays rather than
nodes; the latter change could be beneficial, as tails are just arrays
without a node wrapper).

Hope this helps.

Cheers,
Michał


On 11 August 2014 14:56, Paul Butcher p...@paulbutcher.com wrote:
 Is there any way to benchmark the degree of structural sharing achieved by a
 Clojure algorithm? I'm evaluating two different implementations of an
 algorithm, one which uses zippers and one which uses rrb-vector. It would be
 great if there were some way to quantify the degree to which they both
 achieved (or didn't) structural sharing.

 --
 paul.butcher-msgCount++

 Silverstone, Brands Hatch, Donington Park...
 Who says I have a one track mind?

 http://www.paulbutcher.com/
 LinkedIn: http://www.linkedin.com/in/paulbutcher
 Skype: paulrabutcher

 Author of Seven Concurrency Models in Seven Weeks: When Threads Unravel
 http://pragprog.com/book/pb7con

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: printing lazy lists

[Michał Marczyk]

Use pr-str:

user= (str (lazy-seq (list 1 2 3)))
clojure.lang.LazySeq@7861
user= (pr-str (lazy-seq (list 1 2 3)))
(1 2 3)

Cheers,
Michał


On 15 May 2014 16:29, Phillip Lord phillip.l...@newcastle.ac.uk wrote:


 I am trying to dump a representation of the contents of a list to file.
 I've recently changed how I generated this list and it's now lazy (not really
 by design more by side-effect, if you will excuse the poor choice of words).

 I was using

 (spit file (str lst \n))

 which worked quite nicely, but now it is failing. The problem is that I get a
 file full of clojure.lang.LazySeq@ lines. The problem comes from LazySeq
 directly, as this demonstration with range shows.

 user (str (list 1 2 3 4 5 ))
 (1 2 3 4 5)
 user (str (range 4))
 clojure.lang.LazySeq@e1b83
 user (println (range 4))
 (0 1 2 3)
 nil


 println is using prn and a multimethod to print out. In fact,
 clojure.lang.LazySeq doesn't implement toString, nor does it's super class.

 The best solution that I have come up with so far is to do

 (str (apply list (range 4)))

 I guess I can see why LazySeq doesn't implement toString by printing
 everything out, but is there a better way around my problem?

 Phil

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your 
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups 
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: test.check

[Michał Marczyk]

Sure, here's a possible approach:

user (gen/sample (gen/fmap (partial apply concat) (gen/list (gen/fmap
#(repeat 3 %) gen/int
(()
 ()
 (0 0 0)
 (3 3 3 3 3 3 2 2 2)
 (3 3 3 2 2 2 0 0 0 3 3 3)
 (2 2 2)
 (-4 -4 -4 -6 -6 -6)
 (4 4 4 5 5 5)
 (7 7 7 3 3 3)
 (3 3 3 7 7 7 9 9 9 7 7 7 4 4 4 -4 -4 -4))

Cheers,
Michał


On 14 May 2014 08:50, Maciej Jaśkowski maciej.jaskow...@gmail.com wrote:
 Hi there,

 I would like to create a generator generating a list of ints with each value
 repeated a number of times e.g.

 (gen/sample gen-repeater [ 1 1 1 2 2 2 -1 -1 -1 9 9 9])

 this
 (gen/sample (gen/bind gen/int (fn [i] (gen/return (take 5 (repeat i))
 returns e.g. ((0 0 0 0 0) (-1 -1 -1 -1 -1) (0 0 0 0 0))
 which is of no interest to me :-(

 Is that even possible under test.check to have a generator like that?

 Best,
 Maciej Jaśkowski

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Expected inconsistency between set and map w/ ArrayList?

[Michał Marczyk]

It's not just to facilitate inlining, but also to limit the perf hit
for hashing non-collections, some of which make completely reasonable
map keys and set members. I've used the classes Alex Miller mentioned
he was interested in for benchmarking: Class, Character, Var; these
are all good examples, and only one is under Clojure's control.

I used a map or map entry or collection test in earlier versions of
the patch, but switched to the class name hack because the three-way
type check seems to slow down getting to the default case much more
than examining the class name. For types that actually implement
IHashEq, however, the original patch was faster, so I think HotSpot is
happy to inline either version.


On 13 May 2014 14:00, Mike Fikes mikefi...@me.com wrote:
 To facilitate inlining, the patch calls out to a separate larger method
 which handles a group of cases.

 +if(o.getClass().getName().startsWith(java.util.))
 +return doalienhasheq(o);
 +return o.hashCode();


 I was wondering whether an efficient improvement is possible that would
 support things like Guava ImmutableList.


 In particular, I was wonder which default cases are currently handled by
 the return o.hashCode() above. Replacing the three lines above with


 +return doalienhasheq(o);


 would allow the patch to also handle non-java.util collection
 implementations, but push the default cases down into the bottom of that
 method.


 On Tuesday, May 13, 2014 12:38:54 AM UTC-4, Michał Marczyk wrote:

 I've posted a patch that makes java.util.{List,Map,Map.Entry,Set}
 hashes consistent with those of appropriate Clojure collection types
 on the ticket.

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.

 For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Expected inconsistency between set and map w/ ArrayList?

[Michał Marczyk]

Also, = defers to equals in all cases except two: (1) numeric
arguments, (2) at least one IPersistentCollection among the arguments.

Clojure collections are allowed to determine whether they are = to the
other thing or not. So, we should calculate special hashCodes for the
java.util collections Clojure collections know about.

Other collection types will just use equals for comparisons and so for
them we have to return hashCode to preserve =/hash semantics by
relying on the equals/hashCode contract -- there is nothing else we
can do. Well, strictly speaking we could apply an arbitrary function
to hashCode, so murmuring it (as with Strings) is an option; that
would come at a performance cost, however (I benchmarked a version
that did that), and I don't think there'd be much benefit -- certainly
none at all for classes using identity hash codes.

Cheers,
Michał


On 13 May 2014 14:20, Michał Marczyk michal.marc...@gmail.com wrote:
 It's not just to facilitate inlining, but also to limit the perf hit
 for hashing non-collections, some of which make completely reasonable
 map keys and set members. I've used the classes Alex Miller mentioned
 he was interested in for benchmarking: Class, Character, Var; these
 are all good examples, and only one is under Clojure's control.

 I used a map or map entry or collection test in earlier versions of
 the patch, but switched to the class name hack because the three-way
 type check seems to slow down getting to the default case much more
 than examining the class name. For types that actually implement
 IHashEq, however, the original patch was faster, so I think HotSpot is
 happy to inline either version.


 On 13 May 2014 14:00, Mike Fikes mikefi...@me.com wrote:
 To facilitate inlining, the patch calls out to a separate larger method
 which handles a group of cases.

 +if(o.getClass().getName().startsWith(java.util.))
 +return doalienhasheq(o);
 +return o.hashCode();


 I was wondering whether an efficient improvement is possible that would
 support things like Guava ImmutableList.


 In particular, I was wonder which default cases are currently handled by
 the return o.hashCode() above. Replacing the three lines above with


 +return doalienhasheq(o);


 would allow the patch to also handle non-java.util collection
 implementations, but push the default cases down into the bottom of that
 method.


 On Tuesday, May 13, 2014 12:38:54 AM UTC-4, Michał Marczyk wrote:

 I've posted a patch that makes java.util.{List,Map,Map.Entry,Set}
 hashes consistent with those of appropriate Clojure collection types
 on the ticket.

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.

 For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Expected inconsistency between set and map w/ ArrayList?

[Michał Marczyk]

Unfortunately the Guava example means that this approach doesn't
really solve the problem at hand... Thanks for pointing it out.

The earlier version with the three-way test in place of the
java.util. check should be semantically correct, at least.


On 13 May 2014 14:40, Michał Marczyk michal.marc...@gmail.com wrote:
 Also, = defers to equals in all cases except two: (1) numeric
 arguments, (2) at least one IPersistentCollection among the arguments.

 Clojure collections are allowed to determine whether they are = to the
 other thing or not. So, we should calculate special hashCodes for the
 java.util collections Clojure collections know about.

 Other collection types will just use equals for comparisons and so for
 them we have to return hashCode to preserve =/hash semantics by
 relying on the equals/hashCode contract -- there is nothing else we
 can do. Well, strictly speaking we could apply an arbitrary function
 to hashCode, so murmuring it (as with Strings) is an option; that
 would come at a performance cost, however (I benchmarked a version
 that did that), and I don't think there'd be much benefit -- certainly
 none at all for classes using identity hash codes.

 Cheers,
 Michał


 On 13 May 2014 14:20, Michał Marczyk michal.marc...@gmail.com wrote:
 It's not just to facilitate inlining, but also to limit the perf hit
 for hashing non-collections, some of which make completely reasonable
 map keys and set members. I've used the classes Alex Miller mentioned
 he was interested in for benchmarking: Class, Character, Var; these
 are all good examples, and only one is under Clojure's control.

 I used a map or map entry or collection test in earlier versions of
 the patch, but switched to the class name hack because the three-way
 type check seems to slow down getting to the default case much more
 than examining the class name. For types that actually implement
 IHashEq, however, the original patch was faster, so I think HotSpot is
 happy to inline either version.


 On 13 May 2014 14:00, Mike Fikes mikefi...@me.com wrote:
 To facilitate inlining, the patch calls out to a separate larger method
 which handles a group of cases.

 +if(o.getClass().getName().startsWith(java.util.))
 +return doalienhasheq(o);
 +return o.hashCode();


 I was wondering whether an efficient improvement is possible that would
 support things like Guava ImmutableList.


 In particular, I was wonder which default cases are currently handled by
 the return o.hashCode() above. Replacing the three lines above with


 +return doalienhasheq(o);


 would allow the patch to also handle non-java.util collection
 implementations, but push the default cases down into the bottom of that
 method.


 On Tuesday, May 13, 2014 12:38:54 AM UTC-4, Michał Marczyk wrote:

 I've posted a patch that makes java.util.{List,Map,Map.Entry,Set}
 hashes consistent with those of appropriate Clojure collection types
 on the ticket.

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.

 For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Expected inconsistency between set and map w/ ArrayList?

[Michał Marczyk]

New patch reinstates the three-way check, but puts it in a separate
static method. Performance seems to be just fine.

I wonder if my previous round of benchmarking with the inline
three-way check was somehow flawed... Either that or the JIT is
happier with the new arrangement for some reason. In any case I kind
of like the separate isAlien.


On 13 May 2014 14:47, Michał Marczyk michal.marc...@gmail.com wrote:
 Unfortunately the Guava example means that this approach doesn't
 really solve the problem at hand... Thanks for pointing it out.

 The earlier version with the three-way test in place of the
 java.util. check should be semantically correct, at least.


 On 13 May 2014 14:40, Michał Marczyk michal.marc...@gmail.com wrote:
 Also, = defers to equals in all cases except two: (1) numeric
 arguments, (2) at least one IPersistentCollection among the arguments.

 Clojure collections are allowed to determine whether they are = to the
 other thing or not. So, we should calculate special hashCodes for the
 java.util collections Clojure collections know about.

 Other collection types will just use equals for comparisons and so for
 them we have to return hashCode to preserve =/hash semantics by
 relying on the equals/hashCode contract -- there is nothing else we
 can do. Well, strictly speaking we could apply an arbitrary function
 to hashCode, so murmuring it (as with Strings) is an option; that
 would come at a performance cost, however (I benchmarked a version
 that did that), and I don't think there'd be much benefit -- certainly
 none at all for classes using identity hash codes.

 Cheers,
 Michał


 On 13 May 2014 14:20, Michał Marczyk michal.marc...@gmail.com wrote:
 It's not just to facilitate inlining, but also to limit the perf hit
 for hashing non-collections, some of which make completely reasonable
 map keys and set members. I've used the classes Alex Miller mentioned
 he was interested in for benchmarking: Class, Character, Var; these
 are all good examples, and only one is under Clojure's control.

 I used a map or map entry or collection test in earlier versions of
 the patch, but switched to the class name hack because the three-way
 type check seems to slow down getting to the default case much more
 than examining the class name. For types that actually implement
 IHashEq, however, the original patch was faster, so I think HotSpot is
 happy to inline either version.


 On 13 May 2014 14:00, Mike Fikes mikefi...@me.com wrote:
 To facilitate inlining, the patch calls out to a separate larger method
 which handles a group of cases.

 +if(o.getClass().getName().startsWith(java.util.))
 +return doalienhasheq(o);
 +return o.hashCode();


 I was wondering whether an efficient improvement is possible that would
 support things like Guava ImmutableList.


 In particular, I was wonder which default cases are currently handled by
 the return o.hashCode() above. Replacing the three lines above with


 +return doalienhasheq(o);


 would allow the patch to also handle non-java.util collection
 implementations, but push the default cases down into the bottom of that
 method.


 On Tuesday, May 13, 2014 12:38:54 AM UTC-4, Michał Marczyk wrote:

 I've posted a patch that makes java.util.{List,Map,Map.Entry,Set}
 hashes consistent with those of appropriate Clojure collection types
 on the ticket.

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with 
 your
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.

 For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Expected inconsistency between set and map w/ ArrayList?

[Michał Marczyk]

I've posted a patch that makes java.util.{List,Map,Map.Entry,Set}
hashes consistent with those of appropriate Clojure collection types
on the ticket.

Performance of repeated hasheq lookups on a single PHM actually seems
to be slightly better with this patch applied. Adding the hasheqs of a
PHM, a string and a long takes slightly longer than on 1.6.0. See the
ticket for details.

The patch also changes default return-the-hashCode behaviour of
hasheq to match Strings. That's mostly because I originally wanted to
leave out the String case altogether, but that seems to hurt String
hashing quite badly. The original branch is up on GitHub, however (see
ticket for link).


On 13 May 2014 04:13, 'Mikera' via Clojure Dev
clojure-...@googlegroups.com wrote:
 On Monday, 12 May 2014 00:33:25 UTC+1, Alex Miller wrote:

 On Sun, May 11, 2014 at 2:06 PM, Mikera mike.r.an...@gmail.com wrote:

 OK. this thread is a bit worrying. If I understand correctly, it
 means that we've now got inconsistent hash and equals functions. I suspect
 this hasn't bitten many people yet simply because it is unusual to mix Java
 and Clojure collections as keys in the same structure, but it seems like a
 really nasty issue.


 Reports of real problems in real codebases would change my own opinion of
 the severity of this issue.


 Are you suggesting correctness issues in APIs aren't severe? This is broken
 according to the documentation. From the docstrings:

 clojure.core/= Same as
   Java x.equals(y) except it also works for nil, and compares
   numbers and collections in a type-independent manner.

 clojure.core/hash Returns the hash code of its argument. Note this is the
 hash code
   consistent with =, and thus is different than .hashCode for Integer,
   Short, Byte and Clojure collections.

 This promises consistency, and also promises behaviour that works like
 x.equals(y) and x.hashCode() - except for a a few defined exceptions. Which
 means I should expect them to work on arbitrary Java objects.

 As a result, all sorts of code that depends on hashes directly or indirectly
 now has subtle bugs. clojure.core/merge is now broken for example:

 (let [a {[1 2] :a}
   b {(java.util.ArrayList. [1 2]) :a}
   m (zipmap (range 20) (range 20))]
   [(= a b)   (= (merge m a) (merge m b))])
 = [true false]

 I hope everyone agrees that this isn't the kind of behaviour we want in core
 functions?



 Hash and equality functions generally have a very simple rule: if two
 things are equal they must have the same hashcode. But now we've got:

 (= (java.util.ArrayList. [1 2]) [1 2])
 = true

 (hash (java.util.ArrayList. [1 2]))
 = 994

 (hash [1 2])
 = 156247261

 I consider this a major defect. If we want to define our own equals and
 hashcode that's fine, but then these functions absolutely should be
 consistent with each other.

 If it is really true that non-Clojure collections aren't considered as
 values and aren't expected to participate in hash/= comparisons then I'd
 expect an exception, not an erroneous value. Silent potential corruption of
 data is *much* worse than a noisy failure. But I think even that is a bad
 direction to go: easy Java ecosystem interop is definitely one of the main
 reasons why I'm using Clojure, and I'd like to see this maintained.


 Please be more specific about where an exception could be thrown (equals
 vs hash vs collection usage).

 I can think of only two sensible ways to fix this inconsistency:
 a) Roll back the hash changes. Correctness is much more important than a
 performance edge case.


 We're not doing that.


 (I suspect the hash changes have hurt average case performance anyway...)


 We have made hash calculation slightly slower to generate better hashes
 across the board. Timing info is at
 http://dev.clojure.org/display/design/Better+hashing but in general, the
 time increases per hash are on the order of nanoseconds for simple values
 and sub-millisecond for more complicated strings and collections (where
 hashes are cached and times are likely to be dwarfed by other things). The
 better hashes avoid catastrophically bad (and relatively easy to find)
 cases.


 It is quite challenging to predict the performance impact of the hashing
 changes in Strings, keywords, and symbols due to the effects of usage
 patterns, interning, caching, etc. I am aware of several people that have
 reported better performance overall in 1.6 and I don't think I've heard any
 reports of significantly slower performance in 1.6. I'm calling FUD on this
 unless you have data to report.


 It's absolutely not FUD. I'm generally a pretty big Clojure evangelist
 and my aim here is simply to be very clear about issues so that we can
 improve Clojure. I hope all my comments are taken in that spirit.

 The page you have linked shows that the hashing changes have slowed down
 various workloads (including the full Clojure compile). The Clojure
 micro-benchmarks posted recently also show various slowdowns related 

Re: Immutable or Effectively Immutable?

[Michał Marczyk]

vec with a short array of objects is a special case in that it simply
puts the existing array in a PersistentVector wrapper:

user= (def arr (object-array 16))
#'user/arr
user= (doseq [i (range 16)] (aset arr i i))
nil
user= (def v (vec arr))
#'user/v
user= v
[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15]
user= (aset arr 0 :foo)
:foo
user= v
[:foo 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15]

This works for up to 32 objects, as that is the number that fits in a PV tail.

No member access syntax here -- not sure if object-array/aset count as interop.

NB. vec's docstring is explicit about this.


On 7 May 2014 02:35, Andy Fingerhut andy.finger...@gmail.com wrote:
 Alex, I may be unfamiliar with the definitions of truly immutable and
 effectively immutable being used here, but if I can mutate the contents of a
 Java Object array that is a final field after an object is constructed, does
 it really matter that much if it is final?  It is trivially easy to mutate
 using Java access.  Here is the example that I mentioned earlier in this
 thread, copied here for convenience:

 user= (def v [1 2 3])
 #'user/v
 user= (class v)
 clojure.lang.PersistentVector
 user= v
 [1 2 3]
 user= (aset (.tail v) 1 -2)
 -2
 user= v
 [1 -2 3]

 Andy


 On Tue, May 6, 2014 at 4:49 PM, Alex Miller a...@puredanger.com wrote:

 The Clojure persistent data structures are truly immutable - all fields
 are final and referred objects are not mutated after construction so that
 freeze occurs.  One obvious exception are the transient variants
 (http://clojure.org/transients). You can look at the code in
 https://github.com/clojure/clojure/tree/master/src/jvm/clojure/lang - any of
 the Persistent*.java.


 On Tuesday, May 6, 2014 4:11:49 PM UTC-5, Mike Fikes wrote:

 Are the persistent immutable data structures in Clojure truly immutable
 (using final fields, relying on constructor freezing), or are they mean to
 be merely effectively immutable (as defined in JICP)?

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] data.avl 0.0.12 -- sorted collections with slice, nth, transient support

[Michał Marczyk]

Here's a round of benchmarks for the new functions
(avl/{nearest,subrange,split-at,split-key}), with a lookup benchmark
included as a point of reference.

Cheers,
Michał


(let [m (apply avl/sorted-map
   (interleave (range 10) (range 10)))]
  (c/bench (get m 9))
  (c/bench (avl/nearest m  10))
  (c/bench (avl/subrange m = 1  75000))
  (c/bench (avl/split-at 3 m))
  (c/bench (avl/split-key 3 m)))
WARNING: Final GC required 4.244287716089524 % of runtime
Evaluation count : 213693000 in 60 samples of 3561550 calls.
 Execution time mean : 278.027014 ns
Execution time std-deviation : 4.284738 ns
   Execution time lower quantile : 273.830485 ns ( 2.5%)
   Execution time upper quantile : 287.103701 ns (97.5%)
   Overhead used : 2.289159 ns

Found 10 outliers in 60 samples (16.6667 %)
low-severe 9 (15. %)
low-mild 1 (1.6667 %)
 Variance from outliers : 1.6389 % Variance is slightly inflated by outliers
Evaluation count : 168881400 in 60 samples of 2814690 calls.
 Execution time mean : 352.393290 ns
Execution time std-deviation : 5.226142 ns
   Execution time lower quantile : 346.698762 ns ( 2.5%)
   Execution time upper quantile : 364.246208 ns (97.5%)
   Overhead used : 2.289159 ns

Found 4 outliers in 60 samples (6.6667 %)
low-severe 4 (6.6667 %)
 Variance from outliers : 1.6389 % Variance is slightly inflated by outliers
Evaluation count : 1333200 in 60 samples of 0 calls.
 Execution time mean : 45.605490 µs
Execution time std-deviation : 682.937868 ns
   Execution time lower quantile : 44.839275 µs ( 2.5%)
   Execution time upper quantile : 47.176367 µs (97.5%)
   Overhead used : 2.289159 ns

Found 3 outliers in 60 samples (5. %)
low-severe 3 (5. %)
 Variance from outliers : 1.6389 % Variance is slightly inflated by outliers
Evaluation count : 2338740 in 60 samples of 38979 calls.
 Execution time mean : 25.715076 µs
Execution time std-deviation : 366.924494 ns
   Execution time lower quantile : 25.249912 µs ( 2.5%)
   Execution time upper quantile : 26.592419 µs (97.5%)
   Overhead used : 2.289159 ns

Found 2 outliers in 60 samples (3. %)
low-severe 2 (3. %)
 Variance from outliers : 1.6389 % Variance is slightly inflated by outliers
Evaluation count : 2596800 in 60 samples of 43280 calls.
 Execution time mean : 23.343989 µs
Execution time std-deviation : 308.669720 ns
   Execution time lower quantile : 22.908707 µs ( 2.5%)
   Execution time upper quantile : 24.143221 µs (97.5%)
   Overhead used : 2.289159 ns

Found 4 outliers in 60 samples (6.6667 %)
low-severe 4 (6.6667 %)
 Variance from outliers : 1.6389 % Variance is slightly inflated by outliers


On 24 April 2014 08:51, Michał Marczyk michal.marc...@gmail.com wrote:
 Cheers Alex!

 See below for the results of yesterday's run of my benchmark suite for
 the basic operations. (I'll post some benchmark results for the new
 functions in a separate message sometime soon.)

 In general, it's fair to expect (persistent) assoc/dissoc to be slower
 with data.avl maps, since AVL trees do more work to rebalance
 themselves than red-black trees. (That's on average, of course.)

 The upside is that lookups in AVL trees are faster on average,
 sometimes quite dramatically, due to the trees being shallower. It's
 worth pointing out that this really is on average -- individual
 timings depend on the path through the tree the lookup operation needs
 to traverse, and for nodes at exactly the same position in the tree
 the built-ins are slightly faster (23.9 ns vs. 25.5 ns for a lookup
 succeeding at the root node, see the (get % 131071) benchmark case
 below; as I recall, 0 happens to occupy the same position in both
 benchmark maps as well) -- but if we hit a deeper part of a red-black
 tree, data.avl is likely to win by a large margin, as in the case of
 the 311 ns (data.avl) vs. 588 ns (built-in) result for looking up
 29 in the two benchmark maps.

 For longer chains of updates -- including construction of instances
 with more than a handful of elements -- data.avl wins thanks to its
 support for transients; in particular, inserting 30 entries in
 ascending order of keys took 197 ns with data.avl's sorted-map during
 my benchmark run yesterday vs. 317 ns with the built-in.


 ;;; the setup:

 (def ks   (range 30))
 (def ksks (interleave ks ks))

 (def rb  (apply sorted-map ksks))
 (def avl (apply avl/sorted-map ksks))

 ;;; the actual benchmark run

 Clojure 1.5.1
 user= (all-benchmarks)
 (lookup-benchmarks)
 ===
 (c/bench (get avl 0))
 WARNING: Final GC required 17.95425214438086 % of runtime
 WARNING: Final GC required 1.44548564181991 % of runtime
 Evaluation count : 233348580 in 60 samples of 3889143 calls.
  Execution time mean : 257.269857 ns
 Execution time std-deviation : 2.083817 ns

Re: [ANN] data.avl 0.0.12 -- sorted collections with slice, nth, transient support

[Michał Marczyk]

 samples (1.6667 %)
low-severe 1 (1.6667 %)
 Variance from outliers : 1.6389 % Variance is slightly inflated by outliers
(c/bench (nth avl 29))
WARNING: Final GC required 1.396166903433299 % of runtime
Evaluation count : 62932200 in 60 samples of 1048870 calls.
 Execution time mean : 966.061999 ns
Execution time std-deviation : 12.227245 ns
   Execution time lower quantile : 950.801928 ns ( 2.5%)
   Execution time upper quantile : 994.606117 ns (97.5%)
   Overhead used : 2.115402 ns

Found 4 outliers in 60 samples (6.6667 %)
low-severe 3 (5. %)
low-mild 1 (1.6667 %)
 Variance from outliers : 1.6389 % Variance is slightly inflated by outliers
(c/bench (nth (seq rb) 29))
WARNING: Final GC required 1.411884727250311 % of runtime
Evaluation count : 3480 in 60 samples of 58 calls.
 Execution time mean : 17.404103 ms
Execution time std-deviation : 125.910185 µs
   Execution time lower quantile : 17.180238 ms ( 2.5%)
   Execution time upper quantile : 17.698525 ms (97.5%)
   Overhead used : 2.115402 ns

Found 2 outliers in 60 samples (3. %)
low-severe 2 (3. %)
 Variance from outliers : 1.6389 % Variance is slightly inflated by outliers
(rank-of-benchmarks)

(c/bench (avl/rank-of avl 0))
WARNING: Final GC required 1.410760546097934 % of runtime
Evaluation count : 228328740 in 60 samples of 3805479 calls.
 Execution time mean : 262.229200 ns
Execution time std-deviation : 1.545106 ns
   Execution time lower quantile : 260.026098 ns ( 2.5%)
   Execution time upper quantile : 265.501717 ns (97.5%)
   Overhead used : 2.115402 ns
(c/bench (rb-rank-of rb 0))
WARNING: Final GC required 1.528903400498107 % of runtime
Evaluation count : 122421060 in 60 samples of 2040351 calls.
 Execution time mean : 489.908995 ns
Execution time std-deviation : 4.341988 ns
   Execution time lower quantile : 483.130264 ns ( 2.5%)
   Execution time upper quantile : 497.383919 ns (97.5%)
   Overhead used : 2.115402 ns

Found 1 outliers in 60 samples (1.6667 %)
low-severe 1 (1.6667 %)
 Variance from outliers : 1.6389 % Variance is slightly inflated by outliers
(c/bench (avl/rank-of avl 131071))
WARNING: Final GC required 1.601437516988401 % of runtime
Evaluation count : 1923909240 in 60 samples of 32065154 calls.
 Execution time mean : 29.024185 ns
Execution time std-deviation : 0.271967 ns
   Execution time lower quantile : 28.621628 ns ( 2.5%)
   Execution time upper quantile : 29.530897 ns (97.5%)
   Overhead used : 2.115402 ns

Found 3 outliers in 60 samples (5. %)
low-severe 2 (3. %)
low-mild 1 (1.6667 %)
 Variance from outliers : 1.6389 % Variance is slightly inflated by outliers
(c/bench (rb-rank-of rb 131071))
WARNING: Final GC required 1.5253557008726901 % of runtime
Evaluation count : 2700 in 60 samples of 45 calls.
 Execution time mean : 22.259014 ms
Execution time std-deviation : 176.571734 µs
   Execution time lower quantile : 21.949530 ms ( 2.5%)
   Execution time upper quantile : 22.639924 ms (97.5%)
   Overhead used : 2.115402 ns
(c/bench (avl/rank-of avl 29))
WARNING: Final GC required 1.46116006270368 % of runtime
Evaluation count : 139283400 in 60 samples of 2321390 calls.
 Execution time mean : 429.526612 ns
Execution time std-deviation : 3.425081 ns
   Execution time lower quantile : 425.422751 ns ( 2.5%)
   Execution time upper quantile : 437.939328 ns (97.5%)
   Overhead used : 2.115402 ns

Found 2 outliers in 60 samples (3. %)
low-severe 2 (3. %)
 Variance from outliers : 1.6389 % Variance is slightly inflated by outliers
(c/bench (rb-rank-of rb 29))
WARNING: Final GC required 1.5002581976759541 % of runtime
Evaluation count : 1200 in 60 samples of 20 calls.
 Execution time mean : 50.728340 ms
Execution time std-deviation : 353.191435 µs
   Execution time lower quantile : 50.130720 ms ( 2.5%)
   Execution time upper quantile : 51.368250 ms (97.5%)
   Overhead used : 2.115402 ns
nil


On 23 April 2014 18:37, Alex Miller a...@puredanger.com wrote:
 Great work as always!

 Are there any places where data.avl performs worse than the built-in sorted
 maps and sets?


 On Wednesday, April 23, 2014 7:59:59 AM UTC-5, Michał Marczyk wrote:

 Hi,

 I am pleased to announce the 0.0.12 release of data.avl, a Clojure
 Contrib library providing drop-in replacements for Clojure(Script)'s
 built-in sorted maps and sets with additional functionality:

  1. real (non-view) logarithmic-time slicing and splits;

  2. logarithmic-time nearest neighbour lookups and rank queries;

  3. support for transients.

 This release is identical to 0.0.12-alpha1 except for the fact that
 the Vars clojure.data.avl/empty-{set,map}-hash{eq,code} are now marked
 private, as they should be.

 Changes since 0.0.11 include an important fix

[ANN] data.avl 0.0.12 -- sorted collections with slice, nth, transient support

[Michał Marczyk]

Hi,

I am pleased to announce the 0.0.12 release of data.avl, a Clojure
Contrib library providing drop-in replacements for Clojure(Script)'s
built-in sorted maps and sets with additional functionality:

 1. real (non-view) logarithmic-time slicing and splits;

 2. logarithmic-time nearest neighbour lookups and rank queries;

 3. support for transients.

This release is identical to 0.0.12-alpha1 except for the fact that
the Vars clojure.data.avl/empty-{set,map}-hash{eq,code} are now marked
private, as they should be.

Changes since 0.0.11 include an important fix to the rebalancing
algorithm, so I would strongly advise all users of 0.0.11 to upgrade
as soon as possible. There are also several new additions to the API,
on which more below.

  [org.clojure/data.avl 0.0.12]

  dependency
groupIdorg.clojure/groupId
artifactIddata.avl/artifactId
version0.0.12/version
  /dependency

  org.clojure:data.avl:0.0.12

The following transcript of a REPL session includes calls to each of
the new public functions:

  (require '[clojure.data.avl :as avl])

  (avl/nearest (avl/sorted-set 0 1 2)  2)
  ;= 1

  (avl/subrange (avl/sorted-set 0 1 2 3 4 5) = 1  4)
  ;= #{1 2 3}

  (avl/split-at 2 (avl/sorted-set 0 1 2 3 4 5))
  ;= [#{0 1} #{2 3 4 5}]

  (avl/split-key 3 (avl/sorted-set 0 2 4 6))
  ;= [#{0 2} nil #{4 6}]

All of these functions operate in time logarithmic in the size of
their inputs. avl/subrange returns a data.avl collection of the same
type as its input and completely independent from it -- that is, it
shares structure with the input where possible, but does not hold on
to any parts of the tree containing keys that should not be present in
the output. avl/split-at and avl/split-key return similarly
independent data.avl collections; the middle element of the vector
returned by split-key is the element at the split key if present in
the input, nil otherwise.

split is the name usually applied in the literature to the operation
that splits a tree into a left tree, a middle element and a right
tree. subrange could also be called slice. The split-* functions
take their collection argument last to be consistent with
clojure.core's split-at and split-with. Arguments accepted by subrange
are analogous to those taken by subseq/rsubseq.

Many thanks to the two new contributors to this release (listed here
in chronological order of ticket creation): Pepijn de Vos, who
prompted me to develop the above-mentioned new features by creating
the ticket asking for what is now avl/nearest and mentioning
java.util.Navigable{Map,Set}, and who provided new tests for
avl/nearest; and Andy Fingerhut, who kept on top of the hashing
changes and provided the initial implementation of new-style hasheq
for data.avl with further tests.

Cheers,
Michał

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Catching exception - lazy eval, lambdas and protocols

[Michał Marczyk]

The problem is that inner-lazy is a method of AProto which is
implemented for ARecord, but actually called (in trigger-fn) on a
vector of two ARecords. The exception thrown is an
IllegalArgumentException complaining about the missing method.

inner-lazy2 works, because it's just a regular function. Note that it
could call the-fn rather than the-fn2 -- the-fn is a method of AProto
just like inner-lazy, but in this snippet it's actually called on
ARecord instances, so the method lookup will be successful.

Cheers,
Michał


On 30 March 2014 14:18, Patrik Sundberg patrik.sundb...@gmail.com wrote:
 I'm trying to work out how to catch exceptions close to source in some code
 with structure similar to this:
 https://www.refheap.com/68851

 A doall on line 13 doesn't change the outcome.

 So not involving records and protocol a similar structure does what I expect
 it to do, but with them I seem unable to catch exceptions taking place
 inside the proto fns.

 So I realize I'm missing something around how exception handlers are setup
 and triggered with regards to either lazy eval, lambdas or protocols (or
 some combo therein).

 I'd like my toy example to calculate 20 as a result of calling (trigger-fn),
 same as it does with (trigger-fn2).

 Any explanations of what's going here would be most welcome - I'd really
 like to understand the exception handling logic.

 Thanks,
 Patrik

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] core.rrb-vector 0.0.11 -- Clojure 1.6 compatibility and bug fixes

[Michał Marczyk]

Hi,

I am pleased to announce the 0.0.11 release of core.rrb-vector, a
Clojure Contrib library extending the Clojure vector API with
logarithmic-time concatenation and slicing:

  https://github.com/clojure/core.rrb-vector

  [org.clojure/core.rrb-vector 0.0.11]

  dependency
groupIdorg.clojure/groupId
artifactIdcore.rrb-vector/artifactId
version${version}/version
  /dependency

  compile org.clojure:core.rrb-vector:${version}

This release updates the hashing mechanism used by RRB vectors for
compatibility with Clojure 1.6 (while maintaining compatibility with
1.5.1) and fixes two bugs. Additionally, the README now states that
view vectors produced by clojure.core/subvec are handled correctly in
Clojure = 1.6.0.

Many thanks to the two new contributors: Jozef Wagner, who reported
and fixed a bug to do with transient vectors of primitives and who
reported a bug in sequences over RRB vectors that would surface when
using such sequences with the reducers library; and Andy Fingerhut,
who kept on top of the hashing changes and contributed improvements to
the test suite, which now explicitly checks both hash codes (hasheq
and hashCode).

Cheers,
Michał

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Clojure 1.6.0-RC1 - LAST CHANCE, PLEASE TEST

[Michał Marczyk]

On 22 March 2014 00:47, Stefan Kamphausen ska2...@gmail.com wrote:
 On Saturday, March 22, 2014 12:41:55 AM UTC+1, Andy Fingerhut wrote:
 That is odd.  This is a shot in the dark, and probably unhelpful because I
 do not know a good way to verify whether my guess is true, but perhaps the
 seqFrom method went from being small enough to be inlined by your JIT before
 that change, to being too large to consider for inlining after the change?
 It isn't a big change in the code, so it would have to have been close to
 the threshold if this is true.

 if there is anything I can do to test this, just let me know.

You could experiment with different values of -XX:MaxInlineSize and
-XX:FreqInlineSize, see

http://www.oracle.com/technetwork/java/javase/tech/vmoptions-jsp-140102.html

You can use -XX:+PrintFlagsFinal to discover what values it's currently using.

Additionally differences in -XX:+PrintCompilation ouput might be interesting.

Cheers,
Michał

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN+RFC] data.avl 0.0.12-alpha1 -- sliceable splittable sorted collections

[Michał Marczyk]

Hi,

I am pleased to announce the 0.0.12-alpha1 release of data.avl, a
Clojure Contrib library providing drop-in replacements for
Clojure(Script)'s built-in sorted maps and sets with additional
functionality including support for real (non-view) logarithmic-time
slicing, splits, nearest neighbour lookups, rank queries and
transients. Performance in comparison to the built-ins is in
expectation better for lookups and worse for non-transient updates;
memory use is higher by a reference and two ints per node.

This release includes an important fix to the rebalancing algorithm
and a much more extensive test suite. I would strongly advise all
users of 0.0.11 to upgrade as soon as possible. The alpha
designation is purely to do with the new additions to the API; see
below for details.

  [org.clojure/data.avl 0.0.12-alpha1]

  dependency
groupIdorg.clojure/groupId
artifactIddata.avl/artifactId
version0.0.12-alpha1/version
  /dependency

  org.clojure:data.avl:0.0.12-alpha1

As mentioned above, this is an alpha release because I would like to
solicit comments on several new additions to the API:

  (require '[clojure.data.avl :as avl])

  (avl/nearest (avl/sorted-set 0 1 2)  2)
  ;= 1

  (avl/subrange (avl/sorted-set 0 1 2 3 4 5) = 1  4)
  ;= #{1 2 3}

  (avl/split-at 2 (avl/sorted-set 0 1 2 3 4 5))
  ;= [#{0 1} #{2 3 4 5}]

  (avl/split-key 3 (avl/sorted-set 0 2 4 6))
  ;= [#{0 2} nil #{4 6}]

All of these functions operate in time logarithmic in the size of
their inputs. avl/subrange returns a data.avl collection of the same
type as its input and completely independent from it -- that is, it
shares structure with the input where possible, but does not hold on
to any parts of the tree containing keys that should not be present in
the output. avl/split-at and avl/split-key return similarly
independent data.avl collections; the middle element of the vector
returned by split-key is the element at the split key if present in
the input, nil otherwise.

split is the name usually applied in the literature to the operation
that splits a tree into a left tree, a middle element and a right
tree. subrange could also be called slice. The split-* functions
take their collection argument last to be consistent with
clojure.core's split-at and split-with. Arguments accepted by subrange
are analogous to those taken by subseq/rsubseq.

I would appreciate any and all comments regarding this API.

Many thanks to the two new contributors to this release (listed here
in chronological order of ticket creation): Pepijn de Vos, who
prompted me to develop the above-mentioned new features by creating
the ticket asking for what is now avl/nearest and mentioning
java.util.Navigable{Map,Set}, and who provided new tests for
avl/nearest; and Andy Fingerhut, who kept on top of the hashing
changes and provided the initial implementation of new-style hasheq
for data.avl with further tests.

Cheers,
Michał

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Clojure 1.6.0-RC1 - LAST CHANCE, PLEASE TEST

[Michał Marczyk]

On 19 March 2014 21:14, Alex Miller a...@puredanger.com wrote:

 Rich just pushed a change to the String hashing to address this. We're
 going to grab the string hashcode (which is cached after first call) and
 murmur the result of that. This gives us constant time hashcode after first
 call with better distribution for combinations in nested collections.  Will
 be in presumed RC2.


If I understand correctly, this means that keywords and symbols use
String.hashCode-based hashing in this commit. They are, however, capable of
caching their Murmur3 hashes, so I was wondering if it might be better to
have them use Murmur3 to hash name  ns?

Rationale: murmuring hashCode results in collisions whenever hashCode does,
plus possibly some additonal collisions if ints collide under murmur;
murmur is better all around.

Potential problems: not sure if things like case depend on symbol/keyword
hashes being directly based on string hashes; hopefully not?

Cheers,
Michał





 On Wednesday, March 19, 2014 11:26:51 AM UTC-5, Stefan Kamphausen wrote:



 On Wednesday, March 19, 2014 4:34:45 PM UTC+1, Alex Miller wrote:

 Thanks, this is all really useful. I would appreciate any more detailed
 info.



 * No atoms, agents, refs
 * Almost purely functional plus logging and I/O
 * Multi-threaded using latest core.async with thread, no go
 * JVisualVM reports top methods in CPU sampling c.l.LazySeq.sval,
 c.c$promise$reify_6310.deref, c.l.LazySeq.seq no notable difference for me
 to see.

 A few experiments on two data-sets (dat1 and dat2) using either a call to
 set or to doall:

Clojure Op Time (dat1) Time (dat1) Average dat1 Time (dat2)  1.5.1 set
 19,1 19 19,05 323  1.5.1 doall 19 19,4 19,2 330  1.6.0-RC1 set 19,5 19,8
 19,65 350  1.6.0-RC1 doall 20,2 19,8 20 351




   --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Clojure 1.6.0-RC1 - LAST CHANCE, PLEASE TEST

[Michał Marczyk]

We're vulnerable to this problem anyway as long as hashing is
deterministic, which is why I think it would be cool to use some
universal-hashing-like scheme...

I think Murmur3 actually uses a seed that could be randomized? Not
really safe in the cryptographic sense of the word, but would make
this sort of attack more challenging.

The way to avoid the problem completely is to use BST-based
dictionaries -- slower, but free from pathological edge cases.

Murmuring a short initial fragment could still be cool, just because
we'd probably get a better hash.

Cheers,
Michał


On 20 March 2014 17:40, Tim McCormack group-cloj...@brainonfire.net wrote:
 On Wednesday, March 19, 2014 4:14:38 PM UTC-4, Alex Miller wrote:
 Rich just pushed a change to the String hashing to address this. We're
 going to grab the string hashcode (which is cached after first call) and
 murmur the result of that. This gives us constant time hashcode after first
 call with better distribution for combinations in nested collections. Will
 be in presumed RC2.

 (Discussion continued from IRC and Github.)

 This does make PHM vulnerable to hashDoS attacks again -- [AaAa 
 AaBB BBAa] will all hash to the same value, so an attacker can pass a
 ton of these colliding strings to a webapp as a querystring or POST body and
 really bog down the machine. Best article I could find on this attack:
 http://cryptanalysis.eu/blog/2011/12/28/effective-dos-attacks-against-web-application-plattforms-hashdos/

 Is there some compromise we can make between caching and good hashing?

 My naïve thought would be to combine the native String hashCode with a
 Murmur hash of a fixed chunk of the string, then possibly run that
 combination through Murmur. This avoids hashing unbounded data more than
 once, and would be effective against basic hashDoS. (Intelligently picking
 the fixed chunk of the string would be essential for protecting against an
 adaptive hashDoS attack.)

  - Tim McCormack

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Clojure 1.6.0-RC1 - LAST CHANCE, PLEASE TEST

[Michał Marczyk]

Hashes are cached for Clojure collections, keywords and symbols, but
not for strings.

I wonder if people who report these perf regressions use long string
keys in their maps... For this to explain the regression they'd have
to be looking up the same String objects a lot, of course (else
hashCode caching wouldn't have been helpful before the move to
Murmur3).


On 19 March 2014 15:21, Alex Miller a...@puredanger.com wrote:
 That is the only set of changes that I know of that negatively affect
 performance in 1.6.0. However, the absolute differences are quite small and
 codes are cached. It's hard for me to conjure a scenario where the resulting
 effect is an overall 5-8% slowdown. If this is one, I'd like to understand
 it better.


 On Wednesday, March 19, 2014 8:28:18 AM UTC-5, Nicola Mometto wrote:


 I'm guessing it's because of the minor overhead on hashing added with
 the move to Murmur3?

 Alex Miller writes:

  record scratch what's that about performance now?
 
  Is that something definitive and reproducible? And if so, is there any
  way to track down a cause?

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: weird bug with cljs.core.async + macros

[Michał Marczyk]

On 18 February 2014 02:14, t x txrev...@gmail.com wrote:
   Does this mean:
   (a) the reported behavior is normal (and my bug report is invalid) or
   (b) this error happens in both cljs + clojure ?

I'd say (b).

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: weird bug with cljs.core.async + macros

[Michał Marczyk]

On 18 February 2014 22:15, t x txrev...@gmail.com wrote:
   I've picked case over cond/condp because I want a O(1) dispatch
 rather than linear execution since there are many patterns [though I'm
 probably guilty of premature optimization here].

Actually in ClojureScript case dispatch is O(n), since it's
implemented as a macro expanding to cond.

Cheers,
Michał

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: weird bug with cljs.core.async + macros

[Michał Marczyk]

On 18 February 2014 22:31, Michał Marczyk michal.marc...@gmail.com wrote:
 Actually in ClojureScript case dispatch is O(n), since it's
 implemented as a macro expanding to cond.

As of release 2156:

https://github.com/clojure/clojurescript/blob/r2156/src/clj/cljs/core.clj#L1119

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: weird bug with cljs.core.async + macros

[Michał Marczyk]

Great! First cut at compiling case to switch (when given numbers and
strings only as tests at the moment, but this can be improved) here:

https://github.com/michalmarczyk/clojurescript/tree/713-compile-case-to-switch

I'll post more details on the ticket.

Cheers,
Michał


On 18 February 2014 22:38, David Nolen dnolen.li...@gmail.com wrote:
 Due to asm.js some JS engines are starting to compile switch statements
 where the cases are integers into jump tables. Compiler enhancement that
 compiles optimizable case expressions to JS switch statements would be a
 welcome enhancement - http://dev.clojure.org/jira/browse/CLJS-713.

 David


 On Tue, Feb 18, 2014 at 4:35 PM, t x txrev...@gmail.com wrote:

 Looking at
 https://github.com/clojure/clojurescript/blob/r2156/src/clj/cljs/core.clj#L1144-L1147
 , you win. :-)

 On Tue, Feb 18, 2014 at 1:33 PM, Michał Marczyk
 michal.marc...@gmail.com wrote:
  On 18 February 2014 22:31, Michał Marczyk michal.marc...@gmail.com
  wrote:
  Actually in ClojureScript case dispatch is O(n), since it's
  implemented as a macro expanding to cond.
 
  As of release 2156:
 
 
  https://github.com/clojure/clojurescript/blob/r2156/src/clj/cljs/core.clj#L1119
 
  --
  You received this message because you are subscribed to the Google
  Groups Clojure group.
  To post to this group, send email to clojure@googlegroups.com
  Note that posts from new members are moderated - please be patient with
  your first post.
  To unsubscribe from this group, send email to
  clojure+unsubscr...@googlegroups.com
  For more options, visit this group at
  http://groups.google.com/group/clojure?hl=en
  ---
  You received this message because you are subscribed to the Google
  Groups Clojure group.
  To unsubscribe from this group and stop receiving emails from it, send
  an email to clojure+unsubscr...@googlegroups.com.
  For more options, visit https://groups.google.com/groups/opt_out.

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.


 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: weird bug with cljs.core.async + macros

[Michał Marczyk]

switch can be compiled to a direct array indexing operation or a
binary search. I believe that's what the JVM does for tableswitch and
lookupswitch, respectively.

On 19 February 2014 01:02, t x txrev...@gmail.com wrote:
 With apologies for potential stupidity:

 * (get {... } key) is an operation that is O(log n) worst case (if
 using Red-Black trees) or O(1) average case (hashing)

 * (get { ... } key) does not have integer/string limitation

 Therefore: why are we not compiling down to get ? Are the big-Oh
 constants enormous?

 On Tue, Feb 18, 2014 at 3:51 PM, Michał Marczyk
 michal.marc...@gmail.com wrote:
 Great! First cut at compiling case to switch (when given numbers and
 strings only as tests at the moment, but this can be improved) here:

 https://github.com/michalmarczyk/clojurescript/tree/713-compile-case-to-switch

 I'll post more details on the ticket.

 Cheers,
 Michał


 On 18 February 2014 22:38, David Nolen dnolen.li...@gmail.com wrote:
 Due to asm.js some JS engines are starting to compile switch statements
 where the cases are integers into jump tables. Compiler enhancement that
 compiles optimizable case expressions to JS switch statements would be a
 welcome enhancement - http://dev.clojure.org/jira/browse/CLJS-713.

 David


 On Tue, Feb 18, 2014 at 4:35 PM, t x txrev...@gmail.com wrote:

 Looking at
 https://github.com/clojure/clojurescript/blob/r2156/src/clj/cljs/core.clj#L1144-L1147
 , you win. :-)

 On Tue, Feb 18, 2014 at 1:33 PM, Michał Marczyk
 michal.marc...@gmail.com wrote:
  On 18 February 2014 22:31, Michał Marczyk michal.marc...@gmail.com
  wrote:
  Actually in ClojureScript case dispatch is O(n), since it's
  implemented as a macro expanding to cond.
 
  As of release 2156:
 
 
  https://github.com/clojure/clojurescript/blob/r2156/src/clj/cljs/core.clj#L1119
 
  --
  You received this message because you are subscribed to the Google
  Groups Clojure group.
  To post to this group, send email to clojure@googlegroups.com
  Note that posts from new members are moderated - please be patient with
  your first post.
  To unsubscribe from this group, send email to
  clojure+unsubscr...@googlegroups.com
  For more options, visit this group at
  http://groups.google.com/group/clojure?hl=en
  ---
  You received this message because you are subscribed to the Google
  Groups Clojure group.
  To unsubscribe from this group and stop receiving emails from it, send
  an email to clojure+unsubscr...@googlegroups.com.
  For more options, visit https://groups.google.com/groups/opt_out.

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.


 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your 
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups 
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your 
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr

Re: weird bug with cljs.core.async + macros

[Michał Marczyk]

The same thing happens in Clojure:

(defmacro silly [object pat1 body1 pat2 body2]
  `(case (:tag ~object)
 ~pat1 ~body1
 ~body2))

(def out (java.io.StringWriter.))

(defn log [ args]
  (doseq [arg args]
(.write out (str arg))
(.write out \n)))

(defn init []
  (silly {:tag :dog}
 :dog (log without-go: woof woof)
 cat (log without-go: unrecognized))
  (async/go
   (silly {:tag :dog}
  :dog (log with-go: woof woof)
  cat (log with-go: unrecognized

(init)

(print (str out))
without-go: woof woof
with-go: woof woof
with-go: unrecognized
nil

Cheers,
Michał


On 18 February 2014 01:01, t x txrev...@gmail.com wrote:
 Thanks for verifying!

 @tbaldridge: can you enlighten us on if:

   * I'm doing something stupid or
   * this is an actual bug in cljs/core.async?

 Thanks!

 On Mon, Feb 17, 2014 at 2:10 PM, Manuel Paccagnella
 manuel.paccagne...@gmail.com wrote:
 Tested on Linux x64, Chromium 31.0.1650.63 and Firefox 26.0. Same behaviour
 that you have seen. However, I haven't looked at the code yet.

 Il giorno lunedì 17 febbraio 2014 20:34:22 UTC+1, t x ha scritto:

 Can anyone verify whether this bug is reproducible?

 On Mon, Feb 17, 2014 at 12:28 AM, t x txre...@gmail.com wrote:
  Hi,
 
repo: https://github.com/txrev319/bug-report
 
I have a really weird bug where:
 
* (my-macro ...) == everything works
 
* (async/go (my-macro ...)) == something weird happens
 
 
A minimal failure case is documented at:
  https://github.com/txrev319/bug-report/blob/master/src/app.cljx
 
It depends on the macro defined in:
  https://github.com/txrev319/bug-report/blob/master/src/macros.cljx
 
The bug can be replicated by following the instructions at:
  https://github.com/txrev319/bug-report/blob/master/README.md
 
I *believe* I have included everything required to replicate the
  bug. If I am missing anything, please let me know.
 
 
  Extra info:
 
  $ lein --version
  Leiningen 2.2.0 on Java 1.7.0_45 Java HotSpot(TM) 64-Bit Server VM
 
  I'm on 64-bit OSX.
 
 
  Thanks!

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your 
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups 
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: weird bug with cljs.core.async + macros

[Michał Marczyk]

Just to be clear, the above is a version of t x's bug-report modified
to use a StringWriter instead of console.log.

Cheers,
Michał


On 18 February 2014 01:25, Michał Marczyk michal.marc...@gmail.com wrote:
 The same thing happens in Clojure:

 (defmacro silly [object pat1 body1 pat2 body2]
   `(case (:tag ~object)
  ~pat1 ~body1
  ~body2))

 (def out (java.io.StringWriter.))

 (defn log [ args]
   (doseq [arg args]
 (.write out (str arg))
 (.write out \n)))

 (defn init []
   (silly {:tag :dog}
  :dog (log without-go: woof woof)
  cat (log without-go: unrecognized))
   (async/go
(silly {:tag :dog}
   :dog (log with-go: woof woof)
   cat (log with-go: unrecognized

 (init)

 (print (str out))
 without-go: woof woof
 with-go: woof woof
 with-go: unrecognized
 nil

 Cheers,
 Michał


 On 18 February 2014 01:01, t x txrev...@gmail.com wrote:
 Thanks for verifying!

 @tbaldridge: can you enlighten us on if:

   * I'm doing something stupid or
   * this is an actual bug in cljs/core.async?

 Thanks!

 On Mon, Feb 17, 2014 at 2:10 PM, Manuel Paccagnella
 manuel.paccagne...@gmail.com wrote:
 Tested on Linux x64, Chromium 31.0.1650.63 and Firefox 26.0. Same behaviour
 that you have seen. However, I haven't looked at the code yet.

 Il giorno lunedì 17 febbraio 2014 20:34:22 UTC+1, t x ha scritto:

 Can anyone verify whether this bug is reproducible?

 On Mon, Feb 17, 2014 at 12:28 AM, t x txre...@gmail.com wrote:
  Hi,
 
repo: https://github.com/txrev319/bug-report
 
I have a really weird bug where:
 
* (my-macro ...) == everything works
 
* (async/go (my-macro ...)) == something weird happens
 
 
A minimal failure case is documented at:
  https://github.com/txrev319/bug-report/blob/master/src/app.cljx
 
It depends on the macro defined in:
  https://github.com/txrev319/bug-report/blob/master/src/macros.cljx
 
The bug can be replicated by following the instructions at:
  https://github.com/txrev319/bug-report/blob/master/README.md
 
I *believe* I have included everything required to replicate the
  bug. If I am missing anything, please let me know.
 
 
  Extra info:
 
  $ lein --version
  Leiningen 2.2.0 on Java 1.7.0_45 Java HotSpot(TM) 64-Bit Server VM
 
  I'm on 64-bit OSX.
 
 
  Thanks!

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your 
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups 
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: rseq for subvec in clojurescript

[Michał Marczyk]

It is now, thanks for the report!

Ticket with patch:

http://dev.clojure.org/jira/browse/CLJS-765


On 16 February 2014 17:48, Sunil S Nandihalli
sunil.nandiha...@gmail.com wrote:
 Hi Everybody,

 I get the following error when I do a rseq on a subvec in clojurescript
 Uncaught Error: No protocol method IReversible.-rseq defined for type
 cljs.core/Subvec: [(:red :clockwise) (:blue :clockwise) (:yellow :clockwise)
 (:yellow :clockwise) (:orange :clockwise)]

 Is this a known problem?
 Thanks, Sunil

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: range-sum

[Michał Marczyk]

str is much faster with apply than with reduce; in fact, with reduce
it is O(n^2), whereas with apply it is O(n). (The constants are better
with apply too.)

In general, (apply foo ...) takes advantage of any optimizations the
author of foo might be aware of for speeding up its operation when
given multiple arguments. (reduce foo ...) is always a simple-minded
fold. Not that that's never useful; it is with reducers and with
anonymous functions.


On 12 February 2014 21:06, Vincent vhenneb...@gmail.com wrote:
 On Wednesday, February 12, 2014 2:47:07 PM UTC, Stuart Sierra wrote:

 On Wednesday, February 12, 2014 8:46:41 AM UTC-5, Vincent wrote:

 On a slightly different topic: why reduce and not apply?


 The implementation of `+` with more than 2 arguments uses `reduce`
 internally. So they amount to the same thing.


 Ah ok, interesting. I do seem to have, in the past, come across a function
 that was significantly faster when used with apply rather than reduce, but I
 can't remember which one.



 There isn't really a performance difference:

 user= (dotimes [i 7] (time (reduce + (range 1000
 Elapsed time: 354.475 msecs
 Elapsed time: 346.235 msecs
 Elapsed time: 348.124 msecs
 Elapsed time: 348.894 msecs
 Elapsed time: 379.9 msecs
 Elapsed time: 356.337 msecs
 Elapsed time: 362.788 msecs
 nil
 user= (dotimes [i 7] (time (apply + (range 1000
 Elapsed time: 360.067 msecs
 Elapsed time: 353.281 msecs
 Elapsed time: 345.694 msecs
 Elapsed time: 355.162 msecs
 Elapsed time: 346.511 msecs
 Elapsed time: 350.61 msecs
 Elapsed time: 353.674 msecs
 nil


 Thanks,
 Vincent

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: map semantics

[Michał Marczyk]

The Contrib library algo.generic provides a function fmap which does
preserve the type of its input.

As for the idea that clojure.core/map should preserve type, it's worth
remembering that in order to map a function over a set and return a
set, we must do two things: (1) apply the function to each element of
the input set, (2) produce a set containing the resulting values. In
general, there is no better way to do this than to pour (map f s) into
a new set, because the shape of the output is for the most part
undetermined by the input (except if the input is empty or is a
*sorted* set of size 1; actually in the latter case there is the
comparator issue, see below).

So, the fundamental operations here are (1) mapping a function over a
sequence / iterator, (2) pouring a sequence of items into a
collection. As it happens, both (1) and (2) are useful in many
different scenarios, separately and together. Thus, they make great
primitives, and Clojure chooses to expose them as such.

Then of course map and filter are lazy and can be used to do things
like (- some-sorted-set (filter p?) (map f) (take n)); take could be
reordered with map, but not with filter, and if it's ok for filter to
convert to seq implicitly, then it is also ok for map.

In the case of sorted sets and maps, there is no guarantee that the
input collection's comparator can deal with outputs from the function
passed to map.

So, these are some of the available conceptual arguments. There is
also a rather convincing practical argument in the form of the
existing body of Clojure code, written using the existing Clojure core
library and its conventions and achieving, in many cases, amazing
levels of clarity and concision.

Cheers,
Michał


On 9 February 2014 06:07, Mars0i marsh...@logical.net wrote:
 Maybe physical identity is too strong of a requirement for equality.  So
 another way to think about it is that it's 'hash-set', 'set', and '#{}' that
 are--you know--broken, but that there's a fix, which is to always use
 'sorted-set'.  (I'm assuming that calling 'seq' on any two sorted sets that
 are = always returns seqs that are =.)

 (Don't take offense at the tone of my remarks.  I think we're all on the
 same side here.)


 On Saturday, February 8, 2014 10:14:37 PM UTC-6, Mars0i wrote:

 Maybe another way to put it is that what is, uh, broken isn't 'map' or
 'seq', but '=', which is willing to tell you that two things (sets) are the
 same when they're not!  We also have the non-broken predicate 'identical?',
 however, that gets it right.  It's nice to also have a set-equal predicate,
 which ignores differences in how sets are stored, and ... that's what '='
 is!  However, if we interpret '=' as implying that when the same function is
 applied to things that are equal in its sense, then we are making a
 mistake: '=' doesn't mean that.  According to this reasoning, nothing here
 is broken, even from an extra-linguistic perspective.  '=' just shouldn't be
 misunderstood.  (In a language with different design and style goals, it
 might been preferable to define = to mean what Clojure means by
 'identical?', and use something else--perhaps equivalent?--for a predicate
 analogous to Clojure's '='.)

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Confused by Clojure floating-point differences (compared to other languages)

[Michał Marczyk]

This returns

(.getTotalPhysicalMemorySize
(java.lang.management.ManagementFactory/getOperatingSystemMXBean))

You could use this in your project.clj, perhaps by including

~(str -Xms (quot (.getTotalPhysicalMemorySize ...) appropriate-number))

in :jvm-opts.

Also, you can absolutely use your own :jvm-opts with :replace.

Cheers,
Michał


PS. getTotalPhysicalMemorySize is declared by
com.sun.management.OperatingSystemMXBean. getOperatingSystemMXBean's
return type is actually java.lang.management.OperatingSystemMXBean,
but the actual returned value does implement the com.sun interface. I
just tested this on a Linux system, hopefully it'll also work on other
platforms.


On 6 February 2014 03:15, Lee Spector lspec...@hampshire.edu wrote:

 On Feb 5, 2014, at 8:50 PM, Bruce Adams wrote:
 Modern JVM's pick default heap sizes based on the physical memory in
 your machine. With more than 1GB of physical memory, initial heap is
 1/64 and maximum heap is 1/4 of physical memory.[1]

 For OpenJDK and Oracle, this command:
java -XX:+PrintFlagsFinal -version | grep HeapSize
 will show the initial and maximum heap sizes (along with a few other
 numbers).

 Thanks Bruce. Do you happen to know if there's a way to specify different 
 fractions? I'd like something more like 3/4 (or 7/8) than 1/4. Nothing else 
 will be running on the machine (aside from the OS and maybe tiny other 
 things), and I want it to take everything it might need. I realize I can 
 hardcode specific sizes, but then I have to change it for every machine 
 configuration it runs on, which is what I was hoping to avoid. I have 64GB on 
 some of my machines, 12GB on others, etc., and I'd like to use most of what's 
 available wherever I run, preferably without changing project.clj every time.

 Also, you may not want to set the initial heap size as large as the
 maximum heap size. Oracle[2] says (in part):

 Setting -Xms and -Xmx to the same value increases predictability by 
 removing the most important sizing decision from the virtual machine. 
 However, the virtual machine is then unable to compensate if you make a 
 poor choice.

 The choice of using the same, maximal limit for both -Xms and -Xmx, could 
 only be poor in the sense of using more than the necessary memory, right? 
 Since I'm happy to use every available byte and am only concerned about speed 
 it seems like this should be okay.

 Thanks,

  -Lee

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your 
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups 
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: how does source work?

[Michał Marczyk]

source-fn is a private Var in the clojure.repl namespace; you can
still use source to display its definition, but you'll have to use the
fully qualified name:

user= (source clojure.repl/source-fn)
(defn source-fn
  Returns a string of the source code for the given symbol, if it can
  find it.  This requires that the symbol resolve to a Var defined in
  a namespace for which the .clj is in the classpath.  Returns nil if
  it can't find the source.  For most REPL usage, 'source' is more
  convenient.

  Example: (source-fn 'filter)
  [x]
  (when-let [v (resolve x)]
(when-let [filepath (:file (meta v))]
  (when-let [strm (.getResourceAsStream (RT/baseLoader) filepath)]
(with-open [rdr (LineNumberReader. (InputStreamReader. strm))]
  (dotimes [_ (dec (:line (meta v)))] (.readLine rdr))
  (let [text (StringBuilder.)
pbr (proxy [PushbackReader] [rdr]
  (read [] (let [i (proxy-super read)]
 (.append text (char i))
 i)))]
(read (PushbackReader. pbr))
(str text)))

Cheers,
Michał


On 4 February 2014 21:34, t x txrev...@gmail.com wrote:
 Hi,

 user (source source)
 (defmacro source
   Prints the source code for the given symbol, if it can find it.
   This requires that the symbol resolve to a Var defined in a
   namespace for which the .clj is in the classpath.

   Example: (source filter)
   [n]
   `(println (or (source-fn '~n) (str Source not found
 nil
 user (source source-fn)
 Source not found
 nil


 Is there a way, from pure-clojure that I can implement my-def +
 my-source (without using def+source) which behaves as def+source does
 ... or does doing so require writing Java code?

 Thanks!

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your 
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups 
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: order of returned values from keys and vals

[Michał Marczyk]

On 2 February 2014 04:54, Ambrose Bonnaire-Sergeant 
abonnaireserge...@gmail.com wrote:

 zipmap could also potentially use transients (which would be a nice
 addition).


My patch for this has been waiting in JIRA since end of May 2012:

http://dev.clojure.org/jira/browse/CLJ-1005

Cheers,
Michał



 keys/vals are also lazy, so I would be surprised if there was any
 performance
 difference with walking the seq twice.

 Thanks,
 Ambrose


 On Sun, Feb 2, 2014 at 11:35 AM, Justin Smith noisesm...@gmail.comwrote:

 Realistically, how many situations are there where running keys and vals
 independently is preferable to running seq once and using the two element
 vectors that returns?

 Usually this way one can avoid walking the whole thing twice.

 (into {} (map (fn [[k v]] [k (inc v)]) {:a 0 :b 1})) is representative of
 the idiom I usually use. As a bonus, into uses transients, which can create
 the resulting structure in fewer cycles / less time.


 On Saturday, February 1, 2014 10:00:33 AM UTC-8, Sam Ritchie wrote:

 Looks like Rich just chimed in with:

 keys order == vals order == seq order 

   Matching Socks
  January 31, 2014 7:31 PM
 Actually, http://dev.clojure.org/jira/browse/CLJ-1302 keys and vals
 consistency not mentioned in docstring was declined, with the comment The
 absence of this property in the docs is correct. You should not rely on
 this.



 On Wednesday, August 11, 2010 6:03:39 PM UTC-4, Chouser wrote:
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com

 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com

 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to clojure+u...@googlegroups.com.

 For more options, visit https://groups.google.com/groups/opt_out.


 --
 Sam Ritchie (@sritchie)
  Paddleguru Co-Founder
 703.863.8561
 www.paddleguru.com
 Twitter http://twitter.com/paddleguru // 
 Facebookhttp://facebook.com/paddleguru

  --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.


  --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: order of returned values from keys and vals

[Michał Marczyk]

On 2 February 2014 05:14, Justin Smith noisesm...@gmail.com wrote:

 Pardon my ignorance but if this is producing a lazy result, how is it
 doing so?

 https://github.com/clojure/clojure/blob/dff9600387b962f16fc78e6477e10e34651fd366/src/jvm/clojure/lang/APersistentMap.java#L134


APersistentMap's KeySeq.create and ValSeq.create are pretty much equivalent
to (map key entry-seq) and (map val entry-seq). The logic producing
entry-seq lives in the individual map types and is indeed lazy.

Cheers,
Michał



 On Saturday, February 1, 2014 7:54:32 PM UTC-8, Ambrose Bonnaire-Sergeant
 wrote:

 zipmap could also potentially use transients (which would be a nice
 addition).

 keys/vals are also lazy, so I would be surprised if there was any
 performance
 difference with walking the seq twice.

 Thanks,
 Ambrose


 On Sun, Feb 2, 2014 at 11:35 AM, Justin Smith noise...@gmail.com wrote:

 Realistically, how many situations are there where running keys and vals
 independently is preferable to running seq once and using the two element
 vectors that returns?

 Usually this way one can avoid walking the whole thing twice.

 (into {} (map (fn [[k v]] [k (inc v)]) {:a 0 :b 1})) is representative
 of the idiom I usually use. As a bonus, into uses transients, which can
 create the resulting structure in fewer cycles / less time.


 On Saturday, February 1, 2014 10:00:33 AM UTC-8, Sam Ritchie wrote:

 Looks like Rich just chimed in with:

 keys order == vals order == seq order 

   Matching Socks
  January 31, 2014 7:31 PM
 Actually, http://dev.clojure.org/jira/browse/CLJ-1302 keys and vals
 consistency not mentioned in docstring was declined, with the comment The
 absence of this property in the docs is correct. You should not rely on
 this.



 On Wednesday, August 11, 2010 6:03:39 PM UTC-4, Chouser wrote:
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com

 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com

 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to clojure+u...@googlegroups.com.

 For more options, visit https://groups.google.com/groups/opt_out.


 --
 Sam Ritchie (@sritchie)
  Paddleguru Co-Founder
 703.863.8561
 www.paddleguru.com
 Twitter http://twitter.com/paddleguru // 
 Facebookhttp://facebook.com/paddleguru

  --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to clojure+u...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.


  --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: order of returned values from keys and vals

[Michał Marczyk]

That doesn't mean that we're not walking the seq twice, however. keys and
vals work off of separate calls to seq on the map, which of necessity
produce distinct seq objects (otherwise traversing a seq over a map would
permanently increase its memory consumption).


On 2 February 2014 05:18, Michał Marczyk michal.marc...@gmail.com wrote:

 On 2 February 2014 05:14, Justin Smith noisesm...@gmail.com wrote:

 Pardon my ignorance but if this is producing a lazy result, how is it
 doing so?

 https://github.com/clojure/clojure/blob/dff9600387b962f16fc78e6477e10e34651fd366/src/jvm/clojure/lang/APersistentMap.java#L134


 APersistentMap's KeySeq.create and ValSeq.create are pretty much
 equivalent to (map key entry-seq) and (map val entry-seq). The logic
 producing entry-seq lives in the individual map types and is indeed lazy.

 Cheers,
 Michał



 On Saturday, February 1, 2014 7:54:32 PM UTC-8, Ambrose Bonnaire-Sergeant
 wrote:

 zipmap could also potentially use transients (which would be a nice
 addition).

 keys/vals are also lazy, so I would be surprised if there was any
 performance
 difference with walking the seq twice.

 Thanks,
 Ambrose


 On Sun, Feb 2, 2014 at 11:35 AM, Justin Smith noise...@gmail.comwrote:

 Realistically, how many situations are there where running keys and
 vals independently is preferable to running seq once and using the two
 element vectors that returns?

 Usually this way one can avoid walking the whole thing twice.

 (into {} (map (fn [[k v]] [k (inc v)]) {:a 0 :b 1})) is representative
 of the idiom I usually use. As a bonus, into uses transients, which can
 create the resulting structure in fewer cycles / less time.


 On Saturday, February 1, 2014 10:00:33 AM UTC-8, Sam Ritchie wrote:

 Looks like Rich just chimed in with:

 keys order == vals order == seq order 

   Matching Socks
  January 31, 2014 7:31 PM
 Actually, http://dev.clojure.org/jira/browse/CLJ-1302 keys and vals
 consistency not mentioned in docstring was declined, with the comment 
 The
 absence of this property in the docs is correct. You should not rely on
 this.



 On Wednesday, August 11, 2010 6:03:39 PM UTC-4, Chouser wrote:
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com

 Note that posts from new members are moderated - please be patient
 with your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com

 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to clojure+u...@googlegroups.com.

 For more options, visit https://groups.google.com/groups/opt_out.


 --
 Sam Ritchie (@sritchie)
  Paddleguru Co-Founder
 703.863.8561
 www.paddleguru.com
 Twitter http://twitter.com/paddleguru // 
 Facebookhttp://facebook.com/paddleguru

  --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to clojure+u...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.


  --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.




-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails

Re: order of returned values from keys and vals

[Michał Marczyk]

I'd expect

(persistent!
  (reduce-kv (fn [acc k v] (assoc! acc k (inc v)))
(transient {})
input-map))

to be the fastest solution.

Cheers,
Michał


On 2 February 2014 05:30, Justin Smith noisesm...@gmail.com wrote:

 Excellent, thanks for the information.

 I just did a benchmark of zipmap over keys and vals vs. into {} over a
 map, and despite my intuition, they are actually nearly identical in
 performance. So if zipmap were using transients zipmap with keys and vals
 would actually be the better performing option here.

 user (def testmap (zipmap (range 1000) (range 2000 3000)))
 user (crit/bench (zipmap (keys testmap) (map inc (vals testmap
 Evaluation count : 150900 in 60 samples of 2515 calls.
  Execution time mean : 401.458681 µs
 Execution time std-deviation : 4.912358 µs
Execution time lower quantile : 393.882254 µs ( 2.5%)
Execution time upper quantile : 409.534676 µs (97.5%)
Overhead used : 2.203076 ns

 Found 1 outliers in 60 samples (1.6667 %)
 low-severe 1 (1.6667 %)
  Variance from outliers : 1.6389 % Variance is slightly inflated by
 outliers
 nil
 user (crit/bench (into {} (map (fn [[k v]] [k (inc v)]) testmap)))
 Evaluation count : 150060 in 60 samples of 2501 calls.
  Execution time mean : 400.684810 µs
 Execution time std-deviation : 3.489015 µs
Execution time lower quantile : 395.127605 µs ( 2.5%)
Execution time upper quantile : 407.132405 µs (97.5%)
Overhead used : 2.203076 ns
 nil


 On Saturday, February 1, 2014 8:22:28 PM UTC-8, Ambrose Bonnaire-Sergeant
 wrote:

 It also might help to notice KeySeq defines first/next, but never
 explicitly walks the seq.

 That's left to the user of the KeySeq.

 Thanks,
 Ambrose


 On Sun, Feb 2, 2014 at 12:18 PM, Michał Marczyk michal@gmail.comwrote:

 On 2 February 2014 05:14, Justin Smith noise...@gmail.com wrote:

 Pardon my ignorance but if this is producing a lazy result, how is it
 doing so?
 https://github.com/clojure/clojure/blob/dff9600387b962f16fc78e6477e10e
 34651fd366/src/jvm/clojure/lang/APersistentMap.java#L134


 APersistentMap's KeySeq.create and ValSeq.create are pretty much
 equivalent to (map key entry-seq) and (map val entry-seq). The logic
 producing entry-seq lives in the individual map types and is indeed lazy.

 Cheers,
 Michał



 On Saturday, February 1, 2014 7:54:32 PM UTC-8, Ambrose
 Bonnaire-Sergeant wrote:

 zipmap could also potentially use transients (which would be a nice
 addition).

 keys/vals are also lazy, so I would be surprised if there was any
 performance
 difference with walking the seq twice.

 Thanks,
 Ambrose


 On Sun, Feb 2, 2014 at 11:35 AM, Justin Smith noise...@gmail.comwrote:

 Realistically, how many situations are there where running keys and
 vals independently is preferable to running seq once and using the two
 element vectors that returns?

 Usually this way one can avoid walking the whole thing twice.

 (into {} (map (fn [[k v]] [k (inc v)]) {:a 0 :b 1})) is
 representative of the idiom I usually use. As a bonus, into uses
 transients, which can create the resulting structure in fewer cycles / 
 less
 time.


 On Saturday, February 1, 2014 10:00:33 AM UTC-8, Sam Ritchie wrote:

 Looks like Rich just chimed in with:

 keys order == vals order == seq order 

   Matching Socks
  January 31, 2014 7:31 PM
 Actually, http://dev.clojure.org/jira/browse/CLJ-1302 keys and
 vals consistency not mentioned in docstring was declined, with the 
 comment
 The absence of this property in the docs is correct. You should not 
 rely
 on this.



 On Wednesday, August 11, 2010 6:03:39 PM UTC-4, Chouser wrote:
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com

 Note that posts from new members are moderated - please be patient
 with your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com

 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To unsubscribe from this group and stop receiving emails from it,
 send an email to clojure+u...@googlegroups.com.

 For more options, visit https://groups.google.com/groups/opt_out.


 --
 Sam Ritchie (@sritchie)
  Paddleguru Co-Founder
 703.863.8561
 www.paddleguru.com
 Twitter http://twitter.com/paddleguru // 
 Facebookhttp://facebook.com/paddleguru

  --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com
 Note that posts from new members are moderated - please be patient
 with your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because

Re: order of returned values from keys and vals

[Michał Marczyk]

Oh, sorry, for some reason Gmail didn't alert me to some of the recent
messages coming in while I was viewing this thread.


On 2 February 2014 05:32, Michał Marczyk michal.marc...@gmail.com wrote:

 I'd expect

 (persistent!
   (reduce-kv (fn [acc k v] (assoc! acc k (inc v)))
 (transient {})
 input-map))

 to be the fastest solution.

 Cheers,
 Michał


 On 2 February 2014 05:30, Justin Smith noisesm...@gmail.com wrote:

 Excellent, thanks for the information.

 I just did a benchmark of zipmap over keys and vals vs. into {} over a
 map, and despite my intuition, they are actually nearly identical in
 performance. So if zipmap were using transients zipmap with keys and vals
 would actually be the better performing option here.

 user (def testmap (zipmap (range 1000) (range 2000 3000)))
 user (crit/bench (zipmap (keys testmap) (map inc (vals testmap
 Evaluation count : 150900 in 60 samples of 2515 calls.
  Execution time mean : 401.458681 µs
 Execution time std-deviation : 4.912358 µs
Execution time lower quantile : 393.882254 µs ( 2.5%)
Execution time upper quantile : 409.534676 µs (97.5%)
Overhead used : 2.203076 ns

 Found 1 outliers in 60 samples (1.6667 %)
 low-severe 1 (1.6667 %)
  Variance from outliers : 1.6389 % Variance is slightly inflated by
 outliers
 nil
 user (crit/bench (into {} (map (fn [[k v]] [k (inc v)]) testmap)))
 Evaluation count : 150060 in 60 samples of 2501 calls.
  Execution time mean : 400.684810 µs
 Execution time std-deviation : 3.489015 µs
Execution time lower quantile : 395.127605 µs ( 2.5%)
Execution time upper quantile : 407.132405 µs (97.5%)
Overhead used : 2.203076 ns
 nil


 On Saturday, February 1, 2014 8:22:28 PM UTC-8, Ambrose Bonnaire-Sergeant
 wrote:

 It also might help to notice KeySeq defines first/next, but never
 explicitly walks the seq.

 That's left to the user of the KeySeq.

 Thanks,
 Ambrose


 On Sun, Feb 2, 2014 at 12:18 PM, Michał Marczyk michal@gmail.comwrote:

 On 2 February 2014 05:14, Justin Smith noise...@gmail.com wrote:

 Pardon my ignorance but if this is producing a lazy result, how is it
 doing so?
 https://github.com/clojure/clojure/blob/dff9600387b962f16fc78e6477e10e
 34651fd366/src/jvm/clojure/lang/APersistentMap.java#L134


 APersistentMap's KeySeq.create and ValSeq.create are pretty much
 equivalent to (map key entry-seq) and (map val entry-seq). The logic
 producing entry-seq lives in the individual map types and is indeed lazy.

 Cheers,
 Michał



 On Saturday, February 1, 2014 7:54:32 PM UTC-8, Ambrose
 Bonnaire-Sergeant wrote:

 zipmap could also potentially use transients (which would be a nice
 addition).

 keys/vals are also lazy, so I would be surprised if there was any
 performance
 difference with walking the seq twice.

 Thanks,
 Ambrose


 On Sun, Feb 2, 2014 at 11:35 AM, Justin Smith noise...@gmail.comwrote:

 Realistically, how many situations are there where running keys and
 vals independently is preferable to running seq once and using the two
 element vectors that returns?

 Usually this way one can avoid walking the whole thing twice.

 (into {} (map (fn [[k v]] [k (inc v)]) {:a 0 :b 1})) is
 representative of the idiom I usually use. As a bonus, into uses
 transients, which can create the resulting structure in fewer cycles / 
 less
 time.


 On Saturday, February 1, 2014 10:00:33 AM UTC-8, Sam Ritchie wrote:

 Looks like Rich just chimed in with:

 keys order == vals order == seq order 

   Matching Socks
  January 31, 2014 7:31 PM
 Actually, http://dev.clojure.org/jira/browse/CLJ-1302 keys and
 vals consistency not mentioned in docstring was declined, with the 
 comment
 The absence of this property in the docs is correct. You should not 
 rely
 on this.



 On Wednesday, August 11, 2010 6:03:39 PM UTC-4, Chouser wrote:
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com

 Note that posts from new members are moderated - please be patient
 with your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com

 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To unsubscribe from this group and stop receiving emails from it,
 send an email to clojure+u...@googlegroups.com.

 For more options, visit https://groups.google.com/groups/opt_out.


 --
 Sam Ritchie (@sritchie)
  Paddleguru Co-Founder
 703.863.8561
 www.paddleguru.com
 Twitter http://twitter.com/paddleguru // 
 Facebookhttp://facebook.com/paddleguru

  --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com
 Note that posts from new members are moderated - please be patient
 with your first

Re: core.async over websocket + cljs + clojure

[Michał Marczyk]

If the application is structured in such a way that messages from the
other side of the client/server divide are put on channels upon
arrival, which seems to be a reasonable idea, why not encapsulate the
connection-handling logic in a black box exposing a (bunch of)
channel(s)?

On 26 January 2014 00:01, Patrick Logan patrickdlo...@gmail.com wrote:
 This seems like more trouble than it is worth. There are almost certainly 
 suitable but more established protocols and implementations for the problem 
 at hand. Anyway, maybe it's worth exploring. To me it seems to muddy the 
 waters for what core.async seems intended to provide, which seems to me to be 
 fairly straight-forward CSP.

 --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your 
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups 
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.

-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: avoiding repetition in ns declarations

[Michał Marczyk]

Not addressing the main problem, but require takes multiple libspecs,
so you can write

(ns foo
  (:require [foo.bar :as bar]
  [foo.quux :as quux]))

to avoid repeating the keyword, at least. :require in ns expands to a
call to require-the-function, so the latter supports multiple libspecs
too (and so require-standard could be written more succinctly).

Cheers,
Michał


On 22 January 2014 08:15, t x txrev...@gmail.com wrote:
 Hi,

   I have the following problem:

 (ns foo.bar
   ...
   ...
   ... )

 There's basically 10 lines of require that I want as part of nearly _every_
 ns I declare is there a way to define some soft of alias / abbrevraviation
 that is used in namespaces at will?

 For example:

 ## somewhere, in a magic file:

 (def require-standard
   (require [ ... :as ... ])
   (require [ ... :as ... ])
   (require [ ... :as ... ])
   (require [ ... :as ... ]))

 (ns foo.bar
   require-standard
   ...)

 Thanks!




 --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.

-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: closing-buffer

[Michał Marczyk]

Ah, of course, the mutex is not reentrant.

So, one possible way to fix the situation would be to close the owned
channel manually by resetting its closed atom to true. Thinking
through all the implications should make for an interesting exercise.
:-) (For example, I think cleanup is performed correctly anyway, but I
haven't put in the time to make sure properly. Clearly this relies on
the implementation details of channels, so even if it is in fact not
broken with the current alpha, it might be in the future, and so it
might still be a better idea to implement a new channel. And so on.)

Here's the implementation, see below for an example from the REPL:

(defn closing-buffer [n]
  (let [b (async/buffer n)
c (clojure.lang.Box. nil)]
(reify
  clojure.core.async.impl.protocols/Buffer
  (full? [this]
false)
  (remove! [this]
(clojure.core.async.impl.protocols/remove! b))
  (add! [this itm]
(if (clojure.core.async.impl.protocols/full? b)
  (do
(reset! (.-closed
^clojure.core.async.impl.channels.ManyToManyChannel (.-val c)) true)
nil)
  (clojure.core.async.impl.protocols/add! b itm)))
  clojure.core.async.impl.protocols/UnblockingBuffer
  clojure.lang.Counted
  (count [this]
(count b))
  PBufferThatMightCloseAChannel
  (this-is-your-channel! [this given-c]
(if (nil? (.-val c))
  (do
(set! (.-val c) given-c)
true)
  false)

(defn closing-channel [n]
  (let [b (closing-buffer n)
c (async/chan b)]
(this-is-your-channel! b c)
c))

From the REPL:

user (def test-chan (closing-channel 2))
#'user/test-chan
user (async/!! test-chan :foo)
nil
user (async/!! test-chan :foo)
nil
user (async/!! test-chan :foo)
nil
user (async/!! test-chan)
:foo
user (async/!! test-chan)
:foo
user (async/!! test-chan)
nil
user (async/!! test-chan :foo)
nil
user (async/!! test-chan)
nil

Cheers,
Michał


On 14 January 2014 07:41, t x txrev...@gmail.com wrote:
 If I understand your strategy correctly, it works as follows:

 * write new buffer class
 * extend buffer class with a field/atom that stores the channel that owns
 the buffer

 * on overflow, call (async/close!) on the owned channel

 However, looking at :

 *
 https://github.com/clojure/core.async/blob/master/src/main/clojure/clojure/core/async/impl/channels.clj#L179
 *
 https://github.com/clojure/core.async/blob/master/src/main/clojure/clojure/core/async/impl/channels.clj#L55

 isn't this a deadlock by acquiring the mutex while holding the mutex?


 On Mon, Jan 13, 2014 at 10:27 PM, Michał Marczyk michal.marc...@gmail.com
 wrote:

 Instead of actually implementing a new channel type, you could use the
 regular MTMCs, but with your own factory function, custom buffers and
 a sprinkle of mutability:

 (defprotocol PBufferThatMightCloseAChannel
   (this-is-your-channel! [this c]
 Informs the buffer that c is the channel it might need to close))

 ;; implement a buffer with an impl for the above protocol;
 ;; have it close the appropriate channel on overflow;
 ;; suppose closing-buffer is the factory function
 ;; producing such channels

 (defn closing-channel [n]
   (let [b (closing-buffer n)
 c (chan b)]
 (this-is-your-channel! b c)
 c))

 Cheers,
 Michał


 On 14 January 2014 05:47, t x txrev...@gmail.com wrote:
  I am aware of:
 
DroppingBuffer and SliddingBuffer
 
  I would like to build a channel with different semantics:
 
When I try to put an item on a full channel:
 
* DroppingBuffer drops the new item
* SliddingBuffer evicts the oldest item
 
* ClosingBuffer should _close the channel_
 
  (thus, the receiver eventually gets a nil, and realizes ah, was
  disconnected)
 
  Now, I am looking at the github core.async code:
 
  DroppingBuffer:
 
  https://github.com/clojure/core.async/blob/76317035d386ce2a1d98c2c349da9b898b480c55/src/main/clojure/clojure/core/async/impl/buffers.clj#L33-L45
 
  ManyToManyChannel:
 
  https://github.com/clojure/core.async/blob/76317035d386ce2a1d98c2c349da9b898b480c55/src/main/clojure/clojure/core/async/impl/channels.clj#L31-L198
 
  ## Problem:
 
  DroppingBuffer is easy to understand. However, I don't think I can
  implement
  ClosingBuffer as Buffer.
 
  I feel taht ClosingBuffer can only be implemneted at the Channel
  Level.
 
  However, the Channel Code is complicated.
 
  Is there a neat hack / trick to do what I described above?
 
  Thanks!
 
  --
  --
  You received this message because you are subscribed to the Google
  Groups Clojure group.
  To post to this group, send email to clojure@googlegroups.com
  Note that posts from new members are moderated - please be patient with
  your
  first post.
  To unsubscribe from this group, send email to
  clojure+unsubscr...@googlegroups.com
  For more options, visit this group at
  http://groups.google.com/group/clojure?hl=en
  ---
  You received this message because you

Re: closing-buffer

[Michał Marczyk]

A sketch of a different approach:

alt!  / alts! on the main channel and an extra channel with :priority
true when putting. Use an unblocking buffer in the extra channel. If
you end up putting on the extra channel, close the main channel.
(core.async channels can be closed multiple times.)

Here a consumer could take something off the channel before we close,
making it possible to a different producer to put something on the
channel possibly before we close, meaning that the channel doesn't,
strictly speaking, get closed as soon as an overflowing put arrives.
Of course if the timing is that close, the end result is no different
to what would happen had we arrived at a slightly later time.


On 14 January 2014 09:37, Michał Marczyk michal.marc...@gmail.com wrote:
 Ah, of course, the mutex is not reentrant.

 So, one possible way to fix the situation would be to close the owned
 channel manually by resetting its closed atom to true. Thinking
 through all the implications should make for an interesting exercise.
 :-) (For example, I think cleanup is performed correctly anyway, but I
 haven't put in the time to make sure properly. Clearly this relies on
 the implementation details of channels, so even if it is in fact not
 broken with the current alpha, it might be in the future, and so it
 might still be a better idea to implement a new channel. And so on.)

 Here's the implementation, see below for an example from the REPL:

 (defn closing-buffer [n]
   (let [b (async/buffer n)
 c (clojure.lang.Box. nil)]
 (reify
   clojure.core.async.impl.protocols/Buffer
   (full? [this]
 false)
   (remove! [this]
 (clojure.core.async.impl.protocols/remove! b))
   (add! [this itm]
 (if (clojure.core.async.impl.protocols/full? b)
   (do
 (reset! (.-closed
 ^clojure.core.async.impl.channels.ManyToManyChannel (.-val c)) true)
 nil)
   (clojure.core.async.impl.protocols/add! b itm)))
   clojure.core.async.impl.protocols/UnblockingBuffer
   clojure.lang.Counted
   (count [this]
 (count b))
   PBufferThatMightCloseAChannel
   (this-is-your-channel! [this given-c]
 (if (nil? (.-val c))
   (do
 (set! (.-val c) given-c)
 true)
   false)

 (defn closing-channel [n]
   (let [b (closing-buffer n)
 c (async/chan b)]
 (this-is-your-channel! b c)
 c))

 From the REPL:

 user (def test-chan (closing-channel 2))
 #'user/test-chan
 user (async/!! test-chan :foo)
 nil
 user (async/!! test-chan :foo)
 nil
 user (async/!! test-chan :foo)
 nil
 user (async/!! test-chan)
 :foo
 user (async/!! test-chan)
 :foo
 user (async/!! test-chan)
 nil
 user (async/!! test-chan :foo)
 nil
 user (async/!! test-chan)
 nil

 Cheers,
 Michał


 On 14 January 2014 07:41, t x txrev...@gmail.com wrote:
 If I understand your strategy correctly, it works as follows:

 * write new buffer class
 * extend buffer class with a field/atom that stores the channel that owns
 the buffer

 * on overflow, call (async/close!) on the owned channel

 However, looking at :

 *
 https://github.com/clojure/core.async/blob/master/src/main/clojure/clojure/core/async/impl/channels.clj#L179
 *
 https://github.com/clojure/core.async/blob/master/src/main/clojure/clojure/core/async/impl/channels.clj#L55

 isn't this a deadlock by acquiring the mutex while holding the mutex?


 On Mon, Jan 13, 2014 at 10:27 PM, Michał Marczyk michal.marc...@gmail.com
 wrote:

 Instead of actually implementing a new channel type, you could use the
 regular MTMCs, but with your own factory function, custom buffers and
 a sprinkle of mutability:

 (defprotocol PBufferThatMightCloseAChannel
   (this-is-your-channel! [this c]
 Informs the buffer that c is the channel it might need to close))

 ;; implement a buffer with an impl for the above protocol;
 ;; have it close the appropriate channel on overflow;
 ;; suppose closing-buffer is the factory function
 ;; producing such channels

 (defn closing-channel [n]
   (let [b (closing-buffer n)
 c (chan b)]
 (this-is-your-channel! b c)
 c))

 Cheers,
 Michał


 On 14 January 2014 05:47, t x txrev...@gmail.com wrote:
  I am aware of:
 
DroppingBuffer and SliddingBuffer
 
  I would like to build a channel with different semantics:
 
When I try to put an item on a full channel:
 
* DroppingBuffer drops the new item
* SliddingBuffer evicts the oldest item
 
* ClosingBuffer should _close the channel_
 
  (thus, the receiver eventually gets a nil, and realizes ah, was
  disconnected)
 
  Now, I am looking at the github core.async code:
 
  DroppingBuffer:
 
  https://github.com/clojure/core.async/blob/76317035d386ce2a1d98c2c349da9b898b480c55/src/main/clojure/clojure/core/async/impl/buffers.clj#L33-L45
 
  ManyToManyChannel:
 
  https://github.com/clojure/core.async/blob/76317035d386ce2a1d98c2c349da9b898b480c55/src/main/clojure/clojure/core/async

Re: closing-buffer

[Michał Marczyk]

Just to avoid any possibility of giving the impression that I'm
offering the above as a well-tested component: I wouldn't actually use
closing-channel as defined above without giving much more thought to
all implications. The current version is just a quick sketch.

Also, if the alt! / alts! approach works satisfactorily, or if another
approach based on the available public primitives does, I'd definitely
avoid implementing new funky stuff.

Fun problem to think about, though.


On 14 January 2014 09:47, Michał Marczyk michal.marc...@gmail.com wrote:
 A sketch of a different approach:

 alt!  / alts! on the main channel and an extra channel with :priority
 true when putting. Use an unblocking buffer in the extra channel. If
 you end up putting on the extra channel, close the main channel.
 (core.async channels can be closed multiple times.)

 Here a consumer could take something off the channel before we close,
 making it possible to a different producer to put something on the
 channel possibly before we close, meaning that the channel doesn't,
 strictly speaking, get closed as soon as an overflowing put arrives.
 Of course if the timing is that close, the end result is no different
 to what would happen had we arrived at a slightly later time.


 On 14 January 2014 09:37, Michał Marczyk michal.marc...@gmail.com wrote:
 Ah, of course, the mutex is not reentrant.

 So, one possible way to fix the situation would be to close the owned
 channel manually by resetting its closed atom to true. Thinking
 through all the implications should make for an interesting exercise.
 :-) (For example, I think cleanup is performed correctly anyway, but I
 haven't put in the time to make sure properly. Clearly this relies on
 the implementation details of channels, so even if it is in fact not
 broken with the current alpha, it might be in the future, and so it
 might still be a better idea to implement a new channel. And so on.)

 Here's the implementation, see below for an example from the REPL:

 (defn closing-buffer [n]
   (let [b (async/buffer n)
 c (clojure.lang.Box. nil)]
 (reify
   clojure.core.async.impl.protocols/Buffer
   (full? [this]
 false)
   (remove! [this]
 (clojure.core.async.impl.protocols/remove! b))
   (add! [this itm]
 (if (clojure.core.async.impl.protocols/full? b)
   (do
 (reset! (.-closed
 ^clojure.core.async.impl.channels.ManyToManyChannel (.-val c)) true)
 nil)
   (clojure.core.async.impl.protocols/add! b itm)))
   clojure.core.async.impl.protocols/UnblockingBuffer
   clojure.lang.Counted
   (count [this]
 (count b))
   PBufferThatMightCloseAChannel
   (this-is-your-channel! [this given-c]
 (if (nil? (.-val c))
   (do
 (set! (.-val c) given-c)
 true)
   false)

 (defn closing-channel [n]
   (let [b (closing-buffer n)
 c (async/chan b)]
 (this-is-your-channel! b c)
 c))

 From the REPL:

 user (def test-chan (closing-channel 2))
 #'user/test-chan
 user (async/!! test-chan :foo)
 nil
 user (async/!! test-chan :foo)
 nil
 user (async/!! test-chan :foo)
 nil
 user (async/!! test-chan)
 :foo
 user (async/!! test-chan)
 :foo
 user (async/!! test-chan)
 nil
 user (async/!! test-chan :foo)
 nil
 user (async/!! test-chan)
 nil

 Cheers,
 Michał


 On 14 January 2014 07:41, t x txrev...@gmail.com wrote:
 If I understand your strategy correctly, it works as follows:

 * write new buffer class
 * extend buffer class with a field/atom that stores the channel that owns
 the buffer

 * on overflow, call (async/close!) on the owned channel

 However, looking at :

 *
 https://github.com/clojure/core.async/blob/master/src/main/clojure/clojure/core/async/impl/channels.clj#L179
 *
 https://github.com/clojure/core.async/blob/master/src/main/clojure/clojure/core/async/impl/channels.clj#L55

 isn't this a deadlock by acquiring the mutex while holding the mutex?


 On Mon, Jan 13, 2014 at 10:27 PM, Michał Marczyk michal.marc...@gmail.com
 wrote:

 Instead of actually implementing a new channel type, you could use the
 regular MTMCs, but with your own factory function, custom buffers and
 a sprinkle of mutability:

 (defprotocol PBufferThatMightCloseAChannel
   (this-is-your-channel! [this c]
 Informs the buffer that c is the channel it might need to close))

 ;; implement a buffer with an impl for the above protocol;
 ;; have it close the appropriate channel on overflow;
 ;; suppose closing-buffer is the factory function
 ;; producing such channels

 (defn closing-channel [n]
   (let [b (closing-buffer n)
 c (chan b)]
 (this-is-your-channel! b c)
 c))

 Cheers,
 Michał


 On 14 January 2014 05:47, t x txrev...@gmail.com wrote:
  I am aware of:
 
DroppingBuffer and SliddingBuffer
 
  I would like to build a channel with different semantics:
 
When I try to put an item on a full channel

Re: closing-buffer

[Michał Marczyk]

Instead of actually implementing a new channel type, you could use the
regular MTMCs, but with your own factory function, custom buffers and
a sprinkle of mutability:

(defprotocol PBufferThatMightCloseAChannel
  (this-is-your-channel! [this c]
Informs the buffer that c is the channel it might need to close))

;; implement a buffer with an impl for the above protocol;
;; have it close the appropriate channel on overflow;
;; suppose closing-buffer is the factory function
;; producing such channels

(defn closing-channel [n]
  (let [b (closing-buffer n)
c (chan b)]
(this-is-your-channel! b c)
c))

Cheers,
Michał


On 14 January 2014 05:47, t x txrev...@gmail.com wrote:
 I am aware of:

   DroppingBuffer and SliddingBuffer

 I would like to build a channel with different semantics:

   When I try to put an item on a full channel:

   * DroppingBuffer drops the new item
   * SliddingBuffer evicts the oldest item

   * ClosingBuffer should _close the channel_

 (thus, the receiver eventually gets a nil, and realizes ah, was
 disconnected)

 Now, I am looking at the github core.async code:

 DroppingBuffer:
 https://github.com/clojure/core.async/blob/76317035d386ce2a1d98c2c349da9b898b480c55/src/main/clojure/clojure/core/async/impl/buffers.clj#L33-L45

 ManyToManyChannel:
 https://github.com/clojure/core.async/blob/76317035d386ce2a1d98c2c349da9b898b480c55/src/main/clojure/clojure/core/async/impl/channels.clj#L31-L198

 ## Problem:

 DroppingBuffer is easy to understand. However, I don't think I can implement
 ClosingBuffer as Buffer.

 I feel taht ClosingBuffer can only be implemneted at the Channel Level.

 However, the Channel Code is complicated.

 Is there a neat hack / trick to do what I described above?

 Thanks!

 --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.

-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[ANN] data.avl 0.0.11 -- transient-enabled sorted maps and sets with nth

[Michał Marczyk]

Hi,

I am pleased to announce that version 0.0.11 of data.avl, a Clojure
Contrib library implementing drop-in replacements for
Clojure(Script)'s core sorted collections, is now available from Maven
Central. See below for more information.

Changes in this release:

 * two bugs squashed in the ClojureScript version;

 * in Clojure, data.avl maps and sets are now properly
   java.io.Serializable;

 * in Clojure, rank-of now returns primitive longs where appropriate.

data.avl's maps and sets are constructed with the
clojure.data.avl/sorted-{map,set} functions and their *-by variants
and behave like their clojure.core counterparts, with the following
differences:

 * support for the transient API;

 * support for efficient rank queries via clojure.core/nth and
   clojure.data.avl/rank-of;

 * superior lookup performance at some cost in assoc/dissoc
   performance.

My original announcement of data.avl 0.0.10 [0] provides additional
details (including some Criterium benchmarks).

Dependency information:

  [org.clojure/data.avl 0.0.11]

  dependency
groupIdorg.clojure/groupId
artifactIddata.avl/artifactId
version0.0.11/version
  /dependency

  compile org.clojure:data.avl:0.0.11

Project repository and issue tracker:

  https://github.com/clojure/data.avl

  http://dev.clojure.org/jira/browse/DAVL

Cheers,
Michał


[0] https://groups.google.com/d/msg/clojure/8T-Dorhq6xQ/qne94v7Zot0J

-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] data.avl 0.0.10 -- fast sorted maps and sets with log-time rank queries

[Michał Marczyk]

On 6 January 2014 00:33, Brandon Bloom brandon.d.bl...@gmail.com wrote:
 Michał: This is awesome. Thanks for the continued awesome work on data
 structures!


   ;; if the key is not present in the collection, -1 is returned:
   (avl/rank-of (avl/sorted-set 3 4 5) 0)
   ;= -1


 Curious: Why not return nil instead of -1?

Thank you, happy to hear this might be useful!

As for rank-of's -1: the reason is as Mikera said. I wanted to be
consistent with primitive-returning Java methods like indexOf and to
be able to hint rank-of to ^long (although I haven't yet -- 0.0.11, I
guess!).

This is not the only sensible thing to do; I was thinking about maybe
adding variants of rank-of which would find the index of the first =
/ = element according to the comparator (which would still return -1
for empty collections and elements less / greater than the extreme
elements of the collection, but not for those in between a pair of
existing keys):

(rank-of-= (avl/sorted-set 0 2 4) 3)
;= 1

(rank-of-= (avl/sorted-set 0 2 4) 3)
;= 2

(rank-of-= (avl/sorted-set 0 2 4) 2)
;= 1

Alternatively rank-of could take an optional argument to force this
behaviour; we could even use  / = / = /  like subseq does.

In fact, apparently there's a relevant ticket in the tracker already:
DAVL-1 [0].

It's doable right now with subseq / rsubseq:

(defn rank-of-= [coll x]
  (if (empty? coll)
-1
(let [r (avl/rank-of coll x)]
  (if (== -1 r)
(let [s (rsubseq coll = x)]
  (avl/rank-of coll (first s)))
r

(mapv #(rank-of-= (avl/sorted-set 0 2 4) %) [-1 0 1 2 3 4 5])
;= [-1 0 0 1 1 2 2]

But a direct approach would no doubt be faster.

Any ideas re: possible API, comments on the usefulness of such
additions etc. would be very welcome!

Cheers,
Michał


[0] http://dev.clojure.org/jira/browse/DAVL-1



 --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.

-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[ANN] core.rrb-vector 0.0.10 -- vector concatenation for Clojure(Script)

[Michał Marczyk]

Hi,

I am pleased to announce the immediate availability of version 0.0.10
of core.rrb-vector, a Clojure Contrib library enabling
logarithmic-time concatenation and slicing of Clojure(Script) vectors.

Leiningen:

  [org.clojure/core.rrb-vector 0.0.10]

Maven:

  dependency
groupIdorg.clojure/groupId
artifactIdcore.rrb-vector/artifactId
version0.0.10/version
  /dependency

Gradle:

  compile org.clojure:core.rrb-vector:0.0.10

The public API is exported by the clojure.core.rrb-vector namespace:

  (require '[clojure.core.rrb-vector :as fv])

Most users will only be interested in two functions: fv/catvec and
fv/subvec. These can be used with regular Clojure vectors (including
vectors of primitives and view vectors created with
clojure.core/subvec); there is no need to construct RRB vectors by
hand.

  (fv/catvec (fv/subvec [0 1 2] 1 2) [3 4 5])
  ;= [1 3 4 5]

New in this release:

 * Several important bug fixes; existing users should upgrade ASAP!

 * Stress testing using Zach Tellman's great collection-check library
   (which helped catch some of the aforementioned bugs); many thanks
   for this one, Zach!

 * Support for the transient API.

 * ClojureScript version (same API minus vector-of).

 * Some forgotten Java interfaces implemented for RRB vectors.

 * New behaviour of fv/vec (when passed a vector, reuses its tree).

The repository for this project is located here:

  https://github.com/clojure/core.rrb-vector

See the README or (doc clojure.core.rrb-vector) for additional details.

Cheers,
Michał

-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[ANN] data.avl 0.0.10 -- fast sorted maps and sets with log-time rank queries

[Michał Marczyk]

Hi,

I am pleased to announce the initial Clojure Contrib release of
data.avl (previously known as avl.clj), a library implementing drop-in
replacements for Clojure(Script)'s sorted maps and sets with faster
lookups, support for the transient API (leading to unambiguously
improved performance for many usage patterns) and log-time rank
queries (via clojure.core/nth and clojure.data.avl/rank-of).

There are two relevant trade-offs: higher memory usage (a pointer and
two ints per key) and somewhat slower single updates. See below for
an extended discussion and some benchmark results.

For users of avl.clj 0.0.9: this release fixes a bug in rank-of and a
single reflective call I missed previously. Upgrading requires no
changes other than to the namespace name, which is now
clojure.data.avl.

The Maven artefact is now available from Maven Central; dependency
information is as follows (Leiningen / Maven / Gradle):

  [org.clojure/data.avl 0.0.10]

  dependency
groupIdorg.clojure/groupId
artifactIddata.avl/artifactId
version0.0.10/version
  /dependency

  compile org.clojure:data.avl:0.0.10

The repository for this project is located here:

  https://github.com/clojure/data.avl

data.avl uses persistent AVL trees as the underlying data structure.
AVL trees tend to be significantly shallower than red-black trees;
since the built-in sorted collections use red-black trees, data.avl
maps and sets offer noticeably improved expected lookup times.

The trade-off here is that updates (assoc / dissoc / conj / disj)
must perform more work in rebalancing the trees, so they are typically
slower than with the built-ins. However, data.avl maps and sets
support the transient API, which can be used to speed up long chains
of updates. Thanks to this, data.avl types tend to outperform the
built-ins for batch updates. One particularly important case of this
type is that of the initial construction of large instances; this
tends to be noticeably faster with data.avl types (on a par with the
built-ins worst-case in my benchmarking so far).

(Of course the time to complete all operations varies with tree
structure. When comparing lookup times for nodes occupying the same
positions in both trees, the built-in red-black trees have a very
slight edge; see for example (get avl 0) and (get rb 0) in the
benchmark results below. Updates involving identically placed nodes
may be faster in either tree type depending on the overall shape of
the tree; see for example (dissoc avl 131071) and (dissoc rb 131071)
below, where the AVL map completes the operation twice as fast. On
average, however, performance is as described above.)

Besides supporting transients, data.avl types offer two additions to
the sorted collection API:

  (require '[clojure.data.avl :as avl])

  ;; nth can be used to look up the nth smallest item in a sorted
  ;; collection, as determined by the comparator being used:

  (nth (avl/sorted-set 0 2 4) 2)
  ;= 4

  (nth (avl/sorted-set-by  0 2 4) 2)
  ;= 0

  ;; for maps, nth returns map entry objects:

  (nth (avl/sorted-map 0 0 2 2 4 4) 2)
  ;= [4 4]

  ;; clojure.data.avl/rank-of can be used to query for the index of a
  ;; given key in a data.avl collection:

  (avl/rank-of (avl/sorted-set 3 4 5) 4)
  ;= 1

  ;; if the key is not present in the collection, -1 is returned:
  (avl/rank-of (avl/sorted-set 3 4 5) 0)
  ;= -1

As mentioned above, the new functionality comes at a cost in memory
consumption. To support transients, each node in a data.avl tree must
carry an extra reference field -- that's one pointer per key. To
support efficient rank queries, each pointer needs to carry an extra
int field. Finally, another int field is used by the rebalancing
algorithm itself.

The library carries its own test suite used in both Clojure and
ClojureScript. Additionally, the Clojure version is stress-tested with
Zach Tellman's collection-check library.

I attach some Criterium benchmark results below.

Cheers,
Michał


Benchmark environment:

  OpenJDK 1.7, Clojure 1.5.1, Criterium 0.4.2.

Relevant defs:

  (def ks   (range 30))
  (def ksks (interleave ks ks))

  (def rb  (apply sorted-map ksks))
  (def avl (apply avl/sorted-map ksks))

  (defn rb-rank-of [rb-map k]
(if (contains? rb k)
  (count (take-while #(not= k (key %)) (seq rb)))
  -1))

Benchmark results:

(lookup-benchmarks)
===
(c/bench (get avl 0))
WARNING: Final GC required 17.30960512486499 % of runtime
WARNING: Final GC required 1.487642242862071 % of runtime
Evaluation count : 184965780 in 60 samples of 3082763 calls.
 Execution time mean : 320.595187 ns
Execution time std-deviation : 2.710112 ns
   Execution time lower quantile : 316.756609 ns ( 2.5%)
   Execution time upper quantile : 325.833642 ns (97.5%)
   Overhead used : 2.128084 ns
(c/bench (get rb 0))
WARNING: Final GC required 1.481744139127142 % of runtime
Evaluation count : 192022740 in 60 samples of 3200379 calls.
 

Re: bug in clojure.zip when calling next on empty list node?

[Michał Marczyk]

Ticket with patch at

http://dev.clojure.org/jira/browse/CLJ-1317

The problem is that seq-zip uses clojure.core/identity as the
children argument to zipper. Applied to (), this returns (), but
clojure.zip expects the children function to return nil when there
are no children. The patch attached to the ticket above changes
identity to seq inside seq-zip. This suffices to fix the behaviour
discovered by Lee.

Cheers,
Michał


On 1 January 2014 00:15, Lee Spector lspec...@hampshire.edu wrote:

 On Dec 31, 2013, at 5:08 PM, Armando Blancas wrote:

 The implementation of seq-zip uses seq? as its branching predicate. As a 
 result the zipper goes down on () thinking it can have children:

 user= (seq? ())
 true
 user= (seq? {})
 false
 user= (seq? #{})
 false
 user= (seq? [])
 false

 Does that mean that you think that the behavior is correct and expected?

 It goes down () thinking it can have children -- fine -- but when there 
 aren't any children shouldn't zip/next continue until it hits the next thing? 
 Why should it land on a non-existent nil instead?

 Compare:

 Traversing '((1) 0) with zip/next we get 4 items: ((1) 0), (1), 1, 0

 Traversing '(() 0) with zip/next we also get 4 items: (() 0), (), nil, 0

 It seems to me that these shouldn't both give 4 things, since the second 
 clearly contain one less thing. That alleged nil just isn't there. Note also 
 that:

 Traversing '((nil) 0)) with zip/next we also get 4 items: ((nil) 0), (nil), 
 nil, 0

 That one seems right to me -- there really IS a nil there this time. But the 
 one with () doesn't.

 If I'm alone in this then I guess I'll just write my own zip_really_next that 
 does what I expect, but I'm curious if others also think that the current 
 behavior is correct.

 Thanks,

  -Lee


 --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your 
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups 
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.

-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: bug in clojure.zip when calling next on empty list node?

[Michał Marczyk]

Oh, and of course you can use the amended version now to obtain the
expected results:

(defn seq-zip
  Returns a zipper for nested sequences, given a root sequence
  {:added 1.0}
  [root]
(zipper seq?
seq
(fn [node children] (with-meta children (meta node)))
root))

As noted in a comment on the ticket, clojure.zip/zipper's documetation
actuals asks that the children argument return a seq of nodes. The
rest of clojure.zip, however, expects nil when there are no children.
One could argue that it is that expectation that needs fixing, but I
think the current behaviour makes sense; perhaps the docstring should
be adjusted, though.

Cheers,
Michał


On 1 January 2014 00:53, Michał Marczyk michal.marc...@gmail.com wrote:
 Ticket with patch at

 http://dev.clojure.org/jira/browse/CLJ-1317

 The problem is that seq-zip uses clojure.core/identity as the
 children argument to zipper. Applied to (), this returns (), but
 clojure.zip expects the children function to return nil when there
 are no children. The patch attached to the ticket above changes
 identity to seq inside seq-zip. This suffices to fix the behaviour
 discovered by Lee.

 Cheers,
 Michał


 On 1 January 2014 00:15, Lee Spector lspec...@hampshire.edu wrote:

 On Dec 31, 2013, at 5:08 PM, Armando Blancas wrote:

 The implementation of seq-zip uses seq? as its branching predicate. As a 
 result the zipper goes down on () thinking it can have children:

 user= (seq? ())
 true
 user= (seq? {})
 false
 user= (seq? #{})
 false
 user= (seq? [])
 false

 Does that mean that you think that the behavior is correct and expected?

 It goes down () thinking it can have children -- fine -- but when there 
 aren't any children shouldn't zip/next continue until it hits the next 
 thing? Why should it land on a non-existent nil instead?

 Compare:

 Traversing '((1) 0) with zip/next we get 4 items: ((1) 0), (1), 1, 0

 Traversing '(() 0) with zip/next we also get 4 items: (() 0), (), nil, 0

 It seems to me that these shouldn't both give 4 things, since the second 
 clearly contain one less thing. That alleged nil just isn't there. Note also 
 that:

 Traversing '((nil) 0)) with zip/next we also get 4 items: ((nil) 0), (nil), 
 nil, 0

 That one seems right to me -- there really IS a nil there this time. But the 
 one with () doesn't.

 If I'm alone in this then I guess I'll just write my own zip_really_next 
 that does what I expect, but I'm curious if others also think that the 
 current behavior is correct.

 Thanks,

  -Lee


 --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your 
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups 
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.

-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: bug in clojure.zip when calling next on empty list node?

[Michał Marczyk]

No worries.

Incidentally, akhudek's fast-zip seems to have the same issue. Here's
a PR fixing it:

https://github.com/akhudek/fast-zip/pull/3

Cheers,
Michał


On 1 January 2014 02:12, Lee Spector lspec...@hampshire.edu wrote:

 On Dec 31, 2013, at 6:53 PM, Michał Marczyk wrote:

 Ticket with patch at

 http://dev.clojure.org/jira/browse/CLJ-1317

 [and]
 Oh, and of course you can use the amended version now to obtain the
 expected results: ///


 Thank you so much Michał!

  -Lee

 --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your 
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups 
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.

-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: When does clojure.lang.PersistentArrayMap become clojure.lang.PersistentHashMap

[Michał Marczyk]

;; persistent maps only
(instance? clojure.lang.IPersistentMap %)

;; all Java maps
(instance? java.util.Map %)

On 24 December 2013 04:28, larry google groups
lawrencecloj...@gmail.com wrote:
 I find this surprising. I do this:

 (supers %)

 inside of my :post condition, and I get:

 java.lang.ClassCastException: clojure.lang.PersistentHashMap cannot be cast
 to java.lang.Class
 at clojure.core$bases.invoke(core.clj:4985)
 at clojure.core$supers.invoke(core.clj:4994)
 at admin.secretary$fetch.invoke(secretary.clj:327)

 How else do I find what interfaces the return value might be implementing?





 On Monday, December 23, 2013 10:04:54 PM UTC-5, larry google groups wrote:

  enforce that the return is a subtype of java.util.Map rather than
  checking for a specific concrete class of map.


 Thank you. I'll do that. All the same, can anyone tell me why this
 function changes the type of the value? All it does is call fetch. The
 fetch function has this post condition:

:post [(= (type %) clojure.lang.PersistentHashMap)]}

 And get-distinct has the same restriction, but in get-distinct I get
 this error:

 java.lang.AssertionError: Assert failed: (= (type %)
 clojure.lang.PersistentHashMap)
 at admin.controller$get_distinct.invoke(controller.clj:40)

 The :post condition of fetch does not throw an error, so I know that
 fetch is returning clojure.lang.PersistentHashMap.  However, when I print
 the return type of get-distinct, I am amazed to see that it is now
 clojure.lang.PersistentArrayMap. This function does almost nothing, so I am
 surprised it changes the concrete implementation type of the return value.






 On Monday, December 23, 2013 3:43:09 AM UTC-5, Cedric Greevey wrote:

 The two classes have essentially the same semantics, but performance
 differences, which is why Clojure sometimes uses one and sometimes the
 other. If you want to enforce that a map is returned, enforce that the
 return is a subtype of java.util.Map rather than checking for a specific
 concrete class of map.


 On Sun, Dec 22, 2013 at 3:07 PM, larry google groups
 lawrenc...@gmail.com wrote:

 Hmm, I see. get-distinct was returning an empty lazyseq, which
 apparently made the difference.


 On Sunday, December 22, 2013 2:56:01 PM UTC-5, larry google groups
 wrote:

 Hmm, the different return types seem tied to the 2 different functions
 being called, but both functions have the same return type, which is a
 lazyseq. I am using Monger to get data from MongoDb. These functions are
 private:

 (defn- get-distinct [request]
   {:pre [(= (type request) clojure.lang.PersistentHashMap)]
:post [(= (type %) clojure.lang.LazySeq)]}
   (monger/get-distinct (:item-type request)))

 (defn- paginate-results [request]
   {:pre [ (= (type request) clojure.lang.PersistentHashMap)]
:post [(= (type %) clojure.lang.LazySeq)]}
   (monger/paginate-results (:item-type request) (if (:which-page
 request)
   (:which-page request)
   0)))

 Both of these functions return lazyseqs, as expected. The results from
 both get run through a (reduce) function that does some minor filtering. 
 Yet
 in one case the return type (from the reduce function) is
 clojure.lang.PersistentArrayMap and in the other it is
 clojure.lang.PersistentHashMap. I'd like to be able to write a :post
 condition that enforces strictness, but that seems impossible because I 
 can
 not figure out what the rule is that handles the conversion. I don't care 
 if
 the return type is clojure.lang.PersistentArrayMap or
 clojure.lang.PersistentHashMap, all I want is it for it be consistently 
 one
 or the other.





 On Sunday, December 22, 2013 2:31:45 PM UTC-5, larry google groups
 wrote:


 I am surprised that a map literal is clojure.lang.PersistentArrayMap
 but as soon as I assign it to a var, it becomes
 clojure.lang.PersistentHashMap. Are there any rules for being able to
 predict when these conversions occur?

 user (type {})
 clojure.lang.PersistentArrayMap

 user (type {:what why?})
 clojure.lang.PersistentArrayMap

 user (def curious {:what why?})
 #'user/curious

 user (type curious)
 clojure.lang.PersistentHashMap

 user (def sug (assoc curious :whodoneit mikey))
 #'user/sug

 user (type sug)
 clojure.lang.PersistentHashMap

 I am curious because I wrote a (reduce) function which mostly just
 builds a map:

  (assoc map-of-data (:item-name next-item) next-item))

 Since I was using assoc I was certain I would get
 clojure.lang.PersistentHashMap back, but instead I got
 clojure.lang.PersistentArrayMap.






 --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com
 For more options, visit this 

Re: When does clojure.lang.PersistentArrayMap become clojure.lang.PersistentHashMap

[Michał Marczyk]

On 24 December 2013 07:35, Michał Marczyk michal.marc...@gmail.com wrote:
 ;; persistent maps only
 (instance? clojure.lang.IPersistentMap %)

Additionally, this one's better written

(map? %)

(map? is in fact defined as instance? IPM.)

Cheers,
M.


 ;; all Java maps
 (instance? java.util.Map %)

 On 24 December 2013 04:28, larry google groups
 lawrencecloj...@gmail.com wrote:
 I find this surprising. I do this:

 (supers %)

 inside of my :post condition, and I get:

 java.lang.ClassCastException: clojure.lang.PersistentHashMap cannot be cast
 to java.lang.Class
 at clojure.core$bases.invoke(core.clj:4985)
 at clojure.core$supers.invoke(core.clj:4994)
 at admin.secretary$fetch.invoke(secretary.clj:327)

 How else do I find what interfaces the return value might be implementing?





 On Monday, December 23, 2013 10:04:54 PM UTC-5, larry google groups wrote:

  enforce that the return is a subtype of java.util.Map rather than
  checking for a specific concrete class of map.


 Thank you. I'll do that. All the same, can anyone tell me why this
 function changes the type of the value? All it does is call fetch. The
 fetch function has this post condition:

:post [(= (type %) clojure.lang.PersistentHashMap)]}

 And get-distinct has the same restriction, but in get-distinct I get
 this error:

 java.lang.AssertionError: Assert failed: (= (type %)
 clojure.lang.PersistentHashMap)
 at admin.controller$get_distinct.invoke(controller.clj:40)

 The :post condition of fetch does not throw an error, so I know that
 fetch is returning clojure.lang.PersistentHashMap.  However, when I print
 the return type of get-distinct, I am amazed to see that it is now
 clojure.lang.PersistentArrayMap. This function does almost nothing, so I am
 surprised it changes the concrete implementation type of the return value.






 On Monday, December 23, 2013 3:43:09 AM UTC-5, Cedric Greevey wrote:

 The two classes have essentially the same semantics, but performance
 differences, which is why Clojure sometimes uses one and sometimes the
 other. If you want to enforce that a map is returned, enforce that the
 return is a subtype of java.util.Map rather than checking for a specific
 concrete class of map.


 On Sun, Dec 22, 2013 at 3:07 PM, larry google groups
 lawrenc...@gmail.com wrote:

 Hmm, I see. get-distinct was returning an empty lazyseq, which
 apparently made the difference.


 On Sunday, December 22, 2013 2:56:01 PM UTC-5, larry google groups
 wrote:

 Hmm, the different return types seem tied to the 2 different functions
 being called, but both functions have the same return type, which is a
 lazyseq. I am using Monger to get data from MongoDb. These functions are
 private:

 (defn- get-distinct [request]
   {:pre [(= (type request) clojure.lang.PersistentHashMap)]
:post [(= (type %) clojure.lang.LazySeq)]}
   (monger/get-distinct (:item-type request)))

 (defn- paginate-results [request]
   {:pre [ (= (type request) clojure.lang.PersistentHashMap)]
:post [(= (type %) clojure.lang.LazySeq)]}
   (monger/paginate-results (:item-type request) (if (:which-page
 request)
   (:which-page request)
   0)))

 Both of these functions return lazyseqs, as expected. The results from
 both get run through a (reduce) function that does some minor filtering. 
 Yet
 in one case the return type (from the reduce function) is
 clojure.lang.PersistentArrayMap and in the other it is
 clojure.lang.PersistentHashMap. I'd like to be able to write a :post
 condition that enforces strictness, but that seems impossible because I 
 can
 not figure out what the rule is that handles the conversion. I don't 
 care if
 the return type is clojure.lang.PersistentArrayMap or
 clojure.lang.PersistentHashMap, all I want is it for it be consistently 
 one
 or the other.





 On Sunday, December 22, 2013 2:31:45 PM UTC-5, larry google groups
 wrote:


 I am surprised that a map literal is clojure.lang.PersistentArrayMap
 but as soon as I assign it to a var, it becomes
 clojure.lang.PersistentHashMap. Are there any rules for being able to
 predict when these conversions occur?

 user (type {})
 clojure.lang.PersistentArrayMap

 user (type {:what why?})
 clojure.lang.PersistentArrayMap

 user (def curious {:what why?})
 #'user/curious

 user (type curious)
 clojure.lang.PersistentHashMap

 user (def sug (assoc curious :whodoneit mikey))
 #'user/sug

 user (type sug)
 clojure.lang.PersistentHashMap

 I am curious because I wrote a (reduce) function which mostly just
 builds a map:

  (assoc map-of-data (:item-name next-item) next-item))

 Since I was using assoc I was certain I would get
 clojure.lang.PersistentHashMap back, but instead I got
 clojure.lang.PersistentArrayMap.






 --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com
 Note

Re: core.async timeout channels are values - is this intended

[Michał Marczyk]

The reason = considers you timeout channels to be equal is that they
are, in fact, the same object. In fact, they may end up being the same
object even with different timeout values:

(identical? (timeout 1) (timeout 2))
;= true

Except I got a false just now with a fresh REPL and same timeout
value... All subsequent invocations return true though, and I'm sure
with a little digging the reason for the false would become clear.

The reason for them being the same object is that timeout goes out of
its way to avoid creating to many timeout channels and will reuse
existing ones if their timeouts are due within a small amount of time
(clojure.core.async.impl.timers/TIMEOUT_RESOLUTION_MS, currently 10
ms) of the requested timeout.

Cheers,
Michał

On 20 November 2013 10:08, Thomas G. Kristensen
thomas.g.kristen...@gmail.com wrote:
 Hi all,

 I ran into a core.async behaviour that confused me a bit the other day. In
 some of our systems, we need to fire different timeouts, perform actions and
 schedule a new timeout. The problem is, that if the timeouts are of the same
 number of ms, we can't distinguish them, and therefore not keep track of and
 remove them from a set (at least not easily).

 That sounds a bit fuzzy. Hopefully this spike will make it clearer what I'm
 trying to say:

 (require '[clojure.core.async :refer [chan timeout alts!! alts!]])

 (= (chan) (chan))
 ;; false

 (= (timeout 1) (timeout 2))
 ;; false

 (= (timeout 1) (timeout 1))
 ;; true

 (do (loop [ch-v (into {} (for [v [1 2 3]] [(timeout 1000) v]))]
   (when-let [chs (keys ch-v)]
 (let [[_ ch] (alts!! chs)]
   (println (ch-v ch))
   (recur (dissoc ch-v ch)
 (println done))
 ;; only fires 3, the last channel in the map

 The intended behaviour of the last loop is to print 1, 2 and 3 (not
 necessarily in that order). However, the ch-v map will only contain one
 key, as timeouts with the same duration are considered the same value. In
 the real example, a new timeout with the same value should be scheduled
 again, by being put in the map.

 So, my questions are:

 - Is this intended behaviour?
 - Is there a different pattern for achieving the scheduling behaviour I'm
 looking for?

 Thanks for your help,

 Thomas

 --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.

-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: core.async timeout channels are values - is this intended

[Michał Marczyk]

The behaviour of timeout channels is to close after the timeout
elapses. This will be visible everywhere the channel is used.

Cheers,
Michał


On 20 November 2013 11:22, Cedric Greevey cgree...@gmail.com wrote:
 Isn't that not only violating least astonishment, but potentially
 introducing a terrible bug into the library? One could use two different
 timeout channels in two different alts, and if the timeout channels are
 aliased, then perhaps only one of the alts would actually get notified.


 On Wed, Nov 20, 2013 at 5:05 AM, Michał Marczyk michal.marc...@gmail.com
 wrote:

 The reason = considers you timeout channels to be equal is that they
 are, in fact, the same object. In fact, they may end up being the same
 object even with different timeout values:

 (identical? (timeout 1) (timeout 2))
 ;= true

 Except I got a false just now with a fresh REPL and same timeout
 value... All subsequent invocations return true though, and I'm sure
 with a little digging the reason for the false would become clear.

 The reason for them being the same object is that timeout goes out of
 its way to avoid creating to many timeout channels and will reuse
 existing ones if their timeouts are due within a small amount of time
 (clojure.core.async.impl.timers/TIMEOUT_RESOLUTION_MS, currently 10
 ms) of the requested timeout.

 Cheers,
 Michał

 On 20 November 2013 10:08, Thomas G. Kristensen
 thomas.g.kristen...@gmail.com wrote:
  Hi all,
 
  I ran into a core.async behaviour that confused me a bit the other day.
  In
  some of our systems, we need to fire different timeouts, perform actions
  and
  schedule a new timeout. The problem is, that if the timeouts are of the
  same
  number of ms, we can't distinguish them, and therefore not keep track of
  and
  remove them from a set (at least not easily).
 
  That sounds a bit fuzzy. Hopefully this spike will make it clearer what
  I'm
  trying to say:
 
  (require '[clojure.core.async :refer [chan timeout alts!! alts!]])
 
  (= (chan) (chan))
  ;; false
 
  (= (timeout 1) (timeout 2))
  ;; false
 
  (= (timeout 1) (timeout 1))
  ;; true
 
  (do (loop [ch-v (into {} (for [v [1 2 3]] [(timeout 1000) v]))]
(when-let [chs (keys ch-v)]
  (let [[_ ch] (alts!! chs)]
(println (ch-v ch))
(recur (dissoc ch-v ch)
  (println done))
  ;; only fires 3, the last channel in the map
 
  The intended behaviour of the last loop is to print 1, 2 and 3 (not
  necessarily in that order). However, the ch-v map will only contain one
  key, as timeouts with the same duration are considered the same value.
  In
  the real example, a new timeout with the same value should be scheduled
  again, by being put in the map.
 
  So, my questions are:
 
  - Is this intended behaviour?
  - Is there a different pattern for achieving the scheduling behaviour
  I'm
  looking for?
 
  Thanks for your help,
 
  Thomas
 
  --
  --
  You received this message because you are subscribed to the Google
  Groups Clojure group.
  To post to this group, send email to clojure@googlegroups.com
  Note that posts from new members are moderated - please be patient with
  your
  first post.
  To unsubscribe from this group, send email to
  clojure+unsubscr...@googlegroups.com
  For more options, visit this group at
  http://groups.google.com/group/clojure?hl=en
  ---
  You received this message because you are subscribed to the Google
  Groups
  Clojure group.
  To unsubscribe from this group and stop receiving emails from it, send
  an
  email to clojure+unsubscr...@googlegroups.com.
  For more options, visit https://groups.google.com/groups/opt_out.

 --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.


 --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr

Re: core.async timeout channels are values - is this intended

[Michał Marczyk]

As for scheduling the printlns, this prints out 1, 2, 3 (in some order):

(let [c (chan)]
  (doseq [v [1 2 3]]
(take! (async/timeout 1000) (fn [_] (put! c v
  (go-loop [v (! c)]
(println v)
(recur (! c

Cheers,
Michał


On 20 November 2013 11:34, Michał Marczyk michal.marc...@gmail.com wrote:
 The behaviour of timeout channels is to close after the timeout
 elapses. This will be visible everywhere the channel is used.

 Cheers,
 Michał


 On 20 November 2013 11:22, Cedric Greevey cgree...@gmail.com wrote:
 Isn't that not only violating least astonishment, but potentially
 introducing a terrible bug into the library? One could use two different
 timeout channels in two different alts, and if the timeout channels are
 aliased, then perhaps only one of the alts would actually get notified.


 On Wed, Nov 20, 2013 at 5:05 AM, Michał Marczyk michal.marc...@gmail.com
 wrote:

 The reason = considers you timeout channels to be equal is that they
 are, in fact, the same object. In fact, they may end up being the same
 object even with different timeout values:

 (identical? (timeout 1) (timeout 2))
 ;= true

 Except I got a false just now with a fresh REPL and same timeout
 value... All subsequent invocations return true though, and I'm sure
 with a little digging the reason for the false would become clear.

 The reason for them being the same object is that timeout goes out of
 its way to avoid creating to many timeout channels and will reuse
 existing ones if their timeouts are due within a small amount of time
 (clojure.core.async.impl.timers/TIMEOUT_RESOLUTION_MS, currently 10
 ms) of the requested timeout.

 Cheers,
 Michał

 On 20 November 2013 10:08, Thomas G. Kristensen
 thomas.g.kristen...@gmail.com wrote:
  Hi all,
 
  I ran into a core.async behaviour that confused me a bit the other day.
  In
  some of our systems, we need to fire different timeouts, perform actions
  and
  schedule a new timeout. The problem is, that if the timeouts are of the
  same
  number of ms, we can't distinguish them, and therefore not keep track of
  and
  remove them from a set (at least not easily).
 
  That sounds a bit fuzzy. Hopefully this spike will make it clearer what
  I'm
  trying to say:
 
  (require '[clojure.core.async :refer [chan timeout alts!! alts!]])
 
  (= (chan) (chan))
  ;; false
 
  (= (timeout 1) (timeout 2))
  ;; false
 
  (= (timeout 1) (timeout 1))
  ;; true
 
  (do (loop [ch-v (into {} (for [v [1 2 3]] [(timeout 1000) v]))]
(when-let [chs (keys ch-v)]
  (let [[_ ch] (alts!! chs)]
(println (ch-v ch))
(recur (dissoc ch-v ch)
  (println done))
  ;; only fires 3, the last channel in the map
 
  The intended behaviour of the last loop is to print 1, 2 and 3 (not
  necessarily in that order). However, the ch-v map will only contain one
  key, as timeouts with the same duration are considered the same value.
  In
  the real example, a new timeout with the same value should be scheduled
  again, by being put in the map.
 
  So, my questions are:
 
  - Is this intended behaviour?
  - Is there a different pattern for achieving the scheduling behaviour
  I'm
  looking for?
 
  Thanks for your help,
 
  Thomas
 
  --
  --
  You received this message because you are subscribed to the Google
  Groups Clojure group.
  To post to this group, send email to clojure@googlegroups.com
  Note that posts from new members are moderated - please be patient with
  your
  first post.
  To unsubscribe from this group, send email to
  clojure+unsubscr...@googlegroups.com
  For more options, visit this group at
  http://groups.google.com/group/clojure?hl=en
  ---
  You received this message because you are subscribed to the Google
  Groups
  Clojure group.
  To unsubscribe from this group and stop receiving emails from it, send
  an
  email to clojure+unsubscr...@googlegroups.com.
  For more options, visit https://groups.google.com/groups/opt_out.

 --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.


 --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr

Re: Functions using locks are slowed even when locks are never taken

[Michał Marczyk]

You have a typo in foo -- monitor-exit's argument is 0 (zero) rather
than o (the sentinel object).

Besides that, in foo both monitor-enter and monitor-exit get their
arguments from a Var. Rewriting to use locking, which first puts the
object whose monitor will be used in a local (that is, (let [lockee o]
...), where ... performs the locking using the newly introduced
local), gives timings identical to those of bar and baz:

(defn foo' [x]
  (if ( x 0)
(inc x)
(let [res (locking o (dec x))] res)))

So this is one reason not to use monitor-enter and monitor-exit
directly. Another reason is that locking guarantees that the monitor
will be released (by using try / finally, and of course by preventing
situations where the matching monitor-enter  monitor-exit operate on
different objects).

In fact, both monitor-enter and monitor-exit carry docstrings which
explicitly say that they should not be used in user code and point to
locking as the user-facing equivalent to Java's synchronized.

Cheers,
Michał


On 1 November 2013 19:34, Michael Blume blume.m...@gmail.com wrote:
 https://github.com/MichaelBlume/perf-test

 (ns perf-test
   (:use (criterium core))
   (:gen-class))

 (def o (Object.))

 (defn foo [x]
   (if ( x 0)
 (inc x)
 (do
   (monitor-enter o)
   (let [res (dec x)]
 (monitor-exit 0)
 res

 (defn bar [x]
   (if ( x 0)
 (inc x)
 (dec x)))

 (defn locking-part [x l]
   (monitor-enter l)
   (let [res (dec x)]
 (monitor-exit l)
 res))

 (defn baz [x]
   (if ( x 0)
 (inc x)
 (locking-part x o)))

 (defn -main []
   (println benching foo)
   (bench (foo 5) :verbose)
   (println benching bar)
   (bench (bar 5) :verbose)
   (println benching baz)
   (bench (baz 5) :verbose)
   (println done benching))



 I'm only ever calling these functions with positive values, so the
 monitor-enter branch should never be entered. Nevertheless, the performance
 of foo is much worse than bar or baz.

 The best guess I've got is that the fact that lock-taking is involved
 somehow changes how the function is compiled, somehow making the function
 slower. If the practical upshot is that I shouldn't write functions that
 only sometimes lock -- that the locking part of a function should always be
 its own function -- then I can do that, but I'm curious why.

 $ lein uberjar
 Compiling perf-test
 Created /Users/mike/perf-test/target/perf-test-0.1.0-SNAPSHOT.jar
 Created /Users/mike/perf-test/target/perf-test-0.1.0-SNAPSHOT-standalone.jar
 $ java -jar -server target/perf-test-0.1.0-SNAPSHOT-standalone.jar
 benching foo
 WARNING: Final GC required 1.5974571326266802 % of runtime
 x86_64 Mac OS X 10.8.3 4 cpu(s)
 Java HotSpot(TM) 64-Bit Server VM 24.0-b28
 Runtime arguments:
 Evaluation count : 391582560 in 60 samples of 6526376 calls.
   Execution time sample mean : 167.426696 ns
  Execution time mean : 167.459429 ns
 Execution time sample std-deviation : 4.079466 ns
 Execution time std-deviation : 4.097819 ns
Execution time lower quantile : 160.742869 ns ( 2.5%)
Execution time upper quantile : 175.453376 ns (97.5%)
Overhead used : 1.634996 ns

 Found 2 outliers in 60 samples (3. %)
 low-severe 2 (3. %)
  Variance from outliers : 12.5602 % Variance is moderately inflated by
 outliers
 benching bar
 x86_64 Mac OS X 10.8.3 4 cpu(s)
 Java HotSpot(TM) 64-Bit Server VM 24.0-b28
 Runtime arguments:
 Evaluation count : 2174037300 in 60 samples of 36233955 calls.
   Execution time sample mean : 26.068923 ns
  Execution time mean : 26.066422 ns
 Execution time sample std-deviation : 0.887937 ns
 Execution time std-deviation : 0.916861 ns
Execution time lower quantile : 23.996763 ns ( 2.5%)
Execution time upper quantile : 27.911936 ns (97.5%)
Overhead used : 1.634996 ns

 Found 3 outliers in 60 samples (5. %)
 low-severe 1 (1.6667 %)
 low-mild 1 (1.6667 %)
 high-mild 1 (1.6667 %)
  Variance from outliers : 22.1874 % Variance is moderately inflated by
 outliers
 benching baz
 x86_64 Mac OS X 10.8.3 4 cpu(s)
 Java HotSpot(TM) 64-Bit Server VM 24.0-b28
 Runtime arguments:
 Evaluation count : 2270676660 in 60 samples of 37844611 calls.
   Execution time sample mean : 25.834142 ns
  Execution time mean : 25.837429 ns
 Execution time sample std-deviation : 0.718382 ns
 Execution time std-deviation : 0.729431 ns
Execution time lower quantile : 24.837925 ns ( 2.5%)
Execution time upper quantile : 27.595781 ns (97.5%)
Overhead used : 1.634996 ns

 Found 4 outliers in 60 samples (6.6667 %)
 low-severe 2 (3. %)
 low-mild 2 (3. %)
  Variance from outliers : 15.7591 % Variance is moderately inflated by
 outliers
 done benching

 --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are 

Re: Functions using locks are slowed even when locks are never taken

[Michał Marczyk]

I should perhaps make clear that with direct use of monitor-enter and
monitor-exit with a Var it's possible for monitor-enter and
monitor-exit to operate on different objects even in the absence of
typos, namely if somebody rebinds the Var. To illustrate this with
print at the REPL (a regular Clojure REPL, as opposed to nREPL or
similar, in case there are any issues with reproducing this in
different environments):

(def x 1)
(future (print x) (Thread/sleep 5000) (print x))
; prints 1 immediately, but then goes to sleep;
; in the meantime, I say
(def x 2)
@*2
; and after a moment 2 is printed

NB. this could also happen with alter-var-root or set! to a Var with a
thread-local binding.

Cheers,
Michał


On 3 November 2013 18:30, Michał Marczyk michal.marc...@gmail.com wrote:
 You have a typo in foo -- monitor-exit's argument is 0 (zero) rather
 than o (the sentinel object).

 Besides that, in foo both monitor-enter and monitor-exit get their
 arguments from a Var. Rewriting to use locking, which first puts the
 object whose monitor will be used in a local (that is, (let [lockee o]
 ...), where ... performs the locking using the newly introduced
 local), gives timings identical to those of bar and baz:

 (defn foo' [x]
   (if ( x 0)
 (inc x)
 (let [res (locking o (dec x))] res)))

 So this is one reason not to use monitor-enter and monitor-exit
 directly. Another reason is that locking guarantees that the monitor
 will be released (by using try / finally, and of course by preventing
 situations where the matching monitor-enter  monitor-exit operate on
 different objects).

 In fact, both monitor-enter and monitor-exit carry docstrings which
 explicitly say that they should not be used in user code and point to
 locking as the user-facing equivalent to Java's synchronized.

 Cheers,
 Michał


 On 1 November 2013 19:34, Michael Blume blume.m...@gmail.com wrote:
 https://github.com/MichaelBlume/perf-test

 (ns perf-test
   (:use (criterium core))
   (:gen-class))

 (def o (Object.))

 (defn foo [x]
   (if ( x 0)
 (inc x)
 (do
   (monitor-enter o)
   (let [res (dec x)]
 (monitor-exit 0)
 res

 (defn bar [x]
   (if ( x 0)
 (inc x)
 (dec x)))

 (defn locking-part [x l]
   (monitor-enter l)
   (let [res (dec x)]
 (monitor-exit l)
 res))

 (defn baz [x]
   (if ( x 0)
 (inc x)
 (locking-part x o)))

 (defn -main []
   (println benching foo)
   (bench (foo 5) :verbose)
   (println benching bar)
   (bench (bar 5) :verbose)
   (println benching baz)
   (bench (baz 5) :verbose)
   (println done benching))



 I'm only ever calling these functions with positive values, so the
 monitor-enter branch should never be entered. Nevertheless, the performance
 of foo is much worse than bar or baz.

 The best guess I've got is that the fact that lock-taking is involved
 somehow changes how the function is compiled, somehow making the function
 slower. If the practical upshot is that I shouldn't write functions that
 only sometimes lock -- that the locking part of a function should always be
 its own function -- then I can do that, but I'm curious why.

 $ lein uberjar
 Compiling perf-test
 Created /Users/mike/perf-test/target/perf-test-0.1.0-SNAPSHOT.jar
 Created /Users/mike/perf-test/target/perf-test-0.1.0-SNAPSHOT-standalone.jar
 $ java -jar -server target/perf-test-0.1.0-SNAPSHOT-standalone.jar
 benching foo
 WARNING: Final GC required 1.5974571326266802 % of runtime
 x86_64 Mac OS X 10.8.3 4 cpu(s)
 Java HotSpot(TM) 64-Bit Server VM 24.0-b28
 Runtime arguments:
 Evaluation count : 391582560 in 60 samples of 6526376 calls.
   Execution time sample mean : 167.426696 ns
  Execution time mean : 167.459429 ns
 Execution time sample std-deviation : 4.079466 ns
 Execution time std-deviation : 4.097819 ns
Execution time lower quantile : 160.742869 ns ( 2.5%)
Execution time upper quantile : 175.453376 ns (97.5%)
Overhead used : 1.634996 ns

 Found 2 outliers in 60 samples (3. %)
 low-severe 2 (3. %)
  Variance from outliers : 12.5602 % Variance is moderately inflated by
 outliers
 benching bar
 x86_64 Mac OS X 10.8.3 4 cpu(s)
 Java HotSpot(TM) 64-Bit Server VM 24.0-b28
 Runtime arguments:
 Evaluation count : 2174037300 in 60 samples of 36233955 calls.
   Execution time sample mean : 26.068923 ns
  Execution time mean : 26.066422 ns
 Execution time sample std-deviation : 0.887937 ns
 Execution time std-deviation : 0.916861 ns
Execution time lower quantile : 23.996763 ns ( 2.5%)
Execution time upper quantile : 27.911936 ns (97.5%)
Overhead used : 1.634996 ns

 Found 3 outliers in 60 samples (5. %)
 low-severe 1 (1.6667 %)
 low-mild 1 (1.6667 %)
 high-mild 1 (1.6667 %)
  Variance from outliers : 22.1874 % Variance is moderately inflated by
 outliers
 benching baz
 x86_64 Mac OS X 10.8.3 4 cpu(s)
 Java HotSpot(TM) 64-Bit Server VM 24.0-b28
 Runtime arguments

Re: Functions using locks are slowed even when locks are never taken

[Michał Marczyk]

Well, that is interesting.

The difference between the compiled versions of

(defn foo [x]
  (if ( x 0)
(inc x)
(locking o
  (dec x

and

(defn bar [x]
  (if ( x 0)
(inc x)
(let [res (locking o
(dec x))]
  res)))

is quite significant. foo gets compiled to a single class, with
invocations handled by a single invoke method; bar gets compiled to a
class for bar + an extra class for an inner function which handles the
(locking o (dec x)) part -- probably very similar to the output for
the version with the hand-coded locking-part (although I haven't
really looked at that yet). The inner function is a closure, so
calling it involves an allocation of a closure object; its ctor
receives the closed-over locals as arguments and stores them in two
fields (lockee and x). Then they get loaded from the fields in the
body of the closure's invoke method etc.

I guess I'll have to play around with Java equivalents too...

Cheers,
Michał


On 3 November 2013 20:46, Michael Blume blume.m...@gmail.com wrote:
 I mean, I'm probably being naive, but this suggests that one could write

 (defmacro locking' [ forms]
   `(let [res# (locking ~@forms)] res#))

 and use locking' in place of locking for improved performance. Is this
 wrong? If it's right, does that suggest the macro in clojure.core should be
 changed?


 On Sun, Nov 3, 2013 at 11:09 AM, Michael Blume blume.m...@gmail.com wrote:

 Huh, interesting.

 I have:

 (defn foo' [x]
   (if ( x 0)
 (inc x)
 (let [res (locking o (dec x))] res)))

 (defn foo'' [x]
   (if ( x 0)
 (inc x)
 (locking o
   (dec x

 foo' is fast, but foo'' is slow. So something about wrapping the locking
 clause in a let makes it fast. Still no idea why.

 On Sunday, November 3, 2013 9:30:45 AM UTC-8, Michał Marczyk wrote:

 You have a typo in foo -- monitor-exit's argument is 0 (zero) rather
 than o (the sentinel object).

 Besides that, in foo both monitor-enter and monitor-exit get their
 arguments from a Var. Rewriting to use locking, which first puts the
 object whose monitor will be used in a local (that is, (let [lockee o]
 ...), where ... performs the locking using the newly introduced
 local), gives timings identical to those of bar and baz:

 (defn foo' [x]
   (if ( x 0)
 (inc x)
 (let [res (locking o (dec x))] res)))

 So this is one reason not to use monitor-enter and monitor-exit
 directly. Another reason is that locking guarantees that the monitor
 will be released (by using try / finally, and of course by preventing
 situations where the matching monitor-enter  monitor-exit operate on
 different objects).

 In fact, both monitor-enter and monitor-exit carry docstrings which
 explicitly say that they should not be used in user code and point to
 locking as the user-facing equivalent to Java's synchronized.

 Cheers,
 Michał


 On 1 November 2013 19:34, Michael Blume blume...@gmail.com wrote:
  https://github.com/MichaelBlume/perf-test
 
  (ns perf-test
(:use (criterium core))
(:gen-class))
 
  (def o (Object.))
 
  (defn foo [x]
(if ( x 0)
  (inc x)
  (do
(monitor-enter o)
(let [res (dec x)]
  (monitor-exit 0)
  res
 
  (defn bar [x]
(if ( x 0)
  (inc x)
  (dec x)))
 
  (defn locking-part [x l]
(monitor-enter l)
(let [res (dec x)]
  (monitor-exit l)
  res))
 
  (defn baz [x]
(if ( x 0)
  (inc x)
  (locking-part x o)))
 
  (defn -main []
(println benching foo)
(bench (foo 5) :verbose)
(println benching bar)
(bench (bar 5) :verbose)
(println benching baz)
(bench (baz 5) :verbose)
(println done benching))
 
 
 
  I'm only ever calling these functions with positive values, so the
  monitor-enter branch should never be entered. Nevertheless, the
  performance
  of foo is much worse than bar or baz.
 
  The best guess I've got is that the fact that lock-taking is involved
  somehow changes how the function is compiled, somehow making the
  function
  slower. If the practical upshot is that I shouldn't write functions
  that
  only sometimes lock -- that the locking part of a function should
  always be
  its own function -- then I can do that, but I'm curious why.
 
  $ lein uberjar
  Compiling perf-test
  Created /Users/mike/perf-test/target/perf-test-0.1.0-SNAPSHOT.jar
  Created
  /Users/mike/perf-test/target/perf-test-0.1.0-SNAPSHOT-standalone.jar
  $ java -jar -server target/perf-test-0.1.0-SNAPSHOT-standalone.jar
  benching foo
  WARNING: Final GC required 1.5974571326266802 % of runtime
  x86_64 Mac OS X 10.8.3 4 cpu(s)
  Java HotSpot(TM) 64-Bit Server VM 24.0-b28
  Runtime arguments:
  Evaluation count : 391582560 in 60 samples of 6526376 calls.
Execution time sample mean : 167.426696 ns
   Execution time mean : 167.459429 ns
  Execution time sample std-deviation : 4.079466 ns
  Execution time std-deviation : 4.097819 ns
 Execution time lower quantile : 160.742869

Re: [ANN] avl.clj: sorted maps and sets with fast rank queries via nth

[Michał Marczyk]

As of 0.0.3, avl.clj offers sorted maps with lookups unambiguously
faster than those of the built-ins. Construction times for large
instances are also faster thanks to transients. The lookup performance
comes at a cost in the form of slower non-transient assoc/dissoc.

I've posted some benchmark results to clojure-dev; I'll post a similar
message here once I'm ready to cut an 0.1.0 release (which basically
needs more tests for added confidence, a test suite on the
ClojureScript side and such).

Cheers,
Michał


On 21 September 2013 04:28, Michał Marczyk michal.marc...@gmail.com wrote:
 On 21 September 2013 02:42, Michał Marczyk michal.marc...@gmail.com wrote:
 [avl.clj 0.0.1]

 dependency
   groupIdavl.clj/groupId
   artifactIdavl.clj/artifactId
   version0.0.1/version
 /dependency

 Make that

 [avl.clj 0.0.2]

 and

  dependency
groupIdavl.clj/groupId
artifactIdavl.clj/artifactId
version0.0.2/version
  /dependency

 Cheers,
 Michał

-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: About with-meta source, can explain it by itself?

[Michał Marczyk]

Actually with-meta's definition does not refer to with-meta. Rather,
it calls withMeta, a method in the clojure.lang.IObj interface which
the first argument to with-meta is supposed to implement.

Cheers,
Michał


On 21 September 2013 09:01,  ljcppu...@gmail.com wrote:
 Hi,
   I read the source about with-meta, and find def with-meta using with-meta,
 can it?  someone give a explain?

 user= (source with-meta)
 (def
  ^{:arglists '([^clojure.lang.IObj obj m])
:doc Returns an object of the same type and value as obj, with
 map m as its metadata.
:added 1.0
:static true}
  with-meta (fn ^:static with-meta [^clojure.lang.IObj x m]
  (. x (withMeta m

 --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.

-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: About with-meta source, can explain it by itself?

[Michał Marczyk]

The with-meta in (fn ^:static with-meta ...) is just the name the
function created by the fn form will know itself by. It will also be
used as the final segment of the name of the class of this function
object. It's not being evaluated in this position, and in fact it
could be replaced with, say, foo (and then you'd get mentions of a
class with name ending with foo in stack traces caused by improper
uses of with-meta). On the other hand, the fact that the function has
a name means that it could call itself through this name, return
itself as a value etc.; effectively giving a function a name creates a
local holding the function itself within the function body.

Cheers,
Michał

On 21 September 2013 11:36,  ljcppu...@gmail.com wrote:
  You are right, i noticed  it calls withMeta, but  in the expression,
 actually use with-meta to define with-meta, it's very strange, thank you
 very much!
 (def with-meta (fn ^:static with-meta [^clojure.lang.IObj x m]
   (. x (withMeta m


 On Saturday, September 21, 2013 3:30:30 PM UTC+8, Michał Marczyk wrote:

 Actually with-meta's definition does not refer to with-meta. Rather,
 it calls withMeta, a method in the clojure.lang.IObj interface which
 the first argument to with-meta is supposed to implement.

 Cheers,
 Michał


 On 21 September 2013 09:01,  ljcp...@gmail.com wrote:
  Hi,
I read the source about with-meta, and find def with-meta using
  with-meta,
  can it?  someone give a explain?
 
  user= (source with-meta)
  (def
   ^{:arglists '([^clojure.lang.IObj obj m])
 :doc Returns an object of the same type and value as obj, with
  map m as its metadata.
 :added 1.0
 :static true}
   with-meta (fn ^:static with-meta [^clojure.lang.IObj x m]
   (. x (withMeta m
 
  --
  --
  You received this message because you are subscribed to the Google
  Groups Clojure group.
  To post to this group, send email to clo...@googlegroups.com
  Note that posts from new members are moderated - please be patient with
  your
  first post.
  To unsubscribe from this group, send email to
  clojure+u...@googlegroups.com
  For more options, visit this group at
  http://groups.google.com/group/clojure?hl=en
  ---
  You received this message because you are subscribed to the Google
  Groups
  Clojure group.
  To unsubscribe from this group and stop receiving emails from it, send
  an
  email to clojure+u...@googlegroups.com.
  For more options, visit https://groups.google.com/groups/opt_out.

 --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.

-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[ANN] avl.clj: sorted maps and sets with fast rank queries via nth

[Michał Marczyk]

Hi,

I have just released avl.clj, a library implementing drop-in
replacements for Clojure(Script)'s sorted maps and sets which
additionally support the transients API and logarithmic time rank
queries via clojure.core/nth:

(require '[avl.clj :as avl])

(nth (apply avl/sorted-set (range 10)) 5)
;= 5

Performance is mostly on a par with the built-in red-black-tree-based
variants in my preliminary benchmarking.

If you'd like to take it for a spin, here's the Leiningen/Maven
dependency information for the initial release:

[avl.clj 0.0.1]

dependency
  groupIdavl.clj/groupId
  artifactIdavl.clj/artifactId
  version0.0.1/version
/dependency

Project repository:

https://github.com/michalmarczyk/avl.clj

Artefact's page on Clojars:

https://clojars.org/avl.clj

Cheers,
Michał

-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] avl.clj: sorted maps and sets with fast rank queries via nth

[Michał Marczyk]

On 21 September 2013 02:42, Michał Marczyk michal.marc...@gmail.com wrote:
 [avl.clj 0.0.1]

 dependency
   groupIdavl.clj/groupId
   artifactIdavl.clj/artifactId
   version0.0.1/version
 /dependency

Make that

[avl.clj 0.0.2]

and

 dependency
   groupIdavl.clj/groupId
   artifactIdavl.clj/artifactId
   version0.0.2/version
 /dependency

Cheers,
Michał

-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] Leiningen 2.3.0 released

[Michał Marczyk]

On 10 August 2013 08:39, Michael Klishin michael.s.klis...@gmail.com wrote:
 I personally think there should be a fallback location for the jar that does
 not use SSL.

As long as it isn't used automatically or after flashing a y/n prompt.
(Offering a flag like --no-ssl and mentioning it to the user when
appropriate would be fine.)

To explain, while I don't mind the occasional wait for an upgrade at
all, I would very much mind if lein tricked me into downloading its
jar over a connection not using SSL.

I'll also take this opportunity to note that if one has an earlier
self-install in place, downgrading can be accomplished by replacing
the lein script with the version from the appropriate tag in the
Leiningen repo:

$ cp /path/to/source/of/leiningen  git checkout 2.1.3  cp bin/lein ~/bin

Or go to GitHub and download manually in absence of a local clone.

This works at least in Linux; hopefully adjusting paths (and the
script name on Windows) would make it work on other platforms.

Cheers,
Michał

-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: primitive array casts

[Michał Marczyk]

I think it's more about clojure.core/{ints,longs,...}. These are
basically tools for removing reflection:

(let [xs (ints (get-some-ints))]
   (aget xs 0)) ; no reflection here thanks to the cast

Cheers,
Michał


On 20 July 2013 10:47, Mikera mike.r.anderson...@gmail.com wrote:
 Do you mean something like (int-array [1 2 3 4])?

 This is a cast-like operation that produces a primitive array of ints,
 exactly like int[] in Java. You can then use it with array operations like
 aget.

 If you want to type-hint int-array parameters then you need to use something
 like ^ints, e.g.

 (defn third-int-in-array [^ints xs]
  (aget xs 2))

 On Saturday, 20 July 2013 02:28:15 UTC+1, Brian Craft wrote:

 I'm unable to find any examples of primitive array casts, and the docs are
 not helpful. Can someone point me to examples or docs that explain what they
 do?

 --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.



-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: primitive array casts

[Michał Marczyk]

More complete example:

(defn get-some-ints []
  (int-array [1 2 3]))

(set! *warn-on-reflection* true)

;; causes a reflection warning
(let [xs (get-some-ints)]
  (aget xs 0))

;; do not cause reflection warnings
(let [xs (ints (get-some-ints))]
  (aget xs 0))

(let [xs ^ints (get-some-ints)]
  (aget xs 0))

(let [^ints xs (get-some-ints)]
  (aget xs 0))

Cheers,
Michał


On 20 July 2013 11:25, Michał Marczyk michal.marc...@gmail.com wrote:
 I think it's more about clojure.core/{ints,longs,...}. These are
 basically tools for removing reflection:

 (let [xs (ints (get-some-ints))]
(aget xs 0)) ; no reflection here thanks to the cast

 Cheers,
 Michał


 On 20 July 2013 10:47, Mikera mike.r.anderson...@gmail.com wrote:
 Do you mean something like (int-array [1 2 3 4])?

 This is a cast-like operation that produces a primitive array of ints,
 exactly like int[] in Java. You can then use it with array operations like
 aget.

 If you want to type-hint int-array parameters then you need to use something
 like ^ints, e.g.

 (defn third-int-in-array [^ints xs]
  (aget xs 2))

 On Saturday, 20 July 2013 02:28:15 UTC+1, Brian Craft wrote:

 I'm unable to find any examples of primitive array casts, and the docs are
 not helpful. Can someone point me to examples or docs that explain what they
 do?

 --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.



-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: primitive array casts

[Michał Marczyk]

Also, ^foo-style hints are soft hints in the following sense:

(let [^ints xs [1 2 3]] xs) ; no complaint from compiler or RTE

In contrast, ints  Co. demand that the casts be correct:

(let [xs (ints [1 2 3])] xs) ; ClassCastException


On 20 July 2013 11:29, Michał Marczyk michal.marc...@gmail.com wrote:
 More complete example:

 (defn get-some-ints []
   (int-array [1 2 3]))

 (set! *warn-on-reflection* true)

 ;; causes a reflection warning
 (let [xs (get-some-ints)]
   (aget xs 0))

 ;; do not cause reflection warnings
 (let [xs (ints (get-some-ints))]
   (aget xs 0))

 (let [xs ^ints (get-some-ints)]
   (aget xs 0))

 (let [^ints xs (get-some-ints)]
   (aget xs 0))

 Cheers,
 Michał


 On 20 July 2013 11:25, Michał Marczyk michal.marc...@gmail.com wrote:
 I think it's more about clojure.core/{ints,longs,...}. These are
 basically tools for removing reflection:

 (let [xs (ints (get-some-ints))]
(aget xs 0)) ; no reflection here thanks to the cast

 Cheers,
 Michał


 On 20 July 2013 10:47, Mikera mike.r.anderson...@gmail.com wrote:
 Do you mean something like (int-array [1 2 3 4])?

 This is a cast-like operation that produces a primitive array of ints,
 exactly like int[] in Java. You can then use it with array operations like
 aget.

 If you want to type-hint int-array parameters then you need to use something
 like ^ints, e.g.

 (defn third-int-in-array [^ints xs]
  (aget xs 2))

 On Saturday, 20 July 2013 02:28:15 UTC+1, Brian Craft wrote:

 I'm unable to find any examples of primitive array casts, and the docs are
 not helpful. Can someone point me to examples or docs that explain what 
 they
 do?

 --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.



-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Proposed change to let- syntax

[Michał Marczyk]

On 19 July 2013 10:24, Daniel Dinnyes dinny...@gmail.com wrote:
 Why don't we have a candidate name-space, a separate library, like the
 contrib before, which most people who prefer to be on the cutting edge just
 include automatically. That would give reasonable feedback on how much
 traction new stuff like these get, and keeps it physically obvious what is
 considered The Core.

https://github.com/clojure/core.incubator

Cheers,
Michał

-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.