And to further explicate "bewildering" -- I mean that I only figured out
what was happening because I was explicitly testing a library against
1.7.0-RC1 and even then had to `git bisect` *Clojure* to find the offending
commit. Otherwise the resulting behavior is just that values generated via
the deep guts of a complex Java API suddenly became nonsensical.
On Tue, May 26, 2015 at 9:45 PM Marshall Bockrath-Vandegrift <
llas...@gmail.com> wrote:
> Ugh -- looks like the iterator value re-use behavior for EnumMap entrySet
> was "fixed" post Java 1.6 (my example was under Java 1.6, which I believe
> is still a Clojure-supported version of Java?). I can throw together a
> synthetic example, but I think the people following this thread get what's
> happening. The point isn't whether this pattern is a "good idea" or not
> (it certainly isn't) but whether existing Java APIs people want to interop
> with use it (they certainly do).
>
> I presently depend on no less than 3 separate Java library APIs I
> currently know for a fact depend on this behavior:
> - Hadoop ReduceContextImpl$ValueIterator
> - Mahout DenseVector$AllIterator/NonDefaultIterator
> - LensKit FastIterators
>
> It is an option to explicitly construct `IteratorSeq` instances (I
> actually had verified that approach this afternoon for the Hadoop API in
> Parkour), but I'm not happy about it. That approach places a definite
> burden on integration library maintainers to implement the change, and on
> end-users to realize they need to upgrade for Clojure 1.7 compatibility.
> The `Iterator` interface just fundamentally in no way guarantees that the
> `next()` yielded values are functional-safe in the sense necessary to
> support chunking. I understand the desire to increase performance, but I
> don't think it's worth the potential silent and bewildering breakage in
> interop.
>
>
> On Tue, May 26, 2015 at 9:18 PM Alex Miller <a...@puredanger.com> wrote:
>
>> That's not what I see with 1.7.0-RC1 (or any of the betas). I tried with
>> both Java 1.7.0_25 and 1.8.0-b132.
>>
>> user=> *clojure-version*
>> {:major 1, :minor 7, :incremental 0, :qualifier "RC1"}
>> user=> (->> (map vector (java.util.EnumSet/allOf
>> java.util.concurrent.TimeUnit) (range)) (into {}) (java.util.EnumMap.)
>> (.entrySet) (map str) (into []))
>> ["NANOSECONDS=0" "MICROSECONDS=1" "MILLISECONDS=2" "SECONDS=3"
>> "MINUTES=4" "HOURS=5" "DAYS=6"]
>>
>> Re "implementing the not-uncommon Java pattern of mutating and
>> re-yielding the same object on each `next()` invocation". I'm assuming that
>> you're somehow expecting to traverse one seq node, then having an
>> opportunity to mutate something (the source, the iterator, the return
>> object) in between each new advancement of the seq node? That seems a) not
>> common at all, b) a bad idea even in Java and c) dangerous even before this
>> change. In either case you end up with a seq that points to a succession of
>> the same repeated (mutable and mutating) object - this violates most
>> expectations we as Clojure users have of sequences. Any sort of chunking
>> (map, filter, etc) over the top of that seq would force realization up to
>> 32 elements beyond the head causing the same issue.
>>
>> The original one-at-a-time IteratorSeq is still there (for now) and you
>> can still make one if you want via (clojure.lang.IteratorSeq/create iter)
>> but I would consider it deprecated. I think a custom lazy-seq or a
>> loop-recur would be a better way to handle this case, which in my opinion
>> is highly unusual. That said, my ears are open if this is an issue for a
>> large number of people.
>>
>>
>> On Tuesday, May 26, 2015 at 6:24:54 PM UTC-5, Marshall
>> Bockrath-Vandegrift wrote:
>>>
>>> The difference is that the original behavior allowed room to transform
>>> the mutated object into an object which *could* be safely cached in a
>>> "downstream" seq, while the new behavior pumps the iterator through 32
>>> mutations before user-level code has a chance to see it. Contrived example
>>> using the Java standard libary:
>>>
>>> Clojure 1.6.0:
>>> (->> (map vector (java.util.EnumSet/allOf java.util.concurrent.TimeUnit)
>>> (range)) (into {}) (java.util.EnumMap.) (.entrySet) (map str) (into []))
>>> #=> ["NANOSECONDS=0" "MICROSECONDS=1" "MILLISECONDS=2" "SECONDS=3"
>>> "MINUTES=4" "HOURS=5" "DAYS=6"]
>>>
>>> Clojure 1.7.0-RC1:
>>> (->> (map vector (java.util.EnumSet/allOf java.util.concurrent.TimeUnit)
>>> (range)) (into {}) (java.util.EnumMap.) (.entrySet) (map str) (into []))
>>> #=> ["DAYS=6" "DAYS=6" "DAYS=6" "DAYS=6" "DAYS=6" "DAYS=6" "DAYS=6"]
>>>
>>> IMHO the latter behavior demonstrates a mismatch where chunked seqs and
>>> iterators are simple incompatible.
>>>
>>> On Tue, May 26, 2015 at 5:33 PM Alex Miller <a...@puredanger.com> wrote:
>>>
>>>> In what way is it broken? Both before and after wrapped a mutable
>>>> iterator into a caching seq. The new one is different in that it chunks so
>>>> reads 32 at a time instead of 1. However combining either with other
>>>> chunking sequence operations would have the same effect which is to say
>>>> that using that mutable iterator with anything else, or having expectations
>>>> about its rate of consumption was as dubious before as it is now.
>>>>
>>>> Unless of course I misunderstand your intent, which possible because I
>>>> am on a phone without easy access to look further at the commit and am
>>>> going by memory.
>>>>
>>>>
>>>>
>>>> On May 26, 2015, at 2:17 PM, Marshall Bockrath-Vandegrift <
>>>> llas...@gmail.com> wrote:
>>>>
>>>> Some of my code is broken by
>>>> commit c47e1bbcfa227723df28d1c9e0a6df2bcb0fecc1, which landed in
>>>> 1.7.0-alpha6 (I lasted tested with -alpha5 and have been unfortunately busy
>>>> since). The culprit is the switch to producing seqs over iterators as
>>>> chunked iterators. This would appear to break seq-based traversal of any
>>>> iterator implementing the not-uncommon Java pattern of mutating and
>>>> re-yielding the same object on each `next()` invocation.
>>>>
>>>> I'm unable to find an existing ticket for this apparent-regression.
>>>> Should I create one, or did I miss the existing ticket, or is there some
>>>> mitigating issue which makes this a non-problem?
>>>>
>>>> Thanks.
>>>>
>>>> -Marshall
>>>>
>>>> On Thu, May 21, 2015 at 12:31 PM Alex Miller <a...@puredanger.com>
>>>> wrote:
>>>>
>>>>> Clojure 1.7.0-RC1 is now available.
>>>>>
>>>>> Try it via
>>>>> - Download:
>>>>> https://repo1.maven.org/maven2/org/clojure/clojure/1.7.0-RC1/
>>>>> - Leiningen: [org.clojure/clojure "1.7.0-RC1"]
>>>>>
>>>>> The only change since 1.7.0-beta3 is CLJ-1706, which makes reader
>>>>> conditional splicing an error at the top level (previously it would
>>>>> silently drop all but the first spliced element).
>>>>>
>>>>> For a full list of changes since 1.6.0, see:
>>>>> https://github.com/clojure/clojure/blob/master/changes.md
>>>>>
>>>>> Please give it a try and let us know if things are working (or not).
>>>>> The more and quicker feedback we get, the sooner we can release 1.7.0
>>>>> final!
>>>>>
>>>>> - Alex
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "Clojure Dev" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to clojure-dev+unsubscr...@googlegroups.com.
>>>>> To post to this group, send email to clojure-...@googlegroups.com.
>>>>> Visit this group at http://groups.google.com/group/clojure-dev.
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Clojure Dev" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to clojure-dev+unsubscr...@googlegroups.com.
>>>> To post to this group, send email to clojure-...@googlegroups.com.
>>>> Visit this group at http://groups.google.com/group/clojure-dev.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Clojure Dev" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to clojure-dev+unsubscr...@googlegroups.com.
>>>> To post to this group, send email to clojure-...@googlegroups.com.
>>>> Visit this group at http://groups.google.com/group/clojure-dev.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>> --
>> You received this message because you are subscribed to the Google Groups
>> "Clojure Dev" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to clojure-dev+unsubscr...@googlegroups.com.
>> To post to this group, send email to clojure-...@googlegroups.com.
>> Visit this group at http://groups.google.com/group/clojure-dev.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
