Are you using Aleph in production? Tell us about it.

[Zach Tellman]

We're in the process of figuring out long-term plans for governance of 
Aleph [1], and it would like to understand who the current users are, and 
how they're using the library.  If you are using it, please respond to this 
issue: https://github.com/ztellman/aleph/issues/450.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] 'Elements of Clojure' is complete

[Zach Tellman]

I'm very happy to announce, only two and a half years after the release of 
the first chapter, that Elements of Clojure is completely finished.  
Further details can be found 
here: https://groups.google.com/forum/#!topic/elements-of-clojure/UUJjqU1rllU.

If you've never heard of the book before, please check out its website: 
http://elementsofclojure.com/

Zach

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] A complete draft of "Elements of Clojure" is available

[Zach Tellman]

Almost exactly two years ago, I announced on this mailing list that I was 
working on an intermediate-level book on Clojure [1], and released the 
first chapter.  I've confined updates since then to a book-specific list, 
but I feel like it's fair to post here again to say that the fourth and 
final chapter has been released.  Details can be found on the book's 
mailing 
list: https://groups.google.com/forum/#!topic/elements-of-clojure/t_Uqc8F0Ch0.

If you haven't heard about the book before, please check out its 
website: http://elementsofclojure.com/

Zach


[1] https://groups.google.com/forum/#!msg/clojure/Nh_Z0XaxhTA/P-lPdf2NDQAJ

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN and RFC] Bifurcan: impure functional data strucures

[Zach Tellman]

Are you relying on the immutability of these structures, or are they
effectively always transient?
On Sun, Apr 23, 2017 at 11:02 AM Dave Dixon  wrote:

> FWIW, the use-case I have essentially involves Monte Carlo simulations. So
> we start with a state (non-empty map), and then make a series of
> modifications to it. Various statistics are held in hash-maps keyed by the
> state, so there's a lot of lookups and modifications in those maps.
>
> That said, I'm not sure if for this particular case I care too much using
> Clojure idioms vs. direct API access. The algorithms tend to be
> hand-tweaked for performance anyway. The big win for me in wrapping
> bifurcan would be the ability to use spec without having to write
> specialized specs, generators, etc.
>
>
> On Thursday, April 20, 2017 at 9:53:56 PM UTC-7, Zach Tellman wrote:
>
>> Sure, happy to elaborate.  Bifurcan offers potential performance wins a
>> few different ways:
>>
>> * We can use standard Java equality semantics, bypassing all the overhead
>> of the hash calculations and enhanced numeric equality checks (this can
>> lead to moderate performance gains)
>> * We can use a mutable data structure as long as it never escapes a local
>> context (this can lead to significant performance gains)
>> * We can use the extra capabilities the data structures expose, like
>> concatenation, slicing, set operations, etc. (this is too dependent on the
>> use case to really quantify)
>>
>
>> it would be easy to have a `map` and `map*` method that expose Clojure
>> and Java equality semantics, respectively, but that puts a big onus on the
>> developer to determine if the latter is safe for their use case.  I've been
>> bit by this when I've used j.u.c.ConcurrentHashMap before, so I expect
>> people will suffer similarly weird bugs.
>>
>> However, I think there's a way to use the mutable data structures.
>> Technically, transient data structures allow arbitrary persistent data
>> structures to be batch updated, but in practice they tend to be empty, and
>> after they're populated they tend to be treated as read-only.
>>
>> If we're convinced this is common enough, every empty transient data
>> structure could be mutable, and when we make it persistent we could wrap it
>> in a "virtual" collection [1] which allows updates without touching the
>> base collection.  This would allow for faster writes, faster reads, and
>> only marginally slower updates if those are required.
>>
>> This is all predicated on a bunch of assumptions that are hard to
>> validate, but if this describes enough real-world use cases, it could lead
>> to a big, easy performance win.  It's even possible to automatically
>> replace the base Clojure collections with these alternatives using
>> something like Sleight [2].
>>
>> Anyway, that's what I've been mulling over.  If anyone has opinions, I'm
>> happy to hear them.
>>
>> Zach
>>
>> [1]
>> https://github.com/lacuna/bifurcan/blob/master/src/io/lacuna/bifurcan/Maps.java#L103
>> [2] https://github.com/ztellman/sleight
>>
>> On Thu, Apr 20, 2017 at 8:55 AM Dave Dixon  wrote:
>>
>>> Sounds great. If you have time, I'd certainly like to hear your thoughts
>>> on the issues of equality semantics and transients, maybe I can ponder and
>>> make some suggestions based on my target use-case.
>>>
>>>
>>> On Tuesday, April 18, 2017 at 9:32:32 AM UTC-7, Zach Tellman wrote:
>>>
>>>> To be clear, my intention was always to wrap the implementations in the
>>>> appropriate Clojure interfaces, and I don't believe that will cause much,
>>>> if any, of a performance hit (inlining is magic).  However, there are some
>>>> real questions regarding how to expose non-standard equality semantics, and
>>>> whether transients should be represented using the immutable or mutable
>>>> collection variants.
>>>>
>>>> For what it's worth, I have about 1/3 of an implementation of
>>>> Clojure-compatible versions of these data structures, I just wanted to mull
>>>> on the above questions a bit before going further.  I'm happy to discuss
>>>> them here in more depth if you have any questions or opinions.
>>>>
>>>> Zach
>>>>
>>>> On Tue, Apr 18, 2017 at 6:53 AM Dave Dixon  wrote:
>>>>
>>> Stared at this a bit yesterday. Seems like if you want to leverage spec
>>>>> while using bifurcan, the

Re: [ANN and RFC] Bifurcan: impure functional data strucures

[Zach Tellman]

Sure, happy to elaborate.  Bifurcan offers potential performance wins a few
different ways:

* We can use standard Java equality semantics, bypassing all the overhead
of the hash calculations and enhanced numeric equality checks (this can
lead to moderate performance gains)
* We can use a mutable data structure as long as it never escapes a local
context (this can lead to significant performance gains)
* We can use the extra capabilities the data structures expose, like
concatenation, slicing, set operations, etc. (this is too dependent on the
use case to really quantify)

it would be easy to have a `map` and `map*` method that expose Clojure and
Java equality semantics, respectively, but that puts a big onus on the
developer to determine if the latter is safe for their use case.  I've been
bit by this when I've used j.u.c.ConcurrentHashMap before, so I expect
people will suffer similarly weird bugs.

However, I think there's a way to use the mutable data structures.
Technically, transient data structures allow arbitrary persistent data
structures to be batch updated, but in practice they tend to be empty, and
after they're populated they tend to be treated as read-only.

If we're convinced this is common enough, every empty transient data
structure could be mutable, and when we make it persistent we could wrap it
in a "virtual" collection [1] which allows updates without touching the
base collection.  This would allow for faster writes, faster reads, and
only marginally slower updates if those are required.

This is all predicated on a bunch of assumptions that are hard to validate,
but if this describes enough real-world use cases, it could lead to a big,
easy performance win.  It's even possible to automatically replace the base
Clojure collections with these alternatives using something like Sleight
[2].

Anyway, that's what I've been mulling over.  If anyone has opinions, I'm
happy to hear them.

Zach

[1]
https://github.com/lacuna/bifurcan/blob/master/src/io/lacuna/bifurcan/Maps.java#L103
[2] https://github.com/ztellman/sleight

On Thu, Apr 20, 2017 at 8:55 AM Dave Dixon  wrote:

> Sounds great. If you have time, I'd certainly like to hear your thoughts
> on the issues of equality semantics and transients, maybe I can ponder and
> make some suggestions based on my target use-case.
>
>
> On Tuesday, April 18, 2017 at 9:32:32 AM UTC-7, Zach Tellman wrote:
>
>> To be clear, my intention was always to wrap the implementations in the
>> appropriate Clojure interfaces, and I don't believe that will cause much,
>> if any, of a performance hit (inlining is magic).  However, there are some
>> real questions regarding how to expose non-standard equality semantics, and
>> whether transients should be represented using the immutable or mutable
>> collection variants.
>>
>> For what it's worth, I have about 1/3 of an implementation of
>> Clojure-compatible versions of these data structures, I just wanted to mull
>> on the above questions a bit before going further.  I'm happy to discuss
>> them here in more depth if you have any questions or opinions.
>>
>> Zach
>>
>> On Tue, Apr 18, 2017 at 6:53 AM Dave Dixon  wrote:
>>
> Stared at this a bit yesterday. Seems like if you want to leverage spec
>>> while using bifurcan, then the bifurcan types need to have the Clojure
>>> wrapper. The alternative appears to be re-implementing at least a large
>>> subset of collection-related spec code, which is a lot to bite off. Also
>>> tried updating some existing code to use bifurcan. Similar to spec, there
>>> are going to be cases which are less perf sensitive, where it would be nice
>>> to use code that is polymorphic for collections, and drop down to the fast
>>> interface in perf-sensitive parts.
>>>
>>>
>>> On Monday, April 17, 2017 at 1:52:39 PM UTC-7, Dave Dixon wrote:
>>>>
>>>> What is the issue with wrapping in Clojure interfaces? Added overhead
>>>> of function calls?
>>>>
>>>> I'm finding myself in the process of doing some of this, at least for
>>>> constructors. Also thinking of generating predicates/generators for use
>>>> with spec.
>>>>
>>>> On Monday, March 27, 2017 at 9:51:46 AM UTC-7, Zach Tellman wrote:
>>>>
>>>>> This is a slightly irregular announcement, because it's not for a
>>>>> Clojure library.  Rather, it's for a library written purely in Java:
>>>>> https://github.com/lacuna/bifurcan.
>>>>>
>>>>> This is a collection of mutable and immutable data structures,
>>>>> designed to address some of my 

Re: [ANN and RFC] Bifurcan: impure functional data strucures

[Zach Tellman]

To be clear, my intention was always to wrap the implementations in the
appropriate Clojure interfaces, and I don't believe that will cause much,
if any, of a performance hit (inlining is magic).  However, there are some
real questions regarding how to expose non-standard equality semantics, and
whether transients should be represented using the immutable or mutable
collection variants.

For what it's worth, I have about 1/3 of an implementation of
Clojure-compatible versions of these data structures, I just wanted to mull
on the above questions a bit before going further.  I'm happy to discuss
them here in more depth if you have any questions or opinions.

Zach

On Tue, Apr 18, 2017 at 6:53 AM Dave Dixon  wrote:

> Stared at this a bit yesterday. Seems like if you want to leverage spec
> while using bifurcan, then the bifurcan types need to have the Clojure
> wrapper. The alternative appears to be re-implementing at least a large
> subset of collection-related spec code, which is a lot to bite off. Also
> tried updating some existing code to use bifurcan. Similar to spec, there
> are going to be cases which are less perf sensitive, where it would be nice
> to use code that is polymorphic for collections, and drop down to the fast
> interface in perf-sensitive parts.
>
>
> On Monday, April 17, 2017 at 1:52:39 PM UTC-7, Dave Dixon wrote:
>>
>> What is the issue with wrapping in Clojure interfaces? Added overhead of
>> function calls?
>>
>> I'm finding myself in the process of doing some of this, at least for
>> constructors. Also thinking of generating predicates/generators for use
>> with spec.
>>
>> On Monday, March 27, 2017 at 9:51:46 AM UTC-7, Zach Tellman wrote:
>>
>>> This is a slightly irregular announcement, because it's not for a
>>> Clojure library.  Rather, it's for a library written purely in Java:
>>> https://github.com/lacuna/bifurcan.
>>>
>>> This is a collection of mutable and immutable data structures, designed
>>> to address some of my personal frustrations with what's available in the
>>> Clojure and Java ecosystems.  Notably, they have pluggable equality
>>> semantics, so while they *can* use Clojure's expensive hash and equality
>>> checks, they don't *have* to.  They also provide high-performance mutable
>>> variants of the data structure which share an API with their immutable
>>> cousins.
>>>
>>> I'm posting it here to ask for people's thoughts on how, if at all, this
>>> should be exposed as a Clojure library.  It would be simple to simply wrap
>>> them in the Clojure interfaces and make them behave identically to
>>> Clojure's own data structures, but that kind of obviates the point.
>>> However, creating an entirely new set of accessors means that we can't
>>> leverage Clojure's standard library.
>>>
>>> It's possible that I'm alone in my frustrations, and no Clojure wrapper
>>> is necessary.  But if this does solve a problem you have, I'd like to hear
>>> more about what it is, and how you think Bifurcan might help.  Please feel
>>> free to reply here, or to grab me at Clojure/West and talk about it there.
>>>
>>> Thanks in advance,
>>> Zach
>>>
>> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/1m_I7IrDGb0/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN and RFC] Bifurcan: impure functional data strucures

[Zach Tellman]

Both the 'List' and 'Map' data structures in Bifurcan use innovative
approaches that were published after Clojure's original release [1] [2].
In the case of the immutable map, you get faster iteration and the
structural invariants allow for some clever stuff w.r.t. equality checks
and set operations.  In the case of the immutable list/vector, you get fast
concatenation, the ability to add and remove from both ends of the
collection, and a `subvec` that doesn't hold onto the entire underlying
data structure.

All of this is MIT licensed, so please feel welcome to open a PR against
Clojure to change the core data structures using my code, but I'd rate the
chance of that being accepted as somewhere between "low" and
"nonexistent".  Also, it should be noted that Clojure's implementation is
much more battle-tested than my own at this point.  But if anyone wants to
tilt at that particular windmill, feel free to ask me any questions you may
have about the implementation.

Zach

[1] https://michael.steindorfer.name/publications/oopsla15.pdf
[2] https://infoscience.epfl.ch/record/169879/files/RMTrees.pdf

On Mon, Mar 27, 2017 at 12:30 PM Luke Burton  wrote:

>
> I'm not well versed enough in these data structures to know this without
> asking (apologies if it's really obvious to some people): is there
> opportunity to improve Clojure's built-in data structures with Bifurcan
> rather than trying to wrap Bifurcan's structures in Clojure?
>
> As an aside, I want to draw people's attention to the sweet little
> criterium + gnuplot setup you have there for generating benchmarking plots.
> Nice!
>
>
> On Mar 27, 2017, at 10:13 AM, Zach Tellman  wrote:
>
> Benchmarks are available here, and the Clojure benchmarks make use of
> transients wherever possible:
> https://github.com/lacuna/bifurcan/blob/master/doc/benchmarks.md.
>
> More generally, while transients are often used in practice to quickly
> construct a read-only data structure, the more formal definition is that
> they provide an O(1) mechanism for transforming between immutable and
> mutable forms.  This isn't possible with purely mutable data structures
> like Java's HashMap or Bifurcan's LinearMap.  So while wrapping these data
> structures in the Clojure API would provide better performance for
> construction and lookups, it wouldn't be quite the same thing as a
> transient.
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/1m_I7IrDGb0/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN and RFC] Bifurcan: impure functional data strucures

[Zach Tellman]

Benchmarks are available here, and the Clojure benchmarks make use of
transients wherever possible:
https://github.com/lacuna/bifurcan/blob/master/doc/benchmarks.md.

More generally, while transients are often used in practice to quickly
construct a read-only data structure, the more formal definition is that
they provide an O(1) mechanism for transforming between immutable and
mutable forms.  This isn't possible with purely mutable data structures
like Java's HashMap or Bifurcan's LinearMap.  So while wrapping these data
structures in the Clojure API would provide better performance for
construction and lookups, it wouldn't be quite the same thing as a
transient.



On Mon, Mar 27, 2017 at 10:05 AM Michael Gardner 
wrote:

>
> > On Mar 27, 2017, at 09:51, Zach Tellman  wrote:
> >
> > They also provide high-performance mutable variants of the data
> structure which share an API with their immutable cousins.
>
> How does their performance compare to Clojure's transients? Transients are
> slower than Java's native mutable collections, so if the mutable
> collections in this library deliver the same performance as the latter,
> they could act as a drop-in replacement for the former (given a compatible
> Clojure wrapper).
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/1m_I7IrDGb0/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN and RFC] Bifurcan: impure functional data strucures

[Zach Tellman]

This is a slightly irregular announcement, because it's not for a Clojure 
library.  Rather, it's for a library written purely in 
Java: https://github.com/lacuna/bifurcan.

This is a collection of mutable and immutable data structures, designed to 
address some of my personal frustrations with what's available in the 
Clojure and Java ecosystems.  Notably, they have pluggable equality 
semantics, so while they *can* use Clojure's expensive hash and equality 
checks, they don't *have* to.  They also provide high-performance mutable 
variants of the data structure which share an API with their immutable 
cousins.  

I'm posting it here to ask for people's thoughts on how, if at all, this 
should be exposed as a Clojure library.  It would be simple to simply wrap 
them in the Clojure interfaces and make them behave identically to 
Clojure's own data structures, but that kind of obviates the point. 
 However, creating an entirely new set of accessors means that we can't 
leverage Clojure's standard library.

It's possible that I'm alone in my frustrations, and no Clojure wrapper is 
necessary.  But if this does solve a problem you have, I'd like to hear 
more about what it is, and how you think Bifurcan might help.  Please feel 
free to reply here, or to grab me at Clojure/West and talk about it there.

Thanks in advance,
Zach

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] Virgil 0.1.6

[Zach Tellman]

The code is split into a library and a minimal Leiningen plugin which runs 
it at startup.  If someone wants to contribute a Boot equivalent, I'm happy 
to accept that PR.

On Thursday, March 16, 2017 at 11:18:16 AM UTC-7, Gregg Reynolds wrote:
>
>
>
> On Mar 16, 2017 1:13 PM, "Zach Tellman" > 
> wrote:
>
> I figured it was worth reminding everyone that this library exists: 
> https://github.com/ztellman/virgil.  It now seems to work on Java 
> projects of arbitrary size and structure (the in-process compiler is very 
> fussy about compile order, you need to topologically sort the classes), and 
> I've been using it extensively this last month while writing a bunch of 
> Java [1].  I can tweak some Java, wait a second, and then test that code at 
> the REPL.  It's been pretty game-changing for my workflow.
>
> If anyone has questions, I'm happy to answer them.
>
>
> wow!  looks great, when's the boot version due out? 😊
>
>
> Zach
>
> [1] https://github.com/lacuna/bifurcan
>
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com 
> Note that posts from new members are moderated - please be patient with 
> your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com 
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> --- 
> You received this message because you are subscribed to the Google Groups 
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to clojure+u...@googlegroups.com .
> For more options, visit https://groups.google.com/d/optout.
>
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] Virgil 0.1.6

[Zach Tellman]

I figured it was worth reminding everyone that this library 
exists: https://github.com/ztellman/virgil.  It now seems to work on Java 
projects of arbitrary size and structure (the in-process compiler is very 
fussy about compile order, you need to topologically sort the classes), and 
I've been using it extensively this last month while writing a bunch of 
Java [1].  I can tweak some Java, wait a second, and then test that code at 
the REPL.  It's been pretty game-changing for my workflow.

If anyone has questions, I'm happy to answer them.

Zach

[1] https://github.com/lacuna/bifurcan

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] Aleph 0.4.1

[Zach Tellman]

This is considerably simpler than the suggestions in this thread, but I've 
written a very basic statsd client and server in the literate 
examples: http://aleph.io/aleph/literate.html#aleph.examples.udp.  Feedback 
is welcome.

On Saturday, April 2, 2016 at 11:44:53 PM UTC-7, Zach Tellman wrote:
>
> This release represents a number of incremental improvements to 0.4.0, 
> which has been handling billions of daily requests for close to a year.  
>
> * Documentation can be found at http://aleph.io/
> * Literate examples of usage can be found at 
> http://aleph.io/aleph/literate.html
> * Comparative benchmarks can be found at 
> https://www.techempower.com/benchmarks/#section=data-r12&hw=peak&test=plaintext&l=4,
>  
> which may or may not be relevant to your particular use case
>
> If anyone has questions, I'm happy to answer them.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] Aleph 0.4.1

[Zach Tellman]

Hi Adrian,

Glad to hear that you're getting good use out of Aleph.  As for the UDP
side of things, if you're using it (as games often do) to reimplement half
of TCP, that may not work well with the functional operators in
manifold.stream.  However, using `put!`, `take!`, and a few operators from
manifold.deferred like `chain` and `loop`, you should be able to easily
build any arbitrary call and response logic you need.  The code you write
will end up being pretty imperative, but sometimes that's just the best way
to reason about your problem.

I'm not sure I'll have time to look at the spec you linked anytime soon,
but I think it's reasonable to have some UDP-related code in the literate
examples.  I'll think about what would be a decent use case.

Best,
Zach

On Sun, Apr 3, 2016 at 12:52 PM  wrote:

> Awesome! At Vital Labs we use Aleph in production for our HTTP and (soon
> to be) websocket services. I have nothing but good things to say about it.
> It makes using Netty a breeze in Clojure.
>
> For an unrelated side project, I have been using Aleph to communicate over
> UDP with an old MMORPG called Star Wars Galaxies. Since UDP is not stream
> oriented, some of the architecture which make Aleph easy to reason about
> through Manifold for HTTP/TCP do not seem to carry over as cleanly when
> dealing with UDP services.
>
> Would you be open to writing a comprehensive UDP example for Aleph so that
> potential users could see how the author intends such services to be
> written in tandem with your other libraries, namely Manifold, Gloss, and
> byte-streams?
>
> In particular, SWG uses a convoluted protocol which involves optional XOR
> decryption of the body using an integer exchanged in the first packet you
> see from the game client (and saved for all future communication with the
> client until they go link dead), followed by optional decompression with
> DEFLATE of the body (not the header nor footer) depending on a value in the
> decrypted first byte of the footer, followed by variable-length (opcode
> driven) body decoding. To deal with the problem of unreliable transmission,
> you also need to ack sequenced packets before responding to a specific kind
> of message. Sometimes this needs to be repeated due to loss, etc. To
> respond you also need to invert the decoding process, so compress ->
> encrypt -> etc.
>
> Here's an overview of the protocol if you're interested in seeing the
> potential complexities that come up with protocols like these:
> http://wiki.swganh.org/index.php/Packet_Guides
>
> Since many of these requirements break the more functional, streaming
> nature of Manifold's design I have found that the code quickly devolves
> into madness. I imagine it would be very enlightening to see how you would
> solve these or similar problems with Aleph.
>
> Thanks for the great library,
> Adrian
>
> On Sunday, April 3, 2016 at 2:44:53 AM UTC-4, Zach Tellman wrote:
>>
>> This release represents a number of incremental improvements to 0.4.0,
>> which has been handling billions of daily requests for close to a year.
>>
>> * Documentation can be found at http://aleph.io/
>> * Literate examples of usage can be found at
>> http://aleph.io/aleph/literate.html
>> * Comparative benchmarks can be found at
>> https://www.techempower.com/benchmarks/#section=data-r12&hw=peak&test=plaintext&l=4,
>> which may or may not be relevant to your particular use case
>>
>> If anyone has questions, I'm happy to answer them.
>>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/cNRTnvlPVG4/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] Aleph 0.4.1

[Zach Tellman]

This release represents a number of incremental improvements to 0.4.0, 
which has been handling billions of daily requests for close to a year.  

* Documentation can be found at http://aleph.io/
* Literate examples of usage can be found 
at http://aleph.io/aleph/literate.html
* Comparative benchmarks can be found 
at 
https://www.techempower.com/benchmarks/#section=data-r12&hw=peak&test=plaintext&l=4,
 
which may or may not be relevant to your particular use case

If anyone has questions, I'm happy to answer them.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] Elements of Clojure

[Zach Tellman]

I changed the link to the Leanpub one. I think the content type may have
been wrong before, which maybe Android is overly sensitive to. Glad it's
working now.
On Sat, Mar 19, 2016 at 6:24 PM Gregg Reynolds  wrote:

>
> On Mar 19, 2016 4:16 PM, "Zach Tellman"  wrote:
> >
> > The files have worked everywhere I've tried them. I've linked everything
> to the Leanpub download, just to reduce any change of divergent behavior,
> but I'm not sure there's more I can do. Thank you for the report, though,
> I'm sure you're not the only one seeing this.
>
> dunno what happened but now when I follow the link to you page and then
> click on the first chapter link it comes up immediately.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/Nh_Z0XaxhTA/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] Elements of Clojure

[Zach Tellman]

The files have worked everywhere I've tried them. I've linked everything to
the Leanpub download, just to reduce any change of divergent behavior, but
I'm not sure there's more I can do. Thank you for the report, though, I'm
sure you're not the only one seeing this.
On Sat, Mar 19, 2016 at 2:08 PM Gregg Reynolds  wrote:

>
> On Mar 19, 2016 3:46 PM, "Zach Tellman"  wrote:
> >
> > Do you get the same error for
> http://samples.leanpub.com/elementsofclojure-sample.pdf?
> >
>
> Oy.  it might be the email client, who knows?  when I click on the
> leanpub.com link in my email client I get a "downloading" msg, then
> nothing.  when I go to the "myfiles" folder I see several versions of
> elemwntsofclojure, and I can open them all.  no idea what's going on but
> I'm inclined to blame samsung.
>
> fwiw I've seen this with many other pdf files, some work, some dont.  so I
> guess the lesson is just make a reasonable check that your file are good
> (run them through some version of acrobat? ) and blame any prob on
> cellphone vendors ;).
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/Nh_Z0XaxhTA/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] Elements of Clojure

[Zach Tellman]

Do you get the same error for
http://samples.leanpub.com/elementsofclojure-sample.pdf?

On Sat, Mar 19, 2016 at 1:40 PM Gregg Reynolds  wrote:

>
> On Mar 19, 2016 3:34 PM, "Gregg Reynolds"  wrote:
> >
> >
> > On Mar 19, 2016 3:32 PM, "Zach Tellman"  wrote:
> > >
> > > Is this the PDF from elementsofclojure.com or leanpub.com?  They
> should be the same, but I just want to make sure I know all the details.
> >
> > the former.
>
> fwiw, I get the same result with the mobi link and the epub link: "invalid
> format".
>
> > >
> > > On Sat, Mar 19, 2016 at 1:07 PM Gregg Reynolds 
> wrote:
> > >>
> > >>
> > >> On Mar 17, 2016 12:47 PM, "Zach Tellman"  wrote:
> > >> >
> > >> > I'm writing a book about Clojure, aimed at people who already know
> the core concepts, and want to use them more effectively.  The first
> chapter, "Names", is complete and can be read for free.  Details can be
> found at http://elementsofclojure.com/.
> > >>
> > >> fyi on my galaxy s5 the pdf is declared invalid so I can't read it.
> > >>
> > >> >
> > >> > I'm happy to answer any questions here, or on the book's mailing
> list at https://groups.google.com/forum/#!forum/elements-of-clojure.
> > >> >
> > >> > --
> > >> > You received this message because you are subscribed to the Google
> > >> > Groups "Clojure" group.
> > >> > To post to this group, send email to clojure@googlegroups.com
> > >> > Note that posts from new members are moderated - please be patient
> with your first post.
> > >> > To unsubscribe from this group, send email to
> > >> > clojure+unsubscr...@googlegroups.com
> > >> > For more options, visit this group at
> > >> > http://groups.google.com/group/clojure?hl=en
> > >> > ---
> > >>
> > >> > You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> > >>
> > >> > To unsubscribe from this group and stop receiving emails from it,
> send an email to clojure+unsubscr...@googlegroups.com.
> > >>
> > >>
> > >> > For more options, visit https://groups.google.com/d/optout.
> > >>
> > >> --
> > >> You received this message because you are subscribed to the Google
> > >> Groups "Clojure" group.
> > >> To post to this group, send email to clojure@googlegroups.com
> > >> Note that posts from new members are moderated - please be patient
> with your first post.
> > >> To unsubscribe from this group, send email to
> > >> clojure+unsubscr...@googlegroups.com
> > >> For more options, visit this group at
> > >> http://groups.google.com/group/clojure?hl=en
> > >> ---
> > >> You received this message because you are subscribed to a topic in
> the Google Groups "Clojure" group.
> > >> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/Nh_Z0XaxhTA/unsubscribe.
> > >> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> > >>
> > >> For more options, visit https://groups.google.com/d/optout.
> > >
> > > --
> > > You received this message because you are subscribed to the Google
> > > Groups "Clojure" group.
> > > To post to this group, send email to clojure@googlegroups.com
> > > Note that posts from new members are moderated - please be patient
> with your first post.
> > > To unsubscribe from this group, send email to
> > > clojure+unsubscr...@googlegroups.com
> > > For more options, visit this group at
> > > http://groups.google.com/group/clojure?hl=en
> > > ---
> > > You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> > > To unsubscribe from this group and stop receiving emails from it, send
> an email to clojure+unsubscr...@googlegroups.com.
> > > For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from thi

Re: [ANN] Elements of Clojure

[Zach Tellman]

Is this the PDF from elementsofclojure.com or leanpub.com?  They should be
the same, but I just want to make sure I know all the details.

On Sat, Mar 19, 2016 at 1:07 PM Gregg Reynolds  wrote:

>
> On Mar 17, 2016 12:47 PM, "Zach Tellman"  wrote:
> >
> > I'm writing a book about Clojure, aimed at people who already know the
> core concepts, and want to use them more effectively.  The first chapter,
> "Names", is complete and can be read for free.  Details can be found at
> http://elementsofclojure.com/.
>
> fyi on my galaxy s5 the pdf is declared invalid so I can't read it.
>
> >
> > I'm happy to answer any questions here, or on the book's mailing list at
> https://groups.google.com/forum/#!forum/elements-of-clojure.
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "Clojure" group.
> > To post to this group, send email to clojure@googlegroups.com
> > Note that posts from new members are moderated - please be patient with
> your first post.
> > To unsubscribe from this group, send email to
> > clojure+unsubscr...@googlegroups.com
> > For more options, visit this group at
> > http://groups.google.com/group/clojure?hl=en
> > ---
>
> > You received this message because you are subscribed to the Google
> Groups "Clojure" group.
>
> > To unsubscribe from this group and stop receiving emails from it, send
> an email to clojure+unsubscr...@googlegroups.com.
>
>
> > For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/Nh_Z0XaxhTA/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] Elements of Clojure

[Zach Tellman]

Thanks Bozhidar.  I agree that the "default name" sections can be expanded
a bit, and some related rules may also fall into the second chapter.  I
don't think I'll stray too far from what's in your guide.

On Sat, Mar 19, 2016 at 11:05 AM Bozhidar Batsov 
wrote:

> Seems to me that some more Clojure-specific naming rules can be
> incorporated into the first chapter (e.g. like the ones we have here
> https://github.com/bbatsov/clojure-style-guide/#naming). Other than this
> small remark - excellent work!
>
> On 19 March 2016 at 13:20, Val Waeselynck  wrote:
>
>> The chapter on naming is brilliant, I rarely learned so much in so few
>> pages :)
>>
>> So far I have found the content to be more about 'philosophical'
>> programming notions than Clojure specifically, but the parts about Clojure
>> are useful and pratical.
>>
>> You may want to state more explicitly how this books positions itself
>> relative to others.
>>
>> Looking forward to the rest,
>>
>> Val
>>
>>
>> On Thursday, 17 March 2016 18:47:55 UTC+1, Zach Tellman wrote:
>>>
>>> I'm writing a book about Clojure, aimed at people who already know the
>>> core concepts, and want to use them more effectively.  The first chapter,
>>> "Names", is complete and can be read for free.  Details can be found at
>>> http://elementsofclojure.com/.
>>>
>>> I'm happy to answer any questions here, or on the book's mailing list at
>>> https://groups.google.com/forum/#!forum/elements-of-clojure.
>>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clojure@googlegroups.com
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>>
> You received this message because you are subscribed to the Google Groups
>> "Clojure" group.
>>
> To unsubscribe from this group and stop receiving emails from it, send an
>> email to clojure+unsubscr...@googlegroups.com.
>>
>
>> For more options, visit https://groups.google.com/d/optout.
>>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/Nh_Z0XaxhTA/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] Elements of Clojure

[Zach Tellman]

I'm writing a book about Clojure, aimed at people who already know the core 
concepts, and want to use them more effectively.  The first chapter, 
"Names", is complete and can be read for free.  Details can be found 
at http://elementsofclojure.com/.  

I'm happy to answer any questions here, or on the book's mailing list 
at https://groups.google.com/forum/#!forum/elements-of-clojure.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] Elements of Clojure

[Zach Tellman]

To quote from the introduction:

> Where possible, this book will give specific, prescriptive advice on how
to write idiomatic Clojure. Everywhere else, it will describe the space of
possible approaches, and provide a framework for deciding which to use.

In my mind, a style guide confines itself to prescriptive advice, and
ignores anything which doesn't lend itself to that.  The latter part, which
is definitely demonstrated in the first chapter, makes this more than a
book about style.

However, it is definitely focused on giving practical advice about how to
write Clojure.  I would hope that the material isn't too surprising, given
the introduction.  If you feel like something is unclear or misleading,
please let me know what it is.

Zach

On Thu, Mar 17, 2016 at 12:10 PM JvJ  wrote:

> So far, this appears to be primarily focused on style and programming
> practice.  Is that going to be the primary focus of the book?
>
>
> On Thursday, 17 March 2016 10:47:55 UTC-7, Zach Tellman wrote:
>>
>> I'm writing a book about Clojure, aimed at people who already know the
>> core concepts, and want to use them more effectively.  The first chapter,
>> "Names", is complete and can be read for free.  Details can be found at
>> http://elementsofclojure.com/.
>>
>> I'm happy to answer any questions here, or on the book's mailing list at
>> https://groups.google.com/forum/#!forum/elements-of-clojure.
>>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/Nh_Z0XaxhTA/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Help] core.async and jetty

[Zach Tellman]

For what it's worth, Aleph will handle pipelined requests in 
parallel: https://github.com/ztellman/aleph.

On Tuesday, February 23, 2016 at 4:46:23 AM UTC-8, Miguel Ping wrote:
>
> Thanks, thats what I eventually found out.
>
> On Tuesday, February 23, 2016 at 12:39:00 PM UTC, jonah wrote:
>>
>> Hi Miguel- pipelining is essentially http keep alive. A very old jetty 
>> thread
>>
>>
>> http://jetty.4.x6.nabble.com/HTTP-1-1-Request-Pipelining-handling-td18682.html
>>  
>> 
>>
>> indicates that for simplicity jetty will execute subsequent requests on 
>> the kept-alive socket serially. 
>>
>> Jonah
>>   
>>
>> On Mon, Feb 22, 2016 at 3:33 PM, Miguel Ping  wrote:
>>
>>> Hi guys,
>>>
>>> I'm trying to replicate an experiment on nodejs and http pipelining: 
>>> http://blog.yld.io/2016/02/08/squeeze-the-juice-out-of-node/
>>>
>>> This is what I got right now: 
>>> https://gist.github.com/mping/98bb8eb9faf3c51f9889 (using 
>>> *com.ninjudd/ring-async*)
>>>
>>> Problem is I can't get pipelining to work as in nodejs; by doing two 
>>> reqs they are sequential, ie, the dates that I'm printing have two secs.
>>> Basically I want that the server prints two "identical" accpt dates, and 
>>> the  response should show that (now not working):
>>>
>>> $ tail -f reqs.txt | nc 127.0.0.1    
>>> 
>>>   
>>> HTTP/1.1 200 OK
>>> Date: Mon, 22 Feb 2016 20:29:10 GMT
>>> Transfer-Encoding: chunked
>>> Server: Jetty(7.6.8.v20121106)
>>>
>>> 48
>>> id: 1, uri: /a
>>> Mon Feb 22 20:29:10 WET 2016
>>> Mon Feb 22 20:29:12 WET 2016
>>> 0
>>>
>>> HTTP/1.1 200 OK
>>> Date: Mon, 22 Feb 2016 20:29:12 GMT
>>> Transfer-Encoding: chunked
>>> Server: Jetty(7.6.8.v20121106)
>>>
>>> 48
>>> id: 2, uri: /b
>>> Mon Feb 22 20:29:12 WET 2016
>>> Mon Feb 22 20:29:14 WET 2016
>>>
>>>
>>> I'm guessing that this is either my mistake with core.async or the 
>>> adapter.
>>>
>>> Thanks
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To post to this group, send email to clo...@googlegroups.com
>>> Note that posts from new members are moderated - please be patient with 
>>> your first post.
>>> To unsubscribe from this group, send email to
>>> clojure+u...@googlegroups.com
>>> For more options, visit this group at
>>> http://groups.google.com/group/clojure?hl=en
>>> --- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Clojure" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to clojure+u...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: What's the best option similar to Vert.x, Reactor, Nodejs for use with Clojure?

[Zach Tellman]

This is incidental to the topic of the thread, but to address why Manifold 
exists when there's already core.async, there's a write-up of the rationale 
linked in the readme: http://aleph.io/manifold/rationale.html.

On Saturday, January 23, 2016 at 6:50:34 AM UTC-5, qsys wrote:
>
> Depends on what you want... what I really like is the easy modularity: 
> deploy new 'verticles' somewhere in your network, and they're just all 
> connected through a the eventbus. I make a 'new' module, I put it somewhere 
> and it's automatically picked up by the appication and I can communicate 
> with it using that event bus. Is there something similar in clojure, or can 
> I achieve something similar in clojure? I checked 
> - catacumba: web toolkit, not what I'm after
> - sente: is about the web
> - aleph: comes closer: does have tcp-servers, but no autodiscovery, and 
> well, no 'event bus' (a kind of wrapper around netty etc).
> - manifold: I don't really see the added value, having core.async - but I 
> may fail to see something important here
> - pomegranate: is possibly very interesting in adding new modules... not 
> for removing 'old versions' of a module.
> - pedestal: have to have a deeper look into it, but so far, I have a 
> feeling it's more about web than 'intra-program' communication
>
> So, so far, I don't see how to implement something like the vert.x event 
> bus, with autodiscovery, in clojure (although I would love to see something 
> like it), expanding to the browser. I don't mind (and prefer) composing it 
> using different libraries, but I feel to see how to have this functionality 
> in clojure (now, I wrap the vertx eventbus in my clojure programs), and 
> since I already load vertx for the eventbus, I use it as web server as 
> well, if I need one... So well, if someone has an idea, it might be a nice 
> project I'd love to work on :).
>
> thx, qsys
>
> Op zondag 3 januari 2016 22:25:04 UTC+1 schreef tbc++:
>>
>> I've done some evaluations of Vert.x in the past and was rather 
>> underwhelmed. What is it that you are trying to accomplish? Stuff like 
>> Pedestal offers async web services, but without the complexity of an 
>> traditional evented server. So perhaps if we had a better idea of your 
>> requirements we could be a bit more helpful.
>>
>> So I'd say, look into Pedestal and then define what you need that it 
>> cannot do. Same for other toolkits like ring and httpkit. 
>>
>> Timothy
>>
>> On Sun, Jan 3, 2016 at 12:59 PM, adrians  wrote:
>>
>>>
>>> It used to be that Vert.x 2.x had integration for Clojure, but version 
>>> 3.x hasn't added it yet. Has anyone used this version through the Java API 
>>> and if so, how painful was it? Is Reactor any 
>>> better in that respect? What are people using when they want this kind of 
>>> back end?
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To post to this group, send email to clo...@googlegroups.com
>>> Note that posts from new members are moderated - please be patient with 
>>> your first post.
>>> To unsubscribe from this group, send email to
>>> clojure+u...@googlegroups.com
>>> For more options, visit this group at
>>> http://groups.google.com/group/clojure?hl=en
>>> --- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Clojure" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to clojure+u...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
>>
>> -- 
>> “One of the main causes of the fall of the Roman Empire was that–lacking 
>> zero–they had no way to indicate successful termination of their C 
>> programs.”
>> (Robert Firth) 
>>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: lambda detector?

[Zach Tellman]

The "correct" way to do this is:

(defn function-form? [x]
  (let [x' (macroexpand x)]
(and (sequential? x') (= 'fn* (first x')

However, the approach you're describing won't work with let-bound 
variables, only inline function definitions and vars, which means that your 
macro is a limited subset of Clojure, rather than a small extension of it. 
 People generally find that to be confusing.  If this is just for your own 
personal use, go nuts, but if this is meant to be understood and used by 
others, I suggest thinking carefully about whether this the approach you 
want to use.

Zach

On Tuesday, December 8, 2015 at 9:18:05 AM UTC-8, Gregg Reynolds wrote:
>
>
>
> On Tue, Dec 8, 2015 at 7:16 AM, Angel Java Lopez  > wrote:
>
>> Hi!
>>
>> Sorry, this is not an answer, but a question: Why you need to know if an 
>> argument is a lambda? What is your use case?
>>
>
> A simple use case (somewhat different from my actual case, but clearer) 
> would be dynamic function dispatch.  You have a vector of functions and a 
> dispatch function that takes a vector of args and selects the "best match" 
> from the fns vector.
>
> If things are expressed like so:
>
> (defn f1 ...) (defn f2 ...) ...
> (def fns [f1 f2 ...])
>
> then it's easy: the dispatcher can pull the :arglists from the var 
> metadata for the fn symbols and compare it with the incoming args.  But if 
> you allow inline lambdas you can't (at least I don't know how to) access 
> the var metadata.  In my case this is a problem because some of my 
> functions will have a bit of custom metadata.
>
> To complicate things, let's say we allow clients to register functions, 
> using a register-fn function or macro, for example.  In my case, I have a 
> function and I use it to pre-register some fns defined using #(...) 
> syntax.  The problem I'm encountering is that these functions do not seem 
> to satisfy fn? or any other predicate I can think of to detect that they 
> are in fact functions.  Their types look like this:  class 
> miraj.sync$config_polymer_reqs$fn__16463 
> miraj.sync$config_polymer_reqs$fn__16463@541276dc - from this it is clear 
> that they are function objects, but I don't want to rely on parsing out 
> this kind of stuff.  FYI mirj.sync is my namespace, config-polymer-reqs is 
> my function, and #(...) is expressed in the result of a call to register-fn 
> within config-polymer-reqs.
>
> (My use case involves macros and alter-var-root stuff to manage system 
> config at startup.  I don't have the heart to try to describe it at the 
> moment.)
>
> Thanks,
>
> Gregg
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] lein-virgil, a plugin for mixed Clojure/Java development

[Zach Tellman]

It's been an uncomfortably long time since I released a new library, so 
here we go.  This plugin solves a problem that has endlessly bothered me 
(and hopefully a number of other people which is greater than zero), which 
is that changing any Java files in a project requires completely reloading 
the REPL session.  But no longer: https://github.com/ztellman/virgil.  This 
project, named for everyone's favorite psychopomp, leverages the compiler 
APIs within the JDK to live-reload the classes without a bit of fuss.

This code works for me, and will probably work for most simple use cases, 
but I wouldn't be surprised if there are some more complex cases that don't 
work perfectly.  Bug reports are welcome.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Manifold questions: Creating sinks/sources

[Zach Tellman]

Hi Atamert,

For future reference, posting these questions 
to https://groups.google.com/forum/#!forum/aleph-lib will ensure I'll see 
them sooner.  

The `source-only` method is just a way to make sure that the chance for 
confusion is minimized, it doesn't prevent the underlying object from being 
used normally.  As for your scenario, I'm not sure I completely understand. 
 In a standard stream, the flow of data is:

[ sink -> source ]

So if you `put!` something into the sink, you can `take!` it from the 
source.  If you use `source-only` and connect that to some other sink, then 
the flow of data will be

[ sink -> source ] -> sink

So the source is only connected to one sink, not two.  Maybe I'm 
misunderstanding you, though.

Zach

On Thursday, August 27, 2015 at 4:13:35 AM UTC-7, Atamert Ölçgen wrote:
>
> Hi,
>
> AFAIK the only way to create (just) a source (or sink) is:
>
> (def my-source (s/source-only (s/stream ...)))
>
> This results in creating a stream and then wrapping it with a SourceProxy. 
> We don't keep a reference to the stream and the SourceProxy doesn't allow 
> taking.
>
> But if I'm not missing something since SourceProxy keeps a reference of 
> the original stream, there is a sink nobody is using there.
>
> If I create a sink separately and connect my-source to it, now 
> my-source would be connected to two sinks.
>
> My questions are:
>
> 1. Is there another method for creating only sinks or sources?
> 2. Should the extra/unused source/sink I mentioned above cause concern?
>
> PS: I'm not talking about a scenario where we're creating thousands 
> streams and connecting them all like there's no tomorrow.
>
>
> -- 
> Kind Regards,
> Atamert Ölçgen
>
> ◻◼◻
> ◻◻◼
> ◼◼◼
>
> www.muhuk.com
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: What does Zach Tellman mean by "factored out for greater inlining joy"

[Zach Tellman]

It's fluid, but the output that Norman uses in his post (and that motivated 
the changes to Aleph) were from the JVM explicitly saying "I would have 
inlined this, but it was too big".  That may not be true under other 
circumstances, but you're only helping by factoring out uncommon, large 
code branches.

On Tuesday, August 11, 2015 at 2:01:59 PM UTC-7, Colin Fleming wrote:
>
> That's a really interesting post, thanks for that. I actually cornered 
> Cliff Click at Curry On because I was interested in knowing how well the 
> JVM inlined var indirection. The short answer is "it's complicated". But if 
> the JVM does inline your method, then the var indirection shouldn't cost 
> you as long as the var content is stable.
>
> Tom Crayford's great talk at EuroClojure also pointed out that another 
> thing which affects inlining is the stack depth, which can mean you're 
> damned if you do and you're damned if you don't when trying to refactor to 
> help with inlining. However Cliff said that all these limits are pretty 
> fluid - it's not like a method over 325 bytecodes will never be inlined, if 
> it's identified as hot that limit goes way up, and a relatively cold method 
> can still be inlined even if it's small.
>
> I'd actually love to sit down and test this with Clojure sometime, but I 
> never seem to find time for it.
>
> On 11 August 2015 at 21:00, Zach Tellman > 
> wrote:
>
>> The inlining part is explained very well by this blog post 
>> http://normanmaurer.me/blog/2014/05/15/Inline-all-the-Things/
>>
>> As for why I left all the repetition in there, I tend to let code expand 
>> before getting annoyed and compacting it.  Sometimes there's a commit 
>> between those two events, sometimes there's not.  In this case, you get to 
>> see how the macro/abstraction sausage is made.
>>
>> Happy to answer any other questions,
>> Zach
>>
>>
>> On Sunday, August 9, 2015 at 9:38:51 AM UTC-7, Lawrence Krubner wrote:
>>>
>>>
>>> Reid, thank you. I think you answer half the question. You make a good 
>>> point about giving the JVM a way to better optimize a hot path. I think you 
>>> are right about that. But, given the large amount of repetition between 
>>> "chain'-" and "chain-" I'm wondering why this wasn't done with a macro? 
>>>
>>>
>>>
>>> On Sunday, August 9, 2015 at 2:08:47 AM UTC-4, Reid McKenzie wrote:
>>>>
>>>> -BEGIN PGP SIGNED MESSAGE- 
>>>> Hash: SHA256 
>>>>
>>>> Lawrence, 
>>>>
>>>> This is just a theory, but in the interests of response time, the JVM 
>>>> uses a large number of heuristics to determine what optimizations will 
>>>> likely prove profitable. One of them is a budget for method size. I 
>>>> would guess that lifting this code out into a separate fn made the JVM 
>>>> see that it was optimizing a hot path between the main body and 
>>>> several small but tightly related methods thus giving itself more 
>>>> leeway to inline and optimize in ways that it would otherwise presume 
>>>> are more expensive and not pursue. 
>>>>
>>>> Reid 
>>>>
>>> -- 
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clo...@googlegroups.com 
>> 
>> Note that posts from new members are moderated - please be patient with 
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+u...@googlegroups.com 
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to clojure+u...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: What does Zach Tellman mean by "factored out for greater inlining joy"

[Zach Tellman]

The inlining part is explained very well by this blog 
post http://normanmaurer.me/blog/2014/05/15/Inline-all-the-Things/

As for why I left all the repetition in there, I tend to let code expand 
before getting annoyed and compacting it.  Sometimes there's a commit 
between those two events, sometimes there's not.  In this case, you get to 
see how the macro/abstraction sausage is made.

Happy to answer any other questions,
Zach

On Sunday, August 9, 2015 at 9:38:51 AM UTC-7, Lawrence Krubner wrote:
>
>
> Reid, thank you. I think you answer half the question. You make a good 
> point about giving the JVM a way to better optimize a hot path. I think you 
> are right about that. But, given the large amount of repetition between 
> "chain'-" and "chain-" I'm wondering why this wasn't done with a macro? 
>
>
>
> On Sunday, August 9, 2015 at 2:08:47 AM UTC-4, Reid McKenzie wrote:
>>
>> -BEGIN PGP SIGNED MESSAGE- 
>> Hash: SHA256 
>>
>> Lawrence, 
>>
>> This is just a theory, but in the interests of response time, the JVM 
>> uses a large number of heuristics to determine what optimizations will 
>> likely prove profitable. One of them is a budget for method size. I 
>> would guess that lifting this code out into a separate fn made the JVM 
>> see that it was optimizing a hot path between the main body and 
>> several small but tightly related methods thus giving itself more 
>> leeway to inline and optimize in ways that it would otherwise presume 
>> are more expensive and not pursue. 
>>
>> Reid 
>>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Manifold streams/deferreds raising errors downstream of a sink

[Zach Tellman]

Hi Andy,

To give you some context on this, `put!` will return true if the thing 
directly downstream accepts the value (in this case, the stream).  The fact 
that the callback in `consume` fails is between the stream and the 
callback, and is not propagated all the way back (though the exception will 
cancel the `consume`, which will cause the stream to close).  The approach 
described by Atamert is a reasonable one to get the desired result.

Zach

On Sunday, August 9, 2015 at 9:13:30 PM UTC-7, Andy Chambers wrote:
>
> Hey All,
>
> I'm trying to build a library that exposes manifold streams for data 
> producing applications. The intention
> would be for these apps to s/put! to a sink stream returned by the 
> producer function. However, I need to
> take the value that is put in and pipe it into some other function before 
> I know whether there is an error
> and if there is an error, I'd like to propagate it back to the caller.
>
> I understand that the value returned by put! is a deferred, and how to set 
> the error state of a deferred but I
> don't understand how I can arrange for the deferred to be manipulated 
> downstream of where it has been
> taken off the stream.
>
> (defn producer []
>   (let [in (s/stream)]
>
> (s/consume (fn [msg]
>  (d/future
>(prn "oops, going to error now")
>(throw (Exception. (str msg) in)
> 
> in))
>
> @(s/put! (producer) 42)
>
> As written, you get a success response even though an exception was 
> triggered by the function that consumes
> the "in" stream. Is it possible to re-write this so that the exception 
> bubbles up to where the put! is deref'd?
>
> Cheers,
> Andy
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] data.int-map 0.2.1

[Zach Tellman]

https://github.com/clojure/data.int-map

This is a small but important update.  Where 0.2.0 added the ability to use 
negative integers, it did *not* retain the ordered key invariant when 
negative numbers were used (negative numbers came after the positive 
numbers, because binary representations are a continual delight).  I've 
pushed an update such that the ordering is correct, and also pushed some of 
the int-set implementation into Java, yielding a 20-30% performance 
improvement.  This should be a drop-in replacement for anyone using earlier 
versions of the library.

Zach

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] Clojure 1.8.0-alpha2

[Zach Tellman]

A similar issue was reported earlier in the thread, target [potemkin 
"0.4.1"] and see if that fixes it.

On Tuesday, July 21, 2015 at 12:24:43 PM UTC-7, Rangel Spasov wrote:
>
> Hey guys,
>
> Getting this error with 1.8.0-alpha2, I think related to aleph (using 
> 0.4.0, latest version at the moment).
>
> #error {
>
>  :cause IllegalName: 
> compile__stub.aleph.http.core.aleph.http.core/HeaderMap
>
>  :via
>
>  [{:type clojure.lang.Compiler$CompilerException
>
>:message java.lang.NoClassDefFoundError: IllegalName: 
> compile__stub.aleph.http.core.aleph.http.core/HeaderMap, 
> compiling:(aleph/http/core.clj:81:1)
>
>:at [clojure.lang.Compiler analyzeSeq Compiler.java 6798]}
>
>   {:type java.lang.NoClassDefFoundError
>
>:message IllegalName: 
> compile__stub.aleph.http.core.aleph.http.core/HeaderMap
>
>:at [java.lang.ClassLoader preDefineClass ClassLoader.java 654]}]
>
>  :trace
>
>  [[java.lang.ClassLoader preDefineClass ClassLoader.java 654]
>
>   [java.lang.ClassLoader defineClass ClassLoader.java 758]
>
>   [java.lang.ClassLoader defineClass ClassLoader.java 642]
>
>   [clojure.lang.DynamicClassLoader defineClass DynamicClassLoader.java 46]
>
>   [clojure.lang.Compiler$NewInstanceExpr compileStub Compiler.java 7815]
>
>   [clojure.lang.Compiler$NewInstanceExpr build Compiler.java 7680]
>
>   [clojure.lang.Compiler$NewInstanceExpr$DeftypeParser parse Compiler.java 
> 7590]
>
>   [clojure.lang.Compiler analyzeSeq Compiler.java 6791]
>
>   [clojure.lang.Compiler analyze Compiler.java 6592]
>
>   [clojure.lang.Compiler analyze Compiler.java 6553]
>
>   [clojure.lang.Compiler$BodyExpr$Parser parse Compiler.java 5929]
>
>   [clojure.lang.Compiler$LetExpr$Parser parse Compiler.java 6247]
>
>   [clojure.lang.Compiler analyzeSeq Compiler.java 6791]
>
>   [clojure.lang.Compiler analyze Compiler.java 6592]
>
>   [clojure.lang.Compiler analyze Compiler.java 6553]
>
>   [clojure.lang.Compiler$BodyExpr$Parser parse Compiler.java 5929]
>
>   [clojure.lang.Compiler$FnMethod parse Compiler.java 5359]
>
>   [clojure.lang.Compiler$FnExpr parse Compiler.java 3959]
>
>   [clojure.lang.Compiler analyzeSeq Compiler.java 6789]
>
>   [clojure.lang.Compiler analyze Compiler.java 6592]
>
>   [clojure.lang.Compiler eval Compiler.java 6847]
>
>   [clojure.lang.Compiler eval Compiler.java 6839]
>
>   [clojure.lang.Compiler load Compiler.java 7295]
>
>   [clojure.lang.RT loadResourceScript RT.java 372]
>
>   [clojure.lang.RT loadResourceScript RT.java 363]
>
>   [clojure.lang.RT load RT.java 453]
>
>   [clojure.lang.RT load RT.java 419]
>
>   [clojure.core$load$fn__5448 invoke core.clj 5866]
>
>   [clojure.core$load doInvoke core.clj 5865]
>
>   [clojure.lang.RestFn invoke RestFn.java 408]
>
>   [clojure.core$load_one invoke core.clj 5671]
>
>   [clojure.core$load_lib$fn__5397 invoke core.clj 5711]
>
>   [clojure.core$load_lib doInvoke core.clj 5710]
>
>   [clojure.lang.RestFn applyTo RestFn.java 142]
>
>   [clojure.core$apply invoke core.clj 632]
>
>   [clojure.core$load_libs doInvoke core.clj 5749]
>
>   [clojure.lang.RestFn applyTo RestFn.java 137]
>
>   [clojure.core$apply invoke core.clj 632]
>
>   [clojure.core$require doInvoke core.clj 5832]
>
>   [clojure.lang.RestFn invoke RestFn.java 551]
>
>   [aleph.http.server$eval9251$loading__5340__auto9252 invoke 
> server.clj 1]
>
>   [aleph.http.server$eval9251 invoke server.clj 1]
>
>   [clojure.lang.Compiler eval Compiler.java 6850]
>
>   [clojure.lang.Compiler eval Compiler.java 6839]
>
>   [clojure.lang.Compiler load Compiler.java 7295]
>
>   [clojure.lang.RT loadResourceScript RT.java 372]
>
>   [clojure.lang.RT loadResourceScript RT.java 363]
>
>   [clojure.lang.RT load RT.java 453]
>
>   [clojure.lang.RT load RT.java 419]
>
>   [clojure.core$load$fn__5448 invoke core.clj 5866]
>
>   [clojure.core$load doInvoke core.clj 5865]
>
>   [clojure.lang.RestFn invoke RestFn.java 408]
>
>   [clojure.core$load_one invoke core.clj 5671]
>
>   [clojure.core$load_lib$fn__5397 invoke core.clj 5711]
>
>   [clojure.core$load_lib doInvoke core.clj 5710]
>
>   [clojure.lang.RestFn applyTo RestFn.java 142]
>
>   [clojure.core$apply invoke core.clj 632]
>
>   [clojure.core$load_libs doInvoke core.clj 5753]
>
>   [clojure.lang.RestFn applyTo RestFn.java 137]
>
>   [clojure.core$apply invoke core.clj 632]
>
>   [clojure.core$require doInvoke core.clj 5832]
>
>   [clojure.lang.RestFn invoke RestFn.java 457]
>
>   [aleph.http$eval1594$loading__5340__auto1595 invoke http.clj 1]
>
>   [aleph.http$eval1594 invoke http.clj 1]
>
>   [clojure.lang.Compiler eval Compiler.java 6850]
>
>   [clojure.lang.Compiler eval Compiler.java 6839]
>
>   [clojure.lang.Compiler load Compiler.java 7295]
>
>   [clojure.lang.RT loadResourceScript RT.java 372]
>
>   [clojure.lang.RT loadResourceScript RT.java 363]
>
>   [clojure.lang.RT load RT.java 453]
>
>   [clojure.lang.RT load RT.java 419]
>
>   [clojure.core$load$fn__5448 invoke core.clj 5866]
>
>   [clojure.core$load doInvoke core.clj 5865]
>
>

Re: [ANN] Clojure 1.8.0-alpha2

[Zach Tellman]

You're also going to have to target [clj-tuple "0.2.2"], since something 
else seems to be shadowing that depedency.  Sorry for all the fuss.

On Sunday, July 19, 2015 at 5:02:13 PM UTC-7, Sean Corfield wrote:
>
> Bumping clj-http to 2.0.0 got me past that problem (and it uses Potemkin 
> 0.4.1) but then I hit the problem below, which I’ve run into several times 
> trying to use updated versions of clj-http over the last year.
>
> Messagejava.lang.RuntimeException: No such var: clj-tuple/vector, 
> compiling:(potemkin/utils.clj:110:10)Cause
> clojure.lang.Compiler$CompilerExceptionStacktraceThe Error Occurred in
> *core.clj: line 5866* 
> *called from* core.clj: line 5865 
> *called from* core.clj: line 5671 
> *called from* core.clj: line 5711 
> *called from* core.clj: line 5710 
> *called from* core.clj: line 632 
> *called from* core.clj: line 5753 
> *called from* core.clj: line 634 
> *called from* core.clj: line 5843 
> *called from* collections.clj: line 1 
> *called from* collections.clj: line 1 
> *called from* core.clj: line 5866 
> *called from* core.clj: line 5865 
> *called from* core.clj: line 5671 
> *called from* core.clj: line 5711 
> *called from* core.clj: line 5710 
> *called from* core.clj: line 632 
> *called from* core.clj: line 5753 
> *called from* core.clj: line 632 
> *called from* core.clj: line 5832 
> *called from* potemkin.clj: line 1 
> *called from* potemkin.clj: line 1 
> *called from* core.clj: line 5866 
> *called from* core.clj: line 5865 
> *called from* core.clj: line 5671 
> *called from* core.clj: line 5711 
> *called from* core.clj: line 5710 
> *called from* core.clj: line 632 
> *called from* core.clj: line 5749 
> *called from* core.clj: line 632 
> *called from* core.clj: line 5832 
> *called from* headers.clj: line 1 
> *called from* headers.clj: line 1 
> *called from* core.clj: line 5866 
> *called from* core.clj: line 5865 
> *called from* core.clj: line 5671 
> *called from* core.clj: line 5711 
> *called from* core.clj: line 5710 
> *called from* core.clj: line 632 
> *called from* core.clj: line 5749 
> *called from* core.clj: line 632 
> *called from* core.clj: line 5832 
> *called from* core.clj: line 1 
> *called from* core.clj: line 1 
> *called from* core.clj: line 5866 
> *called from* core.clj: line 5865 
> *called from* core.clj: line 5671 
> *called from* core.clj: line 5711 
> *called from* core.clj: line 5710 
> *called from* core.clj: line 632 
> *called from* core.clj: line 5749 
> *called from* core.clj: line 632 
> *called from* core.clj: line 5832 
> *called from* client.clj: line 1 
>
>
> On Jul 19, 2015, at 4:53 PM, Michael Blume  > wrote:
>
> Looks like it has, pinning to Potemkin 0.4.1 should probably sort you out
>
> On Sun, Jul 19, 2015 at 4:50 PM Michael Blume  > wrote:
>
>> Sean, I think that was identified as a bug in Potemkin. The pull was 
>> merged but I'm not sure if there's been a release since. Zack?
>>
>> https://github.com/ztellman/potemkin/pull/40
>>
>>
>> http://dev.clojure.org/jira/browse/CLJ-1208?focusedCommentId=39632&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-39632
>>
>> On Sun, Jul 19, 2015 at 4:47 PM Sean Corfield > > wrote:
>>
>>> On Jul 18, 2015, at 10:33 PM, Sean Corfield >> > wrote:
>>> > Wow, that's a fast timeline. Thank you. We'll upgrade to Alpha 2 this 
>>> week. We may go to production with it fairly quickly.
>>>
>>> Switched out 1.7.0 for 1.8.0-alpha2 and got the exception below. Posting 
>>> here in case anyone knows immediately what the cause is, while I go 
>>> debugging...
>>>
>>> Exception in thread "main" java.lang.NoClassDefFoundError: IllegalName: 
>>> compile__stub.clj_http.headers.clj-http.headers/HeaderMap, 
>>> compiling:(clj_http/headers.clj:105:1)
>>> at clojure.lang.Compiler.analyzeSeq(Compiler.java:6798)
>>> at clojure.lang.Compiler.analyze(Compiler.java:6592)
>>> at clojure.lang.Compiler.analyze(Compiler.java:6553)
>>> at 
>>> clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:5929)
>>> at clojure.lang.Compiler$LetExpr$Parser.parse(Compiler.java:6247)
>>> at clojure.lang.Compiler.analyzeSeq(Compiler.java:6791)
>>> at clojure.lang.Compiler.analyze(Compiler.java:6592)
>>> at clojure.lang.Compiler.analyze(Compiler.java:6553)
>>> at 
>>> clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:5929)
>>> at clojure.lang.Compiler$FnMethod.parse(Compiler.java:5359)
>>> at clojure.lang.Compiler$FnExpr.parse(Compiler.java:3959)
>>> at clojure.lang.Compiler.analyzeSeq(Compiler.java:6789)
>>> at clojure.lang.Compiler.analyze(Compiler.java:6592)
>>> at clojure.lang.Compiler.eval(Compiler.java:6847)
>>> at clojure.lang.Compiler.eval(Compiler.java:6839)
>>> at clojure.lang.Compiler.load(Compiler.java:7295)
>>> at clojure.lang.RT.loadResourceScript(RT.java:372)
>>> at clojure.lang.RT.loadResourceScript(RT.java:363)
>

Re: RejectedExecutionException? I don't think I was using anything concurrent?

[Zach Tellman]

A RejectedExecutionException is thrown when a thread-pool's queue is full. 
 Try using 'jstack' to take a thread dump, and see who has spun up a thread 
pool under the covers.  Possible culprits include 'at-at' and 'carmine'.  

On Friday, July 17, 2015 at 11:01:05 AM UTC-7, sugarste...@gmail.com wrote:
>
>
> Okay, this error seems to be triggered by a NullPointerException which 
> seems to arise when I try to fetch a non-existent document out of Redis. I 
> am a bit of a noob, so perhaps someone can explain how this works. I'm not 
> aware of using any of Clojure's concurrency tools in this (very simple) 
> app, so how might the NullPointerException lead to 
> RejectedExecutionException?  
>
>
> On Friday, July 17, 2015 at 1:17:47 PM UTC-4, sugarste...@gmail.com wrote:
>>
>> My app starts up, runs for a few minutes, and then: 
>>
>> #> java.util.concurrent.RejectedExecutionException: Task 
>> java.util.concurrent.FutureTask@4fe887e6 rejected from 
>> java.util.concurrent.ThreadPoolExecutor@aea107f[Terminated, pool size = 0, 
>> active threads = 0, queued tasks = 0, completed tasks = 3]>
>>
>> This is a very simple app, and I am not aware of using any concurrency. I 
>> am not using Futures or Promises or pmap or anything else I can think of. 
>> My first hunch would be this comes from some library I am using, but I am 
>> not sure what. I searched on Google but none of the reported problems seem 
>> to apply to me. This is what I am using, do any of these seem likely to 
>> throw this error? 
>>
>> [org.clojure/clojure "1.6.0"]
>>  [com.taoensso/timbre "4.0.2"]
>>  [dire "0.5.1"]
>>  [slingshot "0.12.2"]
>>  [org.clojure/data.json "0.2.5"]
>>  [org.clojure/tools.namespace "0.2.4"]
>>  [me.raynes/fs "1.4.4"]
>>  [clj-stacktrace "0.2.8"]
>>  [overtone/at-at "1.2.0"]
>>  [org.clojure/core.cache "0.6.4"]
>>  [cheshire "5.5.0"]
>>  [com.taoensso/carmine "2.11.1"]
>>
>>
>>
>>
>>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Opinion on core.async vs callbacks in abstract APIs?

[Zach Tellman]

The problem with using bare callbacks is that there's no way for the 
invoked callback to exert backpressure, except by blocking or passing in a 
CPS-style callback to the callback, neither of which is ideal.  If you're 
looking for a "neutral" choice, I'd suggest an infinite lazy-seq over 
callbacks.  If you absolutely need it to be async, something like Manifold 
might be warranted, in that it allows people to consume it however they 
like (though they will have to be at least a little familiar with Manifold 
to do so).

Zach

On Monday, June 1, 2015 at 12:18:19 PM UTC-7, Christopher Small wrote:
>
> Greetings
>
> I imagine most of us here would rather use core.async channels over 
> callbacks in their application code, particularly with more complicated 
> applications. But is it okay/preferable for Clojure libraries to force 
> their users to use core.async channels as part of an API (an event channel, 
> for example)? 
>
> As much as I love core.async, I can't help but wonder whether sticking 
> with callbacks for an API isn't a simpler/better design strategy. It's easy 
> enough to drop messages on a channel in a callback, and this let's users 
> opt-in. But if one expects core.async channels are what most would prefer 
> anyway, is it okay to foist them upon everyone?
>
> As a follow up, does your opinion on the matter change if implementations 
> of an API become simpler using core.async channels?
>
>
> Looking forward to your thoughts :-)
>
> Chris Small
>
>
>
> PS I'm asking because I'm working on a physical computing API (
> https://github.com/clj-bots/pin-ctrl) and debating between using channels 
> vs callbacks for the edge detection functionality (if you're not familiar, 
> edge detection let's you asynchronously handle changes in pin state, such 
> as button pushes). If you're interested in this question as it applies 
> specifically to this application, feel free to join the discussion on our 
> gitter channel: https://gitter.im/clj-bots/chat
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] data.int-map 0.2.0

[Zach Tellman]

https://github.com/clojure/data.int-map

The 0.1.0 release of this library was a faithful Clojure implementation of 
Okasaki's "Fast Mergeable Integer Maps" paper [1].  While it was much 
faster than Clojure's sorted-maps, it was in some cases slower than 
hash-maps, which was mostly due to the int-maps being a binary tree, and 
hash-maps being a 32-ary tree.  It also could only take non-negative 
integers, which was a limitation of the approach used in the paper.  In the 
0.2.0 release, I reimplemented the core data structure in Java, but more 
importantly adapted the data structure to use 16-ary trees under the 
covers.  This has greatly improved performance (by a factor of 2-5x, 
depending on the benchmark), and has also allowed the map to have negative 
integers for keys.

I recommend anyone using an earlier version of this library upgrade to the 
latest, which will be a drop-in replacement.

Zach

[1] http://ittc.ku.edu/~andygill/papers/IntMap98.pdf

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] lein-jammin

[Zach Tellman]

https://github.com/ztellman/lein-jammin

This one's pretty simple: you put `lein jammin ` in front of any 
other Leiningen task, and if it gets stuck for the specified duration, it 
prints out a thread dump.  This especially useful for tests, and 
extra-especially useful for tests in a CI environment where using `jstack` 
or similar tools is difficult or impossible.

Enjoy,
Zach

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN} Aleph 0.4.0 released, plus Manifold, Dirigiste, and a whole host of other libraries

[Zach Tellman]

I suspect it would, I think I was just letting the mechanics of Manifold's
let-flow macro color my judgment. Happy to accept any pull requests which
make my core.async examples more idiomatic.
On Apr 18, 2015 8:33 AM, "Matthias Lange"  wrote:

> In your examples, you put a let around the reads from timeouts.
>
> (let [_ (a/
> As far as i know, that is not neccessary. So your first example could be:
>
> (defn delayed-hello-world-handler
>   [req]
>   (d/->deferred
> (a/go
>   (a/   (hello-world-handler req
>
> Would that not work?
>
>  --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/KH8Js8URKwA/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN} Aleph 0.4.0 released, plus Manifold, Dirigiste, and a whole host of other libraries

[Zach Tellman]

Hey Dmitri,

I haven't used any sort of dev-mode before (I just update stuff in the REPL
when necessary), but it seems like something like that belongs in
middleware, not the server.  The server is just calling a function, it
shouldn't care if something else is changing that function's behavior.

If other servers have this behavior embedded, maybe it can be extracted
into a standalone library?

Zach

On Fri, Apr 17, 2015 at 3:47 PM, Dmitri  wrote:

> I'd like to add Aleph to the Luminus template and I was wondering if
> there's an equivalent of dev mode available for other servers where it
> watches for changes in source and reloads them. I did a cursory look but
> didn't spot anything like a -dev option.
>
> On Friday, April 17, 2015 at 5:06:30 PM UTC-4, Zach Tellman wrote:
>
>> Hey all,
>>
>> In preparation for Clojure/West, I'm formally releasing the latest Aleph
>> and the libraries that surround it.  Aleph 0.4.0 has been running in
>> production at Factual for half a year now, and across a variety of services
>> is handling at peak 600k HTTP requests/sec (spread across 15-20 machines).
>>
>> Since the landscape of Clojure HTTP servers is pretty crowded these days,
>> it's worth taking some time to explain how Aleph differs.  To be clear,
>> most Clojure deployments likely use Jetty, and should continue to do so.
>> However, Aleph has some unique properties:
>>
>> * It uses the Netty library, which is a high-performance and very
>> battle-tested network layer for the JVM
>> * It is the only HTTP server that has *ubiquitous* asynchronous streams
>> wherever data can be received or sent (all other libraries can only
>> represent streaming requests using InputStreams, or like http-kit don't
>> support streaming HTTP requests at all)
>> * It is the only server that has a WebSocket implementation with any
>> support for per-connection backpressure.  I won't make this post even
>> longer by going into why this is important, but this will be a central
>> theme of my talk at Clojure/West next week if you're interested in hearing
>> more.
>> * It uses consistent abstractions to represent network connections over a
>> variety of protocols, which makes it straightforward to use the same
>> application logic for all of them.
>>
>> Again, none of these points mean you should immediately drop whatever
>> you're using and move over to Aleph instead.  However, I do feel it
>> represents the only (current) good option for using core.async or a similar
>> stream abstraction to represent network data, which is an idea a number of
>> people seem to be playing with lately.  Some examples of this can be found
>> at http://ideolalia.com/aleph/literate.html.
>>
>> A full list of the libraries:
>>
>> aleph - https://github.com/ztellman/aleph - uses the excellent Netty
>> library to expose HTTP, TCP, and UDP using a consistent asynchronous stream
>> representation.
>>
>> manifold - https://github.com/ztellman/manifold - an unopinionated
>> stream representation designed to cleanly interoperate with other stream
>> representations (Clojure's seqs, core.async channels, Java's
>> BlockingQueues, and others).  This is the base stream representation for
>> all network sources and sinks in Aleph.
>>
>> dirigiste -  https://github.com/ztellman/dirigiste - a pure-Java library
>> that provides instrumented, dynamically sized thread and object pools.
>> This is used for thread pools in Aleph's HTTP server, and for connection
>> pools in Aleph's HTTP client.
>>
>> byte-streams -  https://github.com/ztellman/byte-streams - a means of
>> translating any byte representation into another.  Want to turn a
>> core.async channel that emits byte-arrays into an InputStream, or maybe the
>> other way around?  Look no further.  The library's conversion mechanism is
>> extensible, which is used in Aleph to make Netty's custom byte
>> representations interoperable with more familiar representations.
>>
>> byte-transforms -  https://github.com/ztellman/byte-transforms - a
>> curated collection of byte compression, hashing, and encoding mechanisms,
>> which can work on anything byte-streams can convert.
>>
>> While all these libraries are used in concert to create Aleph, I've been
>> very careful to make sure any of them can be used by themselves.  If anyone
>> has questions about them, the best place to get my attention is the Aleph
>> mailing list: https://groups.google.com/forum/#!forum/aleph-lib.
>>
>> I will be mentioning some of t

[ANN} Aleph 0.4.0 released, plus Manifold, Dirigiste, and a whole host of other libraries

[Zach Tellman]

Hey all,

In preparation for Clojure/West, I'm formally releasing the latest Aleph 
and the libraries that surround it.  Aleph 0.4.0 has been running in 
production at Factual for half a year now, and across a variety of services 
is handling at peak 600k HTTP requests/sec (spread across 15-20 machines).  

Since the landscape of Clojure HTTP servers is pretty crowded these days, 
it's worth taking some time to explain how Aleph differs.  To be clear, 
most Clojure deployments likely use Jetty, and should continue to do so. 
 However, Aleph has some unique properties:

* It uses the Netty library, which is a high-performance and very 
battle-tested network layer for the JVM
* It is the only HTTP server that has *ubiquitous* asynchronous streams 
wherever data can be received or sent (all other libraries can only 
represent streaming requests using InputStreams, or like http-kit don't 
support streaming HTTP requests at all)
* It is the only server that has a WebSocket implementation with any 
support for per-connection backpressure.  I won't make this post even 
longer by going into why this is important, but this will be a central 
theme of my talk at Clojure/West next week if you're interested in hearing 
more.
* It uses consistent abstractions to represent network connections over a 
variety of protocols, which makes it straightforward to use the same 
application logic for all of them.

Again, none of these points mean you should immediately drop whatever 
you're using and move over to Aleph instead.  However, I do feel it 
represents the only (current) good option for using core.async or a similar 
stream abstraction to represent network data, which is an idea a number of 
people seem to be playing with lately.  Some examples of this can be found 
at http://ideolalia.com/aleph/literate.html.

A full list of the libraries:

aleph - https://github.com/ztellman/aleph - uses the excellent Netty 
library to expose HTTP, TCP, and UDP using a consistent asynchronous stream 
representation.

manifold - https://github.com/ztellman/manifold - an unopinionated stream 
representation designed to cleanly interoperate with other stream 
representations (Clojure's seqs, core.async channels, Java's 
BlockingQueues, and others).  This is the base stream representation for 
all network sources and sinks in Aleph.

dirigiste -  https://github.com/ztellman/dirigiste - a pure-Java library 
that provides instrumented, dynamically sized thread and object pools. 
 This is used for thread pools in Aleph's HTTP server, and for connection 
pools in Aleph's HTTP client.

byte-streams -  https://github.com/ztellman/byte-streams - a means of 
translating any byte representation into another.  Want to turn a 
core.async channel that emits byte-arrays into an InputStream, or maybe the 
other way around?  Look no further.  The library's conversion mechanism is 
extensible, which is used in Aleph to make Netty's custom byte 
representations interoperable with more familiar representations.

byte-transforms -  https://github.com/ztellman/byte-transforms - a curated 
collection of byte compression, hashing, and encoding mechanisms, which can 
work on anything byte-streams can convert.

While all these libraries are used in concert to create Aleph, I've been 
very careful to make sure any of them can be used by themselves.  If anyone 
has questions about them, the best place to get my attention is the Aleph 
mailing list: https://groups.google.com/forum/#!forum/aleph-lib.

I will be mentioning some of these libraries at my upcoming Clojure/West 
talk (http://clojurewest.org/speakers#ztellman), but I've also set aside an 
Unsession for specifically discussing these 
libraries: https://github.com/clojurewest/clojurewest2015/wiki/Unsessions. 
 If you're interested, please add your name to the list.

Zach

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Use core.async executor with aleph

[Zach Tellman]

Hi Robin,

You can absolutely specify an executor of :none if you're sure you won't be 
doing any blocking in your request handler.  If everything's wrapped by a 
go-block, that's certainly the case, and is probably the most efficient 
approach.  However, Aleph just needs some java.util.concurrent.Executor, so 
using the core.async executor is also a valid approach.

By the way, I'm more likely to notice these sorts of questions if you ask 
them on the Aleph mailing 
list: https://groups.google.com/forum/#!forum/aleph-lib

Zach

On Tuesday, February 17, 2015 at 1:35:53 AM UTC-8, Robin Heggelund Hansen 
wrote:
>
> From what I can see, aleph allows me to set a executor to handle client 
> requests. I'm already using core.async pretty heavily. Is there any reason 
> why I shouldn't pass core.async's executor to aleph? I see I can also make 
> every client request start on aleph's dispatch thread. Considering 
> absolutely every request spawns a go-block, might it even be a good idea to 
> not run aleph with an executor at all?
>
> Thanks!
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Newbie Gloss questions - dynamic buffer structure

[Zach Tellman]

I'm sorry, that should have been :ubyte. Being able to insert arbitrary
data into a codec is a feature, though.

On Mon, Jan 12, 2015 at 6:27 AM, Tzach  wrote:

> Hi Zach
> Thanks for the detailed response
>
> On Wednesday, January 7, 2015 at 7:28:06 AM UTC+2, Zach Tellman wrote:
>>
>> Hey Tzach,
>>
>> If I understand what you're trying to do, you want something like this:
>>
>> [ :uint32
>>   :ubyte ;; or bit-seq, depending
>>   (delimited-block (prefix uint24 #(- % 5) #(+ % 5))) ]
>>
>> And then you'll need to parse the final block separately.  I've left
>> defining the uint24 as an exercise for the reader (sorry, long day), but
>> using (compile-frame [:uint16 :uint8] ...) should be pretty straightforward.
>>
> This is what work for me, in case anyone is interested or want to comment
>
> (defn valid-length [l]
>  (and (< l 16777216)
>   (>= l 0)))
>
> (def uint24 (compile-frame [:ubyte :uint16]
>(fn [u]
>  {:pre  [(valid-length u)]}
>  (let [u8 (bit-shift-right u 16)
>u16 (- u (bit-shift-left u8 16))]
>[u8 u16]))
>(fn [[u8 u16]]
>  {:post [(valid-length %)]}
>  (+ (bit-shift-left u8 16) u16
>
> (def avp
>   (compile-frame
>[:uint32
> :ubyte
> (repeated :int32 :prefix (prefix uint24 #(- % 5) #(+ % 5))) ]
> ))
>
> My next step is to parse the bits from the header :ubyte and base on it,
> read an extra uint32 before the repeated part.
> The header is probably the answer.
>
>
>>
>> Hope that helps, happy to answer any followup questions.
>>
>
> It did help! Thanks
>
> BTW, I spent too much time trying to use (compile-frame [:uint16 :uint8])
> before realizing :uint8 is not defined.
> The result is simply encoding of the :uint8 symbol.
> I wonder if its a bug or a feature.
>
>
>
>>
>> Zach
>>
>> On Saturday, January 3, 2015 11:51:14 PM UTC-8, Tzach wrote:
>>>
>>> I'm trying to work with Gloss binary encoder/decoder, and need some help
>>> to kick start.
>>>
>>> My first task is simple(not for me ;)
>>> I have the following binary buffer to read/write:
>>>
>>>- 4 byte (32 bit) - code (uint)
>>>- 8 bit - misc flags
>>>- 3 byte (24 bit) - the entire buffer length
>>>- 4 byte (32 bit) uint value -  optional, depending on on of the
>>>flags.
>>>- List of 4 byte (uints) data elements - size dependent on the
>>>overall size
>>>
>>> How should I represent this structure?
>>> Clearly I need to use *prefix* and *repeated* for the last element, but
>>> I failed to understand how.
>>> I also struggle with the optional element and how to represent a 3 byte
>>> element.
>>>
>>> Thanks!
>>>
>>  --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/NxsCPBk12mE/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: A (foolish) plan to re-invent IO on top of core.async

[Zach Tellman]

Aleph has always allowed that, but the latest release allows the asynchrony 
to be modeled using core.async channels if that's to your taste.

On Wednesday, January 7, 2015 at 9:52:53 AM UTC-8, Paul deGrandis wrote:
>
> There was another discussion on this list regarding async IO and web 
> servers.  It may be rather informative to you: 
> https://groups.google.com/d/msg/clojure/rKqT13Ofy4k/H9xvkZA9Yy4J
>
> To my knowledge, Pedestal is the only web library that let's you go async 
> all the way down to the wire (potentially the latest Aleph also allows for 
> this).  The benefit provided is that Pedestal manages the NIO integration 
> directly with the container for you - optimized to the specific container.  
> You may thumb through the implementation for some ideas.
>
> You also have to be very mindful about back-pressure when using core.async 
> in certain combinations.  Zach Tellman has covered the major points here: 
> https://groups.google.com/d/msg/clojure/TVMQJwaij1U/dQxyBxxbIjQJ
>
> Cheers,
> Paul
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Newbie Gloss questions - dynamic buffer structure

[Zach Tellman]

Hey Tzach,

If I understand what you're trying to do, you want something like this:

[ :uint32
  :ubyte ;; or bit-seq, depending
  (delimited-block (prefix uint24 #(- % 5) #(+ % 5))) ]

And then you'll need to parse the final block separately.  I've left 
defining the uint24 as an exercise for the reader (sorry, long day), but 
using (compile-frame [:uint16 :uint8] ...) should be pretty straightforward.

Hope that helps, happy to answer any followup questions.

Zach

On Saturday, January 3, 2015 11:51:14 PM UTC-8, Tzach wrote:
>
> I'm trying to work with Gloss binary encoder/decoder, and need some help 
> to kick start.
>
> My first task is simple(not for me ;)
> I have the following binary buffer to read/write:
>
>- 4 byte (32 bit) - code (uint)
>- 8 bit - misc flags
>- 3 byte (24 bit) - the entire buffer length
>- 4 byte (32 bit) uint value -  optional, depending on on of the flags.
>- List of 4 byte (uints) data elements - size dependent on the overall 
>size 
>
> How should I represent this structure?
> Clearly I need to use *prefix* and *repeated* for the last element, but I 
> failed to understand how.
> I also struggle with the optional element and how to represent a 3 byte 
> element.
>  
> Thanks!
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Server Sent Events under Http-Kit

[Zach Tellman]

In Aleph, you simply return the channel as the body of the response [1],
and messages are consumed only as they're able to be sent over the
network.  This means that backpressure "just works" without any need to
poll whether downstream buffers are full.  Obviously you can poll using the
put-with-timeout mechanism in core.async et al if you want, though.

[1] https://gist.github.com/ztellman/613629a2a32908d096d5

On Wed, Dec 17, 2014 at 10:42 AM, Malcolm Sparks  wrote:
>
> http-kit's send! doesn't block normally -
>
> From http://www.http-kit.org/server.html :-
>
> send!: Sends data to client and returns true if the data was successfully
> written to the output queue, or false if the channel is closed. Normally,
> checking the returned value is not needed. This function returns
> immediately (does not block).
>
> http-kit will attempt to write to the output queue, otherwise will place
> the message in a pending queue backed by a
> java.util.LinkedList, so there's no back-pressure (except for
> that exerted by an OutOfMemory exception)
>
> AFAICT there's no way of finding whether http-kit's output channel is
> empty or not, that would certainly be a feature I'd like to see.
>
> Does aleph support this?
>
>
>
>
> On 17 December 2014 at 18:23, Zach Tellman  wrote:
>>
>> Does `send!` block if it needs to exert backpressure?  If so, the
>> `go-loop` seems dangerous.  If not, how does backpressure work?
>>
>> On Wed, Dec 17, 2014 at 8:07 AM, Malcolm Sparks  wrote:
>>>
>>> Here is a solution for SSE over http-kit channels
>>>
>>> (defn server-event-source [ch] ;; Adding a mult here is dangerous
>>> because it bleeds the underlying ;; channel dry. We should provide the
>>> mult via modular.async (let [m (async/mult ch)] (fn [req] (let [ch
>>> (async/chan 16)] (async/tap m ch) (with-channel req net-ch (on-close
>>> net-ch (fn [_] (async/untap m ch) (async/close! ch))) (send! net-ch {
>>> :headers headers} false) (go-loop [] (when-let [data (>> data! " data) (send! net-ch (->message data) false) (recur
>>>
>>>
>>> https://github.com/juxt/modular/blob/master/modules/http-kit-events/src/modular/http_kit/events.clj
>>>
>>>
>>> On Saturday, 6 December 2014 17:03:41 UTC, Lars Ole Avery Simonsen wrote:
>>>>
>>>> Hi guys
>>>>
>>>> I am trying to implement Server Sent Event support in a small hobby
>>>> project based on the http-kit server framework and compojure.
>>>>
>>>> I am still quite new to clojure in general, so it may very well be that
>>>> I am missing something obvious, but this SSE detail has me stumped.
>>>>
>>>> What I have tried is to take inspiration from the Eventual
>>>> <https://github.com/ninjudd/eventual> library for the SSE
>>>> implementation (which is aimed at jetty). I have changed the Eventual
>>>> implementation so that it uses the http-kit async-channel implementation
>>>> for data transmission. What I am seeing is that everything seems to run
>>>> without errors except for the fact that no data is being received at the
>>>> EventSource in the browser.
>>>>
>>>> If I allow the sse channel to close immediately after opening (which is
>>>> pretty useless), the initial message is received at the browser side, but
>>>> if I keep the connection open, nothing gets received.
>>>>
>>>> Having had a look at the http-kit internals, my suspicion is that the
>>>> socket doesn't get flushed and so all my attempts at communication are
>>>> sitting in some buffer somewhere.
>>>>
>>>> All SSE implementations I have looked at for other platforms have
>>>> explicit calls to flush the socket buffers on the long lived connection
>>>> after each message transmission.
>>>>
>>>> Does anyone have any suggestions as to a better way to achieve SSE
>>>> support in a http-kit based server? Or maybe ideas as to what I might be
>>>> doing wrong?
>>>>
>>>> Thanks in advance.
>>>>
>>>> LOAS
>>>>
>>>>  --
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To post to this group, send email to clojure@googlegroups.com
>>> Note that posts from new members are moderated - please be patient with
>>> your first post.
>>> To unsubscribe from this group, s

Re: Server Sent Events under Http-Kit

[Zach Tellman]

Does `send!` block if it needs to exert backpressure?  If so, the `go-loop`
seems dangerous.  If not, how does backpressure work?

On Wed, Dec 17, 2014 at 8:07 AM, Malcolm Sparks  wrote:
>
> Here is a solution for SSE over http-kit channels
>
> (defn server-event-source [ch] ;; Adding a mult here is dangerous because
> it bleeds the underlying ;; channel dry. We should provide the mult via
> modular.async (let [m (async/mult ch)] (fn [req] (let [ch (async/chan 16)]
> (async/tap m ch) (with-channel req net-ch (on-close net-ch (fn [_]
> (async/untap m ch) (async/close! ch))) (send! net-ch {:headers headers}
> false) (go-loop [] (when-let [data ( (send! net-ch (->message data) false) (recur
>
>
> https://github.com/juxt/modular/blob/master/modules/http-kit-events/src/modular/http_kit/events.clj
>
>
> On Saturday, 6 December 2014 17:03:41 UTC, Lars Ole Avery Simonsen wrote:
>>
>> Hi guys
>>
>> I am trying to implement Server Sent Event support in a small hobby
>> project based on the http-kit server framework and compojure.
>>
>> I am still quite new to clojure in general, so it may very well be that I
>> am missing something obvious, but this SSE detail has me stumped.
>>
>> What I have tried is to take inspiration from the Eventual
>>  library for the SSE implementation
>> (which is aimed at jetty). I have changed the Eventual implementation so
>> that it uses the http-kit async-channel implementation for data
>> transmission. What I am seeing is that everything seems to run without
>> errors except for the fact that no data is being received at the
>> EventSource in the browser.
>>
>> If I allow the sse channel to close immediately after opening (which is
>> pretty useless), the initial message is received at the browser side, but
>> if I keep the connection open, nothing gets received.
>>
>> Having had a look at the http-kit internals, my suspicion is that the
>> socket doesn't get flushed and so all my attempts at communication are
>> sitting in some buffer somewhere.
>>
>> All SSE implementations I have looked at for other platforms have
>> explicit calls to flush the socket buffers on the long lived connection
>> after each message transmission.
>>
>> Does anyone have any suggestions as to a better way to achieve SSE
>> support in a http-kit based server? Or maybe ideas as to what I might be
>> doing wrong?
>>
>> Thanks in advance.
>>
>> LOAS
>>
>>  --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/DbBzM3AG9wM/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Server Sent Events under Http-Kit

[Zach Tellman]

I'm not sure about http-kit's streaming response implementation, but in 
Aleph [1] if you use a stream/channel as a response body each message from 
the stream will be immediately sent as an HTTP chunk.

Best,
Zach

[1] https://github.com/ztellman/aleph

On Saturday, December 6, 2014 9:03:41 AM UTC-8, Lars Ole Avery Simonsen 
wrote:
>
> Hi guys
>
> I am trying to implement Server Sent Event support in a small hobby 
> project based on the http-kit server framework and compojure.
>
> I am still quite new to clojure in general, so it may very well be that I 
> am missing something obvious, but this SSE detail has me stumped.
>
> What I have tried is to take inspiration from the Eventual 
>  library for the SSE implementation 
> (which is aimed at jetty). I have changed the Eventual implementation so 
> that it uses the http-kit async-channel implementation for data 
> transmission. What I am seeing is that everything seems to run without 
> errors except for the fact that no data is being received at the 
> EventSource in the browser.
>
> If I allow the sse channel to close immediately after opening (which is 
> pretty useless), the initial message is received at the browser side, but 
> if I keep the connection open, nothing gets received.
>
> Having had a look at the http-kit internals, my suspicion is that the 
> socket doesn't get flushed and so all my attempts at communication are 
> sitting in some buffer somewhere.
>
> All SSE implementations I have looked at for other platforms have explicit 
> calls to flush the socket buffers on the long lived connection after each 
> message transmission.
>
> Does anyone have any suggestions as to a better way to achieve SSE support 
> in a http-kit based server? Or maybe ideas as to what I might be doing 
> wrong?
>
> Thanks in advance.
>
> LOAS
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] riffle, a high-performance write-once key/value storage engine

[Zach Tellman]

The second of the libraries Factual is open sourcing today is Riffle, which 
is a write-once key/value storage engine.  By disallowing random writes, it 
allows for O(1) on-disk reads, and a read throughput is both faster and 
significantly more stable than LevelDB and other, similar databases.  Of 
course, any changes to the database require merging two existing databases 
or writing another one from scratch, which can be done either in-process or 
using Hadoop.  

A blog post describing the library can be found 
at 
http://blog.factual.com/how-factual-uses-persistent-storage-for-its-real-time-services,
 
and the library itself can be found at https://github.com/factual/riffle. 
 We look forward to see how it's used by others.

Zach

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] s3-journal, a library for high-throughput log journaling

[Zach Tellman]

Today Factual is open sourcing two libraries, the first of which is a 
relatively simple library that journals data to Amazon's S3 service [1], 
and buffers data on disk using durable-queue [2], making it much more 
robust than most S3 clients to process failure, network issues, and a host 
of other issues.  We have used it to write over a trillion entries this 
year, and expect it will become equally indispensable for others in the 
Clojure community.

A blog post describing it in more detail can be found at 
http://blog.factual.com/how-factual-uses-persistent-storage-for-its-real-time-services.
 
 The library itself can be found at https://github.com/factual/s3-journal.

Zach 

[1] http://aws.amazon.com/s3/
[2] https://github.com/factual/durable-queue

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

updated web server benchmarks

[Zach Tellman]

New benchmarks have been run for a variety of Clojure web servers, and the 
results can be found 
at 
https://github.com/ptaoussanis/clojure-web-server-benchmarks#ubuntu-1404--2x-xeon-x5650.
 
 The top-performing servers are based on Undertow (ring-undertow, Immutant 
v2), Netty (Aleph), and nginx (nginx-clojure) respectively.

The benchmark hardware was dual-socket 2010-era Xeons, which is the 
hardware profile of choice at Factual.  This means that there were 24 
hardware threads, but each core was noticeably slower than those on modern 
desktop processors.  This caused http-kit, which only uses a single thread 
for I/O, to perform worse than it would have on a system with fewer, faster 
cores.  

As always, these benchmarks are very contrived, and not representative of 
most real-world applications.  Adding something as simple as Ring's 
`wrap-keyword-params` middleware on Clojure 1.6.0 would reduce throughput 
on all servers by a factor of 3-4x.  Luckily keyword creation will be much 
cheaper in Clojure 1.7.0, so this particular problem won't exist for much 
longer, but it's important to remember that performance is a property of 
your entire application, not just the underlying web server.

With that said, it's great to see that so many people are taking server 
performance in Clojure seriously.  However contrived these benchmarks are, 
a lot of people use them as a proxy for a language's "maturity", and 
further effort in this space can only help Clojure's adoption.

Zach

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] async-sockets - work with sockets using core.async channels

[Zach Tellman]

A slightly more straightforward implementation can be found 
at https://gist.github.com/ztellman/fb64e81d1d7f0b261ccd.  I'm fairly sure 
it's equivalent to Cristophe's, but I may be missing some nuance.  At any 
rate, I've found using the async/put! and callback mechanism to be a much 
more straightforward way to do interop with non-core.async code.

And yes, reimplementing a TCP-like ack mechanism on top of WebSockets is 
not something you want to do.  The existing stack will do it better and 
faster than you can.  Just to be clear, this is a large part of why I wrote 
Manifold [1], which can easily be turned into a core.async channel, but 
provides an API which is designed for interop with other stream mechanisms 
(including synchronous ones like Java's BlockingQueues).  core.async is a 
framework, meaning it brings not only a stream representation, but an 
entire execution model; using core.async should be an application-level 
decision, not one made for you by your libraries

Zach

[1] https://github.com/ztellman/manifold


On Sunday, October 12, 2014 9:42:32 AM UTC-7, Ryan Waters wrote:
>
> I was just starting to use Sente [1] (which relies on httpkit [2]) and 
> this conversation is a real eye opener.  Unless a person uses a library 
> that supports backpressure, as mentioned earlier, your 
> transport-concern-made-opaque-because-of-core-async must become an 
> application-level concern.  The far-side of the communication would have to 
> respond with an application level acknowledgement for local sends and the 
> local side would need to not send data unless acks were received for 
> previously sent data.
>
> E.g. this could be implemented with core.async by using a pair of channels 
> (instead of a single channel) for all 'sends' where one channel is used for 
> data while the other channel waits for acknowledgement of put data (a 
> 'control' channel).  This would have the unfortunate side effect of hurting 
> throughput.  A better system would be to allow for a certain number of 
> unacknowledged sends before backing off.  Of course, now a person is 
> implementing what was created for TCP at the level of their application.
>
> Christophe's approach means you at least wouldn't have to do the above, 
> replacing it instead with a per backend implementation.  I hope somebody 
> else is able to explain it better.
>
> Looking forward to an Aleph rewrite!!
>
> - - -
> [1] https://github.com/ptaoussanis/sente
> [2] https://github.com/http-kit/http-kit
>
> On Sat, Oct 11, 2014 at 8:01 PM, Julian 
> > wrote:
>
>> Hi Zach, 
>>
>> Thanks for the clarity of thought that went into this post. 
>>
>> Perhaps it is obvious to everyone but me, but I saw this post by 
>> Christophe Grande yesterday that appears to address these concerns:
>> "Back-pressurized interop for core.async" 
>> https://twitter.com/cgrand/status/520566182194450432
>> https://gist.github.com/cgrand/767673242b7f7c27f35a
>>
>> I'm interested to hear if this solves your problem or is about something 
>> else. 
>>
>> Cheers
>> Julian
>>
>>
>> On Wednesday, 8 October 2014 17:00:02 UTC+11, Zach Tellman wrote:
>>>
>>> The reason the thread-per-connection approach is nice is because it 
>>> correctly propagates backpressure.  If we're copying data from a source to 
>>> a sink (let's say reading it in from the network and writing to a file), 
>>> it's possible that the production of data may outstrip the consumption.  If 
>>> this happens, we need to make sure the producer slows down, or we risk 
>>> running out of memory.  In Java, the producer is typically connected to the 
>>> consumer via a blocking queue, and if the queue fills up the producer can't 
>>> send anything more to the consumer.  A Java socket is one such queue, and 
>>> if it fills up it will exert backpressure via TCP.  This will work no 
>>> matter how many queues or other mechanisms separate the producer and 
>>> consumer.
>>>
>>> However, every attempt I've seen to marry core.async to an async network 
>>> stack has been fundamentally broken, in that it doesn't do this.  Often, 
>>> they'll just use 'put!', which works fine until the channel's queue fills 
>>> up, and 1024 pending puts are accumulated, and finally the channel throws 
>>> an exception.  Alternately, they'll use a blocking put on the channel, 
>>> which means that any backpressure will also extend to whatever other 
>>> connections are sharing that thread or the thread pool.  Note that the 
>>> software 

Re: [ANN] async-sockets - work with sockets using core.async channels

[Zach Tellman]

Yes, I didn't mean to imply that there are no blocking operations anywhere,
only that nothing in the worker threads (where the put! occurs) will block,
and that backpressure isn't exerted by causing an active thread to hang.

As to the second point, I'm not sure why you'd think that.  These aren't
futures that are being accessed via blocking dereference, so there's no
reason any thread would block as a result of an unrealized future/deferred.



On Wed, Oct 8, 2014 at 12:22 PM, Jozef Wagner 
wrote:

> Thank you! Using put! callback to control backpressure is a very elegant
> solution. BTW there always has to be blocking somewhere. Netty uses
> selector to block at [1] and .setAutoRead causes respective channel to
> deregister itself from a selector [2], until put! completes.
>
> Regarding the other way around, it seems to me from the quick look (sorry
> if I misinterpreted) that using futures to represent write completion will
> cause the eventual backpressure to block the thread, causing analogous
> drawback as a blocking put! in the "reading from multiple connection" case.
>
> [1]
> https://github.com/netty/netty/blob/220660e351b2a22112b19c4af45e403eab1f73ab/transport/src/main/java/io/netty/channel/nio/NioEventLoop.java#L625
> [2]
> https://github.com/netty/netty/blob/220660e351b2a22112b19c4af45e403eab1f73ab/transport/src/main/java/io/netty/channel/nio/AbstractNioChannel.java#L173
>
> Jozef
>
> On Wednesday, October 8, 2014 7:16:44 PM UTC+2, Zach Tellman wrote:
>>
>> The documentation for Manifold can explain the API better than I can
>> here.  The point where that interacts with Netty w.r.t. backpressure is
>> here: https://github.com/ztellman/aleph/blob/0.4.0/src/
>> aleph/netty.clj#L109.  Here the stream represents data coming off the
>> wire, and if the put onto the stream is not immediately successful,
>> backpressure is enabled until the put completes.  No blocking required
>> anywhere.
>>
>> On Wed, Oct 8, 2014 at 10:10 AM, Jozef Wagner 
>> wrote:
>>
>>> If you want to handle multiple TCP connections and async channels in one
>>> thread, you need a way how to block on both connections (wait for new input
>>> to arrive) and channels (wait for a free space in a buffer). Blocking only
>>> on connections will get you a busy loop if channels are full. If you could
>>> point me to the part of Aleph sources that handles this issue, I would be
>>> very grateful. I'm not familiar with netty API nor manifold's concepts, so
>>> I'm having trouble navigating in the Aleph sources.
>>>
>>> Thanks,
>>> Jozef
>>>
>>> On Wednesday, October 8, 2014 6:15:47 PM UTC+2, Zach Tellman wrote:
>>>>
>>>> I wasn't aware of hermod, that's interesting.  I would still
>>>> characterize its approach to backpressure as "broken", though, since when
>>>> the queues get full it silently drops messages on the ground.  In fairness,
>>>> this is very clearly documented, so it's less pernicious than some of the
>>>> other cases out there.
>>>>
>>>> Both core.async buffers and SelectableChannels have very particular
>>>> semantics, and I would be very surprised if they could be combined in that
>>>> way.  It's perfectly possible to feed one into the other and handle
>>>> backpressure properly (again, I'm doing just that with Aleph 0.4.0, using
>>>> Netty), but it's a nuanced integration and easy to get wrong.
>>>>
>>>> On Wed, Oct 8, 2014 at 7:12 AM,  wrote:
>>>>
>>>>> Check out https://github.com/halgari/com.tbaldridge.hermod for an
>>>>> interesting take on this.
>>>>>
>>>>> On Wednesday, October 8, 2014 1:17:11 AM UTC-4, Sun Ning wrote:
>>>>>>
>>>>>>  BTW, is there any network based core.async channel available now?
>>>>>>
>>>>>> On 10/08/2014 04:36 AM, adrian...@mail.yu.edu wrote:
>>>>>>
>>>>>>  It's not about 'safety' (depending on what that means in this
>>>>>> context), but as Zach pointed out, if you aren't careful about 
>>>>>> backpressure
>>>>>> you can run into performance bottlenecks with unrestrained async IO
>>>>>> operations because although they let you code as if you could handle an
>>>>>> unlimited amount of connections, obviously that isn't true. There is 
>>>>>> only a
>>>>>> finite amount of data th

Re: [ANN] async-sockets - work with sockets using core.async channels

[Zach Tellman]

Sorry, didn't cover converse case.  That's handled by the ChannelSink
directly underneath.  Note that each write returns a Netty ChannelFuture
representing completion of the write, which is transformed into a Manifold
deferred.  Any time a Manifold put returns an unrealized deferred, that
creates upstream backpressure.

On Wed, Oct 8, 2014 at 10:16 AM, Zach Tellman  wrote:

> The documentation for Manifold can explain the API better than I can
> here.  The point where that interacts with Netty w.r.t. backpressure is
> here:
> https://github.com/ztellman/aleph/blob/0.4.0/src/aleph/netty.clj#L109.
> Here the stream represents data coming off the wire, and if the put onto
> the stream is not immediately successful, backpressure is enabled until the
> put completes.  No blocking required anywhere.
>
> On Wed, Oct 8, 2014 at 10:10 AM, Jozef Wagner 
> wrote:
>
>> If you want to handle multiple TCP connections and async channels in one
>> thread, you need a way how to block on both connections (wait for new input
>> to arrive) and channels (wait for a free space in a buffer). Blocking only
>> on connections will get you a busy loop if channels are full. If you could
>> point me to the part of Aleph sources that handles this issue, I would be
>> very grateful. I'm not familiar with netty API nor manifold's concepts, so
>> I'm having trouble navigating in the Aleph sources.
>>
>> Thanks,
>> Jozef
>>
>> On Wednesday, October 8, 2014 6:15:47 PM UTC+2, Zach Tellman wrote:
>>>
>>> I wasn't aware of hermod, that's interesting.  I would still
>>> characterize its approach to backpressure as "broken", though, since when
>>> the queues get full it silently drops messages on the ground.  In fairness,
>>> this is very clearly documented, so it's less pernicious than some of the
>>> other cases out there.
>>>
>>> Both core.async buffers and SelectableChannels have very particular
>>> semantics, and I would be very surprised if they could be combined in that
>>> way.  It's perfectly possible to feed one into the other and handle
>>> backpressure properly (again, I'm doing just that with Aleph 0.4.0, using
>>> Netty), but it's a nuanced integration and easy to get wrong.
>>>
>>> On Wed, Oct 8, 2014 at 7:12 AM,  wrote:
>>>
>>>> Check out https://github.com/halgari/com.tbaldridge.hermod for an
>>>> interesting take on this.
>>>>
>>>> On Wednesday, October 8, 2014 1:17:11 AM UTC-4, Sun Ning wrote:
>>>>>
>>>>>  BTW, is there any network based core.async channel available now?
>>>>>
>>>>> On 10/08/2014 04:36 AM, adrian...@mail.yu.edu wrote:
>>>>>
>>>>>  It's not about 'safety' (depending on what that means in this
>>>>> context), but as Zach pointed out, if you aren't careful about 
>>>>> backpressure
>>>>> you can run into performance bottlenecks with unrestrained async IO
>>>>> operations because although they let you code as if you could handle an
>>>>> unlimited amount of connections, obviously that isn't true. There is only 
>>>>> a
>>>>> finite amount of data that can be buffered in and out of any network
>>>>> according to its hardware. When you don't regulate that, your system will
>>>>> end up spending an inordinate amount of time compensating for this. You
>>>>> don't need to worry about this with "regular io" because the "thread per
>>>>> connection" abstraction effectively bounds your activity within the
>>>>> acceptable physical constraints of the server.
>>>>>
>>>>> On Tuesday, October 7, 2014 2:49:30 PM UTC-4, Brian Guthrie wrote:
>>>>>>
>>>>>>
>>>>>> On Mon, Oct 6, 2014 at 12:10 AM,  wrote:
>>>>>>
>>>>>>> Zach makes an excellent point; I've used AsyncSocketChannels and its
>>>>>>> irk (http://docs.oracle.com/javase/8/docs/api/java/nio/channels/
>>>>>>> AsynchronousServerSocketChannel.html), with core.async in the past.
>>>>>>> Perhaps replacing your direct java.net.Sockets with nio classes that 
>>>>>>> can be
>>>>>>> given CompletionHandlers (http://docs.oracle.com/javase
>>>>>>> /7/docs/api/java/nio/channels/CompletionHandler.html) would be a
>>>>>>> better fit.
>>>&g

Re: [ANN] async-sockets - work with sockets using core.async channels

[Zach Tellman]

The documentation for Manifold can explain the API better than I can here.
The point where that interacts with Netty w.r.t. backpressure is here:
https://github.com/ztellman/aleph/blob/0.4.0/src/aleph/netty.clj#L109.
Here the stream represents data coming off the wire, and if the put onto
the stream is not immediately successful, backpressure is enabled until the
put completes.  No blocking required anywhere.

On Wed, Oct 8, 2014 at 10:10 AM, Jozef Wagner 
wrote:

> If you want to handle multiple TCP connections and async channels in one
> thread, you need a way how to block on both connections (wait for new input
> to arrive) and channels (wait for a free space in a buffer). Blocking only
> on connections will get you a busy loop if channels are full. If you could
> point me to the part of Aleph sources that handles this issue, I would be
> very grateful. I'm not familiar with netty API nor manifold's concepts, so
> I'm having trouble navigating in the Aleph sources.
>
> Thanks,
> Jozef
>
> On Wednesday, October 8, 2014 6:15:47 PM UTC+2, Zach Tellman wrote:
>>
>> I wasn't aware of hermod, that's interesting.  I would still characterize
>> its approach to backpressure as "broken", though, since when the queues get
>> full it silently drops messages on the ground.  In fairness, this is very
>> clearly documented, so it's less pernicious than some of the other cases
>> out there.
>>
>> Both core.async buffers and SelectableChannels have very particular
>> semantics, and I would be very surprised if they could be combined in that
>> way.  It's perfectly possible to feed one into the other and handle
>> backpressure properly (again, I'm doing just that with Aleph 0.4.0, using
>> Netty), but it's a nuanced integration and easy to get wrong.
>>
>> On Wed, Oct 8, 2014 at 7:12 AM,  wrote:
>>
>>> Check out https://github.com/halgari/com.tbaldridge.hermod for an
>>> interesting take on this.
>>>
>>> On Wednesday, October 8, 2014 1:17:11 AM UTC-4, Sun Ning wrote:
>>>>
>>>>  BTW, is there any network based core.async channel available now?
>>>>
>>>> On 10/08/2014 04:36 AM, adrian...@mail.yu.edu wrote:
>>>>
>>>>  It's not about 'safety' (depending on what that means in this
>>>> context), but as Zach pointed out, if you aren't careful about backpressure
>>>> you can run into performance bottlenecks with unrestrained async IO
>>>> operations because although they let you code as if you could handle an
>>>> unlimited amount of connections, obviously that isn't true. There is only a
>>>> finite amount of data that can be buffered in and out of any network
>>>> according to its hardware. When you don't regulate that, your system will
>>>> end up spending an inordinate amount of time compensating for this. You
>>>> don't need to worry about this with "regular io" because the "thread per
>>>> connection" abstraction effectively bounds your activity within the
>>>> acceptable physical constraints of the server.
>>>>
>>>> On Tuesday, October 7, 2014 2:49:30 PM UTC-4, Brian Guthrie wrote:
>>>>>
>>>>>
>>>>> On Mon, Oct 6, 2014 at 12:10 AM,  wrote:
>>>>>
>>>>>> Zach makes an excellent point; I've used AsyncSocketChannels and its
>>>>>> irk (http://docs.oracle.com/javase/8/docs/api/java/nio/channels/
>>>>>> AsynchronousServerSocketChannel.html), with core.async in the past.
>>>>>> Perhaps replacing your direct java.net.Sockets with nio classes that can 
>>>>>> be
>>>>>> given CompletionHandlers (http://docs.oracle.com/javase
>>>>>> /7/docs/api/java/nio/channels/CompletionHandler.html) would be a
>>>>>> better fit.
>>>>>>
>>>>>
>>>>> Once I do some performance instrumentation I'll give that a shot. I
>>>>> admit that I'm not familiar with all the implications of using the nio
>>>>> classes; were I to switch, is it safe to continue using go blocks, or is 
>>>>> it
>>>>> worth explicitly allocating a single thread per socket?
>>>>>
>>>>>  Brian
>>>>>
>>>>  --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Clojure" group.
>>>> To post to this group, send email to clo...@googlegroups.com
>

Re: [ANN] rmap - define lazy, recursive maps

[Zach Tellman]

Hi Arnout,

This is interesting, but may be a lot less useful than you think without 
filling in all the other methods that a normal map has.  For instance, you 
cannot do an equality check with another map (no use of 
clojure.lang.MapEquivalence or implementation of equals and equiv), nor use 
it in a set or key in another map (no implementation of hashCode and 
hasheq), etc. etc.  Unless you and everyone else using it is certain that 
every line of code in your application and the libraries you depend on 
don't do this, this is a ticking timebomb.

For an example of a full map implementation, it might be helpful to take a 
look at 
https://github.com/ztellman/potemkin/blob/master/src/potemkin/collections.clj, 
and will definitely be useful to 
run https://github.com/ztellman/collection-check against your map 
implementation.  One issue that collection-check would uncover is the fact 
that the underlying LinkedHashMap that you're using has different key 
equality semantics than Clojure does, so calling (-> m (assoc 1 :foo) 
(dissoc 1N)) would likely break a bunch of stuff.  

Please don't be discouraged by this, making custom map-like data structures 
is much harder right now than it should be.  I just want to make sure no 
one gets an unpleasant surprise in production somewhere down the line.

Best,
Zach

On Monday, October 6, 2014 1:45:02 AM UTC-7, Arnout Roemers wrote:
>
> Thank you for your kind responses! I will look into writing a post that 
> shows what the motivation was (i.e. how I use it), how it works, and how it 
> compares to Prismatic's Graph. 
>
> Cheers,
> Arnout
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] async-sockets - work with sockets using core.async channels

[Zach Tellman]

I wasn't aware of hermod, that's interesting.  I would still characterize
its approach to backpressure as "broken", though, since when the queues get
full it silently drops messages on the ground.  In fairness, this is very
clearly documented, so it's less pernicious than some of the other cases
out there.

Both core.async buffers and SelectableChannels have very particular
semantics, and I would be very surprised if they could be combined in that
way.  It's perfectly possible to feed one into the other and handle
backpressure properly (again, I'm doing just that with Aleph 0.4.0, using
Netty), but it's a nuanced integration and easy to get wrong.

On Wed, Oct 8, 2014 at 7:12 AM,  wrote:

> Check out https://github.com/halgari/com.tbaldridge.hermod for an
> interesting take on this.
>
> On Wednesday, October 8, 2014 1:17:11 AM UTC-4, Sun Ning wrote:
>>
>>  BTW, is there any network based core.async channel available now?
>>
>> On 10/08/2014 04:36 AM, adrian...@mail.yu.edu wrote:
>>
>>  It's not about 'safety' (depending on what that means in this context),
>> but as Zach pointed out, if you aren't careful about backpressure you can
>> run into performance bottlenecks with unrestrained async IO operations
>> because although they let you code as if you could handle an unlimited
>> amount of connections, obviously that isn't true. There is only a finite
>> amount of data that can be buffered in and out of any network according to
>> its hardware. When you don't regulate that, your system will end up
>> spending an inordinate amount of time compensating for this. You don't need
>> to worry about this with "regular io" because the "thread per connection"
>> abstraction effectively bounds your activity within the acceptable physical
>> constraints of the server.
>>
>> On Tuesday, October 7, 2014 2:49:30 PM UTC-4, Brian Guthrie wrote:
>>>
>>>
>>> On Mon, Oct 6, 2014 at 12:10 AM,  wrote:
>>>
 Zach makes an excellent point; I've used AsyncSocketChannels and its
 irk (http://docs.oracle.com/javase/8/docs/api/java/nio/channels/
 AsynchronousServerSocketChannel.html), with core.async in the past.
 Perhaps replacing your direct java.net.Sockets with nio classes that can be
 given CompletionHandlers (http://docs.oracle.com/
 javase/7/docs/api/java/nio/channels/CompletionHandler.html) would be a
 better fit.

>>>
>>> Once I do some performance instrumentation I'll give that a shot. I
>>> admit that I'm not familiar with all the implications of using the nio
>>> classes; were I to switch, is it safe to continue using go blocks, or is it
>>> worth explicitly allocating a single thread per socket?
>>>
>>>  Brian
>>>
>>  --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clo...@googlegroups.com
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+u...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to clojure+u...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>>   --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/TVMQJwaij1U/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] async-sockets - work with sockets using core.async channels

[Zach Tellman]

The reason the thread-per-connection approach is nice is because it 
correctly propagates backpressure.  If we're copying data from a source to 
a sink (let's say reading it in from the network and writing to a file), 
it's possible that the production of data may outstrip the consumption.  If 
this happens, we need to make sure the producer slows down, or we risk 
running out of memory.  In Java, the producer is typically connected to the 
consumer via a blocking queue, and if the queue fills up the producer can't 
send anything more to the consumer.  A Java socket is one such queue, and 
if it fills up it will exert backpressure via TCP.  This will work no 
matter how many queues or other mechanisms separate the producer and 
consumer.

However, every attempt I've seen to marry core.async to an async network 
stack has been fundamentally broken, in that it doesn't do this.  Often, 
they'll just use 'put!', which works fine until the channel's queue fills 
up, and 1024 pending puts are accumulated, and finally the channel throws 
an exception.  Alternately, they'll use a blocking put on the channel, 
which means that any backpressure will also extend to whatever other 
connections are sharing that thread or the thread pool.  Note that the 
software that uses core.async in this way may work flawlessly in a wide 
variety of cases, but there's still an intractable failure mode lying in 
wait.

In some cases, such as http-kit's websocket mechanism, there's no way to 
even exert backpressure (you register a callback, and have no way to 
indicate in your callback that you can't handle more messages).  This means 
that any attempt to use http-kit in conjunction with core.async will be 
subtly but fundamentally broken.  Arguably, even without core.async in the 
equation it's broken.  This is not a good state of affairs.  I'll admit 
that it took me a few failures in production to realize how important 
correct handling of backpressure is, but this isn't something that our 
ecosystem can afford to ignore, especially as Clojure is used for 
larger-scale projects.

I will note that I am working on a solution to this, in the form of the 
upcoming Aleph release [1].  This will model every network connection via 
streams that can trivially be converted into core.async channels [2], and 
which exert backpressure over TCP wherever necessary without requiring a 
thread per connection.  A formal beta should be available in the near 
future (it's already handling billions of requests a day in production 
without issue).

Zach

[1] https://github.com/ztellman/aleph/tree/0.4.0
[2] https://github.com/ztellman/manifold



On Tuesday, October 7, 2014 1:36:16 PM UTC-7, adrian...@mail.yu.edu wrote:
>
> It's not about 'safety' (depending on what that means in this context), 
> but as Zach pointed out, if you aren't careful about backpressure you can 
> run into performance bottlenecks with unrestrained async IO operations 
> because although they let you code as if you could handle an unlimited 
> amount of connections, obviously that isn't true. There is only a finite 
> amount of data that can be buffered in and out of any network according to 
> its hardware. When you don't regulate that, your system will end up 
> spending an inordinate amount of time compensating for this. You don't need 
> to worry about this with "regular io" because the "thread per connection" 
> abstraction effectively bounds your activity within the acceptable physical 
> constraints of the server. 
>
> On Tuesday, October 7, 2014 2:49:30 PM UTC-4, Brian Guthrie wrote:
>>
>>
>> On Mon, Oct 6, 2014 at 12:10 AM,  wrote:
>>
>>> Zach makes an excellent point; I've used AsyncSocketChannels and its irk 
>>> (
>>> http://docs.oracle.com/javase/8/docs/api/java/nio/channels/AsynchronousServerSocketChannel.html),
>>>  
>>> with core.async in the past. Perhaps replacing your direct java.net.Sockets 
>>> with nio classes that can be given CompletionHandlers (
>>> http://docs.oracle.com/javase/7/docs/api/java/nio/channels/CompletionHandler.html)
>>>  
>>> would be a better fit. 
>>>
>>
>> Once I do some performance instrumentation I'll give that a shot. I admit 
>> that I'm not familiar with all the implications of using the nio classes; 
>> were I to switch, is it safe to continue using go blocks, or is it worth 
>> explicitly allocating a single thread per socket?
>>
>> Brian
>>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to c

Re: [ANN] async-sockets - work with sockets using core.async channels

[Zach Tellman]

Please note that if you use core.async with java.nio, you need to make sure 
backpressure is properly propagated (this happens automatically with 
java.io, assuming you have a thread per connection).

On Sunday, October 5, 2014 9:10:24 PM UTC-7, adrian...@mail.yu.edu wrote:
>
> Zach makes an excellent point; I've used AsyncSocketChannels and its irk (
> http://docs.oracle.com/javase/8/docs/api/java/nio/channels/AsynchronousServerSocketChannel.html),
>  
> with core.async in the past. Perhaps replacing your direct java.net.Sockets 
> with nio classes that can be given CompletionHandlers (
> http://docs.oracle.com/javase/7/docs/api/java/nio/channels/CompletionHandler.html)
>  
> would be a better fit. 
>
> On Sunday, October 5, 2014 11:57:18 PM UTC-4, Zach Tellman wrote:
>>
>> If I'm reading this correctly, you're using non-blocking thread pools for 
>> blocking operations on the sockets.  Given more than N connections (last 
>> time I looked the thread pool's size was 42), you risk deadlock or at the 
>> very least poor average throughput.
>>
>> On Sunday, October 5, 2014 7:06:56 PM UTC-7, Brian Guthrie wrote:
>>>
>>> Hi all,
>>>
>>> I'm releasing a little library for working with sockets. Feedback and 
>>> pull requests gratefully appreciated.
>>>
>>> The skinny
>>> ---
>>>
>>> This library allows you to create socket servers and socket clients and 
>>> interact with them asynchronously using channels. Servers return a record 
>>> with a :connections field, a channel which yields one socket per incoming 
>>> connection. Clients return the same socket record. Socket records each have 
>>> an :in and :out channel each which allow you to receive and send data 
>>> respectively on a line-by-line basis. The raw java.net.Socket is also 
>>> available (as :socket).
>>>
>>> Servers and clients are defined using the Component framework and must 
>>> be explicitly started using (component/start ), though 
>>> sockets will clean up after themselves if they are terminated for some 
>>> reason.
>>>
>>> Further information is available on Github here: 
>>> https://github.com/bguthrie/async-sockets
>>>
>>> Releases
>>> --
>>>
>>> This is the first release, which I've tagged for now as 0.0.1-SNAPSHOT. 
>>> Leiningen dependency: [com.gearswithingears/async-sockets "0.0.1-SNAPSHOT"].
>>>
>>> If this is useful to you, please let me know, but any and all feedback 
>>> is great.
>>>
>>> Happy hacking,
>>>
>>> Brian
>>> @bguthrie
>>> btgu...@gmail.com
>>>
>>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Re: Handling java streams..

[Zach Tellman]

Calling (-> stream .iterator iterator-seq) should give the desired result.

On Wednesday, October 1, 2014 6:39:11 AM UTC-7, José Ricardo wrote:
>
> By nicer I meant something like being able to use, for example, map and 
> filter, just like I can do on a java.util.ArrayList (in clojure) and on a 
> Stream (in java).
>
> On Saturday, September 27, 2014 1:47:38 AM UTC-4, Sean Corfield wrote:
>>
>> On Fri, Sep 26, 2014 at 11:51 AM, José Ricardo  
>> wrote: 
>> > Hi, I'm not sure if resurrecting this thread is the right approach, but 
>> what 
>> > about Java 8 Streams (java.util.stream)? 
>> > 
>> > Are there any libraries out there for making java 8 streams handling 
>> nicer? 
>> > :) 
>>
>> Well, this thread is certainly a blast from the past :) 
>>
>> Can you be a bit more specific about what you'd like to see "nicer" 
>> with Java 8 streams, and what you'd like to see in a Clojure wrapper? 
>>
>> I ask because Java 8 is the first version that I've been interested in 
>> since I felt it went off the rails with Java 5 - and streams are part 
>> of what makes Java 8 attractive again :) 
>> -- 
>> Sean A Corfield -- (904) 302-SEAN 
>> An Architect's View -- http://corfield.org/ 
>> World Singles, LLC. -- http://worldsingles.com/ 
>>
>> "Perfection is the enemy of the good." 
>> -- Gustave Flaubert, French realist novelist (1821-1880) 
>>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] async-sockets - work with sockets using core.async channels

[Zach Tellman]

If I'm reading this correctly, you're using non-blocking thread pools for 
blocking operations on the sockets.  Given more than N connections (last 
time I looked the thread pool's size was 42), you risk deadlock or at the 
very least poor average throughput.

On Sunday, October 5, 2014 7:06:56 PM UTC-7, Brian Guthrie wrote:
>
> Hi all,
>
> I'm releasing a little library for working with sockets. Feedback and pull 
> requests gratefully appreciated.
>
> The skinny
> ---
>
> This library allows you to create socket servers and socket clients and 
> interact with them asynchronously using channels. Servers return a record 
> with a :connections field, a channel which yields one socket per incoming 
> connection. Clients return the same socket record. Socket records each have 
> an :in and :out channel each which allow you to receive and send data 
> respectively on a line-by-line basis. The raw java.net.Socket is also 
> available (as :socket).
>
> Servers and clients are defined using the Component framework and must be 
> explicitly started using (component/start ), though 
> sockets will clean up after themselves if they are terminated for some 
> reason.
>
> Further information is available on Github here: 
> https://github.com/bguthrie/async-sockets
>
> Releases
> --
>
> This is the first release, which I've tagged for now as 0.0.1-SNAPSHOT. 
> Leiningen dependency: [com.gearswithingears/async-sockets "0.0.1-SNAPSHOT"].
>
> If this is useful to you, please let me know, but any and all feedback is 
> great.
>
> Happy hacking,
>
> Brian
> @bguthrie
> btgu...@gmail.com 
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] data.int-map 0.1.0

[Zach Tellman]

https://github.com/clojure/data.int-map

This contrib library represents the union of two other libraries [1] [2], 
which are now both deprecated.  There's nothing too surprising here, but 
I'm happy to answer any questions.

Zach

[1] https://github.com/ztellman/immutable-int-map
[2] https://github.com/ztellman/immutable-bitset

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Howto Use Automat ?

[Zach Tellman]

Hi Tim,

Glad to see you're trying Automat out.

I'm not sure what  you mean by "forks in the road".  If your FSM is defined 
as (a/or [1 3] [1 2 3]), then you can do any of the following:

(reduce #(a/advance fsm %1 %2) nil [1 2 3])

(a/advance-stream fsm nil [1 2 3] ::rejected)

(let [adv (partial a/advance fsm)]
  (-> nil (adv 1) (adv 2) (adv 3)))

Does that answer your first two questions?  If not, you'll have to clarify 
what you're asking.

Automat doesn't allow you to name your states, because that breaks your 
ability to freely compose automata.  If you want to differentiate between 
two different paths, you can use actions:

  (view (a/or [:sent :acknowledged :accepted] [:sent :rejected (a/$ 
:not-accepted)]))

I can go into more depth on this, but the README gives a fairly complete 
explanation of how actions can be used.

Hope that helps,
Zach




On Tuesday, August 26, 2014 11:42:24 AM UTC-7, frye wrote:
>
> I'm trying to use Automat to map out transitions that are either of the 
> paths below. 
>
>- There can be many claims sent to an external service. Each claim, 
>after being sent, can be :acknowledged or :rejected. 
>- If a claim is :acknowledged, it can then either be :accepted or 
>:rejected. 
>- There will be many such claims 
>
> [:sent :acknowledged :accepted]
> [:sent :acknowledged :rejected]
> [:sent :rejected]
>
>
> I'm trying to figure out how to use Automat to handle this use case. 
>
>- The only way to advance through your FSM, seems to be to call 
>fsm/advance (or fsm/advance-stream). How do we deal with forks in the road?
>- Looking at the documentation 
> and tests 
>
> , 
>it's not clear to me if I can pass multiple tokens (or claims) through 1 
>FSM. Are we meant to use 1 FSM per token (claim in this case)? 
>- The semantics for Automat have you declare transitions, instead of 
>states. States are implicit numbers between the transitions. Pallet-fsm 
> seems to follow the state 
>declaration pattern, but hasn't been updated in a few years. Can I name 
> the 
>states and transitions between them? 
>- There always seems to be one output state, even if they come from 2 
>different transitions. So the below 2 FSMs are equivalent. However, they 
>both go to the same end state. Is there a way to define different output 
>states?  
>
> => (require '[automat.viz :refer (view)])
> => (require '[automat.core :as a])
>
> => (view (a/or [:sent :acknowledged :accepted] [:sent :acknowledged 
> :rejected] [:sent :rejected]))
> => (view [:sent (a/or [:acknowledged (a/or :accepted :rejected)] 
> [:rejected])])
>
>
>
> Tim Washington 
> Interruptsoftware.com  
>
>  

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] Automat 0.1.2

[Zach Tellman]

https://github.com/ztellman/automat

The initial release of this was just a little ways back, where I alluded to 
using the library to react to browsing behavior.  A few people asked how to 
actually accomplish that, and I realized that this sort of use, while 
possible, was harder than it needed to be.  So I added a function designed 
for this use case, and added some more detailed documentation on how the 
library can be used effectively.

As always, questions and feedback are welcome.

Zach

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Clojure Office Hours - Experience Report and Future Plans

[Zach Tellman]

For the in-person variety, I've written up some thoughts on why office 
hours are a good format for meetups, and ideas on the underlying 
processes: http://blog.factual.com/clojure-office-hours.

On Thursday, April 24, 2014 8:44:08 PM UTC-7, Leif wrote:
>
> This message is aimed at people that want to *hold* office hours 
> primarily, but of course others can chime in with
> opinions, suggestions, cheerleading, etc.
>
> I recently held "office hours" where I chatted / pair programmed with 
> "less experienced" clojure programmers (some
> were in fact more experienced).
>
> Lessons learned:
>
> 1. It's fun!  Do it!  Online like me, or convince your local clojure user 
> group to do it.
> 2. As I expected, I was more help to less experienced people, but learned 
> a lot *from* the others, and hopefully
>I was at least useful as a sounding board.
> 3. An hour is less time than it sounds.
> 4. If possible, test your pair programming setup beforehand (see point 3 
> above)
>a) corollary: if someone is asking about a library that takes some 
> setup, it's probably best if *they* do the
>   setup and host the pairing session.
> 5. Any remote sharing software (tmux, teamviewer, etc) will mangle *some* 
> input.  Be prepared to work around that.
> 6. Educate people how to cancel, and to cancel ASAP, since some will 
> inevitably need to.
> 7. For beginners (at clojure, but not programming), pick a specific 
> problem and work through it, or have a
>solution and explain it step-by-step; that seemed to work best.  Code 
> review of some OSS project they are
>interested in might also work, I didn't try it (but again, see point 3)
> 8. Unfortunately, no one completely new to programming booked with me, so 
> others will have to give advice here.
> 9. Many people outside of the western hemisphere were interested, so it 
> would be nice to have coverage across the
>globe.
>
> Future plans:
>
> Small plug: I used youcanbook.me to manage the office hours, with no 
> problems.  I encourage you to use their
> service, say nice things about them, and possibly give them money, 
> *because*:
>
> These fine folks allow non-profits to use their advanced features for 
> free, or at a reduced price.  So, I requested
> that the Clojure community's office hours get this status.  They said yes, 
> so my account (for now, for testing, we
> can move it later) can have unlimited "team members" and "services".  So, 
> I'd like to ask if there is interest in
> setting up a community clearinghouse for giving/receiving more office 
> hours, possibly of more types.  Some ideas
> (chime in with your own):
>
> 1. General Office Hours
>Basically what I did, except with more people offering office hours, so 
> that:
>a. Any one person will only have to offer a small number of hours a 
> week (1, even).
>b. Hopefully more coverage across time zones.
>c. People can tag what kinds of programming / projects they have 
> expertise in, so that "beginners" picking up
>  clojure for a specific reason or library can have a more productive 
> session.  E.g. some descriptions could read:
>
>Leif Poorman
>Location: Eastern USA
>Languages: en
>Tags: beginners, absolute beginners, web, data analysis, machine 
> learning
>
>Rich Hickey (obviously this is just an example)
>Location: USA
>Languages: en, Bynar
>Tags: distributed systems, functional databases, Datomic, concurrency, 
> alien technology, everything else
>
> 2. Office Hours for Beginners
>Specifically geared toward beginners in FP, absolute beginners in 
> programming, etc.  This could be covered by
>the description tags as above.  Or this could be more of a hangout, 
> where a set number of beginners get led
>through the ClojureBridge curriculum, or similar.
> 3. Project Specific Hours
>a) Someone with knowledge of an open source project gives a demo of its 
> capabilities/weaknesses to prospective
>   users (kind of a technical sales pitch, but for OSS)
>b) The maintainer of a fairly complex open source project walks some 
> people that want to contribute through the
>   codebase, to kickstart their contributions (I've seen this 
> done/proposed for Midje and Cascalog, at least).
>
> Alternatively, we could just start with 1-on-1, or 1-on-1 and small group, 
> and see where it goes from there.
>
> Comments?  Questions?  Suggestions?
>
> Cheers,
> Leif
>
> P.S. If you are interested in holding a few office hours, email me, and we 
> can start testing out the more advanced youcanbook.me features.
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojur

[ANN] Automat: better FSMs through combinators

[Zach Tellman]

https://github.com/ztellman/automat

This has been languishing in my Github for a while, for lack of a few 
finishing touches on the code and documentation.  I think this is one of 
cooler libraries I've written, and beyond the obvious use for parsers, the 
set theoretic operators could be a powerful way to specify actions in 
response to complex browsing behaviors on a site, or any other number of 
things.  I'm excited to see how it's used.

I'm happy to answer any questions.  Suggestions on how to improve the 
documentation are encouraged.

Zach

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] immutable-int-map

[Zach Tellman]

Correct. I'll clarify that I mean 64 bit integers in the readme.
On Apr 21, 2014 5:26 AM, "Alex Miller"  wrote:

> Never mind, look like you mean integer in the generic sense and you are
> using longs, right?
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/0EysWMml6Ks/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] immutable-int-map

[Zach Tellman]

I could represent the map {0 :foo, 100 :bar} as an array, but it would 
have to be a million element array with a lot of empty space. This would be 
(maybe) faster w.r.t. lookups, but would be vastly slower for enumerating 
entries, merging other maps, and adding keys that are larger than the 
boundaries of the underlying array.  This data structure, which is 
described by the paper linked in the readme,  I also mention the 
possibility of using this to represent a sparse vector (as opposed to the 
array, which is dense) in the readme.

Hope that answers your question,
Zach

On Sunday, April 20, 2014 9:44:21 PM UTC-7, Plinio Balduino wrote:
>
> "it's an immutable map that can only have positive integers as keys" -- Like 
> an array?
>
> (My question is child of my complete ignorance, and I'm not questioning 
> your knowledge or motivation)
>
> Plínio
>
>
> On Mon, Apr 21, 2014 at 1:40 AM, Zach Tellman 
> > wrote:
>
>> This one's pretty simple: it's an immutable map that can only have 
>> positive integers as keys.  It can be found at 
>> https://github.com/ztellman/immutable-int-map.
>>
>> The one interesting aspect of this is that it has an efficient merge 
>> mechanism, which means it plays better with Clojure's reducer framework 
>> than Clojure's own data structures.  This was a non-obvious consequence to 
>> me before I started working on it, so I figure others might also find it 
>> interesting.
>>  
>> -- 
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clo...@googlegroups.com
>> Note that posts from new members are moderated - please be patient with 
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+u...@googlegroups.com 
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to clojure+u...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] immutable-int-map

[Zach Tellman]

This one's pretty simple: it's an immutable map that can only have positive 
integers as keys.  It can be found at 
https://github.com/ztellman/immutable-int-map.

The one interesting aspect of this is that it has an efficient merge 
mechanism, which means it plays better with Clojure's reducer framework 
than Clojure's own data structures.  This was a non-obvious consequence to 
me before I started working on it, so I figure others might also find it 
interesting.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] durable-queue: an in-process disk-backed queue

[Zach Tellman]

Hey Leif,

When using :fsync-interval, the actual calls to .force() on the underlying 
ByteBuffers occur on another thread, making it effectively a background 
process.  This is contrasted with :fsync-threshold, which will 
synchronously call fsync when the write threshold is hit.  Note that if the 
fsync-interval is less than the time it takes to actually fsync, it will 
simply continuously fsync at whatever pace it can.

If you'd like to verify, I suggest using strace.

Zach

On Sunday, March 9, 2014 6:54:50 PM UTC-7, Leif wrote:
>
> Hi, Zach.
>
> I was trying to benchmark at different values of the :fysnc-* parameters, 
> and I noticed that it didn't matter what value of :fsync-interval I set, 
> the performance was constant, and about what it is with both :fsync-put? 
> and :fsync-take? disabled.
>
> Any suggestions on how to test if data is actually being synced to disk at 
> my specified interval?
>
> Please forgive my suspicious nature,
> Leif
>
> On Friday, March 7, 2014 4:21:44 PM UTC-5, Zach Tellman wrote:
>>
>> I added the above-described features a few weeks back, but only got 
>> around to marking 0.1.1 today.  Fsync batching is described at the end of 
>> the README, let me know if you have any questions.
>>
>> On Friday, February 7, 2014 11:52:11 AM UTC-8, Zach Tellman wrote:
>>>
>>> Hi Bob,
>>>
>>> Right now the API only allows for single puts, and fsyncing is 
>>> all-or-nothing.  However, this is just an artifact of my major use case for 
>>> the library, which relies on upstream batching of tasks.  I'm planning an 
>>> 0.1.1 release which has an explicit `sync` method, and support for 
>>> sync-intervals (i.e. sync twice a second) and sync-thresholds (i.e. sync 
>>> every ten puts or takes).  The use case you describe could be achieved by 
>>> disabling automatic syncing, and doing a series of puts and takes followed 
>>> by a call to `sync`.
>>>
>>> If you have thoughts or suggestions on how this can be more useful for 
>>> you, please let me know.
>>>
>>> Zach
>>>
>>>
>>> On Fri, Feb 7, 2014 at 5:26 AM, Bob Hutchison wrote:
>>>
>>>>
>>>> On Feb 6, 2014, at 6:45 PM, Zach Tellman  wrote:
>>>>
>>>> At Factual we get a lot of data thrown at us, and often don't have 
>>>> control over the rate at which it comes in.  As such, it's preferable that 
>>>> our buffer isn't bounded by the process' memory, since a temporary blip in 
>>>> throughput may cause GC pauses, OOM exceptions, and other things that will 
>>>> only exacerbate the problem.  It's also preferable that if the process 
>>>> dies, we won't lose any data which hasn't yet escaped the process.  A 
>>>> disk-backed queue satisfies both of these requirements.
>>>>
>>>> As such, I'm happy to announce that we're open sourcing 
>>>> 'durable-queue': https://github.com/Factual/durable-queue.  It's a 
>>>> small, fast, pure-Clojure implementation that in our production systems is 
>>>> responsible for processing billions of entries daily.  We believe it has 
>>>> broad applications, and are excited to see how others will use it.
>>>>
>>>>
>>>> What excellent timing! I’ve been looking at ZeroMQ, RabbitMQ, and Kafka 
>>>> for the last week or so. ZMQ is awfully attractive for what I’m trying to 
>>>> do, but there are a few things it doesn’t do that I need done. I had begun 
>>>> thinking of building something similar on top of Redis.
>>>>
>>>> You mention the idea of batching to reduce the impact of fsync. Is 
>>>> there an API for batching puts? Is there a way to batch a complete! and 
>>>> put! new tasks to the queue?
>>>>
>>>> One pattern that keeps coming up is:
>>>>- take a single task from the queue
>>>>- execute the task, which might generate a set of new tasks to be 
>>>> queued on the same queue (and likely on other queues too)
>>>>- signal completion, and put the new tasks
>>>>
>>>> Cheers,
>>>> Bob
>>>>
>>>>
>>>> Zach
>>>>
>>>> P.S. If this sort of work is interesting to you, Factual is hiring: 
>>>> https://groups.google.com/forum/#!searchin/clojure/factual/clojure/8bPIEnNpfyQ/lvv-9gkVozAJ
>>>>
>>>> -- 
>>>> You received this message because you are subscribed to 

Re: [ANN] durable-queue: an in-process disk-backed queue

[Zach Tellman]

I added the above-described features a few weeks back, but only got around 
to marking 0.1.1 today.  Fsync batching is described at the end of the 
README, let me know if you have any questions.

On Friday, February 7, 2014 11:52:11 AM UTC-8, Zach Tellman wrote:
>
> Hi Bob,
>
> Right now the API only allows for single puts, and fsyncing is 
> all-or-nothing.  However, this is just an artifact of my major use case for 
> the library, which relies on upstream batching of tasks.  I'm planning an 
> 0.1.1 release which has an explicit `sync` method, and support for 
> sync-intervals (i.e. sync twice a second) and sync-thresholds (i.e. sync 
> every ten puts or takes).  The use case you describe could be achieved by 
> disabling automatic syncing, and doing a series of puts and takes followed 
> by a call to `sync`.
>
> If you have thoughts or suggestions on how this can be more useful for 
> you, please let me know.
>
> Zach
>
>
> On Fri, Feb 7, 2014 at 5:26 AM, Bob Hutchison 
> 
> > wrote:
>
>>
>> On Feb 6, 2014, at 6:45 PM, Zach Tellman > 
>> wrote:
>>
>> At Factual we get a lot of data thrown at us, and often don't have 
>> control over the rate at which it comes in.  As such, it's preferable that 
>> our buffer isn't bounded by the process' memory, since a temporary blip in 
>> throughput may cause GC pauses, OOM exceptions, and other things that will 
>> only exacerbate the problem.  It's also preferable that if the process 
>> dies, we won't lose any data which hasn't yet escaped the process.  A 
>> disk-backed queue satisfies both of these requirements.
>>
>> As such, I'm happy to announce that we're open sourcing 'durable-queue': 
>> https://github.com/Factual/durable-queue.  It's a small, fast, 
>> pure-Clojure implementation that in our production systems is responsible 
>> for processing billions of entries daily.  We believe it has broad 
>> applications, and are excited to see how others will use it.
>>
>>
>> What excellent timing! I’ve been looking at ZeroMQ, RabbitMQ, and Kafka 
>> for the last week or so. ZMQ is awfully attractive for what I’m trying to 
>> do, but there are a few things it doesn’t do that I need done. I had begun 
>> thinking of building something similar on top of Redis.
>>
>> You mention the idea of batching to reduce the impact of fsync. Is there 
>> an API for batching puts? Is there a way to batch a complete! and put! new 
>> tasks to the queue?
>>
>> One pattern that keeps coming up is:
>>- take a single task from the queue
>>- execute the task, which might generate a set of new tasks to be 
>> queued on the same queue (and likely on other queues too)
>>- signal completion, and put the new tasks
>>
>> Cheers,
>> Bob
>>
>>
>> Zach
>>
>> P.S. If this sort of work is interesting to you, Factual is hiring: 
>> https://groups.google.com/forum/#!searchin/clojure/factual/clojure/8bPIEnNpfyQ/lvv-9gkVozAJ
>>
>> -- 
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clo...@googlegroups.com
>> Note that posts from new members are moderated - please be patient with 
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+u...@googlegroups.com 
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to clojure+u...@googlegroups.com .
>>
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>>  -- 
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clo...@googlegroups.com
>> Note that posts from new members are moderated - please be patient with 
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+u...@googlegroups.com 
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> --- 
>> You received this message because you are subscribed to a topic in the 
>> Google Groups "Clojure" group.
>> To unsubscribe from this topic, visit 
>> https://groups.google.com/d/topic/clojure/4tZFWdMKvjw/unsubscribe.
>> To unsubscribe from this group and al

Re: [ANN] durable-queue: an in-process disk-backed queue

[Zach Tellman]

Hi Bob,

Right now the API only allows for single puts, and fsyncing is
all-or-nothing.  However, this is just an artifact of my major use case for
the library, which relies on upstream batching of tasks.  I'm planning an
0.1.1 release which has an explicit `sync` method, and support for
sync-intervals (i.e. sync twice a second) and sync-thresholds (i.e. sync
every ten puts or takes).  The use case you describe could be achieved by
disabling automatic syncing, and doing a series of puts and takes followed
by a call to `sync`.

If you have thoughts or suggestions on how this can be more useful for you,
please let me know.

Zach


On Fri, Feb 7, 2014 at 5:26 AM, Bob Hutchison wrote:

>
> On Feb 6, 2014, at 6:45 PM, Zach Tellman  wrote:
>
> At Factual we get a lot of data thrown at us, and often don't have control
> over the rate at which it comes in.  As such, it's preferable that our
> buffer isn't bounded by the process' memory, since a temporary blip in
> throughput may cause GC pauses, OOM exceptions, and other things that will
> only exacerbate the problem.  It's also preferable that if the process
> dies, we won't lose any data which hasn't yet escaped the process.  A
> disk-backed queue satisfies both of these requirements.
>
> As such, I'm happy to announce that we're open sourcing 'durable-queue':
> https://github.com/Factual/durable-queue.  It's a small, fast,
> pure-Clojure implementation that in our production systems is responsible
> for processing billions of entries daily.  We believe it has broad
> applications, and are excited to see how others will use it.
>
>
> What excellent timing! I've been looking at ZeroMQ, RabbitMQ, and Kafka
> for the last week or so. ZMQ is awfully attractive for what I'm trying to
> do, but there are a few things it doesn't do that I need done. I had begun
> thinking of building something similar on top of Redis.
>
> You mention the idea of batching to reduce the impact of fsync. Is there
> an API for batching puts? Is there a way to batch a complete! and put! new
> tasks to the queue?
>
> One pattern that keeps coming up is:
>- take a single task from the queue
>- execute the task, which might generate a set of new tasks to be
> queued on the same queue (and likely on other queues too)
>- signal completion, and put the new tasks
>
> Cheers,
> Bob
>
>
> Zach
>
> P.S. If this sort of work is interesting to you, Factual is hiring:
> https://groups.google.com/forum/#!searchin/clojure/factual/clojure/8bPIEnNpfyQ/lvv-9gkVozAJ
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
>
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>  --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/4tZFWdMKvjw/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[ANN] durable-queue: an in-process disk-backed queue

[Zach Tellman]

At Factual we get a lot of data thrown at us, and often don't have control 
over the rate at which it comes in.  As such, it's preferable that our 
buffer isn't bounded by the process' memory, since a temporary blip in 
throughput may cause GC pauses, OOM exceptions, and other things that will 
only exacerbate the problem.  It's also preferable that if the process 
dies, we won't lose any data which hasn't yet escaped the process.  A 
disk-backed queue satisfies both of these requirements.

As such, I'm happy to announce that we're open sourcing 'durable-queue': 
https://github.com/Factual/durable-queue.  It's a small, fast, pure-Clojure 
implementation that in our production systems is responsible for processing 
billions of entries daily.  We believe it has broad applications, and are 
excited to see how others will use it.

Zach

P.S. If this sort of work is interesting to you, Factual is 
hiring: 
https://groups.google.com/forum/#!searchin/clojure/factual/clojure/8bPIEnNpfyQ/lvv-9gkVozAJ

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Factual is hiring Clojure engineers

[Zach Tellman]

We will consider candidates from outside the US if they're willing to
relocate here.  I'm happy to talk in more detail if you like, just email me
at the above address.


On Sat, Jan 18, 2014 at 3:31 AM, Petr  wrote:

> Do you consider non US candidates?
> Wihout extensive data analysis experience?
>
>
> On Saturday, January 18, 2014 7:34:38 AM UTC+7, Zach Tellman wrote:
>>
>> It's been six months since I last announced this, so: Factual is
>> perpetually hiring smart, driven engineers to solve deep problems in data
>> analysis and systems engineering.  We have offices in both SF and LA, and
>> remote candidates with a strong background in analysis will be considered.
>>
>> Factual provides an index of things in the world (businesses, places, and
>> products).  You can either use this directly via our API [1] to ask
>> questions like "what are the three coffee shops nearest to me", or
>> indirectly by giving us your own real world data to enrich and refine [2]
>> [3].  Factual uses Clojure extensively [4], both for realtime systems and
>> batch data analysis.  Systems written purely in Clojure handle the
>> following:
>>
>> * realtime intake of 70k datapoints/sec from our partners
>> * batch processing of 7 billion new datapoints every day, comprising
>> terabytes of compressed textual data
>> * generating, storing, and serving 150 million records, refreshed daily
>>
>> Both of these numbers have at least doubled in the last six months, and
>> could very easily do so again.  There are also other projects that use
>> Clojure alongside Java to work at similar scales.
>>
>> Libraries developed in the course of working on these projects include:
>>
>> https://github.com/factual/skuld
>> https://github.com/factual/clj-leveldb
>> https://github.com/aphyr/merkle
>> https://github.com/ztellman/clj-tuple
>> https://github.com/ztellman/narrator
>>
>> Several other libraries are on the cusp of being open sourced, when we
>> get a chance.
>>
>> We need people who enjoy using technology as a lever to accomplish
>> amazing things (wasn't it Archimedes who said "give me a large enough data
>> center..."), who want to work alongside people they can continuously learn
>> from, and who want to be just a little uncomfortable with the scale and
>> scope of their responsibilities.
>>
>> If this sounds interesting, contact me at zach at factual.com.  I'm
>> heading up the small but growing SF office, and am happy to talk in person
>> with anyone in the Bay Area who'd like to know more.
>>
>> [1] http://developer.factual.com/working-with-factual-places/
>> [2] http://www.factual.com/products/geopulse-audience
>> [3] http://www.factual.com/products/resolve
>> [4] http://www.factual.com/jobs/clojure
>>
>  --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/8bPIEnNpfyQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Factual is hiring Clojure engineers

[Zach Tellman]

It's been six months since I last announced this, so: Factual is 
perpetually hiring smart, driven engineers to solve deep problems in data 
analysis and systems engineering.  We have offices in both SF and LA, and 
remote candidates with a strong background in analysis will be considered.

Factual provides an index of things in the world (businesses, places, and 
products).  You can either use this directly via our API [1] to ask 
questions like "what are the three coffee shops nearest to me", or 
indirectly by giving us your own real world data to enrich and refine [2] 
[3].  Factual uses Clojure extensively [4], both for realtime systems and 
batch data analysis.  Systems written purely in Clojure handle the 
following:

* realtime intake of 70k datapoints/sec from our partners
* batch processing of 7 billion new datapoints every day, comprising 
terabytes of compressed textual data
* generating, storing, and serving 150 million records, refreshed daily

Both of these numbers have at least doubled in the last six months, and 
could very easily do so again.  There are also other projects that use 
Clojure alongside Java to work at similar scales.

Libraries developed in the course of working on these projects include:

https://github.com/factual/skuld
https://github.com/factual/clj-leveldb
https://github.com/aphyr/merkle
https://github.com/ztellman/clj-tuple
https://github.com/ztellman/narrator

Several other libraries are on the cusp of being open sourced, when we get 
a chance.

We need people who enjoy using technology as a lever to accomplish amazing 
things (wasn't it Archimedes who said "give me a large enough data 
center..."), who want to work alongside people they can continuously learn 
from, and who want to be just a little uncomfortable with the scale and 
scope of their responsibilities.

If this sounds interesting, contact me at zach at factual.com.  I'm heading 
up the small but growing SF office, and am happy to talk in person with 
anyone in the Bay Area who'd like to know more.

[1] http://developer.factual.com/working-with-factual-places/
[2] http://www.factual.com/products/geopulse-audience
[3] http://www.factual.com/products/resolve
[4] http://www.factual.com/jobs/clojure

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] Narrator: expressive, composable stream analysis

[Zach Tellman]

Riemann is a service for receiving streams of events, and causing one or
more side-effects (sending email, routing to Graphite, etc).  It can do
arbitrary transformations on event streams (the effects from an input may
be arbitrarily time shifted), and assumes that the inputs are fixed
structure (numbers, shallow maps, etc).

Narrator is a library for analyzing streams of events, and returns either a
single value representing the stream, or values representing fixed, regular
intervals within the stream.  The analysis can only on that interval (or
multiples of that interval, using 'moving').  Using the 'recur' operator,
it can do analysis on arbitrarily nested structures.

At Factual, we use both of these in tandem.  Since the trace data for
function call trees is arbitrarily nested, we use Narrator, and then
separate the data into flattened sub-parts, and pass it onto Riemann.  We
typically have a fixed set of functions that we're tracing (entry points
for HTTP requests, primarily), and automatically send them along via UDP to
Omphalos [1].  Obviously the instrumented functions that are called in the
process of creating a response may change, this is within the control of
the authors of the libraries we use.

Does this answer all your questions?  I'm happy to elaborate on any of the
above.

Zach

[1] This is discussed in more detail here:
http://www.infoq.com/presentations/analyze-running-system


On Sun, Nov 10, 2013 at 3:03 AM, dm3  wrote:

> I've read about Lamina and Narrator, watched the linked videos and I think
> I understand how it all fits together:
>
> 1) Instrument the applications using Lamina's `instrument` or `trace`
> 2) Probe the instrumented code somehow by channeling the traces to some
> endpoint (how do you do this? do you automatically probe everything and
> channel to some remote service? Do you have some method of dynamically
> enabling/disabling probes on your services (e.g. embedded repl or some
> management endpoint)?
> 3) Analyze the traces remotely using Narrator + networking code + a UI
> (that's the gist of the Omphalos as I've understood). If so, what would you
> say is the largest difference between Riemann's[1] stream analysis
> functionality and what is provided by Narrator? Would you say that Omphalos
> and Riemann serve completely different puproses?
>
> Am I on the right track?
>
> [1] http://riemann.io/
>
>
> On Sunday, 3 November 2013 00:28:27 UTC+2, Zach Tellman wrote:
>>
>> https://github.com/ztellman/narrator
>>
>> This is a reimplementation of an approach I've discussed in several talks
>> [1] [2], with an eye towards performance, memory efficiency, and
>> flexibility w.r.t. how the event stream is represented.  The readme does a
>> good job of explaining how it works, but there have been a number of new
>> event processing libraries recently (core.async, EEP, etc.), so I'll spend
>> some time here describing how this differs.
>>
>> First, this library is focused on aggregations over event streams, not
>> arbitrary transformations.  It is designed such that these aggregations can
>> be automatically parallelized, and use non-thread-safe data structures
>> (such as those in the excellent stream-lib [3]) without having to worry
>> about coordination.  As such, within this narrower application it has a
>> richer set of operators, and should be a fair bit faster (millions of
>> messages/sec/core).
>>
>> Second, this has support for time-series analysis of ordered streams,
>> either historical or in real time.  The input for either type of analysis
>> can be normal sequences, core.async channels, or Lamina channels. At
>> Factual we use this for aggregations across many of our real-time systems,
>> and I also use it for both ad hoc queries and daily rollups of logs and
>> other historical data.
>>
>> On a personal note, I think this is one of the most interesting and
>> useful libraries I've written.  I'm really looking forward to seeing how
>> people use it, and encourage feedback on how to make it better.
>>
>> Zach
>>
>> [1] http://www.infoq.com/presentations/analyze-running-system
>> [2] http://vimeo.com/45132054#!
>> [3] https://github.com/addthis/stream-lib
>>
>  --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.go

Re: [ANN] Narrator: expressive, composable stream analysis

[Zach Tellman]

I was aware of Babbage, but haven't used it.  There is a certain similarity
to the syntax, but I think most (if not all) of the things I listed
differentiate Narrator from Babbage, as well.  Please correct me if I'm
wrong.


On Sat, Nov 2, 2013 at 3:36 PM, Ben Wolfson  wrote:

> seems kind of similar to babbage:
> https://github.com/ReadyForZero/babbage/tree/1.1
>
>
> On Sat, Nov 2, 2013 at 3:28 PM, Zach Tellman  wrote:
>
>> https://github.com/ztellman/narrator
>>
>> This is a reimplementation of an approach I've discussed in several talks
>> [1] [2], with an eye towards performance, memory efficiency, and
>> flexibility w.r.t. how the event stream is represented.  The readme does a
>> good job of explaining how it works, but there have been a number of new
>> event processing libraries recently (core.async, EEP, etc.), so I'll spend
>> some time here describing how this differs.
>>
>> First, this library is focused on aggregations over event streams, not
>> arbitrary transformations.  It is designed such that these aggregations can
>> be automatically parallelized, and use non-thread-safe data structures
>> (such as those in the excellent stream-lib [3]) without having to worry
>> about coordination.  As such, within this narrower application it has a
>> richer set of operators, and should be a fair bit faster (millions of
>> messages/sec/core).
>>
>> Second, this has support for time-series analysis of ordered streams,
>> either historical or in real time.  The input for either type of analysis
>> can be normal sequences, core.async channels, or Lamina channels. At
>> Factual we use this for aggregations across many of our real-time systems,
>> and I also use it for both ad hoc queries and daily rollups of logs and
>> other historical data.
>>
>> On a personal note, I think this is one of the most interesting and
>> useful libraries I've written.  I'm really looking forward to seeing how
>> people use it, and encourage feedback on how to make it better.
>>
>> Zach
>>
>> [1] http://www.infoq.com/presentations/analyze-running-system
>> [2] http://vimeo.com/45132054#!
>> [3] https://github.com/addthis/stream-lib
>>
>> --
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clojure@googlegroups.com
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to clojure+unsubscr...@googlegroups.com.
>>
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>
>
>
> --
> Ben Wolfson
> "Human kind has used its intelligence to vary the flavour of drinks, which
> may be sweet, aromatic, fermented or spirit-based. ... Family and social
> life also offer numerous other occasions to consume drinks for pleasure."
> [Larousse, "Drink" entry]
>
>  --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/gXGVXgqd9Xs/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[ANN] Narrator: expressive, composable stream analysis

[Zach Tellman]

https://github.com/ztellman/narrator

This is a reimplementation of an approach I've discussed in several talks 
[1] [2], with an eye towards performance, memory efficiency, and 
flexibility w.r.t. how the event stream is represented.  The readme does a 
good job of explaining how it works, but there have been a number of new 
event processing libraries recently (core.async, EEP, etc.), so I'll spend 
some time here describing how this differs.

First, this library is focused on aggregations over event streams, not 
arbitrary transformations.  It is designed such that these aggregations can 
be automatically parallelized, and use non-thread-safe data structures 
(such as those in the excellent stream-lib [3]) without having to worry 
about coordination.  As such, within this narrower application it has a 
richer set of operators, and should be a fair bit faster (millions of 
messages/sec/core).

Second, this has support for time-series analysis of ordered streams, 
either historical or in real time.  The input for either type of analysis 
can be normal sequences, core.async channels, or Lamina channels. At 
Factual we use this for aggregations across many of our real-time systems, 
and I also use it for both ad hoc queries and daily rollups of logs and 
other historical data.

On a personal note, I think this is one of the most interesting and useful 
libraries I've written.  I'm really looking forward to seeing how people 
use it, and encourage feedback on how to make it better.

Zach

[1] http://www.infoq.com/presentations/analyze-running-system
[2] http://vimeo.com/45132054#!
[3] https://github.com/addthis/stream-lib

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] collection-check: validation for data structure variants

[Zach Tellman]

Yes, this is some very simple sugar atop simple-check, Reid is owed most of
the credit here.
On Oct 30, 2013 10:44 PM, "Alex Baranosky" 
wrote:

> Thanks Zach for this cool testing library, and thanks Reid for
> simple-check!
>
> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/W2oJzZnJ-9c/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[ANN] collection-check: validation for data structure variants

[Zach Tellman]

It's relatively rare that people write variants of Clojure data structures. 
 Partially, this is because the existing data structures are quite good, 
but it's also because it's surprisingly difficult.  Clojure's vectors, 
sets, and maps each implement about a dozen interfaces with overlapping 
functionality, making it easy to think you've written something correct 
until someone accesses it in a way you didn't think to test.  I have 
written a number of alternate data structures [1] [2] [3], and each time 
there's been some lacunae of the standard API that I overlooked.

To make this easier for myself and others, I've written a library for the 
express purpose of validating data structures that are extensions of the 
standard three data 
structures: https://github.com/ztellman/collection-check.  It's already 
uncovered some minor bugs in my own libraries, as well as an issue with 
Clojure's own hash-sets and hash-maps [4].  I strongly encourage anyone 
who's playing around in this space to use it, and to let me know if there 
are invariants I've overlooked.

Enjoy,
Zach

[1] https://github.com/ztellman/clj-tuple
[2] https://github.com/ztellman/immutable-bitset
[3] https://github.com/ztellman/potemkin
[4] https://groups.google.com/forum/#!topic/clojure-dev/HvppNjEH5Qc

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: streams, reading bytes

[Zach Tellman]

The byte-transforms library will also take care of the gzip stuff for 
you: https://github.com/ztellman/byte-transforms

On Thursday, October 10, 2013 10:26:35 PM UTC-4, Brian Craft wrote:
>
> I like the blurb:
>
> "This library is a Rosetta stone for all the byte representations Java has 
> to offer, and gives you the freedom to forget all the APIs you never wanted 
> to know in the first place."
>
> My feelings, exactly. ;)
>
> Thanks.
>
> On Thursday, October 10, 2013 6:44:32 PM UTC-7, Ben Mabey wrote:
>>
>> On 10/10/13 7:21 PM, Brian Craft wrote: 
>> > I'm struggling with how to gzip/gunzip byte arrays from clojure. I 
>> > don't really get how clojure.java.io streams supposed to be used. 
>> > 
>> > This appears to gzip a string: 
>> > 
>> > (let [baos (java.io.ByteArrayOutputStream.) 
>> >   gzos (java.util.zip.GZIPOutputStream. baos)] 
>> > (.write gzos (.getBytes "foo")) 
>> > (.finish gzos) 
>> > (.toByteArray baos)) 
>> > 
>> > 
>> > and I can expand on this to gunzip the string afterwards: 
>> > 
>> > (let [baos (java.io.ByteArrayOutputStream.) 
>> >   gzos (java.util.zip.GZIPOutputStream. baos)] 
>> >   (.write gzos (.getBytes "foo")) 
>> >   (.finish gzos) 
>> >   (let [gzbytes (.toByteArray baos) 
>> > bais (java.io.ByteArrayInputStream. gzbytes) 
>> > gzis (java.util.zip.GZIPInputStream. bais)] 
>> > (slurp gzis))) 
>> > 
>> > But slurp builds strings. I need to do this with byte arrays.  Do I 
>> > have to rewrite slurp to build a byte array? Is there any simpler way 
>> > to do this? 
>>
>> Zach Tellman's byte-streams library is very handy for these kinds of 
>> conversions: 
>>
>> https://github.com/ztellman/byte-streams 
>>
>> -Ben 
>>
>>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[ANN] Sleight: whole program transformations

[Zach Tellman]

This is a lein plugin which hijacks the Clojure reader and allows all forms 
to be transformed before they're passed to the compiler.  Usage and 
potential applications are described in the readme: 
https://github.com/ztellman/sleight

I wrote the original version of this a year ago, but due to the lack of 
good code walking facilities it was more a sketch than anything else. 
 Luckily, with the recent release of Riddley [1], this is newly useful. 
 I'm looking forward to seeing what people use it for.

Zach

[1] https://github.com/ztellman/riddley

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: New Functional Programming Job Opportunities

[Zach Tellman]

If anyone has questions about working at Factual, I'm happy to answer them, 
and can be reached at zach at factual.com.  The short version, though, is 
that it's great and you should come work with us.

For some examples of Clojure we're writing and using at Factual, check out:

https://github.com/factual/skuld
https://github.com/factual/clj-leveldb
https://github.com/aphyr/merkle
https://github.com/ztellman/clj-tuple
https://github.com/ztellman/narrator

among many others.

Anyone at Strange Loop who's curious should feel free to track me down. 
 Kyle Kingsbury (better known to some as "aphyr") will be giving a talk, 
and will also happily answer any questions about Factual.

Zach

On Sunday, September 15, 2013 11:00:02 PM UTC-7, Sean Murphy wrote:
>
> Here are some functional programming job opportunities that were posted 
> recently: 
>
> Clojure Engineers Needed at Factual 
> http://functionaljobs.com/jobs/8646-clojure-engineers-needed-at-factual 
>
> Cheers, 
> Sean Murphy 
> FunctionalJobs.com 
>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Clojure, floats, ints and OpenGL

[Zach Tellman]

If you're worried about NIO buffer marshalling, Vertigo [1] lets you treat 
them as native Clojure data structures.  This doesn't save you from the 
float<->double coercion for arithmetic operations, but as James said 
earlier in the thread, this tends to be insignificant.  However, if you're 
absolutely convinced this is a problem, it's pretty straightforward to 
extend the approach used in primitive-math [2] to work around this as well. 
 The current implementation hews to the standard Clojure approach of 
coercing everything to longs and doubles, but that was mostly because it 
saved me from having to do too much copy/pasting.  

I would point out, though that this:

"Where there were 500 fps, sometimes happen to be 480 fps and sometimes 
0.01 fps"

sounds an awful lot like garbage collection to me.  Type coercion has a 
fixed amount of overhead, and should have a fixed effect on the performance 
of your code. 

Zach

[1] https://github.com/ztellman/vertigo
[2] 
https://github.com/ztellman/primitive-math/blob/master/src/primitive_math/Primitives.java


On Monday, September 16, 2013 2:40:24 PM UTC-7, Alex Fowler wrote:
>
> Timothy, thank you very much for such a good explanation! I fully agree 
> with you. Actually, except for a few details, that is really how I thought 
> the things are. It is good that you describe the technical base for 
> problems with the issue. As well as clearified the human factor. What 
> you're saying is very much like what I would say if I were in your shoes. 
> Currently, what you're saying, is the present. In the present we make our 
> choices for currents. And I also agree with Mikera, that an attempt should 
> be taken. I would participate in it myself if I was into JVM and Clojure 
> internals. I plan to learn it, but it will take time.
>
> Mikera, thanks for the reference for the library, I will have a look. Also 
> I strongly encourage you in the attempt and I will be happy if I can be of 
> any help, although, you seem to know much more than me. I agree with Tim 
> that some more decisive steps have to be taken into this direction.
>
> James, ah.. yes, we did benchmarks. We use highly optimized mathematical 
> calculations, derived from numerical analysis. Every single instruction 
> matters there. Due to the nature of the calculations, the amout of casts, 
> however, varies from 3 to N in one computation. There can be 1 to M such 
> computation for a data unit. And thre can be 1 to P data units. At the end, 
> what worked on plain Java or Java-stylized Scala, worked that times faster, 
> NxMxP more instructions to be computed. Where there were 500 fps, sometimes 
> happen to be 480 fps and sometimes 0.01 fps. And I am not even talking 
> about operations on vast arrays or NIO buffers. But imagine for example, 
> you have a 1.5GB float NIO-buffer, that came from an OpenGL data-type and 
> will go back to it. Operation on each float takes, say 5 casts 
> double<->float in Clojure.
>
> Also, as Mikera points out, bus speeds come into play too.
>
> Of course, as Timothy said, why don't we go back to C for that kind of 
> stuff? This is a popular question in such cases. I am not going to go into 
> details on that, except for saying that we have mostly migrated from C to 
> JVM and it satisfies our requirements, while giving huge benifits. However, 
> we still do some things in C. To prevent arguments, here is an 
> article 
> that 
> somewhat describes what one can lose in computation, but does not describe 
> what one gains in other aspects. The loss is what can be tolerated. But 
> that loss for JVM is understandable. The one more loss for Clojure is 
> understandable too, but it can be changed. I think that the discussion 
> branch of Java vs C may be laid to rest.
>
>
> On Mon, Sep 16, 2013 at 9:03 PM, James Reeves 
> 
> > wrote:
>
>>
>>
>>
>> On 16 September 2013 09:03, Mikera > >wrote:
>>
>>>
>>> Obviously this is just a microbenchmark, but it fits my general 
>>> experience that floats are a reasonable bit faster than doubles, typically 
>>> 20-100% (doubles are closer when it is pure number crunching since 64-bit 
>>> CPUs are actually pretty good at doubles, floats have a bigger advantage 
>>> when you are manipulating a lot of data points and hence memory bandwidth 
>>> matters more)
>>>
>>> Code here for those interested:
>>> src/test/java/mikera/vectorz/performance/FloatVsDoubleBenchmark.java
>>>
>>
>> That's a pretty interesting result. I ran some tests of my own, based on 
>> your code, as I wondered whether or not the time to instantiate the array 
>> of doubles was biasing the test. My goal was to see whether or not I'd get 
>> a similar result running an array of floats through a method that processed 
>> doubles. (See: https://gist.github.com/weavejester/6583367)
>>
>> It turns out that I get a similar result. Passing floats to a method that 
>> takes doubles slows things down by a simila

Re: Handling name collisions with clojure.core

[Zach Tellman]

Not doing actual replacement via code-walking will prevent any functions
with inline definitions actually being able to benefit from this.  I'm not
sure if those are used in core.matrix, though.


On Thu, Sep 5, 2013 at 4:54 AM, Colin Fleming
wrote:

> This is actually probably not a bad solution. You wouldn't even need to
> rewrite, couldn't you just expand to a let?
>
> (let [* clojure.core.matrix.*
>   + clojure.core.matrix.+]
>   (+ ... (* ...)))
>
> Although thinking about it, you'd have to let-bind all possible operators
> every time, and the compiler doesn't do any dead-code elimination (although
> Hotspot might).
>
>
> On 5 September 2013 17:35, Dave Ray  wrote:
>
>> Maybe this is a dumb idea, but could you have a macro that rewrites code
>> to use your ops?
>>
>>   (require '[clojure.core.matrix :as m])
>>   (m/with-ops (+ ... (* ...) ...))
>>
>> and then all the "special" symbols get rewritten/qualified with
>> clojure.core.matrix?
>>
>> Dave
>>
>>
>>
>> On Wed, Sep 4, 2013 at 10:26 PM, Sean Corfield wrote:
>>
>>> You only get the warning if you 'use' the namespace or 'refer all'
>>> tho', correct?
>>>
>>> And we've recently seen a lot of discussion that basically says "don't
>>> do that" so it seems that either users of core.matric are going to
>>> have two "approved" choices:
>>> * require core.matrix with an alias, or choose to rename colliding
>>> names however you want
>>> * exclude the colliding symbols via refer-clojure and require them
>>> from core.matrix as referred symbols
>>>
>>> That's seems right to me: it is explicit and provides no surprises; it
>>> gives the user control over how to manage things.
>>>
>>> Sean
>>>
>>> On Wed, Sep 4, 2013 at 6:22 PM, Mikera 
>>> wrote:
>>> > Hi all,
>>> >
>>> > While building the API for core.matrix, I've fun into a few cases
>>> where the
>>> > "best" name is a direct clash with clojure.core.
>>> >
>>> > Examples are "+", "zero?", "vector?", "=="
>>> >
>>> > In many of these cases, the core.matrix behaviour is a natural
>>> extension of
>>> > the clojure.core function (i.e. it extends the same functionality to
>>> > arbitrary N-dimensional arrays).
>>> >
>>> > I'm not very happy with any of the options I can see for handling this:
>>> >
>>> > A) Use the good names in the "clojure.core.matrix" namespace. Problem:
>>> that
>>> > gives you a ton of nasty warnings of the type "WARNING: + already
>>> refers to:
>>> > #'clojure.core/+ in namespace: test.blank, being replaced by:
>>> > #'clojure.core.matrix/+". Significant boilerplate must be maintained
>>> by the
>>> > user in their ns declaration to prevent these warnings. I don't like
>>> forcing
>>> > users to maintain boilerplate, and I think that normal idiomatic usage
>>> > should be warning-free.
>>> >
>>> > B) Separate the name-clashing functions into separate namespaces - e.g.
>>> > "clojure.core.matrix.operators". Problem: that's something of an
>>> artificial
>>> > division, and again it forces users to do extra ns-management work to
>>> access
>>> > the functions they want.
>>> >
>>> > C) Use different names. Problem: names would be worse, and this would
>>> be
>>> > inconsistent and confusing, especially for functions that do
>>> effectively the
>>> > same thing.
>>> >
>>> > D) Encourage users to use aliases. Problem: that's horrendously ugly
>>> and
>>> > inconvenient for numerical code. Users with any sense of elegance in
>>> their
>>> > coding style would quite rightly throw their hands up in disgust.
>>> >
>>> > Currently we're doing B), I'd prefer to do A) but can't figure out a
>>> way to
>>> > automatically suppress the warnings.
>>> >
>>> > Any better ideas?
>>> >
>>> >
>>> >
>>> > --
>>> > --
>>> > You received this message because you are subscribed to the Google
>>> > Groups "Clojure" group.
>>> > To post to this group, send email to clojure@googlegroups.com
>>> > Note that posts from new members are moderated - please be patient
>>> with your
>>> > first post.
>>> > To unsubscribe from this group, send email to
>>> > clojure+unsubscr...@googlegroups.com
>>> > For more options, visit this group at
>>> > http://groups.google.com/group/clojure?hl=en
>>> > ---
>>> > You received this message because you are subscribed to the Google
>>> Groups
>>> > "Clojure" group.
>>> > To unsubscribe from this group and stop receiving emails from it, send
>>> an
>>> > email to clojure+unsubscr...@googlegroups.com.
>>> > For more options, visit https://groups.google.com/groups/opt_out.
>>>
>>>
>>>
>>> --
>>> Sean A Corfield -- (904) 302-SEAN
>>> An Architect's View -- http://corfield.org/
>>> World Singles, LLC. -- http://worldsingles.com/
>>>
>>> "Perfection is the enemy of the good."
>>> -- Gustave Flaubert, French realist novelist (1821-1880)
>>>
>>> --
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To post to this group, send email to clojure@googlegroups.com
>>> Note that posts from new members are moderated - please be patien

Re: [ANN] riddley: code-walking without caveats

[Zach Tellman]

Hi Nils,

Thanks for the link, I hadn't seen that before.  Happily, what you describe
is pretty much exactly what Riddley does, in this case by accessing the
compiler internals.  An example of how macrolet or symbol-macrolet could be
implemented using this is linked above in my response to Konrad.  This
means that macros which rely on the clojure.lang.Compiler$LocalBinding
(which includes [1] and [2]) will still work.  As far as I can tell, this
is not the case in either tools.macro or core.async.

If you know of any ways to make the code-walking more complete, please let
me know.

Zach

[1] https://github.com/ztellman/proteus/blob/master/src/proteus.clj#L60
[2]
https://github.com/flatland/useful/blob/develop/src/flatland/useful/datatypes.clj#L62


On Thu, Sep 5, 2013 at 3:31 AM, bertschi
wrote:

> Hi Zach,
>
> you might want to look at this paper explaining how to write a correct
> macroexpand-all (which requires a code walker) in Common Lisp:
> http://www.merl.com/publications/TR1993-017/
>
> The compiler certainly has to do something like that, but might not do all
> of the macroexpansion before starting any compilation as Konrad explained.
> What the compiler needs to do is track the lexical environment while
> walking down the source forms. When a code walker wants to introduce
> additional bindings, such as macrolet (for local macros) or symbol-macrolet
> (for new symbols) it needs to be able to extend the environment
> accordingly. So, you either have to access the compiler internals,
> especially its environment handling, or track the environment yourself (as
> Konrad suggested).
> As an aside: The problem in Common Lisp is mainly that the environment
> handling is not exposed in the standard, thus you cannot write a portable
> code walker without doing some environment handling yourself.
>
> You might also want to look at core.async, which uses a code walker to
> transform go blocks into state machines. I have not (yet) checked its
> restrictions (someone told me, that it cannot even look into anonymous fn
> forms within its body!), but it is generally very hard to write a code
> walker that can handle all special forms (in Common Lisp I don't know any).
>
> +10 for having a library that supports writing correct and (almost)
> complete code walkers
>
> Best,
>
>Nils
>
> On Thursday, September 5, 2013 12:09:28 PM UTC+2, Konrad Hinsen wrote:
>>
>> Zach Tellman writes:
>>
>>  > I guess I'm confused, then.  You contrast "complete recursive
>>  > expansion" with what the compiler does, and then say it's recursive
>>  > prewalk expansion, which is exactly what the compiler does.  Can
>>  > you clarify the difference between what you're doing and what the
>>  > compiler does?
>>
>> Here's an example:
>>
>>(defmacro foo [x]
>>  `(list ~x ~x))
>>
>>(defmacro bar [x]
>>  `[~x ~x])
>>
>> Now let's work on the form
>>
>>(foo (bar 'baz))
>>
>> Plain macroexpand returns
>>
>>(list (bar 'baz) (bar 'baz))
>>
>> whereas tools.macro/mexpand-all gives
>>
>>(list ['baz 'baz] ['baz 'baz])
>>
>> It does this by first calling macroexpand, so foo gets called exactly
>> as during Clojure compilation and returns
>>
>>(list (bar 'baz) (bar 'baz))
>>
>> mexpand-all then goes through that form and expands the two subforms
>> (bar 'baz).
>>
>> So mexpand-all does exactly what the compiler does, in particular it
>> calls the macros with exactly the same arguments. But the compiler
>> interleaves macro expansion with compilation, so it never gives you
>> access to the fully expanded but uncompiled form which is
>>
>>(list ['baz 'baz] ['baz 'baz])
>>
>> Konrad
>>
>  --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/a68aThpvP4o/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
>

Re: [ANN] riddley: code-walking without caveats

[Zach Tellman]

Sorry, that documentation reflected 0.1.1-SNAPSHOT, which I've just
released as 0.1.1.  Let me know if you have any other issues.

Zach


On Thu, Sep 5, 2013 at 6:26 AM, Stathis Sideris  wrote:

> Thanks for this library Zach,
>
> It seems that the released version is a bit behind in comparison to the
> generated documentation [1]. For example, walk-exprs is advertised as being
> able to accept a special-forms parameter, but that's not the case in the
> jar that leiningen retrieved when I used [riddley "0.1.0"] in my
> project.clj.
>
> Thanks,
>
> Stathis
>
> [1] http://ideolalia.com/riddley/
>
>
>
> On Monday, 2 September 2013 21:49:01 UTC+1, Zach Tellman wrote:
>>
>> When I announced Proteus [1], it was rightfully pointed out that it
>> didn't play nicely with macros which rely on &env, as well as a few forms
>> like 'letfn' that I hadn't explicitly handled.  This flaw has been shared
>> by pretty much every library of this sort, and since this is a problem I've
>> half-solved two or three times already, I figured something more general
>> and lasting was in order.
>>
>> The resulting library is called Riddley [2].  For obvious reasons, I've
>> named it after a book which is written entirely in a barely-readable pidgin
>> dialect. While there may be lingering issues, it's good enough to replace
>> the code-walking mechanism in Proteus, which I think makes it the best game
>> in town right now.  Bug reports and pull requests are welcome.
>>
>> Zach
>>
>> [1] https://groups.google.com/**forum/#!searchin/clojure/**
>> proteus/clojure/7HNNiJJTte4/**iMBWn8p6tZAJ<https://groups.google.com/forum/#!searchin/clojure/proteus/clojure/7HNNiJJTte4/iMBWn8p6tZAJ>
>> [2] 
>> https://github.com/**ztellman/riddley<https://github.com/ztellman/riddley>
>>
>  --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/a68aThpvP4o/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] riddley: code-walking without caveats

[Zach Tellman]

Hi Konrad,

Okay, I think I was just being dense.  I thought you were talking about a
different macroexpansion strategy, rather than just doing full
macroexpansion without interleaved compilation.  Thanks for your patience
in explaining what you meant.

I will note, though, that &env is an implicit argument to the macros, so
anything which works "exactly" like the compiler needs to mimic that as
well.

Zach


On Thu, Sep 5, 2013 at 3:09 AM, Konrad Hinsen <
googlegro...@khinsen.fastmail.net> wrote:

> **
> Zach Tellman writes:
>
>  > I guess I'm confused, then.  You contrast "complete recursive
>  > expansion" with what the compiler does, and then say it's recursive
>  > prewalk expansion, which is exactly what the compiler does.  Can
>  > you clarify the difference between what you're doing and what the
>  > compiler does?
>
> Here's an example:
>
>(defmacro foo [x]
>  `(list ~x ~x))
>
>(defmacro bar [x]
>  `[~x ~x])
>
> Now let's work on the form
>
>(foo (bar 'baz))
>
> Plain macroexpand returns
>
>(list (bar 'baz) (bar 'baz))
>
> whereas tools.macro/mexpand-all gives
>
>(list ['baz 'baz] ['baz 'baz])
>
> It does this by first calling macroexpand, so foo gets called exactly
> as during Clojure compilation and returns
>
>(list (bar 'baz) (bar 'baz))
>
> mexpand-all then goes through that form and expands the two subforms
> (bar 'baz).
>
> So mexpand-all does exactly what the compiler does, in particular it
> calls the macros with exactly the same arguments. But the compiler
> interleaves macro expansion with compilation, so it never gives you
> access to the fully expanded but uncompiled form which is
>
>(list ['baz 'baz] ['baz 'baz])
>
> Konrad
>
> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/a68aThpvP4o/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Handling name collisions with clojure.core

[Zach Tellman]

It is probably instructive to look at how (use-primitive-operators) works 
in primitive-math [1], though maybe not something you want to emulate.  The 
basic mechanism is pretty simple: use 'ns-unmap' to get rid of the 
operators you want to shadow, and bring in the operators from the alternate 
namespace.

Unfortunately, the next time you load the namespace, you'll get the same 
mess of collision warnings you were trying to avoid, because you haven't 
added the ':refer-clojure :exclude' clause to the ns declaration.  To get 
around this, use-primitive-operator hijacks 'ns' via alter-var-root so that 
this is automatically added if the alternate operators are detected [2], 
but otherwise leaves it unchanged.  This hijacking is undone if 
clojure.core is ever reloaded (usually by a :reload-all somewhere), 
however.  This could be fixed by adding a watcher to #'clojure.core/ns and 
forcing it back whenever it changes, but I haven't done that yet because it 
doesn't really affect me, and it feels hokey enough already.

Despite all that, though, it works pretty well.  Feel free to use this 
approach if you like, or create a less questionable variant if you can 
think of one.

Zach

[1] 
https://github.com/ztellman/primitive-math/blob/master/src/primitive_math.clj#L154
[2] 
https://github.com/ztellman/primitive-math/blob/master/src/primitive_math.clj#L135

On Wednesday, September 4, 2013 6:22:08 PM UTC-7, Mikera wrote:
>
> Hi all,
>
> While building the API for core.matrix, I've fun into a few cases where 
> the "best" name is a direct clash with clojure.core.
>
> Examples are "+", "zero?", "vector?", "=="
>
> In many of these cases, the core.matrix behaviour is a natural extension 
> of the clojure.core function (i.e. it extends the same functionality to 
> arbitrary N-dimensional arrays). 
>
> I'm not very happy with any of the options I can see for handling this:
>
> A) Use the good names in the "clojure.core.matrix" namespace. Problem: 
> that gives you a ton of nasty warnings of the type "WARNING: + already 
> refers to: #'clojure.core/+ in namespace: test.blank, being replaced by: 
> #'clojure.core.matrix/+". Significant boilerplate must be maintained by the 
> user in their ns declaration to prevent these warnings. I don't like 
> forcing users to maintain boilerplate, and I think that normal idiomatic 
> usage should be warning-free.
>
> B) Separate the name-clashing functions into separate namespaces - e.g. 
> "clojure.core.matrix.operators". Problem: that's something of an artificial 
> division, and again it forces users to do extra ns-management work to 
> access the functions they want.
>
> C) Use different names. Problem: names would be worse, and this would be 
> inconsistent and confusing, especially for functions that do effectively 
> the same thing.
>
> D) Encourage users to use aliases. Problem: that's horrendously ugly and 
> inconvenient for numerical code. Users with any sense of elegance in their 
> coding style would quite rightly throw their hands up in disgust. 
>
> Currently we're doing B), I'd prefer to do A) but can't figure out a way 
> to automatically suppress the warnings.
>
> Any better ideas?
>
>
>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] riddley: code-walking without caveats

[Zach Tellman]

I guess I'm confused, then.  You contrast "complete recursive expansion"
with what the compiler does, and then say it's recursive prewalk expansion,
which is exactly what the compiler does.  Can you clarify the difference
between what you're doing and what the compiler does?


On Wed, Sep 4, 2013 at 11:54 AM, Konrad Hinsen <
googlegro...@khinsen.fastmail.net> wrote:

> --On 4 septembre 2013 09:27:12 -0700 Zach Tellman 
> wrote:
>
>  So "complete recursive expansion" is postwalk macroexpansion?  It seems
>> like that could break anaphoric macros, and likely others.  A macro has
>> the option of calling macroexpand-all on its own contents if it wants
>> only special forms, but it shouldn't be forced to take only special forms.
>>
>
> Recursive macro expansion still works from outside in, so each macro gets
> to see the unexpanded form. It's only after the macro has done its
> transformation that the inner forms get expanded.
>
>
>  Also, here's a sketch of how you could do symbol macros using
>> Riddley: 
>> https://gist.github.**com/ztellman/6439318<https://gist.github.com/ztellman/6439318>.
>>  Please let me know
>> if I'm missing something w.r.t. how symbol macros are done in
>> tools.macros.
>>
>
> It's on my reading list for tomorrow!
>
>
> Konrad.
>
> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscribe@**googlegroups.com
> For more options, visit this group at
> http://groups.google.com/**group/clojure?hl=en<http://groups.google.com/group/clojure?hl=en>
> --- You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/**
> topic/clojure/a68aThpvP4o/**unsubscribe<https://groups.google.com/d/topic/clojure/a68aThpvP4o/unsubscribe>
> .
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscribe@**googlegroups.com
> .
> For more options, visit 
> https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out>
> .
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] riddley: code-walking without caveats

[Zach Tellman]

Actually, postwalk expansion (if that is in fact what you were describing)
would ignore any binding forms created by the outer macro.  This means that
something simple like:

(defmacro with-db [db & body]
  `(with-open [~db (create-db)]
 ~@body))

would be expanded without any knowledge of the 'db' local variable, since
that would only get turned into a let form later.  Prewalk is pretty much
the only way this works.


On Wed, Sep 4, 2013 at 9:50 AM, Ben Wolfson  wrote:

> Postwalk expansion would break macros that inspect their argument forms
> for e.g. writing special-purpose queries, if they *also* adopt the symbols
> "and" and "or" for conjunction or disjunction. Korma's "where", for
> instance, does this; one can write
>
> (select my-table (where (and (...) (...
>
> And the "where" detects the "and".
>
> Arguably this is wrongheaded behavior from the get-go (it can be somewhat
> confusing and makes it necessary to use something like clojure.core/and
> within "where" if you want normal clojure-land "and" semantics), but it's a
> style of non-anaphoric macro that relies on receiving an unexpanded form.
>
>
> On Wed, Sep 4, 2013 at 9:27 AM, Zach Tellman  wrote:
>
>> So "complete recursive expansion" is postwalk macroexpansion?  It seems
>> like that could break anaphoric macros, and likely others.  A macro has the
>> option of calling macroexpand-all on its own contents if it wants only
>> special forms, but it shouldn't be forced to take only special forms.
>>
>> Also, here's a sketch of how you could do symbol macros using Riddley:
>> https://gist.github.com/ztellman/6439318.  Please let me know if I'm
>> missing something w.r.t. how symbol macros are done in tools.macros.
>>
>> Zach
>>
>>
>> On Wed, Sep 4, 2013 at 2:16 AM, Konrad Hinsen <
>> googlegro...@khinsen.fastmail.net> wrote:
>>
>>> **
>>> On Wed, Sep 4, 2013, at 09:25 AM, Zach Tellman wrote:
>>>
>>> I'm not sure what you mean by "complete recursive expansion".  Could
>>> you expand
>>> on that?
>>>
>>> Completely ;-)
>>>
>>> By complete recursive expansion I mean that you get a form that is fully
>>> reduced to
>>> the core language, i.e. it contains no more macro applications at any
>>> level.
>>>
>>> If you leave macro expansion to the compiler, it does it when it arrives
>>> at the
>>> macro during evaluation. Then it does a plain non-recursive macroexpand
>>> and goes on
>>> evaluating. Any macro thus has access to the unexpanded contents of its
>>> form, but
>>> not to what it eventually expands to. For many applications that's just
>>> fine, which
>>> is why this approach has been the default in the Lisp world for a long
>>> time.
>>>
>>> As for replicating the behavior of the compiler, I'd assert that
>>> unless &env is
>>> precisely what it would be without ahead of time macroexpansion, the
>>> compiler's
>>> behavior isn't being replicated.
>>>
>>> I agree. tools.macro predates &env, which is why it is not supported.
>>> Since I have
>>> never need &env support and nobody ever asked for it (before now), it's
>>> not there.
>>> I don't see any reason why it couldn't be supported.
>>>
>>> Konrad.
>>>
>>>
>>> --
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To post to this group, send email to clojure@googlegroups.com
>>> Note that posts from new members are moderated - please be patient with
>>> your first post.
>>> To unsubscribe from this group, send email to
>>> clojure+unsubscr...@googlegroups.com
>>> For more options, visit this group at
>>> http://groups.google.com/group/clojure?hl=en
>>> ---
>>> You received this message because you are subscribed to a topic in the
>>> Google Groups "Clojure" group.
>>> To unsubscribe from this topic, visit
>>> https://groups.google.com/d/topic/clojure/a68aThpvP4o/unsubscribe.
>>> To unsubscribe from this group and all its topics, send an email to
>>> clojure+unsubscr...@googlegroups.com.
>>>
>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>
>>
>>  --
>> --
>> You receive

Re: [ANN] riddley: code-walking without caveats

[Zach Tellman]

So "complete recursive expansion" is postwalk macroexpansion?  It seems
like that could break anaphoric macros, and likely others.  A macro has the
option of calling macroexpand-all on its own contents if it wants only
special forms, but it shouldn't be forced to take only special forms.

Also, here's a sketch of how you could do symbol macros using Riddley:
https://gist.github.com/ztellman/6439318.  Please let me know if I'm
missing something w.r.t. how symbol macros are done in tools.macros.

Zach


On Wed, Sep 4, 2013 at 2:16 AM, Konrad Hinsen <
googlegro...@khinsen.fastmail.net> wrote:

> **
> On Wed, Sep 4, 2013, at 09:25 AM, Zach Tellman wrote:
>
> I'm not sure what you mean by "complete recursive expansion".  Could
> you expand
> on that?
>
> Completely ;-)
>
> By complete recursive expansion I mean that you get a form that is fully
> reduced to
> the core language, i.e. it contains no more macro applications at any
> level.
>
> If you leave macro expansion to the compiler, it does it when it arrives
> at the
> macro during evaluation. Then it does a plain non-recursive macroexpand
> and goes on
> evaluating. Any macro thus has access to the unexpanded contents of its
> form, but
> not to what it eventually expands to. For many applications that's just
> fine, which
> is why this approach has been the default in the Lisp world for a long
> time.
>
> As for replicating the behavior of the compiler, I'd assert that
> unless &env is
> precisely what it would be without ahead of time macroexpansion, the
> compiler's
> behavior isn't being replicated.
>
> I agree. tools.macro predates &env, which is why it is not supported.
> Since I have
> never need &env support and nobody ever asked for it (before now), it's
> not there.
> I don't see any reason why it couldn't be supported.
>
> Konrad.
>
>
> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/a68aThpvP4o/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] riddley: code-walking without caveats

[Zach Tellman]

I'm not sure what you mean by "complete recursive expansion".  Could you
expand on that?

As for replicating the behavior of the compiler, I'd assert that unless
&env is precisely what it would be without ahead of time macroexpansion,
the compiler's behavior isn't being replicated.  The tools.macro library
emulates an aspect of its behavior, certainly, and the fact that Clojure's
existed this long without anyone doing something like this indicates that
maybe this isn't such a huge omission, but without there remains an uncanny
valley.


On Wed, Sep 4, 2013 at 12:09 AM, Konrad Hinsen <
googlegro...@khinsen.fastmail.net> wrote:

> Zach Tellman writes:
>
>  > I see.  This is honestly something I hadn't considered, but since
>  > Riddley actually uses the Clojure compiler internals to track
>  > locals, this would be as simple as a (when-not (contains?
>  > (riddley.compiler/locals) (first expr)) ...) guard in the
>  > macroexpansion.
>
> If you don't need complete recursive expansion, that's indeed an
> approach worth exploring. For tools.macros that's not an option
> because the compiler knows nothing about local macros and symbol
> macros.
>
>  > As Ben points out, using the compiler this way is the only way to
>  > make sure that locals are consistent everywhere, rather than just
>  > in your own targeted use to track shadowing.
>
> Well, either you use the compiler or you replicate what it does.  For
> tools.macro I had to choose the second approach. I don't claim it has
> no bugs, I just claim I haven't had any bug reports ;-) (until today
> at least).
>
> Konrad
>
> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/a68aThpvP4o/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[ANN] clj-leveldb, idiomatic LevelDB bindings from Factual

[Zach Tellman]

In the first of what I hope will be many annoucements, we're open sourcing 
a library we've found useful at Factual: 
https://github.com/Factual/clj-leveldb.  This is just a simple wrapper 
around LevelDB, which is an in-process persistent k/v store from Google.

If anyone has questions, I'm happy to answer them.

Zach

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] riddley: code-walking without caveats

[Zach Tellman]

I see.  This is honestly something I hadn't considered, but since Riddley
actually uses the Clojure compiler internals to track locals, this would be
as simple as a (when-not (contains? (riddley.compiler/locals) (first expr))
...) guard in the macroexpansion.  As Ben points out, using the compiler
this way is the only way to make sure that locals are consistent
everywhere, rather than just in your own targeted use to track shadowing.

Hope that helps,
Zach


On Tue, Sep 3, 2013 at 12:41 PM, Konrad Hinsen <
googlegro...@khinsen.fastmail.net> wrote:

> --On 3 septembre 2013 02:08:23 -0700 Zach Tellman 
> wrote:
>
>  Hey Konrad, you can maybe speak with more authority as to what
>> tools.macro does and doesn't provide, but my reading of it is that it
>> does expression walking to prevent bound variables from being incorrectly
>> symbol-macroexpanded.  This seems only important in the context of
>> symbol macros, however; if you don't use symbol macros it's functionally
>> equivalent to clojure.walk/macroexpand-all.
>>
>
> Not quite. It expands only terms that are evaluated, using a built-in
> table of special forms, and it allows local macro definitions (macrolet).
> But most importantly, it tracks local bindings and expands only macros that
> are not shadowed. So if you have
>
>  (defmacro foo [] ...)
>  (let [foo (fn [] ...)]
> (foo 'bar))
>
> the form (foo 'bar) is not expanded because its local binding is a
> function. The version in clojure.walk doesn't take this into account, and
> can therefore produce incorrect code, which is a major pain to debug. I
> know because it happened to me, that's why I ended up writing my own macro
> expander. And that's why I wonder how riddley handled this.
>
>
> Konrad.
>
> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscribe@**googlegroups.com
> For more options, visit this group at
> http://groups.google.com/**group/clojure?hl=en<http://groups.google.com/group/clojure?hl=en>
> --- You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/**
> topic/clojure/a68aThpvP4o/**unsubscribe<https://groups.google.com/d/topic/clojure/a68aThpvP4o/unsubscribe>
> .
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscribe@**googlegroups.com
> .
> For more options, visit 
> https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out>
> .
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] riddley: code-walking without caveats

[Zach Tellman]

Hey Konrad, you can maybe speak with more authority as to what tools.macro
does and doesn't provide, but my reading of it is that it does expression
walking to prevent bound variables from being incorrectly
symbol-macroexpanded.  This seems only important in the context of symbol
macros, however; if you don't use symbol macros it's functionally
equivalent to clojure.walk/macroexpand-all.

This means that it suffers from all the same issues mentioned in Riddley's
readme, namely no &env and no expansion of inlined functions.  The code
walking is also only used to do expansion, no generic code walking
mechanism is exposed for more general transformations a la Proteus.

Hope that helps,
Zach


On Mon, Sep 2, 2013 at 11:20 PM, Konrad Hinsen <
googlegro...@khinsen.fastmail.net> wrote:

> --On 2 septembre 2013 13:49:01 -0700 Zach Tellman 
> wrote:
>
>  The resulting library is called Riddley [2].  For obvious reasons, I've
>> named it after a book which is written entirely in a barely-readable
>> pidgin dialect. While there may be lingering issues, it's good enough to
>> replace the code-walking mechanism in Proteus, which I think makes it the
>> best game in town right now.  Bug reports and pull requests are welcome.
>>
>
> How does this compare to mexpand-all in clojure.tools.macro?
>
> Konrad.
>
>
> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscribe@**googlegroups.com
> For more options, visit this group at
> http://groups.google.com/**group/clojure?hl=en<http://groups.google.com/group/clojure?hl=en>
> --- You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/**
> topic/clojure/a68aThpvP4o/**unsubscribe<https://groups.google.com/d/topic/clojure/a68aThpvP4o/unsubscribe>
> .
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscribe@**googlegroups.com
> .
> For more options, visit 
> https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out>
> .
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[ANN] riddley: code-walking without caveats

[Zach Tellman]

When I announced Proteus [1], it was rightfully pointed out that it didn't 
play nicely with macros which rely on &env, as well as a few forms like 
'letfn' that I hadn't explicitly handled.  This flaw has been shared by 
pretty much every library of this sort, and since this is a problem I've 
half-solved two or three times already, I figured something more general 
and lasting was in order.

The resulting library is called Riddley [2].  For obvious reasons, I've 
named it after a book which is written entirely in a barely-readable pidgin 
dialect. While there may be lingering issues, it's good enough to replace 
the code-walking mechanism in Proteus, which I think makes it the best game 
in town right now.  Bug reports and pull requests are welcome.

Zach

[1] 
https://groups.google.com/forum/#!searchin/clojure/proteus/clojure/7HNNiJJTte4/iMBWn8p6tZAJ
[2] https://github.com/ztellman/riddley

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: ANN: clj-tuple, efficient small collections

[Zach Tellman]

I've updated the library so tuples behave like vectors, rather than lists. 
 The macros have gotten kind of elephantine, but since I've already added 
every conceivable Clojure interface at least they won't get any bigger.  If 
there are any lingering gaps, though, please let me know.

Zach 

On Saturday, August 24, 2013 7:38:25 PM UTC-7, Zach Tellman wrote:
>
> I just pulled some code I wrote while trying to optimize 'memoize' into 
> its own library: https://github.com/ztellman/clj-tuple.  It only has the 
> one function, so I assume no one will need too much explanation.  However, 
> there may still be room for performance improvements, so if anyone wants to 
> take a stab, pull requests are welcome.
>
> Zach
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: ANN: clj-tuple, efficient small collections

[Zach Tellman]

I'm not sure I understand the motivation behind what you're suggesting.
 Why not just (nth tuple 0)?


On Mon, Aug 26, 2013 at 10:38 AM, Kevin Downey  wrote:

> A Tuple protocol that defines get0 get1 get3 etc for fast element access
> that doesn't tie you to using field names might be a good idea.
>
> On 8/25/13 9:35 AM, Zach Tellman wrote:
> > I don't think so, even the existence of all the Tuple* types are an
> > implementation detail, and you'd need to hint it as the right one to get
> > sane performance.  (nth t n) has good performance, you should prefer
> that.
> >
> > On Saturday, August 24, 2013 8:15:40 PM UTC-7, Ben wrote:
> >>
> >> Are the element names .e0, .e1 etc. considered part of the public
> >> interface of tuple objects?
> >>
> >>
> >> On Sat, Aug 24, 2013 at 7:38 PM, Zach Tellman  
> >>> wrote:
> >>
> >>> I just pulled some code I wrote while trying to optimize 'memoize' into
> >>> its own library: https://github.com/ztellman/clj-tuple.  It only has
> the
> >>> one function, so I assume no one will need too much explanation.
>  However,
> >>> there may still be room for performance improvements, so if anyone
> wants to
> >>> take a stab, pull requests are welcome.
> >>>
> >>> Zach
> >>>
> >>> --
> >>> --
> >>> You received this message because you are subscribed to the Google
> >>> Groups "Clojure" group.
> >>> To post to this group, send email to clo...@googlegroups.com
> 
> >>> Note that posts from new members are moderated - please be patient with
> >>> your first post.
> >>> To unsubscribe from this group, send email to
> >>> clojure+u...@googlegroups.com 
> >>> For more options, visit this group at
> >>> http://groups.google.com/group/clojure?hl=en
> >>> ---
> >>> You received this message because you are subscribed to the Google
> Groups
> >>> "Clojure" group.
> >>> To unsubscribe from this group and stop receiving emails from it, send
> an
> >>> email to clojure+u...@googlegroups.com .
> >>> For more options, visit https://groups.google.com/groups/opt_out.
> >>>
> >>
> >>
> >>
> >> --
> >> Ben Wolfson
> >> "Human kind has used its intelligence to vary the flavour of drinks,
> which
> >> may be sweet, aromatic, fermented or spirit-based. ... Family and social
> >> life also offer numerous other occasions to consume drinks for
> pleasure."
> >> [Larousse, "Drink" entry]
> >>
> >>
> >
>
>
> --
> And what is good, Phaedrus,
> And what is not good—
> Need we ask anyone to tell us these things?
>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: ANN: clj-tuple, efficient small collections

[Zach Tellman]

in my microbenchmarks I've found it to be consistently faster to unroll a 
fixed number of elements rather than iterate over them.  The difference 
might not be large enough to matter for many people's use cases, but the 
stated goal is to make a fast collection, so it's worthwhile to use the 
best known approach.

Currently tuples mimic lists, so conj adds onto the beginning, and does not 
return another tuple.  'rest', however, does return a tuple.  Also, since 
my intent wasn't to emulate vectors, 'get', 'assoc', and calling as 
function don't work with tuples.  This could obviously change, but I'd be 
curious to know why it makes a difference in someone's code.

Also, I played around with returning both a list or a vector in the 
unbounded case, but the performance is too different in each of these 
cases.  As I point out in the readme, vectors are *significantly* slower to 
create, which would make the performance of 'tuple' too inconsistent for my 
liking.  The TupleN solution I came up with was the best compromise I could 
think of.

Zach



On Saturday, August 24, 2013 10:47:11 PM UTC-7, Jozef Wagner wrote:
>
> Hi,
>
> I've did something similar for CLJS, see 
> http://dev.clojure.org/jira/browse/CLJS-453 and 
> https://groups.google.com/forum/#!searchin/clojure/arrayvector/clojure/yDvTRP0gYLA/Tb5MJC8Z0K8J
>
> In CLJS, it is comparable in performance to use array instead of 
> specialized type for each arity. Did you check if it is not the same in 
> Clojure?
>
> If I conjoin to the tuple, does it return a tuple too? Also instead of 
> tupleN, you could fall back to the PersistentVector.
>
> JW
>
>
> On Sun, Aug 25, 2013 at 4:38 AM, Zach Tellman 
> > wrote:
>
>> I just pulled some code I wrote while trying to optimize 'memoize' into 
>> its own library: https://github.com/ztellman/clj-tuple.  It only has the 
>> one function, so I assume no one will need too much explanation.  However, 
>> there may still be room for performance improvements, so if anyone wants to 
>> take a stab, pull requests are welcome.
>>
>> Zach
>>
>> -- 
>> -- 
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clo...@googlegroups.com
>> Note that posts from new members are moderated - please be patient with 
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+u...@googlegroups.com 
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to clojure+u...@googlegroups.com .
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: ANN: clj-tuple, efficient small collections

[Zach Tellman]

I don't think so, even the existence of all the Tuple* types are an 
implementation detail, and you'd need to hint it as the right one to get 
sane performance.  (nth t n) has good performance, you should prefer that.

On Saturday, August 24, 2013 8:15:40 PM UTC-7, Ben wrote:
>
> Are the element names .e0, .e1 etc. considered part of the public 
> interface of tuple objects?
>
>
> On Sat, Aug 24, 2013 at 7:38 PM, Zach Tellman 
> > wrote:
>
>> I just pulled some code I wrote while trying to optimize 'memoize' into 
>> its own library: https://github.com/ztellman/clj-tuple.  It only has the 
>> one function, so I assume no one will need too much explanation.  However, 
>> there may still be room for performance improvements, so if anyone wants to 
>> take a stab, pull requests are welcome.
>>
>> Zach
>>
>> -- 
>> -- 
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clo...@googlegroups.com
>> Note that posts from new members are moderated - please be patient with 
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+u...@googlegroups.com 
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to clojure+u...@googlegroups.com .
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>
>
>
> -- 
> Ben Wolfson
> "Human kind has used its intelligence to vary the flavour of drinks, which 
> may be sweet, aromatic, fermented or spirit-based. ... Family and social 
> life also offer numerous other occasions to consume drinks for pleasure." 
> [Larousse, "Drink" entry]
>
>  

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

ANN: clj-tuple, efficient small collections

[Zach Tellman]

I just pulled some code I wrote while trying to optimize 'memoize' into its 
own library: https://github.com/ztellman/clj-tuple.  It only has the one 
function, so I assume no one will need too much explanation.  However, 
there may still be room for performance improvements, so if anyone wants to 
take a stab, pull requests are welcome.

Zach

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.