Are you using Aleph in production? Tell us about it.

[Zach Tellman]

We're in the process of figuring out long-term plans for governance of 
Aleph [1], and it would like to understand who the current users are, and 
how they're using the library.  If you are using it, please respond to this 
issue: https://github.com/ztellman/aleph/issues/450.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] 'Elements of Clojure' is complete

[Zach Tellman]

I'm very happy to announce, only two and a half years after the release of 
the first chapter, that Elements of Clojure is completely finished.  
Further details can be found 
here: https://groups.google.com/forum/#!topic/elements-of-clojure/UUJjqU1rllU.

If you've never heard of the book before, please check out its website: 
http://elementsofclojure.com/

Zach

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] A complete draft of "Elements of Clojure" is available

[Zach Tellman]

Almost exactly two years ago, I announced on this mailing list that I was 
working on an intermediate-level book on Clojure [1], and released the 
first chapter.  I've confined updates since then to a book-specific list, 
but I feel like it's fair to post here again to say that the fourth and 
final chapter has been released.  Details can be found on the book's 
mailing 
list: https://groups.google.com/forum/#!topic/elements-of-clojure/t_Uqc8F0Ch0.

If you haven't heard about the book before, please check out its 
website: http://elementsofclojure.com/

Zach


[1] https://groups.google.com/forum/#!msg/clojure/Nh_Z0XaxhTA/P-lPdf2NDQAJ

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN and RFC] Bifurcan: impure functional data strucures

[Zach Tellman]

Are you relying on the immutability of these structures, or are they
effectively always transient?
On Sun, Apr 23, 2017 at 11:02 AM Dave Dixon <dave.d.di...@gmail.com> wrote:

> FWIW, the use-case I have essentially involves Monte Carlo simulations. So
> we start with a state (non-empty map), and then make a series of
> modifications to it. Various statistics are held in hash-maps keyed by the
> state, so there's a lot of lookups and modifications in those maps.
>
> That said, I'm not sure if for this particular case I care too much using
> Clojure idioms vs. direct API access. The algorithms tend to be
> hand-tweaked for performance anyway. The big win for me in wrapping
> bifurcan would be the ability to use spec without having to write
> specialized specs, generators, etc.
>
>
> On Thursday, April 20, 2017 at 9:53:56 PM UTC-7, Zach Tellman wrote:
>
>> Sure, happy to elaborate.  Bifurcan offers potential performance wins a
>> few different ways:
>>
>> * We can use standard Java equality semantics, bypassing all the overhead
>> of the hash calculations and enhanced numeric equality checks (this can
>> lead to moderate performance gains)
>> * We can use a mutable data structure as long as it never escapes a local
>> context (this can lead to significant performance gains)
>> * We can use the extra capabilities the data structures expose, like
>> concatenation, slicing, set operations, etc. (this is too dependent on the
>> use case to really quantify)
>>
>
>> it would be easy to have a `map` and `map*` method that expose Clojure
>> and Java equality semantics, respectively, but that puts a big onus on the
>> developer to determine if the latter is safe for their use case.  I've been
>> bit by this when I've used j.u.c.ConcurrentHashMap before, so I expect
>> people will suffer similarly weird bugs.
>>
>> However, I think there's a way to use the mutable data structures.
>> Technically, transient data structures allow arbitrary persistent data
>> structures to be batch updated, but in practice they tend to be empty, and
>> after they're populated they tend to be treated as read-only.
>>
>> If we're convinced this is common enough, every empty transient data
>> structure could be mutable, and when we make it persistent we could wrap it
>> in a "virtual" collection [1] which allows updates without touching the
>> base collection.  This would allow for faster writes, faster reads, and
>> only marginally slower updates if those are required.
>>
>> This is all predicated on a bunch of assumptions that are hard to
>> validate, but if this describes enough real-world use cases, it could lead
>> to a big, easy performance win.  It's even possible to automatically
>> replace the base Clojure collections with these alternatives using
>> something like Sleight [2].
>>
>> Anyway, that's what I've been mulling over.  If anyone has opinions, I'm
>> happy to hear them.
>>
>> Zach
>>
>> [1]
>> https://github.com/lacuna/bifurcan/blob/master/src/io/lacuna/bifurcan/Maps.java#L103
>> [2] https://github.com/ztellman/sleight
>>
>> On Thu, Apr 20, 2017 at 8:55 AM Dave Dixon <dave.d...@gmail.com> wrote:
>>
>>> Sounds great. If you have time, I'd certainly like to hear your thoughts
>>> on the issues of equality semantics and transients, maybe I can ponder and
>>> make some suggestions based on my target use-case.
>>>
>>>
>>> On Tuesday, April 18, 2017 at 9:32:32 AM UTC-7, Zach Tellman wrote:
>>>
>>>> To be clear, my intention was always to wrap the implementations in the
>>>> appropriate Clojure interfaces, and I don't believe that will cause much,
>>>> if any, of a performance hit (inlining is magic).  However, there are some
>>>> real questions regarding how to expose non-standard equality semantics, and
>>>> whether transients should be represented using the immutable or mutable
>>>> collection variants.
>>>>
>>>> For what it's worth, I have about 1/3 of an implementation of
>>>> Clojure-compatible versions of these data structures, I just wanted to mull
>>>> on the above questions a bit before going further.  I'm happy to discuss
>>>> them here in more depth if you have any questions or opinions.
>>>>
>>>> Zach
>>>>
>>>> On Tue, Apr 18, 2017 at 6:53 AM Dave Dixon <dave.d...@gmail.com> wrote:
>>>>
>>> Stared at this a bit yesterday. Seems like if you want to leverage spec
>>>>> while using bifurcan, then the

Re: [ANN and RFC] Bifurcan: impure functional data strucures

[Zach Tellman]

Sure, happy to elaborate.  Bifurcan offers potential performance wins a few
different ways:

* We can use standard Java equality semantics, bypassing all the overhead
of the hash calculations and enhanced numeric equality checks (this can
lead to moderate performance gains)
* We can use a mutable data structure as long as it never escapes a local
context (this can lead to significant performance gains)
* We can use the extra capabilities the data structures expose, like
concatenation, slicing, set operations, etc. (this is too dependent on the
use case to really quantify)

it would be easy to have a `map` and `map*` method that expose Clojure and
Java equality semantics, respectively, but that puts a big onus on the
developer to determine if the latter is safe for their use case.  I've been
bit by this when I've used j.u.c.ConcurrentHashMap before, so I expect
people will suffer similarly weird bugs.

However, I think there's a way to use the mutable data structures.
Technically, transient data structures allow arbitrary persistent data
structures to be batch updated, but in practice they tend to be empty, and
after they're populated they tend to be treated as read-only.

If we're convinced this is common enough, every empty transient data
structure could be mutable, and when we make it persistent we could wrap it
in a "virtual" collection [1] which allows updates without touching the
base collection.  This would allow for faster writes, faster reads, and
only marginally slower updates if those are required.

This is all predicated on a bunch of assumptions that are hard to validate,
but if this describes enough real-world use cases, it could lead to a big,
easy performance win.  It's even possible to automatically replace the base
Clojure collections with these alternatives using something like Sleight
[2].

Anyway, that's what I've been mulling over.  If anyone has opinions, I'm
happy to hear them.

Zach

[1]
https://github.com/lacuna/bifurcan/blob/master/src/io/lacuna/bifurcan/Maps.java#L103
[2] https://github.com/ztellman/sleight

On Thu, Apr 20, 2017 at 8:55 AM Dave Dixon <dave.d.di...@gmail.com> wrote:

> Sounds great. If you have time, I'd certainly like to hear your thoughts
> on the issues of equality semantics and transients, maybe I can ponder and
> make some suggestions based on my target use-case.
>
>
> On Tuesday, April 18, 2017 at 9:32:32 AM UTC-7, Zach Tellman wrote:
>
>> To be clear, my intention was always to wrap the implementations in the
>> appropriate Clojure interfaces, and I don't believe that will cause much,
>> if any, of a performance hit (inlining is magic).  However, there are some
>> real questions regarding how to expose non-standard equality semantics, and
>> whether transients should be represented using the immutable or mutable
>> collection variants.
>>
>> For what it's worth, I have about 1/3 of an implementation of
>> Clojure-compatible versions of these data structures, I just wanted to mull
>> on the above questions a bit before going further.  I'm happy to discuss
>> them here in more depth if you have any questions or opinions.
>>
>> Zach
>>
>> On Tue, Apr 18, 2017 at 6:53 AM Dave Dixon <dave.d...@gmail.com> wrote:
>>
> Stared at this a bit yesterday. Seems like if you want to leverage spec
>>> while using bifurcan, then the bifurcan types need to have the Clojure
>>> wrapper. The alternative appears to be re-implementing at least a large
>>> subset of collection-related spec code, which is a lot to bite off. Also
>>> tried updating some existing code to use bifurcan. Similar to spec, there
>>> are going to be cases which are less perf sensitive, where it would be nice
>>> to use code that is polymorphic for collections, and drop down to the fast
>>> interface in perf-sensitive parts.
>>>
>>>
>>> On Monday, April 17, 2017 at 1:52:39 PM UTC-7, Dave Dixon wrote:
>>>>
>>>> What is the issue with wrapping in Clojure interfaces? Added overhead
>>>> of function calls?
>>>>
>>>> I'm finding myself in the process of doing some of this, at least for
>>>> constructors. Also thinking of generating predicates/generators for use
>>>> with spec.
>>>>
>>>> On Monday, March 27, 2017 at 9:51:46 AM UTC-7, Zach Tellman wrote:
>>>>
>>>>> This is a slightly irregular announcement, because it's not for a
>>>>> Clojure library.  Rather, it's for a library written purely in Java:
>>>>> https://github.com/lacuna/bifurcan.
>>>>>
>>>>> This is a collection of mutable and immutable data structures,
>>>>> designed to address some of my personal fru

Re: [ANN and RFC] Bifurcan: impure functional data strucures

[Zach Tellman]

To be clear, my intention was always to wrap the implementations in the
appropriate Clojure interfaces, and I don't believe that will cause much,
if any, of a performance hit (inlining is magic).  However, there are some
real questions regarding how to expose non-standard equality semantics, and
whether transients should be represented using the immutable or mutable
collection variants.

For what it's worth, I have about 1/3 of an implementation of
Clojure-compatible versions of these data structures, I just wanted to mull
on the above questions a bit before going further.  I'm happy to discuss
them here in more depth if you have any questions or opinions.

Zach

On Tue, Apr 18, 2017 at 6:53 AM Dave Dixon <dave.d.di...@gmail.com> wrote:

> Stared at this a bit yesterday. Seems like if you want to leverage spec
> while using bifurcan, then the bifurcan types need to have the Clojure
> wrapper. The alternative appears to be re-implementing at least a large
> subset of collection-related spec code, which is a lot to bite off. Also
> tried updating some existing code to use bifurcan. Similar to spec, there
> are going to be cases which are less perf sensitive, where it would be nice
> to use code that is polymorphic for collections, and drop down to the fast
> interface in perf-sensitive parts.
>
>
> On Monday, April 17, 2017 at 1:52:39 PM UTC-7, Dave Dixon wrote:
>>
>> What is the issue with wrapping in Clojure interfaces? Added overhead of
>> function calls?
>>
>> I'm finding myself in the process of doing some of this, at least for
>> constructors. Also thinking of generating predicates/generators for use
>> with spec.
>>
>> On Monday, March 27, 2017 at 9:51:46 AM UTC-7, Zach Tellman wrote:
>>
>>> This is a slightly irregular announcement, because it's not for a
>>> Clojure library.  Rather, it's for a library written purely in Java:
>>> https://github.com/lacuna/bifurcan.
>>>
>>> This is a collection of mutable and immutable data structures, designed
>>> to address some of my personal frustrations with what's available in the
>>> Clojure and Java ecosystems.  Notably, they have pluggable equality
>>> semantics, so while they *can* use Clojure's expensive hash and equality
>>> checks, they don't *have* to.  They also provide high-performance mutable
>>> variants of the data structure which share an API with their immutable
>>> cousins.
>>>
>>> I'm posting it here to ask for people's thoughts on how, if at all, this
>>> should be exposed as a Clojure library.  It would be simple to simply wrap
>>> them in the Clojure interfaces and make them behave identically to
>>> Clojure's own data structures, but that kind of obviates the point.
>>> However, creating an entirely new set of accessors means that we can't
>>> leverage Clojure's standard library.
>>>
>>> It's possible that I'm alone in my frustrations, and no Clojure wrapper
>>> is necessary.  But if this does solve a problem you have, I'd like to hear
>>> more about what it is, and how you think Bifurcan might help.  Please feel
>>> free to reply here, or to grab me at Clojure/West and talk about it there.
>>>
>>> Thanks in advance,
>>> Zach
>>>
>> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/1m_I7IrDGb0/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN and RFC] Bifurcan: impure functional data strucures

[Zach Tellman]

Both the 'List' and 'Map' data structures in Bifurcan use innovative
approaches that were published after Clojure's original release [1] [2].
In the case of the immutable map, you get faster iteration and the
structural invariants allow for some clever stuff w.r.t. equality checks
and set operations.  In the case of the immutable list/vector, you get fast
concatenation, the ability to add and remove from both ends of the
collection, and a `subvec` that doesn't hold onto the entire underlying
data structure.

All of this is MIT licensed, so please feel welcome to open a PR against
Clojure to change the core data structures using my code, but I'd rate the
chance of that being accepted as somewhere between "low" and
"nonexistent".  Also, it should be noted that Clojure's implementation is
much more battle-tested than my own at this point.  But if anyone wants to
tilt at that particular windmill, feel free to ask me any questions you may
have about the implementation.

Zach

[1] https://michael.steindorfer.name/publications/oopsla15.pdf
[2] https://infoscience.epfl.ch/record/169879/files/RMTrees.pdf

On Mon, Mar 27, 2017 at 12:30 PM Luke Burton <luke_bur...@me.com> wrote:

>
> I'm not well versed enough in these data structures to know this without
> asking (apologies if it's really obvious to some people): is there
> opportunity to improve Clojure's built-in data structures with Bifurcan
> rather than trying to wrap Bifurcan's structures in Clojure?
>
> As an aside, I want to draw people's attention to the sweet little
> criterium + gnuplot setup you have there for generating benchmarking plots.
> Nice!
>
>
> On Mar 27, 2017, at 10:13 AM, Zach Tellman <ztell...@gmail.com> wrote:
>
> Benchmarks are available here, and the Clojure benchmarks make use of
> transients wherever possible:
> https://github.com/lacuna/bifurcan/blob/master/doc/benchmarks.md.
>
> More generally, while transients are often used in practice to quickly
> construct a read-only data structure, the more formal definition is that
> they provide an O(1) mechanism for transforming between immutable and
> mutable forms.  This isn't possible with purely mutable data structures
> like Java's HashMap or Bifurcan's LinearMap.  So while wrapping these data
> structures in the Clojure API would provide better performance for
> construction and lookups, it wouldn't be quite the same thing as a
> transient.
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/1m_I7IrDGb0/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN and RFC] Bifurcan: impure functional data strucures

[Zach Tellman]

Benchmarks are available here, and the Clojure benchmarks make use of
transients wherever possible:
https://github.com/lacuna/bifurcan/blob/master/doc/benchmarks.md.

More generally, while transients are often used in practice to quickly
construct a read-only data structure, the more formal definition is that
they provide an O(1) mechanism for transforming between immutable and
mutable forms.  This isn't possible with purely mutable data structures
like Java's HashMap or Bifurcan's LinearMap.  So while wrapping these data
structures in the Clojure API would provide better performance for
construction and lookups, it wouldn't be quite the same thing as a
transient.



On Mon, Mar 27, 2017 at 10:05 AM Michael Gardner <gardne...@gmail.com>
wrote:

>
> > On Mar 27, 2017, at 09:51, Zach Tellman <ztell...@gmail.com> wrote:
> >
> > They also provide high-performance mutable variants of the data
> structure which share an API with their immutable cousins.
>
> How does their performance compare to Clojure's transients? Transients are
> slower than Java's native mutable collections, so if the mutable
> collections in this library deliver the same performance as the latter,
> they could act as a drop-in replacement for the former (given a compatible
> Clojure wrapper).
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/1m_I7IrDGb0/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN and RFC] Bifurcan: impure functional data strucures

[Zach Tellman]

This is a slightly irregular announcement, because it's not for a Clojure 
library.  Rather, it's for a library written purely in 
Java: https://github.com/lacuna/bifurcan.

This is a collection of mutable and immutable data structures, designed to 
address some of my personal frustrations with what's available in the 
Clojure and Java ecosystems.  Notably, they have pluggable equality 
semantics, so while they *can* use Clojure's expensive hash and equality 
checks, they don't *have* to.  They also provide high-performance mutable 
variants of the data structure which share an API with their immutable 
cousins.  

I'm posting it here to ask for people's thoughts on how, if at all, this 
should be exposed as a Clojure library.  It would be simple to simply wrap 
them in the Clojure interfaces and make them behave identically to 
Clojure's own data structures, but that kind of obviates the point. 
 However, creating an entirely new set of accessors means that we can't 
leverage Clojure's standard library.

It's possible that I'm alone in my frustrations, and no Clojure wrapper is 
necessary.  But if this does solve a problem you have, I'd like to hear 
more about what it is, and how you think Bifurcan might help.  Please feel 
free to reply here, or to grab me at Clojure/West and talk about it there.

Thanks in advance,
Zach

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] Virgil 0.1.6

[Zach Tellman]

The code is split into a library and a minimal Leiningen plugin which runs 
it at startup.  If someone wants to contribute a Boot equivalent, I'm happy 
to accept that PR.

On Thursday, March 16, 2017 at 11:18:16 AM UTC-7, Gregg Reynolds wrote:
>
>
>
> On Mar 16, 2017 1:13 PM, "Zach Tellman" <ztel...@gmail.com > 
> wrote:
>
> I figured it was worth reminding everyone that this library exists: 
> https://github.com/ztellman/virgil.  It now seems to work on Java 
> projects of arbitrary size and structure (the in-process compiler is very 
> fussy about compile order, you need to topologically sort the classes), and 
> I've been using it extensively this last month while writing a bunch of 
> Java [1].  I can tweak some Java, wait a second, and then test that code at 
> the REPL.  It's been pretty game-changing for my workflow.
>
> If anyone has questions, I'm happy to answer them.
>
>
> wow!  looks great, when's the boot version due out? 
>
>
> Zach
>
> [1] https://github.com/lacuna/bifurcan
>
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com 
> Note that posts from new members are moderated - please be patient with 
> your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com 
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> --- 
> You received this message because you are subscribed to the Google Groups 
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to clojure+u...@googlegroups.com .
> For more options, visit https://groups.google.com/d/optout.
>
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] Virgil 0.1.6

[Zach Tellman]

I figured it was worth reminding everyone that this library 
exists: https://github.com/ztellman/virgil.  It now seems to work on Java 
projects of arbitrary size and structure (the in-process compiler is very 
fussy about compile order, you need to topologically sort the classes), and 
I've been using it extensively this last month while writing a bunch of 
Java [1].  I can tweak some Java, wait a second, and then test that code at 
the REPL.  It's been pretty game-changing for my workflow.

If anyone has questions, I'm happy to answer them.

Zach

[1] https://github.com/lacuna/bifurcan

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] Aleph 0.4.1

[Zach Tellman]

This is considerably simpler than the suggestions in this thread, but I've 
written a very basic statsd client and server in the literate 
examples: http://aleph.io/aleph/literate.html#aleph.examples.udp.  Feedback 
is welcome.

On Saturday, April 2, 2016 at 11:44:53 PM UTC-7, Zach Tellman wrote:
>
> This release represents a number of incremental improvements to 0.4.0, 
> which has been handling billions of daily requests for close to a year.  
>
> * Documentation can be found at http://aleph.io/
> * Literate examples of usage can be found at 
> http://aleph.io/aleph/literate.html
> * Comparative benchmarks can be found at 
> https://www.techempower.com/benchmarks/#section=data-r12=peak=plaintext=4,
>  
> which may or may not be relevant to your particular use case
>
> If anyone has questions, I'm happy to answer them.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] Aleph 0.4.1

[Zach Tellman]

Hi Adrian,

Glad to hear that you're getting good use out of Aleph.  As for the UDP
side of things, if you're using it (as games often do) to reimplement half
of TCP, that may not work well with the functional operators in
manifold.stream.  However, using `put!`, `take!`, and a few operators from
manifold.deferred like `chain` and `loop`, you should be able to easily
build any arbitrary call and response logic you need.  The code you write
will end up being pretty imperative, but sometimes that's just the best way
to reason about your problem.

I'm not sure I'll have time to look at the spec you linked anytime soon,
but I think it's reasonable to have some UDP-related code in the literate
examples.  I'll think about what would be a decent use case.

Best,
Zach

On Sun, Apr 3, 2016 at 12:52 PM <adrian.med...@mail.yu.edu> wrote:

> Awesome! At Vital Labs we use Aleph in production for our HTTP and (soon
> to be) websocket services. I have nothing but good things to say about it.
> It makes using Netty a breeze in Clojure.
>
> For an unrelated side project, I have been using Aleph to communicate over
> UDP with an old MMORPG called Star Wars Galaxies. Since UDP is not stream
> oriented, some of the architecture which make Aleph easy to reason about
> through Manifold for HTTP/TCP do not seem to carry over as cleanly when
> dealing with UDP services.
>
> Would you be open to writing a comprehensive UDP example for Aleph so that
> potential users could see how the author intends such services to be
> written in tandem with your other libraries, namely Manifold, Gloss, and
> byte-streams?
>
> In particular, SWG uses a convoluted protocol which involves optional XOR
> decryption of the body using an integer exchanged in the first packet you
> see from the game client (and saved for all future communication with the
> client until they go link dead), followed by optional decompression with
> DEFLATE of the body (not the header nor footer) depending on a value in the
> decrypted first byte of the footer, followed by variable-length (opcode
> driven) body decoding. To deal with the problem of unreliable transmission,
> you also need to ack sequenced packets before responding to a specific kind
> of message. Sometimes this needs to be repeated due to loss, etc. To
> respond you also need to invert the decoding process, so compress ->
> encrypt -> etc.
>
> Here's an overview of the protocol if you're interested in seeing the
> potential complexities that come up with protocols like these:
> http://wiki.swganh.org/index.php/Packet_Guides
>
> Since many of these requirements break the more functional, streaming
> nature of Manifold's design I have found that the code quickly devolves
> into madness. I imagine it would be very enlightening to see how you would
> solve these or similar problems with Aleph.
>
> Thanks for the great library,
> Adrian
>
> On Sunday, April 3, 2016 at 2:44:53 AM UTC-4, Zach Tellman wrote:
>>
>> This release represents a number of incremental improvements to 0.4.0,
>> which has been handling billions of daily requests for close to a year.
>>
>> * Documentation can be found at http://aleph.io/
>> * Literate examples of usage can be found at
>> http://aleph.io/aleph/literate.html
>> * Comparative benchmarks can be found at
>> https://www.techempower.com/benchmarks/#section=data-r12=peak=plaintext=4,
>> which may or may not be relevant to your particular use case
>>
>> If anyone has questions, I'm happy to answer them.
>>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/cNRTnvlPVG4/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group

[ANN] Aleph 0.4.1

[Zach Tellman]

This release represents a number of incremental improvements to 0.4.0, 
which has been handling billions of daily requests for close to a year.  

* Documentation can be found at http://aleph.io/
* Literate examples of usage can be found 
at http://aleph.io/aleph/literate.html
* Comparative benchmarks can be found 
at 
https://www.techempower.com/benchmarks/#section=data-r12=peak=plaintext=4,
 
which may or may not be relevant to your particular use case

If anyone has questions, I'm happy to answer them.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] Elements of Clojure

[Zach Tellman]

I changed the link to the Leanpub one. I think the content type may have
been wrong before, which maybe Android is overly sensitive to. Glad it's
working now.
On Sat, Mar 19, 2016 at 6:24 PM Gregg Reynolds <d...@mobileink.com> wrote:

>
> On Mar 19, 2016 4:16 PM, "Zach Tellman" <ztell...@gmail.com> wrote:
> >
> > The files have worked everywhere I've tried them. I've linked everything
> to the Leanpub download, just to reduce any change of divergent behavior,
> but I'm not sure there's more I can do. Thank you for the report, though,
> I'm sure you're not the only one seeing this.
>
> dunno what happened but now when I follow the link to you page and then
> click on the first chapter link it comes up immediately.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/Nh_Z0XaxhTA/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] Elements of Clojure

[Zach Tellman]

The files have worked everywhere I've tried them. I've linked everything to
the Leanpub download, just to reduce any change of divergent behavior, but
I'm not sure there's more I can do. Thank you for the report, though, I'm
sure you're not the only one seeing this.
On Sat, Mar 19, 2016 at 2:08 PM Gregg Reynolds <d...@mobileink.com> wrote:

>
> On Mar 19, 2016 3:46 PM, "Zach Tellman" <ztell...@gmail.com> wrote:
> >
> > Do you get the same error for
> http://samples.leanpub.com/elementsofclojure-sample.pdf?
> >
>
> Oy.  it might be the email client, who knows?  when I click on the
> leanpub.com link in my email client I get a "downloading" msg, then
> nothing.  when I go to the "myfiles" folder I see several versions of
> elemwntsofclojure, and I can open them all.  no idea what's going on but
> I'm inclined to blame samsung.
>
> fwiw I've seen this with many other pdf files, some work, some dont.  so I
> guess the lesson is just make a reasonable check that your file are good
> (run them through some version of acrobat? ) and blame any prob on
> cellphone vendors ;).
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/Nh_Z0XaxhTA/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] Elements of Clojure

[Zach Tellman]

Do you get the same error for
http://samples.leanpub.com/elementsofclojure-sample.pdf?

On Sat, Mar 19, 2016 at 1:40 PM Gregg Reynolds <d...@mobileink.com> wrote:

>
> On Mar 19, 2016 3:34 PM, "Gregg Reynolds" <d...@mobileink.com> wrote:
> >
> >
> > On Mar 19, 2016 3:32 PM, "Zach Tellman" <ztell...@gmail.com> wrote:
> > >
> > > Is this the PDF from elementsofclojure.com or leanpub.com?  They
> should be the same, but I just want to make sure I know all the details.
> >
> > the former.
>
> fwiw, I get the same result with the mobi link and the epub link: "invalid
> format".
>
> > >
> > > On Sat, Mar 19, 2016 at 1:07 PM Gregg Reynolds <d...@mobileink.com>
> wrote:
> > >>
> > >>
> > >> On Mar 17, 2016 12:47 PM, "Zach Tellman" <ztell...@gmail.com> wrote:
> > >> >
> > >> > I'm writing a book about Clojure, aimed at people who already know
> the core concepts, and want to use them more effectively.  The first
> chapter, "Names", is complete and can be read for free.  Details can be
> found at http://elementsofclojure.com/.
> > >>
> > >> fyi on my galaxy s5 the pdf is declared invalid so I can't read it.
> > >>
> > >> >
> > >> > I'm happy to answer any questions here, or on the book's mailing
> list at https://groups.google.com/forum/#!forum/elements-of-clojure.
> > >> >
> > >> > --
> > >> > You received this message because you are subscribed to the Google
> > >> > Groups "Clojure" group.
> > >> > To post to this group, send email to clojure@googlegroups.com
> > >> > Note that posts from new members are moderated - please be patient
> with your first post.
> > >> > To unsubscribe from this group, send email to
> > >> > clojure+unsubscr...@googlegroups.com
> > >> > For more options, visit this group at
> > >> > http://groups.google.com/group/clojure?hl=en
> > >> > ---
> > >>
> > >> > You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> > >>
> > >> > To unsubscribe from this group and stop receiving emails from it,
> send an email to clojure+unsubscr...@googlegroups.com.
> > >>
> > >>
> > >> > For more options, visit https://groups.google.com/d/optout.
> > >>
> > >> --
> > >> You received this message because you are subscribed to the Google
> > >> Groups "Clojure" group.
> > >> To post to this group, send email to clojure@googlegroups.com
> > >> Note that posts from new members are moderated - please be patient
> with your first post.
> > >> To unsubscribe from this group, send email to
> > >> clojure+unsubscr...@googlegroups.com
> > >> For more options, visit this group at
> > >> http://groups.google.com/group/clojure?hl=en
> > >> ---
> > >> You received this message because you are subscribed to a topic in
> the Google Groups "Clojure" group.
> > >> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/Nh_Z0XaxhTA/unsubscribe.
> > >> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> > >>
> > >> For more options, visit https://groups.google.com/d/optout.
> > >
> > > --
> > > You received this message because you are subscribed to the Google
> > > Groups "Clojure" group.
> > > To post to this group, send email to clojure@googlegroups.com
> > > Note that posts from new members are moderated - please be patient
> with your first post.
> > > To unsubscribe from this group, send email to
> > > clojure+unsubscr...@googlegroups.com
> > > For more options, visit this group at
> > > http://groups.google.com/group/clojure?hl=en
> > > ---
> > > You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> > > To unsubscribe from this group and stop receiving emails from it, send
> an email to clojure+unsubscr...@googlegroups.com.
> > > For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note t

Re: [ANN] Elements of Clojure

[Zach Tellman]

Is this the PDF from elementsofclojure.com or leanpub.com?  They should be
the same, but I just want to make sure I know all the details.

On Sat, Mar 19, 2016 at 1:07 PM Gregg Reynolds <d...@mobileink.com> wrote:

>
> On Mar 17, 2016 12:47 PM, "Zach Tellman" <ztell...@gmail.com> wrote:
> >
> > I'm writing a book about Clojure, aimed at people who already know the
> core concepts, and want to use them more effectively.  The first chapter,
> "Names", is complete and can be read for free.  Details can be found at
> http://elementsofclojure.com/.
>
> fyi on my galaxy s5 the pdf is declared invalid so I can't read it.
>
> >
> > I'm happy to answer any questions here, or on the book's mailing list at
> https://groups.google.com/forum/#!forum/elements-of-clojure.
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "Clojure" group.
> > To post to this group, send email to clojure@googlegroups.com
> > Note that posts from new members are moderated - please be patient with
> your first post.
> > To unsubscribe from this group, send email to
> > clojure+unsubscr...@googlegroups.com
> > For more options, visit this group at
> > http://groups.google.com/group/clojure?hl=en
> > ---
>
> > You received this message because you are subscribed to the Google
> Groups "Clojure" group.
>
> > To unsubscribe from this group and stop receiving emails from it, send
> an email to clojure+unsubscr...@googlegroups.com.
>
>
> > For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/Nh_Z0XaxhTA/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] Elements of Clojure

[Zach Tellman]

Thanks Bozhidar.  I agree that the "default name" sections can be expanded
a bit, and some related rules may also fall into the second chapter.  I
don't think I'll stray too far from what's in your guide.

On Sat, Mar 19, 2016 at 11:05 AM Bozhidar Batsov <bozhi...@batsov.com>
wrote:

> Seems to me that some more Clojure-specific naming rules can be
> incorporated into the first chapter (e.g. like the ones we have here
> https://github.com/bbatsov/clojure-style-guide/#naming). Other than this
> small remark - excellent work!
>
> On 19 March 2016 at 13:20, Val Waeselynck <val.vval...@gmail.com> wrote:
>
>> The chapter on naming is brilliant, I rarely learned so much in so few
>> pages :)
>>
>> So far I have found the content to be more about 'philosophical'
>> programming notions than Clojure specifically, but the parts about Clojure
>> are useful and pratical.
>>
>> You may want to state more explicitly how this books positions itself
>> relative to others.
>>
>> Looking forward to the rest,
>>
>> Val
>>
>>
>> On Thursday, 17 March 2016 18:47:55 UTC+1, Zach Tellman wrote:
>>>
>>> I'm writing a book about Clojure, aimed at people who already know the
>>> core concepts, and want to use them more effectively.  The first chapter,
>>> "Names", is complete and can be read for free.  Details can be found at
>>> http://elementsofclojure.com/.
>>>
>>> I'm happy to answer any questions here, or on the book's mailing list at
>>> https://groups.google.com/forum/#!forum/elements-of-clojure.
>>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clojure@googlegroups.com
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>>
> You received this message because you are subscribed to the Google Groups
>> "Clojure" group.
>>
> To unsubscribe from this group and stop receiving emails from it, send an
>> email to clojure+unsubscr...@googlegroups.com.
>>
>
>> For more options, visit https://groups.google.com/d/optout.
>>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/Nh_Z0XaxhTA/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] Elements of Clojure

[Zach Tellman]

I'm writing a book about Clojure, aimed at people who already know the core 
concepts, and want to use them more effectively.  The first chapter, 
"Names", is complete and can be read for free.  Details can be found 
at http://elementsofclojure.com/.  

I'm happy to answer any questions here, or on the book's mailing list 
at https://groups.google.com/forum/#!forum/elements-of-clojure.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] Elements of Clojure

[Zach Tellman]

To quote from the introduction:

> Where possible, this book will give specific, prescriptive advice on how
to write idiomatic Clojure. Everywhere else, it will describe the space of
possible approaches, and provide a framework for deciding which to use.

In my mind, a style guide confines itself to prescriptive advice, and
ignores anything which doesn't lend itself to that.  The latter part, which
is definitely demonstrated in the first chapter, makes this more than a
book about style.

However, it is definitely focused on giving practical advice about how to
write Clojure.  I would hope that the material isn't too surprising, given
the introduction.  If you feel like something is unclear or misleading,
please let me know what it is.

Zach

On Thu, Mar 17, 2016 at 12:10 PM JvJ <kfjwhee...@gmail.com> wrote:

> So far, this appears to be primarily focused on style and programming
> practice.  Is that going to be the primary focus of the book?
>
>
> On Thursday, 17 March 2016 10:47:55 UTC-7, Zach Tellman wrote:
>>
>> I'm writing a book about Clojure, aimed at people who already know the
>> core concepts, and want to use them more effectively.  The first chapter,
>> "Names", is complete and can be read for free.  Details can be found at
>> http://elementsofclojure.com/.
>>
>> I'm happy to answer any questions here, or on the book's mailing list at
>> https://groups.google.com/forum/#!forum/elements-of-clojure.
>>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/Nh_Z0XaxhTA/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Help] core.async and jetty

[Zach Tellman]

For what it's worth, Aleph will handle pipelined requests in 
parallel: https://github.com/ztellman/aleph.

On Tuesday, February 23, 2016 at 4:46:23 AM UTC-8, Miguel Ping wrote:
>
> Thanks, thats what I eventually found out.
>
> On Tuesday, February 23, 2016 at 12:39:00 PM UTC, jonah wrote:
>>
>> Hi Miguel- pipelining is essentially http keep alive. A very old jetty 
>> thread
>>
>>
>> http://jetty.4.x6.nabble.com/HTTP-1-1-Request-Pipelining-handling-td18682.html
>>  
>> 
>>
>> indicates that for simplicity jetty will execute subsequent requests on 
>> the kept-alive socket serially. 
>>
>> Jonah
>>   
>>
>> On Mon, Feb 22, 2016 at 3:33 PM, Miguel Ping  wrote:
>>
>>> Hi guys,
>>>
>>> I'm trying to replicate an experiment on nodejs and http pipelining: 
>>> http://blog.yld.io/2016/02/08/squeeze-the-juice-out-of-node/
>>>
>>> This is what I got right now: 
>>> https://gist.github.com/mping/98bb8eb9faf3c51f9889 (using 
>>> *com.ninjudd/ring-async*)
>>>
>>> Problem is I can't get pipelining to work as in nodejs; by doing two 
>>> reqs they are sequential, ie, the dates that I'm printing have two secs.
>>> Basically I want that the server prints two "identical" accpt dates, and 
>>> the  response should show that (now not working):
>>>
>>> $ tail -f reqs.txt | nc 127.0.0.1    
>>> 
>>>   
>>> HTTP/1.1 200 OK
>>> Date: Mon, 22 Feb 2016 20:29:10 GMT
>>> Transfer-Encoding: chunked
>>> Server: Jetty(7.6.8.v20121106)
>>>
>>> 48
>>> id: 1, uri: /a
>>> Mon Feb 22 20:29:10 WET 2016
>>> Mon Feb 22 20:29:12 WET 2016
>>> 0
>>>
>>> HTTP/1.1 200 OK
>>> Date: Mon, 22 Feb 2016 20:29:12 GMT
>>> Transfer-Encoding: chunked
>>> Server: Jetty(7.6.8.v20121106)
>>>
>>> 48
>>> id: 2, uri: /b
>>> Mon Feb 22 20:29:12 WET 2016
>>> Mon Feb 22 20:29:14 WET 2016
>>>
>>>
>>> I'm guessing that this is either my mistake with core.async or the 
>>> adapter.
>>>
>>> Thanks
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To post to this group, send email to clo...@googlegroups.com
>>> Note that posts from new members are moderated - please be patient with 
>>> your first post.
>>> To unsubscribe from this group, send email to
>>> clojure+u...@googlegroups.com
>>> For more options, visit this group at
>>> http://groups.google.com/group/clojure?hl=en
>>> --- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Clojure" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to clojure+u...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: What's the best option similar to Vert.x, Reactor, Nodejs for use with Clojure?

[Zach Tellman]

This is incidental to the topic of the thread, but to address why Manifold 
exists when there's already core.async, there's a write-up of the rationale 
linked in the readme: http://aleph.io/manifold/rationale.html.

On Saturday, January 23, 2016 at 6:50:34 AM UTC-5, qsys wrote:
>
> Depends on what you want... what I really like is the easy modularity: 
> deploy new 'verticles' somewhere in your network, and they're just all 
> connected through a the eventbus. I make a 'new' module, I put it somewhere 
> and it's automatically picked up by the appication and I can communicate 
> with it using that event bus. Is there something similar in clojure, or can 
> I achieve something similar in clojure? I checked 
> - catacumba: web toolkit, not what I'm after
> - sente: is about the web
> - aleph: comes closer: does have tcp-servers, but no autodiscovery, and 
> well, no 'event bus' (a kind of wrapper around netty etc).
> - manifold: I don't really see the added value, having core.async - but I 
> may fail to see something important here
> - pomegranate: is possibly very interesting in adding new modules... not 
> for removing 'old versions' of a module.
> - pedestal: have to have a deeper look into it, but so far, I have a 
> feeling it's more about web than 'intra-program' communication
>
> So, so far, I don't see how to implement something like the vert.x event 
> bus, with autodiscovery, in clojure (although I would love to see something 
> like it), expanding to the browser. I don't mind (and prefer) composing it 
> using different libraries, but I feel to see how to have this functionality 
> in clojure (now, I wrap the vertx eventbus in my clojure programs), and 
> since I already load vertx for the eventbus, I use it as web server as 
> well, if I need one... So well, if someone has an idea, it might be a nice 
> project I'd love to work on :).
>
> thx, qsys
>
> Op zondag 3 januari 2016 22:25:04 UTC+1 schreef tbc++:
>>
>> I've done some evaluations of Vert.x in the past and was rather 
>> underwhelmed. What is it that you are trying to accomplish? Stuff like 
>> Pedestal offers async web services, but without the complexity of an 
>> traditional evented server. So perhaps if we had a better idea of your 
>> requirements we could be a bit more helpful.
>>
>> So I'd say, look into Pedestal and then define what you need that it 
>> cannot do. Same for other toolkits like ring and httpkit. 
>>
>> Timothy
>>
>> On Sun, Jan 3, 2016 at 12:59 PM, adrians  wrote:
>>
>>>
>>> It used to be that Vert.x 2.x had integration for Clojure, but version 
>>> 3.x hasn't added it yet. Has anyone used this version through the Java API 
>>> and if so, how painful was it? Is Reactor any 
>>> better in that respect? What are people using when they want this kind of 
>>> back end?
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To post to this group, send email to clo...@googlegroups.com
>>> Note that posts from new members are moderated - please be patient with 
>>> your first post.
>>> To unsubscribe from this group, send email to
>>> clojure+u...@googlegroups.com
>>> For more options, visit this group at
>>> http://groups.google.com/group/clojure?hl=en
>>> --- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Clojure" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to clojure+u...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
>>
>> -- 
>> “One of the main causes of the fall of the Roman Empire was that–lacking 
>> zero–they had no way to indicate successful termination of their C 
>> programs.”
>> (Robert Firth) 
>>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: lambda detector?

[Zach Tellman]

The "correct" way to do this is:

(defn function-form? [x]
  (let [x' (macroexpand x)]
(and (sequential? x') (= 'fn* (first x')

However, the approach you're describing won't work with let-bound 
variables, only inline function definitions and vars, which means that your 
macro is a limited subset of Clojure, rather than a small extension of it. 
 People generally find that to be confusing.  If this is just for your own 
personal use, go nuts, but if this is meant to be understood and used by 
others, I suggest thinking carefully about whether this the approach you 
want to use.

Zach

On Tuesday, December 8, 2015 at 9:18:05 AM UTC-8, Gregg Reynolds wrote:
>
>
>
> On Tue, Dec 8, 2015 at 7:16 AM, Angel Java Lopez  > wrote:
>
>> Hi!
>>
>> Sorry, this is not an answer, but a question: Why you need to know if an 
>> argument is a lambda? What is your use case?
>>
>
> A simple use case (somewhat different from my actual case, but clearer) 
> would be dynamic function dispatch.  You have a vector of functions and a 
> dispatch function that takes a vector of args and selects the "best match" 
> from the fns vector.
>
> If things are expressed like so:
>
> (defn f1 ...) (defn f2 ...) ...
> (def fns [f1 f2 ...])
>
> then it's easy: the dispatcher can pull the :arglists from the var 
> metadata for the fn symbols and compare it with the incoming args.  But if 
> you allow inline lambdas you can't (at least I don't know how to) access 
> the var metadata.  In my case this is a problem because some of my 
> functions will have a bit of custom metadata.
>
> To complicate things, let's say we allow clients to register functions, 
> using a register-fn function or macro, for example.  In my case, I have a 
> function and I use it to pre-register some fns defined using #(...) 
> syntax.  The problem I'm encountering is that these functions do not seem 
> to satisfy fn? or any other predicate I can think of to detect that they 
> are in fact functions.  Their types look like this:  class 
> miraj.sync$config_polymer_reqs$fn__16463 
> miraj.sync$config_polymer_reqs$fn__16463@541276dc - from this it is clear 
> that they are function objects, but I don't want to rely on parsing out 
> this kind of stuff.  FYI mirj.sync is my namespace, config-polymer-reqs is 
> my function, and #(...) is expressed in the result of a call to register-fn 
> within config-polymer-reqs.
>
> (My use case involves macros and alter-var-root stuff to manage system 
> config at startup.  I don't have the heart to try to describe it at the 
> moment.)
>
> Thanks,
>
> Gregg
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] lein-virgil, a plugin for mixed Clojure/Java development

[Zach Tellman]

It's been an uncomfortably long time since I released a new library, so 
here we go.  This plugin solves a problem that has endlessly bothered me 
(and hopefully a number of other people which is greater than zero), which 
is that changing any Java files in a project requires completely reloading 
the REPL session.  But no longer: https://github.com/ztellman/virgil.  This 
project, named for everyone's favorite psychopomp, leverages the compiler 
APIs within the JDK to live-reload the classes without a bit of fuss.

This code works for me, and will probably work for most simple use cases, 
but I wouldn't be surprised if there are some more complex cases that don't 
work perfectly.  Bug reports are welcome.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Manifold questions: Creating sinks/sources

[Zach Tellman]

Hi Atamert,

For future reference, posting these questions 
to https://groups.google.com/forum/#!forum/aleph-lib will ensure I'll see 
them sooner.  

The `source-only` method is just a way to make sure that the chance for 
confusion is minimized, it doesn't prevent the underlying object from being 
used normally.  As for your scenario, I'm not sure I completely understand. 
 In a standard stream, the flow of data is:

[ sink - source ]

So if you `put!` something into the sink, you can `take!` it from the 
source.  If you use `source-only` and connect that to some other sink, then 
the flow of data will be

[ sink - source ] - sink

So the source is only connected to one sink, not two.  Maybe I'm 
misunderstanding you, though.

Zach

On Thursday, August 27, 2015 at 4:13:35 AM UTC-7, Atamert Ölçgen wrote:

 Hi,

 AFAIK the only way to create (just) a source (or sink) is:

 (def my-source (s/source-only (s/stream ...)))

 This results in creating a stream and then wrapping it with a SourceProxy. 
 We don't keep a reference to the stream and the SourceProxy doesn't allow 
 taking.

 But if I'm not missing something since SourceProxy keeps a reference of 
 the original stream, there is a sink nobody is using there.

 If I create a sink separately and connect my-source to it, now 
 my-source would be connected to two sinks.

 My questions are:

 1. Is there another method for creating only sinks or sources?
 2. Should the extra/unused source/sink I mentioned above cause concern?

 PS: I'm not talking about a scenario where we're creating thousands 
 streams and connecting them all like there's no tomorrow.


 -- 
 Kind Regards,
 Atamert Ölçgen

 ◻◼◻
 ◻◻◼
 ◼◼◼

 www.muhuk.com


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: What does Zach Tellman mean by factored out for greater inlining joy

[Zach Tellman]

It's fluid, but the output that Norman uses in his post (and that motivated 
the changes to Aleph) were from the JVM explicitly saying I would have 
inlined this, but it was too big.  That may not be true under other 
circumstances, but you're only helping by factoring out uncommon, large 
code branches.

On Tuesday, August 11, 2015 at 2:01:59 PM UTC-7, Colin Fleming wrote:

 That's a really interesting post, thanks for that. I actually cornered 
 Cliff Click at Curry On because I was interested in knowing how well the 
 JVM inlined var indirection. The short answer is it's complicated. But if 
 the JVM does inline your method, then the var indirection shouldn't cost 
 you as long as the var content is stable.

 Tom Crayford's great talk at EuroClojure also pointed out that another 
 thing which affects inlining is the stack depth, which can mean you're 
 damned if you do and you're damned if you don't when trying to refactor to 
 help with inlining. However Cliff said that all these limits are pretty 
 fluid - it's not like a method over 325 bytecodes will never be inlined, if 
 it's identified as hot that limit goes way up, and a relatively cold method 
 can still be inlined even if it's small.

 I'd actually love to sit down and test this with Clojure sometime, but I 
 never seem to find time for it.

 On 11 August 2015 at 21:00, Zach Tellman ztel...@gmail.com javascript: 
 wrote:

 The inlining part is explained very well by this blog post 
 http://normanmaurer.me/blog/2014/05/15/Inline-all-the-Things/

 As for why I left all the repetition in there, I tend to let code expand 
 before getting annoyed and compacting it.  Sometimes there's a commit 
 between those two events, sometimes there's not.  In this case, you get to 
 see how the macro/abstraction sausage is made.

 Happy to answer any other questions,
 Zach


 On Sunday, August 9, 2015 at 9:38:51 AM UTC-7, Lawrence Krubner wrote:


 Reid, thank you. I think you answer half the question. You make a good 
 point about giving the JVM a way to better optimize a hot path. I think you 
 are right about that. But, given the large amount of repetition between 
 chain'- and chain- I'm wondering why this wasn't done with a macro? 



 On Sunday, August 9, 2015 at 2:08:47 AM UTC-4, Reid McKenzie wrote:

 -BEGIN PGP SIGNED MESSAGE- 
 Hash: SHA256 

 Lawrence, 

 This is just a theory, but in the interests of response time, the JVM 
 uses a large number of heuristics to determine what optimizations will 
 likely prove profitable. One of them is a budget for method size. I 
 would guess that lifting this code out into a separate fn made the JVM 
 see that it was optimizing a hot path between the main body and 
 several small but tightly related methods thus giving itself more 
 leeway to inline and optimize in ways that it would otherwise presume 
 are more expensive and not pursue. 

 Reid 

 -- 
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com 
 javascript:
 Note that posts from new members are moderated - please be patient with 
 your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com javascript:
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 --- 
 You received this message because you are subscribed to the Google Groups 
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to clojure+u...@googlegroups.com javascript:.
 For more options, visit https://groups.google.com/d/optout.




-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] data.int-map 0.2.1

[Zach Tellman]

https://github.com/clojure/data.int-map

This is a small but important update.  Where 0.2.0 added the ability to use 
negative integers, it did *not* retain the ordered key invariant when 
negative numbers were used (negative numbers came after the positive 
numbers, because binary representations are a continual delight).  I've 
pushed an update such that the ordering is correct, and also pushed some of 
the int-set implementation into Java, yielding a 20-30% performance 
improvement.  This should be a drop-in replacement for anyone using earlier 
versions of the library.

Zach

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: What does Zach Tellman mean by factored out for greater inlining joy

[Zach Tellman]

The inlining part is explained very well by this blog 
post http://normanmaurer.me/blog/2014/05/15/Inline-all-the-Things/

As for why I left all the repetition in there, I tend to let code expand 
before getting annoyed and compacting it.  Sometimes there's a commit 
between those two events, sometimes there's not.  In this case, you get to 
see how the macro/abstraction sausage is made.

Happy to answer any other questions,
Zach

On Sunday, August 9, 2015 at 9:38:51 AM UTC-7, Lawrence Krubner wrote:


 Reid, thank you. I think you answer half the question. You make a good 
 point about giving the JVM a way to better optimize a hot path. I think you 
 are right about that. But, given the large amount of repetition between 
 chain'- and chain- I'm wondering why this wasn't done with a macro? 



 On Sunday, August 9, 2015 at 2:08:47 AM UTC-4, Reid McKenzie wrote:

 -BEGIN PGP SIGNED MESSAGE- 
 Hash: SHA256 

 Lawrence, 

 This is just a theory, but in the interests of response time, the JVM 
 uses a large number of heuristics to determine what optimizations will 
 likely prove profitable. One of them is a budget for method size. I 
 would guess that lifting this code out into a separate fn made the JVM 
 see that it was optimizing a hot path between the main body and 
 several small but tightly related methods thus giving itself more 
 leeway to inline and optimize in ways that it would otherwise presume 
 are more expensive and not pursue. 

 Reid 



-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Manifold streams/deferreds raising errors downstream of a sink

[Zach Tellman]

Hi Andy,

To give you some context on this, `put!` will return true if the thing 
directly downstream accepts the value (in this case, the stream).  The fact 
that the callback in `consume` fails is between the stream and the 
callback, and is not propagated all the way back (though the exception will 
cancel the `consume`, which will cause the stream to close).  The approach 
described by Atamert is a reasonable one to get the desired result.

Zach

On Sunday, August 9, 2015 at 9:13:30 PM UTC-7, Andy Chambers wrote:

 Hey All,

 I'm trying to build a library that exposes manifold streams for data 
 producing applications. The intention
 would be for these apps to s/put! to a sink stream returned by the 
 producer function. However, I need to
 take the value that is put in and pipe it into some other function before 
 I know whether there is an error
 and if there is an error, I'd like to propagate it back to the caller.

 I understand that the value returned by put! is a deferred, and how to set 
 the error state of a deferred but I
 don't understand how I can arrange for the deferred to be manipulated 
 downstream of where it has been
 taken off the stream.

 (defn producer []
   (let [in (s/stream)]

 (s/consume (fn [msg]
  (d/future
(prn oops, going to error now)
(throw (Exception. (str msg) in)
 
 in))

 @(s/put! (producer) 42)

 As written, you get a success response even though an exception was 
 triggered by the function that consumes
 the in stream. Is it possible to re-write this so that the exception 
 bubbles up to where the put! is deref'd?

 Cheers,
 Andy


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] Clojure 1.8.0-alpha2

[Zach Tellman]

A similar issue was reported earlier in the thread, target [potemkin 
0.4.1] and see if that fixes it.

On Tuesday, July 21, 2015 at 12:24:43 PM UTC-7, Rangel Spasov wrote:

 Hey guys,

 Getting this error with 1.8.0-alpha2, I think related to aleph (using 
 0.4.0, latest version at the moment).

 #error {

  :cause IllegalName: 
 compile__stub.aleph.http.core.aleph.http.core/HeaderMap

  :via

  [{:type clojure.lang.Compiler$CompilerException

:message java.lang.NoClassDefFoundError: IllegalName: 
 compile__stub.aleph.http.core.aleph.http.core/HeaderMap, 
 compiling:(aleph/http/core.clj:81:1)

:at [clojure.lang.Compiler analyzeSeq Compiler.java 6798]}

   {:type java.lang.NoClassDefFoundError

:message IllegalName: 
 compile__stub.aleph.http.core.aleph.http.core/HeaderMap

:at [java.lang.ClassLoader preDefineClass ClassLoader.java 654]}]

  :trace

  [[java.lang.ClassLoader preDefineClass ClassLoader.java 654]

   [java.lang.ClassLoader defineClass ClassLoader.java 758]

   [java.lang.ClassLoader defineClass ClassLoader.java 642]

   [clojure.lang.DynamicClassLoader defineClass DynamicClassLoader.java 46]

   [clojure.lang.Compiler$NewInstanceExpr compileStub Compiler.java 7815]

   [clojure.lang.Compiler$NewInstanceExpr build Compiler.java 7680]

   [clojure.lang.Compiler$NewInstanceExpr$DeftypeParser parse Compiler.java 
 7590]

   [clojure.lang.Compiler analyzeSeq Compiler.java 6791]

   [clojure.lang.Compiler analyze Compiler.java 6592]

   [clojure.lang.Compiler analyze Compiler.java 6553]

   [clojure.lang.Compiler$BodyExpr$Parser parse Compiler.java 5929]

   [clojure.lang.Compiler$LetExpr$Parser parse Compiler.java 6247]

   [clojure.lang.Compiler analyzeSeq Compiler.java 6791]

   [clojure.lang.Compiler analyze Compiler.java 6592]

   [clojure.lang.Compiler analyze Compiler.java 6553]

   [clojure.lang.Compiler$BodyExpr$Parser parse Compiler.java 5929]

   [clojure.lang.Compiler$FnMethod parse Compiler.java 5359]

   [clojure.lang.Compiler$FnExpr parse Compiler.java 3959]

   [clojure.lang.Compiler analyzeSeq Compiler.java 6789]

   [clojure.lang.Compiler analyze Compiler.java 6592]

   [clojure.lang.Compiler eval Compiler.java 6847]

   [clojure.lang.Compiler eval Compiler.java 6839]

   [clojure.lang.Compiler load Compiler.java 7295]

   [clojure.lang.RT loadResourceScript RT.java 372]

   [clojure.lang.RT loadResourceScript RT.java 363]

   [clojure.lang.RT load RT.java 453]

   [clojure.lang.RT load RT.java 419]

   [clojure.core$load$fn__5448 invoke core.clj 5866]

   [clojure.core$load doInvoke core.clj 5865]

   [clojure.lang.RestFn invoke RestFn.java 408]

   [clojure.core$load_one invoke core.clj 5671]

   [clojure.core$load_lib$fn__5397 invoke core.clj 5711]

   [clojure.core$load_lib doInvoke core.clj 5710]

   [clojure.lang.RestFn applyTo RestFn.java 142]

   [clojure.core$apply invoke core.clj 632]

   [clojure.core$load_libs doInvoke core.clj 5749]

   [clojure.lang.RestFn applyTo RestFn.java 137]

   [clojure.core$apply invoke core.clj 632]

   [clojure.core$require doInvoke core.clj 5832]

   [clojure.lang.RestFn invoke RestFn.java 551]

   [aleph.http.server$eval9251$loading__5340__auto9252 invoke 
 server.clj 1]

   [aleph.http.server$eval9251 invoke server.clj 1]

   [clojure.lang.Compiler eval Compiler.java 6850]

   [clojure.lang.Compiler eval Compiler.java 6839]

   [clojure.lang.Compiler load Compiler.java 7295]

   [clojure.lang.RT loadResourceScript RT.java 372]

   [clojure.lang.RT loadResourceScript RT.java 363]

   [clojure.lang.RT load RT.java 453]

   [clojure.lang.RT load RT.java 419]

   [clojure.core$load$fn__5448 invoke core.clj 5866]

   [clojure.core$load doInvoke core.clj 5865]

   [clojure.lang.RestFn invoke RestFn.java 408]

   [clojure.core$load_one invoke core.clj 5671]

   [clojure.core$load_lib$fn__5397 invoke core.clj 5711]

   [clojure.core$load_lib doInvoke core.clj 5710]

   [clojure.lang.RestFn applyTo RestFn.java 142]

   [clojure.core$apply invoke core.clj 632]

   [clojure.core$load_libs doInvoke core.clj 5753]

   [clojure.lang.RestFn applyTo RestFn.java 137]

   [clojure.core$apply invoke core.clj 632]

   [clojure.core$require doInvoke core.clj 5832]

   [clojure.lang.RestFn invoke RestFn.java 457]

   [aleph.http$eval1594$loading__5340__auto1595 invoke http.clj 1]

   [aleph.http$eval1594 invoke http.clj 1]

   [clojure.lang.Compiler eval Compiler.java 6850]

   [clojure.lang.Compiler eval Compiler.java 6839]

   [clojure.lang.Compiler load Compiler.java 7295]

   [clojure.lang.RT loadResourceScript RT.java 372]

   [clojure.lang.RT loadResourceScript RT.java 363]

   [clojure.lang.RT load RT.java 453]

   [clojure.lang.RT load RT.java 419]

   [clojure.core$load$fn__5448 invoke core.clj 5866]

   [clojure.core$load doInvoke core.clj 5865]

   [clojure.lang.RestFn invoke RestFn.java 408]

   [clojure.core$load_one invoke core.clj 5671]

   [clojure.core$load_lib$fn__5397 invoke core.clj 5711]

   

Re: [ANN] Clojure 1.8.0-alpha2

[Zach Tellman]

You're also going to have to target [clj-tuple 0.2.2], since something 
else seems to be shadowing that depedency.  Sorry for all the fuss.

On Sunday, July 19, 2015 at 5:02:13 PM UTC-7, Sean Corfield wrote:

 Bumping clj-http to 2.0.0 got me past that problem (and it uses Potemkin 
 0.4.1) but then I hit the problem below, which I’ve run into several times 
 trying to use updated versions of clj-http over the last year.

 Messagejava.lang.RuntimeException: No such var: clj-tuple/vector, 
 compiling:(potemkin/utils.clj:110:10)Cause
 clojure.lang.Compiler$CompilerExceptionStacktraceThe Error Occurred in
 *core.clj: line 5866* 
 *called from* core.clj: line 5865 
 *called from* core.clj: line 5671 
 *called from* core.clj: line 5711 
 *called from* core.clj: line 5710 
 *called from* core.clj: line 632 
 *called from* core.clj: line 5753 
 *called from* core.clj: line 634 
 *called from* core.clj: line 5843 
 *called from* collections.clj: line 1 
 *called from* collections.clj: line 1 
 *called from* core.clj: line 5866 
 *called from* core.clj: line 5865 
 *called from* core.clj: line 5671 
 *called from* core.clj: line 5711 
 *called from* core.clj: line 5710 
 *called from* core.clj: line 632 
 *called from* core.clj: line 5753 
 *called from* core.clj: line 632 
 *called from* core.clj: line 5832 
 *called from* potemkin.clj: line 1 
 *called from* potemkin.clj: line 1 
 *called from* core.clj: line 5866 
 *called from* core.clj: line 5865 
 *called from* core.clj: line 5671 
 *called from* core.clj: line 5711 
 *called from* core.clj: line 5710 
 *called from* core.clj: line 632 
 *called from* core.clj: line 5749 
 *called from* core.clj: line 632 
 *called from* core.clj: line 5832 
 *called from* headers.clj: line 1 
 *called from* headers.clj: line 1 
 *called from* core.clj: line 5866 
 *called from* core.clj: line 5865 
 *called from* core.clj: line 5671 
 *called from* core.clj: line 5711 
 *called from* core.clj: line 5710 
 *called from* core.clj: line 632 
 *called from* core.clj: line 5749 
 *called from* core.clj: line 632 
 *called from* core.clj: line 5832 
 *called from* core.clj: line 1 
 *called from* core.clj: line 1 
 *called from* core.clj: line 5866 
 *called from* core.clj: line 5865 
 *called from* core.clj: line 5671 
 *called from* core.clj: line 5711 
 *called from* core.clj: line 5710 
 *called from* core.clj: line 632 
 *called from* core.clj: line 5749 
 *called from* core.clj: line 632 
 *called from* core.clj: line 5832 
 *called from* client.clj: line 1 


 On Jul 19, 2015, at 4:53 PM, Michael Blume blume...@gmail.com 
 javascript: wrote:

 Looks like it has, pinning to Potemkin 0.4.1 should probably sort you out

 On Sun, Jul 19, 2015 at 4:50 PM Michael Blume blume...@gmail.com 
 javascript: wrote:

 Sean, I think that was identified as a bug in Potemkin. The pull was 
 merged but I'm not sure if there's been a release since. Zack?

 https://github.com/ztellman/potemkin/pull/40


 http://dev.clojure.org/jira/browse/CLJ-1208?focusedCommentId=39632page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-39632

 On Sun, Jul 19, 2015 at 4:47 PM Sean Corfield se...@corfield.org 
 javascript: wrote:

 On Jul 18, 2015, at 10:33 PM, Sean Corfield se...@corfield.org 
 javascript: wrote:
  Wow, that's a fast timeline. Thank you. We'll upgrade to Alpha 2 this 
 week. We may go to production with it fairly quickly.

 Switched out 1.7.0 for 1.8.0-alpha2 and got the exception below. Posting 
 here in case anyone knows immediately what the cause is, while I go 
 debugging...

 Exception in thread main java.lang.NoClassDefFoundError: IllegalName: 
 compile__stub.clj_http.headers.clj-http.headers/HeaderMap, 
 compiling:(clj_http/headers.clj:105:1)
 at clojure.lang.Compiler.analyzeSeq(Compiler.java:6798)
 at clojure.lang.Compiler.analyze(Compiler.java:6592)
 at clojure.lang.Compiler.analyze(Compiler.java:6553)
 at 
 clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:5929)
 at clojure.lang.Compiler$LetExpr$Parser.parse(Compiler.java:6247)
 at clojure.lang.Compiler.analyzeSeq(Compiler.java:6791)
 at clojure.lang.Compiler.analyze(Compiler.java:6592)
 at clojure.lang.Compiler.analyze(Compiler.java:6553)
 at 
 clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:5929)
 at clojure.lang.Compiler$FnMethod.parse(Compiler.java:5359)
 at clojure.lang.Compiler$FnExpr.parse(Compiler.java:3959)
 at clojure.lang.Compiler.analyzeSeq(Compiler.java:6789)
 at clojure.lang.Compiler.analyze(Compiler.java:6592)
 at clojure.lang.Compiler.eval(Compiler.java:6847)
 at clojure.lang.Compiler.eval(Compiler.java:6839)
 at clojure.lang.Compiler.load(Compiler.java:7295)
 at clojure.lang.RT.loadResourceScript(RT.java:372)
 at clojure.lang.RT.loadResourceScript(RT.java:363)
 at clojure.lang.RT.load(RT.java:453)
 at 

Re: RejectedExecutionException? I don't think I was using anything concurrent?

[Zach Tellman]

A RejectedExecutionException is thrown when a thread-pool's queue is full. 
 Try using 'jstack' to take a thread dump, and see who has spun up a thread 
pool under the covers.  Possible culprits include 'at-at' and 'carmine'.  

On Friday, July 17, 2015 at 11:01:05 AM UTC-7, sugarste...@gmail.com wrote:


 Okay, this error seems to be triggered by a NullPointerException which 
 seems to arise when I try to fetch a non-existent document out of Redis. I 
 am a bit of a noob, so perhaps someone can explain how this works. I'm not 
 aware of using any of Clojure's concurrency tools in this (very simple) 
 app, so how might the NullPointerException lead to 
 RejectedExecutionException?  


 On Friday, July 17, 2015 at 1:17:47 PM UTC-4, sugarste...@gmail.com wrote:

 My app starts up, runs for a few minutes, and then: 

 #RejectedExecutionException 
 java.util.concurrent.RejectedExecutionException: Task 
 java.util.concurrent.FutureTask@4fe887e6 rejected from 
 java.util.concurrent.ThreadPoolExecutor@aea107f[Terminated, pool size = 0, 
 active threads = 0, queued tasks = 0, completed tasks = 3]

 This is a very simple app, and I am not aware of using any concurrency. I 
 am not using Futures or Promises or pmap or anything else I can think of. 
 My first hunch would be this comes from some library I am using, but I am 
 not sure what. I searched on Google but none of the reported problems seem 
 to apply to me. This is what I am using, do any of these seem likely to 
 throw this error? 

 [org.clojure/clojure 1.6.0]
  [com.taoensso/timbre 4.0.2]
  [dire 0.5.1]
  [slingshot 0.12.2]
  [org.clojure/data.json 0.2.5]
  [org.clojure/tools.namespace 0.2.4]
  [me.raynes/fs 1.4.4]
  [clj-stacktrace 0.2.8]
  [overtone/at-at 1.2.0]
  [org.clojure/core.cache 0.6.4]
  [cheshire 5.5.0]
  [com.taoensso/carmine 2.11.1]






-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Opinion on core.async vs callbacks in abstract APIs?

[Zach Tellman]

The problem with using bare callbacks is that there's no way for the 
invoked callback to exert backpressure, except by blocking or passing in a 
CPS-style callback to the callback, neither of which is ideal.  If you're 
looking for a neutral choice, I'd suggest an infinite lazy-seq over 
callbacks.  If you absolutely need it to be async, something like Manifold 
might be warranted, in that it allows people to consume it however they 
like (though they will have to be at least a little familiar with Manifold 
to do so).

Zach

On Monday, June 1, 2015 at 12:18:19 PM UTC-7, Christopher Small wrote:

 Greetings

 I imagine most of us here would rather use core.async channels over 
 callbacks in their application code, particularly with more complicated 
 applications. But is it okay/preferable for Clojure libraries to force 
 their users to use core.async channels as part of an API (an event channel, 
 for example)? 

 As much as I love core.async, I can't help but wonder whether sticking 
 with callbacks for an API isn't a simpler/better design strategy. It's easy 
 enough to drop messages on a channel in a callback, and this let's users 
 opt-in. But if one expects core.async channels are what most would prefer 
 anyway, is it okay to foist them upon everyone?

 As a follow up, does your opinion on the matter change if implementations 
 of an API become simpler using core.async channels?


 Looking forward to your thoughts :-)

 Chris Small



 PS I'm asking because I'm working on a physical computing API (
 https://github.com/clj-bots/pin-ctrl) and debating between using channels 
 vs callbacks for the edge detection functionality (if you're not familiar, 
 edge detection let's you asynchronously handle changes in pin state, such 
 as button pushes). If you're interested in this question as it applies 
 specifically to this application, feel free to join the discussion on our 
 gitter channel: https://gitter.im/clj-bots/chat


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] data.int-map 0.2.0

[Zach Tellman]

https://github.com/clojure/data.int-map

The 0.1.0 release of this library was a faithful Clojure implementation of 
Okasaki's Fast Mergeable Integer Maps paper [1].  While it was much 
faster than Clojure's sorted-maps, it was in some cases slower than 
hash-maps, which was mostly due to the int-maps being a binary tree, and 
hash-maps being a 32-ary tree.  It also could only take non-negative 
integers, which was a limitation of the approach used in the paper.  In the 
0.2.0 release, I reimplemented the core data structure in Java, but more 
importantly adapted the data structure to use 16-ary trees under the 
covers.  This has greatly improved performance (by a factor of 2-5x, 
depending on the benchmark), and has also allowed the map to have negative 
integers for keys.

I recommend anyone using an earlier version of this library upgrade to the 
latest, which will be a drop-in replacement.

Zach

[1] http://ittc.ku.edu/~andygill/papers/IntMap98.pdf

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] lein-jammin

[Zach Tellman]

https://github.com/ztellman/lein-jammin

This one's pretty simple: you put `lein jammin seconds` in front of any 
other Leiningen task, and if it gets stuck for the specified duration, it 
prints out a thread dump.  This especially useful for tests, and 
extra-especially useful for tests in a CI environment where using `jstack` 
or similar tools is difficult or impossible.

Enjoy,
Zach

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN} Aleph 0.4.0 released, plus Manifold, Dirigiste, and a whole host of other libraries

[Zach Tellman]

I suspect it would, I think I was just letting the mechanics of Manifold's
let-flow macro color my judgment. Happy to accept any pull requests which
make my core.async examples more idiomatic.
On Apr 18, 2015 8:33 AM, Matthias Lange matthias.la...@gmail.com wrote:

 In your examples, you put a let around the reads from timeouts.

 (let [_ (a/! (a/timeout 1000))] ... )

 As far as i know, that is not neccessary. So your first example could be:

 (defn delayed-hello-world-handler
   [req]
   (d/-deferred
 (a/go
   (a/! (a/timeout 1000))
   (hello-world-handler req

 Would that not work?

  --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to a topic in the
 Google Groups Clojure group.
 To unsubscribe from this topic, visit
 https://groups.google.com/d/topic/clojure/KH8Js8URKwA/unsubscribe.
 To unsubscribe from this group and all its topics, send an email to
 clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN} Aleph 0.4.0 released, plus Manifold, Dirigiste, and a whole host of other libraries

[Zach Tellman]

Hey all,

In preparation for Clojure/West, I'm formally releasing the latest Aleph 
and the libraries that surround it.  Aleph 0.4.0 has been running in 
production at Factual for half a year now, and across a variety of services 
is handling at peak 600k HTTP requests/sec (spread across 15-20 machines).  

Since the landscape of Clojure HTTP servers is pretty crowded these days, 
it's worth taking some time to explain how Aleph differs.  To be clear, 
most Clojure deployments likely use Jetty, and should continue to do so. 
 However, Aleph has some unique properties:

* It uses the Netty library, which is a high-performance and very 
battle-tested network layer for the JVM
* It is the only HTTP server that has *ubiquitous* asynchronous streams 
wherever data can be received or sent (all other libraries can only 
represent streaming requests using InputStreams, or like http-kit don't 
support streaming HTTP requests at all)
* It is the only server that has a WebSocket implementation with any 
support for per-connection backpressure.  I won't make this post even 
longer by going into why this is important, but this will be a central 
theme of my talk at Clojure/West next week if you're interested in hearing 
more.
* It uses consistent abstractions to represent network connections over a 
variety of protocols, which makes it straightforward to use the same 
application logic for all of them.

Again, none of these points mean you should immediately drop whatever 
you're using and move over to Aleph instead.  However, I do feel it 
represents the only (current) good option for using core.async or a similar 
stream abstraction to represent network data, which is an idea a number of 
people seem to be playing with lately.  Some examples of this can be found 
at http://ideolalia.com/aleph/literate.html.

A full list of the libraries:

aleph - https://github.com/ztellman/aleph - uses the excellent Netty 
library to expose HTTP, TCP, and UDP using a consistent asynchronous stream 
representation.

manifold - https://github.com/ztellman/manifold - an unopinionated stream 
representation designed to cleanly interoperate with other stream 
representations (Clojure's seqs, core.async channels, Java's 
BlockingQueues, and others).  This is the base stream representation for 
all network sources and sinks in Aleph.

dirigiste -  https://github.com/ztellman/dirigiste - a pure-Java library 
that provides instrumented, dynamically sized thread and object pools. 
 This is used for thread pools in Aleph's HTTP server, and for connection 
pools in Aleph's HTTP client.

byte-streams -  https://github.com/ztellman/byte-streams - a means of 
translating any byte representation into another.  Want to turn a 
core.async channel that emits byte-arrays into an InputStream, or maybe the 
other way around?  Look no further.  The library's conversion mechanism is 
extensible, which is used in Aleph to make Netty's custom byte 
representations interoperable with more familiar representations.

byte-transforms -  https://github.com/ztellman/byte-transforms - a curated 
collection of byte compression, hashing, and encoding mechanisms, which can 
work on anything byte-streams can convert.

While all these libraries are used in concert to create Aleph, I've been 
very careful to make sure any of them can be used by themselves.  If anyone 
has questions about them, the best place to get my attention is the Aleph 
mailing list: https://groups.google.com/forum/#!forum/aleph-lib.

I will be mentioning some of these libraries at my upcoming Clojure/West 
talk (http://clojurewest.org/speakers#ztellman), but I've also set aside an 
Unsession for specifically discussing these 
libraries: https://github.com/clojurewest/clojurewest2015/wiki/Unsessions. 
 If you're interested, please add your name to the list.

Zach

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN} Aleph 0.4.0 released, plus Manifold, Dirigiste, and a whole host of other libraries

[Zach Tellman]

Hey Dmitri,

I haven't used any sort of dev-mode before (I just update stuff in the REPL
when necessary), but it seems like something like that belongs in
middleware, not the server.  The server is just calling a function, it
shouldn't care if something else is changing that function's behavior.

If other servers have this behavior embedded, maybe it can be extracted
into a standalone library?

Zach

On Fri, Apr 17, 2015 at 3:47 PM, Dmitri dmitri.sotni...@gmail.com wrote:

 I'd like to add Aleph to the Luminus template and I was wondering if
 there's an equivalent of dev mode available for other servers where it
 watches for changes in source and reloads them. I did a cursory look but
 didn't spot anything like a -dev option.

 On Friday, April 17, 2015 at 5:06:30 PM UTC-4, Zach Tellman wrote:

 Hey all,

 In preparation for Clojure/West, I'm formally releasing the latest Aleph
 and the libraries that surround it.  Aleph 0.4.0 has been running in
 production at Factual for half a year now, and across a variety of services
 is handling at peak 600k HTTP requests/sec (spread across 15-20 machines).

 Since the landscape of Clojure HTTP servers is pretty crowded these days,
 it's worth taking some time to explain how Aleph differs.  To be clear,
 most Clojure deployments likely use Jetty, and should continue to do so.
 However, Aleph has some unique properties:

 * It uses the Netty library, which is a high-performance and very
 battle-tested network layer for the JVM
 * It is the only HTTP server that has *ubiquitous* asynchronous streams
 wherever data can be received or sent (all other libraries can only
 represent streaming requests using InputStreams, or like http-kit don't
 support streaming HTTP requests at all)
 * It is the only server that has a WebSocket implementation with any
 support for per-connection backpressure.  I won't make this post even
 longer by going into why this is important, but this will be a central
 theme of my talk at Clojure/West next week if you're interested in hearing
 more.
 * It uses consistent abstractions to represent network connections over a
 variety of protocols, which makes it straightforward to use the same
 application logic for all of them.

 Again, none of these points mean you should immediately drop whatever
 you're using and move over to Aleph instead.  However, I do feel it
 represents the only (current) good option for using core.async or a similar
 stream abstraction to represent network data, which is an idea a number of
 people seem to be playing with lately.  Some examples of this can be found
 at http://ideolalia.com/aleph/literate.html.

 A full list of the libraries:

 aleph - https://github.com/ztellman/aleph - uses the excellent Netty
 library to expose HTTP, TCP, and UDP using a consistent asynchronous stream
 representation.

 manifold - https://github.com/ztellman/manifold - an unopinionated
 stream representation designed to cleanly interoperate with other stream
 representations (Clojure's seqs, core.async channels, Java's
 BlockingQueues, and others).  This is the base stream representation for
 all network sources and sinks in Aleph.

 dirigiste -  https://github.com/ztellman/dirigiste - a pure-Java library
 that provides instrumented, dynamically sized thread and object pools.
 This is used for thread pools in Aleph's HTTP server, and for connection
 pools in Aleph's HTTP client.

 byte-streams -  https://github.com/ztellman/byte-streams - a means of
 translating any byte representation into another.  Want to turn a
 core.async channel that emits byte-arrays into an InputStream, or maybe the
 other way around?  Look no further.  The library's conversion mechanism is
 extensible, which is used in Aleph to make Netty's custom byte
 representations interoperable with more familiar representations.

 byte-transforms -  https://github.com/ztellman/byte-transforms - a
 curated collection of byte compression, hashing, and encoding mechanisms,
 which can work on anything byte-streams can convert.

 While all these libraries are used in concert to create Aleph, I've been
 very careful to make sure any of them can be used by themselves.  If anyone
 has questions about them, the best place to get my attention is the Aleph
 mailing list: https://groups.google.com/forum/#!forum/aleph-lib.

 I will be mentioning some of these libraries at my upcoming Clojure/West
 talk (http://clojurewest.org/speakers#ztellman), but I've also set aside
 an Unsession for specifically discussing these libraries:
 https://github.com/clojurewest/clojurewest2015/wiki/Unsessions.  If
 you're interested, please add your name to the list.

 Zach

  --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com

Re: Use core.async executor with aleph

[Zach Tellman]

Hi Robin,

You can absolutely specify an executor of :none if you're sure you won't be 
doing any blocking in your request handler.  If everything's wrapped by a 
go-block, that's certainly the case, and is probably the most efficient 
approach.  However, Aleph just needs some java.util.concurrent.Executor, so 
using the core.async executor is also a valid approach.

By the way, I'm more likely to notice these sorts of questions if you ask 
them on the Aleph mailing 
list: https://groups.google.com/forum/#!forum/aleph-lib

Zach

On Tuesday, February 17, 2015 at 1:35:53 AM UTC-8, Robin Heggelund Hansen 
wrote:

 From what I can see, aleph allows me to set a executor to handle client 
 requests. I'm already using core.async pretty heavily. Is there any reason 
 why I shouldn't pass core.async's executor to aleph? I see I can also make 
 every client request start on aleph's dispatch thread. Considering 
 absolutely every request spawns a go-block, might it even be a good idea to 
 not run aleph with an executor at all?

 Thanks!


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Newbie Gloss questions - dynamic buffer structure

[Zach Tellman]

I'm sorry, that should have been :ubyte. Being able to insert arbitrary
data into a codec is a feature, though.

On Mon, Jan 12, 2015 at 6:27 AM, Tzach tzach.livya...@gmail.com wrote:

 Hi Zach
 Thanks for the detailed response

 On Wednesday, January 7, 2015 at 7:28:06 AM UTC+2, Zach Tellman wrote:

 Hey Tzach,

 If I understand what you're trying to do, you want something like this:

 [ :uint32
   :ubyte ;; or bit-seq, depending
   (delimited-block (prefix uint24 #(- % 5) #(+ % 5))) ]

 And then you'll need to parse the final block separately.  I've left
 defining the uint24 as an exercise for the reader (sorry, long day), but
 using (compile-frame [:uint16 :uint8] ...) should be pretty straightforward.

 This is what work for me, in case anyone is interested or want to comment

 (defn valid-length [l]
  (and ( l 16777216)
   (= l 0)))

 (def uint24 (compile-frame [:ubyte :uint16]
(fn [u]
  {:pre  [(valid-length u)]}
  (let [u8 (bit-shift-right u 16)
u16 (- u (bit-shift-left u8 16))]
[u8 u16]))
(fn [[u8 u16]]
  {:post [(valid-length %)]}
  (+ (bit-shift-left u8 16) u16

 (def avp
   (compile-frame
[:uint32
 :ubyte
 (repeated :int32 :prefix (prefix uint24 #(- % 5) #(+ % 5))) ]
 ))

 My next step is to parse the bits from the header :ubyte and base on it,
 read an extra uint32 before the repeated part.
 The header is probably the answer.



 Hope that helps, happy to answer any followup questions.


 It did help! Thanks

 BTW, I spent too much time trying to use (compile-frame [:uint16 :uint8])
 before realizing :uint8 is not defined.
 The result is simply encoding of the :uint8 symbol.
 I wonder if its a bug or a feature.




 Zach

 On Saturday, January 3, 2015 11:51:14 PM UTC-8, Tzach wrote:

 I'm trying to work with Gloss binary encoder/decoder, and need some help
 to kick start.

 My first task is simple(not for me ;)
 I have the following binary buffer to read/write:

- 4 byte (32 bit) - code (uint)
- 8 bit - misc flags
- 3 byte (24 bit) - the entire buffer length
- 4 byte (32 bit) uint value -  optional, depending on on of the
flags.
- List of 4 byte (uints) data elements - size dependent on the
overall size

 How should I represent this structure?
 Clearly I need to use *prefix* and *repeated* for the last element, but
 I failed to understand how.
 I also struggle with the optional element and how to represent a 3 byte
 element.

 Thanks!

  --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to a topic in the
 Google Groups Clojure group.
 To unsubscribe from this topic, visit
 https://groups.google.com/d/topic/clojure/NxsCPBk12mE/unsubscribe.
 To unsubscribe from this group and all its topics, send an email to
 clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: A (foolish) plan to re-invent IO on top of core.async

[Zach Tellman]

Aleph has always allowed that, but the latest release allows the asynchrony 
to be modeled using core.async channels if that's to your taste.

On Wednesday, January 7, 2015 at 9:52:53 AM UTC-8, Paul deGrandis wrote:

 There was another discussion on this list regarding async IO and web 
 servers.  It may be rather informative to you: 
 https://groups.google.com/d/msg/clojure/rKqT13Ofy4k/H9xvkZA9Yy4J

 To my knowledge, Pedestal is the only web library that let's you go async 
 all the way down to the wire (potentially the latest Aleph also allows for 
 this).  The benefit provided is that Pedestal manages the NIO integration 
 directly with the container for you - optimized to the specific container.  
 You may thumb through the implementation for some ideas.

 You also have to be very mindful about back-pressure when using core.async 
 in certain combinations.  Zach Tellman has covered the major points here: 
 https://groups.google.com/d/msg/clojure/TVMQJwaij1U/dQxyBxxbIjQJ

 Cheers,
 Paul



-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Newbie Gloss questions - dynamic buffer structure

[Zach Tellman]

Hey Tzach,

If I understand what you're trying to do, you want something like this:

[ :uint32
  :ubyte ;; or bit-seq, depending
  (delimited-block (prefix uint24 #(- % 5) #(+ % 5))) ]

And then you'll need to parse the final block separately.  I've left 
defining the uint24 as an exercise for the reader (sorry, long day), but 
using (compile-frame [:uint16 :uint8] ...) should be pretty straightforward.

Hope that helps, happy to answer any followup questions.

Zach

On Saturday, January 3, 2015 11:51:14 PM UTC-8, Tzach wrote:

 I'm trying to work with Gloss binary encoder/decoder, and need some help 
 to kick start.

 My first task is simple(not for me ;)
 I have the following binary buffer to read/write:

- 4 byte (32 bit) - code (uint)
- 8 bit - misc flags
- 3 byte (24 bit) - the entire buffer length
- 4 byte (32 bit) uint value -  optional, depending on on of the flags.
- List of 4 byte (uints) data elements - size dependent on the overall 
size 

 How should I represent this structure?
 Clearly I need to use *prefix* and *repeated* for the last element, but I 
 failed to understand how.
 I also struggle with the optional element and how to represent a 3 byte 
 element.
  
 Thanks!


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Server Sent Events under Http-Kit

[Zach Tellman]

Does `send!` block if it needs to exert backpressure?  If so, the `go-loop`
seems dangerous.  If not, how does backpressure work?

On Wed, Dec 17, 2014 at 8:07 AM, Malcolm Sparks malc...@juxt.pro wrote:

 Here is a solution for SSE over http-kit channels

 (defn server-event-source [ch] ;; Adding a mult here is dangerous because
 it bleeds the underlying ;; channel dry. We should provide the mult via
 modular.async (let [m (async/mult ch)] (fn [req] (let [ch (async/chan 16)]
 (async/tap m ch) (with-channel req net-ch (on-close net-ch (fn [_]
 (async/untap m ch) (async/close! ch))) (send! net-ch {:headers headers}
 false) (go-loop [] (when-let [data (! ch)] (println Got data!  data)
 (send! net-ch (-message data) false) (recur


 https://github.com/juxt/modular/blob/master/modules/http-kit-events/src/modular/http_kit/events.clj


 On Saturday, 6 December 2014 17:03:41 UTC, Lars Ole Avery Simonsen wrote:

 Hi guys

 I am trying to implement Server Sent Event support in a small hobby
 project based on the http-kit server framework and compojure.

 I am still quite new to clojure in general, so it may very well be that I
 am missing something obvious, but this SSE detail has me stumped.

 What I have tried is to take inspiration from the Eventual
 https://github.com/ninjudd/eventual library for the SSE implementation
 (which is aimed at jetty). I have changed the Eventual implementation so
 that it uses the http-kit async-channel implementation for data
 transmission. What I am seeing is that everything seems to run without
 errors except for the fact that no data is being received at the
 EventSource in the browser.

 If I allow the sse channel to close immediately after opening (which is
 pretty useless), the initial message is received at the browser side, but
 if I keep the connection open, nothing gets received.

 Having had a look at the http-kit internals, my suspicion is that the
 socket doesn't get flushed and so all my attempts at communication are
 sitting in some buffer somewhere.

 All SSE implementations I have looked at for other platforms have
 explicit calls to flush the socket buffers on the long lived connection
 after each message transmission.

 Does anyone have any suggestions as to a better way to achieve SSE
 support in a http-kit based server? Or maybe ideas as to what I might be
 doing wrong?

 Thanks in advance.

 LOAS

  --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to a topic in the
 Google Groups Clojure group.
 To unsubscribe from this topic, visit
 https://groups.google.com/d/topic/clojure/DbBzM3AG9wM/unsubscribe.
 To unsubscribe from this group and all its topics, send an email to
 clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Server Sent Events under Http-Kit

[Zach Tellman]

In Aleph, you simply return the channel as the body of the response [1],
and messages are consumed only as they're able to be sent over the
network.  This means that backpressure just works without any need to
poll whether downstream buffers are full.  Obviously you can poll using the
put-with-timeout mechanism in core.async et al if you want, though.

[1] https://gist.github.com/ztellman/613629a2a32908d096d5

On Wed, Dec 17, 2014 at 10:42 AM, Malcolm Sparks malc...@juxt.pro wrote:

 http-kit's send! doesn't block normally -

 From http://www.http-kit.org/server.html :-

 send!: Sends data to client and returns true if the data was successfully
 written to the output queue, or false if the channel is closed. Normally,
 checking the returned value is not needed. This function returns
 immediately (does not block).

 http-kit will attempt to write to the output queue, otherwise will place
 the message in a pending queue backed by a
 java.util.LinkedListByteBuffer, so there's no back-pressure (except for
 that exerted by an OutOfMemory exception)

 AFAICT there's no way of finding whether http-kit's output channel is
 empty or not, that would certainly be a feature I'd like to see.

 Does aleph support this?




 On 17 December 2014 at 18:23, Zach Tellman ztell...@gmail.com wrote:

 Does `send!` block if it needs to exert backpressure?  If so, the
 `go-loop` seems dangerous.  If not, how does backpressure work?

 On Wed, Dec 17, 2014 at 8:07 AM, Malcolm Sparks malc...@juxt.pro wrote:

 Here is a solution for SSE over http-kit channels

 (defn server-event-source [ch] ;; Adding a mult here is dangerous
 because it bleeds the underlying ;; channel dry. We should provide the
 mult via modular.async (let [m (async/mult ch)] (fn [req] (let [ch
 (async/chan 16)] (async/tap m ch) (with-channel req net-ch (on-close
 net-ch (fn [_] (async/untap m ch) (async/close! ch))) (send! net-ch {
 :headers headers} false) (go-loop [] (when-let [data (! ch)] (println Got
 data!  data) (send! net-ch (-message data) false) (recur


 https://github.com/juxt/modular/blob/master/modules/http-kit-events/src/modular/http_kit/events.clj


 On Saturday, 6 December 2014 17:03:41 UTC, Lars Ole Avery Simonsen wrote:

 Hi guys

 I am trying to implement Server Sent Event support in a small hobby
 project based on the http-kit server framework and compojure.

 I am still quite new to clojure in general, so it may very well be that
 I am missing something obvious, but this SSE detail has me stumped.

 What I have tried is to take inspiration from the Eventual
 https://github.com/ninjudd/eventual library for the SSE
 implementation (which is aimed at jetty). I have changed the Eventual
 implementation so that it uses the http-kit async-channel implementation
 for data transmission. What I am seeing is that everything seems to run
 without errors except for the fact that no data is being received at the
 EventSource in the browser.

 If I allow the sse channel to close immediately after opening (which is
 pretty useless), the initial message is received at the browser side, but
 if I keep the connection open, nothing gets received.

 Having had a look at the http-kit internals, my suspicion is that the
 socket doesn't get flushed and so all my attempts at communication are
 sitting in some buffer somewhere.

 All SSE implementations I have looked at for other platforms have
 explicit calls to flush the socket buffers on the long lived connection
 after each message transmission.

 Does anyone have any suggestions as to a better way to achieve SSE
 support in a http-kit based server? Or maybe ideas as to what I might be
 doing wrong?

 Thanks in advance.

 LOAS

  --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to a topic in the
 Google Groups Clojure group.
 To unsubscribe from this topic, visit
 https://groups.google.com/d/topic/clojure/DbBzM3AG9wM/unsubscribe.
 To unsubscribe from this group and all its topics, send an email to
 clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.

  --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to a topic

Re: Server Sent Events under Http-Kit

[Zach Tellman]

I'm not sure about http-kit's streaming response implementation, but in 
Aleph [1] if you use a stream/channel as a response body each message from 
the stream will be immediately sent as an HTTP chunk.

Best,
Zach

[1] https://github.com/ztellman/aleph

On Saturday, December 6, 2014 9:03:41 AM UTC-8, Lars Ole Avery Simonsen 
wrote:

 Hi guys

 I am trying to implement Server Sent Event support in a small hobby 
 project based on the http-kit server framework and compojure.

 I am still quite new to clojure in general, so it may very well be that I 
 am missing something obvious, but this SSE detail has me stumped.

 What I have tried is to take inspiration from the Eventual 
 https://github.com/ninjudd/eventual library for the SSE implementation 
 (which is aimed at jetty). I have changed the Eventual implementation so 
 that it uses the http-kit async-channel implementation for data 
 transmission. What I am seeing is that everything seems to run without 
 errors except for the fact that no data is being received at the 
 EventSource in the browser.

 If I allow the sse channel to close immediately after opening (which is 
 pretty useless), the initial message is received at the browser side, but 
 if I keep the connection open, nothing gets received.

 Having had a look at the http-kit internals, my suspicion is that the 
 socket doesn't get flushed and so all my attempts at communication are 
 sitting in some buffer somewhere.

 All SSE implementations I have looked at for other platforms have explicit 
 calls to flush the socket buffers on the long lived connection after each 
 message transmission.

 Does anyone have any suggestions as to a better way to achieve SSE support 
 in a http-kit based server? Or maybe ideas as to what I might be doing 
 wrong?

 Thanks in advance.

 LOAS



-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] s3-journal, a library for high-throughput log journaling

[Zach Tellman]

Today Factual is open sourcing two libraries, the first of which is a 
relatively simple library that journals data to Amazon's S3 service [1], 
and buffers data on disk using durable-queue [2], making it much more 
robust than most S3 clients to process failure, network issues, and a host 
of other issues.  We have used it to write over a trillion entries this 
year, and expect it will become equally indispensable for others in the 
Clojure community.

A blog post describing it in more detail can be found at 
http://blog.factual.com/how-factual-uses-persistent-storage-for-its-real-time-services.
 
 The library itself can be found at https://github.com/factual/s3-journal.

Zach 

[1] http://aws.amazon.com/s3/
[2] https://github.com/factual/durable-queue

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] riffle, a high-performance write-once key/value storage engine

[Zach Tellman]

The second of the libraries Factual is open sourcing today is Riffle, which 
is a write-once key/value storage engine.  By disallowing random writes, it 
allows for O(1) on-disk reads, and a read throughput is both faster and 
significantly more stable than LevelDB and other, similar databases.  Of 
course, any changes to the database require merging two existing databases 
or writing another one from scratch, which can be done either in-process or 
using Hadoop.  

A blog post describing the library can be found 
at 
http://blog.factual.com/how-factual-uses-persistent-storage-for-its-real-time-services,
 
and the library itself can be found at https://github.com/factual/riffle. 
 We look forward to see how it's used by others.

Zach

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

updated web server benchmarks

[Zach Tellman]

New benchmarks have been run for a variety of Clojure web servers, and the 
results can be found 
at 
https://github.com/ptaoussanis/clojure-web-server-benchmarks#ubuntu-1404--2x-xeon-x5650.
 
 The top-performing servers are based on Undertow (ring-undertow, Immutant 
v2), Netty (Aleph), and nginx (nginx-clojure) respectively.

The benchmark hardware was dual-socket 2010-era Xeons, which is the 
hardware profile of choice at Factual.  This means that there were 24 
hardware threads, but each core was noticeably slower than those on modern 
desktop processors.  This caused http-kit, which only uses a single thread 
for I/O, to perform worse than it would have on a system with fewer, faster 
cores.  

As always, these benchmarks are very contrived, and not representative of 
most real-world applications.  Adding something as simple as Ring's 
`wrap-keyword-params` middleware on Clojure 1.6.0 would reduce throughput 
on all servers by a factor of 3-4x.  Luckily keyword creation will be much 
cheaper in Clojure 1.7.0, so this particular problem won't exist for much 
longer, but it's important to remember that performance is a property of 
your entire application, not just the underlying web server.

With that said, it's great to see that so many people are taking server 
performance in Clojure seriously.  However contrived these benchmarks are, 
a lot of people use them as a proxy for a language's maturity, and 
further effort in this space can only help Clojure's adoption.

Zach

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] async-sockets - work with sockets using core.async channels

[Zach Tellman]

A slightly more straightforward implementation can be found 
at https://gist.github.com/ztellman/fb64e81d1d7f0b261ccd.  I'm fairly sure 
it's equivalent to Cristophe's, but I may be missing some nuance.  At any 
rate, I've found using the async/put! and callback mechanism to be a much 
more straightforward way to do interop with non-core.async code.

And yes, reimplementing a TCP-like ack mechanism on top of WebSockets is 
not something you want to do.  The existing stack will do it better and 
faster than you can.  Just to be clear, this is a large part of why I wrote 
Manifold [1], which can easily be turned into a core.async channel, but 
provides an API which is designed for interop with other stream mechanisms 
(including synchronous ones like Java's BlockingQueues).  core.async is a 
framework, meaning it brings not only a stream representation, but an 
entire execution model; using core.async should be an application-level 
decision, not one made for you by your libraries

Zach

[1] https://github.com/ztellman/manifold


On Sunday, October 12, 2014 9:42:32 AM UTC-7, Ryan Waters wrote:

 I was just starting to use Sente [1] (which relies on httpkit [2]) and 
 this conversation is a real eye opener.  Unless a person uses a library 
 that supports backpressure, as mentioned earlier, your 
 transport-concern-made-opaque-because-of-core-async must become an 
 application-level concern.  The far-side of the communication would have to 
 respond with an application level acknowledgement for local sends and the 
 local side would need to not send data unless acks were received for 
 previously sent data.

 E.g. this could be implemented with core.async by using a pair of channels 
 (instead of a single channel) for all 'sends' where one channel is used for 
 data while the other channel waits for acknowledgement of put data (a 
 'control' channel).  This would have the unfortunate side effect of hurting 
 throughput.  A better system would be to allow for a certain number of 
 unacknowledged sends before backing off.  Of course, now a person is 
 implementing what was created for TCP at the level of their application.

 Christophe's approach means you at least wouldn't have to do the above, 
 replacing it instead with a per backend implementation.  I hope somebody 
 else is able to explain it better.

 Looking forward to an Aleph rewrite!!

 - - -
 [1] https://github.com/ptaoussanis/sente
 [2] https://github.com/http-kit/http-kit

 On Sat, Oct 11, 2014 at 8:01 PM, Julian julian...@gmail.com javascript:
  wrote:

 Hi Zach, 

 Thanks for the clarity of thought that went into this post. 

 Perhaps it is obvious to everyone but me, but I saw this post by 
 Christophe Grande yesterday that appears to address these concerns:
 Back-pressurized interop for core.async 
 https://twitter.com/cgrand/status/520566182194450432
 https://gist.github.com/cgrand/767673242b7f7c27f35a

 I'm interested to hear if this solves your problem or is about something 
 else. 

 Cheers
 Julian


 On Wednesday, 8 October 2014 17:00:02 UTC+11, Zach Tellman wrote:

 The reason the thread-per-connection approach is nice is because it 
 correctly propagates backpressure.  If we're copying data from a source to 
 a sink (let's say reading it in from the network and writing to a file), 
 it's possible that the production of data may outstrip the consumption.  If 
 this happens, we need to make sure the producer slows down, or we risk 
 running out of memory.  In Java, the producer is typically connected to the 
 consumer via a blocking queue, and if the queue fills up the producer can't 
 send anything more to the consumer.  A Java socket is one such queue, and 
 if it fills up it will exert backpressure via TCP.  This will work no 
 matter how many queues or other mechanisms separate the producer and 
 consumer.

 However, every attempt I've seen to marry core.async to an async network 
 stack has been fundamentally broken, in that it doesn't do this.  Often, 
 they'll just use 'put!', which works fine until the channel's queue fills 
 up, and 1024 pending puts are accumulated, and finally the channel throws 
 an exception.  Alternately, they'll use a blocking put on the channel, 
 which means that any backpressure will also extend to whatever other 
 connections are sharing that thread or the thread pool.  Note that the 
 software that uses core.async in this way may work flawlessly in a wide 
 variety of cases, but there's still an intractable failure mode lying in 
 wait.

 In some cases, such as http-kit's websocket mechanism, there's no way to 
 even exert backpressure (you register a callback, and have no way to 
 indicate in your callback that you can't handle more messages).  This means 
 that any attempt to use http-kit in conjunction with core.async will be 
 subtly but fundamentally broken.  Arguably, even without core.async in the 
 equation it's broken.  This is not a good state of affairs.  I'll admit 
 that it took me a few

Re: [ANN] async-sockets - work with sockets using core.async channels

[Zach Tellman]

The reason the thread-per-connection approach is nice is because it 
correctly propagates backpressure.  If we're copying data from a source to 
a sink (let's say reading it in from the network and writing to a file), 
it's possible that the production of data may outstrip the consumption.  If 
this happens, we need to make sure the producer slows down, or we risk 
running out of memory.  In Java, the producer is typically connected to the 
consumer via a blocking queue, and if the queue fills up the producer can't 
send anything more to the consumer.  A Java socket is one such queue, and 
if it fills up it will exert backpressure via TCP.  This will work no 
matter how many queues or other mechanisms separate the producer and 
consumer.

However, every attempt I've seen to marry core.async to an async network 
stack has been fundamentally broken, in that it doesn't do this.  Often, 
they'll just use 'put!', which works fine until the channel's queue fills 
up, and 1024 pending puts are accumulated, and finally the channel throws 
an exception.  Alternately, they'll use a blocking put on the channel, 
which means that any backpressure will also extend to whatever other 
connections are sharing that thread or the thread pool.  Note that the 
software that uses core.async in this way may work flawlessly in a wide 
variety of cases, but there's still an intractable failure mode lying in 
wait.

In some cases, such as http-kit's websocket mechanism, there's no way to 
even exert backpressure (you register a callback, and have no way to 
indicate in your callback that you can't handle more messages).  This means 
that any attempt to use http-kit in conjunction with core.async will be 
subtly but fundamentally broken.  Arguably, even without core.async in the 
equation it's broken.  This is not a good state of affairs.  I'll admit 
that it took me a few failures in production to realize how important 
correct handling of backpressure is, but this isn't something that our 
ecosystem can afford to ignore, especially as Clojure is used for 
larger-scale projects.

I will note that I am working on a solution to this, in the form of the 
upcoming Aleph release [1].  This will model every network connection via 
streams that can trivially be converted into core.async channels [2], and 
which exert backpressure over TCP wherever necessary without requiring a 
thread per connection.  A formal beta should be available in the near 
future (it's already handling billions of requests a day in production 
without issue).

Zach

[1] https://github.com/ztellman/aleph/tree/0.4.0
[2] https://github.com/ztellman/manifold



On Tuesday, October 7, 2014 1:36:16 PM UTC-7, adrian...@mail.yu.edu wrote:

 It's not about 'safety' (depending on what that means in this context), 
 but as Zach pointed out, if you aren't careful about backpressure you can 
 run into performance bottlenecks with unrestrained async IO operations 
 because although they let you code as if you could handle an unlimited 
 amount of connections, obviously that isn't true. There is only a finite 
 amount of data that can be buffered in and out of any network according to 
 its hardware. When you don't regulate that, your system will end up 
 spending an inordinate amount of time compensating for this. You don't need 
 to worry about this with regular io because the thread per connection 
 abstraction effectively bounds your activity within the acceptable physical 
 constraints of the server. 

 On Tuesday, October 7, 2014 2:49:30 PM UTC-4, Brian Guthrie wrote:


 On Mon, Oct 6, 2014 at 12:10 AM, adrian...@mail.yu.edu wrote:

 Zach makes an excellent point; I've used AsyncSocketChannels and its irk 
 (
 http://docs.oracle.com/javase/8/docs/api/java/nio/channels/AsynchronousServerSocketChannel.html),
  
 with core.async in the past. Perhaps replacing your direct java.net.Sockets 
 with nio classes that can be given CompletionHandlers (
 http://docs.oracle.com/javase/7/docs/api/java/nio/channels/CompletionHandler.html)
  
 would be a better fit. 


 Once I do some performance instrumentation I'll give that a shot. I admit 
 that I'm not familiar with all the implications of using the nio classes; 
 were I to switch, is it safe to continue using go blocks, or is it worth 
 explicitly allocating a single thread per socket?

 Brian



-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, 

Re: [ANN] async-sockets - work with sockets using core.async channels

[Zach Tellman]

I wasn't aware of hermod, that's interesting.  I would still characterize
its approach to backpressure as broken, though, since when the queues get
full it silently drops messages on the ground.  In fairness, this is very
clearly documented, so it's less pernicious than some of the other cases
out there.

Both core.async buffers and SelectableChannels have very particular
semantics, and I would be very surprised if they could be combined in that
way.  It's perfectly possible to feed one into the other and handle
backpressure properly (again, I'm doing just that with Aleph 0.4.0, using
Netty), but it's a nuanced integration and easy to get wrong.

On Wed, Oct 8, 2014 at 7:12 AM, adrian.med...@mail.yu.edu wrote:

 Check out https://github.com/halgari/com.tbaldridge.hermod for an
 interesting take on this.

 On Wednesday, October 8, 2014 1:17:11 AM UTC-4, Sun Ning wrote:

  BTW, is there any network based core.async channel available now?

 On 10/08/2014 04:36 AM, adrian...@mail.yu.edu wrote:

  It's not about 'safety' (depending on what that means in this context),
 but as Zach pointed out, if you aren't careful about backpressure you can
 run into performance bottlenecks with unrestrained async IO operations
 because although they let you code as if you could handle an unlimited
 amount of connections, obviously that isn't true. There is only a finite
 amount of data that can be buffered in and out of any network according to
 its hardware. When you don't regulate that, your system will end up
 spending an inordinate amount of time compensating for this. You don't need
 to worry about this with regular io because the thread per connection
 abstraction effectively bounds your activity within the acceptable physical
 constraints of the server.

 On Tuesday, October 7, 2014 2:49:30 PM UTC-4, Brian Guthrie wrote:


 On Mon, Oct 6, 2014 at 12:10 AM, adrian...@mail.yu.edu wrote:

 Zach makes an excellent point; I've used AsyncSocketChannels and its
 irk (http://docs.oracle.com/javase/8/docs/api/java/nio/channels/
 AsynchronousServerSocketChannel.html), with core.async in the past.
 Perhaps replacing your direct java.net.Sockets with nio classes that can be
 given CompletionHandlers (http://docs.oracle.com/
 javase/7/docs/api/java/nio/channels/CompletionHandler.html) would be a
 better fit.


 Once I do some performance instrumentation I'll give that a shot. I
 admit that I'm not familiar with all the implications of using the nio
 classes; were I to switch, is it safe to continue using go blocks, or is it
 worth explicitly allocating a single thread per socket?

  Brian

  --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+u...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


   --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to a topic in the
 Google Groups Clojure group.
 To unsubscribe from this topic, visit
 https://groups.google.com/d/topic/clojure/TVMQJwaij1U/unsubscribe.
 To unsubscribe from this group and all its topics, send an email to
 clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] rmap - define lazy, recursive maps

[Zach Tellman]

Hi Arnout,

This is interesting, but may be a lot less useful than you think without 
filling in all the other methods that a normal map has.  For instance, you 
cannot do an equality check with another map (no use of 
clojure.lang.MapEquivalence or implementation of equals and equiv), nor use 
it in a set or key in another map (no implementation of hashCode and 
hasheq), etc. etc.  Unless you and everyone else using it is certain that 
every line of code in your application and the libraries you depend on 
don't do this, this is a ticking timebomb.

For an example of a full map implementation, it might be helpful to take a 
look at 
https://github.com/ztellman/potemkin/blob/master/src/potemkin/collections.clj, 
and will definitely be useful to 
run https://github.com/ztellman/collection-check against your map 
implementation.  One issue that collection-check would uncover is the fact 
that the underlying LinkedHashMap that you're using has different key 
equality semantics than Clojure does, so calling (- m (assoc 1 :foo) 
(dissoc 1N)) would likely break a bunch of stuff.  

Please don't be discouraged by this, making custom map-like data structures 
is much harder right now than it should be.  I just want to make sure no 
one gets an unpleasant surprise in production somewhere down the line.

Best,
Zach

On Monday, October 6, 2014 1:45:02 AM UTC-7, Arnout Roemers wrote:

 Thank you for your kind responses! I will look into writing a post that 
 shows what the motivation was (i.e. how I use it), how it works, and how it 
 compares to Prismatic's Graph. 

 Cheers,
 Arnout


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] async-sockets - work with sockets using core.async channels

[Zach Tellman]

The documentation for Manifold can explain the API better than I can here.
The point where that interacts with Netty w.r.t. backpressure is here:
https://github.com/ztellman/aleph/blob/0.4.0/src/aleph/netty.clj#L109.
Here the stream represents data coming off the wire, and if the put onto
the stream is not immediately successful, backpressure is enabled until the
put completes.  No blocking required anywhere.

On Wed, Oct 8, 2014 at 10:10 AM, Jozef Wagner jozef.wag...@gmail.com
wrote:

 If you want to handle multiple TCP connections and async channels in one
 thread, you need a way how to block on both connections (wait for new input
 to arrive) and channels (wait for a free space in a buffer). Blocking only
 on connections will get you a busy loop if channels are full. If you could
 point me to the part of Aleph sources that handles this issue, I would be
 very grateful. I'm not familiar with netty API nor manifold's concepts, so
 I'm having trouble navigating in the Aleph sources.

 Thanks,
 Jozef

 On Wednesday, October 8, 2014 6:15:47 PM UTC+2, Zach Tellman wrote:

 I wasn't aware of hermod, that's interesting.  I would still characterize
 its approach to backpressure as broken, though, since when the queues get
 full it silently drops messages on the ground.  In fairness, this is very
 clearly documented, so it's less pernicious than some of the other cases
 out there.

 Both core.async buffers and SelectableChannels have very particular
 semantics, and I would be very surprised if they could be combined in that
 way.  It's perfectly possible to feed one into the other and handle
 backpressure properly (again, I'm doing just that with Aleph 0.4.0, using
 Netty), but it's a nuanced integration and easy to get wrong.

 On Wed, Oct 8, 2014 at 7:12 AM, adrian...@mail.yu.edu wrote:

 Check out https://github.com/halgari/com.tbaldridge.hermod for an
 interesting take on this.

 On Wednesday, October 8, 2014 1:17:11 AM UTC-4, Sun Ning wrote:

  BTW, is there any network based core.async channel available now?

 On 10/08/2014 04:36 AM, adrian...@mail.yu.edu wrote:

  It's not about 'safety' (depending on what that means in this
 context), but as Zach pointed out, if you aren't careful about backpressure
 you can run into performance bottlenecks with unrestrained async IO
 operations because although they let you code as if you could handle an
 unlimited amount of connections, obviously that isn't true. There is only a
 finite amount of data that can be buffered in and out of any network
 according to its hardware. When you don't regulate that, your system will
 end up spending an inordinate amount of time compensating for this. You
 don't need to worry about this with regular io because the thread per
 connection abstraction effectively bounds your activity within the
 acceptable physical constraints of the server.

 On Tuesday, October 7, 2014 2:49:30 PM UTC-4, Brian Guthrie wrote:


 On Mon, Oct 6, 2014 at 12:10 AM, adrian...@mail.yu.edu wrote:

 Zach makes an excellent point; I've used AsyncSocketChannels and its
 irk (http://docs.oracle.com/javase/8/docs/api/java/nio/channels/
 AsynchronousServerSocketChannel.html), with core.async in the past.
 Perhaps replacing your direct java.net.Sockets with nio classes that can 
 be
 given CompletionHandlers (http://docs.oracle.com/javase
 /7/docs/api/java/nio/channels/CompletionHandler.html) would be a
 better fit.


 Once I do some performance instrumentation I'll give that a shot. I
 admit that I'm not familiar with all the implications of using the nio
 classes; were I to switch, is it safe to continue using go blocks, or is 
 it
 worth explicitly allocating a single thread per socket?

  Brian

  --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to clojure+u...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


   --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to a topic in the
 Google Groups Clojure group.
 To unsubscribe from this topic, visit https://groups.google.com/d/
 topic/clojure

Re: [ANN] async-sockets - work with sockets using core.async channels

[Zach Tellman]

Sorry, didn't cover converse case.  That's handled by the ChannelSink
directly underneath.  Note that each write returns a Netty ChannelFuture
representing completion of the write, which is transformed into a Manifold
deferred.  Any time a Manifold put returns an unrealized deferred, that
creates upstream backpressure.

On Wed, Oct 8, 2014 at 10:16 AM, Zach Tellman ztell...@gmail.com wrote:

 The documentation for Manifold can explain the API better than I can
 here.  The point where that interacts with Netty w.r.t. backpressure is
 here:
 https://github.com/ztellman/aleph/blob/0.4.0/src/aleph/netty.clj#L109.
 Here the stream represents data coming off the wire, and if the put onto
 the stream is not immediately successful, backpressure is enabled until the
 put completes.  No blocking required anywhere.

 On Wed, Oct 8, 2014 at 10:10 AM, Jozef Wagner jozef.wag...@gmail.com
 wrote:

 If you want to handle multiple TCP connections and async channels in one
 thread, you need a way how to block on both connections (wait for new input
 to arrive) and channels (wait for a free space in a buffer). Blocking only
 on connections will get you a busy loop if channels are full. If you could
 point me to the part of Aleph sources that handles this issue, I would be
 very grateful. I'm not familiar with netty API nor manifold's concepts, so
 I'm having trouble navigating in the Aleph sources.

 Thanks,
 Jozef

 On Wednesday, October 8, 2014 6:15:47 PM UTC+2, Zach Tellman wrote:

 I wasn't aware of hermod, that's interesting.  I would still
 characterize its approach to backpressure as broken, though, since when
 the queues get full it silently drops messages on the ground.  In fairness,
 this is very clearly documented, so it's less pernicious than some of the
 other cases out there.

 Both core.async buffers and SelectableChannels have very particular
 semantics, and I would be very surprised if they could be combined in that
 way.  It's perfectly possible to feed one into the other and handle
 backpressure properly (again, I'm doing just that with Aleph 0.4.0, using
 Netty), but it's a nuanced integration and easy to get wrong.

 On Wed, Oct 8, 2014 at 7:12 AM, adrian...@mail.yu.edu wrote:

 Check out https://github.com/halgari/com.tbaldridge.hermod for an
 interesting take on this.

 On Wednesday, October 8, 2014 1:17:11 AM UTC-4, Sun Ning wrote:

  BTW, is there any network based core.async channel available now?

 On 10/08/2014 04:36 AM, adrian...@mail.yu.edu wrote:

  It's not about 'safety' (depending on what that means in this
 context), but as Zach pointed out, if you aren't careful about 
 backpressure
 you can run into performance bottlenecks with unrestrained async IO
 operations because although they let you code as if you could handle an
 unlimited amount of connections, obviously that isn't true. There is only 
 a
 finite amount of data that can be buffered in and out of any network
 according to its hardware. When you don't regulate that, your system will
 end up spending an inordinate amount of time compensating for this. You
 don't need to worry about this with regular io because the thread per
 connection abstraction effectively bounds your activity within the
 acceptable physical constraints of the server.

 On Tuesday, October 7, 2014 2:49:30 PM UTC-4, Brian Guthrie wrote:


 On Mon, Oct 6, 2014 at 12:10 AM, adrian...@mail.yu.edu wrote:

 Zach makes an excellent point; I've used AsyncSocketChannels and its
 irk (http://docs.oracle.com/javase/8/docs/api/java/nio/channels/
 AsynchronousServerSocketChannel.html), with core.async in the past.
 Perhaps replacing your direct java.net.Sockets with nio classes that 
 can be
 given CompletionHandlers (http://docs.oracle.com/javase
 /7/docs/api/java/nio/channels/CompletionHandler.html) would be a
 better fit.


 Once I do some performance instrumentation I'll give that a shot. I
 admit that I'm not familiar with all the implications of using the nio
 classes; were I to switch, is it safe to continue using go blocks, or is 
 it
 worth explicitly allocating a single thread per socket?

  Brian

  --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com
 Note that posts from new members are moderated - please be patient
 with your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to clojure+u...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


   --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com
 Note that posts from new members

Re: [ANN] async-sockets - work with sockets using core.async channels

[Zach Tellman]

Yes, I didn't mean to imply that there are no blocking operations anywhere,
only that nothing in the worker threads (where the put! occurs) will block,
and that backpressure isn't exerted by causing an active thread to hang.

As to the second point, I'm not sure why you'd think that.  These aren't
futures that are being accessed via blocking dereference, so there's no
reason any thread would block as a result of an unrealized future/deferred.



On Wed, Oct 8, 2014 at 12:22 PM, Jozef Wagner jozef.wag...@gmail.com
wrote:

 Thank you! Using put! callback to control backpressure is a very elegant
 solution. BTW there always has to be blocking somewhere. Netty uses
 selector to block at [1] and .setAutoRead causes respective channel to
 deregister itself from a selector [2], until put! completes.

 Regarding the other way around, it seems to me from the quick look (sorry
 if I misinterpreted) that using futures to represent write completion will
 cause the eventual backpressure to block the thread, causing analogous
 drawback as a blocking put! in the reading from multiple connection case.

 [1]
 https://github.com/netty/netty/blob/220660e351b2a22112b19c4af45e403eab1f73ab/transport/src/main/java/io/netty/channel/nio/NioEventLoop.java#L625
 [2]
 https://github.com/netty/netty/blob/220660e351b2a22112b19c4af45e403eab1f73ab/transport/src/main/java/io/netty/channel/nio/AbstractNioChannel.java#L173

 Jozef

 On Wednesday, October 8, 2014 7:16:44 PM UTC+2, Zach Tellman wrote:

 The documentation for Manifold can explain the API better than I can
 here.  The point where that interacts with Netty w.r.t. backpressure is
 here: https://github.com/ztellman/aleph/blob/0.4.0/src/
 aleph/netty.clj#L109.  Here the stream represents data coming off the
 wire, and if the put onto the stream is not immediately successful,
 backpressure is enabled until the put completes.  No blocking required
 anywhere.

 On Wed, Oct 8, 2014 at 10:10 AM, Jozef Wagner jozef@gmail.com
 wrote:

 If you want to handle multiple TCP connections and async channels in one
 thread, you need a way how to block on both connections (wait for new input
 to arrive) and channels (wait for a free space in a buffer). Blocking only
 on connections will get you a busy loop if channels are full. If you could
 point me to the part of Aleph sources that handles this issue, I would be
 very grateful. I'm not familiar with netty API nor manifold's concepts, so
 I'm having trouble navigating in the Aleph sources.

 Thanks,
 Jozef

 On Wednesday, October 8, 2014 6:15:47 PM UTC+2, Zach Tellman wrote:

 I wasn't aware of hermod, that's interesting.  I would still
 characterize its approach to backpressure as broken, though, since when
 the queues get full it silently drops messages on the ground.  In fairness,
 this is very clearly documented, so it's less pernicious than some of the
 other cases out there.

 Both core.async buffers and SelectableChannels have very particular
 semantics, and I would be very surprised if they could be combined in that
 way.  It's perfectly possible to feed one into the other and handle
 backpressure properly (again, I'm doing just that with Aleph 0.4.0, using
 Netty), but it's a nuanced integration and easy to get wrong.

 On Wed, Oct 8, 2014 at 7:12 AM, adrian...@mail.yu.edu wrote:

 Check out https://github.com/halgari/com.tbaldridge.hermod for an
 interesting take on this.

 On Wednesday, October 8, 2014 1:17:11 AM UTC-4, Sun Ning wrote:

  BTW, is there any network based core.async channel available now?

 On 10/08/2014 04:36 AM, adrian...@mail.yu.edu wrote:

  It's not about 'safety' (depending on what that means in this
 context), but as Zach pointed out, if you aren't careful about 
 backpressure
 you can run into performance bottlenecks with unrestrained async IO
 operations because although they let you code as if you could handle an
 unlimited amount of connections, obviously that isn't true. There is 
 only a
 finite amount of data that can be buffered in and out of any network
 according to its hardware. When you don't regulate that, your system will
 end up spending an inordinate amount of time compensating for this. You
 don't need to worry about this with regular io because the thread per
 connection abstraction effectively bounds your activity within the
 acceptable physical constraints of the server.

 On Tuesday, October 7, 2014 2:49:30 PM UTC-4, Brian Guthrie wrote:


 On Mon, Oct 6, 2014 at 12:10 AM, adrian...@mail.yu.edu wrote:

 Zach makes an excellent point; I've used AsyncSocketChannels and
 its irk (http://docs.oracle.com/javase/8/docs/api/java/nio/
 channels/AsynchronousServerSocketChannel.html), with core.async in
 the past. Perhaps replacing your direct java.net.Sockets with nio 
 classes
 that can be given CompletionHandlers (http://docs.oracle.com/javase
 /7/docs/api/java/nio/channels/CompletionHandler.html) would be a
 better fit.


 Once I do some performance instrumentation I'll give that a shot. I

Re: [ANN] async-sockets - work with sockets using core.async channels

[Zach Tellman]

If I'm reading this correctly, you're using non-blocking thread pools for 
blocking operations on the sockets.  Given more than N connections (last 
time I looked the thread pool's size was 42), you risk deadlock or at the 
very least poor average throughput.

On Sunday, October 5, 2014 7:06:56 PM UTC-7, Brian Guthrie wrote:

 Hi all,

 I'm releasing a little library for working with sockets. Feedback and pull 
 requests gratefully appreciated.

 The skinny
 ---

 This library allows you to create socket servers and socket clients and 
 interact with them asynchronously using channels. Servers return a record 
 with a :connections field, a channel which yields one socket per incoming 
 connection. Clients return the same socket record. Socket records each have 
 an :in and :out channel each which allow you to receive and send data 
 respectively on a line-by-line basis. The raw java.net.Socket is also 
 available (as :socket).

 Servers and clients are defined using the Component framework and must be 
 explicitly started using (component/start server-or-client), though 
 sockets will clean up after themselves if they are terminated for some 
 reason.

 Further information is available on Github here: 
 https://github.com/bguthrie/async-sockets

 Releases
 --

 This is the first release, which I've tagged for now as 0.0.1-SNAPSHOT. 
 Leiningen dependency: [com.gearswithingears/async-sockets 0.0.1-SNAPSHOT].

 If this is useful to you, please let me know, but any and all feedback is 
 great.

 Happy hacking,

 Brian
 @bguthrie
 btgu...@gmail.com javascript:


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Re: Handling java streams..

[Zach Tellman]

Calling (- stream .iterator iterator-seq) should give the desired result.

On Wednesday, October 1, 2014 6:39:11 AM UTC-7, José Ricardo wrote:

 By nicer I meant something like being able to use, for example, map and 
 filter, just like I can do on a java.util.ArrayList (in clojure) and on a 
 Stream (in java).

 On Saturday, September 27, 2014 1:47:38 AM UTC-4, Sean Corfield wrote:

 On Fri, Sep 26, 2014 at 11:51 AM, José Ricardo zehz...@gmail.com 
 wrote: 
  Hi, I'm not sure if resurrecting this thread is the right approach, but 
 what 
  about Java 8 Streams (java.util.stream)? 
  
  Are there any libraries out there for making java 8 streams handling 
 nicer? 
  :) 

 Well, this thread is certainly a blast from the past :) 

 Can you be a bit more specific about what you'd like to see nicer 
 with Java 8 streams, and what you'd like to see in a Clojure wrapper? 

 I ask because Java 8 is the first version that I've been interested in 
 since I felt it went off the rails with Java 5 - and streams are part 
 of what makes Java 8 attractive again :) 
 -- 
 Sean A Corfield -- (904) 302-SEAN 
 An Architect's View -- http://corfield.org/ 
 World Singles, LLC. -- http://worldsingles.com/ 

 Perfection is the enemy of the good. 
 -- Gustave Flaubert, French realist novelist (1821-1880) 



-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] async-sockets - work with sockets using core.async channels

[Zach Tellman]

Please note that if you use core.async with java.nio, you need to make sure 
backpressure is properly propagated (this happens automatically with 
java.io, assuming you have a thread per connection).

On Sunday, October 5, 2014 9:10:24 PM UTC-7, adrian...@mail.yu.edu wrote:

 Zach makes an excellent point; I've used AsyncSocketChannels and its irk (
 http://docs.oracle.com/javase/8/docs/api/java/nio/channels/AsynchronousServerSocketChannel.html),
  
 with core.async in the past. Perhaps replacing your direct java.net.Sockets 
 with nio classes that can be given CompletionHandlers (
 http://docs.oracle.com/javase/7/docs/api/java/nio/channels/CompletionHandler.html)
  
 would be a better fit. 

 On Sunday, October 5, 2014 11:57:18 PM UTC-4, Zach Tellman wrote:

 If I'm reading this correctly, you're using non-blocking thread pools for 
 blocking operations on the sockets.  Given more than N connections (last 
 time I looked the thread pool's size was 42), you risk deadlock or at the 
 very least poor average throughput.

 On Sunday, October 5, 2014 7:06:56 PM UTC-7, Brian Guthrie wrote:

 Hi all,

 I'm releasing a little library for working with sockets. Feedback and 
 pull requests gratefully appreciated.

 The skinny
 ---

 This library allows you to create socket servers and socket clients and 
 interact with them asynchronously using channels. Servers return a record 
 with a :connections field, a channel which yields one socket per incoming 
 connection. Clients return the same socket record. Socket records each have 
 an :in and :out channel each which allow you to receive and send data 
 respectively on a line-by-line basis. The raw java.net.Socket is also 
 available (as :socket).

 Servers and clients are defined using the Component framework and must 
 be explicitly started using (component/start server-or-client), though 
 sockets will clean up after themselves if they are terminated for some 
 reason.

 Further information is available on Github here: 
 https://github.com/bguthrie/async-sockets

 Releases
 --

 This is the first release, which I've tagged for now as 0.0.1-SNAPSHOT. 
 Leiningen dependency: [com.gearswithingears/async-sockets 0.0.1-SNAPSHOT].

 If this is useful to you, please let me know, but any and all feedback 
 is great.

 Happy hacking,

 Brian
 @bguthrie
 btgu...@gmail.com



-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] data.int-map 0.1.0

[Zach Tellman]

https://github.com/clojure/data.int-map

This contrib library represents the union of two other libraries [1] [2], 
which are now both deprecated.  There's nothing too surprising here, but 
I'm happy to answer any questions.

Zach

[1] https://github.com/ztellman/immutable-int-map
[2] https://github.com/ztellman/immutable-bitset

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Howto Use Automat ?

[Zach Tellman]

Hi Tim,

Glad to see you're trying Automat out.

I'm not sure what  you mean by forks in the road.  If your FSM is defined 
as (a/or [1 3] [1 2 3]), then you can do any of the following:

(reduce #(a/advance fsm %1 %2) nil [1 2 3])

(a/advance-stream fsm nil [1 2 3] ::rejected)

(let [adv (partial a/advance fsm)]
  (- nil (adv 1) (adv 2) (adv 3)))

Does that answer your first two questions?  If not, you'll have to clarify 
what you're asking.

Automat doesn't allow you to name your states, because that breaks your 
ability to freely compose automata.  If you want to differentiate between 
two different paths, you can use actions:

  (view (a/or [:sent :acknowledged :accepted] [:sent :rejected (a/$ 
:not-accepted)]))

I can go into more depth on this, but the README gives a fairly complete 
explanation of how actions can be used.

Hope that helps,
Zach




On Tuesday, August 26, 2014 11:42:24 AM UTC-7, frye wrote:

 I'm trying to use Automat to map out transitions that are either of the 
 paths below. 

- There can be many claims sent to an external service. Each claim, 
after being sent, can be :acknowledged or :rejected. 
- If a claim is :acknowledged, it can then either be :accepted or 
:rejected. 
- There will be many such claims 

 [:sent :acknowledged :accepted]
 [:sent :acknowledged :rejected]
 [:sent :rejected]


 I'm trying to figure out how to use Automat to handle this use case. 

- The only way to advance through your FSM, seems to be to call 
fsm/advance (or fsm/advance-stream). How do we deal with forks in the road?
- Looking at the documentation 
https://github.com/ztellman/automat#a-short-example and tests 

 https://github.com/ztellman/automat/blob/master/test/automat/fsm_test.clj, 
it's not clear to me if I can pass multiple tokens (or claims) through 1 
FSM. Are we meant to use 1 FSM per token (claim in this case)? 
- The semantics for Automat have you declare transitions, instead of 
states. States are implicit numbers between the transitions. Pallet-fsm 
https://github.com/pallet/pallet-fsm seems to follow the state 
declaration pattern, but hasn't been updated in a few years. Can I name 
 the 
states and transitions between them? 
- There always seems to be one output state, even if they come from 2 
different transitions. So the below 2 FSMs are equivalent. However, they 
both go to the same end state. Is there a way to define different output 
states?  

 = (require '[automat.viz :refer (view)])
 = (require '[automat.core :as a])

 = (view (a/or [:sent :acknowledged :accepted] [:sent :acknowledged 
 :rejected] [:sent :rejected]))
 = (view [:sent (a/or [:acknowledged (a/or :accepted :rejected)] 
 [:rejected])])



 Tim Washington 
 Interruptsoftware.com http://interruptsoftware.com 

  

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] Automat 0.1.2

[Zach Tellman]

https://github.com/ztellman/automat

The initial release of this was just a little ways back, where I alluded to 
using the library to react to browsing behavior.  A few people asked how to 
actually accomplish that, and I realized that this sort of use, while 
possible, was harder than it needed to be.  So I added a function designed 
for this use case, and added some more detailed documentation on how the 
library can be used effectively.

As always, questions and feedback are welcome.

Zach

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Clojure Office Hours - Experience Report and Future Plans

[Zach Tellman]

For the in-person variety, I've written up some thoughts on why office 
hours are a good format for meetups, and ideas on the underlying 
processes: http://blog.factual.com/clojure-office-hours.

On Thursday, April 24, 2014 8:44:08 PM UTC-7, Leif wrote:

 This message is aimed at people that want to *hold* office hours 
 primarily, but of course others can chime in with
 opinions, suggestions, cheerleading, etc.

 I recently held office hours where I chatted / pair programmed with 
 less experienced clojure programmers (some
 were in fact more experienced).

 Lessons learned:

 1. It's fun!  Do it!  Online like me, or convince your local clojure user 
 group to do it.
 2. As I expected, I was more help to less experienced people, but learned 
 a lot *from* the others, and hopefully
I was at least useful as a sounding board.
 3. An hour is less time than it sounds.
 4. If possible, test your pair programming setup beforehand (see point 3 
 above)
a) corollary: if someone is asking about a library that takes some 
 setup, it's probably best if *they* do the
   setup and host the pairing session.
 5. Any remote sharing software (tmux, teamviewer, etc) will mangle *some* 
 input.  Be prepared to work around that.
 6. Educate people how to cancel, and to cancel ASAP, since some will 
 inevitably need to.
 7. For beginners (at clojure, but not programming), pick a specific 
 problem and work through it, or have a
solution and explain it step-by-step; that seemed to work best.  Code 
 review of some OSS project they are
interested in might also work, I didn't try it (but again, see point 3)
 8. Unfortunately, no one completely new to programming booked with me, so 
 others will have to give advice here.
 9. Many people outside of the western hemisphere were interested, so it 
 would be nice to have coverage across the
globe.

 Future plans:

 Small plug: I used youcanbook.me to manage the office hours, with no 
 problems.  I encourage you to use their
 service, say nice things about them, and possibly give them money, 
 *because*:

 These fine folks allow non-profits to use their advanced features for 
 free, or at a reduced price.  So, I requested
 that the Clojure community's office hours get this status.  They said yes, 
 so my account (for now, for testing, we
 can move it later) can have unlimited team members and services.  So, 
 I'd like to ask if there is interest in
 setting up a community clearinghouse for giving/receiving more office 
 hours, possibly of more types.  Some ideas
 (chime in with your own):

 1. General Office Hours
Basically what I did, except with more people offering office hours, so 
 that:
a. Any one person will only have to offer a small number of hours a 
 week (1, even).
b. Hopefully more coverage across time zones.
c. People can tag what kinds of programming / projects they have 
 expertise in, so that beginners picking up
  clojure for a specific reason or library can have a more productive 
 session.  E.g. some descriptions could read:

Leif Poorman
Location: Eastern USA
Languages: en
Tags: beginners, absolute beginners, web, data analysis, machine 
 learning

Rich Hickey (obviously this is just an example)
Location: USA
Languages: en, Bynar
Tags: distributed systems, functional databases, Datomic, concurrency, 
 alien technology, everything else

 2. Office Hours for Beginners
Specifically geared toward beginners in FP, absolute beginners in 
 programming, etc.  This could be covered by
the description tags as above.  Or this could be more of a hangout, 
 where a set number of beginners get led
through the ClojureBridge curriculum, or similar.
 3. Project Specific Hours
a) Someone with knowledge of an open source project gives a demo of its 
 capabilities/weaknesses to prospective
   users (kind of a technical sales pitch, but for OSS)
b) The maintainer of a fairly complex open source project walks some 
 people that want to contribute through the
   codebase, to kickstart their contributions (I've seen this 
 done/proposed for Midje and Cascalog, at least).

 Alternatively, we could just start with 1-on-1, or 1-on-1 and small group, 
 and see where it goes from there.

 Comments?  Questions?  Suggestions?

 Cheers,
 Leif

 P.S. If you are interested in holding a few office hours, email me, and we 
 can start testing out the more advanced youcanbook.me features.



-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To 

[ANN] Automat: better FSMs through combinators

[Zach Tellman]

https://github.com/ztellman/automat

This has been languishing in my Github for a while, for lack of a few 
finishing touches on the code and documentation.  I think this is one of 
cooler libraries I've written, and beyond the obvious use for parsers, the 
set theoretic operators could be a powerful way to specify actions in 
response to complex browsing behaviors on a site, or any other number of 
things.  I'm excited to see how it's used.

I'm happy to answer any questions.  Suggestions on how to improve the 
documentation are encouraged.

Zach

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] immutable-int-map

[Zach Tellman]

Correct. I'll clarify that I mean 64 bit integers in the readme.
On Apr 21, 2014 5:26 AM, Alex Miller a...@puredanger.com wrote:

 Never mind, look like you mean integer in the generic sense and you are
 using longs, right?

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to a topic in the
 Google Groups Clojure group.
 To unsubscribe from this topic, visit
 https://groups.google.com/d/topic/clojure/0EysWMml6Ks/unsubscribe.
 To unsubscribe from this group and all its topics, send an email to
 clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] immutable-int-map

[Zach Tellman]

This one's pretty simple: it's an immutable map that can only have positive 
integers as keys.  It can be found at 
https://github.com/ztellman/immutable-int-map.

The one interesting aspect of this is that it has an efficient merge 
mechanism, which means it plays better with Clojure's reducer framework 
than Clojure's own data structures.  This was a non-obvious consequence to 
me before I started working on it, so I figure others might also find it 
interesting.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] immutable-int-map

[Zach Tellman]

I could represent the map {0 :foo, 100 :bar} as an array, but it would 
have to be a million element array with a lot of empty space. This would be 
(maybe) faster w.r.t. lookups, but would be vastly slower for enumerating 
entries, merging other maps, and adding keys that are larger than the 
boundaries of the underlying array.  This data structure, which is 
described by the paper linked in the readme,  I also mention the 
possibility of using this to represent a sparse vector (as opposed to the 
array, which is dense) in the readme.

Hope that answers your question,
Zach

On Sunday, April 20, 2014 9:44:21 PM UTC-7, Plinio Balduino wrote:

 it's an immutable map that can only have positive integers as keys -- Like 
 an array?

 (My question is child of my complete ignorance, and I'm not questioning 
 your knowledge or motivation)

 Plínio


 On Mon, Apr 21, 2014 at 1:40 AM, Zach Tellman ztel...@gmail.comjavascript:
  wrote:

 This one's pretty simple: it's an immutable map that can only have 
 positive integers as keys.  It can be found at 
 https://github.com/ztellman/immutable-int-map.

 The one interesting aspect of this is that it has an efficient merge 
 mechanism, which means it plays better with Clojure's reducer framework 
 than Clojure's own data structures.  This was a non-obvious consequence to 
 me before I started working on it, so I figure others might also find it 
 interesting.
  
 -- 
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.comjavascript:
 Note that posts from new members are moderated - please be patient with 
 your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com javascript:
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 --- 
 You received this message because you are subscribed to the Google Groups 
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to clojure+u...@googlegroups.com javascript:.
 For more options, visit https://groups.google.com/d/optout.




-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] durable-queue: an in-process disk-backed queue

[Zach Tellman]

Hey Leif,

When using :fsync-interval, the actual calls to .force() on the underlying 
ByteBuffers occur on another thread, making it effectively a background 
process.  This is contrasted with :fsync-threshold, which will 
synchronously call fsync when the write threshold is hit.  Note that if the 
fsync-interval is less than the time it takes to actually fsync, it will 
simply continuously fsync at whatever pace it can.

If you'd like to verify, I suggest using strace.

Zach

On Sunday, March 9, 2014 6:54:50 PM UTC-7, Leif wrote:

 Hi, Zach.

 I was trying to benchmark at different values of the :fysnc-* parameters, 
 and I noticed that it didn't matter what value of :fsync-interval I set, 
 the performance was constant, and about what it is with both :fsync-put? 
 and :fsync-take? disabled.

 Any suggestions on how to test if data is actually being synced to disk at 
 my specified interval?

 Please forgive my suspicious nature,
 Leif

 On Friday, March 7, 2014 4:21:44 PM UTC-5, Zach Tellman wrote:

 I added the above-described features a few weeks back, but only got 
 around to marking 0.1.1 today.  Fsync batching is described at the end of 
 the README, let me know if you have any questions.

 On Friday, February 7, 2014 11:52:11 AM UTC-8, Zach Tellman wrote:

 Hi Bob,

 Right now the API only allows for single puts, and fsyncing is 
 all-or-nothing.  However, this is just an artifact of my major use case for 
 the library, which relies on upstream batching of tasks.  I'm planning an 
 0.1.1 release which has an explicit `sync` method, and support for 
 sync-intervals (i.e. sync twice a second) and sync-thresholds (i.e. sync 
 every ten puts or takes).  The use case you describe could be achieved by 
 disabling automatic syncing, and doing a series of puts and takes followed 
 by a call to `sync`.

 If you have thoughts or suggestions on how this can be more useful for 
 you, please let me know.

 Zach


 On Fri, Feb 7, 2014 at 5:26 AM, Bob Hutchison hutch...@recursive.cawrote:


 On Feb 6, 2014, at 6:45 PM, Zach Tellman za...@factual.com wrote:

 At Factual we get a lot of data thrown at us, and often don't have 
 control over the rate at which it comes in.  As such, it's preferable that 
 our buffer isn't bounded by the process' memory, since a temporary blip in 
 throughput may cause GC pauses, OOM exceptions, and other things that will 
 only exacerbate the problem.  It's also preferable that if the process 
 dies, we won't lose any data which hasn't yet escaped the process.  A 
 disk-backed queue satisfies both of these requirements.

 As such, I'm happy to announce that we're open sourcing 
 'durable-queue': https://github.com/Factual/durable-queue.  It's a 
 small, fast, pure-Clojure implementation that in our production systems is 
 responsible for processing billions of entries daily.  We believe it has 
 broad applications, and are excited to see how others will use it.


 What excellent timing! I’ve been looking at ZeroMQ, RabbitMQ, and Kafka 
 for the last week or so. ZMQ is awfully attractive for what I’m trying to 
 do, but there are a few things it doesn’t do that I need done. I had begun 
 thinking of building something similar on top of Redis.

 You mention the idea of batching to reduce the impact of fsync. Is 
 there an API for batching puts? Is there a way to batch a complete! and 
 put! new tasks to the queue?

 One pattern that keeps coming up is:
- take a single task from the queue
- execute the task, which might generate a set of new tasks to be 
 queued on the same queue (and likely on other queues too)
- signal completion, and put the new tasks

 Cheers,
 Bob


 Zach

 P.S. If this sort of work is interesting to you, Factual is hiring: 
 https://groups.google.com/forum/#!searchin/clojure/factual/clojure/8bPIEnNpfyQ/lvv-9gkVozAJ

 -- 
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com
 Note that posts from new members are moderated - please be patient with 
 your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 --- 
 You received this message because you are subscribed to the Google 
 Groups Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to clojure+u...@googlegroups.com.

 For more options, visit https://groups.google.com/groups/opt_out.


  -- 
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com
 Note that posts from new members are moderated - please be patient with 
 your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 --- 
 You received this message because you

Re: [ANN] durable-queue: an in-process disk-backed queue

[Zach Tellman]

I added the above-described features a few weeks back, but only got around 
to marking 0.1.1 today.  Fsync batching is described at the end of the 
README, let me know if you have any questions.

On Friday, February 7, 2014 11:52:11 AM UTC-8, Zach Tellman wrote:

 Hi Bob,

 Right now the API only allows for single puts, and fsyncing is 
 all-or-nothing.  However, this is just an artifact of my major use case for 
 the library, which relies on upstream batching of tasks.  I'm planning an 
 0.1.1 release which has an explicit `sync` method, and support for 
 sync-intervals (i.e. sync twice a second) and sync-thresholds (i.e. sync 
 every ten puts or takes).  The use case you describe could be achieved by 
 disabling automatic syncing, and doing a series of puts and takes followed 
 by a call to `sync`.

 If you have thoughts or suggestions on how this can be more useful for 
 you, please let me know.

 Zach


 On Fri, Feb 7, 2014 at 5:26 AM, Bob Hutchison 
 hutch...@recursive.cajavascript:
  wrote:


 On Feb 6, 2014, at 6:45 PM, Zach Tellman za...@factual.com javascript: 
 wrote:

 At Factual we get a lot of data thrown at us, and often don't have 
 control over the rate at which it comes in.  As such, it's preferable that 
 our buffer isn't bounded by the process' memory, since a temporary blip in 
 throughput may cause GC pauses, OOM exceptions, and other things that will 
 only exacerbate the problem.  It's also preferable that if the process 
 dies, we won't lose any data which hasn't yet escaped the process.  A 
 disk-backed queue satisfies both of these requirements.

 As such, I'm happy to announce that we're open sourcing 'durable-queue': 
 https://github.com/Factual/durable-queue.  It's a small, fast, 
 pure-Clojure implementation that in our production systems is responsible 
 for processing billions of entries daily.  We believe it has broad 
 applications, and are excited to see how others will use it.


 What excellent timing! I’ve been looking at ZeroMQ, RabbitMQ, and Kafka 
 for the last week or so. ZMQ is awfully attractive for what I’m trying to 
 do, but there are a few things it doesn’t do that I need done. I had begun 
 thinking of building something similar on top of Redis.

 You mention the idea of batching to reduce the impact of fsync. Is there 
 an API for batching puts? Is there a way to batch a complete! and put! new 
 tasks to the queue?

 One pattern that keeps coming up is:
- take a single task from the queue
- execute the task, which might generate a set of new tasks to be 
 queued on the same queue (and likely on other queues too)
- signal completion, and put the new tasks

 Cheers,
 Bob


 Zach

 P.S. If this sort of work is interesting to you, Factual is hiring: 
 https://groups.google.com/forum/#!searchin/clojure/factual/clojure/8bPIEnNpfyQ/lvv-9gkVozAJ

 -- 
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.comjavascript:
 Note that posts from new members are moderated - please be patient with 
 your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com javascript:
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 --- 
 You received this message because you are subscribed to the Google Groups 
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to clojure+u...@googlegroups.com javascript:.

 For more options, visit https://groups.google.com/groups/opt_out.


  -- 
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.comjavascript:
 Note that posts from new members are moderated - please be patient with 
 your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com javascript:
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 --- 
 You received this message because you are subscribed to a topic in the 
 Google Groups Clojure group.
 To unsubscribe from this topic, visit 
 https://groups.google.com/d/topic/clojure/4tZFWdMKvjw/unsubscribe.
 To unsubscribe from this group and all its topics, send an email to 
 clojure+u...@googlegroups.com javascript:.
 For more options, visit https://groups.google.com/groups/opt_out.




-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send

Re: [ANN] durable-queue: an in-process disk-backed queue

[Zach Tellman]

Hi Bob,

Right now the API only allows for single puts, and fsyncing is
all-or-nothing.  However, this is just an artifact of my major use case for
the library, which relies on upstream batching of tasks.  I'm planning an
0.1.1 release which has an explicit `sync` method, and support for
sync-intervals (i.e. sync twice a second) and sync-thresholds (i.e. sync
every ten puts or takes).  The use case you describe could be achieved by
disabling automatic syncing, and doing a series of puts and takes followed
by a call to `sync`.

If you have thoughts or suggestions on how this can be more useful for you,
please let me know.

Zach


On Fri, Feb 7, 2014 at 5:26 AM, Bob Hutchison hutch-li...@recursive.cawrote:


 On Feb 6, 2014, at 6:45 PM, Zach Tellman z...@factual.com wrote:

 At Factual we get a lot of data thrown at us, and often don't have control
 over the rate at which it comes in.  As such, it's preferable that our
 buffer isn't bounded by the process' memory, since a temporary blip in
 throughput may cause GC pauses, OOM exceptions, and other things that will
 only exacerbate the problem.  It's also preferable that if the process
 dies, we won't lose any data which hasn't yet escaped the process.  A
 disk-backed queue satisfies both of these requirements.

 As such, I'm happy to announce that we're open sourcing 'durable-queue':
 https://github.com/Factual/durable-queue.  It's a small, fast,
 pure-Clojure implementation that in our production systems is responsible
 for processing billions of entries daily.  We believe it has broad
 applications, and are excited to see how others will use it.


 What excellent timing! I've been looking at ZeroMQ, RabbitMQ, and Kafka
 for the last week or so. ZMQ is awfully attractive for what I'm trying to
 do, but there are a few things it doesn't do that I need done. I had begun
 thinking of building something similar on top of Redis.

 You mention the idea of batching to reduce the impact of fsync. Is there
 an API for batching puts? Is there a way to batch a complete! and put! new
 tasks to the queue?

 One pattern that keeps coming up is:
- take a single task from the queue
- execute the task, which might generate a set of new tasks to be
 queued on the same queue (and likely on other queues too)
- signal completion, and put the new tasks

 Cheers,
 Bob


 Zach

 P.S. If this sort of work is interesting to you, Factual is hiring:
 https://groups.google.com/forum/#!searchin/clojure/factual/clojure/8bPIEnNpfyQ/lvv-9gkVozAJ

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.

 For more options, visit https://groups.google.com/groups/opt_out.


  --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to a topic in the
 Google Groups Clojure group.
 To unsubscribe from this topic, visit
 https://groups.google.com/d/topic/clojure/4tZFWdMKvjw/unsubscribe.
 To unsubscribe from this group and all its topics, send an email to
 clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[ANN] durable-queue: an in-process disk-backed queue

[Zach Tellman]

At Factual we get a lot of data thrown at us, and often don't have control 
over the rate at which it comes in.  As such, it's preferable that our 
buffer isn't bounded by the process' memory, since a temporary blip in 
throughput may cause GC pauses, OOM exceptions, and other things that will 
only exacerbate the problem.  It's also preferable that if the process 
dies, we won't lose any data which hasn't yet escaped the process.  A 
disk-backed queue satisfies both of these requirements.

As such, I'm happy to announce that we're open sourcing 'durable-queue': 
https://github.com/Factual/durable-queue.  It's a small, fast, pure-Clojure 
implementation that in our production systems is responsible for processing 
billions of entries daily.  We believe it has broad applications, and are 
excited to see how others will use it.

Zach

P.S. If this sort of work is interesting to you, Factual is 
hiring: 
https://groups.google.com/forum/#!searchin/clojure/factual/clojure/8bPIEnNpfyQ/lvv-9gkVozAJ

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Factual is hiring Clojure engineers

[Zach Tellman]

We will consider candidates from outside the US if they're willing to
relocate here.  I'm happy to talk in more detail if you like, just email me
at the above address.


On Sat, Jan 18, 2014 at 3:31 AM, Petr petrg...@gmail.com wrote:

 Do you consider non US candidates?
 Wihout extensive data analysis experience?


 On Saturday, January 18, 2014 7:34:38 AM UTC+7, Zach Tellman wrote:

 It's been six months since I last announced this, so: Factual is
 perpetually hiring smart, driven engineers to solve deep problems in data
 analysis and systems engineering.  We have offices in both SF and LA, and
 remote candidates with a strong background in analysis will be considered.

 Factual provides an index of things in the world (businesses, places, and
 products).  You can either use this directly via our API [1] to ask
 questions like what are the three coffee shops nearest to me, or
 indirectly by giving us your own real world data to enrich and refine [2]
 [3].  Factual uses Clojure extensively [4], both for realtime systems and
 batch data analysis.  Systems written purely in Clojure handle the
 following:

 * realtime intake of 70k datapoints/sec from our partners
 * batch processing of 7 billion new datapoints every day, comprising
 terabytes of compressed textual data
 * generating, storing, and serving 150 million records, refreshed daily

 Both of these numbers have at least doubled in the last six months, and
 could very easily do so again.  There are also other projects that use
 Clojure alongside Java to work at similar scales.

 Libraries developed in the course of working on these projects include:

 https://github.com/factual/skuld
 https://github.com/factual/clj-leveldb
 https://github.com/aphyr/merkle
 https://github.com/ztellman/clj-tuple
 https://github.com/ztellman/narrator

 Several other libraries are on the cusp of being open sourced, when we
 get a chance.

 We need people who enjoy using technology as a lever to accomplish
 amazing things (wasn't it Archimedes who said give me a large enough data
 center...), who want to work alongside people they can continuously learn
 from, and who want to be just a little uncomfortable with the scale and
 scope of their responsibilities.

 If this sounds interesting, contact me at zach at factual.com.  I'm
 heading up the small but growing SF office, and am happy to talk in person
 with anyone in the Bay Area who'd like to know more.

 [1] http://developer.factual.com/working-with-factual-places/
 [2] http://www.factual.com/products/geopulse-audience
 [3] http://www.factual.com/products/resolve
 [4] http://www.factual.com/jobs/clojure

  --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to a topic in the
 Google Groups Clojure group.
 To unsubscribe from this topic, visit
 https://groups.google.com/d/topic/clojure/8bPIEnNpfyQ/unsubscribe.
 To unsubscribe from this group and all its topics, send an email to
 clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.


-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Factual is hiring Clojure engineers

[Zach Tellman]

It's been six months since I last announced this, so: Factual is 
perpetually hiring smart, driven engineers to solve deep problems in data 
analysis and systems engineering.  We have offices in both SF and LA, and 
remote candidates with a strong background in analysis will be considered.

Factual provides an index of things in the world (businesses, places, and 
products).  You can either use this directly via our API [1] to ask 
questions like what are the three coffee shops nearest to me, or 
indirectly by giving us your own real world data to enrich and refine [2] 
[3].  Factual uses Clojure extensively [4], both for realtime systems and 
batch data analysis.  Systems written purely in Clojure handle the 
following:

* realtime intake of 70k datapoints/sec from our partners
* batch processing of 7 billion new datapoints every day, comprising 
terabytes of compressed textual data
* generating, storing, and serving 150 million records, refreshed daily

Both of these numbers have at least doubled in the last six months, and 
could very easily do so again.  There are also other projects that use 
Clojure alongside Java to work at similar scales.

Libraries developed in the course of working on these projects include:

https://github.com/factual/skuld
https://github.com/factual/clj-leveldb
https://github.com/aphyr/merkle
https://github.com/ztellman/clj-tuple
https://github.com/ztellman/narrator

Several other libraries are on the cusp of being open sourced, when we get 
a chance.

We need people who enjoy using technology as a lever to accomplish amazing 
things (wasn't it Archimedes who said give me a large enough data 
center...), who want to work alongside people they can continuously learn 
from, and who want to be just a little uncomfortable with the scale and 
scope of their responsibilities.

If this sounds interesting, contact me at zach at factual.com.  I'm heading 
up the small but growing SF office, and am happy to talk in person with 
anyone in the Bay Area who'd like to know more.

[1] http://developer.factual.com/working-with-factual-places/
[2] http://www.factual.com/products/geopulse-audience
[3] http://www.factual.com/products/resolve
[4] http://www.factual.com/jobs/clojure

-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] Narrator: expressive, composable stream analysis

[Zach Tellman]

Riemann is a service for receiving streams of events, and causing one or
more side-effects (sending email, routing to Graphite, etc).  It can do
arbitrary transformations on event streams (the effects from an input may
be arbitrarily time shifted), and assumes that the inputs are fixed
structure (numbers, shallow maps, etc).

Narrator is a library for analyzing streams of events, and returns either a
single value representing the stream, or values representing fixed, regular
intervals within the stream.  The analysis can only on that interval (or
multiples of that interval, using 'moving').  Using the 'recur' operator,
it can do analysis on arbitrarily nested structures.

At Factual, we use both of these in tandem.  Since the trace data for
function call trees is arbitrarily nested, we use Narrator, and then
separate the data into flattened sub-parts, and pass it onto Riemann.  We
typically have a fixed set of functions that we're tracing (entry points
for HTTP requests, primarily), and automatically send them along via UDP to
Omphalos [1].  Obviously the instrumented functions that are called in the
process of creating a response may change, this is within the control of
the authors of the libraries we use.

Does this answer all your questions?  I'm happy to elaborate on any of the
above.

Zach

[1] This is discussed in more detail here:
http://www.infoq.com/presentations/analyze-running-system


On Sun, Nov 10, 2013 at 3:03 AM, dm3 deadmo...@gmail.com wrote:

 I've read about Lamina and Narrator, watched the linked videos and I think
 I understand how it all fits together:

 1) Instrument the applications using Lamina's `instrument` or `trace`
 2) Probe the instrumented code somehow by channeling the traces to some
 endpoint (how do you do this? do you automatically probe everything and
 channel to some remote service? Do you have some method of dynamically
 enabling/disabling probes on your services (e.g. embedded repl or some
 management endpoint)?
 3) Analyze the traces remotely using Narrator + networking code + a UI
 (that's the gist of the Omphalos as I've understood). If so, what would you
 say is the largest difference between Riemann's[1] stream analysis
 functionality and what is provided by Narrator? Would you say that Omphalos
 and Riemann serve completely different puproses?

 Am I on the right track?

 [1] http://riemann.io/


 On Sunday, 3 November 2013 00:28:27 UTC+2, Zach Tellman wrote:

 https://github.com/ztellman/narrator

 This is a reimplementation of an approach I've discussed in several talks
 [1] [2], with an eye towards performance, memory efficiency, and
 flexibility w.r.t. how the event stream is represented.  The readme does a
 good job of explaining how it works, but there have been a number of new
 event processing libraries recently (core.async, EEP, etc.), so I'll spend
 some time here describing how this differs.

 First, this library is focused on aggregations over event streams, not
 arbitrary transformations.  It is designed such that these aggregations can
 be automatically parallelized, and use non-thread-safe data structures
 (such as those in the excellent stream-lib [3]) without having to worry
 about coordination.  As such, within this narrower application it has a
 richer set of operators, and should be a fair bit faster (millions of
 messages/sec/core).

 Second, this has support for time-series analysis of ordered streams,
 either historical or in real time.  The input for either type of analysis
 can be normal sequences, core.async channels, or Lamina channels. At
 Factual we use this for aggregations across many of our real-time systems,
 and I also use it for both ad hoc queries and daily rollups of logs and
 other historical data.

 On a personal note, I think this is one of the most interesting and
 useful libraries I've written.  I'm really looking forward to seeing how
 people use it, and encourage feedback on how to make it better.

 Zach

 [1] http://www.infoq.com/presentations/analyze-running-system
 [2] http://vimeo.com/45132054#!
 [3] https://github.com/addthis/stream-lib

  --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to a topic in the
 Google Groups Clojure group.
 To unsubscribe from this topic, visit
 https://groups.google.com/d/topic/clojure/gXGVXgqd9Xs/unsubscribe.
 To unsubscribe from this group and all its topics, send an email to
 clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.


-- 
-- 
You received this message because you are subscribed to the Google

[ANN] Narrator: expressive, composable stream analysis

[Zach Tellman]

https://github.com/ztellman/narrator

This is a reimplementation of an approach I've discussed in several talks 
[1] [2], with an eye towards performance, memory efficiency, and 
flexibility w.r.t. how the event stream is represented.  The readme does a 
good job of explaining how it works, but there have been a number of new 
event processing libraries recently (core.async, EEP, etc.), so I'll spend 
some time here describing how this differs.

First, this library is focused on aggregations over event streams, not 
arbitrary transformations.  It is designed such that these aggregations can 
be automatically parallelized, and use non-thread-safe data structures 
(such as those in the excellent stream-lib [3]) without having to worry 
about coordination.  As such, within this narrower application it has a 
richer set of operators, and should be a fair bit faster (millions of 
messages/sec/core).

Second, this has support for time-series analysis of ordered streams, 
either historical or in real time.  The input for either type of analysis 
can be normal sequences, core.async channels, or Lamina channels. At 
Factual we use this for aggregations across many of our real-time systems, 
and I also use it for both ad hoc queries and daily rollups of logs and 
other historical data.

On a personal note, I think this is one of the most interesting and useful 
libraries I've written.  I'm really looking forward to seeing how people 
use it, and encourage feedback on how to make it better.

Zach

[1] http://www.infoq.com/presentations/analyze-running-system
[2] http://vimeo.com/45132054#!
[3] https://github.com/addthis/stream-lib

-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] collection-check: validation for data structure variants

[Zach Tellman]

Yes, this is some very simple sugar atop simple-check, Reid is owed most of
the credit here.
On Oct 30, 2013 10:44 PM, Alex Baranosky alexander.barano...@gmail.com
wrote:

 Thanks Zach for this cool testing library, and thanks Reid for
 simple-check!

 --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to a topic in the
 Google Groups Clojure group.
 To unsubscribe from this topic, visit
 https://groups.google.com/d/topic/clojure/W2oJzZnJ-9c/unsubscribe.
 To unsubscribe from this group and all its topics, send an email to
 clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.


-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[ANN] collection-check: validation for data structure variants

[Zach Tellman]

It's relatively rare that people write variants of Clojure data structures. 
 Partially, this is because the existing data structures are quite good, 
but it's also because it's surprisingly difficult.  Clojure's vectors, 
sets, and maps each implement about a dozen interfaces with overlapping 
functionality, making it easy to think you've written something correct 
until someone accesses it in a way you didn't think to test.  I have 
written a number of alternate data structures [1] [2] [3], and each time 
there's been some lacunae of the standard API that I overlooked.

To make this easier for myself and others, I've written a library for the 
express purpose of validating data structures that are extensions of the 
standard three data 
structures: https://github.com/ztellman/collection-check.  It's already 
uncovered some minor bugs in my own libraries, as well as an issue with 
Clojure's own hash-sets and hash-maps [4].  I strongly encourage anyone 
who's playing around in this space to use it, and to let me know if there 
are invariants I've overlooked.

Enjoy,
Zach

[1] https://github.com/ztellman/clj-tuple
[2] https://github.com/ztellman/immutable-bitset
[3] https://github.com/ztellman/potemkin
[4] https://groups.google.com/forum/#!topic/clojure-dev/HvppNjEH5Qc

-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: streams, reading bytes

[Zach Tellman]

The byte-transforms library will also take care of the gzip stuff for 
you: https://github.com/ztellman/byte-transforms

On Thursday, October 10, 2013 10:26:35 PM UTC-4, Brian Craft wrote:

 I like the blurb:

 This library is a Rosetta stone for all the byte representations Java has 
 to offer, and gives you the freedom to forget all the APIs you never wanted 
 to know in the first place.

 My feelings, exactly. ;)

 Thanks.

 On Thursday, October 10, 2013 6:44:32 PM UTC-7, Ben Mabey wrote:

 On 10/10/13 7:21 PM, Brian Craft wrote: 
  I'm struggling with how to gzip/gunzip byte arrays from clojure. I 
  don't really get how clojure.java.io streams supposed to be used. 
  
  This appears to gzip a string: 
  
  (let [baos (java.io.ByteArrayOutputStream.) 
gzos (java.util.zip.GZIPOutputStream. baos)] 
  (.write gzos (.getBytes foo)) 
  (.finish gzos) 
  (.toByteArray baos)) 
  
  
  and I can expand on this to gunzip the string afterwards: 
  
  (let [baos (java.io.ByteArrayOutputStream.) 
gzos (java.util.zip.GZIPOutputStream. baos)] 
(.write gzos (.getBytes foo)) 
(.finish gzos) 
(let [gzbytes (.toByteArray baos) 
  bais (java.io.ByteArrayInputStream. gzbytes) 
  gzis (java.util.zip.GZIPInputStream. bais)] 
  (slurp gzis))) 
  
  But slurp builds strings. I need to do this with byte arrays.  Do I 
  have to rewrite slurp to build a byte array? Is there any simpler way 
  to do this? 

 Zach Tellman's byte-streams library is very handy for these kinds of 
 conversions: 

 https://github.com/ztellman/byte-streams 

 -Ben 



-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[ANN] Sleight: whole program transformations

[Zach Tellman]

This is a lein plugin which hijacks the Clojure reader and allows all forms 
to be transformed before they're passed to the compiler.  Usage and 
potential applications are described in the readme: 
https://github.com/ztellman/sleight

I wrote the original version of this a year ago, but due to the lack of 
good code walking facilities it was more a sketch than anything else. 
 Luckily, with the recent release of Riddley [1], this is newly useful. 
 I'm looking forward to seeing what people use it for.

Zach

[1] https://github.com/ztellman/riddley

-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: New Functional Programming Job Opportunities

[Zach Tellman]

If anyone has questions about working at Factual, I'm happy to answer them, 
and can be reached at zach at factual.com.  The short version, though, is 
that it's great and you should come work with us.

For some examples of Clojure we're writing and using at Factual, check out:

https://github.com/factual/skuld
https://github.com/factual/clj-leveldb
https://github.com/aphyr/merkle
https://github.com/ztellman/clj-tuple
https://github.com/ztellman/narrator

among many others.

Anyone at Strange Loop who's curious should feel free to track me down. 
 Kyle Kingsbury (better known to some as aphyr) will be giving a talk, 
and will also happily answer any questions about Factual.

Zach

On Sunday, September 15, 2013 11:00:02 PM UTC-7, Sean Murphy wrote:

 Here are some functional programming job opportunities that were posted 
 recently: 

 Clojure Engineers Needed at Factual 
 http://functionaljobs.com/jobs/8646-clojure-engineers-needed-at-factual 

 Cheers, 
 Sean Murphy 
 FunctionalJobs.com 



-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] riddley: code-walking without caveats

[Zach Tellman]

Hi Konrad,

Okay, I think I was just being dense.  I thought you were talking about a
different macroexpansion strategy, rather than just doing full
macroexpansion without interleaved compilation.  Thanks for your patience
in explaining what you meant.

I will note, though, that env is an implicit argument to the macros, so
anything which works exactly like the compiler needs to mimic that as
well.

Zach


On Thu, Sep 5, 2013 at 3:09 AM, Konrad Hinsen 
googlegro...@khinsen.fastmail.net wrote:

 **
 Zach Tellman writes:

   I guess I'm confused, then.  You contrast complete recursive
   expansion with what the compiler does, and then say it's recursive
   prewalk expansion, which is exactly what the compiler does.  Can
   you clarify the difference between what you're doing and what the
   compiler does?

 Here's an example:

(defmacro foo [x]
  `(list ~x ~x))

(defmacro bar [x]
  `[~x ~x])

 Now let's work on the form

(foo (bar 'baz))

 Plain macroexpand returns

(list (bar 'baz) (bar 'baz))

 whereas tools.macro/mexpand-all gives

(list ['baz 'baz] ['baz 'baz])

 It does this by first calling macroexpand, so foo gets called exactly
 as during Clojure compilation and returns

(list (bar 'baz) (bar 'baz))

 mexpand-all then goes through that form and expands the two subforms
 (bar 'baz).

 So mexpand-all does exactly what the compiler does, in particular it
 calls the macros with exactly the same arguments. But the compiler
 interleaves macro expansion with compilation, so it never gives you
 access to the fully expanded but uncompiled form which is

(list ['baz 'baz] ['baz 'baz])

 Konrad

 --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to a topic in the
 Google Groups Clojure group.
 To unsubscribe from this topic, visit
 https://groups.google.com/d/topic/clojure/a68aThpvP4o/unsubscribe.
 To unsubscribe from this group and all its topics, send an email to
 clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.


-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] riddley: code-walking without caveats

[Zach Tellman]

Sorry, that documentation reflected 0.1.1-SNAPSHOT, which I've just
released as 0.1.1.  Let me know if you have any other issues.

Zach


On Thu, Sep 5, 2013 at 6:26 AM, Stathis Sideris side...@gmail.com wrote:

 Thanks for this library Zach,

 It seems that the released version is a bit behind in comparison to the
 generated documentation [1]. For example, walk-exprs is advertised as being
 able to accept a special-forms parameter, but that's not the case in the
 jar that leiningen retrieved when I used [riddley 0.1.0] in my
 project.clj.

 Thanks,

 Stathis

 [1] http://ideolalia.com/riddley/



 On Monday, 2 September 2013 21:49:01 UTC+1, Zach Tellman wrote:

 When I announced Proteus [1], it was rightfully pointed out that it
 didn't play nicely with macros which rely on env, as well as a few forms
 like 'letfn' that I hadn't explicitly handled.  This flaw has been shared
 by pretty much every library of this sort, and since this is a problem I've
 half-solved two or three times already, I figured something more general
 and lasting was in order.

 The resulting library is called Riddley [2].  For obvious reasons, I've
 named it after a book which is written entirely in a barely-readable pidgin
 dialect. While there may be lingering issues, it's good enough to replace
 the code-walking mechanism in Proteus, which I think makes it the best game
 in town right now.  Bug reports and pull requests are welcome.

 Zach

 [1] https://groups.google.com/**forum/#!searchin/clojure/**
 proteus/clojure/7HNNiJJTte4/**iMBWn8p6tZAJhttps://groups.google.com/forum/#!searchin/clojure/proteus/clojure/7HNNiJJTte4/iMBWn8p6tZAJ
 [2] 
 https://github.com/**ztellman/riddleyhttps://github.com/ztellman/riddley

  --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to a topic in the
 Google Groups Clojure group.
 To unsubscribe from this topic, visit
 https://groups.google.com/d/topic/clojure/a68aThpvP4o/unsubscribe.
 To unsubscribe from this group and all its topics, send an email to
 clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.


-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] riddley: code-walking without caveats

[Zach Tellman]

Hi Nils,

Thanks for the link, I hadn't seen that before.  Happily, what you describe
is pretty much exactly what Riddley does, in this case by accessing the
compiler internals.  An example of how macrolet or symbol-macrolet could be
implemented using this is linked above in my response to Konrad.  This
means that macros which rely on the clojure.lang.Compiler$LocalBinding
(which includes [1] and [2]) will still work.  As far as I can tell, this
is not the case in either tools.macro or core.async.

If you know of any ways to make the code-walking more complete, please let
me know.

Zach

[1] https://github.com/ztellman/proteus/blob/master/src/proteus.clj#L60
[2]
https://github.com/flatland/useful/blob/develop/src/flatland/useful/datatypes.clj#L62


On Thu, Sep 5, 2013 at 3:31 AM, bertschi
nils.bertschin...@googlemail.comwrote:

 Hi Zach,

 you might want to look at this paper explaining how to write a correct
 macroexpand-all (which requires a code walker) in Common Lisp:
 http://www.merl.com/publications/TR1993-017/

 The compiler certainly has to do something like that, but might not do all
 of the macroexpansion before starting any compilation as Konrad explained.
 What the compiler needs to do is track the lexical environment while
 walking down the source forms. When a code walker wants to introduce
 additional bindings, such as macrolet (for local macros) or symbol-macrolet
 (for new symbols) it needs to be able to extend the environment
 accordingly. So, you either have to access the compiler internals,
 especially its environment handling, or track the environment yourself (as
 Konrad suggested).
 As an aside: The problem in Common Lisp is mainly that the environment
 handling is not exposed in the standard, thus you cannot write a portable
 code walker without doing some environment handling yourself.

 You might also want to look at core.async, which uses a code walker to
 transform go blocks into state machines. I have not (yet) checked its
 restrictions (someone told me, that it cannot even look into anonymous fn
 forms within its body!), but it is generally very hard to write a code
 walker that can handle all special forms (in Common Lisp I don't know any).

 +10 for having a library that supports writing correct and (almost)
 complete code walkers

 Best,

Nils

 On Thursday, September 5, 2013 12:09:28 PM UTC+2, Konrad Hinsen wrote:

 Zach Tellman writes:

   I guess I'm confused, then.  You contrast complete recursive
   expansion with what the compiler does, and then say it's recursive
   prewalk expansion, which is exactly what the compiler does.  Can
   you clarify the difference between what you're doing and what the
   compiler does?

 Here's an example:

(defmacro foo [x]
  `(list ~x ~x))

(defmacro bar [x]
  `[~x ~x])

 Now let's work on the form

(foo (bar 'baz))

 Plain macroexpand returns

(list (bar 'baz) (bar 'baz))

 whereas tools.macro/mexpand-all gives

(list ['baz 'baz] ['baz 'baz])

 It does this by first calling macroexpand, so foo gets called exactly
 as during Clojure compilation and returns

(list (bar 'baz) (bar 'baz))

 mexpand-all then goes through that form and expands the two subforms
 (bar 'baz).

 So mexpand-all does exactly what the compiler does, in particular it
 calls the macros with exactly the same arguments. But the compiler
 interleaves macro expansion with compilation, so it never gives you
 access to the fully expanded but uncompiled form which is

(list ['baz 'baz] ['baz 'baz])

 Konrad

  --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to a topic in the
 Google Groups Clojure group.
 To unsubscribe from this topic, visit
 https://groups.google.com/d/topic/clojure/a68aThpvP4o/unsubscribe.
 To unsubscribe from this group and all its topics, send an email to
 clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.


-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com

Re: Handling name collisions with clojure.core

[Zach Tellman]

Not doing actual replacement via code-walking will prevent any functions
with inline definitions actually being able to benefit from this.  I'm not
sure if those are used in core.matrix, though.


On Thu, Sep 5, 2013 at 4:54 AM, Colin Fleming
colin.mailingl...@gmail.comwrote:

 This is actually probably not a bad solution. You wouldn't even need to
 rewrite, couldn't you just expand to a let?

 (let [* clojure.core.matrix.*
   + clojure.core.matrix.+]
   (+ ... (* ...)))

 Although thinking about it, you'd have to let-bind all possible operators
 every time, and the compiler doesn't do any dead-code elimination (although
 Hotspot might).


 On 5 September 2013 17:35, Dave Ray dave...@gmail.com wrote:

 Maybe this is a dumb idea, but could you have a macro that rewrites code
 to use your ops?

   (require '[clojure.core.matrix :as m])
   (m/with-ops (+ ... (* ...) ...))

 and then all the special symbols get rewritten/qualified with
 clojure.core.matrix?

 Dave



 On Wed, Sep 4, 2013 at 10:26 PM, Sean Corfield seancorfi...@gmail.comwrote:

 You only get the warning if you 'use' the namespace or 'refer all'
 tho', correct?

 And we've recently seen a lot of discussion that basically says don't
 do that so it seems that either users of core.matric are going to
 have two approved choices:
 * require core.matrix with an alias, or choose to rename colliding
 names however you want
 * exclude the colliding symbols via refer-clojure and require them
 from core.matrix as referred symbols

 That's seems right to me: it is explicit and provides no surprises; it
 gives the user control over how to manage things.

 Sean

 On Wed, Sep 4, 2013 at 6:22 PM, Mikera mike.r.anderson...@gmail.com
 wrote:
  Hi all,
 
  While building the API for core.matrix, I've fun into a few cases
 where the
  best name is a direct clash with clojure.core.
 
  Examples are +, zero?, vector?, ==
 
  In many of these cases, the core.matrix behaviour is a natural
 extension of
  the clojure.core function (i.e. it extends the same functionality to
  arbitrary N-dimensional arrays).
 
  I'm not very happy with any of the options I can see for handling this:
 
  A) Use the good names in the clojure.core.matrix namespace. Problem:
 that
  gives you a ton of nasty warnings of the type WARNING: + already
 refers to:
  #'clojure.core/+ in namespace: test.blank, being replaced by:
  #'clojure.core.matrix/+. Significant boilerplate must be maintained
 by the
  user in their ns declaration to prevent these warnings. I don't like
 forcing
  users to maintain boilerplate, and I think that normal idiomatic usage
  should be warning-free.
 
  B) Separate the name-clashing functions into separate namespaces - e.g.
  clojure.core.matrix.operators. Problem: that's something of an
 artificial
  division, and again it forces users to do extra ns-management work to
 access
  the functions they want.
 
  C) Use different names. Problem: names would be worse, and this would
 be
  inconsistent and confusing, especially for functions that do
 effectively the
  same thing.
 
  D) Encourage users to use aliases. Problem: that's horrendously ugly
 and
  inconvenient for numerical code. Users with any sense of elegance in
 their
  coding style would quite rightly throw their hands up in disgust.
 
  Currently we're doing B), I'd prefer to do A) but can't figure out a
 way to
  automatically suppress the warnings.
 
  Any better ideas?
 
 
 
  --
  --
  You received this message because you are subscribed to the Google
  Groups Clojure group.
  To post to this group, send email to clojure@googlegroups.com
  Note that posts from new members are moderated - please be patient
 with your
  first post.
  To unsubscribe from this group, send email to
  clojure+unsubscr...@googlegroups.com
  For more options, visit this group at
  http://groups.google.com/group/clojure?hl=en
  ---
  You received this message because you are subscribed to the Google
 Groups
  Clojure group.
  To unsubscribe from this group and stop receiving emails from it, send
 an
  email to clojure+unsubscr...@googlegroups.com.
  For more options, visit https://groups.google.com/groups/opt_out.



 --
 Sean A Corfield -- (904) 302-SEAN
 An Architect's View -- http://corfield.org/
 World Singles, LLC. -- http://worldsingles.com/

 Perfection is the enemy of the good.
 -- Gustave Flaubert, French realist novelist (1821-1880)

 --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send
 

Re: [ANN] riddley: code-walking without caveats

[Zach Tellman]

I'm not sure what you mean by complete recursive expansion.  Could you
expand on that?

As for replicating the behavior of the compiler, I'd assert that unless
env is precisely what it would be without ahead of time macroexpansion,
the compiler's behavior isn't being replicated.  The tools.macro library
emulates an aspect of its behavior, certainly, and the fact that Clojure's
existed this long without anyone doing something like this indicates that
maybe this isn't such a huge omission, but without there remains an uncanny
valley.


On Wed, Sep 4, 2013 at 12:09 AM, Konrad Hinsen 
googlegro...@khinsen.fastmail.net wrote:

 Zach Tellman writes:

   I see.  This is honestly something I hadn't considered, but since
   Riddley actually uses the Clojure compiler internals to track
   locals, this would be as simple as a (when-not (contains?
   (riddley.compiler/locals) (first expr)) ...) guard in the
   macroexpansion.

 If you don't need complete recursive expansion, that's indeed an
 approach worth exploring. For tools.macros that's not an option
 because the compiler knows nothing about local macros and symbol
 macros.

   As Ben points out, using the compiler this way is the only way to
   make sure that locals are consistent everywhere, rather than just
   in your own targeted use to track shadowing.

 Well, either you use the compiler or you replicate what it does.  For
 tools.macro I had to choose the second approach. I don't claim it has
 no bugs, I just claim I haven't had any bug reports ;-) (until today
 at least).

 Konrad

 --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to a topic in the
 Google Groups Clojure group.
 To unsubscribe from this topic, visit
 https://groups.google.com/d/topic/clojure/a68aThpvP4o/unsubscribe.
 To unsubscribe from this group and all its topics, send an email to
 clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.


-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] riddley: code-walking without caveats

[Zach Tellman]

So complete recursive expansion is postwalk macroexpansion?  It seems
like that could break anaphoric macros, and likely others.  A macro has the
option of calling macroexpand-all on its own contents if it wants only
special forms, but it shouldn't be forced to take only special forms.

Also, here's a sketch of how you could do symbol macros using Riddley:
https://gist.github.com/ztellman/6439318.  Please let me know if I'm
missing something w.r.t. how symbol macros are done in tools.macros.

Zach


On Wed, Sep 4, 2013 at 2:16 AM, Konrad Hinsen 
googlegro...@khinsen.fastmail.net wrote:

 **
 On Wed, Sep 4, 2013, at 09:25 AM, Zach Tellman wrote:

 I'm not sure what you mean by complete recursive expansion.  Could
 you expand
 on that?

 Completely ;-)

 By complete recursive expansion I mean that you get a form that is fully
 reduced to
 the core language, i.e. it contains no more macro applications at any
 level.

 If you leave macro expansion to the compiler, it does it when it arrives
 at the
 macro during evaluation. Then it does a plain non-recursive macroexpand
 and goes on
 evaluating. Any macro thus has access to the unexpanded contents of its
 form, but
 not to what it eventually expands to. For many applications that's just
 fine, which
 is why this approach has been the default in the Lisp world for a long
 time.

 As for replicating the behavior of the compiler, I'd assert that
 unless env is
 precisely what it would be without ahead of time macroexpansion, the
 compiler's
 behavior isn't being replicated.

 I agree. tools.macro predates env, which is why it is not supported.
 Since I have
 never need env support and nobody ever asked for it (before now), it's
 not there.
 I don't see any reason why it couldn't be supported.

 Konrad.


 --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to a topic in the
 Google Groups Clojure group.
 To unsubscribe from this topic, visit
 https://groups.google.com/d/topic/clojure/a68aThpvP4o/unsubscribe.
 To unsubscribe from this group and all its topics, send an email to
 clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.


-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] riddley: code-walking without caveats

[Zach Tellman]

Actually, postwalk expansion (if that is in fact what you were describing)
would ignore any binding forms created by the outer macro.  This means that
something simple like:

(defmacro with-db [db  body]
  `(with-open [~db (create-db)]
 ~@body))

would be expanded without any knowledge of the 'db' local variable, since
that would only get turned into a let form later.  Prewalk is pretty much
the only way this works.


On Wed, Sep 4, 2013 at 9:50 AM, Ben Wolfson wolf...@gmail.com wrote:

 Postwalk expansion would break macros that inspect their argument forms
 for e.g. writing special-purpose queries, if they *also* adopt the symbols
 and and or for conjunction or disjunction. Korma's where, for
 instance, does this; one can write

 (select my-table (where (and (...) (...

 And the where detects the and.

 Arguably this is wrongheaded behavior from the get-go (it can be somewhat
 confusing and makes it necessary to use something like clojure.core/and
 within where if you want normal clojure-land and semantics), but it's a
 style of non-anaphoric macro that relies on receiving an unexpanded form.


 On Wed, Sep 4, 2013 at 9:27 AM, Zach Tellman ztell...@gmail.com wrote:

 So complete recursive expansion is postwalk macroexpansion?  It seems
 like that could break anaphoric macros, and likely others.  A macro has the
 option of calling macroexpand-all on its own contents if it wants only
 special forms, but it shouldn't be forced to take only special forms.

 Also, here's a sketch of how you could do symbol macros using Riddley:
 https://gist.github.com/ztellman/6439318.  Please let me know if I'm
 missing something w.r.t. how symbol macros are done in tools.macros.

 Zach


 On Wed, Sep 4, 2013 at 2:16 AM, Konrad Hinsen 
 googlegro...@khinsen.fastmail.net wrote:

 **
 On Wed, Sep 4, 2013, at 09:25 AM, Zach Tellman wrote:

 I'm not sure what you mean by complete recursive expansion.  Could
 you expand
 on that?

 Completely ;-)

 By complete recursive expansion I mean that you get a form that is fully
 reduced to
 the core language, i.e. it contains no more macro applications at any
 level.

 If you leave macro expansion to the compiler, it does it when it arrives
 at the
 macro during evaluation. Then it does a plain non-recursive macroexpand
 and goes on
 evaluating. Any macro thus has access to the unexpanded contents of its
 form, but
 not to what it eventually expands to. For many applications that's just
 fine, which
 is why this approach has been the default in the Lisp world for a long
 time.

 As for replicating the behavior of the compiler, I'd assert that
 unless env is
 precisely what it would be without ahead of time macroexpansion, the
 compiler's
 behavior isn't being replicated.

 I agree. tools.macro predates env, which is why it is not supported.
 Since I have
 never need env support and nobody ever asked for it (before now), it's
 not there.
 I don't see any reason why it couldn't be supported.

 Konrad.


 --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to a topic in the
 Google Groups Clojure group.
 To unsubscribe from this topic, visit
 https://groups.google.com/d/topic/clojure/a68aThpvP4o/unsubscribe.
 To unsubscribe from this group and all its topics, send an email to
 clojure+unsubscr...@googlegroups.com.

 For more options, visit https://groups.google.com/groups/opt_out.


  --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.

 For more options, visit https://groups.google.com/groups/opt_out.




 --
 Ben Wolfson
 Human kind has used its intelligence to vary the flavour of drinks, which
 may be sweet, aromatic, fermented or spirit-based. ... Family and social
 life also offer numerous other occasions to consume drinks for pleasure.
 [Larousse, Drink entry]

  --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members

Re: Handling name collisions with clojure.core

[Zach Tellman]

It is probably instructive to look at how (use-primitive-operators) works 
in primitive-math [1], though maybe not something you want to emulate.  The 
basic mechanism is pretty simple: use 'ns-unmap' to get rid of the 
operators you want to shadow, and bring in the operators from the alternate 
namespace.

Unfortunately, the next time you load the namespace, you'll get the same 
mess of collision warnings you were trying to avoid, because you haven't 
added the ':refer-clojure :exclude' clause to the ns declaration.  To get 
around this, use-primitive-operator hijacks 'ns' via alter-var-root so that 
this is automatically added if the alternate operators are detected [2], 
but otherwise leaves it unchanged.  This hijacking is undone if 
clojure.core is ever reloaded (usually by a :reload-all somewhere), 
however.  This could be fixed by adding a watcher to #'clojure.core/ns and 
forcing it back whenever it changes, but I haven't done that yet because it 
doesn't really affect me, and it feels hokey enough already.

Despite all that, though, it works pretty well.  Feel free to use this 
approach if you like, or create a less questionable variant if you can 
think of one.

Zach

[1] 
https://github.com/ztellman/primitive-math/blob/master/src/primitive_math.clj#L154
[2] 
https://github.com/ztellman/primitive-math/blob/master/src/primitive_math.clj#L135

On Wednesday, September 4, 2013 6:22:08 PM UTC-7, Mikera wrote:

 Hi all,

 While building the API for core.matrix, I've fun into a few cases where 
 the best name is a direct clash with clojure.core.

 Examples are +, zero?, vector?, ==

 In many of these cases, the core.matrix behaviour is a natural extension 
 of the clojure.core function (i.e. it extends the same functionality to 
 arbitrary N-dimensional arrays). 

 I'm not very happy with any of the options I can see for handling this:

 A) Use the good names in the clojure.core.matrix namespace. Problem: 
 that gives you a ton of nasty warnings of the type WARNING: + already 
 refers to: #'clojure.core/+ in namespace: test.blank, being replaced by: 
 #'clojure.core.matrix/+. Significant boilerplate must be maintained by the 
 user in their ns declaration to prevent these warnings. I don't like 
 forcing users to maintain boilerplate, and I think that normal idiomatic 
 usage should be warning-free.

 B) Separate the name-clashing functions into separate namespaces - e.g. 
 clojure.core.matrix.operators. Problem: that's something of an artificial 
 division, and again it forces users to do extra ns-management work to 
 access the functions they want.

 C) Use different names. Problem: names would be worse, and this would be 
 inconsistent and confusing, especially for functions that do effectively 
 the same thing.

 D) Encourage users to use aliases. Problem: that's horrendously ugly and 
 inconvenient for numerical code. Users with any sense of elegance in their 
 coding style would quite rightly throw their hands up in disgust. 

 Currently we're doing B), I'd prefer to do A) but can't figure out a way 
 to automatically suppress the warnings.

 Any better ideas?





-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] riddley: code-walking without caveats

[Zach Tellman]

Hey Konrad, you can maybe speak with more authority as to what tools.macro
does and doesn't provide, but my reading of it is that it does expression
walking to prevent bound variables from being incorrectly
symbol-macroexpanded.  This seems only important in the context of symbol
macros, however; if you don't use symbol macros it's functionally
equivalent to clojure.walk/macroexpand-all.

This means that it suffers from all the same issues mentioned in Riddley's
readme, namely no env and no expansion of inlined functions.  The code
walking is also only used to do expansion, no generic code walking
mechanism is exposed for more general transformations a la Proteus.

Hope that helps,
Zach


On Mon, Sep 2, 2013 at 11:20 PM, Konrad Hinsen 
googlegro...@khinsen.fastmail.net wrote:

 --On 2 septembre 2013 13:49:01 -0700 Zach Tellman ztell...@gmail.com
 wrote:

  The resulting library is called Riddley [2].  For obvious reasons, I've
 named it after a book which is written entirely in a barely-readable
 pidgin dialect. While there may be lingering issues, it's good enough to
 replace the code-walking mechanism in Proteus, which I think makes it the
 best game in town right now.  Bug reports and pull requests are welcome.


 How does this compare to mexpand-all in clojure.tools.macro?

 Konrad.


 --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscribe@**googlegroups.comclojure%2bunsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/**group/clojure?hl=enhttp://groups.google.com/group/clojure?hl=en
 --- You received this message because you are subscribed to a topic in the
 Google Groups Clojure group.
 To unsubscribe from this topic, visit https://groups.google.com/d/**
 topic/clojure/a68aThpvP4o/**unsubscribehttps://groups.google.com/d/topic/clojure/a68aThpvP4o/unsubscribe
 .
 To unsubscribe from this group and all its topics, send an email to
 clojure+unsubscribe@**googlegroups.comclojure%2bunsubscr...@googlegroups.com
 .
 For more options, visit 
 https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out
 .


-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] riddley: code-walking without caveats

[Zach Tellman]

I see.  This is honestly something I hadn't considered, but since Riddley
actually uses the Clojure compiler internals to track locals, this would be
as simple as a (when-not (contains? (riddley.compiler/locals) (first expr))
...) guard in the macroexpansion.  As Ben points out, using the compiler
this way is the only way to make sure that locals are consistent
everywhere, rather than just in your own targeted use to track shadowing.

Hope that helps,
Zach


On Tue, Sep 3, 2013 at 12:41 PM, Konrad Hinsen 
googlegro...@khinsen.fastmail.net wrote:

 --On 3 septembre 2013 02:08:23 -0700 Zach Tellman ztell...@gmail.com
 wrote:

  Hey Konrad, you can maybe speak with more authority as to what
 tools.macro does and doesn't provide, but my reading of it is that it
 does expression walking to prevent bound variables from being incorrectly
 symbol-macroexpanded.  This seems only important in the context of
 symbol macros, however; if you don't use symbol macros it's functionally
 equivalent to clojure.walk/macroexpand-all.


 Not quite. It expands only terms that are evaluated, using a built-in
 table of special forms, and it allows local macro definitions (macrolet).
 But most importantly, it tracks local bindings and expands only macros that
 are not shadowed. So if you have

  (defmacro foo [] ...)
  (let [foo (fn [] ...)]
 (foo 'bar))

 the form (foo 'bar) is not expanded because its local binding is a
 function. The version in clojure.walk doesn't take this into account, and
 can therefore produce incorrect code, which is a major pain to debug. I
 know because it happened to me, that's why I ended up writing my own macro
 expander. And that's why I wonder how riddley handled this.


 Konrad.

 --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscribe@**googlegroups.comclojure%2bunsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/**group/clojure?hl=enhttp://groups.google.com/group/clojure?hl=en
 --- You received this message because you are subscribed to a topic in the
 Google Groups Clojure group.
 To unsubscribe from this topic, visit https://groups.google.com/d/**
 topic/clojure/a68aThpvP4o/**unsubscribehttps://groups.google.com/d/topic/clojure/a68aThpvP4o/unsubscribe
 .
 To unsubscribe from this group and all its topics, send an email to
 clojure+unsubscribe@**googlegroups.comclojure%2bunsubscr...@googlegroups.com
 .
 For more options, visit 
 https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out
 .


-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[ANN] clj-leveldb, idiomatic LevelDB bindings from Factual

[Zach Tellman]

In the first of what I hope will be many annoucements, we're open sourcing 
a library we've found useful at Factual: 
https://github.com/Factual/clj-leveldb.  This is just a simple wrapper 
around LevelDB, which is an in-process persistent k/v store from Google.

If anyone has questions, I'm happy to answer them.

Zach

-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[ANN] riddley: code-walking without caveats

[Zach Tellman]

When I announced Proteus [1], it was rightfully pointed out that it didn't 
play nicely with macros which rely on env, as well as a few forms like 
'letfn' that I hadn't explicitly handled.  This flaw has been shared by 
pretty much every library of this sort, and since this is a problem I've 
half-solved two or three times already, I figured something more general 
and lasting was in order.

The resulting library is called Riddley [2].  For obvious reasons, I've 
named it after a book which is written entirely in a barely-readable pidgin 
dialect. While there may be lingering issues, it's good enough to replace 
the code-walking mechanism in Proteus, which I think makes it the best game 
in town right now.  Bug reports and pull requests are welcome.

Zach

[1] 
https://groups.google.com/forum/#!searchin/clojure/proteus/clojure/7HNNiJJTte4/iMBWn8p6tZAJ
[2] https://github.com/ztellman/riddley

-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: ANN: clj-tuple, efficient small collections

[Zach Tellman]

I've updated the library so tuples behave like vectors, rather than lists. 
 The macros have gotten kind of elephantine, but since I've already added 
every conceivable Clojure interface at least they won't get any bigger.  If 
there are any lingering gaps, though, please let me know.

Zach 

On Saturday, August 24, 2013 7:38:25 PM UTC-7, Zach Tellman wrote:

 I just pulled some code I wrote while trying to optimize 'memoize' into 
 its own library: https://github.com/ztellman/clj-tuple.  It only has the 
 one function, so I assume no one will need too much explanation.  However, 
 there may still be room for performance improvements, so if anyone wants to 
 take a stab, pull requests are welcome.

 Zach


-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: ANN: clj-tuple, efficient small collections

[Zach Tellman]

I'm not sure I understand the motivation behind what you're suggesting.
 Why not just (nth tuple 0)?


On Mon, Aug 26, 2013 at 10:38 AM, Kevin Downey redc...@gmail.com wrote:

 A Tuple protocol that defines get0 get1 get3 etc for fast element access
 that doesn't tie you to using field names might be a good idea.

 On 8/25/13 9:35 AM, Zach Tellman wrote:
  I don't think so, even the existence of all the Tuple* types are an
  implementation detail, and you'd need to hint it as the right one to get
  sane performance.  (nth t n) has good performance, you should prefer
 that.
 
  On Saturday, August 24, 2013 8:15:40 PM UTC-7, Ben wrote:
 
  Are the element names .e0, .e1 etc. considered part of the public
  interface of tuple objects?
 
 
  On Sat, Aug 24, 2013 at 7:38 PM, Zach Tellman ztel...@gmail.com
 javascript:
  wrote:
 
  I just pulled some code I wrote while trying to optimize 'memoize' into
  its own library: https://github.com/ztellman/clj-tuple.  It only has
 the
  one function, so I assume no one will need too much explanation.
  However,
  there may still be room for performance improvements, so if anyone
 wants to
  take a stab, pull requests are welcome.
 
  Zach
 
  --
  --
  You received this message because you are subscribed to the Google
  Groups Clojure group.
  To post to this group, send email to clo...@googlegroups.com
 javascript:
  Note that posts from new members are moderated - please be patient with
  your first post.
  To unsubscribe from this group, send email to
  clojure+u...@googlegroups.com javascript:
  For more options, visit this group at
  http://groups.google.com/group/clojure?hl=en
  ---
  You received this message because you are subscribed to the Google
 Groups
  Clojure group.
  To unsubscribe from this group and stop receiving emails from it, send
 an
  email to clojure+u...@googlegroups.com javascript:.
  For more options, visit https://groups.google.com/groups/opt_out.
 
 
 
 
  --
  Ben Wolfson
  Human kind has used its intelligence to vary the flavour of drinks,
 which
  may be sweet, aromatic, fermented or spirit-based. ... Family and social
  life also offer numerous other occasions to consume drinks for
 pleasure.
  [Larousse, Drink entry]
 
 
 


 --
 And what is good, Phaedrus,
 And what is not good—
 Need we ask anyone to tell us these things?



-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: ANN: clj-tuple, efficient small collections

[Zach Tellman]

I don't think so, even the existence of all the Tuple* types are an 
implementation detail, and you'd need to hint it as the right one to get 
sane performance.  (nth t n) has good performance, you should prefer that.

On Saturday, August 24, 2013 8:15:40 PM UTC-7, Ben wrote:

 Are the element names .e0, .e1 etc. considered part of the public 
 interface of tuple objects?


 On Sat, Aug 24, 2013 at 7:38 PM, Zach Tellman ztel...@gmail.comjavascript:
  wrote:

 I just pulled some code I wrote while trying to optimize 'memoize' into 
 its own library: https://github.com/ztellman/clj-tuple.  It only has the 
 one function, so I assume no one will need too much explanation.  However, 
 there may still be room for performance improvements, so if anyone wants to 
 take a stab, pull requests are welcome.

 Zach

 -- 
 -- 
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.comjavascript:
 Note that posts from new members are moderated - please be patient with 
 your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com javascript:
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 --- 
 You received this message because you are subscribed to the Google Groups 
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to clojure+u...@googlegroups.com javascript:.
 For more options, visit https://groups.google.com/groups/opt_out.




 -- 
 Ben Wolfson
 Human kind has used its intelligence to vary the flavour of drinks, which 
 may be sweet, aromatic, fermented or spirit-based. ... Family and social 
 life also offer numerous other occasions to consume drinks for pleasure. 
 [Larousse, Drink entry]

  

-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: ANN: clj-tuple, efficient small collections

[Zach Tellman]

in my microbenchmarks I've found it to be consistently faster to unroll a 
fixed number of elements rather than iterate over them.  The difference 
might not be large enough to matter for many people's use cases, but the 
stated goal is to make a fast collection, so it's worthwhile to use the 
best known approach.

Currently tuples mimic lists, so conj adds onto the beginning, and does not 
return another tuple.  'rest', however, does return a tuple.  Also, since 
my intent wasn't to emulate vectors, 'get', 'assoc', and calling as 
function don't work with tuples.  This could obviously change, but I'd be 
curious to know why it makes a difference in someone's code.

Also, I played around with returning both a list or a vector in the 
unbounded case, but the performance is too different in each of these 
cases.  As I point out in the readme, vectors are *significantly* slower to 
create, which would make the performance of 'tuple' too inconsistent for my 
liking.  The TupleN solution I came up with was the best compromise I could 
think of.

Zach



On Saturday, August 24, 2013 10:47:11 PM UTC-7, Jozef Wagner wrote:

 Hi,

 I've did something similar for CLJS, see 
 http://dev.clojure.org/jira/browse/CLJS-453 and 
 https://groups.google.com/forum/#!searchin/clojure/arrayvector/clojure/yDvTRP0gYLA/Tb5MJC8Z0K8J

 In CLJS, it is comparable in performance to use array instead of 
 specialized type for each arity. Did you check if it is not the same in 
 Clojure?

 If I conjoin to the tuple, does it return a tuple too? Also instead of 
 tupleN, you could fall back to the PersistentVector.

 JW


 On Sun, Aug 25, 2013 at 4:38 AM, Zach Tellman ztel...@gmail.comjavascript:
  wrote:

 I just pulled some code I wrote while trying to optimize 'memoize' into 
 its own library: https://github.com/ztellman/clj-tuple.  It only has the 
 one function, so I assume no one will need too much explanation.  However, 
 there may still be room for performance improvements, so if anyone wants to 
 take a stab, pull requests are welcome.

 Zach

 -- 
 -- 
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.comjavascript:
 Note that posts from new members are moderated - please be patient with 
 your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com javascript:
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 --- 
 You received this message because you are subscribed to the Google Groups 
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to clojure+u...@googlegroups.com javascript:.
 For more options, visit https://groups.google.com/groups/opt_out.




-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

ANN: clj-tuple, efficient small collections

[Zach Tellman]

I just pulled some code I wrote while trying to optimize 'memoize' into its 
own library: https://github.com/ztellman/clj-tuple.  It only has the one 
function, so I assume no one will need too much explanation.  However, 
there may still be room for performance improvements, so if anyone wants to 
take a stab, pull requests are welcome.

Zach

-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] Vertigo: fast, idiomatic C-style structs

[Zach Tellman]

Hi Ezra,

This is admittedly a little confusing, but you're hinting 's' with the type
of the *element*.  Here you've created a sequence containing a single
'ints-and-floats' struct, so you'd want to do this:

user (v/get-in s [0 :floats 4])
4.0
user (v/get-in s [0])
{:ints (0 1 2 3 4 5 6 7 8 9), :floats (0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0
9.0)}

I'll try to make this clearer in the documentation.  Let me know if you
have any other questions.

Zach


On Wed, Jul 31, 2013 at 10:10 AM, Ezra Lee lee.e...@gmail.com wrote:

 Hi,
 I'm trying out vertigo and hoping you can help me figure out what I am
 missing, I get an error when I use get-in:

 ; nREPL 0.1.8-preview
 user (use 'vertigo.structs)
 nil
 user (def-typed-struct ints-and-floats :ints (array uint32 10) :floats
 (array float32 10))
 #'user/ints-and-floats
 user (def x {:ints (range 10) :floats (map float (range 10))})
 #'user/x
 user (require '[vertigo.core :as v])
 nil
 user (def ^:ints-and-floats s (v/marshal-seq ints-and-floats [x]))
 #'user/s
 user (v/get-in s [:floats 4])
 IllegalArgumentException Invalid field '4' for type ints-and-floats
  vertigo.core/validate-lookup (core.clj:177)
 user (v/get-in s [4 :floats])
 IllegalArgumentException   java.nio.Buffer.position (Buffer.java:216)

 Thanks,
 Ezra

 On Tuesday, July 9, 2013 11:56:03 PM UTC-4, Zach Tellman wrote:

 Last year, I gave a talk at the Conj on my attempt to write an AI for the
 board game Go.  Two things I discovered is that it was hard to get
 predictable performance, but even once I made sure I had all the right type
 hints, there was still a lot of room at the bottom for performance
 improvements.  Towards the end [1], I mentioned a few ideas for
 improvements, one of which was simply using ByteBuffers rather than objects
 to host the data.  This would remove all the levels of indirection, giving
 much better cache coherency, and also allow for fast unsynchronized
 mutability when the situation called for it.

 So, ten months and several supporting libraries [2] [3] later, here it
 is: 
 https://github.com/**ztellman/vertigohttps://github.com/ztellman/vertigo

 At a high level, this library is useful whenever your datatype has a
 fixed layout and is used more than once.  Depending on your type, it will
 give you moderate to large memory savings, and if you're willing to forgo
 some of core library in favor of Vertigo's operators, you can get
 significant performance gains on batch operations.  And, in the cases where
 performance doesn't matter, it will behave exactly like any other Clojure
 data structure.

 I want to point out that something like this would be more or less
 impossible in Java; reading from an offset in a ByteBuffer without the
 compile-time inference and validation provided by this library would be
 pointlessly risky.  There's not a lot of low-level Clojure libraries, but
 there's an increasing amount of production usage where people are using
 Clojure for performance-sensitive work.  I'm looking forward to seeing what
 people do with Vertigo and libraries like it.

 Zach

 [1] http://www.youtube.com/**watch?feature=player_**
 detailpagev=v5dYE0CMmHQ#t=**1828shttp://www.youtube.com/watch?feature=player_detailpagev=v5dYE0CMmHQ#t=1828s
 [2] 
 https://github.com/ztellman/**primitive-mathhttps://github.com/ztellman/primitive-math
 [3] 
 https://github.com/**ztellman/byte-streamshttps://github.com/ztellman/byte-streams

  --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to a topic in the
 Google Groups Clojure group.
 To unsubscribe from this topic, visit
 https://groups.google.com/d/topic/clojure/BayfuaqMzvs/unsubscribe.
 To unsubscribe from this group and all its topics, send an email to
 clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.




-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] Vertigo: fast, idiomatic C-style structs

[Zach Tellman]

Actually, looking at the readme, I can see the code you were trying to use.
 Sorry, I'm not sure how I didn't catch that before, but I've fixed it.

Zach


On Wed, Jul 31, 2013 at 10:17 AM, Zach Tellman ztell...@gmail.com wrote:

 Hi Ezra,

 This is admittedly a little confusing, but you're hinting 's' with the
 type of the *element*.  Here you've created a sequence containing a single
 'ints-and-floats' struct, so you'd want to do this:

 user (v/get-in s [0 :floats 4])
 4.0
 user (v/get-in s [0])
 {:ints (0 1 2 3 4 5 6 7 8 9), :floats (0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0
 9.0)}

 I'll try to make this clearer in the documentation.  Let me know if you
 have any other questions.

 Zach


 On Wed, Jul 31, 2013 at 10:10 AM, Ezra Lee lee.e...@gmail.com wrote:

 Hi,
 I'm trying out vertigo and hoping you can help me figure out what I am
 missing, I get an error when I use get-in:

 ; nREPL 0.1.8-preview
 user (use 'vertigo.structs)
 nil
 user (def-typed-struct ints-and-floats :ints (array uint32 10) :floats
 (array float32 10))
 #'user/ints-and-floats
 user (def x {:ints (range 10) :floats (map float (range 10))})
 #'user/x
 user (require '[vertigo.core :as v])
 nil
 user (def ^:ints-and-floats s (v/marshal-seq ints-and-floats [x]))
 #'user/s
 user (v/get-in s [:floats 4])
 IllegalArgumentException Invalid field '4' for type ints-and-floats
  vertigo.core/validate-lookup (core.clj:177)
 user (v/get-in s [4 :floats])
 IllegalArgumentException   java.nio.Buffer.position (Buffer.java:216)

 Thanks,
 Ezra

 On Tuesday, July 9, 2013 11:56:03 PM UTC-4, Zach Tellman wrote:

 Last year, I gave a talk at the Conj on my attempt to write an AI for
 the board game Go.  Two things I discovered is that it was hard to get
 predictable performance, but even once I made sure I had all the right type
 hints, there was still a lot of room at the bottom for performance
 improvements.  Towards the end [1], I mentioned a few ideas for
 improvements, one of which was simply using ByteBuffers rather than objects
 to host the data.  This would remove all the levels of indirection, giving
 much better cache coherency, and also allow for fast unsynchronized
 mutability when the situation called for it.

 So, ten months and several supporting libraries [2] [3] later, here it
 is: 
 https://github.com/**ztellman/vertigohttps://github.com/ztellman/vertigo

 At a high level, this library is useful whenever your datatype has a
 fixed layout and is used more than once.  Depending on your type, it will
 give you moderate to large memory savings, and if you're willing to forgo
 some of core library in favor of Vertigo's operators, you can get
 significant performance gains on batch operations.  And, in the cases where
 performance doesn't matter, it will behave exactly like any other Clojure
 data structure.

 I want to point out that something like this would be more or less
 impossible in Java; reading from an offset in a ByteBuffer without the
 compile-time inference and validation provided by this library would be
 pointlessly risky.  There's not a lot of low-level Clojure libraries, but
 there's an increasing amount of production usage where people are using
 Clojure for performance-sensitive work.  I'm looking forward to seeing what
 people do with Vertigo and libraries like it.

 Zach

 [1] http://www.youtube.com/**watch?feature=player_**
 detailpagev=v5dYE0CMmHQ#t=**1828shttp://www.youtube.com/watch?feature=player_detailpagev=v5dYE0CMmHQ#t=1828s
 [2] 
 https://github.com/ztellman/**primitive-mathhttps://github.com/ztellman/primitive-math
 [3] 
 https://github.com/**ztellman/byte-streamshttps://github.com/ztellman/byte-streams

  --
 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to a topic in the
 Google Groups Clojure group.
 To unsubscribe from this topic, visit
 https://groups.google.com/d/topic/clojure/BayfuaqMzvs/unsubscribe.
 To unsubscribe from this group and all its topics, send an email to
 clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.






-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group

[ANN] immutable-bitset: smalller, faster integer sets

[Zach Tellman]

https://github.com/ztellman/immutable-bitset

There's not much to describe here, this provides an implementation of an 
integer-only set which can take up three orders of magnitude less memory 
under certain conditions.  I needed this to implement a Bloom filter, but I 
figured it had applications elsewhere.  If anyone has questions, I'd be 
happy to answer them.

Zach

-- 
-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.