[ANN] pex, a powerful PEG parsing library

[Ghadi Shayban]

Here is a compliant JSON parser in 99 LOC, implemented with *pex, a new 
parsing library*. [1]

This is alpha software.

Hot off the heels of Colin Fleming's Conj talk on PEGs [2], I'm making 
public an early version of pex [3], a parsing library in the PEG family. 
 For those of you familiar with Lua's LPEG library, this is similar, but 
with a Clojure-y twist.  Like LPEG, pex uses a virtual machine to parse 
rules (as opposed to combinators or packrat.)  Unlike Colin's talk, pex 
operates on traditional character types, not generic sequences/data 
structures.

Why? Parsing Expression Grammars are simpler than most other grammar types, 
but more powerful than regular expressions. They do not introduce ambiguity 
-- you get one valid parse or none.  There exists a nice space *in between 
regexes (yuck) and **instaparse (power but ambiguity)*.

Here is a tiny *example* grammar to match floating point numbers:

(def Number '{number  [digits (? fractional) (? exponent)]
  fractional ["." digits]
  exponent   ["e" (? (/ "+" "-")) digits]
  digits [(class num) (* (class num))]})

The only other input this particular grammar needs is to let pex know what 
a `num` character class is.  (There is an interface that you can implement 
to match things, and several helpers.  I'm planning to have several common 
ones out of the box.)  Well, you also need to tell the grammar compiler 
what rule to start with (number).

The grammar format has user defined *macros* which let you hide a lot of 
boilerplate, or make higher order rules.  For example, it's very common to 
chew whitespace after rules, so hiding that is useful.  There are also 
*captures* and *actions* that operate on a virtual "Value Stack".  For 
example, while parsing a JSON array, you push all the captured values from 
the array onto the stack, then reduce them into a vector with an action.

It's very early, but pex's completely unoptimized engine can parse a 1.5MB 
file in ~58ms vs ~9ms for Cheshire/Jackson, which is a handwritten 
highly-tuned parser with many thousands of lines of code behind it.  I plan 
on closing that gap by a) implementing some of LPEG's compiler 
optimizations and b) improving some of the terribly naive impls in the 
parser. The win here is *high expressive power per unit of performance*, 
not raw performance... 

Internally, the grammar data structure is analyzed, compiled into special 
parsing bytecode, and then subsequently run inside a virtual machine [4].

Hope you can find this useful in your data munging endeavors.  Next up is 
to make CSV & EDN example parsers, tune the performance, make grammar 
debugging better, and write more docs & tests.  I encourage any feedback.

[1] 
https://github.com/ghadishayban/pex/blob/master/src/com/champbacon/pex/examples/json.clj#L7-L39
[2] https://www.youtube.com/watch?v=kt4haSH2xcs
[3] https://github.com/ghadishayban/pex
[4] 
https://github.com/ghadishayban/pex/blob/master/src-java/com/champbacon/pex/impl/PEGByteCodeVM.java#L247-L280

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Poor parallelization performance across 18 cores (but not 4)

[Andy Fingerhut]

There is no STM involved if you only have atoms, and no refs, so it can't
be STM-related.

I have a conjecture, but don't yet have a suggestion for an experiment that
would prove or disprove it.

The JVM memory model requires that changes to values that should be visible
to all threads, like swap! on an atom, require making state changes to
those values visible to all threads, which I think may often be implemented
by flushing any local cache values to main memory, even if no other thread
actually reads the value.

Your f1 code only does thread-local computation with no requirement to make
its results visible to other threads.

Your f2 code must  make its results visible to other threads.  Not only
that, but the values it must make visible are allocating new memory with
each new value (via calls to assoc).

Perhaps main memory is not fast enough to keep up with 18 cores running f2
at full rate, but it is fast enough to keep up with 4 cores running f2 at
full rate?

Maybe collecting data for time to completion for all number of cores
running f2 from 4 up to 18 on the same hardware would be illuminating?
Especially if it showed that there was some maximum number of 'f2
iterations per second' total that was equal across any number of cores
running f2 in parallel?

I am not sure whether that would explain your results of running 18
separate processes each running 1 thread of f2 in parallel getting full
speedup, unless the JVM can tell only one thread is running and thus no
flushes to main memory are required.  Maybe try running 9 processes, each
with 2 f2 threads, to see if it is as bad as 1 process with 18 threads?

Andy


On Mon, Nov 16, 2015 at 9:01 PM, David Iba  wrote:

> I have functions f1 and f2 below, and let's say they run in T1 and T2
> amount of time when running a single instance/thread.  The issue I'm facing
> is that parallelizing f2 across 18 cores takes anywhere from 2-5X T2, and
> for more complex funcs takes absurdly long.
>
>
>1. (defn f1 []
>2.   (apply + (range 2e9)))
>3.
>4. ;; Note: each call to (f2) makes its own x* atom, so the 'swap!'
>should never retry.
>5. (defn f2 []
>6.   (let [x* (atom {})]
>7. (loop [i 1e9]
>8.   (when-not (zero? i)
>9. (swap! x* assoc :k i)
>10. (recur (dec i))
>
>
> Of note:
> - On a 4-core machine, both f1 and f2 parallelize well (roungly T1 and T2
> for 4 runs in parallel)
> - running 18 f1's in parallel on the 18-core machine also parallelizes
> well.
> - Disabling hyperthreading doesn't help.
> - Based on jvisualvm monitoring, doesn't seem to be GC-related
> - also tried on dedicated 18-core ec2 instance with same issues, so not
> shared-tenancy-related
> - if I make a jar that runs a single f2 and launch 18 in parallel, it
> parallelizes well (so I don't think it's machine/aws-related)
>
> Could it be that the 18 f2's in parallel on a single JVM instance is
> overworking the STM with all the swap's?  Any other theories?
>
> Thanks!
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Poor parallelization performance across 18 cores (but not 4)

[David Iba]

I have functions f1 and f2 below, and let's say they run in T1 and T2 
amount of time when running a single instance/thread.  The issue I'm facing 
is that parallelizing f2 across 18 cores takes anywhere from 2-5X T2, and 
for more complex funcs takes absurdly long.


   1. (defn f1 []
   2.   (apply + (range 2e9)))
   3.  
   4. ;; Note: each call to (f2) makes its own x* atom, so the 'swap!' 
   should never retry.
   5. (defn f2 []
   6.   (let [x* (atom {})]
   7. (loop [i 1e9]
   8.   (when-not (zero? i)
   9. (swap! x* assoc :k i)
   10. (recur (dec i))
   

Of note:
- On a 4-core machine, both f1 and f2 parallelize well (roungly T1 and T2 
for 4 runs in parallel)
- running 18 f1's in parallel on the 18-core machine also parallelizes well.
- Disabling hyperthreading doesn't help.
- Based on jvisualvm monitoring, doesn't seem to be GC-related
- also tried on dedicated 18-core ec2 instance with same issues, so not 
shared-tenancy-related
- if I make a jar that runs a single f2 and launch 18 in parallel, it 
parallelizes well (so I don't think it's machine/aws-related)

Could it be that the 18 f2's in parallel on a single JVM instance is 
overworking the STM with all the swap's?  Any other theories?

Thanks!

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: ClassNotFoundException: clojure.lang.AFunction on OS X

[Gary Verhaegen]

I have no idea what this could be. Is there any chance you could build up a
minimal-ish project that reproduces this ? I have been using Clojure on a
Mac for years without any problem.

Also, at much less effort on your part, could you post the Leiningen and
JVM versions ? lein version should print both.

On Monday, 16 November 2015, Phil Segal  wrote:

> We have been happily using clojure on windows and linux, today we tried to
> run on a MacBook Pro and ran into several issues.
>
> The current stopper is a CNFE as per the title (stack trace at the end).
>
> I have now replicated this on my personal Mac and seen the same behaviour.
> I can't find anything that suggests that there is anything special to do.
>
> I installed leiningen using brew, and am using clojure 1.7.0 as a maven
> dependency in our java code.
>
> The issue is occurring in a java constructor which is then calling:
>
> IFn require = Clojure.var("clojure.core", "require");
>
> require.invoke(Clojure.read("com.xxx.our-name-space"), RELOAD);
> aFn = Clojure.var("com.xxx-our-name-space", "a-function");
>
>
> Any assistance would be amazing.
>
> Thanks
> Phil
>
> Caused by: java.lang.NoClassDefFoundError: clojure/lang/AFunction
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:642)
> at clojure.lang.DynamicClassLoader.defineClass(DynamicClassLoader.java:46)
> at clojure.lang.Compiler$ObjExpr.getCompiledClass(Compiler.java:4833)
> at clojure.lang.Compiler$FnExpr.parse(Compiler.java:3993)
> at clojure.lang.Compiler.analyzeSeq(Compiler.java:6721)
> ... 32 more
> Caused by: java.lang.ClassNotFoundException: clojure.lang.AFunction
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at clojure.lang.DynamicClassLoader.findClass(DynamicClassLoader.java:69)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at clojure.lang.DynamicClassLoader.loadClass(DynamicClassLoader.java:77)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 39 more
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> 
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> 
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] Simulant 0.1.8

[Michael Nygard]

Simulant 0.1.8 is now available. This is a maintenance release that
incorporates some bug fixes and updates Clojure and Datomic dependencies

Try it via

   - Leiningen: [com.datomic/simulant "0.1.8"]


Changes since 0.1.7:


   - PR 24  Allow overriding
   action log batch size via :actionLog.service/batch-size
   - PR 22  Make action log
   batch size configurable
   - PR 21  Action log uses
   pr-str but doesn't bind *print-length* to nil
   - PR 18  Schema is not
   built into the distributed jar file
   - PR 13  put non-attributes
   in db.part/user for newer versions of datomic
   - PR 11  Clean up the URI
   used for process-state databases.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: arity-3 `iterate`

[Ben Wolfson]

ncomp would be faster if it's implemented a la exponentiation by squaring.

On Mon, Nov 16, 2015 at 12:55 PM, Ben Wolfson  wrote:

> If you rearrange the arguments of your proposed three-argument iterate:
>
> (defn iterate
>   ...
>   ([f n x]
>(if (zero? n)
>  x
>  (recur f (f x) (dec n
>
> it supports partial application of the multiply-composed function to
> different arguments more easily, which suggests redefinition as:
>
> (defn ncomp [f n]
>   (loop [g identity n n]
> (if (zero? n)
>g
>(recur (comp g f) (dec n)
>
> This seems to come closer to matching what's desired from the
> three-argument iterate without confusingly (as you note) overloading
> "iterate": you want f applied n times to x and the stream of intermediate
> values isn't really of interest.
>
> On Mon, Nov 16, 2015 at 12:37 PM, Jason Felice 
> wrote:
>
>>
>> I *frequently* see:
>>
>> (nth (iterate foo bar) n)
>>
>> And:
>>
>> (->> (iterate foo bar)
>>(drop n)
>>first)
>>
>> I've also coded this in `avi` after being surprised no such thing exists:
>>
>> (defn n-times
>>   [thing n a-fn]
>>   (reduce
>> (fn [thing n]
>>   (a-fn thing))
>> thing
>> (range n)))
>>
>> (which is kind of a bad implementation, now that I think of it):
>>
>> Would it be useful to file a ticket for a new arity for iterate?:
>>
>> (defn iterate
>>   ...
>>   ([f x n]
>>(if (zero? n)
>>  x
>>  (recur f (f x) (dec n
>>
>> There's a little bit of weirdness - this returns a single value, while
>> the other arity returns a lazy sequence.  However, I think it's the correct
>> name (and the original arity might  have better been named "iterations" for
>> symmetry with "reduce" and "reductions").  But ah well.
>>
>> Thoughts?
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clojure@googlegroups.com
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to clojure+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
> Ben Wolfson
> "Human kind has used its intelligence to vary the flavour of drinks, which
> may be sweet, aromatic, fermented or spirit-based. ... Family and social
> life also offer numerous other occasions to consume drinks for pleasure."
> [Larousse, "Drink" entry]
>
>


-- 
Ben Wolfson
"Human kind has used its intelligence to vary the flavour of drinks, which
may be sweet, aromatic, fermented or spirit-based. ... Family and social
life also offer numerous other occasions to consume drinks for pleasure."
[Larousse, "Drink" entry]

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

ClassNotFoundException: clojure.lang.AFunction on OS X

[Phil Segal]

We have been happily using clojure on windows and linux, today we tried to 
run on a MacBook Pro and ran into several issues.

The current stopper is a CNFE as per the title (stack trace at the end).

I have now replicated this on my personal Mac and seen the same behaviour. 
I can't find anything that suggests that there is anything special to do.

I installed leiningen using brew, and am using clojure 1.7.0 as a maven 
dependency in our java code.

The issue is occurring in a java constructor which is then calling:

IFn require = Clojure.var("clojure.core", "require");

require.invoke(Clojure.read("com.xxx.our-name-space"), RELOAD);
aFn = Clojure.var("com.xxx-our-name-space", "a-function");


Any assistance would be amazing. 

Thanks
Phil

Caused by: java.lang.NoClassDefFoundError: clojure/lang/AFunction
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
at java.lang.ClassLoader.defineClass(ClassLoader.java:642)
at clojure.lang.DynamicClassLoader.defineClass(DynamicClassLoader.java:46)
at clojure.lang.Compiler$ObjExpr.getCompiledClass(Compiler.java:4833)
at clojure.lang.Compiler$FnExpr.parse(Compiler.java:3993)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6721)
... 32 more
Caused by: java.lang.ClassNotFoundException: clojure.lang.AFunction
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at clojure.lang.DynamicClassLoader.findClass(DynamicClassLoader.java:69)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at clojure.lang.DynamicClassLoader.loadClass(DynamicClassLoader.java:77)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 39 more

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: arity-3 `iterate`

[Ben Wolfson]

If you rearrange the arguments of your proposed three-argument iterate:

(defn iterate
  ...
  ([f n x]
   (if (zero? n)
 x
 (recur f (f x) (dec n

it supports partial application of the multiply-composed function to
different arguments more easily, which suggests redefinition as:

(defn ncomp [f n]
  (loop [g identity n n]
(if (zero? n)
   g
   (recur (comp g f) (dec n)

This seems to come closer to matching what's desired from the
three-argument iterate without confusingly (as you note) overloading
"iterate": you want f applied n times to x and the stream of intermediate
values isn't really of interest.

On Mon, Nov 16, 2015 at 12:37 PM, Jason Felice 
wrote:

>
> I *frequently* see:
>
> (nth (iterate foo bar) n)
>
> And:
>
> (->> (iterate foo bar)
>(drop n)
>first)
>
> I've also coded this in `avi` after being surprised no such thing exists:
>
> (defn n-times
>   [thing n a-fn]
>   (reduce
> (fn [thing n]
>   (a-fn thing))
> thing
> (range n)))
>
> (which is kind of a bad implementation, now that I think of it):
>
> Would it be useful to file a ticket for a new arity for iterate?:
>
> (defn iterate
>   ...
>   ([f x n]
>(if (zero? n)
>  x
>  (recur f (f x) (dec n
>
> There's a little bit of weirdness - this returns a single value, while the
> other arity returns a lazy sequence.  However, I think it's the correct
> name (and the original arity might  have better been named "iterations" for
> symmetry with "reduce" and "reductions").  But ah well.
>
> Thoughts?
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Ben Wolfson
"Human kind has used its intelligence to vary the flavour of drinks, which
may be sweet, aromatic, fermented or spirit-based. ... Family and social
life also offer numerous other occasions to consume drinks for pleasure."
[Larousse, "Drink" entry]

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

arity-3 `iterate`

[Jason Felice]

I *frequently* see:

(nth (iterate foo bar) n)

And:

(->> (iterate foo bar)
   (drop n)
   first)

I've also coded this in `avi` after being surprised no such thing exists:

(defn n-times
  [thing n a-fn]
  (reduce
(fn [thing n]
  (a-fn thing))
thing
(range n)))

(which is kind of a bad implementation, now that I think of it):

Would it be useful to file a ticket for a new arity for iterate?:

(defn iterate
  ...
  ([f x n]
   (if (zero? n)
 x
 (recur f (f x) (dec n

There's a little bit of weirdness - this returns a single value, while the
other arity returns a lazy sequence.  However, I think it's the correct
name (and the original arity might  have better been named "iterations" for
symmetry with "reduce" and "reductions").  But ah well.

Thoughts?

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

New Functional Programming Job Opportunities

[Functional Jobs]

Here are some functional programming job opportunities that were posted

recently:



Software Developer | Scala, Clojure | Spark, Solr, Agile at Elmar Reizen

https://functionaljobs.com/jobs/8865-software-developer--scala-clojure--spark-solr-agile-at-elmar-reizen



Cheers,

Sean Murphy

FunctionalJobs.com


-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Largest Clojure codebases?

[Michael Willis]

For what it's worth, here's the way that I always count lines of code, 
should work on any unix-like system:

find -name "*.clj" | wc -l

On Sunday, November 15, 2015 at 5:23:01 PM UTC-6, Colin Yates wrote:
>
> Exactly this. I couldn’t find a reliable way of counting LOC but my 
> (Clojure/ClojureSciprt) src tree (excluding test) in the project I have to 
> hand is 1.5MB.
>
>
> On 15 Nov 2015, at 21:27, Timothy Baldridge  > wrote:
>
> It's funny, because most of the larger OOP projects I worked on were large 
> because of class bloat, not because of business concerns. For example, the 
> C# app I used to work on was a more or less simple CRUD app. We had a ORM 
> that served up objects to a Silverlight UI. So if we wanted to display 
> information about a person on the UI we normally had to modify around 5 
> classes in 5 files. We had the following layers
>
> ORM - 1.4MB of generated C# for interfacing with the 200 or so SQL tables 
> we had
> DTO - Data Transfer Object, where used "normal" C# objects to abstract the 
> ORM. This is where we had the "Person" object 
> API - A web service that served up DTOs over HTTP
> Data Model - Processed views of DTOs formatted in a way that was easily 
> viewable by the UI
> View Model - UI classes that would take data from a Data Model and emit UI 
> controls. 
>
> All of that ceremonythat is replaced by one thing in Clojure...data. 
> Hashmaps and vectors replace all the junk you see above. 
>
> So that's where I often assert "Yes, you may have 1 million lines of Java 
> codebut that would only be ~10,000 lines of Clojure." With proper 
> application of data driven systems (data that configures pipelines and 
> writes code) you can easily get a 100:1 ratio of Java to Clojure code. 
>
> Timothy
>
>
> On Sun, Nov 15, 2015 at 12:48 PM, Marc O'Morain  > wrote:
>
>> We have a large app at CircleCI - as of September:
>>
>> "The repo for our main app contains 93,983 lines of Clojure code. The src 
>> directory 
>> of our main app contains 369 namespaces."
>>
>> http://blog.circleci.com/why-were-no-longer-using-core-typed/
>>
>>
>>
>> On Sun, Nov 15, 2015 at 7:22 PM, dilettan...@live.com  <
>> dilettan...@live.com > wrote:
>>
>>> I've been having a (friendly) argument with a friend who is an 
>>> old-school OOP programmer.  He insists that you need objects to make 
>>> large-scale codebases legible and maintainable over the long run.  Quite 
>>> apart from this argument's virtues or lack thereof, this made me curious -- 
>>> what are the largest programs written in Clojure in terms of LOC?  I know 
>>> I've seen mentions of 50k-100k LOC projects (World Singles, if I'm 
>>> remembering correctly), but are there any that are larger?
>>>
>>>Vikram
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To post to this group, send email to clo...@googlegroups.com 
>>> 
>>> Note that posts from new members are moderated - please be patient with 
>>> your first post.
>>> To unsubscribe from this group, send email to
>>> clojure+u...@googlegroups.com 
>>> For more options, visit this group at
>>> http://groups.google.com/group/clojure?hl=en
>>> --- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Clojure" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to clojure+u...@googlegroups.com .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
>> -- 
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clo...@googlegroups.com 
>> 
>> Note that posts from new members are moderated - please be patient with 
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+u...@googlegroups.com 
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to clojure+u...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> -- 
> “One of the main causes of the fall of the Roman Empire was that–lacking 
> zero–they had no way to indicate successful termination of their C 
> programs.”
> (Robert Firth) 
>
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com 
> Note that posts from new members are moderated - please be patient with 
> your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com 
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> --- 
> You received this message because you are subscribed to the Google Groups 
> "Clojure" group.
> To unsubscribe from this group and stop

[ANN] konserve 0.3.0

[Christian Weilbach]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hello,

I am happy to announce the release of version 0.3.0 of konserve, a
clojuresque durable key-value/document store protocol with core.async.

Most importantly serialization protocols are now factored as a
protocol with implementations for Fressian, Transit and
pr-str/read-string. The protocols are also now wrapped by real clojure
functions. For more details have a look at:

https://github.com/replikativ/konserve

I have also updated the CouchDB backend:

https://github.com/whilo/konserve-couch

Feedback is (as always) appreciated,
Christian
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iQEcBAEBAgAGBQJWSfz6AAoJEKel+aujRZMkkkcH/2oJGiMrbEPZL1SmjEHisiBO
F1vGg2QJYpUOpP/CxxtT7EC8n/BcixnlsGM/FdqcHfB3l/2Liit8aSUD9hCWJW8N
Pas5+EFdOtfD73nc4B5CfDuDo+y+EIlExvktbi3ljnuKVRuZ/X6fG7FBX6+tlhBg
itKDrRZYXs9UIA/N/MRoPHeTvs76XBzLsNo2k+FMRIcUe6pu6D/Chdc5klXDKbfc
fyoPi9s9ZuoBCn0xfpSxum+39LODoREjgr7Wxv44vDDSXayEnMkETZLDPj61rxHb
A960OMXVOXsUpSIRVaU0UJRpmZIm7IcFc3Olwu+6YJbSQhMq7BXohqAQPSoe15U=
=8ZEp
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Largest Clojure codebases?

[Kyle R. Burton]

At the last company I was with I used sloccount [1] to analyze the
codebase.  I concatenated all the clj files to a .lisp file so sloccount
could analyze it.  I was curious about the cost estimate that sloccount
performs to see how the team measured up (size varied from 2 to 7 over 5
years).  When I did the analysis (over a year ago) we had about 130k lines
of Clojure that represented about two dozen libraries and bout six
services.  Including the javascript, java, C, Ruby and other languages in
our repositories, sloccount estimated over 5x the person years we actually
spent.  This team was also responsible for the whole stack - production
operations, releases, etc.  If someone is doing research, I'd be happy to
reach out to a colleague to see if they would run the analysis again.


Kyle


[1] http://www.dwheeler.com/sloccount/

On Sun, Nov 15, 2015 at 4:06 PM, dennis zhuang  wrote:

> I use cloc(http://cloc.sourceforge.net/) to counting the LOC of our
> projects, it's total about 41025 lines of Clojure  code.
>
>
>
>
>
>
> 2015-11-16 7:22 GMT+08:00 Colin Yates :
>
>> Exactly this. I couldn’t find a reliable way of counting LOC but my
>> (Clojure/ClojureSciprt) src tree (excluding test) in the project I have to
>> hand is 1.5MB.
>>
>>
>> On 15 Nov 2015, at 21:27, Timothy Baldridge  wrote:
>>
>> It's funny, because most of the larger OOP projects I worked on were
>> large because of class bloat, not because of business concerns. For
>> example, the C# app I used to work on was a more or less simple CRUD app.
>> We had a ORM that served up objects to a Silverlight UI. So if we wanted to
>> display information about a person on the UI we normally had to modify
>> around 5 classes in 5 files. We had the following layers
>>
>> ORM - 1.4MB of generated C# for interfacing with the 200 or so SQL tables
>> we had
>> DTO - Data Transfer Object, where used "normal" C# objects to abstract
>> the ORM. This is where we had the "Person" object
>> API - A web service that served up DTOs over HTTP
>> Data Model - Processed views of DTOs formatted in a way that was easily
>> viewable by the UI
>> View Model - UI classes that would take data from a Data Model and emit
>> UI controls.
>>
>> All of that ceremonythat is replaced by one thing in Clojure...data.
>> Hashmaps and vectors replace all the junk you see above.
>>
>> So that's where I often assert "Yes, you may have 1 million lines of Java
>> codebut that would only be ~10,000 lines of Clojure." With proper
>> application of data driven systems (data that configures pipelines and
>> writes code) you can easily get a 100:1 ratio of Java to Clojure code.
>>
>> Timothy
>>
>>
>> On Sun, Nov 15, 2015 at 12:48 PM, Marc O'Morain 
>> wrote:
>>
>>> We have a large app at CircleCI - as of September:
>>>
>>> "The repo for our main app contains 93,983 lines of Clojure code. The
>>> src directory of our main app contains 369 namespaces."
>>>
>>> http://blog.circleci.com/why-were-no-longer-using-core-typed/
>>>
>>>
>>>
>>> On Sun, Nov 15, 2015 at 7:22 PM, dilettante.co...@live.com <
>>> dilettante.co...@live.com> wrote:
>>>
 I've been having a (friendly) argument with a friend who is an
 old-school OOP programmer.  He insists that you need objects to make
 large-scale codebases legible and maintainable over the long run.  Quite
 apart from this argument's virtues or lack thereof, this made me curious --
 what are the largest programs written in Clojure in terms of LOC?  I know
 I've seen mentions of 50k-100k LOC projects (World Singles, if I'm
 remembering correctly), but are there any that are larger?

Vikram

 --
 You received this message because you are subscribed to the Google
 Groups "Clojure" group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google
 Groups "Clojure" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.

>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To post to this group, send email to clojure@googlegroups.com
>>> Note that posts from new members are moderated - please be patient with
>>> your first post.
>>> To unsubscribe from this group, send email to
>>> clojure+unsubscr...@googlegroups.com
>>> For more options, visit this group at
>>> http://groups.google.com/group/clojure?hl=en
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To un

[ANN] Clojure 1.8.0-RC2

[Alex Miller]

Clojure 1.8.0-RC2 is now available. *This build is a "release candidate"!* We 
would appreciate any and all testing you can do on your own libraries or 
internal projects to find problems. 

Try it via

   - Download: https://repo1.maven.org/maven2/org/clojure/clojure/1.8.0-RC2
   - Leiningen: [org.clojure/clojure "1.8.0-RC2"]

Below are the changes since 1.8.0-RC1. See the full 1.8 change log here: 
https://github.com/clojure/clojure/blob/master/changes.md.

   - CLJ-1846  Fix VerifyError 
   when primitive type hints are incorrect. These cases now throw compiler 
   errors.
  - Example: (defn foo ^long [] 1)  (Integer/bitCount ^int (foo))
  - In this example, foo returns a long but is type hinted 
  (incorrectly) as an int. This case will now throw a compilation error. 
The 
  correct way to do this is with a cast:  (Integer/bitCount (int (foo)))
   - CLJ-1825  Fix compilation 
   errors on direct linking of anonymous recursive functions

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Reflection warning on try/catch inside go block

[Nicola Mometto]

It's possibly a tools.analyzer issue, I'll take a look ASAP

> On 16 Nov 2015, at 12:41, Alice  wrote:
> 
> Sorry, I was having some copy & paste mistake, but I'm seeing the same 
> warning with .printStackTrace.
> 
> On Monday, November 16, 2015 at 9:25:06 PM UTC+9, Herwig Hochleitner wrote:
> The method is called .printStackTrace: 
> http://docs.oracle.com/javase/7/docs/api/java/lang/Throwable.html#printStackTrace()
> 
> I can't speak to why the reflection warning disappears in some cases. Seems 
> fishy,
> ​
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with your 
> first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> --- 
> You received this message because you are subscribed to the Google Groups 
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Reflection warning on try/catch inside go block

[Alice]

Sorry, I was having some copy & paste mistake, but I'm seeing the same 
warning with .printStackTrace.

On Monday, November 16, 2015 at 9:25:06 PM UTC+9, Herwig Hochleitner wrote:
>
> The method is called .printStackTrace: 
> http://docs.oracle.com/javase/7/docs/api/java/lang/Throwable.html#printStackTrace()
>
> I can't speak to why the reflection warning disappears in some cases. 
> Seems fishy,
> ​
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Reflection warning on try/catch inside go block

[Herwig Hochleitner]

The method is called .printStackTrace:
http://docs.oracle.com/javase/7/docs/api/java/lang/Throwable.html#printStackTrace()

I can't speak to why the reflection warning disappears in some cases. Seems
fishy,
​

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Reflection warning on try/catch inside go block

[Alice]

[org.clojure/clojure "1.7.0"]
[org.clojure/core.async "0.2.374"]

On Monday, November 16, 2015 at 8:28:27 PM UTC+9, Alex Miller wrote:
>
> Which version of core.async?

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Locking non-local variable inside macro

[Alex Miller]

Usually it's better to create a sentinel (Object.) to lock on to avoid sharing 
it. (Well really it's best to avoid locking at all and use Clojure state 
constructs if you can.)

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reflection warning on try/catch inside go block

[Alex Miller]

Which version of core.async?

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reflection warning on try/catch inside go block

[Alice]

I get a reflection warning "reference to field printstacktrace can't be 
resolved" from the following code:

(defn foo 
  []
  (go-loop []
(http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.