Re: Feedback on datafy/nav project

[Christian Gonzalez]

I think you could also add some metadata to allow datafy-ing instead of 
extending the protocol:

(with-meta {:name "John Doe" :language "us"} 
{`clojure.core.protocols/datafy (fn [x] ...)})

On Thursday, January 30, 2020 at 10:28:20 AM UTC-8, Jim foo.bar wrote:
>
> Moreover, following the mailing discussion linked from the clojure.org 
> site, leads to the following quote from Rich - about 10 years ago :)
>
> *If a protocol comes with Clojure itself, avoid extending it to types *
> *you don't own, especially e.g. java.lang.String and other core Java *
> *interfaces. Rest assured if a protocol should extend to it, it will, *
> *else lobby for it. *
>
>
>
> On 30/01/2020 17:58, dimitris wrote:
>
> Well, the official Clojure guidelines are to NOT extend protocols, unless 
> you own either the protocol of the type(s) [1].
>
> In particular these lines:
>
> *If you are a library developer, you should not extend if you own neither 
> the protocol nor the target.*
>
> *You should take particular care when extending protocols included with 
> Clojure itself.*
>
>
> Kind regards,
>
> Dimitris
>
> [1]: https://clojure.org/reference/protocols#_extend_via_metadata
>
>
> On 30/01/2020 17:50, Sean Corfield wrote:
>
> Is there a reason you’ve mirrored those protocols/implementations rather 
> than just use Clojure’s built-in versions?
>
>  
>
> As it stands, your library wouldn’t work with other tooling that builds on 
> Clojure’s datafy/nav (REBL, for example).
>
>  
>
> Sean Corfield -- (904) 302-SEAN
> An Architect's View -- http://corfield.org/
>
> "If you're not annoying somebody, you're not really alive."
> -- Margaret Atwood
>
>  
>
> *From: *dimitris 
> *Sent: *Thursday, January 30, 2020 9:05 AM
> *To: *Clojure 
> *Subject: *Feedback on datafy/nav project
>
>  
>
> Hi folks,
>
>  
>
> I'm looking for honest and constructive feedback on [1] - not so much 
>
> about the actual implementation (even though these are welcome too), but 
>
> rather about the general idea of basing everything on top of (mirrored 
>
> versions) `datafy` and `nav` (in this way). There is a TL;DR section at 
>
> the very bottom that describes the approach in 4 short sentences.
>
>  
>
> Many thanks in advance,
>
>  
>
> Dimitris
>
>  
>
> [1]: https://github.com/jimpil/jedi-time
>
>  
>
>  
>
> -- 
>
> You received this message because you are subscribed to the Google
>
> Groups "Clojure" group.
>
> To post to this group, send email to clo...@googlegroups.com 
>
> Note that posts from new members are moderated - please be patient with 
> your first post.
>
> To unsubscribe from this group, send email to
>
> clo...@googlegroups.com 
>
> For more options, visit this group at
>
> http://groups.google.com/group/clojure?hl=en
>
> --- 
>
> You received this message because you are subscribed to the Google Groups 
> "Clojure" group.
>
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to clo...@googlegroups.com .
>
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/clojure/d0db9e09-6419-2e13-3a0b-9bacf2bbf4fd%40gmail.com
> .
>
>  
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com 
> Note that posts from new members are moderated - please be patient with 
> your first post.
> To unsubscribe from this group, send email to
> clo...@googlegroups.com 
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> --- 
> You received this message because you are subscribed to the Google Groups 
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to clo...@googlegroups.com .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/clojure/5e331755.1c69fb81.ff1ad.21cf%40mx.google.com
>  
> 
> .
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/a0a471ae-c4c6-4b92-86a6-5cefe099bdd7%40googlegroups.com.

ANN: Anglican 1.1.0

[Christian Weilbach]

We are happy to announce a new release of the probabilistic programming
language Anglican. This release brings improvements and changes that
have accumulated over the last two years:

  - ClojureScript support for all of Anglican's language and most of its
inference algorithms
  - Clojure 1.9 support
  - EPL relicencing (from GPL)
  - Student's-T, Laplace distribution
  - IPMCMC fixes and more

You can reach us in the #anglican channel on the Clojurians slack and
learn more on http://anglican.ml .

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

ANN: gorilla-converter (clojupyter)

[Christian Weilbach]

Hey fellow Clojurians,

I just wanted to announce that I have written a bidirectional gorilla to
clojupyter converter:

https://bitbucket.org/probprog/gorilla-converter

This is used to make the http://anglican.ml examples accessible to the
data science community, but should in general help to make Clojure more
accessible in a typical data science environment. Right now you need to
clone and install the repository, but I will probably put it in some
form onto clojars. I think a common environment for plotting (I heavily
favour plotly atm.) would make sense, so that plotting code also is
portable. I would like to have feedback and suggestions!

Best,
Christian


-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: OK idea to replace conj and cons with "prepend" and "append" macros that have consistent behavior and return same types as args?

[Christian Seberino]

> Just what is it that you want to teach? Principles of computation, or 
> Clojure? Not the same. If the former, forget Clojure and give them a week 
> to work thru The Little Schemer. Then move on to clojure and the much more 
> complex stuff involved.
>

I think you bring up a good point.
I wonder if we all have one of 2 mental images of what people mean by a 
"Clojure intro class".  Many probably mean something like a real corporate 
class where
people are trying to train developers on Clojure to get real work done.   I 
think more of the SICP/Scheme/MIT class where you have a lot of smart
people with no programming experience and you want to give them a solid 
academic intro.

I agree Scheme is great for that.  However, what I find delicious is that 
Clojure is this >| |<--- close to being *both* a super practical 
language
*and* a better/cleaner teaching language!  This might surprise some but I 
actually think some things are more elegant in Clojure than Scheme!
For example, I like "fn" over "lambda".  Also, I like the seq abstraction 
over nested conses for the fundamental data structure.

cs


-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: OK idea to replace conj and cons with "prepend" and "append" macros that have consistent behavior and return same types as args?

[Christian Seberino]

>
> - conj adds an element in the place best for the collection type.


Is this a valid hypothetical to worry about?...

Imagine you're the teacher and make the comment above.  

Student responds..

"But why, Mr. Teacher, is the 'best' place different for lists and 
vectors?  That seems strange that they are opposite."

<30 minute heated discussion ensues about aesthetics, efficiency and 
nuances of lists vs. vectors you were trying to avoid.>






 

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: OK idea to replace conj and cons with "prepend" and "append" macros that have consistent behavior and return same types as args?

[Christian Seberino]

The last few tests got cut off...

...

(prn (comb {"a" "b", "c" "d"} {"e" "f", "g" "h"}))
(prn (comb {"a" "b"} {"c" "d", "e" "f"}))
(prn (comb {"a" "b", "c" "d"} {"e" "f"}))
(prn (comb "abc"   "def" "ghi"))

On Sunday, July 22, 2018 at 2:21:24 PM UTC-5, Christian Seberino wrote:
>
>
> This really just reinforces what others have already said above that 
>> Clojure's standard library doesn't make it easy for you to do something 
>> inefficient
>>
>
> Do you at least agree it is at least debatable whether an intro class 
> might benefit from avoiding efficiency concerns initially?  
> I incorrectly assumed that that would require macros, but, it really only 
> requires beginner friendly regular functions.
>
> For example, see this...
>
> (defn comb [& args]
>   (let [comb_  (apply concat args)
> first_ (first args)]
>(cond (list?   first_)comb_
>  (vector? first_) (into  []  comb_)
>  (map?first_) (into  {}  comb_)
>  (set?first_) (into  #{} comb_)
>  (string? first_) (apply str comb_
>
> Testing with these...
>
> (prn (comb [1 2] [3 4]))
> (prn (comb [1]   [2 3]))
> (prn (comb [1 2] [3]))
> (prn (comb '(1 2) '(3 4)))
> (prn (comb '(1)   '(2 3)))
> (prn (comb '(1 2) '(3)))
> (prn (comb #{1 2} #{3 4}))
> (prn (comb #{1}   #{2 3}))
> (prn (comb #{1 2} #{3}))
> (prn (comb [1 2]   [3 4] '(5 6)))
> (prn (comb '(1 2) '(3 4) [5 6]))
> (prn (comb {"a" "b", "c" "d"} {"e" "f", "g" "h"}))
>
> Gives these...
>
> [1 2 3 4]
> [1 2 3]
> [1 2 3]
> (1 2 3 4)
> (1 2 3)
> (1 2 3)
> #{1 4 3 2}
> #{1 3 2}
> #{1 3 2}
> [1 2 3 4 5 6]
> (1 2 3 4 5 6)
> {"a" "b", "c" "d", "e" "f", "g" "h"}
> {"a" "b", "c" "d", "e" "f"}
> {"a" "b", "c" "d", "e" "f"}
> "abcdefghi"
>
> Notice that with my comb (combine) function, I eliminated the need for 
> str, prepend, append, conj, etc.  Furthermore, comb gives
> the intuitive answer irrespective of efficiency.  
>
> Is that so bad at least temporarily?
>
> cs
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: OK idea to replace conj and cons with "prepend" and "append" macros that have consistent behavior and return same types as args?

[Christian Seberino]

> This really just reinforces what others have already said above that 
> Clojure's standard library doesn't make it easy for you to do something 
> inefficient
>

Do you at least agree it is at least debatable whether an intro class might 
benefit from avoiding efficiency concerns initially?  
I incorrectly assumed that that would require macros, but, it really only 
requires beginner friendly regular functions.

For example, see this...

(defn comb [& args]
  (let [comb_  (apply concat args)
first_ (first args)]
   (cond (list?   first_)comb_
 (vector? first_) (into  []  comb_)
 (map?first_) (into  {}  comb_)
 (set?first_) (into  #{} comb_)
 (string? first_) (apply str comb_

Testing with these...

(prn (comb [1 2] [3 4]))
(prn (comb [1]   [2 3]))
(prn (comb [1 2] [3]))
(prn (comb '(1 2) '(3 4)))
(prn (comb '(1)   '(2 3)))
(prn (comb '(1 2) '(3)))
(prn (comb #{1 2} #{3 4}))
(prn (comb #{1}   #{2 3}))
(prn (comb #{1 2} #{3}))
(prn (comb [1 2]   [3 4] '(5 6)))
(prn (comb '(1 2) '(3 4) [5 6]))
(prn (comb {"a" "b", "c" "d"} {"e" "f", "g" "h"}))

Gives these...

[1 2 3 4]
[1 2 3]
[1 2 3]
(1 2 3 4)
(1 2 3)
(1 2 3)
#{1 4 3 2}
#{1 3 2}
#{1 3 2}
[1 2 3 4 5 6]
(1 2 3 4 5 6)
{"a" "b", "c" "d", "e" "f", "g" "h"}
{"a" "b", "c" "d", "e" "f"}
{"a" "b", "c" "d", "e" "f"}
"abcdefghi"

Notice that with my comb (combine) function, I eliminated the need for str, 
prepend, append, conj, etc.  Furthermore, comb gives
the intuitive answer irrespective of efficiency.  

Is that so bad at least temporarily?

cs

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: OK idea to replace conj and cons with "prepend" and "append" macros that have consistent behavior and return same types as args?

[Christian Seberino]

Wow thanks.  That was pretty thorough.

cs

On Friday, July 20, 2018 at 10:51:48 AM UTC-5, Gary Johnson wrote:
>
> Hi Christian,
>
> You are looking for "into", which is already part of the Clojure standard 
> library.
>
> Appending:
>
> (into '(1 2) '(3)) ;=>  (1 2 3)
> (into [1 2] [3])   ;=>  [1 2 3]
> (into {:a 1 :b 2} {:c 3})  ;=>  {:a 1, :b 2, :c 3}
> (into #{:a :b} #{:c})  ;=>  #{:c :b :a}
>
> Prepending:
>
> (into '(1) '(2 3)) ;=>  (1 2 3)
> (into [1] [2 3])   ;=>  [1 2 3]
> (into {:a 1} {:b 2 :c 3})  ;=>  {:a 1, :b 2, :c 3}
> (into #{:a} #{:b :c})  ;=>  #{:c :b :a}
>
> The "into" function pours the contents of the second collection into the 
> first collection, returning a collection of the same type as the first 
> argument.
>
> That being said, I agree with Alex and James in this rather lengthy 
> discussion. Clojure is unique among the Lisps in that it moved beyond 
> having linked lists as the only first class data structure.
>
> Prior to Clojure, if you worked in a Lisp like Scheme or Common Lisp, you 
> would design your program around the creation, traversal, and manipulation 
> of linked lists using higher order functions and explicit recursions. The 
> standard library in both languages is heavily focused on these list-related 
> operations. After developing the initial version of your program, if you 
> found that it was too slow or used too much memory, the accepted practice 
> was to profile your application to identify the functions that were getting 
> hammered the most and were using up the majority of your computing 
> resources. You would then often end up rewriting those function bodies in 
> an imperative fashion using loops and mutable data structures (i.e., arrays 
> and hashtables). The "wisdom" here was that this would enable you to "first 
> make it right, then make it fast". If even further performance was required 
> from your program, you might then rewrite part of your program in C, build 
> a foreign function interface (FFI) to link the C code into your Lisp 
> program, and go from there. These were the Bad Old Days of Lisp(TM).
>
> What was IMHO quite possibly Rich's greatest contribution in the design of 
> Clojure to the Lisp world was his decision to make additional data 
> structures first class citizens of the language. Most importantly, he did 
> so by creating Clojure's vectors, maps, and sets to be immutable, 
> persistent, performant, recursively constructed, and representable as data 
> literals. This was already a wonderful improvement over previous Lisps, but 
> it created a new problem: How could we enjoy the pure delight of 
> list-oriented programming that Lisp had always offered us now that the data 
> structure space had been fragmented? A famous quote from Alan Perlis is a 
> popular gem in the Lisp world, and it goes like so:
>
> "It is better to have 100 functions operate on one data structure than to 
> have 10 functions operate on 10 data structures."
>
> Every Lisp had always satisfied this by simply giving programmers only one 
> first class data structure to use: the linked list. As I already mentioned, 
> the bulk of its standard library would then be built around list 
> manipulation functions. Clojure needed a way to preserve this unified style 
> of programming while still providing a collection of performant data 
> structures for real-world programming. So how did Rich accomplish this?
>
> He created the "sequence abstraction". A sequence in Clojure serves a 
> similar role to the linked list of previous Lisps in that it unifies the 
> API for interacting with all of Clojure's first class data structures 
> (list, vector, map, set). By calling the function "seq" on any data 
> structure, you are given a list-like view of that collection that allows 
> you to traverse it from beginning to end one element at a time and to add 
> new elements to the beginning of it. These operations are called "first", 
> "rest", and "cons", and they behave precisely as you would expect them to 
> if you were calling them on a linked list.
>
> By using seq throughout the Clojure sequence library (i.e., the set of 
> standard library functions responsible for creating, traversing, 
> transforming, and manipulating sequences), Clojure is able to have single 
> implementations of all of the common Lispy higher order list transformation 
> functions. For example, we have "map", "filter", "reduce", "iterate", 
> "take", "drop", "repeat", "cycle&q

How define alias for .indexOf? (def somealias .indexOf) does not work.

[Christian Seberino]

How define alias for .indexOf? 

 (def somealias .indexOf) does not work.

cs

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: OK idea to replace conj and cons with "prepend" and "append" macros that have consistent behavior and return same types as args?

[Christian Seberino]

On Thursday, July 19, 2018 at 10:04:39 AM UTC-5, Benoît Fleury wrote:
>
> I agree with Alex. It is important to understand the rationale behind the
> behavior of conj. conj is for adding an element to a collection. It doesn't
> say anything about ordering. It has been chosen as an operation, as
> opposed to append/prepend, because it can be implemented with the
> same time complexity for a lot of data structures.
>
>
Yes and that's important for real hardcore work.  But beginners don't 
necessarily need to be burdened with complexity
concerns immediately.  Eventually yes but not immediately.  And it really 
does has to be hardcore before
that matters...otherwise Python wouldn't be taking over the world and 
running strong for 20+ years.
 

> To convince myself that time complexity mattered in programming, I take
> the example of a stack. If you give me an implementation of a stack that
> push and pop items in linear time, I will write very inefficient programs 
> very quickly. To the point of not working at all. So it makes sense to me 
> that space/time complexity should be part of the signature of an operation.
>

See above.
 

>
> Now, I think the problem here comes from the fact that we have a tendency
> to infer the semantic of an operation from its behavior on certain types. 
> We
> see that (conj [1 2] 3) returns [1 2 3] so we conclude that conj is for 
> adding
> an item at the end of a collection. And then we get surprised when the
> behavior is "different" for other types. As Clojure programmers, it is 
> something
> we need to be careful about and make sure we understand the actual
> semantic of the operations.
>
> However, I think it is easier said than done. I would be ready to bet that 
> a
> lot of Clojure programs rely on the fact that conj *appends* to a vector or
> *prepends* to a list. I agree that it is problematic. First, it makes the
> understanding of the code harder because you need to remember the behavior
> of conj on each type. And it prevents the implementation of conj to change
> because so many programs rely on implementation details. But to all expert
> Clojure programmers here, can you honestly think about what mental model 
> you
> use when using conj on vectors and lists? Are you really thinking: I'm 
> adding
> this element to this collection. I don't really care where it ends up :) ?
>
> So my questions are:
>
> 1. Am I wrong in thinking that most Clojure programmers use append/prepend
> as mental models for the different implementations of conj?
>
> 2. Am I wrong in thinking that they shouldn't?
>
> 3. And if I'm not wrong, how could we make it easier for programmers to 
> make
> sure they code against the semantic of an operation and not its 
> implementation
> details? Is there methods, tools or tests that could help us with that?
>
>>
>>
My suggestion is to just use the natural polymorphic comb-ining function 
below until performance
is an issue.  At that point students will be motivated to learn about 
complexity

(defn comb [& args]
  (let [comb_ (apply concat args)]
   (cond (string? (first args)) (apply str comb_)
 (vector? (first args)) (into  []  comb_)
 (list?   (first args))comb_)))

(prn (comb [1 2]   [3 4]))
(prn (comb '(1 2) '(3 4)))
(prn (comb [1 2]   [3 4] '(5 6)))
(prn (comb '(1 2) '(3 4) [5 6]))
(prn (comb "abc"   "def" "ghi"))

gives this...

[1 2 3 4]
(1 2 3 4)
[1 2 3 4 5 6]
(1 2 3 4 5 6)
"abcdefghi"

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: OK idea to replace conj and cons with "prepend" and "append" macros that have consistent behavior and return same types as args?

[Christian Seberino]

Thanks.  I caught that yesterday.  Like I said, I'm definitely a beginner. 
;)

On Thursday, July 19, 2018 at 2:05:20 AM UTC-5, Vladimir Bokov wrote:
>
> First of all, def in a fn body is antipattern (use let), then do into on a 
> itself
>
> (defn concat_ [a b]
>   (if (vector? a)
> (into a b)
> (concat a b)))
>
> четверг, 19 июля 2018 г., 4:07:46 UTC+7 пользователь Christian Seberino 
> написал:
>>
>> I'm just a Clojure beginner but it seems that the Lisp Way(TM) is to 
>> append and prepend one or more elements
>> with a single command if possible.  The logical name for this command 
>> seems to be concat which led to this..
>>
>> (defn concat_ [a b]
>>   (def c (concat a b))
>>   (if (vector? a)
>>   (into [] c)
>>   c))
>>
>> (concat_ [1 2] [3 4]) => [1 2 3 4]
>>
>> (concat_ '(1 2) '(3 4)) => (1 2 3 4)
>>
>> (concat_ [1] [2 3]) => [1 2 3]
>>
>> Lists return lists and vectors return vectors.  Simple.
>> Yes yes I know it is slow.  I also know I need to expand concat_ to 
>> handle other data structures.
>>
>> In my little newbie world, this "feels" like the ultimate "right" 
>> solution.
>>
>> Chris
>>
>>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: OK idea to replace conj and cons with "prepend" and "append" macros that have consistent behavior and return same types as args?

[Christian Seberino]

>
>  - 'you always get back a value of the concrete type you supplied for 
> argument X' isn't obviously less cognitively burdensome than 'you always 
> get back a sequence'
>

Combining objects of type X should give a result that is of type X.  That 
seems the most natural to me.
 

>  - doesn't Python, in all its teachability, just throw a TypeError if you 
> try and concatenate differently typed collections, even in trivial cases 
> (e.g. (1,) + [2])?  Is there a way to do that in Python which doesn't 
> involve writing a concat function w/ a bunch of type checks
>

Whether to allow vectors and lists to combine is something that can be 
discussed.  Seems like may be a good idea and an improvement over Python.
 

>  - runtime list construction in Clojure is comparatively rare.  I'm not a 
> teacher, but in a practical educational setting I would focus on vectors as 
> the general-purpose ordered collection type - esp. if i were interested in 
> appending to them.
>

I like below.  It also takes over the str command.  Basically a single 
function to do all basic combinations.  Notice whether run time list 
construction is a good idea or not does not
have to be decided by me.  I just make the ability available.
 


(defn comb [& args]
  (let [comb_ (apply concat args)]
   (cond (string? (first args)) (apply str comb_)
 (vector? (first args)) (into  []  comb_)
 (list?   (first args))comb_)))





 

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: OK idea to replace conj and cons with "prepend" and "append" macros that have consistent behavior and return same types as args?

[Christian Seberino]

I'm just a Clojure beginner but it seems that the Lisp Way(TM) is to append
and prepend one or more elements
with a single command if possible.  The logical name for this command seems
to be concat which led to this..

(defn concat_ [a b]
  (def c (concat a b))
  (if (vector? a)
  (into [] c)
  c))

(concat_ [1 2] [3 4]) => [1 2 3 4]

(concat_ '(1 2) '(3 4)) => (1 2 3 4)

(concat_ [1] [2 3]) => [1 2 3]

Lists return lists and vectors return vectors.  Simple.
Yes yes I know it is slow.  I also know I need to expand concat_ to handle
other data structures.

In my little newbie world, this "feels" like the ultimate "right" solution.

Chris

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: OK idea to replace conj and cons with "prepend" and "append" macros that have consistent behavior and return same types as args?

[Christian Seberino]

Actually I was just kicked out of paradise.  concat always returns a list
and does NOT return a vector for this (concat [1 2] [3 4]) sadly.

cs

___

Christian Seberino, Ph.D.
Phone: (936) 235-1139
Email: cseber...@gmail.com
___



On Wed, Jul 18, 2018 at 2:16 AM, Didier  wrote:

> It's never a good idea to use the wrong data structure for the job.
>
> And thus Clojure takes the stance that it won't make bad ideas easy for
> you to use. Yet, it will never prevent you from doing anything.
>
> If you want to do something bad, you'll need to get your own hands dirty.
>
> That's why slow data structure access functions don't exist as standard.
> That's why data transforms are lazy by default. And why the non lazy
> variant (transducers) do loop fusion for you. That's why mutability is ugly
> and requires you to wrap things in extra verbosity. That's why OOP isn't
> there, and forces you to use the host interop if you want it. That's why
> there's only recursive loops. Etc.
>
> The Clojure standard lib is opinionated. It's not trying to make
> everything easy and convenient. It's trying to make things simple to reason
> about, and promote Rich Hickeys opinion of what is a good idea, and what
> isn't.
>
> But, it can afford to be this way, because it made itself a Lisp, meaning
> it gave you all the power needed to disagree and make your own core, which
> follows your own opinions of good and bad.[1]
>
> Now, I recommend that everyone should have a core library of their own
> that they keep around for cases like this, where they disagree.
>
> And for beginners, I mean, what are you trying to teach them? What problem
> requires them to add items to the beginning and end of an ordered
> collection?
>
> Anyways, my advice is to teach them concat. It's even nicer then
> append/prepend. You just give it the arguments where you want them to go.
>
> (concat [1] [2 3])
>
> (concat [1 2] [3])
>
> And it works for any type of ordered collections, even arrays.
>
> Also, this blog I think does a great job at teaching all this to a
> beginner https://medium.com/@greg_63957/conj-cons-concat-oh-my-
> 1398a2981eab
>
>
>
> [1] Except for reader macros. Rich didn't want you to be able to change
> the whole program syntax in unconstrained ways. That's probably a good
> thing to at least keep the foundation universal accross code bases.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: OK idea to replace conj and cons with "prepend" and "append" macros that have consistent behavior and return same types as args?

[Christian Seberino]

>
>
> Anyways, my advice is to teach them concat. It's even nicer then
> append/prepend. You just give it the arguments where you want them to go.
>
> (concat [1] [2 3])
>
> (concat [1 2] [3])


Thanks.  This is perfect.  I'm surprised it didn't up earlier in the
conversation.  concat is a single command that allows the addition of not
only
1 but arbitrary numbers of elements to lists and vectors.  Furthermore, it
returns the same type as the initial args.  Seems like the perfect
solution that would make the Lisp gods proud.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: OK idea to replace conj and cons with "prepend" and "append" macros that have consistent behavior and return same types as args?

[Christian Seberino]

>  But like others have said, that ship sailed in 2008.
>

Well depends what ship you are talking about.  Using prepend and append
only requires two new function definitions.  That is still easily done in
2018.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: OK idea to replace conj and cons with "prepend" and "append" macros that have consistent behavior and return same types as args?

[Christian Seberino]

>
>
> When writing software in Clojure, the data structures are often the
> keystone of the codebase. When I write Clojure, I start by mapping out what
> data structures and keywords I need, and from there write functions around
> them. It's for this reason that I don't think prepend and append functions
> are particularly useful for students, as the teacher should be encouraging
> them to think *more* about the data structures they're using, not less.
>

I presume you mean that thinking more about data structures and thinking
about computational efficiency are inseparable?  You can't do the former
without the latter?

cs

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: OK idea to replace conj and cons with "prepend" and "append" macros that have consistent behavior and return same types as args?

[Christian Seberino]

Ah no worries.  I think you made a significant point..." Shaping your use
of the language's raw materials to build up to your domain/application is
very much a Lisp philosophy".
Everyone can use Clojure as they need.  Amazing really.

cs

On Tue, Jul 17, 2018 at 3:58 PM, Robert Levy  wrote:

> Whoa, I think you are reading way too deeply into what I wrote.  Let me
> explain what I meant: Lisp is great, Clojure is great, the community
> doesn't typically change the core of the language-- the change process is
> very conservative and largely top-down.  The rest of it was a cheap shot at
> the drama in the Python community.
>
> On Tue, Jul 17, 2018 at 1:39 PM Christian Seberino 
> wrote:
>
>>
>> On Tue, Jul 17, 2018 at 3:13 PM, Robert Levy  wrote:
>>
>>> If you want to you can use the prepend and append found in tupelo lib,
>>> or you can write your own training wheels lib for your students.  You have
>>> total creative control over your course design.  Shaping your use of the
>>> language's raw materials to build up to your domain/application is very
>>> much a Lisp philosophy, don't ask Cognitect, go do it yourself.  Versus the
>>> style of language where users beg the BDFL, and the conflicting beggars
>>> create so much stress on the BDFL as to induce permanent burnout and
>>> retirement. ;)
>>>
>>
>>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: OK idea to replace conj and cons with "prepend" and "append" macros that have consistent behavior and return same types as args?

[Christian Seberino]

> data are functions and vice-versa
>

What do you mean? e.g. How is the [1 2 3] a "function"?

cs

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: OK idea to replace conj and cons with "prepend" and "append" macros that have consistent behavior and return same types as args?

[Christian Seberino]

On Tue, Jul 17, 2018 at 3:13 PM, Robert Levy  wrote:

> If you want to you can use the prepend and append found in tupelo lib, or
> you can write your own training wheels lib for your students.  You have
> total creative control over your course design.  Shaping your use of the
> language's raw materials to build up to your domain/application is very
> much a Lisp philosophy, don't ask Cognitect, go do it yourself.  Versus the
> style of language where users beg the BDFL, and the conflicting beggars
> create so much stress on the BDFL as to induce permanent burnout and
> retirement. ;)
>

No one is asking for free dev work.  As you ready said, the tupelo lib
already exists.  Not sure why discussing a question is a threat to anyone's
emotional health.
If anyone finds this thread "threatening" by all means please ignore it.

cs

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: OK idea to replace conj and cons with "prepend" and "append" macros that have consistent behavior and return same types as args?

[Christian Seberino]

>
>
> For example, in Python you can write {1, 2} to produce a set, or {1: 2} to
> produce a dict. But what does {} produce? Is it an empty set, or an empty
> dict? Python chooses the latter, leaving the literal syntax for sets
> incomplete. To my mind this is indicative of a design approach that sees
> literal data syntax as sugar for data constructors, and not something that
> deserves first class consideration.
>

I agree Python's syntax/grammar doesn't have the mathematical elegance of a
Lisp.  On a scale of 1 to 10, with 10 being the perfect beginners'
language, maybe Python is a 9.
I hope with Clojure's greater mathematical beauty, it can get to about a
9.5.


> Clojure, on the other hand, takes great care to ensure that its data can
> always be represented as literals. For data that isn't a standard
> collection type, there are tagged literals. Clojure syntax starts from a
> representation of data, and in order to really understand it, I think it
> needs to be taught from this principle as well.
>
>
Can you elaborate?  I really want to get your point but I'm not so
proficient with the lingo.  What do you mean by a "literal" and "tagged
literals"? And how is a focus
on the representation of data different than Python beyond Python's less
elegant grammar?

cs

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: OK idea to replace conj and cons with "prepend" and "append" macros that have consistent behavior and return same types as args?

[Christian Seberino]

Actually, even "real" programmers can live for a long time with those 
training wheels I was talking about.
As the Python guys have been proclaiming for 20+ years, performance isn't 
as important as we think for many (most?) applications.  
What the Python crowd does, when they're code is slower than required, is 
to profile their code and maybe rewrite their bottlenecks
in C or some other optimization.

By the time a Clojure beginner is doing the kind of hardcore concurrent 
applications that Clojure is great at, he/she will
know how to use a profiler.  At the point,  they'll be ready (and 
motivated!) to learn about the nuances of conj and friends.

I'm open to corrections and ideas.  

cs

>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: OK idea to replace conj and cons with "prepend" and "append" macros that have consistent behavior and return same types as args?

[Christian Seberino]

Actually, even "real" programmers can live for a long time with those
training wheels I was talking about.
As the Python guys have been proclaiming for 20+ years, performance isn't
as important as we think for many (most?) applications.
What the Python crowd does, when they're code is slower than required, is
to profile their code and maybe rewrite their bottlenecks
in C or some other optimization.

By the time a Clojure beginner is doing the kind of hardcore concurrent
applications that Clojure is great at, he/she will
know how to use a profiler.  At the point,  they'll be ready (and
motivated!) to learn about the nuances of conj and friends.

I'm open to corrections and ideas.

cs




_______

Christian Seberino, Ph.D.
Phone: (936) 235-1139
Email: cseber...@gmail.com
___



On Tue, Jul 17, 2018 at 1:33 PM, Christian Seberino 
wrote:

> Understanding the difference and why it's important are far more
>> illuminating than just forcing your prior model (like tupelo's
>> prepend/append). If your goal is education, then it's doubly important to
>> take this journey. It may be a few stops longer, but you'll actually learn
>> a lot more and create a mental model that will help you understand more of
>> Clojure's philosophy.
>>
>
> We both agree *eventually* it is important for students to understand the
> journey you are referring to.  I think where we possibly disagree is what
> to do in the absolute beginner programming courses.  For those I think I'd
> still vote for something like  tupelo's prepend/append for a few weeks
> until they get their programming legs.
>
> What is beautiful is that this does not involve some addition to the
> language standard or controversial vote to make it happen.  Basically,
> teachers can start with a beginners'
> library installed by default which they can discard later.  Think of such
> libraries as training wheels.  They aren't meant to last forever.
>
> cs
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: OK idea to replace conj and cons with "prepend" and "append" macros that have consistent behavior and return same types as args?

[Christian Seberino]

>
> Understanding the difference and why it's important are far more
> illuminating than just forcing your prior model (like tupelo's
> prepend/append). If your goal is education, then it's doubly important to
> take this journey. It may be a few stops longer, but you'll actually learn
> a lot more and create a mental model that will help you understand more of
> Clojure's philosophy.
>

We both agree *eventually* it is important for students to understand the
journey you are referring to.  I think where we possibly disagree is what
to do in the absolute beginner programming courses.  For those I think I'd
still vote for something like  tupelo's prepend/append for a few weeks
until they get their programming legs.

What is beautiful is that this does not involve some addition to the
language standard or controversial vote to make it happen.  Basically,
teachers can start with a beginners'
library installed by default which they can discard later.  Think of such
libraries as training wheels.  They aren't meant to last forever.

cs

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: OK idea to replace conj and cons with "prepend" and "append" macros that have consistent behavior and return same types as args?

[Christian Seberino]

Alex

Thanks for all the replies.  It is clear there are 2 values in language 
design...*simplicity* and *efficiency*.  Sometimes they conflict 
unfortunately.
Clojure sacrificed a tiny amount of simplicity for a huge gain in 
efficiency with the design of conj and friends.

Imagine someone wanted to have Clojure compete with Python and Scheme for 
introductory programming classes. In that space
simplicity is everything.  Maybe it would make sense there to teach using 
"prepend" and "append" functions to keep things as simple as possible.
Then, later when they were more confident and ready, efficiency 
considerations and conj could be explained.

Would that give the best of both worlds?  Everyone could have what they 
want when they want it.

cs




On Monday, July 16, 2018 at 4:30:51 PM UTC-5, Alex Miller wrote:
>
>
>
> On Monday, July 16, 2018 at 4:08:47 PM UTC-5, solussd wrote:
>>
>> Another way to think about it is lists and vectors are different and the 
>> idiomatic way to add items to them is different. 
>>
>
> I would say different data structures have different ways to *efficiently* 
> add items to them, and conj is an operation to add items efficiently 
> (meaning, sub-linear time). So when you see conj, you know it is always a 
> "fast" operation. 
>  
>
>> A (singly-linked) list is usually prepended to (otherwise you have to 
>> walk the entire list to find the end). A vector is usually added to at it’s 
>> n+1 index, where n is the size of the vector. The conj function is 
>> polymorphic.
>>
>> cons takes a seq and returns a seq. It only cares that it can get a seq 
>> on whatever collection you give it and will always prepend to that seq.
>>
>
> Slight modification - I would say cons takes a *seqable* and returns a 
> seq. For example, a vector is seqable, but not a seq.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

OK idea to replace conj and cons with "prepend" and "append" macros that have consistent behavior and return same types as args?

[Christian Seberino]

I'm impressed with Clojure syntax and I'm still a beginner.

I noticed conj behaves differently for lists and vectors.  I also noticed 
cons does not return a vector for a vector input.

Is there any downside to make 2 macros...prepend and append that behave the 
same for lists and vectors, and also,
return same types as inputs?

If I'm not mistaken, I believe it is for performance reasons that these are 
not already in the standard language?
For those that care about performance, it would be beholden on them to pick 
the correct macro for their
specific data structure.

Good idea?

Thanks!

Chris


-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] datahike 0.1.0

[Christian Weilbach]

Hi,

I am happy to finally announce a first version of datahike. From the README:

datahike is a durable database with an efficient datalog query engine.
This project is a port of datascript to the hitchhiker-tree. All
datascript tests are passing, but we are still working on the internals.
Having said this we consider datahike usable for small projects, since
datascript is very mature and deployed in many applications and the
hitchhiker-tree implementation is at least heavily tested through
generative testing. We are building on the two projects and the storage
backends for the hitchhiker-tree through konserve. We would like to hear
experience reports and are happy if you join us.

https://github.com/replikativ/datahike

This is our first release and we are very happy about all types of
feedback :).

Best,
Christian

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

ANN: denisovan 0.1.0 - A core.matrix backend for neanderthal.

[Christian Weilbach]

Hi,

I would like to announce the first release of denisovan.

https://github.com/cailuno/denisovan

"This library provides a core.matrix implementation for neanderthal. The
main focus of this library is to map neanderthal's high performance BLAS
routines to core.matrix protocols as closely as possible, while being
compliant with the rest of core.matrix. For documentation please look at
the core.matrix and neanderthal documentation, as this library is mostly
glue code. If you encounter any issues, including surprising performance
behaviour, please open an issue."

Next steps are the Cuda and OpenCL backends of neanderthal. I hope you
find it useful.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: [ClojureScript] Re: durable datascript experiments

[Christian Weilbach]

It now has its own repo as datahike and is on clojars to play around.

https://github.com/replikativ/datahike

We are also working on replication of the hitchhiker-tree here:

https://gitlab.com/replikativ/index-sync

So hopefully we will be able to stream replicate a datahike db soon.

Best,
Christian

On 31.12.2017 00:13, Laurens Van Houtven wrote:
> Whoa; this looks awesome. Thanks for publishing this.
> 
> On Sat, Dec 30, 2017 at 2:24 PM, Christian Weilbach
> mailto:whitesp...@polyc0l0r.net>> wrote:
> 
> Hi,
> 
> performance is now a lot better, ~3-5x slower than datascript for
> queries on the fractal index trees on my machine. This still needs
> tuning ofc., but I have tried to make it fast with a lot of profiling
> during idle times at 34C3. The fractal tree causes some overhead though.
> Inserts are a lot more expensive, but I expect them to be written to
> disk anyway. Therefore comparing inserts/deletes with datascript is a
> bit pointless, if you do everything in memory then it will always be
> more performant.
> 
> So it should be usable for small test projects now. GC is still TODO,
> but I would be really happy for feedback! We are developing this as a
> toolbox for distributed dataprocessing architectures.
> 
> Best,
> Christian
> 
> On 26.12.2017 12:18, Christian Weilbach wrote:
> > Hi,
> >
> > I have finally taken the time yesterday in a good end-of-year
> tradition
> > of ambitious hacks to bring the hitchhiker-tree (1) and datascript (2)
> > together. I have only touched the db.cljc namespace in datascript and
> > replaced all calls to the in-memory balanced-tree set (btset) with
> calls
> > to the hitchhiker-tree, the rest is vanilla DataScript. All tests pass
> > and I can store and load databases sufficiently with the three
> trees for
> > the indices (no datascript serialization used).
> >
> > I would like to have early feedback, so feel free to play and throw
> > ideas around. You need to install my version of the
> hitchhiker-tree (1)
> > first. Atm. it is a drop-in replacement for DataScript, but this is a
> > prototype, don't use it for anything serious yet.
> >
> > Here is an example interaction with the store:
> >
> >
> 
> https://github.com/whilo/datascript/blob/hitchhiker_tree_support/test/datascript/test/store.cljc
> 
> <https://github.com/whilo/datascript/blob/hitchhiker_tree_support/test/datascript/test/store.cljc>
> >
> > Caveats:
> > 1. There is no working GC for the hitchhiker-trees yet.
> > 2. Performance penalty against DataScript is in the 20-100x range,
> but I
> > haven't done serious profiling yet.
> > 3. There are probably bugs that are not covered by the tests, please
> > open an issue.
> >
> > Goals:
> > This is more of an intermediary goal, although it is very useful
> to have
> > a durable datascript around in itself. The hitchhiker-tree is a
> generic
> > persistent durable datastructure that can be efficiently synched and
> > hence replication is the goal. Next step is a "web-after-tomorrow" (3)
> > architecture similar to datsync, that we are discussing about here:
> >
> > https://gitter.im/metasoarous/datsync
> <https://gitter.im/metasoarous/datsync>
> > https://gitter.im/replikativ/replikativ
> <https://gitter.im/replikativ/replikativ>
> >
> > In the longer run we can allow multiple writers with different
> > conflict-resolution scheme similar to CRDTs and the bloom language
> > (dedalus).
> >
> >
> > Happy hacking :),
> > Christian
> >
> > (1) https://github.com/whilo/hitchhiker-tree
> <https://github.com/whilo/hitchhiker-tree>
> > (2)
> https://github.com/whilo/datascript/tree/hitchhiker_tree_support
> <https://github.com/whilo/datascript/tree/hitchhiker_tree_support>
> > (3) http://tonsky.me/blog/the-web-after-tomorrow/
> <http://tonsky.me/blog/the-web-after-tomorrow/>
> >
> 
> 
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> <mailto:clojure@googlegroups.com>
> Note that posts from new members are moderated - please be patient
> with your first post.
> To unsubscribe f

Re: durable datascript experiments

[Christian Weilbach]

Hi,

performance is now a lot better, ~3-5x slower than datascript for
queries on the fractal index trees on my machine. This still needs
tuning ofc., but I have tried to make it fast with a lot of profiling
during idle times at 34C3. The fractal tree causes some overhead though.
Inserts are a lot more expensive, but I expect them to be written to
disk anyway. Therefore comparing inserts/deletes with datascript is a
bit pointless, if you do everything in memory then it will always be
more performant.

So it should be usable for small test projects now. GC is still TODO,
but I would be really happy for feedback! We are developing this as a
toolbox for distributed dataprocessing architectures.

Best,
Christian

On 26.12.2017 12:18, Christian Weilbach wrote:
> Hi,
> 
> I have finally taken the time yesterday in a good end-of-year tradition
> of ambitious hacks to bring the hitchhiker-tree (1) and datascript (2)
> together. I have only touched the db.cljc namespace in datascript and
> replaced all calls to the in-memory balanced-tree set (btset) with calls
> to the hitchhiker-tree, the rest is vanilla DataScript. All tests pass
> and I can store and load databases sufficiently with the three trees for
> the indices (no datascript serialization used).
> 
> I would like to have early feedback, so feel free to play and throw
> ideas around. You need to install my version of the hitchhiker-tree (1)
> first. Atm. it is a drop-in replacement for DataScript, but this is a
> prototype, don't use it for anything serious yet.
> 
> Here is an example interaction with the store:
> 
> https://github.com/whilo/datascript/blob/hitchhiker_tree_support/test/datascript/test/store.cljc
> 
> Caveats:
> 1. There is no working GC for the hitchhiker-trees yet.
> 2. Performance penalty against DataScript is in the 20-100x range, but I
> haven't done serious profiling yet.
> 3. There are probably bugs that are not covered by the tests, please
> open an issue.
> 
> Goals:
> This is more of an intermediary goal, although it is very useful to have
> a durable datascript around in itself. The hitchhiker-tree is a generic
> persistent durable datastructure that can be efficiently synched and
> hence replication is the goal. Next step is a "web-after-tomorrow" (3)
> architecture similar to datsync, that we are discussing about here:
> 
> https://gitter.im/metasoarous/datsync
> https://gitter.im/replikativ/replikativ
> 
> In the longer run we can allow multiple writers with different
> conflict-resolution scheme similar to CRDTs and the bloom language
> (dedalus).
> 
> 
> Happy hacking :),
> Christian
> 
> (1) https://github.com/whilo/hitchhiker-tree
> (2) https://github.com/whilo/datascript/tree/hitchhiker_tree_support
> (3) http://tonsky.me/blog/the-web-after-tomorrow/
> 


-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

durable datascript experiments

[Christian Weilbach]

Hi,

I have finally taken the time yesterday in a good end-of-year tradition
of ambitious hacks to bring the hitchhiker-tree (1) and datascript (2)
together. I have only touched the db.cljc namespace in datascript and
replaced all calls to the in-memory balanced-tree set (btset) with calls
to the hitchhiker-tree, the rest is vanilla DataScript. All tests pass
and I can store and load databases sufficiently with the three trees for
the indices (no datascript serialization used).

I would like to have early feedback, so feel free to play and throw
ideas around. You need to install my version of the hitchhiker-tree (1)
first. Atm. it is a drop-in replacement for DataScript, but this is a
prototype, don't use it for anything serious yet.

Here is an example interaction with the store:

https://github.com/whilo/datascript/blob/hitchhiker_tree_support/test/datascript/test/store.cljc

Caveats:
1. There is no working GC for the hitchhiker-trees yet.
2. Performance penalty against DataScript is in the 20-100x range, but I
haven't done serious profiling yet.
3. There are probably bugs that are not covered by the tests, please
open an issue.

Goals:
This is more of an intermediary goal, although it is very useful to have
a durable datascript around in itself. The hitchhiker-tree is a generic
persistent durable datastructure that can be efficiently synched and
hence replication is the goal. Next step is a "web-after-tomorrow" (3)
architecture similar to datsync, that we are discussing about here:

https://gitter.im/metasoarous/datsync
https://gitter.im/replikativ/replikativ

In the longer run we can allow multiple writers with different
conflict-resolution scheme similar to CRDTs and the bloom language
(dedalus).


Happy hacking :),
Christian

(1) https://github.com/whilo/hitchhiker-tree
(2) https://github.com/whilo/datascript/tree/hitchhiker_tree_support
(3) http://tonsky.me/blog/the-web-after-tomorrow/

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Static purity analysis.

[Christian Weilbach]

Hey,

I have had finally the time to prototype an idea I had for over a year:

https://github.com/whilo/beichte

It is only a first prototype, but the basic mechanics work fine, i.e. it
is doing a deep recursive walk through your Clojure codebase (everything
that is on the classpath or filesystem) and tracks down impure var
accesses. I would like to get feedback of what you think about this and
would like see integrated if this becomes a linter.

I am thinking in the direction that you have to declare your vars as
impure explicitly, either by adding a "!" at the end of the function
name or by some metadata on vars (opt-out purity). This would allow you
to go as far as Haskell does in regard to enforcing purity without using
the type system and monads for it. This is not meant as a criticism of
Haskell, rather as a disentanglement of purity from the type system.

There will be a considerable effort needed in assigning purity to core
classes and methods and maybe also beyond Clojure to core libraries, but
maybe somebody has some ideas of how to ease this process(?).

Best,
Christian

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Disable namespaced maps formatting in the REPL?

[Christian Johansen]

Thanks! I was on alpha14. In 17, setting *print-namespace-maps* did not 
work for me in either `lein repl` or CIDER, but the print-method solution 
did work, though only in lein repl. One step closer.

Is this feature set in stone for 1.9, or is it possible to cast a vote 
somewhere on the default setting? I think it impairs readability and would 
gladly vote for `false` as the default value.

Christian

tirsdag 27. juni 2017 09.16.20 UTC+2 skrev Alex Miller følgende:
>
> Works for me in the clojure.main REPL (this is with 1.9.0-alpha17):
>
> user=> {:a/b 1}
> #:a{:b 1}
> user=> (set! *print-namespace-maps* false)
> false
> user=> {:a/b 1}
> {:a/b 1}
>
> It's also possible to redefine the map printer which should work 
> regardless:
>
> (defmethod print-method clojure.lang.IPersistentMap [m, ^java.io.Writer w]
>   (#'clojure.core/print-meta m w)
> (#'clojure.core/print-map m #'clojure.core/pr-on w))
>
>
>
>
> On Tuesday, June 27, 2017 at 1:06:27 AM UTC-5, Christian Johansen wrote:
>>
>> Hi,
>>
>> *print-namespace-maps* disables the new namespace maps formatting in 
>> Clojure 1.9. However, it does not seem to affect formatting of return 
>> values in the REPL (tried both the standard REPL as well as in Emacs with 
>> CIDER).
>>
>> Is it possible to completely disable this kind of formatting of maps with 
>> namespaced keys? Or maybe someone knows how to do it specifically for CIDER?
>>
>> https://clojuredocs.org/clojure.core/*print-namespace-maps*
>>
>> Regards,
>> Christian
>>
>>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Disable namespaced maps formatting in the REPL?

[Christian Johansen]

Hi,

*print-namespace-maps* disables the new namespace maps formatting in 
Clojure 1.9. However, it does not seem to affect formatting of return 
values in the REPL (tried both the standard REPL as well as in Emacs with 
CIDER).

Is it possible to completely disable this kind of formatting of maps with 
namespaced keys? Or maybe someone knows how to do it specifically for CIDER?

https://clojuredocs.org/clojure.core/*print-namespace-maps*

Regards,
Christian

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] replikativ 0.2.4 - An open, scalable and distributive infrastructure for a data-driven community of applications.

[Christian Weilbach]

Hi,

we have been very busy since :clojureD to make replikativ more
approachable. Most importantly we have significantly improved our
documentation and material under http://replikativ.io and for the
underlying libraries https://github.com/replikativ/

The most important new features are:

- A test-net running under ws://replikativ.io:
  Feel free to deploy prototypes against the test-net to have serverless
applications or run a peer or deploy your own internal cluster.
- Improved CRDTs, esp. the OR-Map and a new experimental merging OR-Map
- React native support including a demo project (chat42app with re-natal)
- Higher throughput of many MiB/s for high volume CRDTs through a faster
binary transport.
- Status notifications in the browser through hooking into timbre in chat42.
- A demo of IPFS integration in filesync-replikativ demonstrates how to
leverage DHTs over immutable data with replikativ's distributed write
coordination while staying cryptographically safe through joined merkle
hashes. Other possibilities are datproject, bittorrent or S3.

We will also soon have a polished JavaScript API which can be combined
with react nicely.

For 0.3 we currently hammock about index based synchronization with the
hitchhiker tree (1), that we have ported to ClojureScript recently. We
are also very interested in a p2p pub-sub overlay protocol like
PolderCast (2) with WebRTC (4). If you have any experience or
references, please let us know!

We also collaborate with the nice datsync (3) guys to build eventual
consistent datalog synchronization on top of onyx and replikativ for a
programming model for the large (similar to Eve, for example).


Feel free to ask any questions! :)


Best,
Christian

(1) https://github.com/datacrypt-project/hitchhiker-tree
(2) http://acropolis.cs.vu.nl/%7Espyros/www/papers/PolderCast.pdf
(3) https://github.com/metasoarous/datsync
(4) https://github.com/albertlinde/Legion/ already has several overlay
protocols, we know Albert, so maybe we can also figure something out
together.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Able to consume some nom modules, not others

[Christian Johansen]

I suspect that the problem is that the library I'm looking to fetch does 
not actually export anything. It only defines a polyfill. If there's any 
way to make that work I'd like to know.

Christian

fredag 26. mai 2017 10.59.20 UTC+2 skrev Christian Johansen følgende:
>
> I'm in need of an EventSource polyfill for my app to work in IE/Edge. So I 
> figured I'd take a look at the new npm support. I can get the left-pad 
> example from this post to work: 
> https://anmonteiro.com/2017/03/requiring-node-js-modules-from-clojurescript-namespaces/
>
> However, trying to build with the module I actually need does not work:
>
> (ns test.ns
>   (:require event-source-polyfill))
>
> Build with:
>
> (b/build "test/ns.cljs"
>{:optimizations :none
> :main 'test.ns
> :npm-deps {:event-source-polyfill "0.0.9"}
> :output-to "test.js"})
>
> Fails with:
>
> ExceptionInfo No such namespace: event-source-polyfill, could not locate 
> event_source_polyfill.cljs, event_source_polyfill.cljc, or Closure 
> namespace "event-source-polyfill"
>
> I can `require` the module from a Node repl just fine, so there's nothing 
> obviously wrong with the module.
>
> Any ideas?
>
> Christian
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Able to consume some nom modules, not others

[Christian Johansen]

I'm in need of an EventSource polyfill for my app to work in IE/Edge. So I 
figured I'd take a look at the new npm support. I can get the left-pad 
example from this post to work: 
https://anmonteiro.com/2017/03/requiring-node-js-modules-from-clojurescript-namespaces/

However, trying to build with the module I actually need does not work:

(ns test.ns
  (:require event-source-polyfill))

Build with:

(b/build "test/ns.cljs"
   {:optimizations :none
:main 'test.ns
:npm-deps {:event-source-polyfill "0.0.9"}
:output-to "test.js"})

Fails with:

ExceptionInfo No such namespace: event-source-polyfill, could not locate 
event_source_polyfill.cljs, event_source_polyfill.cljc, or Closure 
namespace "event-source-polyfill"

I can `require` the module from a Node repl just fine, so there's nothing 
obviously wrong with the module.

Any ideas?

Christian

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] Neanderthal 0.9.0 with major improvements

[Christian Weilbach]

Well done! :)

Am 28.04.2017 um 12:22 schrieb Dragan Djuric:
> Version 0.10.0 is in clojars.
> 
> On Friday, March 31, 2017 at 4:39:35 PM UTC+2, Dragan Djuric wrote:
> 
> More details in the announcement blog
> post: 
> http://dragan.rocks/articles/17/Neanderthal-090-released-Clojure-high-performance-computing
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to clojure+unsubscr...@googlegroups.com
> .
> For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: The major upgrade to Neanderthal (the matrix library) will be released soon, with lots of new functionality.

[Christian Weilbach]

Am 22.03.2017 um 02:41 schrieb Dragan Djuric:
> More details
> at: http://dragan.rocks/articles/17/Neanderthal-090-is-around-the-corner

Nice work! Hopefully I can play with it soon :).



-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: go local variable binding

[Christian Weilbach]

You cannot do so in cljs though:
http://dev.clojure.org/jira/browse/CLJS-1634

Just in case you expect to write cross-platform code with dynamic bindings.


Am 16.03.2017 um 01:01 schrieb Timothy Baldridge:
> Yes, that should work fine, do your tests confirm otherwise? Also if
> you're not doing a recur there's no reason to use `go-loop` you can just
> use `go`.
> 
> On Wed, Mar 15, 2017 at 4:44 PM, Eran Levi  > wrote:
> 
> Hi,
> can I bind variables, same as I do for threads, for go routines, to
> be local only for a particular go routine, and if I can't, how would
> you mimic this behavior ?
> 
> |
> (defn print-stuff [s]
>   (println s))
> 
> (go-loop []
>(binding [*out*(clojure.java.io/writer "foo.txt")]
>   (print-stuff)))
> 
> (go-loop []
>(binding [*out*(clojure.java.io/writer "bar.txt")]
>   (print-stuff)))
> |
> 
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> 
> Note that posts from new members are moderated - please be patient
> with your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> 
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> 
> ---
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to clojure+unsubscr...@googlegroups.com
> .
> For more options, visit https://groups.google.com/d/optout
> .
> 
> 
> 
> 
> -- 
> “One of the main causes of the fall of the Roman Empire was that–lacking
> zero–they had no way to indicate successful termination of their C
> programs.”
> (Robert Firth)
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to clojure+unsubscr...@googlegroups.com
> .
> For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

ANN: kabel 0.2.0 - A library for simple wire-like connectivity semantics.

[Christian Weilbach]

Hi,

I am pleased to announce the 0.2.0 version of kabel. It mainly provides
a new websocket client for the JVM with tyrus. This finally solved race
conditions (not multithreading, but an unsafe API) I had with
http.async.client (and the underlying apache websocket client). The
http.async.client is still optionally provided. tyrus is also claimed to
be small and android compatible, so kabel should allow systems to extend
both to the browser and to mobile devices now. I haven't tried Android
yet though. Additionally serialization is also implemented with the
middleware concept now. I am fairly confident that one can build solid
cross-platform systems with this release as is done by implementing the
replikativ database as middlewares for kabel.

>From the README:

kabel (German for cable/wire) is a minimal, modern connection library
modelling a bidirectional wire to pass Clojure values between peers.
Peers in Clojure and ClojureScript are symmetric and hence allow
symmetric cross-platform implementations. Clojure peers can connect to
Clojure and ClojureScript peers in the same way and vice versa. kabel
can use any bidirectional messaging channel, currently it supports
web-sockets. It also ships a transit middleware for efficient serialization.

https://github.com/replikativ/kabel

Best,
Christian

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: A more mathematical kind of set object in clojure

[Christian Weilbach]

Am 29.01.2017 um 21:15 schrieb Michael Lindon:
> Not quite what I'm looking for. There is an assigment in the Functional
> Programming in Scala coursera course called "funsets" - purely
> functional sets. A set is defined by its characteristic functions (a
> predicate) and source code can be found here
> 
> https://mwclearning.com/sourcecode/scala/funsets/src/main/scala/funsets/FunSets.scala
> 
> this is the sort of thing I am looking for.

What do you expect exactly from such an implementation? The definition
by the characteristic function might make elements of a set in theory
computably enumerable, but it is not practical to get a materialized
view. You can only query membership. Maybe if you can create a
persistent datastructure over characteristic functions you can actually
compose such a datastructure, but in fact you just compose predicates
(as is also done in FunSets.scala) and reify them as a type there.
If you just want such an implementation to fulfill part of Clojure's
contracts for sets, you would implement the corresponding interface:
https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/IPersistentSet.java

But you would have to throw on some of these methods and I am not sure
what the benefits are. You can always create new interfaces (protocols)
ofc. Do you have a concrete problem in mind?

Best,
Christian



-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

[ANN] replikativ 0.2.0 - An open, scalable and distributive infrastructure for a data-driven community of applications.

[Christian Weilbach]

Why replikativ?

Are you tired of building the same kind of glue code for your
distributed architecture over and over again? Then, once you have build
all this often redundant code coordinating state changes between
different ‘cloud’ systems, web services (e.g. RESTful) and your
client-side applications you try the impossible by bridging strongly
consistent storages with eventual consistent and volatile ones? Do you
think ad hoc solutions to cache invalidation are stupid? Do you wonder
whether there is a better way?


More information on our new project homepage :): http://replikativ.io/
To play around I recommend chat42: https://github.com/replikativ/chat42

The other projects should work, but are not updated yet. Please open an
issue if you fail with anything!



This release in particular offers:

- robustness. We have tested replikativ under serious load and ensured
that the frontend APIs developers face are safe to use concurrently with
our tests. We also have built-in the Erlang-like error handling of
superv.async to never miss an exception (hopefully ;) ).
- performance: We were able to follow Trump shitstorms on twitter over
more than a week including casual online synchronization onto a Laptop
with Datomic for analysis. This in particular included >200.000 commit
operations and more than 10 GiB managed data. There are still small
memory leaks, so you cannot run a peer in such a setting without
supervision for weeks yet, though.
- streaming: We now support stream-into-identity! functions for our
major datatypes: CDVCS, OR-Map and LWWR. In particular you can use them
to replicate data between different instances of databases like Datomic.
- more datatypes: We now have a very handy OR-Map, which also features
chat42, so you don't have to care about conflicts anymore with
distributed writers. The LWWR is also useful to have simple setable
(last writer wins) register.
- libraries: We have improved considerably on our kv-storage layer by
providing more backends and durable datastructures like an append-log
(trees WIP): https://github.com/replikativ/konserve
Our network library kabel has not seen as much love yet, and it probably
has a deadlock somewhere, as its tests get stuck sometimes, this needs
investigation.

I am keen on your feedback. If you miss anything or find something
unclear, please let us know!

Happy hacking,
Christian

P.S.: I will present the system at ClojureD.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

ANN: konserve 0.4.4 kv-protocol with core.async + new backends

[Christian Weilbach]

Hey,

not yet another kv-store? I written a blog post to clearify the design
decisions here:

https://whilo.github.io/articles/16/unified-storage-io

>From the README:

A simple document store protocol defined with core.async semantics to
allow Clojuresque collection operations on associative key-value stores,
both from Clojure and ClojureScript for different backends. Data is
generally serialized with edn semantics or, if supported, as native
binary blobs and can be accessed similar to clojure.core functions
get-in,assoc-in and update-in. update-in especially allows to run
functions atomically and returns old and new value.

https://github.com/replikativ/konserve

There are now two new backends:

- LevelDB: https://github.com/replikativ/konserve-leveldb
- Riak: https://github.com/replikativ/konserve-welle


I hope you find it useful and ping back,
Christian

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

ANN superv.async 0.2.1 - Erlang-inspired error handling for core.async

[Christian Weilbach]

After more than a year of exploration of different approaches to get
error handling right in the Erlang sense with core.async, I am finally
confident enough to release it as superv.async. I have tested it a lot
with replikativ in a distributed system, but I wouldn't consider it
production ready yet.

I have written a blog post about the experiences and different
approaches here:

https://whilo.github.io/org/2016/03/10/error-handling1.html

>From the README:

Let it crash. The Erlang approach to build reliable systems.

Errors happen, and proper error handling cannot be bolted on top of
subsystems. This library draws on the Erlang philosophy of error
handling to help building robust distributed systems that deal with
errors by default.

This is a Clojure(Script) library that extends core.async with error
handling and includes a number of convenience functions and macros. This
library is a fork of full.async. The original attempt to merge this work
with it failed due to the limitations of a dynamic binding approach. The
fork became reasonable, because full.async mostly deals with convenience
functions, but it is not as radically focused on proper error handling.
Since the error handling cannot happen through a transparent dynamic
binding, some convenience is lost in superv.async as you need to carry
around the supervisor lexically. If binding support comes to
ClojureScript for asynchronous contexts the projects might merge again.
The binding approach also has a performance penalty though. Since the
boundary of both error handling libraries is the exception mechanism and
core.async channels, they still compose, but supervision will not
compose with full.async contexts.

https://github.com/replikativ/superv.async

I hope you find it useful and feedback is very much appreciated.

Christian

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Neanderthal (fast CPU & GPU matrix library for Clojure) will also support Windows out of the box

[Christian Weilbach]

Nice! I have seen that the neanderthal licence file is in the libnd4j
repository:

https://github.com/deeplearning4j/libnd4j

Do you have some cooperation with the dl4j people?

Cheers,
Christian

On 04.10.2016 17:53, Dragan Djuric wrote:
> Hi all,
> 
> I've just spent some time building ATLAS for windows, and created a
> windows binary of the current snapshot version. 
> 
> There seems to be no performance tax on windows, or at least it is not
> large. I wasn't unable to compare it on the same machine, but on my i7
> laptop (2.4 GHz) with Windows, some brief tests that I run are at about
> 60% of the speed I get on i7 4770K at 4.4 GHz on Linux (single-thread
> mode). 
> 
> I guess this works as expected, and you can expect speedups as
> in http://neanderthal.uncomplicate.org/articles/benchmarks.html
> 
> So, from the version 0.8.0 and on, Neanderthal will support Linux, OSX,
> AND Windows out of the box! OSX will always run optimized (since it
> comes with its own native BLAS), and on Linux and Windows you'll be able
> to use a generic ATLAS binary or to compile your own optimized version.
> Either of those would be much faster than any pure Java alternative.
> There's something for everyone now (except for the ClojureScript folks -
> that may come in future versions of neanderthal) :)
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to clojure+unsubscr...@googlegroups.com
> <mailto:clojure+unsubscr...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: ANN: durable-persistence for Clojure and ClojureScript

[Christian Weilbach]

On 09.09.2016 01:39, dgrnbrg wrote:
> This sort of thing is also the goal of Datacrypt & Hitchhiker trees
> (https://github.com/datacrypt-project/hitchhiker-tree), which are
> designed to be highly optimized for I/O.

Sorry for the late reply, I was a bit busy. I have also implemented a
redis backend for our async kv-store protocol:
https://github.com/replikativ/konserve/blob/master/src/konserve/carmine.clj

Thanks for pointing out, this is very nice! I have just read
https://github.com/datacrypt-project/hitchhiker-tree/blob/master/doc/hitchhiker.adoc
Using a B+ tree approach would also have been my next take, but the
hitchhiker combination with an append log is really cool. I am
interested in having these sorted datastructures as building blocks for
CRDT metadata in the longer run.

> 
> We've been making progress on the GC and adding new backends--it would
> be awesome to be able add a CLJS port too :)

Cool :) I have only thought about GC so far. To support cljs one needs
to be aware that all blocking IO calls won't work. That is why I have
started with a core.async kv-protocol. Can you imagine using something
like it?

To support Datascript in cljs, one would then also need to use something
along the lines of async operations to fetch index fragments during the
query. But for the current hitchhiker tree it should not be too invasive
and the performance should not be affected. I think IResolve, IBackend
(and maybe some more), need to return go channels, the rest can be added
gradually. I haven't looked into it in detail yet.

Have you had a look at replikativ? I would be interested in feedback and
critique :).

I don't know what your current take on CRDTs is (I have found the nice
OR-SWOT in libdistsys :) ), I will be doing my master thesis with
Annette Bieniusa soon, one of the authors of the paper about it. I
haven't done as much Clojure backend integration as you yet, since I
wanted to have cljs support for an open replication system and couldn't
wire any JVM technology in (including the implementation of synchronous
standard datatype protocols for asynchronous operations). But I think it
is definitely a good way to start in the Clojure backend as you have
done. As far as I can see the CRDTs in libdistsys are supposed to be
used with a network (the simulator is a good idea). Have you plans for a
complete middleware?

Best,
Christian

P.S.: I have played around with piplin two years ago, do you still work
with FPGAs?


-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

ANN: durable-persistence for Clojure and ClojureScript

[Christian Weilbach]

Hi,

I have started some explorations for durable persitent indices in a
cross-platform fashion, that I wanted to for quite some time now. There
are some other implementations of persistent datastructures with
synchronous IO on the JVM, but I would like to go a step further and
generalize over any key-value store like medium. I am primarily
interested in critique and references to other work or experience
implementing efficient indices on disk. In the longer run I want to use
these indices as a building block for databases in Clojure(Script).

https://github.com/replikativ/durable-persistence

>From the README:

This repository is for exploration of persistent datastructures on a
durable medium. The durable medium is abstracted away by a minimalistic
key-value protocol providing atomic operations (ACID) on single keys.
This decouples the implementation of the persistent datastructure from
the host and backend store.

The repository contains an adaptation of the cookbook in-memory version
of a red black tree. Instead of keeping all fragments in memory we
introduce a Reference type to fragments in the store and dynamically
load and store fragments. The root can be assigned to a fixed key,
modelling a durable identity similar to a Clojure atom.

The long term motivation is to provide efficient indices as a building
block for datatype management independent of JVM IO libraries spanning
also to ClojureScript and the Browser. With konserve an IndexedDB
storage backend for the Browser already exists. For this reason IO
should happen asynchronously through core.async in konserve. A durable
Index for DataScript would be nice :).


Feedback is appreciated :),
Christian

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

ANN: konserve 0.4.0 - A clojuresque key-value/document store protocol with core.async.

[Christian Weilbach]

Hi,

I am pleased to announce the 0.4.0 release of konserve. The major change
is key iteration for the file-store and a fast append-log + reducing
function for all stores.

>From the README:

A simple document store protocol defined with core.async semantics to
allow Clojuresque collection operations on associative key-value stores,
both from Clojure and ClojureScript for different backends. Data is
generally serialized with edn semantics or, if supported, as native
binary blobs and can be accessed similar to clojure.core functions
get-in,assoc-in and update-in. update-in especially allows to run
functions atomically and returns old and new value. Each operation is
run atomically and must be consistent (in fact ACID), but further
consistency is not supported (Riak, CouchDB and many scalable solutions
don't have transactions over keys for that reason). This is meant to be
a building block for more sophisticated storage solutions (Datomic also
builds on kv-stores). An append-log for fast writes is also implemented.

https://github.com/replikativ/konserve

Cheers,
Christian

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: [ClojureScript] Re: Preparing proposal for presentation on replikativ

[Christian Weilbach]

@Daniel

> I would be very interested in learning about how to use replikativ:
> what can and can't it store, when is and isn't it a good fit,
> demo/sample code of common and not so common use cases etc
> I've looked at replikativ and even went down the "learn lots about
> CRDTs" rabbit hole and think I have an OK understanding of the
> theory, but I'm still not sure when and how to actually use
> replikativ. Although it's probably best to assume not everyone in the
> audience is familiar with CRDTs, so should probably include a brief
> primer.

Sure, recently there was a CRDT presence inclusion into the Phoenix web
framework for Elixir. So this would be to have simple backend
replication. I agree that the vision is still too broad for concrete
applications. I have mobile applications in mind, but Clojure for
Android doesn't work with core.async atm. The idea is to use it like a
simple in process Java-HashMap etc. For websites and node it works with
cljs, but still has some rough edges. Would this be interesting for you?

@Ashish


> Thanks for replikativ and asking suggestions. I would like to also
> know about general problems (both of domain and language) you faced
> while developing library, and how / how not clojure helped you ?

Ok. I guess this would be core.async error handling, kv-protocol with
konserve and hasch for crypto, stackwise. Helpful was the decomposition
of the stack and the ability to build a cross-platform stack with
core.async. Most parts were missing though, because usually people build
backends on top of JVM functionality/libs. In general as this is a data
management system, Clojure is a fitting language, but at some points the
nested untyped maps caused difficult debugging. I think this will be
better with spec now. On what level would you expect these arguments?

@Christopher

> 
> Would love to see this talk (embarrassed I still haven't watched the
> video you posted in Gitter though...).
> 
> As you'd suspect, I'm definitely like to see a good bit on
> DataScript/Datomic. Aside from that, I think a good focus on the
> strengths and weakness (/challenges/work-to-be-done) would be very
> informative, and help illustrate some of the implications of the
> approach. In particular, challenges in dealing with larg(ish) data sets
> and/or data that changes very frequently are very interesting.

Indeed. Ok, good to know. I wish I already had closer look at datsync
and how the two approaches could be integrated.

Thanks for pointing out,
Christian

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Preparing proposal for presentation on replikativ

[Christian Weilbach]

Hi,

I would like to present replikativ at the EuroClojure 2016 conference
(1). I would talk about the motivation and how it is implemented
(core.async, the foundational libraries, integration with
DataScript...). There is the background theory of CRDTs, the general
problem of data replication and synchronization, the particular case of
DataScript/Datomic replication, mobile or web frontend development etc.
I will have to focus on some aspects. Do you have any particular
interests I should address?

Best,
Christian

(1) https://github.com/replikativ/replikativ

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Clojure with Tensorflow, Torch etc (call for participation, brainstorming etc)

[Christian Weilbach]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 31.05.2016 07:17, Mikera wrote:
> I've been working with a number of collaborators on a deep
> learning library for Clojure.
> 
> Some key features: - An abstract API for key machine learning
> functionality - Ability to declare graphs / stacks of operations
> (somewhat analogous to tensorflow) - Support for multiple
> underlying implementations (ClojureScript, JVM, CPU, GPU) -
> Integration with core.matrix for N-dimensional data processing
> 
> We intend to release as open source. We haven't released yet
> because we want to get the API right first but it is looking very
> promising.

Almost all of the development in deep learning is done in Python, so
having to reproduce this work on a different runtime (and language)
seems non-Clojure-like for me (compared to being hosted on the JVM and
leveraging this ecosystem). deeplearning4j is already attempting it
from the Java side with the argument of distributed scaling, but
again, there is a ton of work done for the Python toolkits including
massive scaling (e.g. with tensorflow) and most research is there, so
my question would be what the actual goals for a Clojure stack would be?

Machine learning systems usually can be implemented very well as a
"microservice" because they have no state (assumed the model is
trained and constant) and just need to return an output given a sample
(act like a pure function). This has worked out fine for me in Clojure
so far.
There are many other important machine learning concepts and papers
beyond deep learning, which can be used that way. The network IO can
be problematic in some cases ofc., e.g. small models+low-latency
requirement, but I think the leverage is much higher and will increase
unless the ML community switches away from Python to the JVM (highly
unlikely atm.). I personally will rather focus on ML papers and
improving the algorithm on paper and in Python than reimplementing a
ton of work in Clojure just to get on parity in my favourite language.

Another point is performance. For many areas it doesn't matter too
much whether you are 5 or 10% faster as long as you can scale. In
machine learning it is often critical though as these 5-10% are
already hours and scaling can be hard model-wise. I have tried to get
performance on par with Theano out of Clatrix (both on CPU) a year ago
and Theano was 2-4x times faster on my machine. Even if a Clojure
stack would address this, there is a lot of work necessary to
constantly optimize the stack for different GPU architectures and
compute graphs. Theano has atm. 40 contributors or so, not to speak of
TensorFlow being backed by Google & DeepMind now.

I think a much better approach would be to bring Python native
bindings to the JVM. There is http://www.jyni.org/ and they seem to be
getting closer to numpy support, but are only two guys atm. (I think).
While this is a more low-level effort and has little to do with
Clojure, it would allow to use all Python libraries natively from
Clojure (together with core.matrix backends for numpy, etc.). Jython
is considered as slow as CPython, so one could expect to have
Python-like performance with a small wrapper library, because most of
the performance critical code is already in the native libraries. From
there emancipating with Clojure through a core.matrix based-stack
would be a non-uphill battle similar to the development of the Clojure
ecosystem.

What points would speak against an approach like this?


Christian



-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iQIcBAEBAgAGBQJXTXq+AAoJEICbLivqiPOFR1sQAKZYWzGz3mEFVQuItOUgFz8p
/zRh3oj2jLYOT5rHxEehZkZfQEjuRVMn6NW5nPR8c6mEzUc2FRNUTJHbDAgqaWSp
LxIOy5qfqzuA1J1x/hlsn1JRGMrvjZv+NvW2PpG8WSgZYwblIxdzzcRCYiRQ4+tQ
IhGDg1CKc2awGOJHLuJmzTuHtI+fIhvhDxRBEjvlfTdIInKugS5K0rwyiXr50jcx
zoO5jnhibcZB9LxskmW0J/8kH/hT2RD8mwjeI5oQYZuHZ/LvZX0+U9ocihyxoL2B
YFd2TwDc7ebgx71gsAnSTPcrIOfIwItprP4ka2gWtmXGJR3PZxfm5JlBir/gJIYf
Aa3uqR19qOFCgxwUCqFCWTVgojVFcF4F+VU7dXtfrQE5hkmBycSwbXiDh4CC11jV
Lhlff+yjv4OL2IYrPMBbVVU/KeWH+o4ETR0GaePRfGuBOEc04048F4Xz84NBZ6ke
lJhGL63JpUKqBJAPjZlU57VMoNMIczHdlMGF1oRhqkWzo0gD4ygX5C8g90xXGXLq
NjF/GiFEUWR1xzPvqLTNIX2kTveW46ZBDTiYCYCD8j+8yxGw04ow5wEoflbhz3Gd
PdWk7wb9bXlSHm6+b0Ax8CGQqMeDbb/RXqHneTDCQBjDW4olmzWfpRokUGj+K0ne
/1dgMs5AjIB+QHMW+PHZ
=3hFS
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails fro

Re: Porting Clojure to Native Platforms

[Christian Weilbach]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 27.04.2016 16:29, Jason Felice wrote:
> 
> 
> On Tue, Apr 26, 2016 at 7:42 PM, Plínio Balduino
> mailto:pbaldu...@gmail.com>> wrote:
> 
> 
> * Is there a way to compile C++ code at runtime?  This would be 
> essential for the REPL and for Macros. - I never heard about on
> demand native code generation and execution. I think it could be a
> paradise for viruses, but I know almost nothing about this field.
> Unless of course you're generating code for (your own|a) virtual
> machine, so the rules are different.
> 
> 
> There is a Scheme (I can't remember which) that generates C, spawns
> the compiler, links and loads a dynamic library, then invokes the
> code.  I'm having a brain fart because I want to say it's Gambit
> but Gambit actually has an interpreter.  Maybe it was CHICKEN?

https://github.com/bertfrees/cljc.repl also works this way.
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iQIcBAEBAgAGBQJXIwRVAAoJEICbLivqiPOFyo8P/0PgnVHkAroD+fTUTFR5ULjX
CzrEBycHYd4ot1qbRKGW4kNJ/MC+NhuT7TPGcMbnnhULrTZX+A04NTaXaUhKahtO
pO4xspkxjXRJdXQwno7AyqLSOnutdSvjU3aECsqTJN17dxt0+YNjMvyb1dp1PHMr
2aofDx6Pu4UwEbBtqDe8qPWADwcBhNj+Ci5oT447qFZyBD4V1+wQwsdX6xJjiYz3
nyghxasZv9kx8U+qNeBEOKTcPKoD7N+WQs5AjcIq9aRiHNcIf0xAKztd68/+bpg7
lzR46fQg6McdzV8ohomcajlpeg0SMe2hOMmugAMUab9Wuz9JzTMBq0RuZ6GZEjQ6
vGVQTwqfVqbD1dU48MmSPivtd8Ijou03dtq/CuwV3//QePUqUqgKUA8OMdoiQlNv
s124MznjQHfA+YkZxNCo5+7Atc9+OafsIlfH7+WDnmJcy3kDYStksKuNRX+heIpH
j4Fv5e9ILvUHYltgM0Pi1VozpLXodS9d9t/IRyG/SZtk/JqUmmHohNbYGzLuLT1Q
dXXJoVxRLo+qHjaT0ZdmKyRayonVNaV7yLczXVyqWjKVLM8Q3NbanaRQofzSM6jD
M9KtW4CRc22skfWaPKS/pNMJjvkIMHmgyMSwD+HF0rG1SIh06u1Qno/1Wqf7ciiF
u/dBpT3Li4ELAkJVojMI
=3h7o
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Porting Clojure to Native Platforms

[Christian Weilbach]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 25.04.2016 22:02, Jason Felice wrote:
> There was talk of an LLVM backend a while back, but I believe LLVM
> was deemed too low-level to be useful.  That was, in fact, why I
> signed the contributor agreement.  So, I'd love to see some
> movement on a C back-end.
> 
> Gambit Scheme has a special form that emits C (or C++) code.  It's
> very useful.  I can't remember which Lisp it was, but there was
> definitely a lisp which had an emit-C as it's *only* special form.
> It might have been one of the embedded schemes.
> 
> Last time I looked, I also saw ClojureC:  It looked useful, but
> hasn't been touched in 3 years.h

Just because you mentioned it, I played a lot with the ClojureC
compiler and implemented the metacircular evaluator for it two years
ago. There is a really nice REPL for it:

https://github.com/bertfrees/cljc.repl

But ClojureC is not fast, because statically compiling all the
abstractions in (look at the emitted C code), makes it slow. You can
directly call C though and get a self-contained binary. I guess Ferret
is better for embedded development though.

I found mjolnir really interesting and would like to see a way to
build native building blocks from Clojure that way. At least one could
implement performance critical numeric routines directly in Clojure
including annotations and call them from the JVM (e.g. on the GPU).
This should cover a lot of what Julia does, but I am not familiar
enough with its internals (e.g. JIT and how it standardizes tensor
memory layout etc.).


Cheers,
Christian
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iQIcBAEBAgAGBQJXH99KAAoJEICbLivqiPOFTNUP/ik2l/Bg528fvcBlX4WG+kOr
zLlNL7YgXu+rx/kZ+yVZuN/IQo5MW/TdRmOuZz7qkqb0nOuYNJ2J/fij4pUXISnn
tvlgVF4Pasj4sV0e/i4ZL5vo7T8KMd4UIb2D8YFdmNMXXFUw1OgBf10ucQvtljxt
eHGiDIf2etLYIfZupJAS5jtZqW17FK+K10sTZGdd5ZlCMUctl708BPXLlwayjpXM
erIXc7/hS5YIemC0iHEgNLYfyFLxIPa/Jid6PW3kZ7bJeB1QmwBTn472nI9GYMln
LF3DVx6QWK9B210qFz4HemgEKhkKj8/wNRR2uNpsBsYtYoMkUvB2tGeWuw6620G+
YFA2VOsVGpHaPPBxPgJLpT6/mN+ELmpmHdDk+L/UlFHw4eP/V/FlCNc1rnoR0HlM
j9Sqz11eJyy4rZRINC1N6PyXNDlTAFVOUypbb9dTAx7s2h/nBvqqLRAuMQ2kHHPf
8+Mp13EsSu2AaHDW/rFkM158zARn04gHiuFvtlCSjqZY6aCSCYAfKgY+v6YKEG1T
lhqyOlVo9i/tXCDCE3xov30Tp1BdqHDjuGyckfkkxFAJdzoylbuyCewR5GMppyie
FH93BZ63niWtwVXo8vuMAQHkiJnnVcw9BNqlFCI5cJH3LUTx0Kt/Jk1LhKxs0256
TZg/fBZqNQAYQtHMorJ1
=VsLc
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: ANN: Eventually Consistent Datatypes 0.1.0

[Christian Weilbach]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 26.04.2016 15:46, Ashish Negi wrote:
> Nice.. I was going through the Readme.. but it looked complicated.
>  Can you write simple cases where this would be useful ?

First of all, one should try the Clojure STM first ofc. It also allows
to mark parts of transactions (like incrementing a counter) as
commutative and non-aborting. You only need to drop to eventual
consistency, when consistency becomes too expensive due to parallelism.
For in-memory datatypes, in contrast to CRDTs, there is no
availability problem (unless you would start considering CPU
failures). But there is the contention problem for the STM. The idea
behind the research is, that to avoid this, you start implementing
eventual consistent in-memory datastructures.

A made-up simple use case would be a global log of summarized
operations for a system with 100 CPUs running mostly agents for
example. In this scenario there is no consistency (or serializability)
anyway, but you would like a global log to track progress of the
system. You could use the add-only-bag in this case without slowing
down the agents. They will merge with the global log/bag from time to
time. While 100 CPU machines might sound a bit extreme, I think we are
approaching them soon. GPUs are already at over 1000 cores.

Should I add some more information somewhere to make it better
understandable?

Christian

-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iQIcBAEBAgAGBQJXH9pwAAoJEICbLivqiPOFkigP/RRnLmHYhrYAAHtSQDu1UN4v
c3tS3zZqB5IgORUuCdcRi6rmm0DmeVTJXool3pELuWBsd/2NwpMxwHXnr72CsNgX
Jw7/73wq3/i9zTutaY6nRTaiUVg64N1zE8QqhJ/0oB8cSlPB/VuecP1Zc5Ie5rWW
TF0LJyyojr/PZg+OpnofZ2InI/EhgabV5hMmpxt1d0aIPqxtxMHXZTQ2A+cAqiHK
ZJncd8iFn4FMydaKTpVsjZegNIdHMH08KZVY1JdI91tf0YFQPwA/VD4c66j7hzBi
s058UurjqWFZedxe5Ic3eIbn6M0fwJMxzfjip91geJzvaSPG8RMifFDiBv5yOSLO
EIdM5yB515QJPla9moYESRvE/gace7mqI4QTOkpK1oVZAmO+Cjrpp/FrQUjgE0+Q
ObmqXJjfetFT8aaed+ZYrK0Kz1vfhNCuBDLvlKobK60j4i7rb+lm2nRfSLoPehLr
ibZ1aH/e0/BZ0fwuDHtiKYe0WdUL7DAheoHAnOeslDtdrrNJQxcGGmGIHbW0rEDx
YE4wKRkvF7DzjyvTHSVJ0ripsPcBGwPhYLAT+Abx6zniTOwBu6Y2qSEicX6vSiuz
BskbTughvbVyn09gF4co70jptYK9GguGHCBoHAyNx5jTY0FpNBoUR86lYVW+fPCa
NmLzLM8KD53dpMqF3zxS
=UoKO
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

ANN: Eventually Consistent Datatypes 0.1.0

[Christian Weilbach]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hey,

I was at EuroSys last week and was fascinated by a paper about
in-memory eventually consistent datatypes for massive parallism, so I
took a shot at them.

- From the README:

This is an implementation of highly-scalable eventually consistent
datatypes for Clojure. When you do massively parallel updates on STM,
the abort rate can become prohibitive and you don't want to wait until
your transactions succeed. One example is the counter from the paper,
where each thread increments the counter in some inner loop. And you
asynchronously merge the thread-local var from time to time with the
global state.

While I haven't had a use case for them yet, I found the research
interesting and wanted to benchmark the approach against the Clojure
STM. Consider the implementation experimental for now. I think the
datatypes become interesting in many-core scenarios where you have
more than 10 (maybe 100+) threads hammering on a datastructure. Since
massive parallelism is a primary design goal of Clojure, I think they
are worth investigating then.

https://github.com/whilo/ecdts

I would be interested if people have thought in this direction already.

Christian
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iQIcBAEBAgAGBQJXHnm3AAoJEICbLivqiPOFt2UP/3Yoi6dqJDzrpjo8l4/5DN8T
3y0njkUvKTIYI/ZrglE7Ow/IhWFO4VmOq5A+0BMJypqIlS9YGBX1qmaK9jxKg8SP
/6JTkz6J/aH2tzZc8pJnTi+tzGc39ERkreXiHxpqSRr5qtH60z/sN1N+1q4PRPGU
Dnn/52ABARlWTz4sdQOturFDToRt73THeFDFFtu6Bf1lnSjA0PzHqllAbkOYJDA6
lrbigieMemgu0fzJGMb/oIYcup2R1vuv8now9MaMsD8SWJCNwcosHI7r983nMeWY
Gwhy9qxslitoghVH+l1G878G4hNwU3mKL+68spG5QRwhILLR5IVKGpoq7EdOLp4/
h8gOIDR44Oj2iz8juSjV/4yKUFzgELG/jNLN8yjdKHOzvGiK/LITHa/HoZeMG2BL
rq/KAbuvF4BTzfZM5etxZTwXYvF9kP7dcjVUpgc+Az6Xovf546V90O1rYZ/GTeIU
MA4v8HRA3a1dpkt6olBGiLvbhvzB+KvT4NBIa0HSTdTDKgNhHRd8HhGZnHod0qCp
9hPfSxdiisyLYffvjkeCqhgNpOW8qpxzlud6PO47L+xX6bmpZ8rfqLXFtveUB3/7
7w591LEf3EZyYut0tkpvXKtsfR4ZwcckftdpHPsUyOVpalz5we3KzvX+K/kXuPFw
ek57mvP6ofrJasByt/Bi
=4B8Q
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] geheimnis 0.1.0 - RSA + AES for clj & cljs

[Christian Weilbach]

>From the README:

Implementation of cross-platform (clj, cljs) cryptography. The library
supports AES/CBC/Pkcs7Padding with a 256 bit key and RSA with
arbitrary keysize. If you need something which is not provided, please
open an issue. While `geheimnis` is not supposed to cover all
cryptographic options like OpenSSL compatibility with a big set of
chiffres, common and useful algorithms should be provided with a solid
implementation. `geheimnis` is supposed to be batteries included on
Clojure level, so doing the right thing should be easy.

https://github.com/replikativ/geheimnis


RSA is implemented directly with BigIntegers, so if you like it feel
free to hack around. Elliptic curve crypto would also be nice, but RSA
should be good enough. The problem is probably rather lack of
randomness in some Browsers.

I hope this is useful to somebody else.

Let's encrypt with Clojure ;)
Christian

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

ANN: hasch 0.3.0

[Christian Weilbach]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hey,

I am pleased to announce another major update for hasch, this time
with a finalized hashing scheme, so you can use it to store values.

- From the README :

A library to consistently crypto-hash
[edn](https://github.com/edn-format/edn) data structures on Clojure
and ClojureScript, e.g. with SHA-1. Commutative data structures like
maps, sets and records are not hashed in order as was the case with
e.g. hashing a simple printed edn string, but have the same hash value
independant of order (read: always and everywhere). UTF-8 is supported
for strings, symbols and keywords.
You can also create UUID5 (using SHA-1) from it. Alternatively you can
use your own hash function.

Underlying is also a new version of
https://github.com/replikativ/incognito

Happy hacking,
Christian
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iQEcBAEBAgAGBQJW9W9cAAoJEKel+aujRZMkRwgIAM1VPnT1DT57IHAANwzAMZNE
tLC078pNETqetN2KNRUQAj4FVXSNd5uECwOd1QsHKUp/egonGjpnSh8+D4FPqJj1
eoZmL11krFe0yVOsXUEag7+Y2SkbSZTaKEXQZcE6l+dkA3lQziR83GW8cZl14vHj
P4N6rSswmLsIOvAxB8FY/8laJ5+8erraIbpkiypq1m6hnsyvfh0fDwgt+lFAIy7v
Hkw8V1KpWNaiR3hbDq7WrGsr5JlsCYHukYCvldqEpbVRUx6bzQI4KaSIpEZd0WS1
qUyKGsiuYaJOT96QNXHG6Fs/Leb8RRx5aa8gw8rF6tVfmzvarGluirvKSFdVufs=
=kYY8
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: ANN replikativ 0.1.0 - strong eventual consistent P2P replication for clj and cljs

[Christian Weilbach]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 22.01.2016 20:35, Teemu Kaukoranta wrote:
> On Friday, 22 January 2016 20:53:25 UTC+2, Christian Weilbach
> wrote:
> 
>> 
>>>> There's two things that make this difficult to understand:
>>>> its academic nature, and the fact that many of us are just so
>>>> used to the central server that it's hard to imagine anything
>>>> else.
>> 
>> I am sorry for this appeal, I really tried to explain it well and
>> give good examples including a non-trivial prototype application
>>  (https://topiq.es). Why do you think it is academic? I have
>> written a paper and teamed up with the syncfree group for CRDT
>> research, because many eventual consistent (NoSQL) systems
>> promise things which they don't deliver and are poorly documented
>> (read aphyrs Call me maybe blog series to get scared...).
>> 
> 
> I want to make clear that I didn't want to imply anything negative
> by "academic". Rather I meant that this topic is probably pretty
> foreign to many (or maybe it's just me), and this is the first time
> I've ever even heard of something called a "CRDT". In fact the
> project's github page immediately made me think that this is a real
> and practical tool that solves a difficult problem. :)

They are cool, but the basic concept is fairly simple. You just need a
datastructure which converges if some operations commute. So think of
sets or monotone updates (increasing timestamp). The fancyness comes
when you combine these techniques to get better tradeoffs for certain
cases, but they can't defeat the CAP limitations. So you cannot have a
CRDT which models a strong sequential datastructure like a list and is
conflict-free. What they really allow you to do is expose the real
tradeoffs in form of datatypes instead of hiding it behind some
"NoSQL" abstraction.

> 
> I've actually read the blog post you linked many times, it's such a
> great, great post. I immediately thought of it when you announced
> replikativ, the blog post is a big reason in why I was so
> interested in this.

Yes, it is well summarized. :) Also Nikita is doing a great job with
Datascript.

> 
> 
>> 
>> basho is in this research group as well btw. and is building 
>> state-based CRDTs into Riak. I wanted to put the ideas and
>> concepts under serious scrutiny and give very high quality
>> documentation about the core design. But my motivation is not
>> academic, it is rather practical and political: I love data and I
>> love machine learning, but the current trend is that data is
>> always privatized. What can I improve to make it more clear that
>> this a very practical project?
>> 
> 
> Christian gave a great example a couple of posts up. To be honest,
> when I first read about replikativ I immediately thought about
> offline capability and rollback on failed transactions. The problem
> is that both of these can already achieved using (admittedly
> horrible) ad-hoc solutions, which can probably make people wonder
> whether they really need replikativ. It's unclear to me how much
> code I have to write to make the "git like" functionality you
> mention work (git is an awesome example by the way), and if it's a
> reasonable amount, why even abandon my previous ad-hoc solution?

I don't think you should abandon a working solution right now.
replikativ is still very young and it is not ready for production yet.
But for prototyping it can already significantly reduce code and allow
you to focus mostly on the app (client-side) in my experience, letting
replikativ take care of distribution and storage.

> 
> Again, this is not really what I think, just trying to put myself
> in the shoes of devs who don't understand replikativ yet. :)

You are right to do so :).

> 
> So to summarize: emphasize the git-like nature & offline and
> rollback capabilities. Show how much code it takes, and how many
> new concepts devs have to learn.

Yes, I want to have interactive time travel in an app, this would be
really cool. Hopefully I soon find the time to build it into topiq (or
something else). Most developers would love this and it is one of the
awesome things about Datomic.

> 
> This project is looking great, and it's clear to me that you're a
> very skilled maintainer who cares about the project!
> 

Thank you :)
Christian
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iQEcBAEBAgAGBQJWo6QfAAoJEKel+aujRZMk4eMH/iq9dovkdzkvfBciQHBKrgIA
ru0ky39m59Ld/XpxXjDlb7FbV/Wcx+4zybTSuK/K2PnojOqx/ighRo+FMTOHGMhu
SHQzXrkNq5jN7DUVzv9+pfv2LuGQGbFZFJt/kfHqG/igC4rNFC7F/F+JwAsOU/Wa
8T3yyg5oq51hwNAGxsOMHQNARh0oG3mP/vZ/

Re: [ClojureScript] Re: ANN replikativ 0.1.0 - strong eventual consistent P2P replication for clj and cljs

[Christian Weilbach]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 23.01.2016 05:11, Christopher Small wrote:
> Can you listen for changes on a server store?

What exactly do you want to do? If you want to track updates to a
CRDT, I would create a stage and subscribe to the updates as I do in
the topiq client:

https://github.com/whilo/topiq/blob/master/src/topiq/core.cljs#L177

This is still very "low-level" and I want to provide a proper function
for this, but maybe you get the idea. I can refactor it and put it
into replikativ, if this helps you.

> 
> Also, have you developed a mechanism for just sharing parts of a 
> distributed data structure?

Do you mean metadata? All metadata needs to be distributed for the
code to work (the commit-graph to be connected etc.), but not the data
(commits and transactions). The syncfree group did some work on
partial metadata replication with swiftcloud:
http://arxiv.org/pdf/1310.3107.pdf
They need a fixed number of replicas in the datacenters for their
vector clocks, this is not possible in an open p2p system. But I am
open to optional metadata reduction strategies with the help of some
centralized coordination, e.g. physical time, vector clocks or a
classical coordination service.

Atm. even all data needs to be backed up by each peer, but I came to
the conclusion that this is not necessary, peers can replicate the
metadata only and updates from them will then be dropped by peers
which need the data, but can't get it from them. I just wanted to
start with the assumption that storage is infinite and I don't need to
worry much about it (but bandwidth is not ofc.).


Christian
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iQEcBAEBAgAGBQJWo6EBAAoJEKel+aujRZMkurwIALavrxwYx7/wfkEp66NUS1Mu
G3XFUxgpT/yQZ4zT/nO9+xlB+6X4ZFGJs+4beLVR8vc6rY+2WrBUwU6O8kxA8BIV
Tt3ZZTUlSh8gxXUiYqO9XGmU605nbpdRYv2FA5xi1kTrmd5tZ2SsjaSMPB6SHbYE
MMqk6LZyxvrrYHizFbopVKdLP3awkCBCUbjVNZm7w8mLtxnTafItiumCepFtqZw6
AOGDbNWkdDG5f39cTTNShfnLE04XFl+b5AwqvSyXB8OfB6bi/w4toYrxXbGV3T/n
e/9tEvrfdsrc6zoHi4sQf8EdGEU5p75acrvb9Z1bbcMeiBhe5G87DwN3SP9KXfo=
=lb1U
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: ANN replikativ 0.1.0 - strong eventual consistent P2P replication for clj and cljs

[Christian Weilbach]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 22.01.2016 18:43, Christopher Small wrote:
> Replicative lets you distribute state, so any time you'd like a
> distributed system to be able to share state, this is something you
> could consider to do that. In particular, every web application is
> a distributed system, so it's something you could use any time you
> wanted eventually consistent state between server(s) and clients.
> So the real question is what shared state offers.
> 
> If you're not doing shared state, you're probably doing something
> where you send messages through some remote transactor that
> (ideally) ensures consistency of the system. But that means two
> things:
> 
> * You can't be working offline (at least, not without a lot of work
> that starts to look like ad-hoc distributed state) * If you want
> optimistic UI updates (good for responsiveness & UX), you have to
> figure out how to undo UI updates if those transactions fail
> 
> Replicative solves both of these problems. You can work offline on
> your copy of the data, and then when you're able to reconnect, it
> does a "merge" (analogous to a git merge). Ideally, the merge is
> clean and the remotes get copies of the work you did, and everyone
> is happy. But, if there are any kind of conflicting changes that
> need to be merged, you have to go through history conflict
> resolution process, again, quite like you would have to in git.
> 
> So in very broad strokes, replicative is sort of like auto-synced
> git for application data.

Yes, this description is pretty accurate :). There is no example for
manual conflict resolution yet, but we already have a commit graph
visualization to allow analysis and time travel (not implemented yet,
but the vega.js plot routine should work in cljs apps):

https://github.com/replikativ/flechtwerk

Originally the plan was just to implement something like git for data.
But over the last year I have refactored it a few times, so now any
CRDT can fit and replikativ just covers the distribution for you. The
C in CRDT most often stands for "conflict-free" (we prefer
"confluent", because CDVCS represents conflicts in its value), so if
you pick other CRDTs, e.g. sets or a LWWR you don't even have to do
conflict resolution and just have an eventual consistent datatype.

> 
> Chris
> 
> 
> 
> On Thursday, January 21, 2016 at 9:00:28 PM UTC-8, Teemu Kaukoranta
> wrote:
>> 
>> Thank you, this looks immensely interesting! I'm still having
>> trouble understanding when exactly one should use replikativ; are
>> you saying it will make it easier to build offline applications?
>> 
You might have a look at Nikita's very nice blog post:
http://tonsky.me/blog/the-web-after-tomorrow/
He has a slightly more central take on distribution, as there still is
one authority ordering the transaction log. Replikativ + CDVCS also
allows you to fork data and dissent on its state, so you are allowed
not to converge with another writer. But even if you only allow
offline edits, you basically have to deal with conflicts, which is not
really avoidable in his centralized design as well.

>> 
>> There's two things that make this difficult to understand: its
>> academic nature, and the fact that many of us are just so used to
>> the central server that it's hard to imagine anything else.

I am sorry for this appeal, I really tried to explain it well and give
good examples including a non-trivial prototype application
(https://topiq.es). Why do you think it is academic? I have written a
paper and teamed up with the syncfree group for CRDT research, because
many eventual consistent (NoSQL) systems promise things which they
don't deliver and are poorly documented (read aphyrs Call me maybe
blog series to get scared...).

basho is in this research group as well btw. and is building
state-based CRDTs into Riak. I wanted to put the ideas and concepts
under serious scrutiny and give very high quality documentation about
the core design. But my motivation is not academic, it is rather
practical and political: I love data and I love machine learning, but
the current trend is that data is always privatized. What can I
improve to make it more clear that this a very practical project?


Christian


-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iQEcBAEBAgAGBQJWonqNAAoJEKel+aujRZMkZeYIAL5LC2wkPGh2prbq0ZHeIXOv
WKFV1BA7uDF9IzfhGBK6Wgs0t++wsHQu+1ovZlDPM0XDliQ7/tIdONsZi2Cud15T
oPrXVmyjJ8KEZXl9Skb9dEwVw79d8ZPYEdO75MoJB9+/bebad2H1ol1n3NEQYdVw
GRbGLiGhOgJPzR5N/VVV8H/OFhtywxy0VYC9PRCB6N2EPpJvMPvBUyObhbGjEOCI
ci2yNrsmA1lVRMoWtxoHlPKfOWdpRqM5Snr2gzGhB6xGa6lUYcF0t7xGy0zP6tcd
luCtdtbcX1BTPu+xYmDJS9s2lP3L0xOiGblVTOv07Kjlgy6bwRNQL/LFRm/GOeU=
=LY3W
-END PGP SIGNATURE-

-- 
You received

Re: ANN replikativ 0.1.0 - strong eventual consistent P2P replication for clj and cljs

[Christian Weilbach]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 21.01.2016 06:22, Zubair Quraishi wrote:
> This is great work! Can it be compared to something like Datomic
> and Datascript?
> 
It is mostly complementary with Datascript and it can be combined with
Datomic as well. I have so far basically replicated the transactions
with replikativ before applying them to a Datomic in memory database
or Datascript so far.

The main difference between Datomic and an eventual consistent system
like this is that Datomic has one writer and therefore always a
consistent view on the world, while CRDTs in general are eventually
consistent. Even worse, to get a strong sequential order in CRDTs you
have to introduce conflict management, so the git-like CRDT "CDVCS"
that we have invented can replicate a transaction log, but needs
conflict resolution in the case of multiple writers (different CAP
theorem tradeoffs). We think there are many cases where this is
acceptable. In section 5 are some thoughts so far:
http://arxiv.org/pdf/1508.05545v1.pdf

One idea is to use a CRDT like an OR-set for commutative transactions
(e.g. mostly additions) and find a proper composition of CRDTs for the
semantic of the distributed application you are building. The easiest
way is to model something like Datomic where replikativ is
distributing the transaction log to Datascript and you coordinate
writes on a single peer (Datomic does a lot more like efficient index
creation and distribution for you though...). That way you don't have
to care about conflict resolution in the beginning and if your app
grows you can find a proper CRDT composition.

So in general you need to think a bit more about how you coordinate
your writes and that you pick the proper datatypes for the semantics
and write-load of your application, but you get writable replicas
everywhere. Datomic alone is much easier to handle if you just need
one central server and don't want to have offline writes for example.
Om next can provide you some offline functionality, but this of course
is limited by similar tradeoffs.

A goal of this first release is to better understand how people
approach state modelling in combination with Datascript and how
replikativ can make this easier. We already have snapshot isolation to
compose CRDTs, but I haven't implemented other CRDTs yet, as I would
like to have your input first :).

Did this comparison help you?

Christian
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iQEcBAEBAgAGBQJWoVT0AAoJEKel+aujRZMkx9EIAJcL8YQbtbP79dguAszbzJdR
iIkYydrrPIAFMF3XFaA65+1Yev97T4SiJbdpf4HBWhz7LUohHfbcz53h/xuVIKhj
jf1JrIgJy7urN3w7WTuLoITQMHFszw6l5wlNm2QK9ijuDlougkfwFgcqu6uddIz2
/pyZ+Efzw/O6S9sLUtVwzKNeE5E1IdJUGuwO3GVyg/kTV3q8ag2mo992yodK7phb
TkwlU5AYikPWCbBg2/daxbOM6Ussgk37n2Gg1VTAJI/B1o3mVpH/SVAbO8/2mKSW
U3yfrX2LGNgaE9rteYF6kdB1tg+eTk8QsWO3BHSp4h8tbVbUPZHYpKc57NqSGLE=
=APNm
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ClojureScript] Re: ANN replikativ 0.1.0 - strong eventual consistent P2P replication for clj and cljs

[Christian Weilbach]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 20.01.2016 17:30, Thomas wrote:
> Looks very interesting and I suspect there were some pretty hard
> problems to solve!!!
Since I started to explicitly use the CRDT formalism, I don't have to
reimplement the wheel and there are a few pretty cool ideas out there,
e.g. the CRDTs for text editing. Most important for replikativ is to
make state transitions atomic (both in memory and on disk), so
implementing CRDTs is fairly straightforward now (you can even do so
in your application through a few protocols and a multimethod). In
general getting there was a bit of work, since you can't just reuse
JVM libraries or some databases. But this also reduced the
dependencies, simplified the design and increased portability a lot.

> 
> Thank you for open sourcing this.
You are welcome :). Actually I think it has to be open-source, because
what we really would like to do is share data between application and
service providers and overcome the current Internet of data silos
(both for political and for pragmatic business reasons). It was after
some inspiration from EuroClojure 2013 that I recognized I really have
to build this, but it was a long and often lonely journey, that is
finally over :).

I am curious about what the Clojure community can come up with in this
area, as Clojure developers mostly open-source libraries, but not so
many build open-source applications as for instance free-software
communities in Python do.
Concretely I hope to get some better picture of how to combine CRDTs
with Datascript (or Datomic), so you can model nice views with React
like Om next does and still have scalability and offline availability
by picking the appropriate CRDT composition. In topiq for instance one
can imagine to put single topiqs in OR-sets and remove the explicit
conflict resolution for the sequential application of transactions of
CDVCS since operations commute, while using a CDVCS with strong
consistency for a personal social network profile etc.

Christian
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iQEcBAEBAgAGBQJWoU7AAAoJEKel+aujRZMkYBsH/1pM8qEnnAhbXAmSwunvc6HR
chIzihaM6I4IdXWjB7KFj78lejwS8/ang/038qUNxLu+5R/4e26f0n2hZg9Xdywu
FEWY7dq5s+D7bODGYtNuW9aNDZwscfls/tFoMs1DZ80RrYQzMYBlsHWE+V6LJ6Co
d1+jC8l0MZKQJ/xiXHB3/21xafPRbqoDj+lUV61Tz4ojgjX33vSysm5JvqgVM/Uy
Sbji2Wmbd45tpRjp7B8x5nzP5akshJv3tH9VtulTEyOh2CdasPF7PxPg2CG67AFH
Zapw92RB9FZ2U0nbV5/S6vD3p6nKgKPGeKqEi9c9uwd0mm3+iCfSfxl8ky7BB3I=
=ukOj
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

ANN replikativ 0.1.0 - strong eventual consistent P2P replication for clj and cljs

[Christian Weilbach]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hello,

after three years of laying ground-work for a cross-platform database
in form of many libraries (1), doing research about CRDTs and
stretching core.async and other libraries as far as possible, I am
happy to finally announce a first release of replikativ:

replikativ is a replication system for confluent replicated data types
(CRDTs). It is primarily designed to work as a decentralized database
for web applications, but can be used to distribute any state durably
between different peers with different runtimes (JVM, js atm.).
Instead of programming thin web-clients around a central server/cloud,
you operate on your local data like a native application both on
client- and (if you wish to) server-side. You can also view it in
reverse as a cloud being expanded to all end-points. You can write to
CRDTs whenever you want and also access values whenever you want no
matter if the remote peer(s) (servers) is available or not. In
combination with our CDVCS datatype you can imagine it as git for data
(expressed e.g. in edn) + automatic eventual consistent replication.

https://github.com/replikativ/replikativ


While there are still some issues and the design is not completely
finished, I am pretty confident from our different design iterations
and our running prototype that the current one can avoid
race-conditions and is robust to errors. The interesting standard
CRDTs are still missing, but I decided to first hear some feedback
before growing the codebase and implementing optimizations.

Let's build more open systems and share data,
Christian

(1) https://github.com/replikativ/

P.S.: The prototype https://topiq.es is currently hosted on a home
server, if it loads too slowly, I will move it, but so far I felt a
bit romantic about my basement and didn't want to spend money for an
AWS instance or something else. Feel free to host your own instances
and to connect them ;), it should be straightforward.
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iQEcBAEBAgAGBQJWnpxRAAoJEKel+aujRZMk/uUIAMbAHVHOo0zNbbRr6QsapiLN
ohQHTVqixkj/8qS3+z6ZmEGy572t2DH+QzXpHOqtqAS3mxGMikFKk078yWAYD3W3
QbZoxssDjgu/CGWsGAjuUetd8DoI1vI1T1oAVTo4IDo9uot5NEDHs3s5ZLB50NIX
WOjm/muSPwkTt6B+oIp8ZsEYCH6RyLzTqkK6rOXxF0OoPv2XuK+TMgQJVzskmiaI
59Yf1TLizERN6DpyZbtFrWiVlgFF0+0K7GyW1qa7Bp7Yf9LE9yGra2WMjRwDdg7z
3SbZb5aXc/yaGztu+yN0wq3BWFCRSZQ9fh+VjrltGMA9BfExr53/aed1bIIcAtY=
=v65c
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ClojureScript] Re: [ANN] kabel 0.1.0 release

[Christian Weilbach]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 26.12.2015 00:51, Jethro Kuan Sheng Yuan wrote:
> Have you seen castra before? Seems like both are trying to achieve
> the same objective.
> 

No, I haven't seen it before. Thanks for pointing it out. I probably
haven't communicated the objective of kabel clearly then, as castra is
on what I consider the opposite of the spectrum of what kabel is
supposed to be.

castra, to my current understanding, models a high-level RPC mechanism
over a REST API and tries to make the connection between server and an
explicit web-client easy by providing the necessary infrastructure and
syntactic abstractions with macros.
On the other hand I tried to break the concept of a connection down to
the minimum of a bidirectional wire (as opposed to a single core.async
channel, which does not have wire semantics, e.g. latency). Building a
wire then can be done in a platform-neutral manner with two core.async
channels. The library is focused on providing this single abstraction
and composes in a ring-like fashion out of plugging wires together.
There is no distinction between server and client, except that
browsers cannot open ports, but this is not baked in the design and
could change with WebRTC for instance. You can also for example
connect to another peer from the JVM in exactly the same way as a
browser would do, which is what I am doing in replikativ. (1)

kabel provides little convenience atm., e.g. no authentication, but
decouples different middlewares by messages as values similar to
Erlang, so all kind of convenience functionality can be implemented
with the same wire "pattern"/abstraction and can be stacked together
in a decoupled fashion.

Does this distinction make sense to you?

Christian

(1) https://github.com/replikativ/replikativ/
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iQEcBAEBAgAGBQJWftnWAAoJEKel+aujRZMkBFwIAIO1romV3n7IsIO9QkfMoxzn
8+f/jyJUYflDrk5e71izN3ZjsyhrY3McXPkeA+u12KJrJghl5ow279pkzSlkcFQx
zkKlf33XcTkN/FZHCWINJH+TkN7b7hf+OZHQQvQ6hRAUkeX1jIyX/JawhlchGV3F
KU0bsZbqVZ5V/1JjNmIdABsSfQxhic+hhdFuRsvMlfqglD5KsCXde+Y1H8v21zFK
CDytK1d0Jcn8Ge6Bf3Dqzf2fKqIPdo143pvdAmOsxBd1WE21mX8KMorKmZzjBvJi
wP9gQPZ60wUM/jZTWY0FVi6PTGIVD8ajcEX4JLxUHA/iROWLEmblJR4VeAEgJCs=
=28UU
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ClojureScript] Re: [ANN] kabel 0.1.0 release

[Christian Weilbach]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 25.12.2015 08:18, Zubair Quraishi wrote:
> I like the idea of this. Isn't there anything similar already in
> Cljs and Clojure though?

Do you know any?

I sadly do not, one and a half years ago when I started implementing
the communication concept in replikativ I had a close look at both
sente and chord, but they make a strong distinction between server and
client (e.g. with AJAX fallbacks) and add more convenience infrastructur
e.

I added another section to the README on how the peer itself has a
core.async pub-sub core. This probably needs a full example at some poin
t.


Christian
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iQEcBAEBAgAGBQJWfamOAAoJEKel+aujRZMkpRMH/RvcvCDCjWY1/hc4iZkP12/j
3v7gSgv2j3mrmdcrJnI2l0k4uC+aiGjTdMdQuqaHLBYmtbvl2rVZ+THmQQeyjxzO
bScp/b7nSRhEgoW8gPzD+hdzQYlq0WWn5qELnHAis1x2T8/bPdQlLdP5iWdsQtUM
TZOamHg9Q12a0Q9BatmIjAY/BhHQbnA5+D3U5a0l3t0qPyA3VjW9ofpDBOY/Hl1N
Geqr0XpKmeRcN4NI6gErXps9YfD4o6xTBvncy2WwOFJSC2aLsS1/nTTCBijjJJ4F
+dGak/i4+jKbR1QUoXXUTl5ARBXPZSG4F1OdOZIuiHqeXgMMfCjtqPsWQqRpcUQ=
=YQuo
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] kabel 0.1.0 release

[Christian Weilbach]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hello,

I am happy to announce the first independent release of kabel (German
for wire or cable), a websocket library that is used by several of my
projects. From the REAMDE:

kabel is a minimal, modern connection library modelling a
bidirectional wire to pass clojure values between endpoints. Peers in
Clojure and ClojureScript are symmetric and hence allow symmetric
cross-platform implementations. At the moment web-sockets are used and
transit is the serialization format.

https://github.com/replikativ/kabel

I hope it is useful for others as well. Feedback is appreciated :)

Christian
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iQEcBAEBAgAGBQJWfCVxAAoJEKel+aujRZMk5g4H/0Wp84eGotnBwQItb17NsGc+
/Vt5HiG9nEO1FriylPMkoGgnwhdYDKs7BXhJJGPB9JIe2adJKZFQgL9Wpx0WiLC/
borxP7LcwmY1sbFquy+AZ5+kl6PSqDyM0eolfKuL+kRlXgaFeMhfopZ3562yLf71
FjQmn2NS+h+i0OP+obSjl+dDJFE8ndv7lJYgWXWklnDZST9ctVNVD+kyY0nXXfo3
db5wcGvtU0MBhngzDMyhzrJtgXp4RqvcIMICesN63osFkfADzKWo2VKV/MSxPNk1
ZsYB89OxBzWuj6fmIEBZQ+v9HDTbExiVr467W0N2DL67YcjqyAzKrDOERu1MaCc=
=y9hg
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] konserve 0.3.0

[Christian Weilbach]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hello,

I am happy to announce the release of version 0.3.0 of konserve, a
clojuresque durable key-value/document store protocol with core.async.

Most importantly serialization protocols are now factored as a
protocol with implementations for Fressian, Transit and
pr-str/read-string. The protocols are also now wrapped by real clojure
functions. For more details have a look at:

https://github.com/replikativ/konserve

I have also updated the CouchDB backend:

https://github.com/whilo/konserve-couch

Feedback is (as always) appreciated,
Christian
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iQEcBAEBAgAGBQJWSfz6AAoJEKel+aujRZMkkkcH/2oJGiMrbEPZL1SmjEHisiBO
F1vGg2QJYpUOpP/CxxtT7EC8n/BcixnlsGM/FdqcHfB3l/2Liit8aSUD9hCWJW8N
Pas5+EFdOtfD73nc4B5CfDuDo+y+EIlExvktbi3ljnuKVRuZ/X6fG7FBX6+tlhBg
itKDrRZYXs9UIA/N/MRoPHeTvs76XBzLsNo2k+FMRIcUe6pu6D/Chdc5klXDKbfc
fyoPi9s9ZuoBCn0xfpSxum+39LODoREjgr7Wxv44vDDSXayEnMkETZLDPj61rxHb
A960OMXVOXsUpSIRVaU0UJRpmZIm7IcFc3Olwu+6YJbSQhMq7BXohqAQPSoe15U=
=8ZEp
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

ANN incognito 0.1.0

[Christian Weilbach]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hey,

I would like to announce the first release of incognito [1]:

- From the README:

"Different Clojure(Script) serialization protocols like `edn`,
`fressian` or `transit` offer different ways to serialize custom
types. In general they fall back to maps for unknown record types,
which is a reasonable default in many situations. But when you build a
distributed data management system parts of your system might not care
about the record types while others do. This library safely wraps
unknown record types and therefore allows to unwrap them later. It
also unifies record serialization between `fressian` and `transit` as
long as you can express your serialization format in Clojure's default
datastructures."

This is motivated by the need to carry records, e.g. of DataScript, in
a p2p system [2].

ClojureScript transit support is still missing. Feedback is very
welcome :).

Cheers,
Christian

[1] https://github.com/replikativ/incognito
[2] https://github.com/replikativ/replikativ
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iQEcBAEBAgAGBQJWEX4nAAoJEKel+aujRZMkWpcIAKxb5otl7TzyLw9EIKsPaLSK
i6Nbv+8cY0+/8urPj5pF5ImIqyu7qaWEf/RxOu5N/ViH3GsqXvrH+QmxrB0B3tXV
eEOx5RTE4yBl1r8C623en2MSX1xV3TsmhbDuEDKDMfwMSmwR4jbBXr7kf/PNwf/F
cBW5jntSTq53ZSDgcU0/M2wVAJxjdUWHSk/l8qV1JmuyMKDXXh70RNZxQMGpaU2c
riZG5Ea2Xy7fydO19p7paRTctKRMQfS0e2oVHHZngRkmuE9G93d1ppvKnhTatmv5
izgpIwZfSTNpnRxuKLgJVstMcR3UI7agQHwIgp3da5PN6hOnN4cqzimxOPFYk5g=
=I1ip
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ClojureScript] Erlang-inspired error handling for core.async

[Christian Weilbach]

Hey Moe,

> 
>>
>> into and pipe should probably be safe.
> 
> 
> I was thinking of a case where a function is returning the equivalent of:
> 
> (async/onto-chan ... [1 2 (Exception.)])
> 
> And the consumer wants to async/into [], obscuring the error from  looks like you've defined < 
> I have some distantly related code here:
> https://github.com/nervous-systems/glossop/blob/master/src/glossop/util.cljc -
> keyed-merge (when used with a/into {}) is a slightly different approach to
>  
> Can you find constructs/cases I am missing?
>>
> 
> I haven't taken everything in, though I think it'd be great if your channel
> operations were usable outside of the context of a supervisor, so pieces of
> the approach could be considered/adopted without going all-in.

Since I have to track all exceptions, you need to have a *super*
supervisor binding for the code to work (and you will later want to have
it in production setups to catch problems slipped out of explicit
supervision scope). I provide now a default supervisor, which for now
printlns errors to stdout so you can see them in the REPL (which is imo
also better than core.async printing stacktraces to stdout of the
nrepl-server through the thread pool).
Is this not modular enough for you? As long as you don't use supervisor
code and have no stale errors you will not hear from the default
supervisor and can minimally use go-try with http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ClojureScript] Erlang-inspired error handling for core.async

[Christian Weilbach]

Hi Moe,

> 
> I haven't felt the need for Erlang-style supervision in core.async,
> but am generally interested in moving around errors
> asynchronously.
> 
> How are you dealing with failures which pass through higher-order
> channel operations, e.g. into, map, pipe, etc.?

into and pipe should probably be safe. The best way to implement pure
channel logic now are transducers. I have added a chan-super
constructer which automatically reports exceptions in the transduction
to the supervisor. But for cljs I need to interleave control flow with
blocking side-effects, in which case the transducers don't help. For
this purpose I have refit the for-comprehension to work with channels
instead of lazy-seqs. I have added an example for both higher-order
constructs here, just uncomment the respective expressions and see how
you receive the exception after 3 retry attempts:

https://github.com/whilo/full.monty/blob/41ddec45f707ca4e29f67df4c8cff09b91bb8de6/full.async/src/full/async.clj#L452

Can you find constructs/cases I am missing?

> 
> Take care, Moe
> 
> P.S. What does >? do?

It is just there to throw an abortion exception when the supervised
context dies. I understand your confusion, but this is my best bet to
abort all nested contexts by intercepting all blocking ops and
throwing exceptions there. This allows you to clean up safely with
try-catch-finally locally, if this is needed (e.g. file-handles,
sockets, db connections...). Best would be some preemption, but this
needs runtime support or very invasive code instrumentation, I think.

Christian

> 
> 
> On Sun, Sep 13, 2015 at 5:30 PM, Christian Weilbach < 
> whitesp...@polyc0l0r.net> wrote:
> 
>> Hi,
>> 
>> I am working on a replication system which also works in the
>> browser (1). So far I have come a long way with core.async and a
>> pub-sub architecture, but I have avoided error-handling in the
>> beginning, just using a cascading close on the pub-sub
>> architecture on errors (e.g. disconnects). Lately I introduced > and go-try as described here (2) and extended that to go-loops
>> with dedicated error-channels. But this never felt right, as
>> error handling in a distributed system should not be an
>> afterthought. Erlang has a very successful and sound error 
>> handling concept, as Rich also pointed out (3).
>> 
>> The idea is basically that uncaught errors will always happen at
>> some point and when they do they just propagate and trigger the
>> restart of whole subsystems. Processes in Erlang terms, which are
>> somewhat an extended version of go-routines, have ids and are
>> explicitly wired to each other to receive exit messages when one
>> of them fails.
>> 
>> To retain the robustness of Erlang, I have reduced the concept to
>> the following 2 requirements:
>> 
>> 
>> 1) All errors (exceptions) need to be caught and propagated. 2)
>> Resources need to be freed reliably.
>> 
>> 
>> Since we operate with channels in core.async and not namend
>> process as in Erlang, the supervision needs to be passed to
>> routines by some other means unless go-routines were to be
>> globally registered as in Erlang. The most natural way is passing
>> a parameter to the routines, but this can be verbose. To reduce
>> this, one could use bindings, but these are thread-local and
>> difficult to reason about, which might break 1).
>> 
>> For requirement 2) the Erlang VM can preemptively terminate
>> processes. Since this is not possible in core.async, I have
>> decided to inject "abort" exceptions on every blocking channel
>> op, ?, alt?, ... This is not perfect, but will eventually
>> throw an exception in every go-routine and free all resources
>> satisfying 2). The supervisor then needs to track all running
>> go-routines under its supervision and only restart the subsystem
>> once all routines are finished. Using the default
>> try-catch-finally exception mechanism inside of go-routines + 
>> bubbling through > along Java/JavaScript semantics.
>> 
>> Example (4):
>> 
>> (let [test-fn (fn [super] (go-super super (try (> 500)) (catch Exception e (println "Caught:" (.getMessage e))) 
>> (finally (> start-fn (fn [super] (go-super super (go-try (throw (ex-info
>> "stale" {}))) (test-fn super) (> "foo" {}] (> :stale-timeout 100)))
>> 
>> Since one can accidentally leave exceptions stale in some
>> channel without taking them, we also need to track these and act
>> after some timeout to satisfy 1). I am really wondering what
>> other people have thought about error handling with core.async so
>> far

Re: Clojure/Pedestal vs Go

[Christian Weilbach]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

> I'm about to start a new web project and they are thinking about
> using Go (golang) instead of a JVM (preferably Clojure) based
> approach.  The idea is "BARE METAL SPEED!!!", but I really think
> the network and DB will be the bottlenecks, not Clojure vs Go.

You might also be interested in
http://docs.paralleluniverse.co/pulsar/ The go take on CSP is limited
when you really want to scale, while having an actor Erlang-like model
can also give you benefits, so in Clojure (or on the JVM), you can
evolve your application differently, while go basically has one
(well-done, but fairly primitive, e.g. error handling) concurrency
concept baked into the language semantics. I haven't used go myself
though and I guess best would be a small prototype application both in
go and Clojure + some experiments. It would be nice if you publish the
results somewhere and ping back.

> 
> Is anybody out there aware of any speed comparisons using
> Clojure/Pedestal and/or Go?  I'm thinking basic measurements like
> connections/sec, latency, simultaneous users, etc.
> 
> Thanks, Alan
> 

Christian
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iQEcBAEBAgAGBQJV9fW6AAoJEKel+aujRZMkxxoH/3bWMiFW/brS2t+o9qcq7gH6
NsIx+WL9YuSe0Fy2/lp/WKD7owLLM6gIoSfQIbYYurRqSlb4+hjKoei5Wqnxg6TF
Meq1I3D0uSJmMcQ7ux3d6Nn/XloXbUR9QDL8IVUZKNiCM+jz9s4XNBhTVRyhp2UI
0ubV7uXJGZV6XVun2UV2zoLDZV4PwXS0O45GB+ta+q+RhJ1bKZtdKtEBq6p449Hs
f6mdHWo+0MtWnHzOpOo7r2sLwkdwkRSi9ItWLB0UUbp4/5UQqmE+zUjYQR3wVIKe
307R8NFAb0sZp9fk/rtklIemIeUXslOTid//sdDHmKBWRCa8vKuEKiYDwQvR0t4=
=itJZ
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Erlang-inspired error handling for core.async

[Christian Weilbach]

Hi,

I am working on a replication system which also works in the browser
(1). So far I have come a long way with core.async and a pub-sub
architecture, but I have avoided error-handling in the beginning, just
using a cascading close on the pub-sub architecture on errors (e.g.
disconnects). Lately I introduced ?, alt?, ...
This is not perfect, but will eventually throw an exception in every
go-routine and free all resources satisfying 2). The supervisor then
needs to track all running go-routines under its supervision and only
restart the subsystem once all routines are finished. Using the
default try-catch-finally exception mechanism inside of go-routines +
bubbling through https://github.com/replikativ/replikativ
(2) http://swannodette.github.io/2013/08/31/asynchronous-error-handling/
(3) http://www.erlang.org/download/armstrong_thesis_2003.pdf
(4)
https://github.com/whilo/full.monty/blob/3ba439c6971c8255d34b2d39f1b7619b3b016236/full.async/src/full/async.clj#L451

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Hashing With Consistent Results

[Christian Weilbach]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi Atamert,

sorry for replying late.

On 11.08.2015 10:29, Atamert Ölçgen wrote:
> Hi Christian,
> 
> hasch looks nice, I might end up just using it. I will be hashing
> smaller collections (maps where keys are keywords and values are
> atomic data like integers).

Ok, then io will probably hurt you much more than some overhead for
hashing, I guess.

> 
> Collisions BTW are not such a big deal for my use case. I will have
> a limited number of fragments (buckets, index pages, etc.) anyway.
> 65536 of them perhaps. The more I think about the problem the more
> I realize I am implementing some sort of hash map.

I guess a durable one. In this case it might be interesting to think
about extending the persistent datastructures of Clojure in a way to
keep them on disk. I am currently experimenting a bit with that on
hash-maps of a commit graph, although I need it to work in
ClojureScript as well and cannot just reimplement core protocols
because of async io.
That way changing metadata of my datatype can have constant size.

Feel free to post any feedback on your progress/findings :).

Christian
-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQEcBAEBAgAGBQJVzITYAAoJEKel+aujRZMkZX4H/j5kpqolsS61y2IH+68Bq55/
sdiME1eXdJ4VgYQH+IF4WDTYtPGZgV7U3XnM7Bqc5SygkGxOvDc5p4piTeSfpJIh
HE8GkBP8RkQNU5rqKu0M6xeSJGQdnAp/1VzQdJux9KAC2+0RG+SLKKft95zka9iz
PHDy+n/m8qTrMSUjpk2tVxuglyjkGaQeBm9bfRN07Cn/96e9XcafzsekMwZiI8HU
70n5ACbBWFXz5zxe0xfoUdA48OJSXrnoQTCmA95zOLnZ9thHgs066jjXCjNtomzD
NRhx7J9hi4lU54VmRcYJb4mVw5JLXQCWnARh8//o6P2SbmYFJkDIsCLtPJ9xJu4=
=H4lz
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Hashing With Consistent Results

[Christian Weilbach]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

I am the author of https://github.com/whilo/hasch

Would calling hasch.core/edn-hash satisfy your performance
requirements? I tried hard to make the recursion of the protocol
performant, but hashing a value is slower than the time needed to
write the data to disk for big collections. You should pick a faster
message-digest like you suggested, e.g. MD5:

(defn ^MessageDigest md5-message-digest []
  (MessageDigest/getInstance "md5"))

(edn-hash {:foo "Bar" :baz 5} md5-message-digest)

You can use the criterium benchmarking snippets in platform.clj to do
benchmarks. Object.hashCode() is a lot faster still and caches the
result, I am not sure how much overhead the protocol dispatch causes.

Note that if some collisions are ok for you, you might find a better
tradeoff, since atm. commutative collections like maps and sets are
hashed key-value wise and then XOR'd for safety. I am interested in
your findings and decision, especially if you pick something else.

Christian

On 10.08.2015 09:00, Atamert Ölçgen wrote:
> Hi,
> 
> I need a way to reduce a compound value, say {:foo "bar"}, into a
> number (like 693d9a0698aff95c in hex). I don't necessarily need a
> very large hash space, 7 hex digits is good enough for my purposes.
> But I need this hash to be consistent between runs and JVM versions
> etc. So I guess that rules out standard object hashes.
> 
> I would like to find a sufficiently fast way to do this. I can live
> with MD5, but are there faster alternatives (but produce smaller
> hashes)? ( clj-digest <https://github.com/tebeka/clj-digest>
> provides a nice interface to what Java provides but there are only
> usual suspects AFAICS 
> <http://docs.oracle.com/javase/7/docs/technotes/guides/security/StandardNames.html#MessageDigest>
>
> 
)
> 
> I will be dealing with unordered collections, but it seems hashing
> is consistent when the input order is changed:
> 
> user=> (.hashCode {:foo "Bar" :baz 5}) 2040536238 user=> (.hashCode
> {:baz 5 :foo "Bar"}) 2040536238
> 
> 
> (It even gave the same hash code in different runs.)
> 
> I will use these hashes to build index tables. My data, that
> contains these things I hash is a set. I will store this as an
> ordered set and keep an index pointing to where records from this
> hash to that hash lives. This is all Clojure, but I can't keep all
> my data in memory. (So Clojure's persistent data structures is out
> of the picture. life would've been much simpler if I could.)
> 
> Thanks for reading. Any insight is appreciated.
> 


-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQEcBAEBAgAGBQJVyJ3vAAoJEKel+aujRZMkbhMIAJ61DGUWM9JoN/JcIxvh2Jph
VohlWbr1yw69D+x4guGOk5AXUh7HMAkmlbuc+YRRnYqGhZtc3r/6C/d/aa5faBAh
NdIeDa8yNHTAuYERDktfviy+q5a/blJRdvIIe7ntyjpDZyd2gD1AwUGYOKctXipS
wMPan7v7yPfPlFfnl+VVXfP8yx/LWyZbwfu0Ugv2B2NhvqPMu8joyondOz7GPcLd
P7EgpIrvfQAElA4c4+UB0BEeJkn+fnpYF3QLJIy5oQny5QwbVtxgVuUNES8EolYl
HkpFY1ECV/M65fvP6wrcYPihuphSYQoPkfY4ZQfzWCq9mo+3Aj1Jq2u7QfG9HxM=
=1UE6
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: clojure, not the go to for data science

[Christian Weilbach]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

>> 

>> http://viewer.gorilla-repl.org/view.html?source=github&user=ghubber&repo=cnc&path=rincanter.clj
>>  I am not sure whether this fits the design atm. though. I also
>> had a look at renjin, but I think the native plugins mandate an
>> RVM integration atm.
>> 
> 
> You might find http://beakernotebook.com interesting, if you
> haven't seen it before. It allows several languages in the same
> notebook, and has  a simple mechanism for sharing data between the
> language environments. It's pretty green at the moment, but it has
> some serious backing, so could be pretty useful once it's got a bit
> more polish on it. I chatted a little with someone who was hoping
> to implement a Clojure plugin for it, although I don't know how
> they've gotten on since.

This is really nice, thank you for pointing out! Effectively though
this means you need to manually(!) convert data between all cells,
which only works if serialization between the environments is a lot
less work than processing the data and you don't want to call alien
procedures e.g. in a loop. It also means that part of datasets need to
be hold in memory in all runtimes at the same time. So I am not sure
whether JSON synchronisation of state between runtimes is good in the
long run, it feels fairly hacky and having that many runtimes already
causes a big type-conversion matrix and probably many subtle
incompatibilities. It is the most pragmatic approach though and
probably the most successful for now.

They are not opinionated about languages, while I would like to
integrate data science tooling in Clojure, so it is easier to bring
people on board, not to make language hopping the top goal, which is
for mentioned performance reasons and from a Lisp/Clojure perspective
a bit questionable imo (esp. since all these languages have to drop to
C to get some performance). I also would like to have the environment
written in Clojure (read: gorilla). I would be happy to integrate
Python and R on the JVM, e.g. through Jython and RServe/renjin, which
allows to share code much better down to direct method dispatch. Do
you think integrating R cells in this way would be reasonable for you?
Or just allowing plugins for new cell-types...
(With RServe manual synchronisation with the RVM is still necessary
btw., renjin would solve this, but needs more support for CRAN
packages imho).

> 
> I also have taken a paper I liked and implemented a quick version
> of
> 
>> SNE, which was fairly nice to do in Gorilla REPL:
>> 
>> http://viewer.gorilla-repl.org/view.html?source=github&user=ghubber&repo=cnc&path=stochastic-neighbour-embedding.clj
>>  A problem seems to be unicode support, I tried to use some math
>>  symbols from the notation in the paper directly, but the viewer
>> seems to have a problem with it.
>> 
> 
> Not sure what the problem is there - my feeling is it should just
> work, on the Gorilla side. If there's a test case you can point to
> (with an expected result) I can take a look at it.

The problem is just that viewer.gorilla-repl.org seems to have some
encoding problems with UTF-8. See the sum sigma symbol for instance:
https://github.com/ghubber/cnc/blob/master/stochastic-neighbour-embedding.clj#L72
which is corrupted in the viewer. I hoped that was easy to fix.


Christian
-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQEcBAEBAgAGBQJVIYNTAAoJEKel+aujRZMk5wsIAIl/CoX0E7u6lSJIHQfWI8U5
HZWUPjHJie+sbIt+G4qm8SozAbmPEc6mRSDC3kP1v2JhoJv4y1j6klWmHH9Xm3dj
eqxNgVTNClk5ZUUcAFhcy2gUCc9RP6AJF27TnSO+ArG4qApueU+s297uHTAfGefk
hj+AbzCGoQOgvw1F6sUOTQ4KV6Z51/pYsjIgxwPC5vvzIKgv+qiKXMWLow1mM9uu
tG4h/HDSxdjCWdVm4sCuipi5tJJmdYEuYHjtFiMVbpVA5YZT3KleL4K6Gn9nMu7u
vrw2YlZyE9mlKPbv4GRS3ANnVv45+e0yLuMMXV1OdlxwKqmvKosmk7k4K6jR8Pc=
=5QnK
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: clojure, not the go to for data science

[Christian Weilbach]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 30.03.2015 13:35, Jony Hudson wrote:
> I propose, instead of this discussion, everyone channels their
> energy into writing an open-source data-science library, or blog
> post/article promoting Clojure for data science. In their favourite
> editor, of course!

I have started working on R integration with the help of rinancanter
(1) and it was nice to have dedicated R code cells with at least
syntax highlighting that I can mix with Clojure code in the same JVM
environment.
http://viewer.gorilla-repl.org/view.html?source=github&user=ghubber&repo=cnc&path=rincanter.clj
I am not sure whether this fits the design atm. though. I also had a
look at renjin, but I think the native plugins mandate an RVM
integration atm.

Sadly JyNI has made little progress towards Numpy support, (2)
otherwise Clojure could integrate more mature data science platforms
like R and Python more easily and leverage its integrative JVM
advantage more. The fragmentation on runtime level is really stupid imo.

It can be difficult to address some data science techniques in
Clojure, because you have to reimplement them from R, Python or
something else compared to Clojure being able to use Java libraries
directly and "sitting on top of the hill" by default for "business"
problems. Java is just not a good data science language and hated by
many people in my environment. But gorilla is really nice in
combination with ggplot already, so I think if one could at least
integrate R there, the disadvantages would shrink significantly.

I also have taken a paper I liked and implemented a quick version of
SNE, which was fairly nice to do in Gorilla REPL:
http://viewer.gorilla-repl.org/view.html?source=github&user=ghubber&repo=cnc&path=stochastic-neighbour-embedding.clj
A problem seems to be unicode support, I tried to use some math
symbols from the notation in the paper directly, but the viewer seems
to have a problem with it.

Christian

(1) https://github.com/jolby/rincanter
(2) https://github.com/Stewori/JyNI/issues/2#
-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQEcBAEBAgAGBQJVGVPFAAoJEKel+aujRZMkLBYIAKp+kzS7l0iiKdKVhRl1e0wP
utRHY+KFwmIRpc0vBf1Y5I5Lo2jFlemslDlWmQb5iPcYfALpLrzhHVP4CL4nRy+c
SMfnHPOS4nD6usAbTjyi3ZP8/wTbL3Du6oGQmwQMcMpklTgh+qgBsLmLdYabwXcm
YMifMYZVjSEtFcAz/FLxukLGGOSa86Nld79Wgj9hu2kTpS3ZFIx2gSpNzeeyiZBU
O+QtKTVAM603vBDQF0dgYu5u45pfsW/SZ0dThOvKN9632MX8Xb+SzzS34pxtxGWr
/JMpffYOIZ7kw0qto1zKKwewoO0zmhCjmOXDS063wYfHWvEqhcia9ChdT/4rxvY=
=0XIe
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: clojure, not the go to for data science

[Christian Weilbach]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 30.03.2015 15:12, Lee Spector wrote:
>> On Mar 30, 2015, at 7:35 AM, Jony Hudson 
>> wrote:
>> 
>> I propose, instead of this discussion, everyone channels their
>> energy into writing an open-source data-science library, or blog
>> post/article promoting Clojure for data science. In their
>> favourite editor, of course!
>> 
>> 
>> Jony
> 
> 
> 
> That's a good idea, but I'd also like to say a bit more about the
> pro/con-emacs discussion, which I hope to be constructive.
> 
> I think I actually agree with most of the comments both by the
> emacs critics and the emacs proponents in this thread. Even the
> most intense ones, on both sides. But rather than worrying about
> who is more correct I want to point out that it's entirely
> possible, and would be gloriously beautiful, for an emacs-based
> Clojure environment to be produced that:
> 
> - Can be downloaded in a single click and run with one more click
> to do basic Clojure development with no further configuration, on
> Mac/Windows/Linux.
> 
> - Provides reasonably standard GUI elements (familiar to any
> computer user without reading a manual) for triggering core
> functionality and for discovering additional features.
> 
> As some have mentioned in this thread, a lot of work has been done
> on easing configuration (by people on this list among others) and
> there are some GUI-based packages out there, but as far as I know
> there's nothing that comes close to meeting both of the bullet
> points above. I think that most emacs-based folks either don't
> think this is possible or don't see it as a priority, but something
> like this must be possible (and there have been things close to
> this for other Lisps in the past), and  if it became a reality that
> I would switch to it for all of my coding and teaching and I'd
> evangelize it from the rooftops.
> 
> I'm not in a position to do development work on this myself, but I
> believe quite fervently that this would be a fabulous thing for the
> Clojure community.
> 
> I'd be happy to discuss this further off-list and/or beta-test
> projects aimed at these goals.

I have failed to setup Emacs for development and used Vim or Eclipse
primarily before entering Clojure-land. emacs-live allowed me to just
execute a shell script and get a decent Clojure environment which I
have used for 2 years until I have reconfigured my emacs having a
better understanding. Its emphasis on live-coding inspired by
composing music is a very compelling. The only thing I had to add to
emacs live was evil-mode, which wasn't too hard.

Also installing packages in emacs 24 is a breeze, no match to Eclipse
plugin repository mess and screw-ups and I don't want to use a closed
source IDE. No matter how many resources you have, being able to run
emacs in a shared tmux session is a big plus. I even have aliased
emacs-client as vim, so I don't have to rethink starting vim for quick
edits and still do it in a common emacs environment. I also find it
interesting that Rich still uses inferior-lisp mode and I think people
get too obsessed with tooling. I usually wait until it really hurts
and then I try to adjust my tooling.

It is really important that the nrepl middleware is shared between
IDEs and I had the impression recently that Cursive and others started
to integrate features directly into the IDE, which I don't think is a
good strategy.

A thankful shoutout again to the emacs-live maintainers!
https://github.com/overtone/emacs-live

Christian
-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQEcBAEBAgAGBQJVGU+dAAoJEKel+aujRZMkMRIH+gJJ1rLqB4NZNDZHmZSz6E2W
7OwsDXcn7bqZeaGv/s/VRjZ9Hk6saxS2hJP+PEzWSnXU48fBvYSAU8tu657U85tk
wFuWqm2A5kl/RyKwukYY1o05GYmVQvkGA2cKM63rTItXL/4pkhMpMQhwkOZHe1Zt
ZesL4A8B1A5jiZqp3bwhHIt/c1Awi33XG2BrFmMqdlSINJ3t0ADDNZra5328G1yZ
eZDIO+ljRISkfxAOz/u45yi/KT9mDWe7AXqhavKwDFXbFZxmofrPNAHYRkB/+WuY
vZAuw+zyVvY8NyBmBNKW1b5oqljQtUBQxSbNdC1sdE9Z1pe2yYsyOD0Jp3EZjR0=
=jt1D
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Advice on core.async and (JDBC) transactions

[Christian Weilbach]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 23.02.2015 18:20, Colin Yates wrote:
> Currently each request gets serviced in its own thread (web
> container) and I am thinking of integrating core.async and I wonder
> how core.async and a JDBC transactional "unit of work" get on.
> 
> Conceptually, this model (thread-per-request) is trivial however
> the problems are well known. Replacing this with core.async right
> at the front is trivial but I can see the benefit of sprinkling
> asynchronous behaviour throughout the (still very trivial)
> pipeline. Or rather I can see the beauty of decomposed components
> communicating via channels.
> 
> My question is about transactionality. I am used to JDBC
> transactions being thread local and I understand core.async
> utilises a thread-pool, so how does one implement a "unit of work"
> that spans multiple channels?
> 
> I want to do something like:
> 
> - request comes in - the appropriate handler/service consumes it
> (either through request mapping, defmethod whatever) - TX starts -
> in parallel some logging happens (and DB is updated) - the message
> is handled (and DB is updated) - performance metrics are stored
> (and DB is updated) - all work on all channels gets finished - TX
> commits
> 
> The point is that channels are used only to communicate between 
> disconnected components but they should all participate in the same
> TX.
> 
> Is anyone else using channels like this?
I have done it in IndexedDB in JavaScript and had to use the proper
callback API (1), because otherwise transactions break. I guess you
cannot do channel operations inside a JDBC operation since they are
executed in the threadpool (by the state-machine) while each JDBC
connection seems to be pinned to a thread, so you cannot participate
in the same transaction from another thread of the threadpool. But if
you can wrap your blocking JDBC stuff in a singular thread call, you
might be able to use async/thread (probably less than you want).

There is also https://github.com/niwibe/suricatta, but I have no
experience with it. jooq seems to solve the the thread per request
problem decoupling into async operations you have in mind.

Cheers,
Christian

(1)
https://github.com/ghubber/konserve/blob/master/src/cljs/konserve/indexeddb.cljs
-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQEcBAEBAgAGBQJU65n6AAoJEKel+aujRZMkwaAH/1Eq2/vr84WqNq6u2rcG4ly+
xkByy602oMy9/ScK1/ytv/kFdPG+qAYmtQ4EHq/LGUma3zLjpPdrf49A1RUGfVii
fbwIUiRgUJWdigrGiSf/2zIn972MgIfc4qKFj/+/2hJNZIAwlAKiPQSvG2VPrHo7
dskeQZeCZI4yGSz4+dQFGmKvjiAum+UConkzn0T/N2XGknerFsLonXyvpbTv/SCC
NwNoRJzI39kjIyhfUuwbwHoy0/DtbxjvkbrLHeyGIKGOJ75QutWY6gvkRsSlzS5g
A2jVtZo/hw02di3mxrD1vwSZ2PWads2juKjCmmOQrIPofSBWyVmoGeQ/qyD/Dgw=
=Wi7E
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: ANN: boltzmann 0.1.1 - a deep-learning library

[Christian Weilbach]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hey,

On 06.01.2015 05:04, Mike Anderson wrote:
> On Tuesday, 6 January 2015 04:27:55 UTC+8, Christian Weilbach
> wrote:
>> 
> On 05.01.2015 03:34, Mike Anderson wrote:
>>>> Very cool stuff!
> 
> Like yours! I wish nurokit was EPLed, then I could have had a look
> at it and try to include it there. Have libraries like this high 
> commercial value? I thought the knowledge to apply them and tune
> them to the problem is still more expensive, this is why I picked
> EPL. Also GPL and EPL don't seem to be compatible due to my
> recherche (which is a pity, because I like the GPL).
> 
> 
>> I think there isn't much commercial value in the library itself -
>> there are many free libraries for machine learning that work just
>> fine. Nobody with enough skill to use the library is going to pay
>> you for something they can get for free.
> 
>> The commercial value is all around : - Building a solution that
>> solves a business problem *using* the library - Integrating with
>> other applications/services (Clojure shines here because of the
>> JVM ecosystem) - Professional services / consulting

Ok, thx. I'd like to move into this direction.

> 
> 
> 
> 
>>>> 
>>>> I notice that you are specialising the RBM to a specific
>>>> matrix implementation (Clatrix / JBlas) in the file
>>>> "jblas.clj". Are you sure you need to do that? Part of the
>>>> beauty of core.matrix is that you should be able to write
>>>> your algorithms in an implementation-independent manner and
>>>> still get the performance benefits of the optimised
>>>> implementation when you need it.
> 
> I started with core.matrix operations and clatrix and then tried to
>  eliminate all overhead showing up in the VisualVM sampling
> profiler. In my experiments the protocol overhead in this inner
> loop in `cond-prob-batch` was something like 10% or so, but I am
> not sure whether I did something wrong. In the mean time I have
> benchmarked my cryptographic hash function, which also uses
> protocols, and sometimes I have seen protocol overhead and
> sometimes not, maybe it was related to tiered compilation and the
> JIT sometimes not optimizing it, but this is only guessing.
> 
> 
>> 10% protocol overhead sounds like you must be doing quite a lot
>> of protocol calls.
> 
>> The usual trick to minimise this is to ensure that a single
>> protocol call does a lot of work (i.e. work on whole arrays at a
>> time rather than individual elements). If you do that, then the
>> protocol overhead should be negligible.

I only do a matrix-multiplication and element-wise calculation of the
sigmoid activation:
https://github.com/ghubber/boltzmann/blob/master/src/boltzmann/jblas.clj#L40
I have not done any inlining without a profiler proving significant
performance benefits, but I can recheck at some point.

> 
> 
> 
> If you replace all the jBlas method calls with core.matrix fns in 
> `cond-prob-batch` (3), which is quick to do, do you see a
> performance difference?
> 
> I really like core.matrix, or in general sound, light protocols and
>  then implementations. Yesterday I found an improved fork for
> clj-hdf5 for instance, which implements some of core.matrix
> protocols and fixed that to read double matrices for me,
> potentially this even allows to read tensors bigger than memory
> partially then. (1) So I didn't want to inline jBlas, but really
> use core.matrix. This internal inlining seemed to be some
> compromise, since it still allows to use clatrix when dealing with
> the jblas implementation (otherwise it was just a mini-batch
> implementation).
> 
> For deep learning most interesting was GPU support in core.matrix
> for typical BLAS routines, e.g. with jCuBLAS or clBLAS, but I just
>  couldn't start work on this yet. You then have to be very careful
> not to access some memory, but if this could work with core.matrix
>  protocols it was a major win.
> 
> 
>> It should certainly be possible to wrap GPU matrix support in a
>> core.matrix implementation, indeed I think there have been a
>> couple of "proof of concept" attempts already.
> 
>> I personally have in the back of my mind a GPU-accelerated
>> extension to Vectorz (i.e. GPU-backed subclasses of AMatrix and
>> AVector), using something like jCuBLAS. Then the full core.matrix
>> support would come for free via vectorz-clj. Would possibly be
>> the easiest way to get comprehensive GPU array programming
>> support in Clojure.

Cool. Maybe we could also just wrap the NDarray library of
deeplearning4j, the

ANN: konserve 0.2.3

[Christian Weilbach]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hello!

I'd like to announce a new version of konserve,

https://github.com/ghubber/konserve

The filestore can be considered beta now, I have put some effort into
flushing all streams and synchronising file-descriptors as is expected
for atomicity and durability on system crash. CouchDB is now
implemnted in a separate project.

https://github.com/ghubber/konserve-couch

Happy storing!
Christian
-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQEcBAEBAgAGBQJUtwO6AAoJEKel+aujRZMkKggIAI/bS+PqZCJ+IiuVchJkYpaZ
VGhnFuyr9j1lNStL635iep3+DxydEJd1JMkKtFNLxe3CiVK8TMSDun5ugsY81hgl
jtx6B5o3+SK2C1Nxft/Mg+NtHofoyDBFNCzoKE8PdpD6c/uKvtWueis9KudUbXnu
s45bdXfOQRybzr0qkepYfnndmyJ/0ZOFbLdKA2wpRy/HBWOpwx4HT3mmeH8zBIXM
xkdsmv9F57qZtGGzq7rk5yRVpJNUsVaumG/039Un563fcF1Ut11hWoEzKsaUnIcn
NmWz/TcPD0ZcxAltTFRsdsCYqTTUF6dOH0PgVzjaIEg8idkzJ8prq+1MPcGEb9M=
=8fBf
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: ANN: boltzmann 0.1.1 - a deep-learning library

[Christian Weilbach]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 05.01.2015 03:34, Mike Anderson wrote:
> Very cool stuff!

Like yours! I wish nurokit was EPLed, then I could have had a look at
it and try to include it there. Have libraries like this high
commercial value? I thought the knowledge to apply them and tune them
to the problem is still more expensive, this is why I picked EPL. Also
GPL and EPL don't seem to be compatible due to my recherche (which is
a pity, because I like the GPL).

> 
> I notice that you are specialising the RBM to a specific matrix 
> implementation (Clatrix / JBlas) in the file "jblas.clj". Are you 
> sure you need to do that? Part of the beauty of core.matrix is
> that you should be able to write your algorithms in an 
> implementation-independent manner and still get the performance 
> benefits of the optimised implementation when you need it.

I started with core.matrix operations and clatrix and then tried to
eliminate all overhead showing up in the VisualVM sampling profiler.
In my experiments the protocol overhead in this inner loop in
`cond-prob-batch` was something like 10% or so, but I am not sure
whether I did something wrong. In the mean time I have benchmarked my
cryptographic hash function, which also uses protocols, and sometimes
I have seen protocol overhead and sometimes not, maybe it was related
to tiered compilation and the JIT sometimes not optimizing it, but
this is only guessing.

If you replace all the jBlas method calls with core.matrix fns in
`cond-prob-batch` (3), which is quick to do, do you see a performance
difference?

I really like core.matrix, or in general sound, light protocols and
then implementations. Yesterday I found an improved fork for clj-hdf5
for instance, which implements some of core.matrix protocols and fixed
that to read double matrices for me, potentially this even allows to
read tensors bigger than memory partially then. (1) So I didn't want
to inline jBlas, but really use core.matrix. This internal inlining
seemed to be some compromise, since it still allows to use clatrix
when dealing with the jblas implementation (otherwise it was just a
mini-batch implementation).

For deep learning most interesting was GPU support in core.matrix for
typical BLAS routines, e.g. with jCuBLAS or clBLAS, but I just
couldn't start work on this yet. You then have to be very careful not
to access some memory, but if this could work with core.matrix
protocols it was a major win.
boltzmann's CPU version is for me 1/3 to 1/4th training speed of
theano (which again is 1/5 of its GPU version on my older gaming
laptop). Theano uses a symbolic compute graph modelled after Python's
numpy API and then emits that either to CPU or GPU (including some
numeric optimizations). I guess my jBlas backend is still slower than
theirs netlib-java (2) recommends building a custom version of
ATLAS (for Ubuntu here), have you experience with this? I probably
should do this for clatrix (and also for numpy).


> 
> For example, the core.matrix protocols (mmul, add!, add, 
> inner-product, transpose etc.) should all call the right Clatrix 
> implementation without any noticeable loss of performance (if they 
> don't that's an implementation issue in Clatrix... would be good
> to unearth these!).

Indeed! I also missed outer-product, which I have implemented for
jBLAS, as this at some point was taking most of the time, seemingly
falling back on a default implementation of core.matrix including
conversion to default types.

> 
> If the core.matrix API is insufficient to implement what you need, 
> then I'd love to get issues / PRs (either for core.matrix or 
> Clatrix).

Ok. Maybe you can verify that you don't see a significant performance
difference between the clatrix and the jblas version of
cond-prob-batch so I can remove the inlining and the rest should be
able to be patched into clatrix.

Christian

(1) https://github.com/ghubber/clj-hdf5/tree/develop
(2) https://github.com/fommil/netlib-java/
(3)
https://github.com/ghubber/boltzmann/blob/master/src/boltzmann/jblas.clj#L33


-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQEcBAEBAgAGBQJUqvPJAAoJEKel+aujRZMkNyEIAKTUqhsZOWI+17Fk9eZCkvLj
0geoshCHdX0K1A6ZmIGblFuRZ+DuJ6fiP/cO95IxRDfkXnK+cm/FIAJAXxz+U5PB
4+cl6x9x86C8VLL7MwrTR0woiP8sSHmnrbGpeefoj5KFBD03GQ0g0P/5ONFIeYPc
4MNOvFIja8EiHmFph2rOgBXvM3WWtbaibSeRbYkAVyq7jZ7D8sHcmM43Ycg+S0kM
Gfweuc3dzWAShxK8WKOazBiu7T4IPwHHIMZgNiPYNK5jFV6C1NIUrUpyU+fkWbTB
Tz7gm4l8i0zpX/M7yfa2l8r6Hgq6B0wtGXivSeurXyJnLDyHvWKvbzUJzvMACec=
=3TgC
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroup

ANN: boltzmann 0.1.1 - a deep-learning library

[Christian Weilbach]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi all,

- From the README:

This library is supposed to implement Boltzmann Machines, Autoencoders
and related deep learning technologies. All implementations should
both have a clean high-level mathematical implementation of their
algorithms (with core.matrix) and if possible, an optimized and
benchmarked version of the core routines for production use. This is
to facilitate learning for new users or potential contributors, to be
able to implement algorithms from papers/other languages and then tune
them for performance if needed.

This repository is supposed to cover techniques building on Restricted
Boltzmann Machines, like Deep Belief Networks, Deep Boltzmann Machines
or temporal extensions thereof as well as Autoencoders (which I am not
familiar enough with yet). Classical back-propagation is also often
used to fine-tune deep models supervisedly, so networks should support
it as well.



I haven't build myself deep belief networks out of it yet, but this
should be fairly straightforward. Also combination with the usual
linear classifiers (logistic regression, SVM) at the top layer can be
explored. If somebody has interest/experience in/with implementing
standard backpropagation, go ahead and open a pull-request :-).

Christian
-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQEcBAEBAgAGBQJUqceeAAoJEKel+aujRZMkJHoIAKkAbgZjvs9pzmJjzJf5Y1sg
EQCwf7W6Vrz0rvDtrkSiRNO+rmSEL4TpWPPlHLTYWs781Wrz9FRmkmHzR0mZ8izT
kWsQ3rP4TjDUDiB8S34CQxA15YLRfbvIxVv2JBfkGBWo64NHSrNUxz+Dfvu2jzbi
at614o/T5lZQ6qzkyputYwzOocX58AcnCtfXDVO2UJt8RU/q33FVugjtXtvsDxgM
AOO4WnW6mzYvLUbrhksDjuLShhs2EoCMB54cB2W5ejz+6X3oFeF/xndFqtNYdwPF
d13q60Ex0s/IqIo3mOwB/O1rOnsBHxiQ6nuSaphMAm7jJF9wHtDaXHWRZHa2RTg=
=BjnJ
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] konserve 0.2.0: A kv-store protocol with clj and cljs implementations

[Christian Weilbach]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi all,

from the README:

A key-value store protocol implemented with core.async to allow
Clojuresque collection operations on associative key-value stores,
both from Clojure and ClojureScript for different backends. Data is
generally serialized with edn semantics or if supported as native
binary blobs and can be accessed similar to clojure.core functions
get-in,assoc-in and update-in. Each operation is run atomically and
must be consistent (in fact ACID), but further consistency is not
supported (Riak, CouchDB and many scalable solutions don't have
transactions over keys for that reason). This is meant to be a
building block for more sophisticated storage solutions (Datomic also
builds on kv-stores), it is not necessarily fast depending on the
usage pattern. The general idea is to write most values once (e.g. in
form of index fragments) and only update one place once all data is
written, similar to Clojure's persistent datastructures.

Supported Backends

The protocol is implemented at the moment for CouchDB with clutch and
an experimental file-system store in Clojure and for IndexedDB in
ClojureScript. The file-system store currently uses fressian and is
quite efficient. For CouchDB and IndexedDB there is no internal
JSON-representation of the underlying store like transit, yet, hence
they are fairly slow. New storage backends, e.g. Riak, MongoDB, Redis,
JDBC, WebSQL, Local-Storage are welcome. This was initially
implemented as a storage protocol for geschichte.

https://github.com/ghubber/konserve

Feel free to give feedback,
Christian
-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQEcBAEBAgAGBQJUnhheAAoJEKel+aujRZMkw9AIAMZo5z+qsyX7ZjOK4218VpRF
sEr4QFnDHwE8/ekYAfgWR+IirB+LkoLhxGcyNjUyFZIbkIupm6SfItAZmIHTe5ar
Uzqz3IAXiwNhi/tgACRsywceduNzfoOrV4bpDvde7GtwjtjGplR002F7PZLtB1Dj
Fqc9Uh/LU6ainZw4SA3Z1g8QXYezdsxVM9rUSo7Nh6RDDriKEnYKfaFLjCNmKsIM
VHA/+0PJPCM7Mh7do5R0o5fnzjoOlNfVza359JWGd0PtvgegkG0Z3A1PVNdYIHwn
VTkypZP3nFuw0XKGpayD/x5eImTNIzhIlVbylycO07PpZKDcYuMtuwH2W+bdWHs=
=1vRb
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Safe and fast clojure + cljs crypto-hashing - input needed

[Christian Weilbach]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hallo,

tldr;
I would like to have some input, because the crypto-hash function for
edn is a basic building block to distribute values and cannot easily
be changed later.


I have developed a cross-platform implementation to create UUID5
values based on hashes to safely store values in untrusted distributed
environments (1) (for a p2p synchronisation system we are working on
(2)).
I am pretty sure that I don't introduce collisions, but please prove
me wrong. The conceptually critical part is to use a commutative
operation for commutative datastructures like maps and sets. I first
hash kv entries and then xor them, so you cannot e.g. easily make two
kv entries collide before entering the crypto-hash function.

Now the problem is that this makes hashing maps more expansive (3). My
goal was to hash a map of 1 million entries (blob size a few
megabytes) in roughly a second or better so it takes as long as
transmission over mobile or slow internet between untrusted peers
(datascript javascript mobile clients and datomic database on the
server for instance) and keeps latency in synchronisation low. I want
to push the speed as far as I can (without sacrificing safety),
because this determines the boundaries of distributed systems that can
be built with it. Inside trusted networks hashing can always be
disabled, treating these as random UUIDs, but this weakens the overall
system. An alternative is to avoid commutative datastructures
completely, which can give up to native byte-array speed but drops
clojure value semantics.

I had a look at cheshire (4) and transit-java (5) and I might be able
to squeeze a factor 2x out by using JVM dispatch on types instead of
protocols. But as this is already a fairly low-level and limited
optimization (sequential hashing is fairly fast and the profiler only
sometimes shows protocol overhead) I might miss some good ideas of how
to improve the commutative crunching, I wanted to ask here first.
Benchmarking has also proven quite adventurous, with significant
performance changes from different jvm runs, even with criterium it
just was half as fast as a few hours ago (probably OS/cpu throtteling
(?...)).

If you have further requirements or ideas, it would make sense to
raise them now. I am also not aware of other solutions, so please
point them out if you know some.


Thanks,
Christian

(1)
https://github.com/ghubber/hasch/blob/master/src/clj/hasch/platform.clj (2
bits of precious entropy are reserved for internal revisions)
(2) https://github.com/ghubber/geschichte
(3) https://github.com/ghubber/hasch#speed
(4) https://github.com/dakrone/cheshire
(5) https://github.com/cognitect/transit-java
-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQEcBAEBAgAGBQJUlyujAAoJEKel+aujRZMkpgYH+QFmNvrAOiu65CMDVNynZTGB
B9Pb7N62gCA3sYZZq6R0taY6ZraVKozwiJ27WOO7u6c0XJeYAMFHSTUvBepbKqkj
qYpgeVkkNMB9MyCLDvhBxaT4aId35CfXJBWYu0pK1jsq8Kwlfwwukq4ThOndUMB/
wp226e7i7YogLB8tetPQ2wHC9wfw1ITFSQtIf9avt1tAlxroeg23scV8NPG0LRSO
p33kFrGWYnZcl27kS2AsdjC5akkS3QUDm3FBiaLVF4swCkir5jOMGOm8zhUincUL
NPKpbBJ2DT6RU2nTgJESWRxK9Sph8+hvI+3Vu9S6VVgmMDGq4uixMqVhtfYipys=
=6W+f
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] bote 0.1.0 - smtp server as a library

[Christian Weilbach]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

https://github.com/ghubber/bote

I have wrapped subethasmtp to allow to run a smtp server easily and
just catch e-mails as values as they are just messages instead of the
horrible configuration nightmares unix mail servers build around mail
setups. I tried postfix, exim, had a look at qmail years ago and got
scared by sendmail. The postfix manual has cost me countless hours and
lately I failed to setup authentication which seemed ridiculous to me
as it will probably be easier to implement it now (TODO).

I will try to clone my mail on my postfix server and store them in
datomic next, as also most mail clients roll fairly resource intensive
indexing operations just to allow to search mail locally (often in
limited ways). This again requires some lecture of postfix manuals...

To really replace a traditional mail setup the most important peace
missing is spam filtering imo. The rest can be written fairly easily
(including procmail style filtering, forwarding, storage to mail
directories) and even nicely distributed with core.async. Another
thing missing which I personally would replace with a web client is
imap (in general in java land I couldn't find a usable library...).

Feedback is appreciated.

Christian
-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQEcBAEBAgAGBQJUi1SZAAoJEKel+aujRZMkvPQH/0emNCLiDldwyJqVj0ShxRv4
watgoJ4a6rVkaBpbFkMV9JmPJFuWMpzlK+yHNcUjxFLy2Q6DiiQq0y6of7k9kY5X
qnD+EPWKHM4FaR//xw7LPtn+IJ4mH/nPjts8Zc6no2X9/KZj76GwUeD1zQPPUQ8F
QF5bOwcuOYvC+VocHNSo+b8Hi7BbZh/kyme0f/xtr4vKp2G/L+O9EmGTptSEtwSw
+cfaDzvd9OVAkjz0b02Up6Pnghub4WP7sojo8z7SKV/GqXLhV/qorSv6TjW/BbdF
ouU9dxJRrukZPQqf51V12Dd0pEWSvOTzN230P9NqJrtph2M6VSVu0c01qeG/Chc=
=kgFy
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Dependency problem between clj-http and com.cemerick/friend

[Christian Egli]

Hi Plínio

Plínio Balduino  writes:

> I just forked friend, updated the library versions and opened a pull
> request.
>
> If you want to try and see if the problem still happens, please check
> https://github.com/pbalduino/friend

AFAIK the problem I have with friends dependencies is not the fact that
it depends on an old version of clj-http but that it depends on an old
version of org.apache.httpcomponents/httpclient. So I think your commit
(https://github.com/pbalduino/friend/commit/926f01ba718c355c82faad4aa0438162708a1efb)
will not help with that.

Thanks
Christian

-- 
Christian Egli
Swiss Library for the Blind, Visually Impaired and Print Disabled
Grubenstrasse 12, CH-8045 Zürich, Switzerland

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Dependency problem between clj-http and com.cemerick/friend

[Christian Egli]

Andy Fingerhut  writes:

> Try running 'lein deps :tree >& deps.txt' in each of those projects,
> and diff them. I can't explain why the differences are there that
> exist, but there are significant differences, including in version
> numbers of some of the dependencies brought in.
>
> Asking on the Leiningen email list or #clojure IRC channel may be more
> fruitful.

Thanks for you answer. Further research indicates that the order of the
dependencies does matter (at least for transitive dependencies). Both
friend and clj-httpclient depend on org.apache.httpcomponents/httpclient,
so if friend comes first in the dependency vector lein will fetch
version 4.2.1. However if clj-httpclient comes first lein will fetch
version 4.3.5.

I guess there are two ways out of this:

1. change the order so that clj-httpclient comes first
2. file a bug report with friend

For my project I changed the order and I just filed a bug with friend
(https://github.com/cemerick/friend/issues/128). In fact there is even
already a pull request that adresses this issue.

Thanks
Christian

-- 
Christian Egli
Swiss Library for the Blind, Visually Impaired and Print Disabled
Grubenstrasse 12, CH-8045 Zürich, Switzerland

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Dependency problem between clj-http and com.cemerick/friend

[Christian Egli]

Hi all

I have a strange interaction between clj-http and com.cemerick/friend. I
don't know if this is a problem in either of the two packages or maybe
even Leiningen.

The problem is very easy to reproduce. Create a project with `lein new`,
add dependencies to clj-http and com.cemerick/friend, start a repl and
simpy require clj-http.client as follows:

$ lein new app friend-and-clj-http
$ cd friend-and-clj-http
$ # add clj-http and friend as a dependency to project.clj
$ lein repl
friend-and-clj-http.core=> (require '[clj-http.client :as client])

If the dependency to friend is before the dependency to clj-http you
will get the following exception:

CompilerException java.lang.ClassNotFoundException: 
org.apache.http.conn.ssl.SSLContexts, compiling:(clj_http/conn_mgr.clj:1:1)

However if you change the order of the dependencies everything works as
expected.

$ # change the order of the dependencies
$ lein repl
friend-and-clj-http.core=> (require '[clj-http.client :as client])
nil
friend-and-clj-http.core=> Bye for now!

I put both versions of project.clj, the shell session and the output of
`lein deps :tree` for both versions in public gists

- https://gist.github.com/egli/8e6086f4d35ff7c11f80 project.clj with
  friend and clj-http as dependencies
- https://gist.github.com/egli/ff26669047b9273741c6 project.clj with
  clj-http and friend as dependencies 
- https://gist.github.com/egli/076b4f450cc237e2ac4a shell session
- https://gist.github.com/egli/b3e149aded64c7628f8e lein deps :tree with
  friend before clj-http
- https://gist.github.com/egli/4fa13bc791e52061f9e9 lein deps :tree with
  clj-http before friend

Where do I report this problem?

Thanks
Christian

-- 
Christian Egli
Swiss Library for the Blind, Visually Impaired and Print Disabled
Grubenstrasse 12, CH-8045 Zürich, Switzerland

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Patch] Support for DOCTYPE when emitting XML (DXML-10)

[Christian Egli]

Hi 

Currently data.xml doesn't currently support the specification of
doctypes when emitting XML (see
http://dev.clojure.org/jira/browse/DXML-10). I created a patch to
support this. Now it appears that data.xml is in the midst of a big
refactoring so I'm wondering if and how I should attach the patch to the
jira bug.

The patch basically just adds another option to emit, emit-str, indent
and indent-str which allows to specify the doctype declaration. It then
simply delegates to the underlying XMLStreamWriter by invoking .writeDTD
if there is a doctype. 

The patch can be found at
https://github.com/egli/data.xml/compare/clojure:master...master?diff=unified.

Signing the CA will take a bit of time, so I thought I could get the
conversation already started while the legal process is under way.

Thanks
Christian

-- 
Christian Egli
Swiss Library for the Blind, Visually Impaired and Print Disabled
Grubenstrasse 12, CH-8045 Zürich, Switzerland

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] Immutant 2.0.0-alpha2

[Christian Romney]

One small nit with the documentation (web guide): like hello and howdy, hola 
begins with an "h", albeit a silent one.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] Prone: Exception/debugging middleware

[Christian Johansen]

Hi,

Prone (http://clojars.org/prone - http://github.com/magnars/prone) is a new 
middleware for Ring apps that replaces the default exception page with an 
interactive page that presents the exception, stack trace and environment 
data in a more easily understandable way. It can also debug arbitrary data, 
and has a nice little client-side data browser. It's intention is to make 
it a lot easier to find the cause of errors when working on Clojure web 
applications.

Here's a short video demoing it's use: 
https://dl.dropboxusercontent.com/u/3378230/prone-demo.mp4

Hope you find it useful!

Christian

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

handling only a part of a hash-map

[Christian]

Hi,

I'm just beginning with clojure.

;(let [conn (mg/connect)
; db   (mg/get-db conn "throughput")
; coll "clojure_throughput_src"]
;;; with a query, as Clojure maps
;  ( doseq [item (mc/find-maps db coll {})]
;(println
;  (get_timeslot
;(get_dt
;  ( get item :ts))

In a mongodb collection I have a timestamp and an id. I'll transform the 
timestamp which works fine.
But I'm struggle with getting  the id  , too   - let  it untransformed and 
going on with this hash-map result.

Thanks in advance
Christian




-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Integration with Mutable Object-Oriented Eventing Hell

[Christian Eitner]

Hi Luca and Phil,

Thanks for your insights.

Especially your example, Phil, is very revealing.

I think I now have a direction for further thought. And of course I did not 
expect there to be a 'magic silver bullet' answer to the question.

Cheers,

Christian

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Integration with Mutable Object-Oriented Eventing Hell

[Christian Eitner]

Hi Luca,

On Thursday, April 3, 2014 11:57:27 AM UTC+2, icamts wrote:
>
> Hi Christian,
> I think you are looking for this.
>
> http://en.wikipedia.org/wiki/Facade_pattern
>
> In clojure you can use a def for each private member of the facade. 
> Alternatively you can write a function to instantiate and return private 
> members. Use a defn for each facade's methods.
>
>
I'm not sure in how the facade pattern would help me on the conceptual 
level.

I'm pretty confident that I can work out the technical details of how to 
get my fingers at the existing objects or how to interface from the OO 
world into Clojure code.

The question is more like, how should I model the integration?

It seems pretty useless and/or dangerous to me to put the mutable objects 
into Clojure's immutable data structures and work on the them directly. 
Would I kind of extract the objects' data (value-like) into my data 
structures, perform operations an these and in the end retransform those 
values onto the objects?

That's more like the patterns I am searching for.

Thanks,

Christian

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Integration with Mutable Object-Oriented Eventing Hell

[Christian Eitner]

Hello everybody,

Given an enormous network of inter-referenced, mutable objects which have 
to change in-place driven by events (= the OO system).

Which strategy would you recommend to plug into such a system with Clojure, 
building an island of immutable functional programming saneness? How to 
build this famous 'stateful bridge' that I have read about here and there, 
but have not found concrete examples for?

Or would you really dissuade me from trying anything like that, and treat 
Clojure's 'hostedness' as simply a way to use the host's array of libraries?

Any hints or references to further literature would be welcome.

Thanks for your consideration,

Christian

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] Clojure 1.6.0-RC3

[Christian Johansen]

Thanks for all your hard work on this!

One thought: I like the idea behind some? and friends, but the naming seems
misleading to me. In most APIs that have an every?-like function, there is a
corresponding some? or any?. In Clojure this is not the case, but we do have
every? and some, which provides the same capabilities. Adding some? into 
this
mix as a completely different kind of function seems to create unnecessary
confusion.

I'm probably too late to the party on this one, and it may even have been
discussed before, if so I apologize for beating a dead horse.

Christian

kl. 22:45:59 UTC+1 søndag 23. mars 2014 skrev Alex Miller følgende:
>
> Clojure 1.6.0-RC3 is now available. 
>  
> Try it via 
> - Download: http://central.maven.org/maven2/org/clojure/clojure/1.6.0-RC3
> - Leiningen: [org.clojure/clojure "1.6.0-RC3"]
>
> See the full change log here:
> https://github.com/clojure/clojure/blob/master/changes.md
>
> Clojure 1.6.0-RC3 has the following changes since 1.6.0-RC2:
>
> - Test temporarily disabled to allow Clojure to build with JDK 1.8 
> (requires ASM upgrade in future release)
> -  Symbol no longer caches hashCode and will lazily cache hasheq. The 
> hasheq is also computed in a different way to leverage the cached String 
> hashCode of the Symbol ns (if it exists).
>
> Please let us know how it looks!
>  

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] Stasis - not another static site framework

[Christian Johansen]

I just posted a quite long and detailed post/tutorial on using Stasis for 
static web sites, including integration testing of all pages, link checker 
++

The post also shows off using Optimus for asset optimization, and Enlive 
and Hiccup for markup generation/manipulation.

http://cjohansen.no/building-static-sites-in-clojure-with-stasis

Christian

kl. 11:16:48 UTC+1 torsdag 23. januar 2014 skrev Magnar Sveen følgende:
>
> Stasis
>
> A Clojure library of tools for developing static web sites.
>
> <https://gist.github.com/magnars/32dbca91bdb0987ea4ba#another-static-site-framework-why>Another
>  
> static site framework? Why?
>
> Well, that's exactly it. I didn't want to use a framework. I don't like 
> the restrained feeling I get when using them. I prefer coding things over 
> messing around with configuration files.
>
> I want to
>
>- code my own pages
>- set up my own configuration
>- choose my own templating library
>- create my own damn stylesheets
>
> *Statis offers a few functions that are pretty useful when creating static 
> web sites.*
>
> No more. There are no batteries included.
>
> If you want a framework that makes it really quick and easy to create a 
> blog, you should take a look at these:
>
>- misaki <https://github.com/liquidz/misaki> is a Jekyll inspired 
>static site generator in Clojure.
>- Madness <http://algernon.github.io/madness/> is a static site 
>generator, based on Enlive and Bootstrap.
>- Static <http://nakkaya.com/static.html> is a simple static site 
>generator written in Clojure.
>- Ecstatic <http://samrat.me/ecstatic/> creates static web pages and 
>blog posts from Hiccup templates and Markdown.
>- incise <https://github.com/RyanMcG/incise> is an extensible static 
>site generator written in Clojure.
>
> They generally come with a folder where you put your blog posts in some 
> templating language, and a set of configuration options about how to set up 
> your blog. They often generate code for you to tweak.
> <https://gist.github.com/magnars/32dbca91bdb0987ea4ba#usage>Usage
>
> The core of Stasis is two functions: serve-pages and export-pages. Both 
> take a map from path to contents:
>
> (def pages {"/index.html" "Welcome!"})
>
> The basic use case is to serve these live on a local server while 
> developing - and then exporting them as static pages to deploy on some 
> server.
>
> <https://gist.github.com/magnars/32dbca91bdb0987ea4ba#serving-live-pages-locally>Serving
>  
> live pages locally
>
> Stasis can create a Ring handler to serve your pages.
>
> (ns example
>   (:require [stasis.core :as stasis]))
> (def app (stasis/serve-pages pages))
>
> Like with any Ring app, you point to your app in project.clj:
>
> :ring {:handler example/app}
>
> and start it with lein ring server-headless.
> <https://gist.github.com/magnars/32dbca91bdb0987ea4ba#exporting-the-pages>Exporting
>  
> the pages
>
> To export, just give Stasis some pages and a target directory:
>
> (defn export []
>   (stasis/export-pages pages target-dir))
>
> When you've got this function, you can create an alias for leiningen:
>
> :aliases {"build-site" ["run" "-m" "example/export"]}
>
> and run lein build-site on the command line. No need for a lein plugin.
> <https://gist.github.com/magnars/32dbca91bdb0987ea4ba#example-apps>Example 
> apps?
>
> The static page that prompted me to write Stasis is currently closed 
> source, but I'm in the process of turning my 4 other static sites over. The 
> simplest, and first to be done, is:
>
>- 
>
>whattheemacsd.com (source) <https://github.com/magnars/what-the-emacsd>
>
>Uses Enlive <https://github.com/cgrand/enlive> for templating, and 
>Optimus <https://github.com/magnars/optimus> for frontend optimization.
>
> I'm also working on the Emacs Rocks! <http://emacsrocks.com/> webpage, 
> where I'll use hiccup instead of Enlive.
> <https://gist.github.com/magnars/32dbca91bdb0987ea4ba#is-it-stable>Is it 
> stable?
>
> It's still on a 0.x release, but I feel the API has jelled enough now to 
> open source it. I don't expect any major changes at this point. I'll likely 
> push it to 1.0 at the end of the month.
>
> <https://gist.github.com/magnars/32dbca91bdb0987ea4ba#again-why-not-use-one-of-the-existing-frameworks>Again,
>  
> why not use one of the existing frameworks?
>
> I think the existing frameworks are great if they fit your style. Stasis 
> 

Re: Good learning resources for Clojure novice but with a long background i programming, both OO and some Fp?

[christian jacobsen]

Great tips, thank you all! :)

On Friday, January 10, 2014 8:34:47 PM UTC+1, Colin Yates wrote:
>
> At the risk of self promotion*, have a read of 
> https://groups.google.com/d/msg/clojure/rt-l_X3gK-I/K80axT77XzwJ - it is 
> an excellent example of iterative compared to functional.  You can see at 
> least 4 distinct approaches to solving a fairly straight forward problem. 
>  It is a startlingly clear example of how elegant Clojure can become.
>
> * it isn't self promotion at all!  It is self deprecation - my solution is 
> clearly lame, hence the original post :).
>
> On Friday, 10 January 2014 12:52:53 UTC, christian jacobsen wrote:
>>
>> I have +10 years experience of OO programming (C++, C# and a little Java) 
>> and a couple of years of FP programming (mainly F#, some Scala and a little 
>> Haskell). 
>> Are there any resources for learning Clojure that are particular good for 
>> someone with the above background?
>>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Good learning resources for Clojure novice but with a long background i programming, both OO and some Fp?

[christian jacobsen]

I have +10 years experience of OO programming (C++, C# and a little Java) 
and a couple of years of FP programming (mainly F#, some Scala and a little 
Haskell). 
Are there any resources for learning Clojure that are particular good for 
someone with the above background?

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.