Re: Seeking a function to partially parallelize collection processing

[Sam Raker]

core.async pub/sub ?

On Friday, June 16, 2017 at 10:13:11 AM UTC-4, Tom Connors wrote:
>
> I'm looking for a function that would likely be named something like 
> "sequential-by" or "parallel-per" that takes some data-producing thing like 
> a lazy seq or a core async channel, a function to split records from that 
> input, and a function to handle each item. Each item with an identical 
> return value from the "split" function would be handled sequentially, while 
> the handling of the collection as a whole would be parallel.
>
> If we assume this signature:
> (parallel-per splitter handler inputs)
>
>
> Calling it like this:
> (parallel-per :id
>   (fn [m] (prn (update m :val inc)))
>   [{:id 1 :val 1}, {:id 1 :val 2}, {:id 3 :val 1}])
>
>
> Would result in the first two maps being handled sequentially, while the 
> third map is handled in parallel with the first two. The order of the 
> printed lines would probably be non-deterministic, except {:id 1 :val 2} 
> would be printed before {:id 1 :val 3}.
>
> Note that for my use case I don't care about return values, but if 
> something like this already exists it's plausible that it returns something 
> equivalent to (map handler inputs).
>
> Does anything like this already exist in core or some lib? If not, any 
> recommendations for how to build it?
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Filtering lazy sequence on itself - Eratosthenes Sieve

[Sam Raker]

Clojure doesn't do TCO with recursive functions--you have to use 
`loop`/`recur`[1]. This is one of the (thankfully few) "gotchas" in 
Clojure, at least/especially for people coming from other functional 
languages (or so I assume--coming from Python, I hadn't heard that much 
about TCO in the first place). There's a pretty good chance whichever intro 
book you're working through will talk about this in a chapter or so :)


[1] http://clojure.org/functional_programming#Functional 
Programming--Recursive Looping


On Thursday, November 5, 2015 at 10:01:44 AM UTC-5, Matthew Ulrich wrote:
>
> Thanks, Sergio - this is a good way to construct the sieve.
>
> It took me a little while to convince myself that it's the same, but it's 
> definitely a good solution.
>
> In general, I guess it's not possible to generate an infinite sequence 
> that utilizes the tail-call optimization it makes sense as there is no 
> "tail".
>
> Have a good day,
>
> Matty
>
> On Wednesday, November 4, 2015 at 10:07:46 AM UTC-5, Sergio Rupena wrote:
>>
>> Hi
>>
>> You can try the following
>>
>> (defn dividers   [primes n]
>>   (take-while #(<= (* % %) n) primes))
>>
>> (defn prime? [primes n]
>>   (every? #(pos? (rem n %))
>>   (dividers primes n)))
>>
>> (defn range-peek [coll]
>>   (iterate inc (-> coll peek inc)))
>>
>> (defn sieve
>>   ([] (cons 2 (lazy-seq (sieve [2]
>>   ([primes]
>>(let [p (->> primes
>> range-peek
>> (filter (partial prime? primes))
>> first)]
>>  (cons p (lazy-seq (sieve (conj primes p)))
>>
>> (last (take 1 (sieve)))
>>
>> This version keeps the visited primes in a vector so it will grow in 
>> memory but won’t overflow otherwise.
>>
>> Sergio
>>
>> On 04 Nov 2015, at 15:44, Matthew Ulrich  wrote:
>>
>> All - 
>>
>> I'm trying to generate a lazy sequence of primes using Erastosthenes 
>> Sieve (from Abelson & Sussman) and am encountering some issues with lazy 
>> sequence.
>>
>> Here's what I have:
>> ---
>>
>> (defn divisible?
>>   [input numerator]
>>   (= 0 (mod input numerator)))
>>
>> (defn sieve
>>   [stream]
>>   (lazy-seq
>> (cons
>>   (first stream)
>>   (sieve (filter #(not (divisible? % (first stream))) (rest 
>> stream))
>>
>> (def primes (sieve (iterate inc 2)))
>>
>> ---
>>
>> This is fine now when I:
>>
>> => (take 5 (drop 1000 primes))
>> (7927 7933 7937 7949 7951)
>>
>>
>> But when I:
>>
>> => (take 5 (drop 4000 primes))
>> StackOverflowErrorclojure.lang.LazySeq.sval  (LazySeq.java:40)
>>
>>
>> --
>>
>> I understand how the StackOverflow occurs with the filter on the 
>> recursively generated sequence... however I can't think of a way to make 
>> this work..
>>
>> I'm pretty new to clojure so I'm not very familiar with all of the 
>> language features; is there some tool to help me realize this or should I 
>> just go about the implementation in a different way?
>>
>> Thanks - have a great morning!
>>
>> Matty
>>
>> -- 
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clo...@googlegroups.com
>> Note that posts from new members are moderated - please be patient with 
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+u...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to clojure+u...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Hashing With Consistent Results

[Sam Raker]

Could you use something like Redis? Use hashes as keys, fake immutability 
by 'popping' kv pairs and inserting new ones keyed to the (presumably 
different) hash of the updated map.

On Thursday, August 13, 2015 at 7:52:06 AM UTC-4, Christian Weilbach wrote:

 -BEGIN PGP SIGNED MESSAGE- 
 Hash: SHA1 

 Hi Atamert, 

 sorry for replying late. 

 On 11.08.2015 10:29, Atamert Ölçgen wrote: 
  Hi Christian, 
  
  hasch looks nice, I might end up just using it. I will be hashing 
  smaller collections (maps where keys are keywords and values are 
  atomic data like integers). 

 Ok, then io will probably hurt you much more than some overhead for 
 hashing, I guess. 

  
  Collisions BTW are not such a big deal for my use case. I will have 
  a limited number of fragments (buckets, index pages, etc.) anyway. 
  65536 of them perhaps. The more I think about the problem the more 
  I realize I am implementing some sort of hash map. 

 I guess a durable one. In this case it might be interesting to think 
 about extending the persistent datastructures of Clojure in a way to 
 keep them on disk. I am currently experimenting a bit with that on 
 hash-maps of a commit graph, although I need it to work in 
 ClojureScript as well and cannot just reimplement core protocols 
 because of async io. 
 That way changing metadata of my datatype can have constant size. 

 Feel free to post any feedback on your progress/findings :). 

 Christian 
 -BEGIN PGP SIGNATURE- 
 Version: GnuPG v1 

 iQEcBAEBAgAGBQJVzITYAAoJEKel+aujRZMkZX4H/j5kpqolsS61y2IH+68Bq55/ 
 sdiME1eXdJ4VgYQH+IF4WDTYtPGZgV7U3XnM7Bqc5SygkGxOvDc5p4piTeSfpJIh 
 HE8GkBP8RkQNU5rqKu0M6xeSJGQdnAp/1VzQdJux9KAC2+0RG+SLKKft95zka9iz 
 PHDy+n/m8qTrMSUjpk2tVxuglyjkGaQeBm9bfRN07Cn/96e9XcafzsekMwZiI8HU 
 70n5ACbBWFXz5zxe0xfoUdA48OJSXrnoQTCmA95zOLnZ9thHgs066jjXCjNtomzD 
 NRhx7J9hi4lU54VmRcYJb4mVw5JLXQCWnARh8//o6P2SbmYFJkDIsCLtPJ9xJu4= 
 =H4lz 
 -END PGP SIGNATURE- 


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Decomplecting Clojure

[Sam Raker]

Default laziness can cause problems, but I'm not sure that they're bad 
problems, necessarily. I think 'opt-out' laziness (and the headaches it can 
cause) push people--especially people coming from non-FP backgrounds--to 
really zero in on writing pure, focused functions. 

On the other hand, from a 'marketing' perspective, I'd talk more about 
transducers than laziness--they're more exotic, at least.

On Thursday, August 13, 2015 at 2:27:21 PM UTC-4, Alan Thompson wrote:

 I must agree with Lee that, IMHO, default laziness can cause unexpected 
 problems. I would argue that it violates the Principle of Least Surprise.

 A better way would be to make laziness optional and explicit, perhaps by 
 adding a z suffix to the lazy version of each function (e.g. map - mapz, 
 for - forz, etc).  Then lazy behavior would be available whenever it was 
 desired, but the code would explicitly indicate when laziness was being 
 employed.

 Alan

 On Thu, Aug 13, 2015 at 11:01 AM, Lee Spector lspe...@hampshire.edu 
 javascript: wrote:



 Thanks Sebastian for the thoughtful and helpful document! I like it and 
 have shared it with my group.

  On Aug 13, 2015, at 1:51 PM, Sebastian Bensusan sbe...@gmail.com 
 javascript: wrote:
 
 
  I never thought of laziness! It's a good point. Retroactively I might 
 add it to the Functional Style section :)

 You did a nice job of presenting both pros and cons for the other 
 features of Clojure, and I expect there would be something similar for your 
 entry on laziness but FWIW from my perspective the cons here are more 
 significant than for any other feature (and I know that opinions will vary 
 on this!).

 I know that no language is perfect for everyone or every purpose, but at 
 least from my perspective pervasive laziness is one of Clojure's very few 
 true mistakes (sort of like my father-in-law says that eggplant is one of 
 god's very few true mistakes).

 Laziness is often glorious and elegant, and the way that it's everywhere 
 by default in Clojure is quite cool. But in a not-purely-functional 
 environment it can cause weird problems at unexpected times and in 
 unexpected places. There are several ways to get into trouble -- there was 
 a nice blog post on some of them years ago, maybe by Chas Emerick? -- but I 
 think most of my own come from the not-purely-functional nature of 
 synchronization, awaiting the completion of tasks dispatched to agents. 
 Symptoms that I get include the saturation of only one core when I expect 
 to be saturating all of them, and sometimes out-of-memory errors. I think 
 the former are due to delayed lazy computations taking place at unexpected 
 times, and that the latter come from GC not collecting things that it would 
 know it could collect if the deferred computations were completed in a 
 non-lazy way.

 It has been hard to replicate or track down the sources of these 
 problems, but they do always go away when I un-lazify everything -- 
 coercing sequences to vectors, calling doall, changing every call to map to 
 mapv, etc.

 Because these problems are so hard to diagnose and think about, I've 
 taken to pre-emptively avoiding laziness whenever possible, unless I really 
 want it (which is relatively rare, for me). This can be a bit awkward, and 
 it's one of the very few things that I feel like I have to apologize for 
 when I introduce Clojure to students.

  -Lee


 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com 
 javascript:
 Note that posts from new members are moderated - please be patient with 
 your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com javascript:
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups 
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to clojure+u...@googlegroups.com javascript:.
 For more options, visit https://groups.google.com/d/optout.




-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

side-effecting re-/transducers

[Sam Raker]

This seems bad, is this bad:

(defn to-db
  [conn]
  (fn
 ([val] (upload-to-my-db conn val))
 ([_ val] (upload-to-my-db conn val)))

(defn -main []
  (transduce my-preprocessing-xf (to-db (get-db-conn)) seq-of-things-to-
preprocess-and-upload))

I ask only because
1) Plugging the side-effecting part into the transducer pipeline seems 
cleaner and potentially more efficient than e.g. 
(doseq [v (sequence my-preprocessing-xf sea-of-things)] (upload-to-my-db 
conn v))
which is what I was doing
2) The addition of `run!`[1] in 1.7 seems to perhaps implicitly condone 
this kind of thing. There's very little out there about it, though, so I 
could very well be wrong.




[1] 
http://conj.io/store/v1/org.clojure/clojure/1.7.0-alpha4/clj/clojure.core/run%21

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: transducer (+ closures) efficiency question

[Sam Raker]

Oh fantastic! I was 100% wrong in literally the best way.


Thanks!

On Tue, Jun 23, 2015 at 7:17 PM, Ghadi Shayban gshay...@gmail.com wrote:

 Good question.

 Clojure's evaluation semantics dictate that the arguments are evaluated
 (computed) *before* calling the function. So(set coll) is computed before
 being passed to `partial`.  Partial receives a function (a value) and
 arguments (also values) and returns back a new function that saves those
 original arguments (which happen to be stuffed away in Java final fields).

 All three of your filters return a transducer.  All three of the inputs to
 `remove` are a partial function.  Each of the arguments to the call to
 partial is a set.  They are all essentially equivalent, and should perform
 the same.  (Except filter3 happens to create the set twice; once in the
 let, and once as the arg to partial).  So over a billion item collection,
 the set in your examples will only be computed once, once, and twice
 respectively.

 Note however that sets *are* functions that evaluate whether the argument
 is in the set. This means you could remove the call to partial and shorten
 to:

 (defn filter-contains1 [edn-file]
   (remove (set (read-edn-file edn-file


 Tangentially:
 (remove even?)
 Will be faster than
 (remove (fn [i] (even? i)))
 because in the first case the dereference of the var 'even?' happens only
 once and the value inside the var will be passed to `remove` at the
 outset.  In the second example the var dereference happens for every single
 item (though it's very cheap).  The second example is equivalent to writing 
 (remove
 #'even?)

 On Tuesday, June 23, 2015 at 6:07:06 PM UTC-4, Sam Raker wrote:

 Let's say that, as part of an xf, I want to filter out everything in a
 sequence that's also in some other sequence. Here are some ways of doing
 that:

 (defn filter-contains1 [edn-file] (remove (partial contains? (set (read-
 edn-file edn-file)

 (defn filter-contains2 [coll] (remove (partial contains? (set coll

 (def filter-contains3 [coll] (let [coll-as-set (set coll)] (remove (
 partial contains? (set coll)

 I have the strong suspicion that `filter-contains3` is the best of the 3,
 and `filter-contains1` the worst. The internal mechanics of transduce are a
 bit of a mystery to me, however: if `filter-contains2` were to be used on a
 collection of, say, a million items, would `coll` be cast to a set a
 million times, or is Clojure/the JVM smarter than that? I'm also wondering
 if anyone has any best practices (or whatever) they can share relating to
 this kind of intersection of transducers/xfs and closures. It seems to me,
 for example, that something like

 (defn my-thing [coll  stuff]
   (let [s (set coll)]
   ...
   (comp
 ...
(map foo)
(filter bar)
(remove (partial contains? s))
...

 is awkward, but that a lot of limited-use transducer factory functions
 (like the ones above) aren't exactly optimal, either.

  --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to a topic in the
 Google Groups Clojure group.
 To unsubscribe from this topic, visit
 https://groups.google.com/d/topic/clojure/H0zoF6jlY-c/unsubscribe.
 To unsubscribe from this group and all its topics, send an email to
 clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.




-- 
.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

transducer (+ closures) efficiency question

[Sam Raker]

Let's say that, as part of an xf, I want to filter out everything in a 
sequence that's also in some other sequence. Here are some ways of doing 
that:

(defn filter-contains1 [edn-file] (remove (partial contains? (set 
(read-edn-file 
edn-file)

(defn filter-contains2 [coll] (remove (partial contains? (set coll

(def filter-contains3 [coll] (let [coll-as-set (set coll)] (remove (partial 
contains? (set coll)

I have the strong suspicion that `filter-contains3` is the best of the 3, 
and `filter-contains1` the worst. The internal mechanics of transduce are a 
bit of a mystery to me, however: if `filter-contains2` were to be used on a 
collection of, say, a million items, would `coll` be cast to a set a 
million times, or is Clojure/the JVM smarter than that? I'm also wondering 
if anyone has any best practices (or whatever) they can share relating to 
this kind of intersection of transducers/xfs and closures. It seems to me, 
for example, that something like

(defn my-thing [coll  stuff]
  (let [s (set coll)]
  ...
  (comp
...
   (map foo)
   (filter bar)
   (remove (partial contains? s))
   ...

is awkward, but that a lot of limited-use transducer factory functions 
(like the ones above) aren't exactly optimal, either.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

neocons get record from cypher query

[Sam Raker]

I'm using neocons to put some data into a Neo4j database. I need to create 
relationships between nodes I've just created and nodes in the DB. I can't 
retrieve the nodes based on id without caching them locally, and they're 
not unique enough to retrieve using e.g. rest.nodes/find, so I need to use 
Cypher to get at them. I'm wondering if there's a better way to do this 
than what I currently am working with, namely 

(nn/get conn (-  (cy/tquery conn START person=node({sid}) MATCH 
person-[:foo]-o RETURN o {:sid 1}) first (get-in [o :metadata :id]

(this is just a toy example, so if the Cypher query is too simple, forgive 
me).

This seems...too complicated? Am I missing something? Is there a more 
idiomatic way to convert cypher results into neocons records?

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: neocons get record from cypher query

[Sam Raker]

Figured it out: `neocons.rest.records/instantiate-record-from` does the 
trick.

On Sunday, May 17, 2015 at 7:09:35 PM UTC-4, Sam Raker wrote:

 I'm using neocons to put some data into a Neo4j database. I need to create 
 relationships between nodes I've just created and nodes in the DB. I can't 
 retrieve the nodes based on id without caching them locally, and they're 
 not unique enough to retrieve using e.g. rest.nodes/find, so I need to use 
 Cypher to get at them. I'm wondering if there's a better way to do this 
 than what I currently am working with, namely 

 (nn/get conn (-  (cy/tquery conn START person=node({sid}) MATCH 
 person-[:foo]-o RETURN o {:sid 1}) first (get-in [o :metadata :id]

 (this is just a toy example, so if the Cypher query is too simple, forgive 
 me).

 This seems...too complicated? Am I missing something? Is there a more 
 idiomatic way to convert cypher results into neocons records?


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Protocols/multimethods vs core.match

[Sam Raker]

The discussion/post-linked-to 
in https://groups.google.com/d/msg/clojure/eoAp6QVimYI/iipmEJNKdrIJ have 
got me thinking about protocols  multimethods, which I admittedly have 
possibly never actually used, now that I think about it. I'm wondering how 
they differ from core.match[1]. I realize protocols, specifically, have a 
niche for when you have concrete type information about your data and want 
better performance. I'm a little less clear about multimethods--in 
particular, I just considered using multimethods for something, and then 
ended up reaching for core.match because it seemed more closure-friendly. 

What, if any, are the benefits of using protocols/multimethods over 
core.match, or vice versa? When would you reach for one and definitely not 
the other(s)?


[1] https://github.com/clojure/core.match

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: contains? on String

[Sam Raker]

I always assumed (contains? foo 2) worked because strings are arrays (i.e.  
vectors) of characters, on some level. 

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Streaming a big file

[Sam Raker]

I've got two really big CSV files that I need to compare. Stay tuned for 
the semi-inevitable how do I optimize over this M x N space? question, 
but for now I'm still trying to get the data into a reasonable format--I'm 
planning on converting each line into a map, with keys coming from either 
the first line of the file, or a separate list I was given. Non-lazy 
approaches run into memory limitations; lazy approaches run into  Stream 
closed exceptions while trying to coordinate `with-open` and `line-seq`. 
Given that memory is already tight, I'd like to avoid leaving open 
files/file descriptors/readers/whatever-the-term-in-clojure-is lying 
around. I've tried writing a macro, I've tried transducers, I've tried 
passing around the open reader along with the lazy seq, none successfully, 
albeit none necessarily particularly well. Any suggestions on streaming 
such big files?



Thanks!

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

code-rewriting with-connection macro (monger and maybe beyond?)

[Sam Raker]

I'm working with Monger[1], and have been thinking about connection 
encapsulation. I've currently got a `myproject.mongo` ns with private `db` 
and `coll` vars. I've got a public `insert` method along the lines of 

(defn insert [m] (mc/insert db coll m))

where `mc/insert` is, perhaps unsurprisingly, monger.collection/insert. To 
properly encapsulate my db-related vars, my options seem to be:

1) Replicate that `insert` func for every potentially-relevant mongo action 
(too much boilerplate/not DRY, I'd have to go back and add new funcs if I 
wanted more functionality)

2) Write a function along the lines of

(defn with-connection [f  args] (apply f (concat [db coll] args))

which is fine I GUESS but is only func-level

3) Write a macro that inserts `db` and `coll` in the proper places, so that 
one could do something like
(defn fake-upsert [m]
  (with-conn 
(if-let [existing (mc/find-one-as-map {:_id (:_id m)})]
  (mc/update existing m)
  (mc/insert m

and then `with-conn` would search through the code, identify the 
`monger.collection`-namespaced vars, and rewrite the code to put in `db` 
and `coll` arguments so it'd end up looking like

...
(if-let [existing (mc/find-one-as-map db coll {:_id (:_id m})]...
   (mc/update db coll existing m)..

and so on. This seems like my cleanest option, especially because it could 
be extended with an options map, à la

(defn mongo-to-another-db [m]
   (with-conn {monger.collection [monger-db monger-coll]
   another-db.adapter [another-db-uri another-db-username 
another-db-password]}
  (adapter/insert (mc/find-one-as-map {:foo bar}

I haven't been able to figure out how quite to do this, yet. I need, 
basically, a way to iterate through tokens in source code, examine each 
token, and then add a token and its relevant args if it's in the options 
map, or the token alone if it's not. 

Is this reasonable? Does this or something like it already exist? Is there 
something obvious my general macro mediocrity is preventing me from seeing?


Thanks!
-sam

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [OT?] Best DB/architecture for n-gram corpus?

[Sam Raker]

That's honestly closer to what I was originally envisioning--I've never 
really looked into graph dbs before, but I'll check out Neo4j tonight. Do 
you know whether you can model multiple edges between the same nodes? I'd 
love to be able to have POS-based wildcarding as a feature, so you could 
search for e.g. the ADJ goose, but that's a whole other layer of stuff, 
so it might go in the eventually, maybe pile.



On Tuesday, March 10, 2015 at 3:47:37 PM UTC-4, Ray Miller wrote:

 On 10 March 2015 at 17:58, Sam Raker sam@gmail.com javascript: 
 wrote: 
  I more meant deciding on a maximum size and storing them qua ngrams--it 
  seems limiting. On the other hand, after a certain size, they stop being 
  ngrams and start being something else--texts, possibly. 

 Exactly. When I first read your post, I almost suggested you model 
 this in a graph database like Neo4j or Titan. Each word would be a 
 node in the graph with an edge linking it to the next word in the 
 sentence. You could define an index on the words (so retrieving all 
 nodes for a given word would be fast), then follow edges to find and 
 count particular n-grams. This is more complicated than the relational 
 model I proposed, and will be a bit slower to query. But if you don't 
 want to put an upper-bound on the length of the n-gram when you index 
 the data, it might be the way to go. 

 Ray. 


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [OT?] Best DB/architecture for n-gram corpus?

[Sam Raker]

POS tagging is a solved-enough problem, at least in most domains. Clojure 
still doesn't have its own NLTK (HINT HINT, GSOC kids!), but I'm sure I can 
find a Java lib or 2 that should do the job well enough.

On Tuesday, March 10, 2015 at 4:27:17 PM UTC-4, Ray Miller wrote:

 On 10 March 2015 at 20:03, Sam Raker sam@gmail.com javascript: 
 wrote: 
  That's honestly closer to what I was originally envisioning--I've never 
  really looked into graph dbs before, but I'll check out Neo4j tonight. 
 Do 
  you know whether you can model multiple edges between the same nodes? 

 Yes, certainly possible. 

 If you go for Neo4j you have two options for Clojure: embedded (with 
 the borneo library and reading Javadoc, or plain Java interop) or 
 stand-alone server with REST API (with the well-documented Neocons 
 library from Clojurewerkz). You'll also have to think about how to 
 model which text (song) each phrase came from - likely another node 
 type in the graph with a linking edge to the start of the phrase. 

 Great book on Neo4j available for free download, also covers data 
 modelling: http://neo4j.com/books/ 

  I'd love to be able to have POS-based wildcarding as a feature, so you 
 could 
  search for e.g. the ADJ goose, but that's a whole other layer of 
 stuff, so 
  it might go in the eventually, maybe pile. 

 Sounds like fun, but means doing some natural language processing on 
 the input texts, which is a much more difficult problem than simply 
 tokenizing. 

 Ray. 


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [OT?] Best DB/architecture for n-gram corpus?

[Sam Raker]

I more meant deciding on a maximum size and storing them qua ngrams--it 
seems limiting. On the other hand, after a certain size, they stop being 
ngrams and start being something else--texts, possibly.

On Tuesday, March 10, 2015 at 1:29:44 PM UTC-4, John Wiseman wrote:

 By hard coding n-grams, do you mean using the simple string 
 representation, e.g. aunt rhodie as the key in your database?  If so, 
 then maybe it helps to think of it from the perspective that it's not 
 really just text, it's a string that encodes an n-gram just like 
 [\aunt\, \rhodie\] is another way to encode an n-gram--the 
 encoding/decoding uses clojure.string/join and clojure.string/split instead 
 of json/write and json/read, and escaping tokens that contain spaces is on 
 your TODO list at a low priority :)

 (And I think the Google n-gram corpus 
 https://catalog.ldc.upenn.edu/LDC2006T13 uses the same format.)


 John


 On Mon, Mar 9, 2015 at 7:09 PM, Sam Raker sam@gmail.com javascript:
  wrote:

 That's interesting. I've been really reluctant to hard code n-grams, 
 but it's probably the best way to go.

 On Monday, March 9, 2015 at 6:12:43 PM UTC-4, John Wiseman wrote:

 One thing you can do is index 1, 2, 3...n-grams and use a simple  fast 
 key-value store (like leveldb etc.)  e.g., you could have entries like

 aunt rhodie - song-9, song-44
 woman - song-12, song-65, song-96


 That's basically how I made the Metafilter N-gram Viewer 
 http://mefingram.appspot.com/, a clone of Google Books Ngram Viewer 
 https://books.google.com/ngrams.

 Another possibility is using Lucene.  Just be aware that Lucene calls 
 n-grams of characters (au, un, nt) n-grams but it calls n-grams of 
 words (that the, the old, old gray) shingles.  So you would end up 
 using (I think, I haven't done this) the ShingleFilter 
 https://lucene.apache.org/core/4_2_0/analyzers-common/org/apache/lucene/analysis/shingle/ShingleFilter.html
 .

 You might also find this article by Russ Cox interesting, where he 
 describes building and using an inverted trigram index: 
 http://swtch.com/~rsc/regexp/regexp4.html


 John





 Three things that you might find interesting:

 Russ Cox' explanation of doing indexing and retrieval with an inverted 
 trigram index: http://swtch.com/~rsc/regexp/regexp4.html


 On Sat, Mar 7, 2015 at 3:22 AM, Matching Socks phill...@gmail.com 
 wrote:

 A lot of guys would use Lucene.  Lucene calls n-grams of words 
 shingles. [1]

 As for architecture, here is a suggestion to use Lucene to find keys 
 to records in your real database. [2]

 [1] https://lucidworks.com/blog/whats-a-shingle-in-lucene-parlance/

 [2] https://groups.google.com/d/msg/datomic/8yrCYxcQq34/GIomGaarX5QJ


  -- 
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com
 Note that posts from new members are moderated - please be patient with 
 your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 --- 
 You received this message because you are subscribed to the Google 
 Groups Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to clojure+u...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


  -- 
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com 
 javascript:
 Note that posts from new members are moderated - please be patient with 
 your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com javascript:
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 --- 
 You received this message because you are subscribed to the Google Groups 
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to clojure+u...@googlegroups.com javascript:.
 For more options, visit https://groups.google.com/d/optout.




-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [OT?] Best DB/architecture for n-gram corpus?

[Sam Raker]

That's interesting. I've been really reluctant to hard code n-grams, but 
it's probably the best way to go.

On Monday, March 9, 2015 at 6:12:43 PM UTC-4, John Wiseman wrote:

 One thing you can do is index 1, 2, 3...n-grams and use a simple  fast 
 key-value store (like leveldb etc.)  e.g., you could have entries like

 aunt rhodie - song-9, song-44
 woman - song-12, song-65, song-96


 That's basically how I made the Metafilter N-gram Viewer 
 http://mefingram.appspot.com/, a clone of Google Books Ngram Viewer 
 https://books.google.com/ngrams.

 Another possibility is using Lucene.  Just be aware that Lucene calls 
 n-grams of characters (au, un, nt) n-grams but it calls n-grams of 
 words (that the, the old, old gray) shingles.  So you would end up 
 using (I think, I haven't done this) the ShingleFilter 
 https://lucene.apache.org/core/4_2_0/analyzers-common/org/apache/lucene/analysis/shingle/ShingleFilter.html
 .

 You might also find this article by Russ Cox interesting, where he 
 describes building and using an inverted trigram index: 
 http://swtch.com/~rsc/regexp/regexp4.html


 John





 Three things that you might find interesting:

 Russ Cox' explanation of doing indexing and retrieval with an inverted 
 trigram index: http://swtch.com/~rsc/regexp/regexp4.html


 On Sat, Mar 7, 2015 at 3:22 AM, Matching Socks phill...@gmail.com 
 javascript: wrote:

 A lot of guys would use Lucene.  Lucene calls n-grams of words 
 shingles. [1]

 As for architecture, here is a suggestion to use Lucene to find keys to 
 records in your real database. [2]

 [1] https://lucidworks.com/blog/whats-a-shingle-in-lucene-parlance/

 [2] https://groups.google.com/d/msg/datomic/8yrCYxcQq34/GIomGaarX5QJ


  -- 
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com 
 javascript:
 Note that posts from new members are moderated - please be patient with 
 your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com javascript:
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 --- 
 You received this message because you are subscribed to the Google Groups 
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to clojure+u...@googlegroups.com javascript:.
 For more options, visit https://groups.google.com/d/optout.




-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[OT?] Best DB/architecture for n-gram corpus?

[Sam Raker]

I'm trying to create an n-gram[1] corpus out of song lyrics. I'm breaking 
individual songs into lines, which are then split into words, so you end up 
with something like

{0 {0 go 1 tell 2 aunt 3 rhodie} 1 {0 the 1 old 2 grey 3 
goose 4 is 5 dead}...}

(Yes, maps with integer keys is kind of dumb; I thought about using 
vectors, but this is all going into MongoDB temporarily, and I'd rather 
just deal with maps instead of messing with Mongo's somewhat lacking 
array-handling stuff.)

The idea, ultimately, is to build a front-end that would allow users to, 
e.g., search for all songs that contain the (sub)string aunt rhodie, or 
see how many times The Rolling Stones use the word woman vs how many 
times the Beatles do, etc. The inspiration comes largely from projects like 
COCA[2]. 

I'm wondering if any of you have opinions about which database to use 
(Mongo is most likely just a stopgap), and how best to architect it. I'm 
most familiar with MySQL and Mongo, but I'd rather not be limited by just 
those two if there's a better option out there. I'm thinking that I'll 
probably end up storing tokens over types--e.g., each word would be stored 
individually, as opposed to having an entry for, e.g., the that stores 
every instance of the word the. I was also thinking that I'll probably 
have to end up storing each token's previous and next, either as full 
references or just as strings. This seems potentially inefficient, however. 

(I could've just gone to StackOverflow with this, but figured I'm more 
likely to get a real answer here, because you all seem so smart and nice?)


Thanks!



[1] https://en.wikipedia.org/wiki/N-gram
[2] http://corpus.byu.edu/coca/

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Better/more idiomatic way to read EDNs than using java.io.PushbackReader

[Sam Raker]

I'm experimenting a little with EDN files. I've currently got this function:

(defn from-edn [edn-file] (with-open [r (clojure.java.io/reader edn-file)]
(edn/read (java.io.PushbackReader. r

Having to explicitly reach into the Java API to read a clojure-only format 
seems wrong. What should I be doing instead?

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Disk based caching for Clojure app

[Sam Raker]

I'm under the impression that, because of the hard limit on writes, OSes 
often already cache writes to SSDs, further limiting their usefulness in 
this kind of application.

On Friday, March 6, 2015 at 4:10:54 PM UTC-5, Fluid Dynamics wrote:

 On Friday, March 6, 2015 at 3:16:09 PM UTC-5, Michael Blume wrote:

 Possibly stupid question: can you just pretend you have more memory than 
 you do and let the operating system do the heavy lifting?

 As in, put the swap partition on the SSD and jack up the virtual memory in 
 the OS config?

 Isn't it a bad idea to put swap on an SSD, because of the limited number 
 of times an SSD byte can be rewritten before it sticks permanently? Thus 
 making SSD more suited to storing stuff where speed counts, but which 
 doesn't change very much, like the operating system kernel and modules and 
 core libraries, plus your most often used applications? Then you get faster 
 boot and app-startup times without constant writes to the SSD (just when 
 something is upgraded).

 Of course, SSD being less well suited to frequently-written data would 
 also militate against using it for a cache managed by the application, 
 rather than by the OS ...
  


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: :reload does not always work correctly in leiningen

[Sam Raker]

does not always work correctly  in what context? In the REPL, `(require 
'[my.ns :as mine] :reload-all)` should work. `defonce` and `defmulti` might 
also be causing confusion. You could also look into 
`clojure.tools.namespace.repl` 
(https://github.com/clojure/tools.namespace#reloading-code-usage).

On Saturday, February 28, 2015 at 9:25:56 AM UTC-5, Cecil Westerhof wrote:

 I discovered:
 (require 'project.core :reload)

 Very handy indeed and a big time saver. But it does not always work 
 correctly. At a certain moment I got strange results. An exit and a new 
 'lein repl' solved the problems.

 -- 
 Cecil Westerhof
  

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: printf does not always generate output

[Sam Raker]

I 
have 
https://jafingerhut.github.io/cheatsheet/grimoire/cheatsheet-tiptip-cdocs-summary.html
 
bookmarked and also like basically always open in a tab forever, fwiw. 

On Friday, February 27, 2015 at 4:37:38 PM UTC-5, Cecil Westerhof wrote:

 2015-02-27 20:36 GMT+01:00 Andy Fingerhut andy.fi...@gmail.com 
 javascript::

 Not every function and macro is documented completely or accurately on 
 ClojureDocs.org, but I would recommend checking it out when you run across 
 something that looks amiss, and see if the examples mention what you are 
 seeing.  In this case, it does:

 http://clojuredocs.org/clojure.core/printf

  
 ​That is certainly a place I should look more often. :-) A wealth of 
 information.


 On Fri, Feb 27, 2015 at 11:12 AM, Cecil Westerhof cldwes...@gmail.com 
 javascript: wrote:

 My core.clj ends with:
 (println (format Started: %s (println) (java.util.Date.)))
 (printf Started: %s (printf)\n (java.util.Date.))

 I do not see the output from the printf when I run 'lein repl'.
 The funny thing is that when I switch the two statements, both are shown.
 Also when I do 'lein run' both are shown.

 What could be happening here?

 For the moment I changed all my printf to println with format, but it is 
 a little annoying.



 -- 
 Cecil Westerhof
  

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How to create jar for data.int-map

[Sam Raker]

@Cecil: while it's a little irritating that there's not more centralization 
of third-party Clojure libs, I've found that Leiningen + some googling 
solves the problem like 99% of the time.

On Thursday, February 19, 2015 at 10:42:43 AM UTC-5, Alex Miller wrote:

 The first line of the readme for the project (
 https://github.com/clojure/data.int-map) has the Leiningen jar dependency 
 info on it:

 [org.clojure/data.int-map 0.1.0]

 Also, the 'mvn package' command should have worked since that's how 
 releases get built. That was a bug in the pom dependencies and I have 
 updated it to match the version in project.clj.




 On Thursday, February 19, 2015 at 5:39:22 AM UTC-6, Cecil Westerhof wrote:

 2015-02-19 11:58 GMT+01:00 Michael Griffiths mikeygr...@gmail.com:

 Clojure and Clojure contrib libraries are uploaded to Sonatype when 
 released. e.g. 
 https://oss.sonatype.org/content/groups/public/org/clojure/data.int-map/
  
 There should be both a jar with sources and jar without sources for each 
 released version of each lib.

  
 ​Thanks. I think information like this should be easier to find. I like 
 Clojure, but sometimes it is difficult to find things.


 -- 
 Cecil Westerhof
  


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How to create jar for data.int-map

[Sam Raker]

I guess by google I really meant using google to search github. AFAICT,
there's 0 reason to opt for anything other than Leiningen when given the
choice. It makes things astonishingly easy, and nearly every
more-than-alpha library has lein dependency info right up near the top. If
they don't, clojars or maven should have it.

On Thu, Feb 19, 2015 at 11:19 AM, Cecil Westerhof cldwester...@gmail.com
wrote:



 2015-02-19 17:00 GMT+01:00 Sam Raker sam.ra...@gmail.com:

 @Cecil: while it's a little irritating that there's not more
 centralization of third-party Clojure libs, I've found that Leiningen +
 some googling solves the problem like 99% of the time.


 ​I did come quit an end with Google, but then I was bitten by a broken pom
 file.
 I think I like Leiningen better as Maven, so I should invest some time in
 Leiningen.
 ​With Google I only found build information about Maven, not about
 Leiningen. But I am enlightened now. :-D
 ​


 On Thursday, February 19, 2015 at 10:42:43 AM UTC-5, Alex Miller wrote:

 The first line of the readme for the project (
 https://github.com/clojure/data.int-map) has the Leiningen jar
 dependency info on it:

 [org.clojure/data.int-map 0.1.0]

 Also, the 'mvn package' command should have worked since that's how
 releases get built. That was a bug in the pom dependencies and I have
 updated it to match the version in project.clj.




 On Thursday, February 19, 2015 at 5:39:22 AM UTC-6, Cecil Westerhof
 wrote:

 2015-02-19 11:58 GMT+01:00 Michael Griffiths mikeygr...@gmail.com:

 Clojure and Clojure contrib libraries are uploaded to Sonatype when
 released. e.g. https://oss.sonatype.org/content/groups/public/org/
 clojure/data.int-map/

 There should be both a jar with sources and jar without sources for
 each released version of each lib.


 ​Thanks. I think information like this should be easier to find. I like
 Clojure, but sometimes it is difficult to find things.


 --
 Cecil Westerhof

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to a topic in the
 Google Groups Clojure group.
 To unsubscribe from this topic, visit
 https://groups.google.com/d/topic/clojure/cDNGgpTeAcc/unsubscribe.
 To unsubscribe from this group and all its topics, send an email to
 clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: getting enlive not to parse something

[Sam Raker]

I think the easiest solution for me is to just use clj-http to get the 
content of the leaf pages and then hit 'em with a regex. It's not pretty, 
but, then again, neither is the tagsoup documentation.

On Saturday, February 14, 2015 at 1:03:23 PM UTC-5, Herwig Hochleitner 
wrote:

 Enlive by default uses tagsoup to parse html and can also use jsoup, so 
 I'd start with parsing options for those. It can also be used with other 
 parsers, provided you can get them to generate clojure's xml map format.

 2015-02-13 18:55 GMT+01:00 Sam Raker sam@gmail.com javascript::

 I'm trying to parse some stuff with enlive, but part of at least of the 
 pages look like:
 ...
 pre
 ...
 8-bar break...
 ...
 /pre

 Enlive interprets the text in the pointy braces as html, which it isn't, 
 and that's messing up my application. Is there a way to tell enlive to not 
 parse anything within a given tag?

 -- 
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com 
 javascript:
 Note that posts from new members are moderated - please be patient with 
 your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com javascript:
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 --- 
 You received this message because you are subscribed to the Google Groups 
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to clojure+u...@googlegroups.com javascript:.
 For more options, visit https://groups.google.com/d/optout.




-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

getting enlive not to parse something

[Sam Raker]

I'm trying to parse some stuff with enlive, but part of at least of the 
pages look like:
...
pre
...
8-bar break...
...
/pre

Enlive interprets the text in the pointy braces as html, which it isn't, 
and that's messing up my application. Is there a way to tell enlive to not 
parse anything within a given tag?

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: 1.7.0-alpha5 clobber warning error?

[Sam Raker]

I think I get it. I still find the message confusing, though--in the 
current ns, update isn't being clobbered by monger.collection/update.

On Wednesday, February 11, 2015 at 9:02:47 PM UTC-5, Andy Fingerhut wrote:

 Sam:

 I believe if you proceed to do the following in your REPL:

 (in-ns 'monger.collection)
 (doc update)

 You will see that the name 'update' _from within the namespace 
 monger.collection_ refers to monger.collection/update, not 
 clojure.core/update.

 That is what the message is intended to convey.

 In the user namespace where you are making your doc calls, the name update 
 still refers to clojure.core/update.

 Andy

 On Wed, Feb 11, 2015 at 5:53 PM, Sam Raker sam@gmail.com 
 javascript: wrote:

 I just added monger as a dependency to my Clojure 1.7.0-alpha5 project, 
 and saw the following in my repl:

 user= (require '(monger [core :as mg] [collection :as mc]))
 WARNING: update already refers to: #'clojure.core/update in namespace: 
 monger.collection, being replaced by: #'monger.collection/update

 That warning seems to be in error, though, because `(doc update)` and 
 `(doc mc/update)` print different (and correct) things.

 Is this a bug? Should I file something somewhere?

 -- 
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com 
 javascript:
 Note that posts from new members are moderated - please be patient with 
 your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com javascript:
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 --- 
 You received this message because you are subscribed to the Google Groups 
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to clojure+u...@googlegroups.com javascript:.
 For more options, visit https://groups.google.com/d/optout.




-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

1.7.0-alpha5 clobber warning error?

[Sam Raker]

I just added monger as a dependency to my Clojure 1.7.0-alpha5 project, and 
saw the following in my repl:

user= (require '(monger [core :as mg] [collection :as mc]))
WARNING: update already refers to: #'clojure.core/update in namespace: 
monger.collection, being replaced by: #'monger.collection/update

That warning seems to be in error, though, because `(doc update)` and `(doc 
mc/update)` print different (and correct) things.

Is this a bug? Should I file something somewhere?

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Extending the LispReader with embeded language lorms

[Sam Raker]

For a competent English speaker/reader, Infinite Jest is hard to read 
because it's dense and elliptical c. c. For that same reader, Tintin in 
the original French is hard to read because it's in French. I think 
that's a relevant distinction to make in this context. Extensibility is 
nice, but there comes a point where, for all intents and purposes, a piece 
of code stops being Clojure and starts being something else. 

On Tuesday, February 10, 2015 at 3:12:46 PM UTC-5, Gary Verhaegen wrote:

 I *think* Alex means read in the very specific and technical sense of a 
 Lisp reader, i.e. a piece of program that turns a stream of characters into 
 data structures in memory, and then I guess the other users are all of 
 the other programs, beside the Clojure compiler itself, that may want to 
 analyze or manipulate Clojure code.

 I certainly hope he's not proposing to excommunicate abybody writing hard 
 to understand code on occasion. ;-)

 On Tuesday, 10 February 2015, Ben Wolfson wol...@gmail.com javascript: 
 wrote:

 On Tue, Feb 10, 2015 at 11:29 AM, Alex Miller a...@puredanger.com 
 wrote:

 Hi Henrik,

 There is a long-standing philosophical position in Clojure that it 
 should not be possible to write programs that cannot be read by other 
 users. 


 What does that mean?
  
 -- 
 Ben Wolfson
 Human kind has used its intelligence to vary the flavour of drinks, 
 which may be sweet, aromatic, fermented or spirit-based. ... Family and 
 social life also offer numerous other occasions to consume drinks for 
 pleasure. [Larousse, Drink entry]

  -- 
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with 
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 --- 
 You received this message because you are subscribed to the Google Groups 
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.



-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

parameterized/-izable transducers

[Sam Raker]

So I've written a transducer to build up urls from relative links:

  (defn build-url
([root]
  (fn [xf]
  (fn 
([] (xf))
([links] (xf links))
([links suffix]
(xf links (let [root (if (.endsWith root .html)
 (string/replace root #\w+\.html$ )
 root)]
(str root suffix

This works great when I know the root in advance. The issue is that I 
want to run this recursively, building up the links as I go. As an example:
1) The main index page is, e.g., http://www.website.com/index.html;. The 
index.html part gets stripped off by the transducer, leaving the root as 
http://www.website.com/index.html;.
2) index.html contains, inter alia, a list of links to other index pages, 
e.g. /index1/, /index2/, etc.
3) Running the transducer once yields `[http://www.website.com/index1/;, 
http://www.website.com/index2/...]`
4) I want to be able to then run the transducer again on the links scraped 
from each of the sub-indexes, ensuring that each is compounded with the 
correct root element, e.g. `[http://www.website.com/index1/foo;, 
http://www.website.com/index1/bar;, http://www.website.com/index2/baz;, 
http://www.website.com/index2/quux...]`

Ideally, I'd like to be able to do this with as little intermediate stuff 
as possible, i.e., not doing something like `(into [] (comp transducer1 
transducer2 build-url) (into [] (comp transducer3 build-url)))`.

Is there a way to capture this with transducers? Something like (fn [foo] 
(mapcat #(somefn foo %) (someotherfn foo))?

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: remote nrepl/gorilla + port forwarding

[Sam Raker]

Oh genius. Thanks! (Also thanks for pointing out the MASSIVE security hole 
I was trying to create for myself.)

On Thursday, January 15, 2015 at 6:19:04 AM UTC-5, Jony Hudson wrote:

 Hi Sam,

  I think `lein gorilla :ip 0.0.0.0 :port 5` should work (it works on 
 my machine). And if your router is forwarding 5 to that machine 
 correctly then it should be accessible from outside. `lein gorilla :ip 
 ROUTER_IP :port 5` shouldn't work because ROUTER_IP isn't an IP address 
 bound to the machine, so it will not be able to run a server on it.

 More importantly, though, DO NOT DO THIS!!! The reason is that this will 
 give full access (with the privileges of the user running Gorilla) to 
 anyone on the internet. Stuff like reading, deleting all of your files etc 
 :-( Gorilla doesn't have any form of authentication mechanism built in, and 
 the server mode is really meant for trusted access only. If you want to run 
 Gorilla over a non-trusted network then you should set it up to use some 
 form of authentication.

 The solution I usually use for this sort of thing is to tunnel through 
 SSH. So, first of all make sure you can access SSH on THAT_COMPUTER ... if 
 your router allows you to forward 5 external to 22 on THAT_COMPUTER 
 then that would work. If you can't control the target port then you might 
 need to run the SSH server on a different port (5).

 Once you've got SSH running then you can run the Gorilla server on the 
 internal (loopback) IP i.e. the default setting. This is not accessible 
 from outside the machine, but it is accessible by the SSH server which is 
 running on the machine. So you can then use ssh tunnelling to access this 
 port from the outside machine.

 So:

 - set up router to forward ssh
 - run `lein gorilla :port 6` on THAT_COMPUTER
 - on the remote machine `ssh -L 8090:127.0.0.1:6 -p 5 
 me@ROUTER_IP`. This connects to the SSH server on 5, and tells it to 
 route traffic from the local machine's port 8080 to the remote machine's 
 port 6, which is the port that Gorilla is running on. This will be done 
 by the ssh server, internal to the remote machine, so does not need Gorilla 
 to be externally accessible.
 - then you should be able to securely access Gorilla on the remote machine 
 at `http://localhost:8090/...` http://localhost:8090/ SSH will 
 route this as described above.

 A diagram would really help here with all of the ports, but hopefully you 
 get the idea :-)


 Jony

 On Thursday, 15 January 2015 01:01:56 UTC, Sam Raker wrote:

 I've got a computer with a bunch of clojure code on it, sitting at home 
 on my home network. I've configured my router to forward port 5 on that 
 computer to port 5 on the router itself, so that, at least in theory, 
 ROUTER_IP:5 should be forwarded to THAT_COMPUTER:5, if that makes 
 sense. I've done this same thing for a number of other things, including 
 SSH, MySQL, etc.

 When I'm at home, I can fire up `lein gorilla :ip THAT_COMPUTER :port 
 5` and then go to http://THAT_COMPUTER:5/worksheet.html, or, 
 more directly, `lein repl :headless THAT_COMPUTER:5`/`lein repl 
 :connect THAT_COMPUTER:5`, and it works. I'm trying to do the same from 
 not-at-home, no dice. (For the curious: I can SSH into THAT_COMPUTER, but 
 the connection is REALLY slow.)

 I've tried `lein gorilla :ip ROUTER_IP :port 5` (from THAT_COMPUTER), 
 but get a java.net.BindException about not being able to assign the 
 requested address--which makes sense--but `lein gorilla :ip 0.0.0.0 :port 
 5` doesn't seem to work either. Same with `lein repl :headless :host 
 0.0.0.0.0 :port 5`. 


 Any suggestions? Is this just not a thing I can do with these tools at 
 this point in their development? Is this a thing that should just work and 
 possibly my dumb router or some other thing is screwing it up?



 Thanks!
 -sam



-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: DSL in RTL (Right to Left) languages.

[Sam Raker]

I think there's something to be said for non-English programming languages, 
primarily for the reasons the OP suggested--toy/beginner languages to get 
more people coding. Languages are judged for readability and similar 
metrics, with (standard) English as the yardstick. Code in English because 
you have to learn English anyway to talk to other people who program in 
English is a bit circular, as well. 

On Wednesday, January 14, 2015 at 11:30:00 AM UTC-5, HamsterofDeath wrote:

 i encountered a german progamming language once. it was terrible. 
 everybody should stick to english when it comes ot programming - you have 
 to do it anyway, and there is no reason not to go ahead and learn a 
 language since that is what brains are built for

 2015-01-14 17:11 GMT+01:00 Jesse Alama jesse...@gmail.com javascript::

 On Wednesday, January 14, 2015 at 2:42:57 PM UTC+1, clojur...@gmail.com 
 wrote:
  

 Thanks Jan,

 Good idea!

 It is just a hobby project for now... I am thinking of a language for 
 kids (8+) . Would be interesting to see how kids react to programming in a 
 more familiar language.


 Some similar work has been with قلب.  It's a Scheme-like language
 written entirely in Arabic.  Article about the language:

 http://www.theregister.co.uk/2013/01/25/arabic_programming_language

 GitHub repo:

 https://github.com/nasser/---

 Jesse
  

 -- 
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com 
 javascript:
 Note that posts from new members are moderated - please be patient with 
 your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com javascript:
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 --- 
 You received this message because you are subscribed to the Google Groups 
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to clojure+u...@googlegroups.com javascript:.
 For more options, visit https://groups.google.com/d/optout.




-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

remote nrepl/gorilla + port forwarding

[Sam Raker]

I've got a computer with a bunch of clojure code on it, sitting at home on 
my home network. I've configured my router to forward port 5 on that 
computer to port 5 on the router itself, so that, at least in theory, 
ROUTER_IP:5 should be forwarded to THAT_COMPUTER:5, if that makes 
sense. I've done this same thing for a number of other things, including 
SSH, MySQL, etc.

When I'm at home, I can fire up `lein gorilla :ip THAT_COMPUTER :port 
5` and then go to http://THAT_COMPUTER:5/worksheet.html, or, more 
directly, `lein repl :headless THAT_COMPUTER:5`/`lein repl :connect 
THAT_COMPUTER:5`, and it works. I'm trying to do the same from 
not-at-home, no dice. (For the curious: I can SSH into THAT_COMPUTER, but 
the connection is REALLY slow.)

I've tried `lein gorilla :ip ROUTER_IP :port 5` (from THAT_COMPUTER), 
but get a java.net.BindException about not being able to assign the 
requested address--which makes sense--but `lein gorilla :ip 0.0.0.0 :port 
5` doesn't seem to work either. Same with `lein repl :headless :host 
0.0.0.0.0 :port 5`. 


Any suggestions? Is this just not a thing I can do with these tools at this 
point in their development? Is this a thing that should just work and 
possibly my dumb router or some other thing is screwing it up?



Thanks!
-sam

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Q: How to parse stream of text (from java.io.Reader) via instaparse with minimal input consuption

[Sam Raker]

How structured/delimited is the other part? Could you write your parser 
in such a way as to parse what you want and then return the rest as a 
single chunk you could then pass to something else?

On Tuesday, January 6, 2015 at 11:50:26 AM UTC-5, henrik42 wrote:

 Hi all,

 I want to parse a stream of text which comes from a java.io.Reader with 
 https://github.com/Engelberg/instaparse.

 But the stream of text will only start with what can be parsed by my 
 grammar. The rest of the text stream must be 
 consumed/parsed with some other grammar. I know of instparse's :total 
 parse which can be used to parse as
 much as possible and it will give you what could not be parsed.
 But the text comes from a stream. And further processing will consume this 
 (statefull) stream/Reader. I cannot just slurp the Reader,
 give instaparse the String and then process the String for the 
 not-parsed-part. And I cannot unread that much (it's a PushbackReader).

 So my question is: is there a way to use instaparse with a Reader/Stream 
 in such a way, that instaparse will consume
 exactly those Chars/Bytes that are covered by the resulting parse-tree 
 and leave any following data un-read in the Reader?
 (Since it is a PushbackReader with buffersize=1 I could accept a 
 1-char-lookahead read)

 Any idea?

 Henrik


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] Neanderthal, a fast, native matrix and linear algebra library for Clojure released + call for help

[Sam Raker]

I'd like to politely add to the calls for this to become a pluggable 
core.matrix backend.

On Tuesday, January 13, 2015 at 4:38:22 PM UTC-5, Dragan Djuric wrote:

 It would be nice if that would be that easy. However, I am sceptical...

 On Tuesday, January 13, 2015 at 8:13:36 PM UTC+1, Christopher Small wrote:

 Awesome project!

 I'll echo the encouragement towards having Neanderthal implement the 
 core.matrix protocols. You'll have much higher adoption if folks know they 
 can just plug your tool in by changing a single line setting the underlying 
 implementation to Neanderthal. And as Mikera points out, it would be nice 
 if we kept the Clojure matrix API space cohesive.



-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Help with Incanter and Emacs

[Sam Raker]

For something that's been deprecated for a while now, `use` sure still 
shows up in A LOT of docs/tutorials/books/etc.

On Monday, January 12, 2015 at 7:48:44 PM UTC-5, Robert Berger wrote:

 Wish this was in the Incanter docs and in the Readme

 On Tuesday, August 13, 2013 at 12:09:20 PM UTC-7, Tim Visher wrote:

 LOL. I just realized I'd been missing that all along. 

 It seems like the community is more and more leaning to something like 
 this, just FYI. 

 (ns default.core 
   (:require (incanter [core  :refer :all] 
 [charts:refer :all] 
 [stats  :refer :all] 
 [datasets :refer :all]))) 

 `:use` has been discussed in the interest of deprecating it many times. 

 I did not test the above declaration. 

 On Tue, Aug 13, 2013 at 2:40 PM, Akhil Wali green.tr...@gmail.com 
 wrote: 
  Well this is embarrassing. 
  
  I was having a wrong use syntax .. 
  Changed the import line to this and it works. 
  
  (ns default.core 
(:use [incanter core charts stats datasets])) 
  
  Thanks for the help though!! 
  
  On Tuesday, August 13, 2013 11:57:46 PM UTC+5:30, Akhil Wali wrote: 
  
  Yes, by nrepl-jack-in. 
  
  On Tuesday, August 13, 2013 11:54:24 PM UTC+5:30, Tim Visher wrote: 
  
  And you're connecting to the project how? 
  
  On Tue, Aug 13, 2013 at 2:11 PM, Akhil Wali green.tr...@gmail.com 
  wrote: 
   Well 
   Here's my project.clj. 
   
   (defproject someproj 0.1.0-SNAPSHOT 
 :dependencies [[org.clojure/clojure 1.5.1] 
[incanter 1.5.2]]) 
   
   Here's my .lein/profiles.clj as well. 
   
   {:user {:plugins [[lein-ritz 0.7.0] 
 [compojure-app/lein-template 0.2.7]] 
   :dependencies [[ritz/ritz-nrepl-middleware 0.7.0]] 
   :repl-options {:nrepl-middleware 
  [ritz.nrepl.middleware.javadoc/wrap-javadoc 
   
   ritz.nrepl.middleware.simple-complete/wrap-simple-complete]}}} 
   
   
   
   On Tuesday, August 13, 2013 11:35:58 PM UTC+5:30, Tim Visher wrote: 
   
   On Tue, Aug 13, 2013 at 1:59 PM, Akhil Wali green.tr...@gmail.com 

   wrote: 
Hi All, 

A really noob question. 

Why do I get FileNotFoundException Could not locate 
incanter__init.class or 
incanter.clj on classpath:   clojure.lang.RT.load (RT.java:443) 
when i 
load 
a file that uses incanter in emacs? 
Here's the file... 

(ns default.core 
  (:require incanter core charts stats datasets)) 

(defn plot [] 
  (view (scatter-plot :Sepal.Length :Sepal.Width 
  :group-by :Species 
  :data (get-dataset :iris 

(plot) 

This works in lein repl just fine. 
I'm using Emacs 24 and nrepl.el 0.1.8. 

This issue was posted on Github way long back. Seems to be 
 solved, 
but 
by 
simply upgrading emacs. Doesn't really work in my case. 
Any advise? 
   
   Couple things. 
   
   1. I'm assuming you've declared the proper dependencies in your 
   `project.clj` file since this works from `lein repl`, however it's 
   worth checking. 
   
   2. How are you connecting to your project? Simply loading the 
   namespace won't work if you haven't properly jack in. There's a 
 number 
   of options here but the simplest is probably to use `M-x 
   nrepl-jack-in` (usually bound to `C-c M-j`) from this file. It 
 should 
   Just Work™. 
   
   If that doesn't work, I'd probably post some more details about 
 the 
   project somewhere. At least the `project.clj` and the whole ns 
 would 
   be helpful in a gist of some sort. 
   
   -- 
   
   In Christ, 
   
   Timmy V. 
   
   http://blog.twonegatives.com/ 
   http://five.sentenc.es/ -- Spend less time on mail 
   
   -- 
   -- 
   You received this message because you are subscribed to the Google 
   Groups Clojure group. 
   To post to this group, send email to clo...@googlegroups.com 
   Note that posts from new members are moderated - please be patient 
 with 
   your 
   first post. 
   To unsubscribe from this group, send email to 
   clojure+u...@googlegroups.com 
   For more options, visit this group at 
   http://groups.google.com/group/clojure?hl=en 
   --- 
   You received this message because you are subscribed to the Google 
   Groups 
   Clojure group. 
   To unsubscribe from this group and stop receiving emails from it, 
 send 
   an 
   email to clojure+u...@googlegroups.com. 
   For more options, visit https://groups.google.com/groups/opt_out. 
  
  -- 
  -- 
  You received this message because you are subscribed to the Google 
  Groups Clojure group. 
  To post to this group, send email to clo...@googlegroups.com 
  Note that posts from new members are moderated - please be patient with 
 your 
  first post. 
  To unsubscribe from this group, send email to 
  clojure+u...@googlegroups.com 
  For more options, visit this group at 
  

Re: Handling increasingly-intensive processes

[Sam Raker]

Now that someone's said it, just store tweets seems like such a duh 
move.

Thanks!
-sam

On Monday, December 15, 2014 6:35:13 AM UTC-5, Thomas Heller wrote:

 Hey,

 without knowing much about your application/business needs its hard to 
 speculate what might be good for you. The root of your problem might be 
 CouchDB since it was never meant for Big Data and since we are talking 
 tweets I generally think a lot. I'm not sure how your map value looks but 
 I think you do something like

 obj = (couch/get hash-tag)
 obj = (my-app/update obj new-tweet)
 (couch/put hash-tag obj)

 Which will always perform badly since you cannot do this concurrently, 
 except with CRDTs which CouchDB doesn't support since it does its own 
 MVCC.  Don't remember exaclty how their conflict resolution works but I 
 think it was last write wins. Caching will not save you for long, since 
 writes will eventually become the bottleneck.

 Why do you not use a CouchDB view to create the hash-tag map on the server 
 and then just append-only the tweets? The views map function can then just 
 emit each tweet under the hash-tag key (once for each tag) and the reduce 
 function can build your map. That should perform alot better up to a 
 certain point and you can control how up-to-date your view index has to be.

 Anyways, might be best to choose another Database. Regardless of what 
 database you are using, updating a single place concurrently is going to be 
 a problem. An Atom in Clojure makes this look like a no-brainer but under 
 high load it can still blow up since it has no back-pressure in any way.

 Bit Data and Distributed Systems are hard and cannot be described in 
 short. Without exact knowledge of what your app/business needs look like it 
 is impossible to make the correct recommendation.

 HTH,
 /thomas

 On Monday, December 15, 2014 4:54:04 AM UTC+1, Sam Raker wrote:

 I'm (still) pulling tweets from twitter, processing them, and storing 
 them in CouchDB with hashtags as doc ids, such that if a tweet contains 3 
 hashtags, that tweet will be indexed under each of those 3 hashtags. My 
 application hits CouchDB for the relevant document and uses Cheshire to 
 convert the resulting string to a map. The map's values consist of a few 
 string values and an array that consists of all the tweets that contain 
 that hashtag. The problem is thus with common hashtags: the more tweets 
 contain a given hashtag, the long that hashtag's tweets array will be, 
 and, additionally, the more often that document will be retrieved from 
 CouchDB. The likelihood and magnitude of performance hits on my app are 
 therefore correlated, which is Bad.

 I'm reaching out to you all for suggestions about how best to deal with 
 this situation. Some way of caching something, somehow? I'm at a loss, but 
 I want to believe there's a solution.


 Thanks,
 -sam



-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Handling increasingly-intensive processes

[Sam Raker]

I'm (still) pulling tweets from twitter, processing them, and storing them 
in CouchDB with hashtags as doc ids, such that if a tweet contains 3 
hashtags, that tweet will be indexed under each of those 3 hashtags. My 
application hits CouchDB for the relevant document and uses Cheshire to 
convert the resulting string to a map. The map's values consist of a few 
string values and an array that consists of all the tweets that contain 
that hashtag. The problem is thus with common hashtags: the more tweets 
contain a given hashtag, the long that hashtag's tweets array will be, 
and, additionally, the more often that document will be retrieved from 
CouchDB. The likelihood and magnitude of performance hits on my app are 
therefore correlated, which is Bad.

I'm reaching out to you all for suggestions about how best to deal with 
this situation. Some way of caching something, somehow? I'm at a loss, but 
I want to believe there's a solution.


Thanks,
-sam

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

futures + refs

[Sam Raker]

I've got some code that's using Twitter's HoseBirdClient to pull tweets 
from the public stream, which I then preprocess and store with CouchDB. 
Right now, my HBC client is being forced to reconnect more than I'd like, 
which occasionally causes my app to hang, for reasons I'm not entirely 
clear on. Regardless, some preliminary research on HBC suggests that the 
reconnections are being caused by my code failing to keep up with the 
endpoint, which in turn suggests that my processing+uploading is taking too 
long. I tried wrapping the processing+uploading part in futures, which 
definitely sped things up, but caused 409 errors when uploading to 
CouchDB--briefly, Couch requires any update operation to include a 
git-style rev string, and if the rev you provide isn't the most recent 
one, it throws a 409 at you. I'm organizing things by hashtag, so tweets 
with multiple copies of the same hashtag, or series of tweets with the same 
hashtag are the culprit--future A gets the current doc from Couch, 
processes it, and uses the rev it got from the currently-existing doc, 
while future B does the same thing, but finishes first, so now future A has 
an outdated rev, and that causes the 409. 

The vague solution I've come up with involves using a map to store the rev 
values, with the last step of the processing/uploading function being to 
store the rev number Clutch helpfully returns to you after a successful 
update. From what I can tell, refs are the way to go, since each future is 
effectively a separate thread. My questions are as follows:
1) Would I have to store the map-of-refs in a ref?
2) Is this even feasible? Would the timing work out?
3) With the addition of all this dereferencing and `dosync`+`alter`-ing, 
would this actually end up speeding things up all that much?

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: futures + refs

[Sam Raker]

So HBC actually does that already--it dumps tweets into a
LinkedBlockingQueue. Right now, I'm doing `(loop [tweet (.take queue)]...`,
which, I think, essentially amounts to what you're suggesting, but I could
be misunderstanding you.

There's a distinct possibility that all the reconnections are caused by my
home's internet connection--my local ssh connections get dropped
constantly, which suggests there might be a problem somewhere. I figured
trying to optimize my code couldn't hurt, though.

On Thu, Dec 11, 2014 at 11:43 AM, László Török ltoro...@gmail.com wrote:

 Hi Sam,

 have you tried putting the incoming (hashtag,tweet) tuples into a queue
 and have another thread pull them out and upload them to couchdb?

 I'm unfamiliar with HBC, but I assume it has a callback-based API, so you
 should be able to have multiple callbacks/connections/streams feed the same
 queue and have a single thread do the upload (and maybe batch if necessary).

 I don't see refs being a particularly good fit for this problem, but I
 could be wrong.


 2014-12-11 16:18 GMT+00:00 Sam Raker sam.ra...@gmail.com:

 I've got some code that's using Twitter's HoseBirdClient to pull tweets
 from the public stream, which I then preprocess and store with CouchDB.
 Right now, my HBC client is being forced to reconnect more than I'd like,
 which occasionally causes my app to hang, for reasons I'm not entirely
 clear on. Regardless, some preliminary research on HBC suggests that the
 reconnections are being caused by my code failing to keep up with the
 endpoint, which in turn suggests that my processing+uploading is taking too
 long. I tried wrapping the processing+uploading part in futures, which
 definitely sped things up, but caused 409 errors when uploading to
 CouchDB--briefly, Couch requires any update operation to include a
 git-style rev string, and if the rev you provide isn't the most recent
 one, it throws a 409 at you. I'm organizing things by hashtag, so tweets
 with multiple copies of the same hashtag, or series of tweets with the same
 hashtag are the culprit--future A gets the current doc from Couch,
 processes it, and uses the rev it got from the currently-existing doc,
 while future B does the same thing, but finishes first, so now future A has
 an outdated rev, and that causes the 409.

 The vague solution I've come up with involves using a map to store the
 rev values, with the last step of the processing/uploading function being
 to store the rev number Clutch helpfully returns to you after a successful
 update. From what I can tell, refs are the way to go, since each future is
 effectively a separate thread. My questions are as follows:
 1) Would I have to store the map-of-refs in a ref?
 2) Is this even feasible? Would the timing work out?
 3) With the addition of all this dereferencing and `dosync`+`alter`-ing,
 would this actually end up speeding things up all that much?

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.




 --
 László Török
 --
 Checkout http://www.lollyrewards.com/

  --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to a topic in the
 Google Groups Clojure group.
 To unsubscribe from this topic, visit
 https://groups.google.com/d/topic/clojure/zRy6dm1Vmcs/unsubscribe.
 To unsubscribe from this group and all its topics, send an email to
 clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group

more colloquial do x n times

[Sam Raker]

I've got a decent-sized corpus of tweets, organized by hashtag, in a 
CouchDB db. I'm doing some initial explorations of my data, and was curious 
about which hashtags show up together in tweets. I want to do a NSA-style 
hops kind of algorithm--get all the hashtags that show up in the same 
tweets as hashtags that show up in the same tweets as hashtags that show up 
in the same tweets as my target hashtag, to an arbitrary depth. I wrote 
this:

(defn co-ocs [db ht  [s]] 
(reduce into (or s #{})
  (map #(map :text %)
 (map #(get-in % [:entities :hashtags])
  (:tweets (clutch/get-document db ht))

(defn co-occurrences [db ht depth]
(loop [tags (co-ocs db 5sos) i 1]
(if (= i depth) (recur
(reduce into tags
(map (partial co-ocs db) tags))
(inc i))
tags)))

It works, but loop + incrementing a counter seems profoundly un-clojuric. I 
suppose I could use `dotimes` + an atom, but that doesn't seem much better. 
Any suggestions?

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: more colloquial do x n times

[Sam Raker]

EDIT

On Wednesday, December 3, 2014 10:45:33 PM UTC-5, Sam Raker wrote:

 I've got a decent-sized corpus of tweets, organized by hashtag, in a 
 CouchDB db. I'm doing some initial explorations of my data, and was curious 
 about which hashtags show up together in tweets. I want to do a NSA-style 
 hops kind of algorithm--get all the hashtags that show up in the same 
 tweets as hashtags that show up in the same tweets as hashtags that show up 
 in the same tweets as my target hashtag, to an arbitrary depth. I wrote 
 this:

 (defn co-ocs [db ht  [s]] 
 (reduce into (or s #{})
   (map #(map :text %)
  (map #(get-in % [:entities 
 :hashtags])
   (:tweets (clutch/get-document db ht))

 (defn co-occurrences [db ht depth]
 (loop [tags (co-ocs db ht) i 1]
 (if (= i depth) (recur
 (reduce into tags
 (map (partial co-ocs db) tags))
 (inc i))
 tags)))

 It works, but loop + incrementing a counter seems profoundly un-clojuric. 
 I suppose I could use `dotimes` + an atom, but that doesn't seem much 
 better. Any suggestions?


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Clutch timeout problem

[Sam Raker]

I asked this in the Clutch group, before I realized the last time anyone 
else had posted there was last year...

I have some code that connects to a CouchDB server using Clutch 
(https://github.com/clojure-clutch/clutch). I recently changed the 
connection to use a non-local connection, i.e. 

(def db (clutch/get-database http://ip addres:port/db))

instead of

(def db (clutch/get-database db))


Since doing so, I've gotten the following error:
ConnectTimeoutException Connect to ip address:port timed out  org.apache
.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:
119)

 The CouchDB server is on my local home network, which isn't the best 
(local SSH connections get dropped, etc.) Is there anything I can do to fix 
my timeout problems? I'd really rather not have to wrap everything in 
try/catch blocks, if I can possibly avoid it.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

loop problems

[Sam Raker]

I'm using Twitter's HBC library to read from Twitter's public stream. HBC 
stores results in a LinkedBlockingQueue, from which you can then `.take` 
tweets and do stuff to them (in my case, doing some 
processing/normalization, then storing them in CouchDB). I've been 
struggling with how exactly best to do this, though. I tried `doseq`, but 
it stops when the queue is empty, which isn't what I want. Since my code is 
basically entirely IO, `map` and other lazy stuff causes me problems. Next, 
I reached for `loop`:

(defn process-stream-nores
   ([in-queue]
  (process-stream-nores in-queue identity))
   ([in-queue process-fn]
  (loop [res (.take in-queue)]
 (process-fn (parse-string res true))
 (recur (.take in-queue)


(where `in-queue` is a LinkedBlockingQueue). Unfortunately, this just 
hangs, even when the LinkedBlockingQueue isn't empty (I'd expect it to hang 
when the queue is empty, since the queue blocks until it gets something). 

I've also tried

...
   (while true (process-fn (parse-string (.take in-queue) true))
...

but that 1) also hangs, and 2) seems profoundly un-idiomatic.

I want to continuously take from the queue, process the tweet, and then put 
it in CouchDB. (I'm planning on putting this in a Thread that I can stop 
when I have enough tweets.) I feel like loop is the right way to go, but I 
don't understand it very well, and can't get it to work. Any help would be 
greatly appreciated.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: loop problems

[Sam Raker]

The plan is to run it in a thread, yeah. The process-fn I'm planning on 
running it with does some stuff to the tweet and then uploads the results 
to a local couchdb instance. I've been periodically checking 
/var/log/couchdb/couch.log to verify it's actually doing stuff.

I *think* this could benefit from some core.async magic, but I don't 
understand that lib yet, and haven't had the time to watch Rich's talk on 
it. Someday...

On Wednesday, November 12, 2014 5:34:14 PM UTC-5, Francis Avila wrote:

 Your loop pattern should work. (I've used this pattern before.)

 Just a sanity check: you *are* running this function in a different 
 thread, right? Because whatever thread calls this function *will* block 
 forever, whether the queue is empty or not. And unless you provide some 
 side-effecting process-fn there will be no evidence that the thread is 
 doing anything. (I.e. your first function will hang forever and appear to 
 do nothing.)


 On Wednesday, November 12, 2014 10:43:57 AM UTC-6, Sam Raker wrote:

 I'm using Twitter's HBC library to read from Twitter's public stream. HBC 
 stores results in a LinkedBlockingQueue, from which you can then `.take` 
 tweets and do stuff to them (in my case, doing some 
 processing/normalization, then storing them in CouchDB). I've been 
 struggling with how exactly best to do this, though. I tried `doseq`, but 
 it stops when the queue is empty, which isn't what I want. Since my code is 
 basically entirely IO, `map` and other lazy stuff causes me problems. Next, 
 I reached for `loop`:

 (defn process-stream-nores
([in-queue]
   (process-stream-nores in-queue identity))
([in-queue process-fn]
   (loop [res (.take in-queue)]
  (process-fn (parse-string res true))
  (recur (.take in-queue)


 (where `in-queue` is a LinkedBlockingQueue). Unfortunately, this just 
 hangs, even when the LinkedBlockingQueue isn't empty (I'd expect it to hang 
 when the queue is empty, since the queue blocks until it gets something). 

 I've also tried

 ...
(while true (process-fn (parse-string (.take in-queue) true))
 ...

 but that 1) also hangs, and 2) seems profoundly un-idiomatic.

 I want to continuously take from the queue, process the tweet, and then 
 put it in CouchDB. (I'm planning on putting this in a Thread that I can 
 stop when I have enough tweets.) I feel like loop is the right way to go, 
 but I don't understand it very well, and can't get it to work. Any help 
 would be greatly appreciated.



-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

io/writer + map/recur

[Sam Raker]

I'm writing some stuff to interact with the Twitter API. I want to be able 
to write tweets (as JSON) to a file, so I can, e.g., test things without 
connecting to the API. I've proxied the LinkedBlockingQueue that Twitter's 
HBC library uses to use an agent, so ideally I want to be able to write the 
contents of the agent AND the LBQ. Here's what I have right now:

(defmulti write-tweets (fn [q f] (class q)))
(defmethod write-tweets clojure.lang.Agent [a f]
  (with-open [w (clojure.java.io/writer f :append true)]
(.write w (apply str (interpose \n @a)
(defmethod write-tweets java.util.concurrent.LinkedBlockingQueue [lbq f]
(loop [res (.take lbq)]
  (if res
(recur 
  (with-open [w (clojure.java.io/writer f :append true)]
(.write w (str (generate-string res) \n)))

My first implementation of this used `(map #(.write w %) @a)` and had the 
`recur` within the `with-open` block. Unfortunately, at least with the 
agent part, I ran into an error about the file being closed when I tried to 
write to it. I assumed `with-open` kept the file open within the block, but 
maybe I'm missing something? I'm worried about the performance of either 
creating a potentially super-huge string in memory for the agent method 
(twitter returns pretty sizable JSON blobs) or repeatedly opening/closing a 
file for the LBQ method (I realize I could collapse this into one problem 
by taking everything out of the LBQ and putting it into an agent, but 
that's not really a solution...)

Does `writer` auto-close the file after it's done? Is there some better way 
of handling this kind of situation? 

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: CCW bug [SEVERE]

[Sam Raker]

I version everything. I don't version something because it's huge, or 
important, or shared, or even worth sharing. I version because Git is an 
important and widely-used tool, and because it's better than regular 
saving. You've already downloaded and learned at least one programming 
language, you've already downloaded and learned how to use at least one 
IDE. One more tool is not a crippling burden. 

On Tuesday, October 28, 2014 6:21:11 PM UTC-4, Fluid Dynamics wrote:

 On Tuesday, October 28, 2014 5:22:03 PM UTC-4, Dylan Butman wrote:

 This still sounds like a case of I haven't learned how to use git yet, 
 and although that means I can't possibly understand how it might improve my 
 workflow, I don't want to take the time to learn a fundamental piece of 
 software that's used by the vast majority of the programming community. 


 I have a full understanding of the benefits of version control, including 
 that they don't scale down to a project this small, at least not when 
 overhead from additional ceremony around commonplace actions like file save 
 is considered.
  


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: filter but for non-sequences

[Sam Raker]

I think the point is to get nil if x does not satisfy some predicate. -- 
exactly.

I basically want something like `if-let`, but which returns x instead of 
f(x). `(if-let [m-with-k (contains? m k)]...` results in `m-with-k` 
evaluating to `true`, not `m`. I *think* it's better to reject something 
that doesn't meet your criteria early, as opposed to guarding against not 
accidentally doing something like `(.toUpper {:foo m})` and getting an 
error every step of the way.

But like I said, I'm totally willing to believe there's a more 
idiomatic/appropriate way of doing things.

On Friday, October 24, 2014 4:59:33 AM UTC-4, Gary Verhaegen wrote:

 I think the point is to get nil if x does not satisfy some predicate.

 The best option would depend on the context in which these expressions are 
 used, but one option would be to write your predicates to return the input 
 as the truthy value (and nil otherwise).

 On Friday, 24 October 2014, Laurens Van Houtven _...@lvh.io wrote:

 Hi Sam,


 I’m not sure I understand. If not for sequences, then what for? `(if 
 (pred x) x)` looks like it’s missing an else-clause; so I don’t know what 
 happens when (not (pred x)).

 From the rest of your e-mail it sounds a little bit like you might want 
 (get m x x)? (i.e. get me x if x isn’t in m; otherwise give me the value 
 mapped to x in m).


 hth
 lvh


 On 24 Oct 2014, at 06:06, Sam Raker sam.ra...@gmail.com wrote:

  Is there anything simpler/more idiomatic than `(if (pred x) x)`? What I 
 really want is something better than `(if (= (get a-map a-key) a-val) 
 a-map)`/`(if (= (get-in a-map some-keys) a-val) a-map)`, but that might be 
 too specific to have been 'cached' somewhere.
 
  --
  You received this message because you are subscribed to the Google
  Groups Clojure group.
  To post to this group, send email to clojure@googlegroups.com
  Note that posts from new members are moderated - please be patient with 
 your first post.
  To unsubscribe from this group, send email to
  clojure+unsubscr...@googlegroups.com
  For more options, visit this group at
  http://groups.google.com/group/clojure?hl=en
  ---
  You received this message because you are subscribed to the Google 
 Groups Clojure group.
  To unsubscribe from this group and stop receiving emails from it, send 
 an email to clojure+unsubscr...@googlegroups.com.
  For more options, visit https://groups.google.com/d/optout.

 

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

reorganizing/naming args with partial

[Sam Raker]

From what I can tell, `partial` goes 'in order'. As such, you can't pass it 
a fn that takes  1 args and the 2nd...nth args. In Python, at least, you 
can explicitly name positional arguments, e.g. 

def my_func(x, y, z):
return x + y * z

p

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

reorganizing/naming args with partial

[Sam Raker]

(Accidentally hit 'enter' prematurely :( )
From what I can tell, `partial` goes 'in order'. As such, you can't pass it 
a fn that takes  1 args and the 2nd...nth args. In Python, at least, you 
can explicitly name positional arguments, e.g. 

def my_func(x, y, z):
return x + y * z

p = partial(my_func, 1, 2)

p2 partial(my_func, y=3, z=4)

Is there a similar thing in Clojure? Or is the answer suck it up and use 
anonymous functions, you baby?

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

filter but for non-sequences

[Sam Raker]

Is there anything simpler/more idiomatic than `(if (pred x) x)`? What I 
really want is something better than `(if (= (get a-map a-key) a-val) 
a-map)`/`(if (= (get-in a-map some-keys) a-val) a-map)`, but that might be 
too specific to have been 'cached' somewhere.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] Clojure instaREPL for web

[Sam Raker]

This is great. A little buggy, but that's to be expected :)
I really appreciate the ability to hop onilne and test the behavior of 
little bits of code without waiting for a local repl to start up!

On Tuesday, October 7, 2014 3:00:08 PM UTC-4, Lauri Hartikka wrote:

 A web based clojure instaREPL,

 Check it out :)

 http://clojurerepl.com




-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Interop nightmare

[Sam Raker]

Thanks for all the help! I knew I could count on you guys.

I saw that there were a bunch of params in the constructor, but naively 
hoped there'd be some kind of default values for them so I didn't have to 
muck around with anything too much. Disappointed once again. I'll look into 
exactly what else I have to move around/import/instantiate before I can get 
the parser to just work.

As for the comments about Clojure being difficult to use w/o knowing Java, 
aside from this unfortunate experience, I've found it pretty easy/not an 
issue. The automatic upgrading of integers to the appropriate underlying 
Java (/JVM) types, and the wrappers around Java's regex stuff are two good 
examples. I'd imagine it'd be much more difficult coming in cold, although 
TJOC and the other intro to Clojure book I read at least covered numerics 
and over/underflow pretty thoroughly. The other stuff (typed arrays, e.g.) 
keep out of your way unless you actively seek them out, so it's not a big 
deal either.

That being said, while Just enough Java for Clojure, as Ivan and Michael 
discussed, might not be big enough on its own for a book/article, I, for 
one, would appreciate more on interop than I've seen, which tends to be 
along the lines of use ClassName/staticThing for static things, use . for 
everything else, also .. and doto exist.

On Monday, September 8, 2014 4:49:12 PM UTC-4, Michael Klishin wrote:

  On 9 September 2014 at 00:33:11, Ivan L (ivan.l...@gmail.com 
 javascript:) wrote: 
  For an enterprising clojure hacker, this is a good opportunity   
  to write Clojure for non-Java Hackers and put it up on Pragprog.   

 Sounds more like Just enough Java for Clojure. Which I think would have 
 too small an audience to be worth the effort. 
 --   
 @michaelklishin, github.com/michaelklishin 


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Interop nightmare

[Sam Raker]

I'm trying to use the Stanford Parser from Clojure, but I don't know hardly 
any Java, and this is my first time working with the interop stuff. All I 
want to do is play around with the class in the REPL. I added 
`[edu.stanford.nlp/stanford-parser 3.4.1]` to my Lein `project.clj`, and 
the download seemed to go fine. The documentation 
is 
http://nlp.stanford.edu/nlp/javadoc/javanlp/edu/stanford/nlp/parser/lexparser/LexicalizedParser.html
 
for those of you playing at home.

Basically, my efforts have been a total failure. I can `(import 
edu.stanford.nlp.parser.lexparser.LexicalizedParser)`, but after that, it's 
just a nightmare of `no matching ctor`, `no matching field`, 
`NoSuchFieldException` and `expected static field` errors. I can't even 
initialize anything -- `(def parser (new LexicalizedParser))` gives me the 
aforementioned `no matching ctor` error.

Like I said before, this is entirely my fault: I don't know Java, I don't 
know interop, and I Google has failed me. So I turn to you, beloved Clojure 
community, to correct my ignorance. I'm sure it's not hard, I'm just 
missing something.


Thanks,
-sam

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Puzzling list comp error: Don't know how to create ISeq from: clojure.lang.Keyword

[Sam Raker]

`-` inserts its first argument into the second position of the next 
argument, and so on, so
(- []
 (conj 1)
 (conj 2))

Turns into
(conj (conj [] 1) 2)

`-` inserts its first argument into the LAST position of the next 
argument, and so on, so
(- 1
 (conj [2])
 (conj [3]))
 

Turns into 
(conj [3] (conj [2] 1))

I guess it is bottom to top, in a way, but I always thought that was kind 
of an artifact of the argument (re-)ordering.



On Saturday, August 30, 2014 12:19:31 PM UTC-4, g vim wrote:

 On 30/08/2014 17:04, Alexey Kachayev wrote: 
  Thread-first macro -will insert list-of-listsas first argument for map, 
  which is definitely not what you expect. Use threading-last -instead. 
  

 I've never quite understood the distinction other than - does 
 everything top to bottom and - does the reverse. From what you're 
 saying the choice also affects which position the argument is inserted? 
 If so this presents a complication in that threading through several 
 functions may require the argument to be inserted in different positions? 

 gvim 


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Resources for intermediate/not-absolute-beginner Clojurians

[Sam Raker]

I worked my way through *Clojure Programming* (Emerick, Carper,  Grand, 
O'Reilly), and I've started writing my own Clojure (porting over an 
unfinished Python project that seemed amenable to the Clojure treatment.) I 
really love the language, but I'm not sure where to go from here. 

My other main language is Python, which I learned in school, and also found 
a bunch of intermediate/non-introductory resources for, like the awesome, 
short, topic-oriented monographs (for lack of a better term) by Matt 
Harrison 
(e.g., 
http://www.amazon.com/Guide-Learning-Iteration-Generators-Python-ebook/dp/B007JR4FCQ/ref=sr_1_3).
 
These really helped me understand some of the less-obvious/less-intro parts 
of Python, and the stuff I learned in school helped me learn what idiomatic 
Python looked/felt like. 

I'm just not sure what to do at this point in my Clojure learning 
experience. I've probably written a few thousand lines of Clojure at this 
point, but I'm not sure that I'm doing things right: I don't know if my 
code is efficient, or even idiomatic. I've know next to nothing about Java, 
and Clojure is my first introduction to functional programming. There are 
so many fun, exciting, awesome-seeming things in Clojure that I want to 
take advantage of, like reference types and futures, but I have no point of 
reference for them and feel like I'm having trouble wrapping my head around 
them.

I've come to realize that I learn best from books, and while code literacy 
is something I need to work on, read the sourcecode [for library X] isn't 
going to help me that much, unless it's aggressively commented/documented. 
I don't really want another intro book, since I'd rather not pay for too 
much overlap, and while I'll happily accept recommendations for 
application-/domain-specific books, I'm more looking for a deeper dive into 
the language itself. 

I'm being really difficult about this, and I'm sorry in advance. Any and 
all suggestions are welcome. Thanks guys!

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

positional + keyword args

[Sam Raker]

I'm trying to write a function that takes (up to) 4 arguments. I want to be 
able to supply every argument positionally, with a keyword or as a default, 
so that

(f)
(f 1)
(f 1 2)
(f 1 2 3)
(f 1 2 3 4)
(f 1 :b 2)
(f 1 2 :c 3)
...
(f :a 1 :b 2 :c 3 :d 4)

are all equivalent. In Python, I could do this by

def f(a=1,b=2,c=3,d=4):...

but I'm not sure how to do it in Clojure.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: positional + keyword args

[Sam Raker]

I hacked this together:

(defn f [ args] (let [default-args-atom (atom {:a 1 :b 2 :c 3 :d 4}) kwds 
(vec (keys @default-args-atom)) 
 kwargs (map-indexed (fn [idx arg] (if 
(keyword? arg) (swap! default-args-atom assoc arg (get args (inc idx))) 
(swap! a assoc (get kwds idx) arg))) args)]
(function-that-actually-does-what-i-want-to-do (:a @default-args-atom) 
(:b @default-args-atom) (:c @default-args-atom) (:d @default-args-atom)))

But it seems kind of gross to me. Is there a cleaner solution?

On Sunday, July 20, 2014 8:16:55 PM UTC-4, Alex Baranosky wrote:

 You can get keyword args like this:

 (defn f [ {:keys [a b c d]
 :or {a 1 b 2 c 3 d 4}}]
   [a b c d])

 But you cannot also get the ability to call:
 (f 5 6 7 8)


 On Sun, Jul 20, 2014 at 4:13 PM, Sam Raker sam@gmail.com 
 javascript: wrote:

 I'm trying to write a function that takes (up to) 4 arguments. I want to 
 be able to supply every argument positionally, with a keyword or as a 
 default, so that

 (f)
 (f 1)
 (f 1 2)
 (f 1 2 3)
 (f 1 2 3 4)
 (f 1 :b 2)
 (f 1 2 :c 3)
 ...
 (f :a 1 :b 2 :c 3 :d 4)

 are all equivalent. In Python, I could do this by

 def f(a=1,b=2,c=3,d=4):...

 but I'm not sure how to do it in Clojure.

 -- 
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com 
 javascript:
 Note that posts from new members are moderated - please be patient with 
 your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com javascript:
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 --- 
 You received this message because you are subscribed to the Google Groups 
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to clojure+u...@googlegroups.com javascript:.
 For more options, visit https://groups.google.com/d/optout.




-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: positional + keyword args

[Sam Raker]

Wait. Should be:

(defn f [ args] (let [default-args-atom (atom {:a 1 :b 2 :c 3 :d 4}) kwds 
(vec [:a :b :c :d) 
 kwargs (map-indexed (fn [idx arg] (if 
(keyword? arg) (swap! default-args-atom assoc arg (get args (inc idx))) 
(swap! a assoc (get kwds idx) arg))) args)]
(function-that-actually-does-what-i-want-to-do (:a @default-args-atom) 
(:b @default-args-atom) (:c @default-args-atom) (:d @default-args-atom)))

because there's no guarantee that (vec (keys @default-args-atom)) will 
return the keys in the same order.

On Sunday, July 20, 2014 8:45:02 PM UTC-4, Sam Raker wrote:

 I hacked this together:

 (defn f [ args] (let [default-args-atom (atom {:a 1 :b 2 :c 3 :d 4}) kwds 
 (vec (keys @default-args-atom)) 
  kwargs (map-indexed (fn [idx arg] (if 
 (keyword? arg) (swap! default-args-atom assoc arg (get args (inc idx))) 
 (swap! a assoc (get kwds idx) arg))) args)]
 (function-that-actually-does-what-i-want-to-do (:a @default-args-atom) 
 (:b @default-args-atom) (:c @default-args-atom) (:d @default-args-atom)))

 But it seems kind of gross to me. Is there a cleaner solution?

 On Sunday, July 20, 2014 8:16:55 PM UTC-4, Alex Baranosky wrote:

 You can get keyword args like this:

 (defn f [ {:keys [a b c d]
 :or {a 1 b 2 c 3 d 4}}]
   [a b c d])

 But you cannot also get the ability to call:
 (f 5 6 7 8)


 On Sun, Jul 20, 2014 at 4:13 PM, Sam Raker sam@gmail.com wrote:

 I'm trying to write a function that takes (up to) 4 arguments. I want to 
 be able to supply every argument positionally, with a keyword or as a 
 default, so that

 (f)
 (f 1)
 (f 1 2)
 (f 1 2 3)
 (f 1 2 3 4)
 (f 1 :b 2)
 (f 1 2 :c 3)
 ...
 (f :a 1 :b 2 :c 3 :d 4)

 are all equivalent. In Python, I could do this by

 def f(a=1,b=2,c=3,d=4):...

 but I'm not sure how to do it in Clojure.

 -- 
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com
 Note that posts from new members are moderated - please be patient with 
 your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 --- 
 You received this message because you are subscribed to the Google 
 Groups Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to clojure+u...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.




-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: positional + keyword args

[Sam Raker]

Wait. Should be: 

(defn f [ args] (let [default-args-atom (atom {:a 1 :b 2 :c 3 :d 4}) kwds 
[:a :b :c :d]
 kwargs (map-indexed (fn [idx arg] (if 
(keyword? arg) (swap! default-args-atom assoc arg (get args (inc idx))) 
(swap! a assoc (get kwds idx) arg))) args)]
(function-that-actually-does-what-i-want-to-do (:a @default-args-atom) 
(:b @default-args-atom) (:c @default-args-atom) (:d @default-args-atom)))

because there's no guarantee (vec (keys @default-args-atom)) will return 
the keys in the same order every time.

On Sunday, July 20, 2014 8:45:02 PM UTC-4, Sam Raker wrote:

 I hacked this together:

 (defn f [ args] (let [default-args-atom (atom {:a 1 :b 2 :c 3 :d 4}) kwds 
 (vec (keys @default-args-atom)) 
  kwargs (map-indexed (fn [idx arg] (if 
 (keyword? arg) (swap! default-args-atom assoc arg (get args (inc idx))) 
 (swap! a assoc (get kwds idx) arg))) args)]
 (function-that-actually-does-what-i-want-to-do (:a @default-args-atom) 
 (:b @default-args-atom) (:c @default-args-atom) (:d @default-args-atom)))

 But it seems kind of gross to me. Is there a cleaner solution?

 On Sunday, July 20, 2014 8:16:55 PM UTC-4, Alex Baranosky wrote:

 You can get keyword args like this:

 (defn f [ {:keys [a b c d]
 :or {a 1 b 2 c 3 d 4}}]
   [a b c d])

 But you cannot also get the ability to call:
 (f 5 6 7 8)


 On Sun, Jul 20, 2014 at 4:13 PM, Sam Raker sam@gmail.com wrote:

 I'm trying to write a function that takes (up to) 4 arguments. I want to 
 be able to supply every argument positionally, with a keyword or as a 
 default, so that

 (f)
 (f 1)
 (f 1 2)
 (f 1 2 3)
 (f 1 2 3 4)
 (f 1 :b 2)
 (f 1 2 :c 3)
 ...
 (f :a 1 :b 2 :c 3 :d 4)

 are all equivalent. In Python, I could do this by

 def f(a=1,b=2,c=3,d=4):...

 but I'm not sure how to do it in Clojure.

 -- 
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com
 Note that posts from new members are moderated - please be patient with 
 your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 --- 
 You received this message because you are subscribed to the Google 
 Groups Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to clojure+u...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.




-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: positional + keyword args

[Sam Raker]

Nope, that doesn't actually work...

On Sunday, July 20, 2014 8:57:47 PM UTC-4, Sam Raker wrote:

 Wait. Should be: 

 (defn f [ args] (let [default-args-atom (atom {:a 1 :b 2 :c 3 :d 4}) kwds 
 [:a :b :c :d]
  kwargs (map-indexed (fn [idx arg] (if 
 (keyword? arg) (swap! default-args-atom assoc arg (get args (inc idx))) 
 (swap! a assoc (get kwds idx) arg))) args)]
 (function-that-actually-does-what-i-want-to-do (:a @default-args-atom) 
 (:b @default-args-atom) (:c @default-args-atom) (:d @default-args-atom)))

 because there's no guarantee (vec (keys @default-args-atom)) will return 
 the keys in the same order every time.

 On Sunday, July 20, 2014 8:45:02 PM UTC-4, Sam Raker wrote:

 I hacked this together:

 (defn f [ args] (let [default-args-atom (atom {:a 1 :b 2 :c 3 :d 4}) 
 kwds (vec (keys @default-args-atom)) 
  kwargs (map-indexed (fn [idx arg] (if 
 (keyword? arg) (swap! default-args-atom assoc arg (get args (inc idx))) 
 (swap! a assoc (get kwds idx) arg))) args)]
 (function-that-actually-does-what-i-want-to-do (:a 
 @default-args-atom) (:b @default-args-atom) (:c @default-args-atom) (:d 
 @default-args-atom)))

 But it seems kind of gross to me. Is there a cleaner solution?

 On Sunday, July 20, 2014 8:16:55 PM UTC-4, Alex Baranosky wrote:

 You can get keyword args like this:

 (defn f [ {:keys [a b c d]
 :or {a 1 b 2 c 3 d 4}}]
   [a b c d])

 But you cannot also get the ability to call:
 (f 5 6 7 8)


 On Sun, Jul 20, 2014 at 4:13 PM, Sam Raker sam@gmail.com wrote:

 I'm trying to write a function that takes (up to) 4 arguments. I want 
 to be able to supply every argument positionally, with a keyword or as a 
 default, so that

 (f)
 (f 1)
 (f 1 2)
 (f 1 2 3)
 (f 1 2 3 4)
 (f 1 :b 2)
 (f 1 2 :c 3)
 ...
 (f :a 1 :b 2 :c 3 :d 4)

 are all equivalent. In Python, I could do this by

 def f(a=1,b=2,c=3,d=4):...

 but I'm not sure how to do it in Clojure.

 -- 
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com
 Note that posts from new members are moderated - please be patient with 
 your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 --- 
 You received this message because you are subscribed to the Google 
 Groups Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to clojure+u...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.




-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: positional + keyword args

[Sam Raker]

OK! After a few false starts, I think I got it:

(defn f [ args] (let [args (vec args) default-args-atom (atom {:a 1 :b 2 
:c 3 :d 4}) kwds [:a :b :c :d]]
(do
  (dorun (map-indexed (fn [idx arg] (if 
(keyword? arg) 

(swap! default-args-atom assoc arg (get args (inc idx))) 

(if (not (keyword? (get args (dec idx (swap! default-args-atom 
assoc (get kwds idx) arg 
  args))
  (the-real-func (:a @default-args-atom) (:b 
@default-args-atom) (:c @default-args-atom) (:d @default-args-atom)

I feel like I should turn this into a macro, but I've spent enough time on 
it as is, for now.

On Sunday, July 20, 2014 8:16:55 PM UTC-4, Alex Baranosky wrote:

 You can get keyword args like this:

 (defn f [ {:keys [a b c d]
 :or {a 1 b 2 c 3 d 4}}]
   [a b c d])

 But you cannot also get the ability to call:
 (f 5 6 7 8)


 On Sun, Jul 20, 2014 at 4:13 PM, Sam Raker sam@gmail.com 
 javascript: wrote:

 I'm trying to write a function that takes (up to) 4 arguments. I want to 
 be able to supply every argument positionally, with a keyword or as a 
 default, so that

 (f)
 (f 1)
 (f 1 2)
 (f 1 2 3)
 (f 1 2 3 4)
 (f 1 :b 2)
 (f 1 2 :c 3)
 ...
 (f :a 1 :b 2 :c 3 :d 4)

 are all equivalent. In Python, I could do this by

 def f(a=1,b=2,c=3,d=4):...

 but I'm not sure how to do it in Clojure.

 -- 
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com 
 javascript:
 Note that posts from new members are moderated - please be patient with 
 your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com javascript:
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 --- 
 You received this message because you are subscribed to the Google Groups 
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to clojure+u...@googlegroups.com javascript:.
 For more options, visit https://groups.google.com/d/optout.




-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Protocol equivalent of Python's `repr`?

[Sam Raker]

I'm working my way through an intro to Clojure book, and I just got to the 
section on protocols. I might be missing something/committing a category 
error, but I'm wondering if there's a way to change the way a protocol is 
represented in the repl, akin to redefining `__repr__`/`__str__` in Python. 
The example protocol in the book involves a matrix (i.e., a vector of 
vectors). I can do this

(extend-protocol Matrix
clojure.lang.IPersistentVector
...
(pprint [vov] (clojure.pprint/pprint vov)))


which works fine when I call (pprint m) on a Matrix m:

(pprint m)
; [[0 0 0]
   [0 0 0]
   [0 0 0]]

What I'd like is to just have the same behavior calling the variable from 
the repl, i.e.

m
; [[0 0 0]
   [0 0 0]
   [0 0 0]]


Can/should this be done? Is this a job for macros? Am I just being 
lazy/un-Clojuric and just get over it and use (pprint m)?

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Protocol equivalent of Python's `repr`?

[Sam Raker]

James: I haven't gotten to multimethods yet--I'll definitely check that out!

Jony: I use Gorilla REPL for basically all my clojure stuff already! It's
really great, thanks so much. I know you can add stuff to project.clj to
pretty-print everything in the repl, I was just wondering about targeting
specific protocols and types.



On Sun, Jul 13, 2014 at 2:29 PM, Jony Hudson jonyepsi...@gmail.com wrote:

 You might also enjoy looking at Gorilla REPL [1], which has an easily
 extensible renderer [2] for just this sort of thing. Disclosure: I'm one of
 its authors!


 Jony

 [1] http://gorilla-repl.org
 [2] http://vimeo.com/89532785


 On Sunday, 13 July 2014 18:51:40 UTC+1, Sam Raker wrote:

 I'm working my way through an intro to Clojure book, and I just got to
 the section on protocols. I might be missing something/committing a
 category error, but I'm wondering if there's a way to change the way a
 protocol is represented in the repl, akin to redefining
 `__repr__`/`__str__` in Python. The example protocol in the book involves a
 matrix (i.e., a vector of vectors). I can do this

 (extend-protocol Matrix
 clojure.lang.IPersistentVector
 ...
 (pprint [vov] (clojure.pprint/pprint vov)))


 which works fine when I call (pprint m) on a Matrix m:

 (pprint m)
 ; [[0 0 0]
[0 0 0]
[0 0 0]]

 What I'd like is to just have the same behavior calling the variable
 from the repl, i.e.

 m
 ; [[0 0 0]
[0 0 0]
[0 0 0]]


 Can/should this be done? Is this a job for macros? Am I just being
 lazy/un-Clojuric and just get over it and use (pprint m)?

  --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to a topic in the
 Google Groups Clojure group.
 To unsubscribe from this topic, visit
 https://groups.google.com/d/topic/clojure/ClQ4LwjyxNo/unsubscribe.
 To unsubscribe from this group and all its topics, send an email to
 clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

IO/memory bottlenecks

[Sam Raker]

I'm super new to Clojure, so apologies if I've missed something.

I'm trying to use an agent to manage IO for code that involves reading a 
bunch of JSON files, doing some processing, then writing the contents to 
separate files. I'm using pmap to speed things up, but I've run into some 
memory overrun issues and other hassles. So now I'm trying to use an agent 
to manage the IO, but I'm not quite sure how to do it. I'm trying to use a 
watch function, but I'm not sure that's right: 

...

(def str-agent (agent {}))

(defn watch-write [k i o n] (let [in-file (:f n) out-str (:s n)]
  (do (println in-file)
(spit (get-fixed-name in-file) out-str

(defn update-file [in-file] (let [lines (parse-file in-file)]
  (doall (map update-both lines


(defn process-file [in-file] (do (println (str in-file))
  (send-off str-agent assoc
:f in-file
:s (string/join \n (update-file 
in-file)

(defn process-files [] (dorun (pmap process-file valid-files)))

(defn -main [] (do  (println Hello, world!)
(add-watch str-agent :writer watch-write)
(process-files)))


Is this the way to handle this kind of IO issue, or should I create an atom 
for each process, or what?


Thanks guys!

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.