I'm using Twitter's HBC library to read from Twitter's public stream. HBC
stores results in a LinkedBlockingQueue, from which you can then `.take`
tweets and do stuff to them (in my case, doing some
processing/normalization, then storing them in CouchDB). I've been
struggling with how exactly best to do this, though. I tried `doseq`, but
it stops when the queue is empty, which isn't what I want. Since my code is
basically entirely IO, `map` and other lazy stuff causes me problems. Next,
I reached for `loop`:
(defn process-stream-nores
([in-queue]
(process-stream-nores in-queue identity))
([in-queue process-fn]
(loop [res (.take in-queue)]
(process-fn (parse-string res true))
(recur (.take in-queue)))))
(where `in-queue` is a LinkedBlockingQueue). Unfortunately, this just
hangs, even when the LinkedBlockingQueue isn't empty (I'd expect it to hang
when the queue is empty, since the queue blocks until it gets something).
I've also tried
...
(while true (process-fn (parse-string (.take in-queue) true))
...
but that 1) also hangs, and 2) seems profoundly un-idiomatic.
I want to continuously take from the queue, process the tweet, and then put
it in CouchDB. (I'm planning on putting this in a Thread that I can stop
when I have enough tweets.) I feel like loop is the right way to go, but I
don't understand it very well, and can't get it to work. Any help would be
greatly appreciated.
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
