Re: Relational mapping - turning an sql query result into array of hashes (with joins)

[Sam Roberton]

On Tuesday, 26 May 2015 06:43:18 UTC+10, Krzysiek Herod wrote:

 Sean, maybe my use case is specific to web applications. In REST API's 
 it's common to respond with something like Example Result here: 
 https://dev.twitter.com/rest/reference/get/statuses/mentions_timeline I 
 completely agree with you about ORMs, but in case of REST API's one just 
 needs to somehow map the queryset to a structure of hashes and arrays (not 
 necessarily objects), and this task is much heavier than I thought in the 
 beginning, from choosing between making many queries to fill in the hash of 
 related objects vs making JOINs, to complexity of extracting data from a 
 set of rows with potential repetitions, rows of null values (like Daniel 
 said happens for instance in case of LEFT JOINS), and different kinds of 
 relations (has-one, has-many, belongs-to). 


I found myself needing this for a REST API as well, so I implemented an 
'unjoin' function which sounds like what you're looking for.

Here's a gist, including test case (not necessarily a hugely comprehensive 
test case, but better than nothing): 
 https://gist.github.com/samroberton/d72cedaf225526d9007a

It's working for me for the few use cases I have for it -- I'd be 
interested to hear whether it works for you too.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Help with timestamp with timezone in YeSQL

[Sam Roberton]

On Sunday, 17 May 2015 01:46:14 UTC+10, g vim wrote:

 snip/

   (c/to-timestamp 1967-07-31 06:30:00 America/Caracas) 

  evaluates to nil. However: 

 (c/to-timestamp 1967-07-31 06:30:00) 

  gives me an:  #inst 1967-07-31T06:30:00.0-00:00 , 
 whatever that is, so I checked the clj-time docs and it appears 
 to-timestamp doesn't handle timezones. 

 Any ideas? 


Looks like you might want this:

user (clj-time.format/parse (clj-time.format/formatter -MM-dd 
HH:mm:ss ZZZ) 1967-07-31 06:30:00 America/Caracas)
#object[org.joda.time.DateTime 0x79e05a0 1967-07-31T10:30:00.000Z]

I suspect you'll then need some further work to make sure that the Joda 
DateTime object correctly converts into what you actually want stored in 
the database when it's set on the SQL PreparedStatement (something like this 
http://tapestryjava.blogspot.ie/2014/09/postgres-jdbc-time-zones.html, 
but extending the type to Joda's DateTime rather than java.util.Date) -- or 
alternatively you might be OK just coercing the above to a java.sql.Date or 
java.sql.Timestamp.
 

 gvim 


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: code-rewriting with-connection macro (monger and maybe beyond?)

[Sam Roberton]

I use partial for a sort-of-similar scenario.  It gets you a reasonable way 
there without adding any real complexity.

(defn do-stuff [db coll m]
  (let [update (partial update db coll)
insert (partial insert db coll)]
(insert m)
(update m)
...))

You could wrap it with a macro if you wanted, but you may find it's not 
worth it.

Cheers,
Sam.



On Monday, 13 April 2015 02:07:26 UTC+10, Sam Raker wrote:

 I'm working with Monger[1], and have been thinking about connection 
 encapsulation. I've currently got a `myproject.mongo` ns with private `db` 
 and `coll` vars. I've got a public `insert` method along the lines of 

 (defn insert [m] (mc/insert db coll m))

 where `mc/insert` is, perhaps unsurprisingly, monger.collection/insert. To 
 properly encapsulate my db-related vars, my options seem to be:

 1) Replicate that `insert` func for every potentially-relevant mongo 
 action (too much boilerplate/not DRY, I'd have to go back and add new funcs 
 if I wanted more functionality)

 2) Write a function along the lines of

 (defn with-connection [f  args] (apply f (concat [db coll] args))

 which is fine I GUESS but is only func-level

 3) Write a macro that inserts `db` and `coll` in the proper places, so 
 that one could do something like
 (defn fake-upsert [m]
   (with-conn 
 (if-let [existing (mc/find-one-as-map {:_id (:_id m)})]
   (mc/update existing m)
   (mc/insert m

 and then `with-conn` would search through the code, identify the 
 `monger.collection`-namespaced vars, and rewrite the code to put in `db` 
 and `coll` arguments so it'd end up looking like

 ...
 (if-let [existing (mc/find-one-as-map db coll {:_id (:_id m})]...
(mc/update db coll existing m)..

 and so on. This seems like my cleanest option, especially because it could 
 be extended with an options map, à la

 (defn mongo-to-another-db [m]
(with-conn {monger.collection [monger-db monger-coll]
another-db.adapter [another-db-uri another-db-username 
 another-db-password]}
   (adapter/insert (mc/find-one-as-map {:foo bar}

 I haven't been able to figure out how quite to do this, yet. I need, 
 basically, a way to iterate through tokens in source code, examine each 
 token, and then add a token and its relevant args if it's in the options 
 map, or the token alone if it's not. 

 Is this reasonable? Does this or something like it already exist? Is there 
 something obvious my general macro mediocrity is preventing me from seeing?


 Thanks!
 -sam


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Enhanced Primitive Support Syntax

[Sam Roberton]

On 16 January 2011 05:35, Jason Wolfe ja...@w01fe.com wrote:
 (a) unsafe/incorrect value on overflow/fastest/unifiable* vs.
 (b) safe/error on overflow/fast/unifiable vs.
 (c) safe/promoting on overflow/slow/not-unifiable

 If I understand correctly, the issue with auto-promotion is that we
 have to box the output of an operation even if it turns out to fit in
 a long, since the compiler must specify whether the result will be
 long or Object.

 This is probably a stupid question, but since Clojure is already
 testing for overflow in (b) to throw an exception, would it be
 possible to jump into an alternative (c)-type compilation of the
 function just-in-time to promote on overflow instead?  It seems like
 this could achieve the performance of (b) while still allowing for
 auto-promotion (albeit perhaps with a performance hit in that case,
 code bloat, compromises about how many versions to compile, etc.).

When Clojure compiles your function, it emits JVM bytecode for a new
class which is then loaded by the classloader.  That JVM bytecode
defines a function (well, a method as far as the JVM is concerned)
which returns either a primitive type or Object.  Your suggestion
would involve redifining the class while it is executing.  That's not
possible on the JVM.  Even if it were possible -- your function now
returns Object instead of long.  But the variable that the result of
your function is about to be assigned to is a long, because that's
what your function used to be defined to return, and the next bytecode
operation in the calling code is the one that subtracts a primitive
long from a primitive long.  Now what?

Fundamentally, Clojure has to contend with the fact that the JVM as a
platform distinguishes between primitives and Objects.  The bytecode
operations which the Clojure compiler emits have to differ based on
that distinction.  Essentially, the distinction cannot be magicked
away, because JVM bytecode is going to statically enforce that we're
either working with a (long or double) or a (Long or BigInt or
BigDecimal or whatever), and never the twain shall meet.  So if we
ever want to be able to access the speed of the primitive bytecode
operations, then the primitive/Object distinction has to leak into
Clojure.  (Or we have to redefine or move away from the JVM.
Obviously not really an option, but I mention it to point out that the
reason Clojure has to make this decision is that it's a hosted
language.  That has a lot of benefits;  this is one of the trade-offs
we have to contend with in return.)

I think everyone agrees that it's important to make the speed of the
primitive bytecode operations available in Clojure (whether or not
it's the default), so that rules out the option of always doing
auto-promotion.  I think it's probably also agreed that allowing
unchecked overflow is not good (at least, no one seems to be arguing
for it).

So we're left with option (b) and a choice about the default behaviour
of the core library functions.  If we default to boxing and treating
everything as an Object then we get the nice comfy numeric tower that
we never have to worry about, but the default case suffers in
performance.  Otherwise, we default to primitive, and accept that if
we're dealing with numbers which might get bigger than Long.MAX_VALUE,
then we might need to explicitly use a BigInt to get contagion, or use
an operator like +' which will always deal with Objects.

By choosing to make speed the default preference in the core library
functions, I suppose there's more for Clojure programmers to think
about, because whenever you're dealing with numbers, you need to have
in the back of your mind the question of whether this might ever need
to be bigger than Long.MAX_VALUE, and so whether you might need +'
instead of +.  Then again, how often do you write code that might be
doing maths with numbers that big and not realise it?  For that
matter, how often do you write code that might be doing maths with
numbers that big and not spend time thinking carefully about its
performance anyway?

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

Re: error with doseq in defrecord function

[Sam Roberton]

 (defrecord Foo [properties]
   Fooable
   (foo [f]
     (let [s (java.util.HashSet.)]
      (doto s
    (doseq [p properties] (.add s p))

 When I load/reload the above code in the repl I get this error

 CompilerException java.lang.IllegalArgumentException: doseq requires a
 vector for its binding (config.clj:22)

The problem is the way you're calling doto, not the defrecord.  If you
macroexpand the doto, you'll get something like:

(clojure.core/let [G__1646 s] (doseq G__1646 [p properties] (.add s p)) G__1646)

doto puts your 's' as the first argument in all the forms.  So the
first argument to doseq is no longer your binding vector, and so doseq
doesn't like it.

 If I change my defrecord to,

 (defrecord Foo [properties]
   Fooable
   (foo [f]
     (let [s (java.util.HashSet.)]
      (doto s (add-props s properties)

This macroexpands to something like:

(clojure.core/let [G__1649 s] (add-props G__1649 s properties) G__1649)

That's not quite what you want either, because the set gets passed to
add-props twice.

You want:

(defrecord Foo [properties]
  Fooable
  (foo [f]
(let [s (java.util.HashSet.)]
 (doto s (add-props properties)

Or, more concisely:

(defrecord Foo [properties]
  Fooable
  (foo [f]
 (doto (java.util.HashSet.) (add-props properties

But there's also a typo in your add-props:  the (.add p) needs to
become (.add s p)

Then again, you may just want (set properties) instead of the whole
doto thing, to save yourself the effort!

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

Re: strange bug in range or lazy-seq?

[Sam Roberton]

 (def nums (cons 2 (lazy-seq (map inc nums
 (def primes (cons (first nums)
              (lazy-seq (-
                (rest nums)
                (remove
                  (fn [x]
                    (let [dividors (take-while #(= (* % %) x)
 primes)]
                      (some #(= 0 (rem x %)) dividors

 It works fine. However if I redefine nums like this
 (def nums (drop 2 (range)))

 It gives me a wrong result
 e.g. (take 5 primes) is (2 3 5 7 9)

I don't have a complete answer, but...

Using your first version (cons and lazy-seq):

user (take 25 primes)
(2 3 5 7 11 13 17 19 23 29 31 37 41 43 47 53 59 61 67 71 73 79 83 89 97)

Using your second version (range):

user (take 25 primes)
(2 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 37 41 43 47 53 59 61 67 71)

Notice that the range version has every odd number right up until 31,
then it switches to starting to get the primes correct from there on.

So now I'm suspicious.  A quick look at the source for (range) and
note it's doing chunking, with 32-item chunks:

(defn range
  ...
  ([start end step]
   (lazy-seq
(let [b (chunk-buffer 32)
  comp (if (pos? step)  )]
  (loop [i start]
(if (and ( (count b) 32)
 (comp i end))
  (do
(chunk-append b i)
(recur (+ i step)))
  (chunk-cons (chunk b)
  (when (comp i end)
(range i end step)

Quite a numerical coincidence...  So is it possible that the
difference is in how the lazy sequences are being called on to produce
their next item?  Perhaps range, since it has a next one available and
pre-calculated already, offers its up more willingly than your
hand-rolled lazy-seq?  Figuring the details of that out is beyond my
limited capabilities at the moment, but it seems worth investigating.

Finally, for interest in figuring out exactly what the two different
behaviours are:

(def right-nums (cons 2 (lazy-seq (map inc right-nums
(def right-primes (cons (first right-nums)
  (lazy-seq (-
 (rest right-nums)
 (remove
  (fn [x]
(let [dividors (take-while #(= (* % %) x)
   right-primes)]
  (some #(= 0 (rem x %)) dividors

(def wrong-nums (drop 2 (range)))
(def wrong-primes (cons (first wrong-nums)
  (lazy-seq (-
 (rest wrong-nums)
 (remove
  (fn [x]
(let [dividors (take-while #(= (* % %) x)
   wrong-primes)]
  (some #(= 0 (rem x %)) dividors

(def wrong-answers
 (filter (fn [x]
   (not (some #(= x %)
  (take-while #(= % x) right-primes
 wrong-primes))

user (take 2 wrong-answers)
(9 15)
user (take 3 wrong-answers)
(9 15 21)
user (take 5 wrong-answers)
(9 15 21 25 27)
user (take 6 wrong-answers)
... much time passes, I get bored, Ctrl-C ...
; Evaluation aborted.

Hope this helps?!

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en