On Sat, Mar 26, 2011 at 13:12, Base <basselh...@gmail.com> wrote: > Thanks All! I guess I didnt even realize that they were so different > (though that certainly makes sense). > > Basically I want to design a system that will have maybe 30M > (eventually) complex graphs of data that need to be searchable through > either pattern matching or unification or ?... > The pattern s could conceivably become very complicated. > > My thought was that this would need to scale out horizontally and > allow for a map-reduce like process to burn through these searches > (though at this point i am still planning this part out and currently > have my data in an RDBMS (using H2 in development - great little > database). >
That helps, so you need: - key:value store - support for graph structures. - mapreduce My experience is with Riak, and none of the others, which can certainly help you here (in the same order as above): - You store your objects, - You can define links between objects, - You can walk these links during map reduce jobs This is very new territory for me, as i am very much used to working > with traditional databases so I kind of dont know where to start... > http://mmcgrana.github.com/2010/08/riak-clojure.html does a decent job of giving you a low down on riak with clojure. I've accentuated the api to permit storing links against your objects: https://github.com/ossareh/clj-riak - this hasn't been pulled into Mark's main branch yet. A high level run through (tested): (ns riak-test (:require [clj-riak.client :as riak] [clojure.contrib.json :as json])) (def rc (riak/init {:host "127.0.0.1" :port 8087})) (defn fetch [type key] (try (let [bucket (name type) val (riak/get rc bucket key) links (:links val) val (->> val :value String. json/read-json)] (with-meta val {:links links})) (catch Exception e nil))) (defn put [bucket key data] (let [bucket (name bucket) links (or (:links (meta data)) '()) data {:value (.getBytes (json/json-str data)) :content-type "application/json" :links links}] (riak/put rc bucket key data))) (defn link [obj to-type to-id type] (let [links (:links (meta obj)) links (conj links {:bucket (name to-type) :key to-id :tag (name type)})] (with-meta obj {:links links}))) ;; now you can store objects with links. (put :test "foo" {:some "data" :about {:your "domain"}}) (put :test "bar" (link {:more "data"} :test "foo" :example)) ;; The following is from the repl riak-test> (fetch :test "foo") {:some "data", :about {:your "domain"}} riak-test> (fetch :test "bar") {:more "data"} riak-test> (:links (meta (fetch :test "bar"))) ({:bucket "test", :key "foo", :tag "example"}) Now you'd have two independent objects in riak, which during map reduce could be used in a graph like manner. In this case the "bar" object points to the "foo" object. A point worth making is that Riak is a dynamo based kv store. Which, most notably to a RDBMS user, means you don't have the concept of updating in place. That is, if I wanted to change the bar object to have a key of :name with a value of "bob" you have to do this: riak-test> (put :test "bar" (assoc (fetch :test "bar") :name "bob")) nil riak-test> (fetch :test "bar") {:name "bob", :more "data"} i.e. fetch bar, assoc bar :name bob put result. HTH > On Mar 26, 2:55 pm, Michael Ossareh <ossa...@gmail.com> wrote: > > On Sat, Mar 26, 2011 at 12:15, Michael Ossareh <ossa...@gmail.com> > wrote: > > > > > On Sat, Mar 26, 2011 at 09:56, Base <basselh...@gmail.com> wrote: > > > > >> hi All - > > > > >> Any recommendations on a NoSQL database to use with clojure? I am > > >> experimenting if it will fit my project better than a SQL db and have > > >> no real experience with them. > > > > > I've replaced my rdbms with Riak (www.basho.com). > > > > >> Strong clojure support is obviously important for this. The only one > > >> I know of is MongoDb... > > > > > It doesn't have the best clojure interface into it unfortunately, so if > > > you're looking for 0-60 in the shortest time choose something else. It > is a > > > medium term plan of mine to build a clojure implementation. > > > > .... a clojure implementation of the interface into riak. > > > > > > > > > > > > > > > > > > > > >> Thoughts?? > > > > > Were I making the decisions again, i'd definitely look into Redis. Not > > > least because every page of their documentation has a live redis > connection > > > on it, so you can play with the function which you're reading about: > > >http://redis.io/commands/expire > > > > >> Thanks > > > > >> Base > > > > >> -- > > >> You received this message because you are subscribed to the Google > > >> Groups "Clojure" group. > > >> To post to this group, send email to clojure@googlegroups.com > > >> Note that posts from new members are moderated - please be patient > with > > >> your first post. > > >> To unsubscribe from this group, send email to > > >> clojure+unsubscr...@googlegroups.com > > >> For more options, visit this group at > > >>http://groups.google.com/group/clojure?hl=en > > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with > your first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
