On Fri, Oct 1, 2010 at 17:55, Ross Gayler <r.gay...@gmail.com> wrote:
> Hi, > > This is probably an abuse of the Clojure forum, but it is a bit > Clojure-related and strikes me as the sort of thing that a bright, > eclectic bunch of Clojure users might know about. (Plus I'm not really > a software person, so I need all the help I can get.) > > I am looking at the possibility of finding/building a declarative data > aggregation language operating on a small relational representation. > Each query identifies a set of rows satisfying some relational > predicate and calculates some aggregate function of a set of values > (e.g. min, max, sum). There might be ~20 input tables of up to ~1k > rows. The data is immutable - it gets loaded and never changed. The > results of the queries get loaded as new rows in other tables and are > eventually used as input to other computations. There might be ~1k > queries. There is no requirement for transaction management or any > inherent concurrency (there is only one consumer of the results). > There is no requirement for persistent storage - the aggregation is > the only thing of interest. I would like the query language to map as > directly as possible to the task (SQL is powerful enough, but can get > very contorted and opaque for some of the queries). There is > considerable scope for optimisation of the calculations over the total > set of queries as partial results are common across many of the > queries. > > I would like to be able to do this in Clojure (which I have not yet > used), partly for some very practical reasons to do with Java interop > and partly because Clojure looks very cool. > > * Is there any existing Clojure functionality which looks like a good > fit to this problem? > > I have looked at Clojure-Datalog. It looks like a pretty good fit > except that it lacks the aggregation operators. Apart from that the > deductive power is probably greater than I need (although that doesn't > necessarily cost me anything). I know that there are other (non- > Clojure) Datalog implementations that have been extended with > aggregation operators (e.g. DLV > http://www.mat.unical.it/dlv-complex/dlv-complex). > > Tutorial D (what SQL should have been > http://en.wikipedia.org/wiki/D_%28data_language_specification%29#Tutorial_D > ) > might be a good fit, although once again, there is probably a lot of > conceptual and implementation baggage (e.g. Rel > http://dbappbuilder.sourceforge.net/Rel.php) > that I don't need. > > * Is there a Clojure implementation of something like Tutorial D? > > If there is no implementation of anything that meets my requirements > then I would be willing to look at the possibility of creating a > Domain Specific language. However, I am wary of launching straight > into that because of the probability that anything I dreamed up would > be an ad hoc kludge rather than a semantically complete and consistent > language. Optimised execution would be a whole other can of worms. > > * Does anyone know of any DSLs/formalisms for declaratively specifying > relational data aggregations? > > Thanks > > Ross This sounds very similar to NoSQL and Map/Reduce? http://www.basho.com/Riak.html Where your predicate is a reduce fn? -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
