Thanks Daniel,
That sounds like it's in the right direction (although that is
probably true of anything that gives database-like functionality).
I would need some filtered-join-like functionality between tables in
order to select some of the rows of interest.
As to the declarative part: leaving aside the terminology (which I am
probably abusing) I currently have a bunch of colleagues who are even
less computer literate than me doing these queries in an imperative
language and it is a wonderfully fertile breeding ground for errors.
(If you really want to know, they are statisticians using SAS to
calculate things like "Over all the companies of which this person was
a director at the time a loan application was made, count the number
of loan applications made in the last year where the purpose was "debt
restructuring" and the amount sought was less than $100k"). I guess I
was trying to achieve two things by asking for a declarative query
language: 1) reduce the accidental complexity of the specification,
thereby reducing the opportunity for specification error, 2) I
presumed that optimizing the calculation would be easier from a
declarative specification (because there is no baked in ordering of
operations).
I think my next steps are to collect a small zoo (maybe only a
menagerie) of different query languages; assemble a collection of test
queries; code the queries in each of the query languages as a pencil
and paper exercise (or in the real software if it is trivial enough to
set up); try to rank the different query languages by usability for my
colleagues, potential for optimization, and compatibility with our
software environment and my minimal computing skills.
> Does this sound like something you could use? I'm happy to share it.
Thanks. That would be great. At this stage (not knowing Clojure more
than a tiny bit) I'm likely to make the most use of examples of how it
would be used.
Cheers
Ross Gayler
On Oct 2, 2:26 pm, Daniel Phelps <phelps...@gmail.com> wrote:
> Hi Ross,
> I am working on something that may be of help to you, but it's very
> early in development.
>
> Basically I wanted to see if I could write a database server in
> Clojure and what I have now sounds (kinda) like what you're after. It
> was really simple.
>
> Imagine a list of maps as a database table. To query the structure
> we simply filter that list based on some predicate, and now you have a
> new structure. My select function takes a database (a map of maps) a
> table name and a predicate and does exactly that. Then, arbitrary
> functions can be applied to the result. For example, suppose the
> result is as follows:
>
> ({:name "Daniel" :salary 1000} {:name "Ross" :salary 1500})
>
> The total of salaries can be calculated like this:
>
> (defn sum [attr relation]
> (reduce + (map attr relation)))
>
> (sum :salary results)
> 2500
>
> You can see how min, max and friends could just as easily be
> implemented.
>
> As far as building the database, I use classes as keys for the tables
> and instances of those classes are the tuples (see defrecord). In
> that sense what I have is declarative; the database is built by
> successively applying Clojure functions to Clojure data structures.
>
> Does this sound like something you could use? I'm happy to share it.
>
> Daniel Phelps
>
> On Oct 1, 8:55 pm, Ross Gayler <r.gay...@gmail.com> wrote:
>
> > Hi,
>
> > This is probably an abuse of the Clojure forum, but it is a bit
> > Clojure-related and strikes me as the sort of thing that a bright,
> > eclectic bunch of Clojure users might know about. (Plus I'm not really
> > a software person, so I need all the help I can get.)
>
> > I am looking at the possibility of finding/building a declarative data
> > aggregation language operating on a small relational representation.
> > Each query identifies a set of rows satisfying some relational
> > predicate and calculates some aggregate function of a set of values
> > (e.g. min, max, sum). There might be ~20 input tables of up to ~1k
> > rows. The data is immutable - it gets loaded and never changed. The
> > results of the queries get loaded as new rows in other tables and are
> > eventually used as input to other computations. There might be ~1k
> > queries. There is no requirement for transaction management or any
> > inherent concurrency (there is only one consumer of the results).
> > There is no requirement for persistent storage - the aggregation is
> > the only thing of interest. I would like the query language to map as
> > directly as possible to the task (SQL is powerful enough, but can get
> > very contorted and opaque for some of the queries). There is
> > considerable scope for optimisation of the calculations over the total
> > set of queries as partial results are common across many of the
> > queries.
>
> > I would like to be able to do this in Clojure (which I have not yet
> > used), partly for some very practical reasons to do with Java interop
> > and partly because Clojure looks very cool.
>
> > * Is there any existing Clojure functionality which looks like a good
> > fit to this problem?
>
> > I have looked at Clojure-Datalog. It looks like a pretty good fit
> > except that it lacks the aggregation operators. Apart from that the
> > deductive power is probably greater than I need (although that doesn't
> > necessarily cost me anything). I know that there are other (non-
> > Clojure) Datalog implementations that have been extended with
> > aggregation operators (e.g.
> > DLVhttp://www.mat.unical.it/dlv-complex/dlv-complex).
>
> > Tutorial D (what SQL should have
> > beenhttp://en.wikipedia.org/wiki/D_%28data_language_specification%29#Tuto...)
> > might be a good fit, although once again, there is probably a lot of
> > conceptual and implementation baggage (e.g.
> > Relhttp://dbappbuilder.sourceforge.net/Rel.php)
> > that I don't need.
>
> > * Is there a Clojure implementation of something like Tutorial D?
>
> > If there is no implementation of anything that meets my requirements
> > then I would be willing to look at the possibility of creating a
> > Domain Specific language. However, I am wary of launching straight
> > into that because of the probability that anything I dreamed up would
> > be an ad hoc kludge rather than a semantically complete and consistent
> > language. Optimised execution would be a whole other can of worms.
>
> > * Does anyone know of any DSLs/formalisms for declaratively specifying
> > relational data aggregations?
>
> > Thanks
>
> > Ross
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
