Persistant variable handling is one of the things which I have spent much time on as a beginner and former SQL-illiterate (Among getting the swank to finally work (it's a dream!)). I have however got into databases quite a bit among the way - but it was not my main goal and it has taken some time from "the real task".
I have looked into clj-record, which seems to be highly usable as well, but the persistant refs-idea IMHO may feel a bit more elegant. An easy way of persistence would be highly valuable when developing web applications with compojure in :reload-mode, since it seems to lose normal persistant refs after each reload in the browser. It would be valuable in production environments as well, of course. /Linus 2010/9/2 Alyssa Kwan <alyssa.c.k...@gmail.com> > I'll go one step further and say that we shouldn't have to call > "persist namespace". It should be automatic such that a change to the > state of an identity is transactionally written. > > Let's start with refs. We can tackle the other identities later. > > The API is simple. Call (refp) instead of (ref) when creating a > persisted ref. Passed into the call are a persistence address (file > path, DB connection string, etc.) and a name that has to be unique to > that persistence address. Not all refs end up being referred to by a > top-level symbol in a package, and multi-process systems are hard... > Ensuring uniqueness of name is up to the programmer. Upon creation, > Clojure checks to see if the refp exists in the store; if so it > instantiates in memory with that state, else it uses the default in > the call. > > In a dosync block, the function runs as normal until commit time. > Then Clojure acquires a transactional write lock on each refp that is > alter-ed or ensure-d. It checks the value against memory. If it's > the same, commit data store changes. If not, retry after refreshing > memory with the current contents of the store. If the data store > commit fails, retry a number of times. If the data store commit still > can't proceed, roll back the whole thing. commute and refset are > slightly different, but for an initial implementation, just treat > commute as an alter, and ignore refset. > > Does this make sense? > > My intention is to cover the 80% case. The implementation would > necessarily be chatty, since the API is chatty. That's OK for most > systems. > > This API has the benefit of being able to be shared across Clojure > instances. It's a nice bonus. > > A dosync block may contain symbols pointing to refp's spanning > different data stores, which isn't too hard to handle. It simply > requires that if this is the case, each data store must support two- > phase commit or some other distributed transaction supporting > protocol. For an initial implementation, I would just throw an > exception. > > I've begun working on an implementation using BDB. > > What do people think? > > On Aug 30, 5:02 pm, nchubrich <nchubr...@gmail.com> wrote: > > I'm not aware of any, but +1 for seeing persistence handled as part of > > the language. A big project and a long-term one, to be sure, but > > could it not be considered a goal? > > > > In my student days, I was talking to a well-known Lisper (name > > suppressed for fear of Google indexing) about some data structures in > > MIT Scheme. When I asked about saving them to disk, he said in > > effect, "You're on your own----that's something that \should be > > handled, but never is". > > > > I think people are so used to this state of affairs they forget how > > ugly it really is. Programming languages are like Moses without > > Joshua: they lead your data in the wilderness, but when it comes to > > finding it a permanent home, you have to talk to someone else. And > > these "someone elses" (who seem to be as numberless as the sons of > > Abraham) each have their own habits and ways of talking. > > > > Persistence libraries always end up warping the entire codebase; I've > > never succeeded in keeping them at bay. Using data with Incanter is > > different from ClojureQL, which is different from just using > > contrib.sql, and all of it is different from dealing with just > > Clojure. (I've never even tried Clojure + Hibernate.) You might as > > well rewrite the program from scratch depending on what you use. > > Maybe other people have had better luck; but whatever luck they have, > > I'm sure it is a fight to keep programs abstracted from persistence. > > > > I'd like to be able to work with mere Clojure until my program is > > complete, and then work in a completely separate manner on how to read > > and write data. Or maybe there would be off-the-shelf solutions I > > could plug in for different needs: low latency, high read, high write, > > large stores, etc. > > > > On the Clojure side, you would simply call something like "persist > > namespace", which would save the state of your current or given > > namespace (unless you pass it the names of variables as well, in which > > case it only saves those). And to read data, you would simply require > > or use it into your namespace: you could choose what granularity to > > give first-class status: just tables, or columns as well, etc. And > > you could do this equally well for XML, JSON, relational data, or a > > graph store; your choice. And the only difference between these and > > ordinary variables would be----heaven forbid!----a disk access might > > happen when you deal with them! > > > > To have such a system work well, you would need to enrich the way you > > query Clojure datastructures. I have some ideas on that, but I'd like > > to make sure I'm not shouting in the dark first. > > > > I'd like to see a day when programmers need to worry about persistence > > about as much as they worry about garbage collection now. > > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with > your first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com<clojure%2bunsubscr...@googlegroups.com> > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
