I'm looking ahead to a time when I'll extend my application to be
multi-user. This would mean multiple users accessing and updating the
databases. Will Metakit support this? Or are the plans to support it in
the future?
If not, what is a reasonable workaround? Write a server that will handle
all interactions with Metakit databases?
There are many sides to multi-user:
- single process (threads)?
- multi-process, single file machine?
- locking (not just contention), but application level "logical" interaction
- isolation requirements (the "I" in "ACID"), nested commits
Have a look at Brian Kelley's Metaserve. It's a Python-based server, with clients proxying all their requests through sockets to the server. Listed on the MK links page:
http://www.equi4.com/mklinks.html
I've used a similar (but simpler) approach in the past in Tcl, using a little server called "Tequila". Has been tremendously effective for me in two commercial projects, but evidently there are many parameters which will affect how well it can work - scalability, responsiveness, parallelism, etc.
Yes, I do intend to support concurrent access. The commit-aside design (which is not finished - it uses huge amounts of disk space right now) is a way to provide contention-free concurrency. It can allow readers to proceed without ever locking. It should work well in multiple-reader/single-writer scenario's, and can be extended to serialize multiple writers.
Concurrency in MK is not trivial, due to the very high level of caching which can be going on (and memory mapped files introduce yet more cache coherence headaches). It's very easy to open a file and read without problems, it's also trivial to commit *once* and not have it affect current readers. The trouble is the next commit, and the need to sync all readers - or offer them a way to keep looking at frozen state and provide that isolation level.
I'm currently involved in writing a layer on top of MK which makes it usable as a multi-user system (but at the Tcl level, alas). The lessons from that will help design and implement proper mechanism at the core level. it's likely to go from where commit-aside stands today, and simply finish that job.
But all of this is not going to help you further any day soon. If you need something operational, I'd definitely suggest looking at Kelley's Metaserve, and perhaps contributing to that so it becomes more capable.
FWIW, I'll be following that closely, and along with my own work will look for ways to solve this at a core level. As is becoming clearer all the time, nobody seems to be very interested in solving things at any other level than the language they happen to use - but I am, more than ever. If I had the capability now, multi-user capability would be in MK today, and I would not have to solve it in "just" Tcl.
At the risk of making a nuisance of myself... let me repeat:
I'll go into a little hand-waving explanation of why it hasn't happened:
* time, priorities, funding - or any suitable mix there-of
* the desire to do this at the core level, not "just" Python, Tcl, whatever
* the need for a higher-level approach to such things than C++
* me having new code which is not good enough yet to replace the current MK
* waiting for that new code to get there, so I can code such ideas at a higher level
* time, priorities, funding - or any suitable mix there-of
It's really quite simple: if anyone wants *specific* things such as these to happen, you'll just have to reach out and fund me. Because as long as no-one does, I can only work on my own priorities, and these are: a next-generation Metakit design and contract work. Don't hold your breath on the next-gen stuff, btw. I'm not going to do more than mention it, since there is *no* time scale to put next to it.
Don't get me wrong, btw. I'm not asking for one penny from anyone. There's plenty going on, and I'm in fact buried in work at the moment. The spare moments I do have left go fully into working on new stuff, and I'm having an absolutely terrific time with that. So if people want to change my to do list, they will have to do more than post requests.
Having said that, I'm always willing to discuss - on-list or off-list - how to help you make MK do new things in ways which I can integrate and which leaves MK's multi-lingual perspective intact.
-jcw
_____________________________________________ Metakit mailing list - [EMAIL PROTECTED] http://www.equi4.com/mailman/listinfo/metakit
