1) Having a view level commit would be incredibly useful.
Hard. I can only suggest: apply only changes to that view, then commit.
I'm not sure what you're after - sometimes a consistent datafile will require changes in more than one view before getting committed (e.g. one view containing totals of another one).
I suppose what I'm getting at is that if two users are working on different tables and one user commits, the changes the other user has should not be commited, i.e. they can't be rolled back. I'm second guessing this though. The combined view that I'm using already has this property so I'm not so sure that it is necessary anymore.
2) The combined view is really a special form of a blocked view.
The whole mechanism of basic view and derived views is reaching its limits. There are *tons* of views one can come up with which could be defined in terms of basic ones. This is one of the areas where language boundaries bite: with a blocked view in C++, Mk4py cannot subclass it somehow, and adjust it for other purposes such as the above.
This is not insurmountable, this might just be a function of the Mk4py wrapper. The boost C++ wrappers, or even swig these days, allow python to subclass C++ classes although I see your point, a whole lot more of the meat would have to be exposed to be able to do this.
I've mentioned this before, but I can even create a combined view using views from different storages!
Yes. With modifiable combinations, things become more complex of course - but that's to be expected with updatable derived views (which are not fully supported right now).
My python wrapper takes care of this for now.
All in all, given the fact that I have been learning as I'm going, I think that metakit is really, really close to providing almost full ACID support. Am I kidding myself here?
It is, to a pretty large extent, I agree.
Atomicity: yes, MK uses stable storage.
Consistency: yes, same reason, as long as all views are in the same datafile.
Isolation: this one needs work, it does not apply in single-user usage.
Durability: yes, again stable storage.
Isolation requires more than just collecting changes and being careful to apply them in ways which do not affect others too soon. It also needs to deal with coherence & contention, i.e. what if A adds $100 to a bank account, and B does the same:
A: start: temp = balance; temp = temp + 100
B: start: temp = balance; temp = temp + 100; balance = temp; commit
A: balance = temp; commit
Either A's or B's changes have to be prevented or re-calculated.
I'm doing this now using row level locking. It's pretty easy in metakit and adds negligible overhead. I just add a lock column which is 0/1 Of course, I run into problems if the client crashes somehow and leaves rows locked, so I probably need a more sophisticated mechanism. On the plus side, these are really easy to implement in metakit. Already I have foreign keys, transaction implementations on views, triggers, internal functions and commit/rollback. MySQL took what, five years to do this :)
Note also that MK's commit-aside is another mechanism to start addressing these issues. The aside file essentially stores the difference between the old and the new state, in a transaction-safe way. This goes even further than the above cases, because commit-aside also properly covers all storage re-structuring changes. Commit aside works at the column level, with the potential to be particularly compact and efficient.
The way I think everything can be made to work, is to store changes (similar to a journal) without actually committing (i.e. a "pseudo commit"), until all users are known to have an up to date view of things. For readers/writers which come to the party later, they can open the unmodified original, and each play back the pseudo-commits. IOW a bunch of layered views, as follows:
| original state | | | pseudo-commits | | | uncommitted changes
This is pretty much the technique that I am using in the Combined View. The client provides a data-playground that keeps track of changes to the base views. This I see as sort of a multi-user commit-aside. During a commit, the data is sent to the server, checked for collisions and the like. The upside is that this is easily implemented, the downside is that data-changes are slower than normal.
Of course this brings up the question of can metakit support multiple aside files, each from a different user? The answer, obviously is no, but intriguing.
All of this can be coded at the scripting level IMO. It looks like I will be able to explore exactly such an approach in an upcoming Tcl project. My agony continues because this means that solving it in one language will not make it usable with any other language binding!
The methodology will be in-place though, that is what rapid prototyping is for.
Since I know what is around the corner, I expect this will be a short agony :)
-jcw
_____________________________________________ Metakit mailing list - [EMAIL PROTECTED] http://www.equi4.com/mailman/listinfo/metakit
_____________________________________________ Metakit mailing list - [EMAIL PROTECTED] http://www.equi4.com/mailman/listinfo/metakit
