Thanks Julian.
I covered Migrations above. See reference [4].
I would view migrations as a way to encapsulate formed meanings.
Something that has always struck me as funny about the NoSQL movement
is the complaints about how much of a PITA they find schema versioning
in an RDBMS. I've never served millions of pages a day, but I've
always been deeply suspicious of the cause-effect relationship
described by the engineers at the biggest sites. Put simply, it
doesn't make any sense. It's a bad correlation. The next big
architectural step forward will rectify this with a better
correlation.
On 4/10/11, Julian Leviston jul...@leviston.net wrote:
You should probably have a look at ActiveRecord::Migration which is part of
Rails if you're interested in SQL-based systems, and in fact ActiveRecord in
general is a really wonderful abstraction system - and a very good mix of
do what you *can* in a programming-language based DSL, and what you can't
in direct SQL.
http://api.rubyonrails.org/classes/ActiveRecord/Migration.html
Julian.
On 11/04/2011, at 2:58 AM, John Nilsson wrote:
Wow, thanks. This will keep me occupied for a while ;-)
Regarding AI completness and the quest for automation. In my mind its
better to start with making it simpler for humans to do, and just keep
making it simpler until you can remove the human element. This way you
can put something out there much quicker and get feedback on what
works, and what doesn't. Most importantly something out there for
other developers (people smarter than me) to extend and improve.
Regarding the database I have some ideas of experimenting with a
persistent approach to data storage, employing a scheme with
branching and human assisted merging to handle evolution of the
system. For a fast moving target such as a typical OLTP-system there
must be automated merging of course, but I see no reason for the
algorithms to be either general or completley automatic. I think we
can safley assume that there will be some competend people around to
assist with the merging. Afterall there must be humans involved to
trigger the evolution to being with.
To solve the impedance missmatch between the dynamic world of the
databse and the static world of application development I'm thinking
the best approach is to simply remove it. Why have a static,
unconnected, version of the application at all? After all code is
data, and data belongs in the database.
To have any hope of getting any kind of prototype out there I have,
for now, decided to avoid thinking of distributed systems and/or
untrusted systemcomponents. I guess this will be a 3.0 concern ;-)
BR,
John
On Sun, Apr 10, 2011 at 4:40 PM, John Zabroski johnzabro...@gmail.com
wrote:
John,
Disagree it is a simple thing, but it is a good example.
It also demonstrates blending well, since analogies are used all the
time in this domain to circumvent impedance mismatches.
For example, versioning very large database systems' schema is
non-trivial since the default methods don't scale:
alter table BigTable add /*column*/ foo int
This will lock out all readers and writers until it completes.
Effectively it is a denial of service attack. Predicting its
completion time is difficult, since it will depend on how the table
was previously built (e.g. if anything fancy was done storing sparse
columns; if there is still storage space available in-row to store the
int required by this new column thus avoiding a complete rebuild; if
the table needs to be completely rebuilt, then so do its indices; if
the table is sharded across many independent disks, then the storage
engine can parallelize the task). The *intention* is to add a column
to a table, presumably for some new requirement. But there is a latent
requirement on the intention, forming a new meaning, that nobody
should observe a delay during the schema upgrade.
Now, if the default method isn't robust enough, then What is? and What
do we call it?
Well, what I did to solve this problem was type in how to add a
column to a large table into Google [1].
As for naming it, well, the enterprise software community came up with
this concept called Database Refactorings [2] [3] or simply
Migrations [4], which are a heuristic system for approximating the
Holy Grail of having a reversible logic for schema operations
(generally difficult due to destructive changes and other problems).
Programmers procedurally embed knowledge on how to change the schema,
and then just pass messages to a server that has all of this
procedural knowledge embedded in it. It i interesting (to me, anyway)
that programmers have developed a human process for working around a
complex theoretical problem (e.g., see [5] for a discussion of the
challenges in building a lingua franca for data integration, schema
evolution, database design, and model management), without ever
knowing the problems. Good designers realize there is a structural
