Hi,
lately we've been experiencing some problems on our Invenio test server
loaded with 6.5 millions documents. After some investigation I found out
that the table definitions of our database were a bit old and that in
the 7 months since the initial pull from the Git repository.
First I thought that it was strange because I've never experienced such
a problem before. And then I understood: I've always done all my work on
the Atlantis collection and reloaded the full install several times a
week (maybe day), so my database was always up-to-date.
When running a development server on a non-Atlantis collection (and
especially if the test collection is big), it is not possible to reload
the database entirely. So we rely entirely on updates.
Additions to the latest Makefile update script (currently v0.99.1) are
added regularly but differences between the code and the MySQL table
updating scripts remain, e.g. the creation script for the index tables
in tabcreate.sql and in bibindex_engine.py (although this is only a
minor difference, it shows my point).
So how can we avoid this problem? Here are a few opinions:
* Reject all the commits that touches the database structure without
updating the Makefile.
* Using the DROP TABLE statement in the update scripts should be
allowed only if the table to remove will actually disappear from the
system (c.f. update-v0.92.1-tables). Table definitions should be updated
and not recreated. People might have useful information in there.
* Create an automated database updating system that would rely on an
internal database version number. This number could be the version of
Invenio when the database was last updated. Each time the install script
is used, we could compare the version of Invenio and the version of
database and automatically run the appropriate database update script(s).
I suspect that setting up such a system would simplify greatly the life
of both the admins and the support team.
Best regards,
Benoit.
