On 15/09/11 03:14, Julien Coloos wrote:
Le 12/09/2011 23:09, Bron Gondwana a écrit :
- for 'old' mailboxes (those created before the annotation storage
usage field in the index header), current annotations usage shall be
computed (and added to the quota entry) upon upgrading; this way
users won't have to run 'quota -f' for all quotaroots after
switching to this new version ;)
Definitely. Upgrading usually handles things like that. It's
the right way[tm].
Pushed a few lines of code in the 'upgrade_index' function. Though it
still lacks the annotation storage usage computing.
It's in the branch 'gnb/annotate/fixes'; commit
ed61f721487804b205e399c538fc35ddc153bd93.
Thanks Julien, I've cherry-picked this commit and the Fixes commit
from the other day into my annotate branch.
BTW I'm working on making reconstruct calculate the X-ANNOTATION-STORAGE
quota. Some of that code may also be useful during upgrade and for
handling message expunging.
Actually I just gave up the 'old' test: there is no easy way to
simulate upgrading mailbox index, or at least I don't feel confident
enough to make it in cassandane :(
No, there isn't a really good way at the moment.
Easiest way is the same way I did for the broken quotas test.
Have a tar file with the contents of the old mailbox which
you unpack onto the filesystem, and then open the mailbox and
check that the upgraded fields are what you would expect.
I also did something similar for the crash on thread test,
where there were 5 messages which were known to be able to
crash the THREAD command. I unpacked the folder contents
with those messages in a test.
Nice idea :)
Bron's method of saving a tarball of the state of a mailbox generated by
an older release is reasonably straightforward. The difficulty however
is going to be working out whether the result of the upgrade is correct.
In the general case, you can't just test that the first open of the
mailbox doesn't report an error from upgrading, you need to test at least:
* the mailbox is re-written with the latest version number
* all the new fields have correct or at least feasible values
* the messages previously present are still present and in the same
order with the same contents and the same metadata (uid, flags,
internaldate, etc)
To do this properly you really need to save, independently of the Cyrus
mailbox store, a whole lot of information about the expected results of
the mailbox open so that Cassandane can check it. The two other
alternatives are both worse: a) don't check and b) hardcode checks into
the Cassandane code which depend intimately on details of the data in
the mailbox state tarball.
I'm thinking that what we need is a standalone utility built from the
Cassandane source tree. Someone would point this utility at a live
working Cyrus instance (not one controlled by Cassandane), and it would
create a new folder and fill the folder with known data, then harvest
the on-disk state left by Cyrus and packages that state up into a
tarball along with a separate description of the messages. That tarball
could then be checked into the Cassandane tree and used to run upgrade
tests with reasonable post-upgrade checks. The advantages I see for
this approach are:
* when (not 'if') we need to improve either the data generated or the
checks run, we can improve this utility and then re-capture new state as
time permits.
* the versions of Cyrus tested can be on separate machines from
Cassandane itself
- which allows testing upgrades from really old versions which don't
install or run on modern OSes due to library issues, and
- allows testing upgrades from Cyrus on other platforms (e.g.
platforms with different endianness or wordsize).
BTW at some point in the future I want Cassandane to be able to handle
multiple installed versions of Cyrus, so that we can test configurations
like cross-version replication. But that's not really the same problem
as upgrade testing. For upgrade testing, we don't need functioning code
for the old Cyrus versions, just a snapshot of their spoor. For
cross-version replication or murder testing we need live installs to
poke. The difference is critical, because I figure we'll need to
support upgrade from much older versions than we support in a
cross-version murder, and those older versions will be quite challenging
to install or get running on the same platform that we develop on.
Pushed a few things.
Thanks. I pulled in your commit Added quota MESSAGE resource (RFC 2087)
test cases. yesterday and then refactored the new tests a bit, but
didn't get a chance to push it out again until today.
Added an option hash so that when running commands:
- test can run system commands (based on the current code which would
run cyrus commands inside the current cassandane instance directory)
- finer I/O redirections are possible
- working directory can be specified
While adding those new 'features', I still kept
