MINUTES, 2009.03.12
Present: Alex, Chris, Marek, Namshin, Titus
1. The new test framework
Titus: the new framework is now in master. Features:
- simplified structure
- removes dependence on Nose
- integrates coverage analysis ('python runtest.py --coverage')
Chris: unfortunately, the new framework has currently got a number of
issues:
- individual tests frequently lack isolation from each other; note
that since most (if not all) of problems of this sort stem from
pygr.Data, they should disappear with the metabase (see below) rollout
- there is no automatic conditional skipping of certain tests
- Internet connectivity is not checked for despite some tests
depending on it
- if PYTHONPATH is not set, tests which should fail as not knowing
where Pygr should be are simply skipped instead; as a result, this
problem can only be detected by knowing beforehand the expected
number of successful tests
- it requires a bit of careful reading to determine which of the tests
have actually failed (Titus: this is because each test file is
considered an independent suite by runtest; one could work around this
by having runtest compile one suite from the contents of all files
before running tests)
- runtest's error reporting in general leaves much to be desired of:
on one hand many tests produce extremely verbose output, on the other
some errors are extremely hard to figure out (e.g the one appearing
when runtest is called without '-b' for build-dir testing)
- unittest test cases fail when run without arguments, which makes it
more difficult to run individual tests from the Python command line
(Titus: this can be addressed by using a different runner)
Marek: It seems some of the tests have been 'neutered' during framework
conversion, making it possible for them to return false positives. One
of such mistakes (the pygr.Data.dir() test - in the new framework,
instead of comparing an array of dir() output with one containing
expected keys using '==' they were compared by checking whether their
size was identical) has been identified and fixed, it is however not
know whether all other tests are fine.
Titus+Chris: this kind of mistakes ought to be caught by code reviews
performed on new commits, this can however be difficult if commits are
as large as e.g. Istvan's framework rewrite. Therefore, all developers
providing code to be incorporated into master are asked to observe the
following guidelines:
- maintain granularity of changes by committing frequently to your
local repository
- maintain isolation between issues by developing code of each of them
in a separate side branch
- merge the official master with your active side branch(es) on a
regular basis
- when submitting to the official master, preserve commit logs from
development in your repository
- if you change someone else's code, have the original author review
your changes to make sure the new code does all the old code did or was
supposed to
Megatests will not be re-written for the new framework until issues with
the latter have been addressed.
2. The buildbot
Titus: progress on the subject was hindered by the spring break, then
again Justin can now run the bot under Linux. When he's back the
following week, he will begin by setting it up to run on a regular
basis - then proceed with deploying under Mac OS X and Windows.
Furthermore, a new student will be working on buildbot code the
upcoming quarter to make it more simple.
Chris asked Titus for Justin to register on Google Code.
3. Megatests
Namshin has agreed to review recent portability changes (issue 69)
instead of Titus.
Deployment of a new MBI megatest machine hindered by RAID failure;
presently the system hasn't got enough disc space to hold all data
files needed by the tests, let alone test output files. Work is ongoing
on providing a replacement array.
4. Metabase
Chris: metabase is a reorganisation of pygr.Data code which attempts to
address a large number of existing and potential issues with that code.
A post on the subject was sent to pygr-dev. The code has been posted
for review to a non-master branch on github.
Titus: The problem is, nobody but Chris knows pygr.Data internals well
enough to review the code. Titus will try to, within two weeks, find
someone to review the code; if it doesn't work, fallback validation
would involve feedback from pygr.Data users (mostly UCLA people).
An issue will be created in the tracker, with Titus as assigned
reviewer.
5. SequenceDB review
Titus: pygr-dev post 'short notes on seqdb review' contains observation
from the first pass, combined with integrating Alex's short-read code.
Mostly formatting-related issues; "separate into modules" request has
been preempted by Chris after Titus's last pull from master. Changes
will be submitted as discrete commits and clearly tagged as
non-functional. Moving on to functional issues will be announced.
One peculiarity observed so far has been is that sequences establish
connection to databases, which can be considered a violation of
modularity. Chris: this has been done on purpose and is in fact somewhat
modular, as only two methods per sequence class (strslice and __init__)
contain database-related code. Titus: if we don't move this code to
database classes, the rationale should be documented in docstrings. All
in all, this issue will be revisited.
The second pass will follow soon, to be announced on pygr-dev.
Another matter: both development of the short-read back-end and the
code review would benefit from a short list of required feature of a
SequenceDB. Chris will provide such a list and post it in (suggested by
Titus, in the order of preference): docstrings, tests or the
development guide.
6. Verifying FixedNeedsReview issues
The following convention has been established for specifying the
reviewer:
- add a 'reviewby-<name>' tag to facilitate searching (Titus has
reconfigured the tracker accordingly)
- add <name> as the *first* one on the Cc list
Titus: when closing a verified issue, please provide a short comment
that the check has indeed been successful.
MS: in the process of writing up the tracking process; will add today's
changes.
7. The alpha release
It has been clarified that alpha == feature freeze. Titus: we shouldn't
announce this version too widely, leave this up to beta. Chris: agreed.
Timeline - the current plan is to declare the freeze after the review
and incorporation of the metabase patchset, but since that may take a
while everyone not involved can carry on verifying current
FixedNeedsReview issues and/or working on documentation updates (see
below).
Titus: Short-read code will remain a plug-in for now. Incorporation
into the core tentatively planned for Pygr-1.0.
8. Documentation updates
Chris: Right now all updates are blocked by having to migrate from
LaTeX to restructured docs (following the move made by the Python core
between 2.5 and 2.6). The first pass of this migration will therefore
be minimal, i.e. concentrating on reproducing the old effect in the new
format; enhancements such as doctests will come later. Titus:
suggests breaking pygr.tex into multiple files combined using \include,
then converting these files to REST one-by-one - but in general, the
rewrite itself should be simple enough.
9. AOB
Since the UCLA XML-RPC server is based on Pygr from Git, it would make
sense to update it to support regular expressions in pygr.Data.dir() -
however, it must be confirmed that the upgrade won't break 0.7.1
clients. Chris and MS will confirm this, then let Namshin know how to
proceed.
--
MS
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"pygr-dev" group.
To post to this group, send email to pygr-dev@googlegroups.com
To unsubscribe from this group, send email to
pygr-dev+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/pygr-dev?hl=en
-~----------~----~----~----~------~----~------~--~---
