Hi,
I'm jumping in that thread late and wanted to stress some points from a
project management standpoint:
- We care about performance: right now, poor performance is the biggest
recognized issue for Chandler's adoption. This is in discussing this
that we decided to devote 3 weeks of the entire team working on this
matter alone, trying to understand what can be done, make headways in
some places, allow devs to work collaboratively on some issues, etc...
We'll have more work to do on performance after those 3 weeks for sure
but the objective was to have everyone involved in it and caring for it
for a while. The message is: everyone owns performance.
- Baseline and regression are two different things: one aspect of the
discussion that started the thread but was somewhat muddled afterward is
the difference between "baseline performance" and "performance
regression", resulting in people talking pass each other. Both are
important in their own right though, clearly, a "baseline" must be
reached before we can consider "regressions". This is why we modified
the performance matrix so that scenarios that do not reach the
"acceptable" level yet are clearly identified, regardless on how best or
worse they compared to the previous run. It's true (as stated by
stearns) that for Preview, we'll likely aim for "acceptable" and not get
to "ideal", leaving the matrix in an all-orange state. Let's be clear
here: orange is good, we like orange.
- Regression tracking is important moving forward: we'll need to have a
performance tracking system so that we get alerted when performance of
the base scenarios is falling behind for *any* reason. Everything that
has been said in the thread is true and fair and what needs to be done
when a regression is detected can be debated (log a bug, send an email,
rollback a change, chat on IRC, etc...) and we need to find our own
Chandler way of doing it. One thing is sure though: if not monitored and
cared for, performance will fall behind. That's a law of software
development...
- Bugzilla is our tool to track incidents : this came up during the
discussion. Bugzilla is not perfect, it has issues but, as a team, we
need to use one single tracking tool and this is the one we use in that
team right now. We even use it to track feature work (tasks) at least in
the Apps team. A perf drop is a bona fide "incident" and using Bugzilla
to eventually track it seems fine. In any case, the "tracking perf
regression" issue is orthogonal to the "is Bugzilla an adequate tracking
system" issue. That's something we may want to discuss on its own thread.
In conclusion, considering the current objective (get to baseline) and
to come back to the title of the thread, I'm voting for a moratorium on
the current "log a regression bug" policy, at least till we reach
"baseline" (which may happen *after* the end of the current perf
tweaking period).
In the meantime, we can discuss of what the perf regression alert policy
should be.
Cheers,
- Philippe
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Open Source Applications Foundation "chandler-dev" mailing list
http://lists.osafoundation.org/mailman/listinfo/chandler-dev
