Build, Release and QA
---------------------
*Checkpoint*
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008108.html
*PPD triage*
Sheila and Mimi went through all of the alpha5 and 0.7release desktop
bugs, prioritizing them for Preview. They put them in buckets:
P1: Bugs that prevent users from trying out the app or particular features
P2: Bugs that block dogfooders
P3: Bugs that are actively confusing people
P4: Bug fixes that would make it more likely for people to stick with
Chandler (nice to have)
http://wiki.osafoundation.org/Journal/ChandlerBugTriage20070328
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008128.html
*Intermittent test failure*
Morgen observed that PerfLargeDataScrollTable fails or succeeds based on
the time of day (as events cycle in and out of various triage sections).
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008133.html
Performance
-----------
*Notifications and redrawing*
John was exploring improvements to notification handling to eliminate
unnecessary redrawing. (1) Don't draw if a change doesn't affect the
widget. (2) Make a small change to a widget instead of recreating the
widget. Example: if an item is removed from the table, delete the single
row instead of recreating the table. To pursue the example, John needs
the deleted item's index. He asked Andi if that information could be
included in the notification.
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008095.html
Andi explained that he doesn't have the old index location readily
available. He noted that indexes are versioned, and that one could ask
the index location given a UUID (but performance of such an operation
should be considered). John thought keeping a widget up to date should
be similar to keeping a selection up to date.
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008098.html
Andi also mentioned keeping information about the "page" being displayed
at the UI level, tracking the position of the items having changed there.
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008097.html
*Performance regressions*
Heikki kicked off a thread questioning the value of tracking performance
regressions. He was working under the assumption that tracking down
performance regressions was valuable, a first path to pursue that is
easier than regular performance work. He noted that other developers
were questioning the perf regression bugs he logs, and wondered if he
should continue to do so.
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008099.html
Andi felt that logging performance regression bugs when (expected)
slowdowns occur do to feature additions or bug fixes was unhelpful. He
wanted to see regression bugs if a test became slower as an (unexpected)
side effect of some other change.
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008100.html
Heikki found this criteria confusing: all checkins are bug fixes or
feature additions. Heikki won't always know what slowdowns are expected,
he logs bugs as a task/reminder for someone to go check it out. He
checks the graphs once a day, and doesn't log bugs for temporary spikes.
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008102.html
Bryan felt that logging performance regression bugs as new features land
was not especially productive and somewhat demoralizing. Performance
analysis works better after the feature chain is in (not commit by
commit); backing out the new feature isn't the productive path forward.
Bryan also noted that the measurements vary widely, so the graphs cover
too short a period to reliably see the result of a commit. The graphs
indicate areas where investigation is necessary, but Bryan focuses on
his profiles and doesn't rely too much on differences in the graphs per
commit. Bryan also asked that we make green the "acceptable" color
instead of orange.
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008101.html
Heikki explained that he worked on another project where the expectation
was that no new feature is committed if it jeopardized performance --
some projects do operate that way. He's not proposing that for Chandler
at this stage. He acknowledged some of the measurement problems that
Bryan brought up; improving them hasn't been the top priority but we
might want to tackle them later.
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008104.html
Philippe wrapped up the thread with some important points:
- Performance is everyone's problem. We added a 3 weeks period to the
schedule to allow everyone to focus on performance, after feature complete.
- "Baseline performance" and "performance regression" are two different
things. Before we can worry about regressions, we need to hit an
acceptable baseline.
- We're shooting for "acceptable" right now. Orange is good, orange is
the current goal.
- We're worried about any case that is not "acceptable" -- we care more
about whether or not we're in that goal range than whether or not it is
faster or slower than previous runs.
- Once we've hit the baseline or acceptable values, we'll need to be
tracking regressions. Regression tracking is important, if we don't
monitor performance and pay attention to it, performance will fall behind.
- Bugzilla is our tool to track issues. A marked performance drop is a
real "incident" that needs tracking; Bugzilla is an appropriate tool to
use for this kind of tracking.
Philippe asked that we stop automatically logging performance
regressions until we hit baseline numbers.
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008118.html
*Quick item entry test*
Heikki wrote a performance test for quick item entry. The operation
ended up being significantly faster than creating an event by double
clicking -- we don't select the event and display it in the detail view.
This case should get faster as other similar cases get faster. Heikki
decided not to add the test to the Tinderbox suite, as it doesn't yield
new/different information.
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008110.html
*Performance tools poll*
Heikki sent out a poll asking questions about performance tracking
graphs and tools. A couple people questioned the statistical
value/correctness of the standard deviation calculations. Read the
thread for more detailed info on how people use rt:
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008112.html
*Performance test variability*
Heikki looked at variability of performance test results, and asked at
what threshold the results became meaningless.
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008122.html
Heikki turned off real time virus protection on the performance
Tinderboxes to reduce variability.
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008129.html
*Minor perf tip*
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008127.html
Dev
---
*Reload*
Heikki wrapped up an earlier thread about reload:
- File a bug to have File-->Reload restart Chandler with --reload
- File an enhancement request for a feature that dumps/reloads a
particular collection (like import/export)
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008113.html
Morgen planned on adding a menu to Tools allowing a reload without
blowing away current data, as he finds it useful for testing.
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008116.html
Meetings, Announcements
-----------------------
Desktop team meeting (both apps and platform groups):
http://wiki.osafoundation.org/Journal/AppsMeeting20070419
Desktop release meeting:
http://wiki.osafoundation.org/Journal/EngineeringMeetingNotes20070419
Aparna explained that the QA team will schedule additional informal
testing sessions on Fridays at 1pm PDT. They'll be conducted in the
#osaf-qa channel.
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008114.html
Migration testing QA session
http://wiki.osafoundation.org/Journal/CosmoZeroSixOneMigrationTestPlan
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008124.html
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Open Source Applications Foundation "chandler-dev" mailing list
http://lists.osafoundation.org/mailman/listinfo/chandler-dev
