David W. Van Couvering wrote:
I would like to have an email list set up that sends out test
results, for those of us who would like a "push" model to see what's
going on. Personally I would want to see test failures sent to
derby-dev. If you don't like seeing them, you can filter them out,
but I think it's good for the developers to have awareness of what's
going on.
I think that combined with logging JIRA issues would be good. We
could use Kathey's idea to get a report on open test bugs by
filtering on the right component.
If it is agreed upon that we need to change the way developers get
alerted about test failures, I basically support what what is quoted
above. I've seen many good suggestions the past 24 hours for how to
increase awareness and potentially trigger action when a test in
derbyall fails. However....
With regards to the _awareness_ part of the challenge, what we need
(IMHO) more than yet another wiki-page or other kind of web page that
people have to manually update and/or navigate to, is some kind of
"push" mechanism such as automatically sending e-mails to some mailing list.
Adding JIRA-entries sounds good to me, since e-mails are sent to
derby-dev when issues are created/updated (although this strategy seemed
a bit chaotic to me yesterday), thus increasing awareness. But, it
requires a certain amount of *manual* work. Someone has to have the itch
to check the actual test results (whether they are provided by Sun or
IBM or someone else), pick a test failure (if any exist), check if it
has been reported in JIRA already, and, if not, create a new JIRA issue,
etc. Because of this, I do not think this strategy is sufficient as a
long-term solution (but it may be a part of it).
I suggest that we focus on deciding:
a) Do we want to get automated e-mails reporting test results?
b) If so, how often should it be reported (after every test run, after
every test run with failures, after every test run with a certain
number of failures, once a day, once a day if failures occured, ...)?
(I guess the number of sources producing public test reports regularly
may be a factor here in the long run, but today there is only one such
source)
c) Where should such e-mails be sent (derby-dev, derby-test-results,
derby-commits, ...)?
...and that we try to keep this separate from the discussions of adding
new web-pages with test results, wiki-pages with test results, new
JIRA-types, etc.
--
John
