This is a summary of a spontaneous IRC session from 2006-03-23. Present were Jean, Kathey and Dag. Items touched upon were: state of stand-alone tests, the amount of open bugs/quality and what could be done to improve on that situation. The IRC log is attached at the end.
* Dag said he was trying to run running all tests in stand-alone mode in all three frameworks, with useprocess true and false, checking for success, skip, failure, undecided status, and offered to put the results on the wiki. Kathey said Mike and Myrna had been trying to run more tests with useprocess=false to improve on test running time. Being able to run more tests in-process would also be useful for debugging. * Kathey expressed concerns over the (growing) number of open bugs and quality in general. We were wondering what other projects do to handle bug triage besides registering bugs in a bug data base; questions include how to prioritize and preserve architectural integrity on a "scratch your own itch" setting. We agreed we don't have much data on how other open source projects do this, although Jean noted that other, albeit smaller projects, tend to be more pro-active on bugs and get them resolved faster. Presently the number of outstanding code bugs is 155 according to Kathey. * We touched on how to make fixing bugs more attractive and meritable, suggestions including "bug fixer of the month" and tee-shirts to express peer recognition. An issue could be that contributors prefer to work on features over bug fixes due to priorities set by organizations contributors work for. * Is it a problem that Derby could be judged on the sheer number of JIRA issues, bugs or not? Jean said remarks had been made on the high number of Derby issues and that issues can equated with bugs by some. * Committing available patches is becoming a bottleneck as well, presently 18 patches are pending. It is OK to ping the list if patches are left hanging for long. Dag IRC log: (19:08:45) daghw: just as an experiment i am running all tests standalone in all 3 frameworks with and without useprocess and checking for success, skip, failure, undecided status. (19:26:31) marsden: to work to try to get the test runs faster by running more with useprocess? (19:27:00) daghw: that might be one benefit; i just wanted an overview... :) (19:28:00) daghw: i might post the results on the wiki... (19:28:19) marsden: great (19:28:24) daghw: if you think it could be useful (19:29:45) marsden: I know that Mike and Myrna had mentioned getting more running with useprocess to speed up the runs. This overview would help with that but I have no idea how it all fits into the Junit framework. (19:30:40) daghw: IMHO they should be runnable in-profcess, too, to help debugging. Not sure if they are at this point, we'll see (19:31:38) marsden: Changing the subject, I am curious. does anyone in your group do regular bug reviews? (19:32:33) daghw: I think it is ad hoc at this point, most are working hard on the features right now. (19:32:50) daghw: we need to get there though (19:35:34) marsden: I see. I am just so concerned about the amount of stuff that is going in and the bugs that customers are hitting. I am about to file one that the client connection state for pooled connections is not getting reset properly for isolation, autocommit or holdability. Manjula hit a serious issue blocking system tests. Perhaps is is just a cycle but I have customer cases coming in like a... (19:35:35) marsden: ...flood. (19:36:02) daghw: All junit tests failed stand-alone in embedded framework when useprocess=false :( (19:36:13) marsden: oh well. (19:36:25) marsden: just more work I guess. (19:36:30) daghw: yup (19:37:24) daghw: what do other communities do as far as bug triage, do you know? (19:37:38) marsden: I don't know. (19:38:10) daghw: Personally, I agree with you, the number of open bugs is concerning.. (19:38:50) marsden: I am not sure how the general quality and architectural issues work in the scratch your own itch world. (19:39:15) jta [EMAIL PROTECTED] entered the room. (19:39:38) daghw: the thought did cross my mind, too ..:) (19:44:02) marsden: Hi Jean, Dag and I were discussing bugs and quality issues. Dag had asked how other projects handle bug triage. (19:44:08) marsden: Do you know? (19:45:09) jta: I'm watching other projects very lightly (19:45:29) jta: and most of these have far fewer developers active (19:45:38) jta: ojb seems to have like 2 active (19:45:56) jta: and when something goes amiss, they're very proactive about pointing it out -- and fixing it (19:46:12) jta: forrest dev likewise (19:46:29) jta: we should look at one of the big projects with lots of components to it - httpd comes to mind (19:47:01) daghw: In general there needs to be a balance between feature work and bug fixing.. (19:47:35) jta: yes (19:47:46) jta: there's no glory in bug fixing, but it's extremely important (19:47:55) jta: how do we enhance the glory? (19:48:06) daghw: Bug fixer of the month? ;) (19:48:17) jta: another project is giving out "awards" (19:48:21) jta: let me chase that down (19:48:27) daghw: seriosuly, if should give merit (19:48:39) jta: the danger of awards is it can miss the quiet, but valuable efforts (19:49:08) marsden: Yes. I had considered. "Can you break Derby?" contest on the QA side and "Can you fix Derby?" contest on the fixing side. (19:49:14) jta: and a similar thing on the doc front (19:49:21) marsden: Problem is all I have to hand out is chocolate. (19:49:23) jta: to try to promote the value of dox fixes (19:49:43) jta: public thank you's might beat chocolate :-) (19:50:05) jta: back in a second .. the foxhound commands me to the front door ... (19:51:14) daghw: saw Daffodil (One$DB) had put up a Derby comparison... with some FUD in it.. (19:51:22) marsden: I like bug fixer of the month, but maybe make it more generic to be more of a "quality star" there are so many testing tasks that need to be done. (19:51:50) daghw: I think community recognition is the best tool we have for motivation. (19:52:08) daghw: SO, yes, giving it attention is necessary. (19:52:40) jta: how about a derby t-shirt? (19:52:51) jta: maybe we could get sun or ibm to do up a bunch (19:52:57) daghw: We should all have those anyway .!! (19:53:04) jta: :-) (19:53:21) jta: how about a special t-shirt -- "I fixed a bug and all I got was this damn shirt" (19:53:50) jta: oops, maybe not such a good byeline (19:55:06) daghw: We should heed the "broken windows" effect (19:55:12) jta: ? (19:55:20) daghw: too many bugs foster indifference (19:55:37) jta: the number of entries in Jira for derby has been noticed -- and commented on (19:55:56) jta: folks asked me at the hackathon in december -- how come derby has so many bugs? (19:55:58) daghw: i didnt know... where? (19:56:01) daghw: ah ok (19:56:21) daghw: data bases are complex products... is one answer! (19:56:23) jta: people equate a jira number with "bug" so DERBY-1023 means Derby has had 1023 bugs so far (19:56:37) jta: in fact, Jira issues include features and subtasks (19:56:54) daghw: Well, MS KB numbers in the hundred thousands..! (19:57:01) marsden: There are 155 outstanding cod ebugs (19:57:04) daghw: but it coul dbe a perception problem (19:58:01) marsden: But one thing is client. It is a brand new thing as of 10.1. It needs more testing and work. (19:58:09) jta: so back to the original question ... how do we entice people to fix bugs? (19:58:44) daghw: agreed. the client code has a way to go (19:59:40) jta: is fixing client code a good entrypoint for potential contributors? (20:00:13) jta: --I think Kathey has done some good work trying to highlight what might be easy to fix (20:00:17) daghw: depends.... if error envolves communication issues, answer is probably no. (20:00:32) daghw: (DRDA is intimidating at first) (20:00:51) marsden: I think that the newcomer tasks are a good way to identify issues for newcomers. (20:01:07) daghw: yes i like that! (20:01:21) jta: could a pointer to be added to http://wiki.apache.org/db-derby/ForNewDevelopers ? (20:01:23) marsden: But it is interesting that most everyone that seems to want to contribute wants to pick up some new feature. (20:01:35) marsden: Is it not there? (20:01:40) jta: I'm looking .... (20:02:23) marsden: I shouldn't say that. Some folks have come in fixing a lot of bugs like Bryan. (20:02:36) jta: oh I see it -- the "First Code Changes" section points to the Jira "Newcomer" component (20:02:41) daghw: Some people's "itches" are not ncessarily their own at all times ;) (20:03:02) daghw: Bryan has been a great example (20:03:07) marsden: Also some bug fixes that have been contributed have been ignored by committers (20:03:27) marsden: DERBY-974 (20:03:27) daghw: i suspect he has more latitude than some though, as far as priorities. (20:04:31) marsden: Yes I guess that is the key point a lot might be related to what the paid volunteers are paid to work on. (20:04:34) jta: re DERBY-974, it looks like Deepa commented on it yesterday (20:04:51) daghw: in deed! (20:05:07) marsden: Yes. I asked her to. It has been sitting since Feb 13. (20:05:28) jta: Satheesh commented on it on Feb 14, and there it sat (20:05:41) daghw: if a bug gets forgotten, i think prompting the list again is ok (20:05:48) jta: it's definitely ok (20:05:55) marsden: There are 18 available patches. (20:06:09) jta: but new contributors might need to be encouraged to prompt the list "don't be shy! your work is valuable!" (20:06:27) marsden: And so in addition to bug review we need outstanding patch review. (20:06:46) daghw: yes, i sometimes worry newcomers can be put off (20:07:21) marsden: Dag, would you mind summarizing this conversation and posting to the list? (20:07:36) daghw: Sure, no prob. (20:07:42) marsden: Thanks! -- Dag H. Wanvik Sun Microsystems, Database Technology Group (DBTG) Haakon VII gt. 7b, N-7485 Trondheim, Norway Tel: x43496/+47 73842196, Fax: +47 73842101
