Hi,
I think Dimitry makes some valid points. I think most users understand
that the beta version is likely to have bugs, and if you tell them about
the bugs you're aware of, they'll be reasonable, especially if the
program is, by and large, stable. I also believe that any user community
is sensitive to perceived responsiveness on the part of the developers.
They are more willing to put up with problems if they see that those
problems are addressed fairly quickly.
As Dimitry points out, a lot of the bugs found by Beta users are
difficult to nail down. To me, that suggests that we may need improved
logging on Beta versions. We can, of course, disable the logging later
in the production release, and we'll need to take precautions to make
sure we don't fill up someone's disk with log files, but if we do find
that some bugs are nearly impossible to find, adding additional logging
and asking the Beta community to help is a valid approach.
I have found a number of really nice tools that we can use to help our
testing. I'm trying some of them out now, when I have time, trying to
see how best to integrate them into our development process. One of the
things I'd like to try to do, is write a new test case to match each bug
that is reported. That will help whoever is working on it fix it.
(Obviously, this won't be necessary for really minor bugs.)
I've also learned that Eclipse can integrate quite closely with
Bugzilla, making it quite easy to track bugs, who is working on them,
and the progress made towards fixing them. If we settle on Eclipse as
our standard development platform, we could set up the Bugzilla
integration. This would make it easier for bug reporters to find out
what's going on with the bug they submitted, and much of the work would
be automated.
A while back, I submitted a bug to Open Office. It was a rather obscure
bug that occurred when working with large master documents. (My novel is
organized as a master document with each chapter in a separate file.) I
didn't expect very rapid response. All my previous bug reporting
experience had been with proprietary software companies. I was extremely
pleased that I got regular emails (automated) alerting me to the
progress made on my bug and letting me know what release my bug was
scheduled to be fixed in. Two releases later, about 3 months, the bug
was fixed. It impressed me.
By using the right tools and setting up the right processes, we can do
much of what Dimitry is suggesting without making someone use up all
their free time on it. I'll be happy to help.
-----------------
On a related topic. We are badly in need of updated documentation. Do we
have any technical writers, or is it just up to us? If it's up to us, I
can put some time in on that as well.
Ray
Dimitry Polivaev wrote:
Hello,
the most important task of today is the coming release. We are
interested in getting it stable work ASAP. That's why I am starting with
the discussion about how to let the community participate on the testing.
(NB: Certainly there are two sides: the FreeMind team and the others
called community. Because the discussed topics concerns the both of
them, I write into the closed freemind-developer list for the internal
team discussion and to the open discussion forum to let the community to
participate in the discussion too.)
I start with analysis of the current project practices.
Currently the Beta versions are released rather seldom: the Beta 9 is
released on the 2007-02-21, the Beta 12 is released exactly 5 months
later on the 2007-07-21. (The Beta 10 and Beta 11 are also released in
Juli, but they have been hidden because of some bugs).
Each new Beta version encourage the beta users to submit the bug reports
so that the remaining bugs can be removed. In about two weeks after the
beta release all most important bugs are found, and the users want to
get them away. But after fixing of the annoying submitted bugs (which
takes in the most cases about a week) the freemind team does not
immediately produce the next bug fix release. Instead it works on
further testing and improvements not requested by the users.
The users get disappointed and send mails like
I am also wondering when beta 13 will be published. I still see beta
12 in the download list, and the images are still broken, and editing
a node(not long), still always has the text and cursor as black, even
if I set my selected node colors otherwise.
Such mails are not always answered but often just ignored. It probably
reduce both the will to submit bug reports and the consequently the
chances that the bugs are found and reported by the users. As a
consequence the team has to perform more tests by itself which course an
additional delay of the next beta release.
Another point: developers are generally less effective in performing the
tests than the users, because the most bugs require some special
conditions to become visible. Different users handling the program in
different ways are more likely to find a bug than a single developer
always trying more ore less the same things. The bugs happen mostly
because of some wrong dependencies between different program parts. The
program seems to work fine, and one need a coincidence to see the bug.
That's why many beta tester are more likely to find the bug than one
developer.
Now I would like to describe what could be done another way.
Instead of full concentration on the testing the team (or at least one
developer) could concentrate on producing new beta releases and on the
dialog with the community giving them response and paying the tribute to
the people submitting the bug reports.
The more people get involved in the testing and above all in submitting
the bug reports, the more bugs can be recognized and fixed.
It requires that both the team and the community accept that the bugs
are still not unlikely to be present. And I have never had an impression
that the community is not satisfied with the quality of the current beta
versions. Because the newer versions fixing the old bugs are better than
the older once, we produce a positive feedback loop: if the bugs get
fixed, submitting of the new bug reports seems to make more sense.
Further I think that releasing of the next beta does not require all of
the known bugs to be fixed. The remaining bugs could be just listed in
the version description, because it is better to work with a version
where 70% of the reported bugs have been fixed than to wait for a
version without bugs for many months.
Let me conclude.
I think that more frequently producing of the beta versions and staying
in the dialog with the community is a better way for achieving both
higher user satisfaction and less error rate in the release. The Beta
versions could be produced every month. I would like to do the job, but
first of all the proposed ideas should be discussed within the team and
within the community.
I am looking forward to your responses.
Best regards, Dimitry
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Freemind-developer mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/freemind-developer
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Freemind-developer mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/freemind-developer
