I agree it makes sense for a end user debug version to send info back
to us and and a developer debug version wouldn't do that. Perhaps we
could have a a single debug version with a developer command line flag,
or a launcher that specifies an end user flag.
In past projects I turned asserts and crashes into automated reports
and it was amazing how much I learned from those reports. Most of the
problems that weren't found during "in house" testing were related to
OS configuration differences between machines. Also, asserts found many
more problems than crashes -- perhaps because crashes were rarer. It
was not unusual to add extra debugging code to narrow down rare, non
reproducable problems, and have the info sent back identify the source
of the bug.
John
Alec Flett wrote:
The "talkback" feature that heikki describes is more of an end-user
feature. Typically used to allow developers to track crashes of their
application out in the wild, it sends a stack trace to a central server
on a crash.. (with permission of the user, sometimes even with comments
that included the steps required to make the crash happen)
The stack trace gets aggregated with other similar stack traces on the
server. Then we, as the developers, can look at the data on this
server, and look at a "top crash" report to see what crashes are seen
most often in the real world, with real users..
In our case, I think heikki is saying that for end users, if we get an
exception at the top level, where developers would normally see a
dialog box with all sorts of ugly code, end-users should see a
user-friendly error message recommending the user quits & restarts
chandler. Someday it would be nice if the stack trace were sent back to
OSAF to be combined with others so we could see a "top exception"
report and fix such errors.
Obviously, developers want the actual stack trace of the exception (an
assert being just one kind of exception) and the exception would NOT be
sent to a central talkback server, since developers are constantly
throwing exceptions during day-to-day debugging :)
Alec
John Anderson wrote:
I like the talkback feature and have used it in the past. However, I
perfer using it in the debug version since it gives you feedback about
asserts as well as crashes. I've found as many rare failures in asserts
as crashes using the talkback technique -- and Python errors are often
ignored instead of causing crashes like C++ programs. So I'd vote for
making the debug version available for customers as an optional install
and providing talkback for all errors.
John
Heikki Toivonen wrote:
Maybe you meant this message to go to dev? You sent to me only, which is
why I am replying in private. Feel free to forward/reply to dev.
My personal bias is to avoid the bug by having good testing rather than
writing code to report bugs to users, expecially because there are so
many possible bugs.
I'd rather leave this in the debug version and have some of our "power"
customers using the debug version.
My point is that we can never find all the bugs in testing. Like you
said, "there are so many possible bugs". For example, suppose a bug
shows up in 1 instance of every 1,000 Chandler installation. We'd be
extremely lucky to have enough "power" customers to uncover that. On the
other hand, if we had 10,000 or more users, the likelihood that someone
will run into it is getting significant and if we make it easy to report
it is a pretty good chance we can act on it.
Also, silently ignoring exceptions is potentially really, really bad.
(It is likely a program written in some compiled language would have
even crashed. And even in our case some users will run into bugs where
they crash the whole application.) We should give the user some kind of
indication that a problem we didn't anticipate cropped up and give them
a chance to do something about it, rather than hiding it under the covers.
And suppose we did hide the problem. It is likely that at some point in
the future the users start experiencing weird, very noticeable problems.
But if they report those problems to us, we will have a very difficult
time finding out what is causing them. If we fix those symptoms, the
original bug could still go unfixed.
Mozilla has had huge improvements in stability from the Talkback crash
reporting tool. I can remember many cases where the developers were
unable to reproduce a bug, but thousands of people were experiencing it.
The only way the bugs were found and verified to be fixed was by using
Talkback. It is an optional install, and it is optional to send the
crash reports.
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Open Source Applications Foundation "Dev" mailing list
http://lists.osafoundation.org/mailman/listinfo/dev
|
