Message: A new issue has been created in JIRA.
--------------------------------------------------------------------- View the issue: http://nagoya.apache.org/jira/secure/ViewIssue.jspa?key=GUMP-21 Here is an overview of the issue: --------------------------------------------------------------------- Key: GUMP-21 Summary: FOG factor is misleading and requires improvement Type: Improvement Status: Unassigned Priority: Major Project: Gump Components: Python Assignee: Reporter: Daniel Savarese Created: Thu, 4 Mar 2004 5:21 PM Updated: Thu, 4 Mar 2004 5:21 PM Description: FOG is defined right now as: def getFOGFactor(self): return round((float(self.successes) / (float(self.failures) + float(self.prereqs))), 2) This is not helpful. Last month, jakart-oro failed to build by coincidence around the time of a Gump run and the problem was fixed before the next Gump run, yet its FOG factor dropped in half. The responsiveness to resolving build and integration problems is not taken into account. It is likely that it is impossible for FOG to mean anything because the factors that determine a project's "health" (to borrow the term used by Adam Jack in an email to the commons-dev and gump lists) are largely orthogonal. FOG should probably be a vector and if you really wanted a single number you could use the length of that vector. One of the major problems with the current factor is that a project's FOG factor continues to increase when no changes have been made to the repository. It's not meaningful to count only successful builds towards improving the factor because no changes may have been made to the code base. Ideally, you count only builds that cause no failures after a change to the code base. FOG doesn't factor in how quickly (or slowly) projects recover from build failures, and how often changes to a code base cause failures (either in itself or in other projects). Right now, a project with few code changes (say once a month) that causes a failure 50% (1 every 2 months) of the time the code is changed with a mean time of failure correction of 1 week is deemed more reliable (higher FOG factor) than a project that makes frequent changes (say every day) to its code base and causes a failure 10% of the time (1 every 10 days) with a mean time of failure correction of 1 day. That's because FOG is based almost entirely on consecutive successful Gump builds, which is not useful. Rather than propose a new meaningless way of computing FOG, I suggest FOG stop being used temporarily. It would be more useful to break out and report all of the independently useful metrics and through experience over time determine how each impacts the overall notion of FOG and eventually derive a way of condensing the information (perhaps a FOG factor, perhaps a vector) in a useful manner. Right now the state of affairs is that someone decided FOG was a cool thing, so it's being reported and people see this number and mistakenly think a project with half the FOG factor of another is less dependable and therefore avoid using it. Alternatively an explanation of what FOG is intended to compute and how it is computed should be placed on Gump reports that list FOG so that consumers of the information will not be misled. --------------------------------------------------------------------- JIRA INFORMATION: This message is automatically generated by JIRA. If you think it was sent incorrectly contact one of the administrators: http://nagoya.apache.org/jira/secure/Administrators.jspa If you want more information on JIRA, or have a bug to report see: http://www.atlassian.com/software/jira --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
