This is the WowBar bug tracking system, which probably should have a better name. I was talking about it at Wiki Wednesday, and some people expressed some interest.
I've put it up for download, including a list of all of its bugs, at http://pobox.com/~kragen/sw/wowbarbts-0.tar.gz as well as in this email. Here's my README file: To get started running wowbarbts ================================ 1. Install Python, Twisted, and twisted.web. I'm using Python 2.4, Twisted 2.4.0, and Twisted.web 0.6.0, but somewhat older versions probably work too. 2. Run "./run.sh". If you're not on Unix, you're on your own for this step. You should see a message that says "On 7777". That means it's accepting connections on port 7777. 3. Go to http://localhost:7777/ and start playing. What is wowbarbts? ================== This is the bug/issue tracking system I wrote for WowBar while I was an employee of CommerceNet, so it's copyrighted by CommerceNet, who licensed it under the GNU GPL along with the rest of WowBar. It's really a kind of Wiki. It's also included in the WowBar distributions, although I don't know if any of those are still around. I'm packaging this one up separately to make it easier for people to try it out. It has several interesting features that make it different from other bug trackers and Wikis: * Unlike other Wikis, each page has multiple fields. Your default view is an editable table whose columns are those fields, in which you can edit many fields at a time. You can add new fields by adding them to the tabular view, then typing into them. * Unlike other bug-tracking systems, each bug is stored in a separate text file, which automatically provides a certain amount of integration with your source-control system --- the idea is that you can mark a bug as "fixed" or "closed" or whatever in the same patch that includes the fix, so anyone who merges that patch in will also merge in your change to the bug status. * To support this kind of decentralized development, new bugs have randomly-generated ids. * Unlike other bug-tracking systems, very little structure is defined up-front; any bug can contain any combination of fields, any field can contain any value, you can add or delete fields on the fly, and you can query on fields that don't exist yet. * Unlike other Wikis, editing is nearly modeless --- you click on text to edit it, and it stays in pretty much the same place. Most text you see is editable this way. It's very straightforward to use it for a variety of simple "database" applications that require slightly more structure than a normal WikiWikiWeb: signing up BoF rooms at a conference, planning a shopping trip that involves going to multiple grocery stores, planning a potluck dinner (to make sure that not everyone brings potato salad), budgeting a vacation, and so on. Setting up wowbarbts for your project ===================================== The shortest path is as follows: 1. Create an empty directory 2. Copy btsbts/new-bug-template and btsbts/bts.conf into it 3. cd into it 4. Run bts.py there. 5. Go to http://localhost:7777/ and start entering bugs. bts.conf configures a few basic things, including most importantly tells wowbarbts which file to get defaults from, which is normally called "new-bug-template". If it's missing, you need to have at least one existing bug to serve as a template for new bugs. The stuff in new-bug-template is important for three reasons: 1. The stuff in it is what you will see every time you enter a new bug. This is a good place to describe the purpose and meaning of each field. 2. If you put a field in there, any bugs that don't specify that field themselves (say, because you created them before you put the field there, or because you deleted the field from them with Emacs) will inherit that field from new-bug-template. 3. Generally, at the moment, due to UI bugs, it's kind of a pain to change a field between being multiline and non-multiline, and it's kind of a pain if you include multiline fields in a query screen. So it's a good idea to provide a multiline default value for fields that you want to be multiline. You can edit new-bug-template either by going to http://localhost:7777/new-bug-template or with a text editor, which is more useful because it lets you add and delete fields. bts.conf configures a few other things --- the title of your bug tracker, the order of fields displayed in a bug, the mouseover link title text that appears over a link to a bug, and the list views linked at the top of the bug list that you see by default. The default behavior is as if the following were in bts.conf: views: default sec=component&col=title:50+component:10+state:6 ::: by milestone sec=milestone&col=title:50+state:6+milestone+commitment:4+actual:4&tot=commitment+actual The "::: " is how it stores continuation lines of multiline fields in its file format. Further documentation is in the first 90 lines of bts.py. Bugs ==== The example data here is actually a list of the known bugs and desired features in this program, so you can peruse there to your heart's content. Those in "state: closed" have already been fixed. More globally, though, it needs deeper integration with a source control system so that it can answer questions like the following: - What bugs were state=open in release 0.9 that are state=closed in release 0.95? - What release was this bug last updated in? - Who checked in the last change to this bug? - Who checked in the change that set the "state" of this bug to "open"? - What was the identifier of the patch that last changed this bug (so I can see what code changes accompanied it)? Or what are the identifiers of all the patches that changed this bug? - What did this field say in previous versions? Kragen Sitaker, 2007-08-09 (end of README file) Here's the bts.conf: defaults: new-bug-template field order: title estimate body link title: %(title)s (%(actual)s/%(estimate)s, %(state)s) title: Bug tracking system bug tracking system And here's the new-bug-template: actual: body: (describe the bug at greater length here, ::: including what you did, what happened, what you ::: think should happen instead, and ideally how to reproduce.) estimate: 40 milestone: 500 state: open title: new bug title (a brief, four-to-ten-word description) Here's bugpage.js, which is a little bit of JavaScript that is referenced from each bug page: // -*- mode: c++ -*- function contains(list, item) { for (var ii = 0; ii < list.length; ii++) if (list[ii] == item) return true return false } function replace_with_textarea(elem, textarea) { // XXX try to match click point return function (ev) { if (ev.target != elem) return elem.style.display = 'none' textarea.style.display = '' textarea.focus() } } function next_element(elem) { var next = elem.nextSibling while (next.nodeType != next.ELEMENT_NODE) next = next.nextSibling return next } function make_replaceable(elem) { var textarea = next_element(elem) elem.addEventListener("click", replace_with_textarea(elem, textarea), true) textarea.style.display = 'none' } function elements_having_class(className) { var rv = [] for (var ii = 0; ii < document.all.length; ii++) { var elem = document.all[ii] if (elem.nodeType != elem.ELEMENT_NODE) continue if (contains(elem.className.split(/\s+/), className)) rv.push(elem) } return rv } function foreach(list, func) { for (var ii = 0; ii < list.length; ii++) func(list[ii]) } function init() { foreach(elements_having_class('editable'), make_replaceable) } Finally, here's the program bts.py itself: #!/usr/bin/python # WowBar bug tracking system # Copyright (C) 2006 CommerceNet # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA """Web-based bug tracking system. The contents of all bugs are kept in text files in a specified directory. Updating the bug atomically updates the text files. Every file in the directory whose name begins with 'bug', ends with '.txt', and is otherwise ASCII-alphanumeric, is considered to be a bug report. Optionally the bug tracker will commit every change made through its web user interface to a version control system. Because version control systems exist, the bug tracker doesn't feel that it should track changes over time or authorship itself, leaving that up to the version control system. The bug files are free-form sets of name-value pairs. The filename of each bug controls its URL. Bug files follow the following simple line-oriented format. Each line is either a name-value pair (syntax: NAME ': ' VALUE '\n', where the NAME contains no colons) or a continuation line (syntax: '::: ' VALUE '\n') which represents another newline-separated line belonging to the value of the most recent name. Values ending in a newline therefore have a representation ending in '::: \n'. If something, such as you editing a bug file with Emacs or your version control system handling a conflict, inserts lines that don't match one of these formats, the bug tracking system will only display the bug as raw text in a textarea, and will display it as the first item on all reporting displays, so you'll have to fix it by hand. One piece of data is not stored in the bug file: 'last modified' --- instead we rely on the filesystem, in order to avoid spurious version-control system conflicts. Each bug file can only contain a particular name once, so you'll have to use something such as commas or newlines to separate multiple values, and the names are normally alphabetized to keep the order consistent and make life easier for the veral contortion system. (Incomplete; most things that follow are wishful thinking.) Report formats are defined by URLs. There are five variables in a report format: - inclusion criteria 'q' (default, include everything) - sorting criteria 'sort' (default, sort by 'last modified' descending) - columns 'col' (default, a column for any field that occurs in all included bugs) - section criteria 'sec' (default, no sections) which are sorting criteria each of whose new values displays as a header of a section - details 'etc' to be displayed below the summary line (default, all fields omitted from any bug or containing newlines) The bug tracking system is configured by an optional file called 'bts.conf' in the same directory as the bugs; its format is the same as the format used by the bugs. It has the following variables: - 'title': title for main bug list - 'default report': lists the query to be performed when you hit the BTS front page, e.g. q=open+in+status&sec=severity - 'patterns': lists a bunch of regular expressions and URL patterns they map to. - 'defaults': has the name of a file to be copied for the creation of new bugs, with default field values. In general these default field values will be used whenever a bug has no value for the field. Change notification is accomplished as follows. There's a field entitled 'nosy' containing the names of people to be notified of changes to a particular bug, and a field entitled 'not-notified' containing the (alphabetized) names of people who have not been notified of the current state of the bug. Edits through the web UI overwrite the 'not-notified' field with the 'nosy' field. """ import twisted.web.resource, twisted.web.server, twisted.internet.reactor, sys import os, cgi, twisted.web.static, stat, twisted.web.error, sha, time, re import string, urllib # completed story list, in some vague kind of order: # D run web server # D read file format # D display single bug read-only # D display all bugs in title list read-only # D add bug links to table # D put all known WowBar bugs and planned tasks into the system # D make title list editable # D make single bug editable # D support creating new bugs from bts.conf 'new bug template' # D add order for fields on bug page # D remove duplication on bug save # D add default sectioning for title list # D support sectioning title list by user-specified field # D make title list display specified fields instead of just title # D it is now possible in the list view to add new fields by viewing # nonexistent fields # D make POST redirects not lose query params # D change sibLink (which returns a relative URL) to something better, maybe # use URLPath.sibling() # D display state as well as component in bug list # D insert customizable hyperlinks into text; maybe use Markdown or something? # D currently indentation in text is ugly # D add support for section totals # D support word searches. (Don't need an index any time soon.) css_link = ' <link href="bugs.css" rel="stylesheet" />\n' def sorted(alist, key=lambda x: x, reverse=True): rv = [(key(x), x) for x in alist] rv.sort() if reverse: rv.reverse() return [v for k, v in rv] class BugResource(twisted.web.resource.Resource): def __init__(self, dirname, bugname, buglist): twisted.web.resource.Resource.__init__(self) self.bug = buglist.open_bug(bugname) def render_GET(self, request): return self.bug.as_html() def render_POST(self, request): updates = self.get_updates(self.bug, request) if updates: self.bug.update(**updates) request.redirect(request.prePathURL()) return '' def get_updates(self, oldbug, request): updates = {} for name, value in request.args.items(): newvalue = value[0].replace('\r\n', '\n') assert '\r\n' not in newvalue if newvalue != oldbug.get(name, None): updates[name] = newvalue return updates class NewBugMaker(BugResource): def __init__(self, buglist): twisted.web.resource.Resource.__init__(self) self.buglist = buglist def render_POST(self, request): newbug = Record([]) newbug.update(self.get_updates(newbug, request)) newbugname, newbugpath = self.buglist.get_new_bugname() newbug.overwrite_filename(newbugpath) request.redirect(str(request.URLPath().sibling(newbugname))) return '' def render_GET(self, request): return self.buglist.getdefaults().as_html() class Query: def __init__(self, searchstring): self.terms = [term for term in searchstring.split() if term] def matches(self, bug): for term in self.terms: if term.startswith('-'): if term[1:] in bug: return False else: if term not in bug: return False return True class BugListColumn: def __init__(self, spec, totfields): specbits = spec.split(':') if len(specbits) == 2: self.field, self.width = specbits else: self.field, self.width = specbits[0], 10 if totfields: self.totfields = totfields[0].split() else: self.totfields = [] def display(self, bug): value = cgi.escape(bug.get(self.field, ''), 1) return ' <td><input name="%s:%s" value="%s" size="%s" class="simple"/>' % ( self.field, bug['name'], value, self.width) + ( '<input name="old:%s:%s" value="%s" type="hidden" /></td>' % ( self.field, bug['name'], value)) def needs_total(self): return self.field in self.totfields def numeric_value(self, bug): try: return float(bug[self.field]) except: return 0.0 def total(self, bugs): return sum([self.numeric_value(bug) for bug in bugs]) def __repr__(self): return '<BugListColumn %s:%s>' % (self.field, self.width) bugname_re = r'bug[a-zA-Z0-9]+\.txt' class BugList(twisted.web.resource.Resource): def __init__(self, bugdir): twisted.web.resource.Resource.__init__(self) self.bugdir = bugdir def getparam(self, paramname): try: btsconf = self.open_bug('bts.conf') except: return None return btsconf.get(paramname, None) def open_bug(self, bugname): return Bug(self.bugdir, bugname, self) def get_new_bugname(self): ii = 0 while 1: ss = sha.sha('%s %s %s' % (os.getpid(), time.time(), ii)) bugname = 'bug%s.txt' % ss.hexdigest()[:4] # 32 bits path = os.path.join(self.bugdir, bugname) # If 32 bits were really enough to prevent collisions, # this would be unnecessary; this reduces the collisions # to the outstanding unsynchronized changes: if not os.path.exists(path): return bugname, path ii += 1 def bugs(self): for fname in os.listdir(self.bugdir): if re.match(bugname_re + r'\Z', fname): yield self.open_bug(fname) def all_fields(self): rv = {} for bug in self.bugs(): for field in bug.keys(): rv.setdefault(field, 0) rv[field] += 1 return rv def display_bug(self, bug, fields): name = bug['name'] return ('<tr><td><a href="%s" title="%s">%s</a></td>\n' % (name, cgi.escape(bug.link_title(), 1), name) + '\n'.join([field.display(bug) for field in fields]) + '</tr>') def calc_totals(self, sec, fields): rv = ['<th>totals</th>'] need_totals_row = False for field in fields: if field.needs_total(): rv.append('<td>%s</td>' % field.total(sec)) need_totals_row = True else: rv.append('<td></td>') if need_totals_row: return '<tr>%s</tr>\n' % ''.join(rv) else: return '' def viewlink(self, view): # XXX this should have a regression test written for it! spindex = view.rfind(' ') if spindex == -1: spindex = len(view) text = view[:spindex] url = urllib.quote(view[spindex+1:], safe='&=+:') return '<a href="?%s">%s</a>' % (cgi.escape(url, quote=1), cgi.escape(text)) def render_GET(self, req): tot = req.args.get('tot') fields = [BugListColumn(x, tot) for x in req.args.get('col', ['title:50 component:10 state:6'])[0].split()] sec = req.args.get('sec', ['component'])[0] secs = {} query = Query(req.args.get('q', [''])[0]) for bug in self.bugs(): if query.matches(bug): secs.setdefault(bug.get(sec, ''), []).append(bug) blist = '\n'.join(['<tr><th colspan="%d">%s</th></tr>' % ( 1 + len(fields), (name or 'misc') ) + '\n'.join([self.display_bug(bug, fields) for bug in sec]) + self.calc_totals(sec, fields) for name, sec in sorted(secs.items())]) title = (self.getparam('title') or 'Bugs') views = (self.getparam('views') or 'default sec=component&col=title:50+component:10+state:6\n' + 'by milestone sec=milestone&col=title:50+state:6+milestone+' + 'commitment:4+actual:4&tot=commitment+actual').split('\n') vtxt = ('<p>See ' + ', or '.join([self.viewlink(view) for view in views]) + '.</p>') text = (vtxt + '<form method="get"><p><input name="q" />\n' + '<input type="submit" value="Search" /></p></form>') return '<html><head>%s</head><body>%s</body></html>\n' % ( '<title>%s</title>\n%s\n' % (title, css_link), '<h1>%s</h1>' % title + text + '<form method="POST"><table>' + blist + '</table>\n<input type="submit" value="Save">' + '</form>\n<a href="new">New</a>\n') def getChild(self, path, request): assert '/' not in path try: return BugResource(self.bugdir, path, self) except OSError: return twisted.web.error.NoResource("No such bug.") def render_POST(self, req): for name, value in req.args.items(): parts = name.split(':') if len(parts) == 2: field, bugname = parts newvalue = value[0] if req.args['old:' + name][0] != newvalue: bug = self.open_bug(bugname) bug.update(**{field: newvalue}) # XXX there must be a better way than this: req.redirect(str(req.URLPath().click(req.uri))) return '' def getdefaults(self): templatefile = self.getparam('defaults') if templatefile: bug = self.open_bug(templatefile) else: bugs = list(self.bugs()) bug = bugs[-1] bug['title'] = 'new bug' return bug class Record: def __init__(self, infile, **kwargs): self.kwargs = kwargs.copy() # save a copy to keep them separate self.values = kwargs currentname = None for line in infile: assert line.endswith('\n') if line == '\n': continue if line.startswith('::: '): self.values[currentname] += '\n' + line[4:-1] else: # this will raise an exception if there's no : currentname, value = line.split(': ', 1) self.values[currentname] = value[:-1] def __getitem__(self, name): return self.values[name] def __setitem__(self, name, val): self.values[name] = val def get(self, name, default): return self.values.get(name, default) def keys(self): return self.values.keys() def items(self): return self.values.items() def update(self, update): return self.values.update(update) def editable(self, name): return name not in self.kwargs def serialized(self): items = [(name, value.replace('\n', '\n::: ')) for name, value in self.items() if self.editable(name)] return ''.join(["%s: %s\n" % (name, value) for name, value in sorted(items, reverse=False)]) def overwrite_filename(self, filename): tmpname = filename + '.tmp' f = file(tmpname, 'w') f = file(tmpname, 'w') f.write(self.serialized()) f.close() os.rename(tmpname, filename) class Bug: def __init__(self, dirname, filename, buglist): self.pathname = os.path.join(dirname, filename) updatetime = os.stat(self.pathname)[stat.ST_MTIME] updated = time.strftime("%Y-%m-%d %H:%M", time.localtime(updatetime)) self.values = Record(file(self.pathname), name=filename, updated=updated) self.buglist = buglist self.patterns = { '\n': '<br />\n', '\\b(https?://[^ \\n()<>\'"]*[^ \\n()<>,.\'"])': r'<a href="\1">\1</a>', r'\b(%s)\b' % bugname_re: lambda mo: self.buglink(mo.group(1)), r'(?m)^(\s+.*)$': lambda mo: self.indent(mo.group(1)), '\s---\s': ' — ', } def __contains__(self, term): """For string searches. Not related to normal mapping 'in'.""" for name, value in self.values.items(): if term.lower() in value.lower(): return True return False def buglink(self, bugname): return '<a href="%s" title="%s">%s</a>' % ( bugname, self.other_bug_link_title(bugname), bugname) def other_bug_link_title(self, bugname): try: otherbug = self.buglist.open_bug(bugname) return cgi.escape(otherbug.link_title(), quote=1) except: type, value, tb = sys.exc_info() return "surprise: %s" % (value,) def link_title(self): template = self.buglist.getparam('link title') if template is None: template = '%(title)s' return template % self def indent(self, line): wsp = 0 for char in line: if char in string.whitespace: wsp += 1 else: break # XXX I feel dirty! return ' ' * wsp + line[wsp:] def __getitem__(self, name): try: return self.values[name] except KeyError: defaults = self.buglist.getdefaults() if name in defaults.keys(): return defaults[name] else: raise def get(self, name, default): try: return self[name] except KeyError: return default def keys(self): return self.values.keys() # XXX note that this does not propagate back to the filesystem def __setitem__(self, name, val): self.values[name] = val def add_new_field_html(self): all_fields = self.buglist.all_fields() for key in self.keys(): del all_fields[key] return ('<select name="new_field">\n' + ' <option value="">Add new field...</option>\n ' + '\n '.join(['<option value="%s">%s</option>' % (field, field) for field in sorted(all_fields.keys(), key=all_fields.get)])) def display_text(self, text): rv = cgi.escape(text) for pat, repl in self.patterns.items(): rv = re.compile(pat).sub(repl, rv) return rv def as_html(self): order = self.buglist.getparam('field order') if order: order = order.split() else: order = [] def field_order((field, value)): try: return len(order) - order.index(field) except ValueError: return -1 rv = [] for name, value in sorted(self.values.items(), key=field_order): if not self.values.editable(name): rv.append("<p><b>%s</b>: %s</p>" % (name, cgi.escape(value))) continue valuelines = value.split('\n') if len(valuelines) > 1: textarea = ('<textarea name="%s" rows="%d" cols="80" ' + 'wrap="hard" style="display: none">' + '%s</textarea>') % (name, len(valuelines), cgi.escape(value)) display = '<p class="editable">%s</p>\n' % self.display_text(value) valuepart = display + textarea else: valuepart = ('<input name="%s" value="%s" size="80" class="simple">' % (name, cgi.escape(value, 1))) rv.append('<h3>%s</h3>\n%s' % (name, valuepart)) # XXX doesn't work yet: #rv.append(self.add_new_field_html()) # XXX html escaping problem title = '%s (%s)' % (self['title'], self['name']) form = ('<form method="POST">%s\n' + '<input type="submit" value="Save" />\n' + '</form>') % ''.join(rv) js_link = ' <script src="bugpage.js"></script>\n' return ('<html><head>\n <title>%s</title>\n%s</head><body %s>' + '<h1>%s</h1>\n%s</body></html>') % (title, css_link + js_link, 'onload="init()"', title, form) def update(self, **kwargs): self.values.update(kwargs) self.values.overwrite_filename(self.pathname) def ok(a, b): assert a == b, (a, b) def test(): bug1text = ["title: This is a bug\n", "severity: bug\n", "space: \n", "extro: field\n"] rec = Record(bug1text) ok(rec['title'], 'This is a bug') ok(rec['severity'], 'bug') ok(rec['space'], ' ') keys = rec.keys() keys.sort() ok(keys, ['extro', 'severity', 'space', 'title']) rec = Record(['title: This bug\n', '::: has continuation lines\n', 'severity: wish\n']) ok(rec['title'], 'This bug\nhas continuation lines') ok(rec['severity'], 'wish') # named extras rec = Record(bug1text, name="bug305.txt") ok(rec['title'], 'This is a bug') ok(rec['name'], "bug305.txt") ok(rec.serialized(), ("extro: field\n" + "severity: bug\n" + "space: \n" + "title: This is a bug\n")) # serialization of continuation lines. # # I had the expected bug here at first and it corrupted every bug # file; I had to svn revert them. ok(Record(["a: b\n", "::: c\n", "::: d\n"]).serialized(), "a: b\n::: c\n::: d\n") test() bugs_css = """ input.simple, textarea, p { border: 0; font: inherit; margin: 0 1ex } body { margin: 0 } h1, h3, th { font-family: sans-serif; font-weight: normal; margin: 0 } h1 { text-align: right; font-size: 16pt; background-color: #aaf; padding: 0.25em; } h3 { font-size: 13pt; padding: 0.25em 1ex; } h3, b, th { background-color: #ddf } td { padding: 0 1ex } """ def main(port=7777): # Unreal Tournament port root = BugList('.') root.putChild('', root) root.putChild('bugs.css', twisted.web.static.Data(bugs_css, 'text/css')) home = os.path.dirname(__file__) bugpage_js = os.path.join(home, 'bugpage.js') bugpage = twisted.web.static.File(bugpage_js) bugpage.openForReading() # to ensure it exists root.putChild('bugpage.js', bugpage) root.putChild('new', NewBugMaker(root)) twisted.internet.reactor.listenTCP(port, twisted.web.server.Site(root)) print "On", port twisted.internet.reactor.run() if __name__ == '__main__': if len(sys.argv) > 1: main(int(sys.argv[1])) else: main()
