New submission from Ezio Melotti <ezio.melo...@gmail.com>:
The attached patch adds an autocomplete to the nosy list.
TL;DR: see screenshot.
This is how it works:
* server-side, the entries are taken from 2 places:
1) the experts index of the devguide. This is done downloading and
parsing the rst source. The new user.experts.html page returns
this data as JSON.
2) the list of committers from the Roundup database. This is done
retrieving the username and realname from the database. The new
user.devs.html page returns this data as JSON to logged in users.
The Python code to retrieve this data and convert to JSON is in a
new extension called jnosy.py.
* client-side, when the page loads, the browser:
1) checks if the two JSON files are cached in the HTML5 storage;
2) if HTML storage is not supported or if they are not cached,
it requests from the JSON from the server (through the two
user.*.html pages), and possibly caches the result;
3) the two JSON files are adapted and combined, in order to
form a list of entries for the autocomplete.
When the user writes in the nosy list, the autocomplete will list all the
elements that match divided by category.
The categories are the ones of the experts index (one per table), plus a
"developer" category.
The "developer" entries are listed first, and look like "Name Surname
(username)". Once selected, only the username is added to the nosy.
The "experts" entries list the name of the module/interest area/etc. followed
by the list of names (e.g. 'unittest: michael.foord,ezio.melotti'), and once
selected it adds all the names to the nosy.
A few more random comments about the patch:
* HTML5 storage allows the script to save data on the client's
machine and it's available on all the major browsers (FF3.5+,
IE8+, Chrome4+, Opera10.5+, Safari4+).
* If the HTML5 storage is not available the script will request
the data from the server, once for each issue page visited
(all the data are downloaded only once when the page loads).
* Both the pages return less than 5 kB of JSON, and both seem
quite fast.
* If JS is not available the autocomplete is not available, with
no other side effects.
* Having autocomplete for all the registered users, even if
technically doable, has two major problems:
1) it takes about 10s for the server to put together a JSON
object with all the names and usernames of 10k+ users and
send it to the client (that's about 200+ kB of JSON);
2) it's probably really slow to process on the client-side
and it will flood the autocomplete with lot of matching
names (I haven't tried it though).
* The user.devs.html page is not visible to not-logged in users.
* Some server-side caching should be added, so that at least the
expert page is not downloaded and parsed every time.
I'm not sure if this should be done via the Cache-Control
and/or Expires HTTP headers (that would prevent the browsers
to send requests at all), or just keep the generated JSON
in memory or in a local file, and update it, say, once a day.
* The HTML5 storage cache should also be invalidated after a
while, and the JSON requested again to the server.
* Instead of having two separates files, a single file that
returns a single JSON object with both developers and experts
can be used.
Unless there are problems with this design, I'll commit it and see what the
users think about it.
----------
assignedto: ezio.melotti
files: issue417.diff
messages: 2216
nosy: ezio.melotti
priority: feature
status: in-progress
title: Add autocomplete for the nosy list
_______________________________________________________
PSF Meta Tracker <metatrac...@psf.upfronthosting.co.za>
<http://psf.upfronthosting.co.za/roundup/meta/issue417>
_______________________________________________________Index: extensions/jnosy.py
===================================================================
--- extensions/jnosy.py (revision 0)
+++ extensions/jnosy.py (revision 0)
@@ -0,0 +1,84 @@
+"""
+This module provides two helper functions used by the Javascript autocomplete
+of the nosy list:
+ 1) a simple state machine to parse the tables of the
+ experts index and turn them in a JSON object;
+ 2) a function to get the list of developers as a JSON object;
+"""
+
+import urllib
+try:
+ import json
+except ImportError:
+ import simplejson as json
+
+url = 'http://hg.python.org/devguide/raw-file/default/experts.rst'
+
+# possible states
+no_table = 0 # not parsing a table
+table_header = 1 # parsing the header
+table_content = 2 # parsing the content
+table_end = 3 # reached the end of the table
+
+def experts_as_json():
+ """
+ Parse the tables of the experts index and turn them into a JSON object.
+ """
+ data = {}
+ table_state = no_table
+
+ try:
+ page = urllib.urlopen(url)
+ except Exception:
+ # if something goes wrong just return an empty JSON object
+ return '{}'
+
+ for line in page:
+ columns = [column.strip() for column in line.split(' ', 1)]
+ # all the tables have 2 columns (some entries might not have experts,
+ # so we just skip them)
+ if len(columns) != 2:
+ continue
+ first, second = columns
+ # check if we found a table separator
+ if set(first) == set(second) == set('='):
+ table_state += 1
+ if table_state == table_end:
+ table_state = no_table
+ continue
+ if table_state == table_header:
+ # create a dict for the category (e.g. 'Modules', 'Interest areas')
+ category = first
+ data[category] = {}
+ if table_state == table_content:
+ # add to the category dict the entries for that category
+ # (e.g.module names) and the list of experts
+ # if the entry is empty the names belong to the previous entry
+ entry = first or entry
+ names = (name.strip(' *') for name in second.split(','))
+ names = ','.join(name for name in names if '(inactive)' not in
name)
+ if not first:
+ data[category][entry] += names
+ else:
+ data[category][entry] = names
+ return json.dumps(data, separators=(',',':'))
+
+
+def devs_as_json(cls):
+ """
+ Generate a JSON object that contains the username and realname of all
+ the committers.
+ """
+ users = []
+ for user in cls.filter(None, {'iscommitter': 1}):
+ username = user.username.plain()
+ realname = user.realname.plain(unchecked=1)
+ if not realname:
+ continue
+ users.append([username, realname])
+ return json.dumps(users, separators=(',',':'))
+
+
+def init(instance):
+ instance.registerUtil('experts_as_json', experts_as_json)
+ instance.registerUtil('devs_as_json', devs_as_json)
Index: html/issue.item.js
===================================================================
--- html/issue.item.js (revision 88880)
+++ html/issue.item.js (working copy)
@@ -62,3 +62,153 @@
}
});
})
+
+$(document).ready(function() {
+ /* Add an autocomplete to the nosy list that searches the term in two
lists:
+ 1) the list of users (both the user and the real name);
+ 2) the list of experts in the devguide;
+ Note: most of this code comes from the "categories" and "multiple
values"
+ examples in http://jqueryui.com/demos/autocomplete/. */
+
+ // create a custom widget to group the entries in categories
+ $.widget("custom.catcomplete", $.ui.autocomplete, {
+ _renderMenu: function(ul, items) {
+ var self = this, current_category = "";
+ // loop through the items, adding a <li> when a new category is
+ // found, and then render the item in the <ul>
+ $.each(items, function(index, item) {
+ if (item.category != current_category) {
+ ul.append("<li class='ui-autocomplete-category'>" +
item.category + "</li>");
+ current_category = item.category;
+ }
+ self._renderItem(ul, item);
+ });
+ }
+ });
+
+ function split(val) {
+ return val.split(/,\s*/);
+ }
+ function extractLast(term) {
+ return split(term).pop();
+ }
+
+ // this will be called once we have retrieved the data
+ function add_autocomplete(data) {
+ $("[name=nosy]")
+ // don't navigate away from the field on tab when selecting an item
+ .bind("keydown", function(event) {
+ if (event.keyCode === $.ui.keyCode.TAB &&
+ $(this).data("autocomplete").menu.active) {
+ event.preventDefault();
+ }
+ })
+ .catcomplete({
+ minLength: 2, // this doesn't seem to work
+ delay: 0,
+ source: function(request, response) {
+ // delegate back to autocomplete, but extract the last term
+ response($.ui.autocomplete.filter(
+ data, extractLast(request.term)));
+ },
+ focus: function() {
+ // prevent value inserted on focus
+ return false;
+ },
+ select: function(event, ui) {
+ var terms = split(this.value);
+ // remove the current input
+ terms.pop();
+ // add the selected item
+ terms.push(ui.item.value);
+ // add placeholder to get the comma at the end
+ terms.push("");
+ this.value = terms.join(",") ;
+ return false;
+ }
+ });
+ }
+
+
+ // check if we have HTML5 storage available
+ try {
+ var supports_html5_storage = !!localStorage.getItem;
+ } catch(e) {
+ var supports_html5_storage = false;
+ }
+ alert('storage ' + supports_html5_storage);
+
+ // this object receives the entries for the devs and experts and
+ // when it has both it calls add_autocomplete
+ var data = {
+ devs: null,
+ experts: null,
+ add: function(data, type) {
+ // type is either 'devs' or 'experts'
+ this[type] = data;
+ if (this.devs && this.experts)
+ add_autocomplete(this.devs.concat(this.experts))
+ }
+ };
+
+ /* Note: instead of using a nested structure like:
+ {"Platform": {"plat1": "name1,name2", "plat2": "name3,name4", ...},
+ "Module": {"mod1": "name1,name2", "mod2": "name3,name4", ...},
+ ...}
+ (i.e. the same format sent by the server), we have to flat it down and
+ repeat the category for each entry, because the autocomplete wants a
+ flat structure like:
+ [{label: "plat1: name1,name2", value: "name1,name2", category:
"Platform"},
+ {label: "plat2: name3,name4", value: "name3,name4", category:
"Platform"},
+ {label: "mod1: name1,name2", value: "name1,name2", category: "Module"},
+ {label: "mod2: name3,name4", value: "name3,name4", category: "Module"},
+ ...].
+ Passing a nested structure to ui.autocomplete.filter() and attempt
+ further parsing in _renderMenu doesn't seem to work.
+ */
+ function get_json(file, callback) {
+ // Get the JSON from either the HTML5 storage or the server.
+ // file is either 'devs' or 'experts',
+ // the callback is called once the json is retrieved
+ var json;
+ if (supports_html5_storage &&
+ (json = localStorage[file]) != null) {
+ // if we have HTML5 storage and already cached the JSON, use it
+ callback(JSON.parse(json), file);
+ }
+ else {
+ // if we don't have HTML5 storage or the cache is empty, request
+ // the JSON to the server
+ $.getJSON('user?@template='+file, function(rawdata) {
+ var objects = []; // array of objs with label, value, category
+ if (file == 'devs') {
+ // save devs as 'Name Surname (user.name)'
+ $.each(users, function(index, names) {
+ objects.push({label: names[1] + ' (' + names[0] + ')',
+ value: names[0], category: 'Developer'});
+ });
+ }
+ else {
+ // save experts as e.g. 'modname: user1,user2'
+ $.each(rawdata, function(category, entries) {
+ $.each(entries, function(entry, names) {
+ objects.push({label: entry + ': ' + names,
+ value: names, category: category});
+ });
+ });
+ }
+ // cache the objects if we have HTML5 storage
+ if (supports_html5_storage)
+ localStorage[file] = JSON.stringify(objects);
+ callback(objects, file);
+ });
+ }
+ }
+
+ // request the JSON. This will get it from the HTML5 storage if it's there
+ // or request it to the server if it's not, The JSON will be passed to the
+ // data object, that will wait to get both the files before calling the
+ // add_autocomplete function.
+ get_json('experts', data.add);
+ get_json('devs', data.add);
+});
Index: html/user.experts.html
===================================================================
--- html/user.experts.html (revision 0)
+++ html/user.experts.html (revision 0)
@@ -0,0 +1,5 @@
+<tal:block content="python:utils.experts_as_json()">
+{"Platform":{"platname":"name1,name2",...},
+ "Module":{"modname":"name1,name2",...},
+ ...}
+</tal:block>
Index: html/user.devs.html
===================================================================
--- html/user.devs.html (revision 0)
+++ html/user.devs.html (revision 0)
@@ -0,0 +1,4 @@
+<tal:block tal:condition="context/is_view_ok"
+ content="python:utils.devs_as_json(context)">
+ [["username1","Real Name1"],["username2", "Real Name2"],...]
+</tal:block>
Index: html/issue.item.html
===================================================================
--- html/issue.item.html (revision 88880)
+++ html/issue.item.html (working copy)
@@ -11,8 +11,11 @@
</title>
<metal:slot fill-slot="more-javascript">
-<script type="text/javascript"
src="http://ajax.googleapis.com/ajax/libs/jquery/1.6.1/jquery.min.js";></script>
+<script type="text/javascript"
src="http://ajax.googleapis.com/ajax/libs/jquery/1.6.2/jquery.min.js";></script>
+<script type="text/javascript"
+src="http://ajax.googleapis.com/ajax/libs/jqueryui/1.8.15/jquery-ui.js";></script>
<script type="text/javascript" src="@@file/issue.item.js"></script>
+<link rel="stylesheet" type="text/css"
href="http://ajax.googleapis.com/ajax/libs/jqueryui/1.8/themes/smoothness/jquery-ui.css";
/>
</metal:slot>
<tal:block metal:fill-slot="body_title">
Index: html/style.css
===================================================================
--- html/style.css (revision 88880)
+++ html/style.css (working copy)
@@ -513,3 +513,16 @@
.calendar_display .today {
background-color: #afafaf;
}
+
+.ui-autocomplete-category {
+ font-weight: bold;
+ padding: 0 .2em;
+ line-height: 1.2;
+}
+
+.ui-autocomplete {
+ font-size: 75% !important;
+ max-height: 25em;
+ max-width: 20em;
+ overflow: auto;
+}
_______________________________________________
Tracker-discuss mailing list
Tracker-discuss@python.org
http://mail.python.org/mailman/listinfo/tracker-discuss
