There's also a great presentation by a Google Relevance guy: https://builtvisible.com/how-google-works/
Judgement platform is halfway down. Video of the talk: https://www.youtube.com/watch?v=iJPu4vHETXw (video is good; and a great way to spend 30min of a Friday afternoon) --justin On Fri, Apr 29, 2016 at 2:22 PM, Justin Ormont <[email protected]> wrote: > If you want to get deep in to user instructions for Discernatron, check > out Google's: > https://static.googleusercontent.com/media/www.google.com/en//insidesearch/howsearchworks/assets/searchqualityevaluatorguidelines.pdf > > It's likely a bit excessive, but there are some interesting examples and > their rating scale beginning on page 76. > > > Rating scale & short descriptions of the scale: > > > > Instructions for how to rate ambiguous queries: > > > > Their judgement UI (or one of): > > > The map at the top shows where the querying user was located. WMF could do > the same to try to better capture the intent of the user. For example a > user searching for q={national gallery}, is looking for a different > wikipage <https://en.wikipedia.org/wiki/National_Gallery_(disambiguation)> > depending on where they are located. > > > Feedback button: > > > Search engine land also has a writeup on Bing's equivalent: > http://searchengineland.com/bing-search-quality-rating-guidelines-130592 > > --justin > >> On Thu, Apr 21, 2016 at 4:13 PM, Justin Ormont <[email protected]> > wrote: > There's an example of user instructions for a judgement platform on pages > 27-27[sic] of these slides: > > http://resources.mpi-inf.mpg.de/d5/teaching/ws14_15/atir/slides/2014-atir-ch09-evaluation.pdf > > Judgment instructions generally need a couple of rounds to get right as > they are quite task specific. For example, not defining what to do with a > disambiguation page will cause some judges to rate a page a "1", and some > rate a "3". > > --justin > > >>>
_______________________________________________ discovery mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/discovery
