It is a manual rating system, which can be used for quality improvements.
Cost-less rating systems have a inherit problem with gaming. That can be
counteracted with rating of the raters, often called meta rating. You use
reputation of the raters by observing disagreement and then use that to
calculate a trust when they rate articles or statements about the articles.

If there is a cost to do rating the quality of the rating will usually go
up. If there is no cost with doing a rating the quality goes down, as it
become easier to game the system.

Note that manual rating only works for subjective quality assessment.

On Sun, Apr 16, 2017 at 10:04 PM, James Salsman <jsals...@gmail.com> wrote:

> John, the AROWF project GSoC student implemented your proposal last year:
>
> https://github.com/priyankamandikal/arowf/blob/master/backlog.py
>
> She also used WikiWho to suggest review of out-of-date passages, and both
> categories and readability metrics to suggest review of unclear passages:
>
> https://github.com/priyankamandikal/arowf/blob/master/recent_script.py
>
>
> https://github.com/priyankamandikal/arowf/blob/master/copy_edit.py
>
> This year she has agreed to co-mentor a voice-interactive tutorial system
> for instructing on the use of her project, with which we plan to
> simultaneously coach speech pronunciation.
>
> On Sun, Apr 16, 2017 at 11:23 AM John Erling Blad <jeb...@gmail.com>
> wrote:
>
> > I wrote a proposal a few years ago on how we could identfy some types of
> > bias. The idea was to compare ranking of pageviews, and notify other
> > projects about missing articles. I don't think anyone has done any
> followup
> > om that
> >
> > Den søn. 16. apr. 2017, 19.12 skrev Gerard Meijssen <
> > gerard.meijs...@gmail.com>:
> >
> > > Hoi,
> > > Humans are overrated. I saw this answer on Facebook [1] and [2] compare
> > the
> > > two and tell me why we accept the bias in our editors. Why are we
> > satisfied
> > > with what we write about when there is more to inform about. Remember
> > what
> > > we aim to achieve. It does not say text, it says share the sum of all
> > > knowledge.
> > > Thanks,
> > >         GerardM
> > >
> > > [1]
> > >
> > >
> > https://upload.wikimedia.org/wikipedia/commons/0/07/
> Geotagged_articles_in_enWP_map_RENDER_small.png
> > > [2]
> > >
> > >
> > https://upload.wikimedia.org/wikipedia/commons/2/2b/
> WorldmapGeonamesallCountries.jpg
> > >
> > > On 16 April 2017 at 18:59, Ziko van Dijk <zvand...@gmail.com> wrote:
> > >
> > > > Hello John,
> > > >
> > > > Article quality is an interesting subject. I guess that it depends
> > > > extremely on what is the scientific discipline you come from, and
> what
> > > > questions you want to be answered. A linguist will have a very
> > different
> > > > approach than a computer scientist, for example. If you ask me, only
> a
> > > > human being can judge an article if it comes to content quality and
> > > textual
> > > > quality, by the way. Maybe you want to elaborate on what are your
> > > > questions?
> > > >
> > > > Kind regards
> > > > Ziko
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > 2017-04-16 9:44 GMT+02:00 Gerard Meijssen <gerard.meijs...@gmail.com
> >:
> > > >
> > > > > Hoi,
> > > > > How can you check for consistency when you are not able to
> appreciate
> > > if
> > > > > certain facts (like date of death) exist and are the same? What can
> > you
> > > > say
> > > > > about sources when some Wikipedias insist on sources in their own
> > > > language
> > > > > and sources in other languages you cannot read? How do you check
> for
> > > > > consistency when we have over 280 Wikipedias with possible content?
> > > > >
> > > > > Do know that only Wikidata approaches a state where it knows about
> > all
> > > > our
> > > > > projects and we have not, to the best of my knowledge, assessed
> what
> > > the
> > > > > quality of Wikidata is on interwiki links.. Case in point, I fixed
> an
> > > > error
> > > > > today about a person that was said to be dead because a Commons
> > > category
> > > > > was not correctly linked.
> > > > >
> > > > > When you study the consistency of English Wikipedia only, you only
> > add
> > > to
> > > > > the current bias in research.
> > > > >
> > > > > When you want to know about the half life of an error, you can find
> > in
> > > > the
> > > > > history when for instance a date was mentioned for a first time and
> > > find
> > > > > the same date in another language. This is not trivial as the
> format
> > > of a
> > > > > language is diverse think Thai for instance.
> > > > > Thanks,
> > > > >         GerardM
> > > > >
> > > > > On 16 April 2017 at 02:08, John Erling Blad <jeb...@gmail.com>
> > wrote:
> > > > >
> > > > > > This is more about checking consistency between projects. It is
> > > > > > interesting, but not quite what I was asking about. It is very
> > > > > interesting
> > > > > > if it would be possible to say something about half-life of an
> > error.
> > > > I'm
> > > > > > pretty sure this follows number of page views if ordinary
> logged-in
> > > > > editing
> > > > > > is removed.
> > > > > >
> > > > > > On Sun, Apr 16, 2017 at 12:08 AM, Gerard Meijssen <
> > > > > > gerard.meijs...@gmail.com
> > > > > > > wrote:
> > > > > >
> > > > > > > Hoi,
> > > > > > > Would checking if a date of death exists in articles be of
> > interest
> > > > to
> > > > > > you.
> > > > > > > The idea is that Wikidata knows about dates of death and for
> > > "living
> > > > > > > people" the fact of a death should be the same in all projects.
> > > When
> > > > > the
> > > > > > > date of death is missing, there is either an issue at Wikidata
> > (not
> > > > the
> > > > > > > same precision is one) or at a project.
> > > > > > >
> > > > > > > When a difference is found, the idea is that it is each
> projects
> > > > > > > responsibility to do what is needed. No further automation.
> > > > > > > Thanks,
> > > > > > >        GerardM
> > > > > > >
> > > > > > > On 15 April 2017 at 23:50, John Erling Blad <jeb...@gmail.com>
> > > > wrote:
> > > > > > >
> > > > > > > > Are anyone doing any work on automated quality assurance of
> > > > articles?
> > > > > > Not
> > > > > > > > the ORES-stuff, that is about creating hints from measured
> > > > features.
> > > > > > I'm
> > > > > > > > thinking about verifying existence and completeness of
> > citations,
> > > > and
> > > > > > > > structure of logical arguments.
> > > > > > > >
> > > > > > > > John
> > > > > > > > _______________________________________________
> > > > > > > > Wikimedia-l mailing list, guidelines at:
> > > > https://meta.wikimedia.org/
> > > > > > > > wiki/Mailing_lists/Guidelines and
> https://meta.wikimedia.org/
> > > > > > > > wiki/Wikimedia-l
> > > > > > > > New messages to: Wikimedia-l@lists.wikimedia.org
> > > > > > > > Unsubscribe: https://lists.wikimedia.org/
> > > > > mailman/listinfo/wikimedia-l,
> > > > > > > > <mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=
> > > > unsubscribe>
> > > > > > > _______________________________________________
> > > > > > > Wikimedia-l mailing list, guidelines at:
> > > https://meta.wikimedia.org/
> > > > > > > wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/
> > > > > > > wiki/Wikimedia-l
> > > > > > > New messages to: Wikimedia-l@lists.wikimedia.org
> > > > > > > Unsubscribe: https://lists.wikimedia.org/
> > > > mailman/listinfo/wikimedia-l,
> > > > > > > <mailto:wikimedia-l-requ...@lists.wikimedia.org
> > > ?subject=unsubscribe>
> > > > > > _______________________________________________
> > > > > > Wikimedia-l mailing list, guidelines at:
> > https://meta.wikimedia.org/
> > > > > > wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/
> > > > > > wiki/Wikimedia-l
> > > > > > New messages to: Wikimedia-l@lists.wikimedia.org
> > > > > > Unsubscribe:
> > > https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
> > > > > > <mailto:wikimedia-l-requ...@lists.wikimedia.org
> > ?subject=unsubscribe>
> > > > > >
> > > > > _______________________________________________
> > > > > Wikimedia-l mailing list, guidelines at:
> https://meta.wikimedia.org/
> > > > > wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/
> > > > > wiki/Wikimedia-l
> > > > > New messages to: Wikimedia-l@lists.wikimedia.org
> > > > > Unsubscribe:
> > https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
> > > > > <mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=
> unsubscribe>
> > > > >
> > > > _______________________________________________
> > > > Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/
> > > > wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/
> > > > wiki/Wikimedia-l
> > > > New messages to: Wikimedia-l@lists.wikimedia.org
> > > > Unsubscribe: https://lists.wikimedia.org/
> mailman/listinfo/wikimedia-l,
> > > > <mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe>
> > > >
> > > _______________________________________________
> > > Wikimedia-l mailing list, guidelines at:
> > > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and
> > > https://meta.wikimedia.org/wiki/Wikimedia-l
> > > New messages to: Wikimedia-l@lists.wikimedia.org
> > > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
> > > <mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe>
> > _______________________________________________
> > Wikimedia-l mailing list, guidelines at:
> > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and
> > https://meta.wikimedia.org/wiki/Wikimedia-l
> > New messages to: Wikimedia-l@lists.wikimedia.org
> > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
> > <mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe>
> _______________________________________________
> Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/
> wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/
> wiki/Wikimedia-l
> New messages to: Wikimedia-l@lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
> <mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe>
>
_______________________________________________
Wikimedia-l mailing list, guidelines at: 
https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and 
https://meta.wikimedia.org/wiki/Wikimedia-l
New messages to: Wikimedia-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, 
<mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe>

Reply via email to