Michael added a subscriber: Ladsgroup. Michael added a comment.
> What would be the optimal way to use these 2.000 human-categorized revisions? Can we focus on hard and useful stuff without jeopardizing the balance of ORES? Could we for example treat bot edits and massive classes like "academic papers" and "astronomical objects" differently? (e.g. segmented sample for external evaluation and self-evaluate the "easy" stuff) Yes, we can just skip edits by bots, and instances of scholarly articles <https://www.wikidata.org/wiki/Q13442814> and astronomical objects <https://www.wikidata.org/wiki/Q6999>. Because they are not what we are looking for when we want ORES to evaluate human //edits//. This would be different if our goal were improving the model for Item quality (though even then there would be ways to optimize things). That being said, after the revisions haven been scored by humans, we can still significantly improve the amount of training data we have by using self-training: training ORES with the data, letting it score more revisions, and then adding the revisions where it is very confident to the training data and repeating that process as often as needed. At least that is what I learned from @Ladsgroup 🙇😊 TASK DETAIL https://phabricator.wikimedia.org/T297347 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Michael Cc: Ladsgroup, Lucas_Werkmeister_WMDE, Michael, Manuel, Aklapper, Lydia_Pintscher, Invadibot, maantietaja, Akuckartz, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, Mbch331
_______________________________________________ Wikidata-bugs mailing list -- [email protected] To unsubscribe send an email to [email protected]
