[Wikidata-bugs] [Maniphest] T261850: compare model accuracy with and without property suggester

Michael moved this task from Peer Review to Done on the Item Quality Scoring Improvement (Item Quality Scoring Improvement - Sprint 3) board. Michael closed this task as "Resolved". Michael added a comment. This has been done in #159 .

[Wikidata-bugs] [Maniphest] T261850: compare model accuracy with and without property suggester

Lydia_Pintscher added a comment. In T261850#6449787 , @Michael wrote: > Great! I'll make a pull request for removing it.  > > Removing property suggester has also the positive side-effect that our scores for dumps and the API

[Wikidata-bugs] [Maniphest] T261850: compare model accuracy with and without property suggester

Halfak added a comment. It's not quite fair to compare the old an new feature sets. It does look like the property suggestor was having a minor positive effect, but that seems like it was not worth the additional API call. Everything that follows is just me nerding out about the stats.

[Wikidata-bugs] [Maniphest] T261850: compare model accuracy with and without property suggester

Michael added a comment. Great! I'll make a pull request for removing it.  Removing property suggester has also the positive side-effect that our scores for dumps and the API should be the same again. cc @Lydia_Pintscher TASK DETAIL https://phabricator.wikimedia.org/T261850 EMAIL

[Wikidata-bugs] [Maniphest] T261850: compare model accuracy with and without property suggester

Ladsgroup added a comment. In T261850#6449735 , @Michael wrote: > Thank you for your thorough research. That means we can effectively drop property suggester? Not having to do that extra network request should speed some things up.

[Wikidata-bugs] [Maniphest] T261850: compare model accuracy with and without property suggester

Michael added a comment. Thank you for your thorough research. That means we can effectively drop property suggester? Not having to do that extra network request should speed some things up. TASK DETAIL https://phabricator.wikimedia.org/T261850 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T261850: compare model accuracy with and without property suggester

Ladsgroup added a comment. So I did a little bit of statistics. First I rebuilt the old model with the old features multiple times to build a distribution of roc_auc and other metrics it produced. - For roc_auc the mean is 0.965, the std is 0.000655 - For accuracy, the mean is 0.921

[Wikidata-bugs] [Maniphest] T261850: compare model accuracy with and without property suggester

Ladsgroup moved this task from Doing to Peer Review on the Item Quality Scoring Improvement (Item Quality Scoring Improvement - Sprint 3) board. Ladsgroup added a comment. (All are micro average

[Wikidata-bugs] [Maniphest] T261850: compare model accuracy with and without property suggester

Ladsgroup claimed this task. Ladsgroup moved this task from To Do to Doing on the Item Quality Scoring Improvement (Item Quality Scoring Improvement - Sprint 3) board. Restricted Application added a project: User-Ladsgroup. TASK DETAIL https://phabricator.wikimedia.org/T261850 WORKBOARD

[Wikidata-bugs] [Maniphest] T261850: compare model accuracy with and without property suggester

Lydia_Pintscher moved this task from Backlog to Item Quality Scoring Improvement - Sprint 3 on the Item Quality Scoring Improvement board. Lydia_Pintscher edited projects, added Item Quality Scoring Improvement (Item Quality Scoring Improvement - Sprint 3); removed Item Quality Scoring

[Wikidata-bugs] [Maniphest] T261850: compare model accuracy with and without property suggester

Lydia_Pintscher created this task. Lydia_Pintscher added projects: Item Quality Scoring Improvement, Wikidata. TASK DESCRIPTION **Problem:** The property suggester is only taken into account when scoring an Item live. It is not taken into account when scoring an Item in the dump. We want to