All fine points. As you can see, I've filed some phab tasks where I saw a clear opportunity to do so.
> as mentioned before all the models that currently run on ORES are available in both ores-legacy and Lift Wing. I thought I read that damaging and goodfaith models are going to be replaced. Should I instead read that they are likely to remain available for the foreseeable future? When I asked about a community discussion about the transition from damaging/goodfaith to revertrisk, I was imagining that many people who use those predictions might have an opinion about them going away. E.g. people who use the relevant filters in RecentChanges. Maybe I missed the discussions about that. I haven't seen a mention of the article quality or article topic models in the docs. Are those also going to remain available? I have some user scripts that use these models and are relatively widely used. I didn't notice anyone reaching out. ... So I checked and setting a User-Agent on my user scripts doesn't actually change the User-Agent. I've read that you need to set "Api-User-Agent" instead, but that causes a CORS error when querying ORES. I'll file a bug. On Fri, Sep 22, 2023 at 1:22 PM Luca Toscano <[email protected]> wrote: > > > On Fri, Sep 22, 2023 at 8:59 PM Aaron Halfaker <[email protected]> > wrote: > >> We could definitely file a task. However, it does seem like highlighting >> the features that will no longer be available is an appropriate topic for a >> discussion about migration in a technical mailing list. >> > > A specific question related to a functionality is the topic for a task, I > don't think that we should discuss every detail that differs from the ORES > API (Wikitech-l doesn't seem a good medium for it). We are already > following up on Phabricator, let's use tasks if possible to keep the > conversation as light and targeted as possible. > > Is there a good reference for which features have been excluded from >> ores-legacy? It looks like https://wikitech.wikimedia.org/wiki/ORES covers >> some of the excluded features/models, but not all of them. >> > > We spent the last months helping the community to migrate away from the > ORES API (to use Lift Wing instead), the remaining traffic is only related > to few low traffic IPs that we are not able to contact. We didn't add > feature injection or threshold optimization to ores-legacy, for example, > since there was no indication on our logs that users were relying on it. We > have always stated everywhere (including all emails sent in this mailing > list) that we are 100% open to add a functionality if it is backed up by a > valid use case. > > >> I see now that it looks like the RevertRisk model will be replacing the >> *damaging >> *and *goodfaith *models that differentiate intentional damage from >> unintentional damage. There's a large body of research on why this is >> valuable and important to the social functioning of the wikis. This >> literature also discusses why being reverted is not a very good signal for >> damage/vandalism and can lead to problems when used as a signal for >> patrolling. Was there a community discussion about this deprecation that I >> missed? I have some preliminary results (in press) that demonstrate that >> the RevertRisk model performs significantly worse than the damaging and >> goodfaith models in English Wikipedia for patrolling work. Do you have >> documentation for how you evaluated this model and compared it to >> damaging/goodfaith? >> > > We have model cards related to both Revert Risk models, all of them linked > in the API portal docs (more info: > https://api.wikimedia.org/wiki/Lift_Wing_API). All the community folks > that migrated their bots/tools/etc.. to Revert Risk were very happy about > the change, and we haven't had any request to switch back since then. > > The ML team provides all the models deployed on ORES on Lift Wing, so any > damaging and goodfaith variant is available in the new API. We chose to not > pursue the development of those models for several reasons: > - We haven't had any indication/request from the community about those > models in almost two years, except few Phabricator updates that we followed > up on. > - Managing several hundreds models for goodfaith and damaging is not very > scalable in a modern micro-service architecture like Lift Wing (since we > have a model for each supported wiki). We (both Research and ML) are > oriented on having fewer models that manage more languages at the same > time, and this is the direction that we are following at the moment. It may > not be the perfect one but so far it seems a good choice. If you want to > chime in and provide your inputs we are 100% available in hearing > suggestions/concerns/doubts/recommendations/etc.., please follow up in any > of our channels (IRC, mailing lists, Phabricator for example). > - Last but not the least, most of the damaging/goodfaith models have been > trained with data coming from years ago, and never re-trained. The efforts > to keep several hundreds models up-to-date with recent data versus doing > the same of few models (like revert risk) weights in favor of the latter > for a relatively small team of engineers like us. > > >> FWIW, from my reading of these announcement threads, I believed that >> generally functionality and models would be preserved in >> ores-legacy/LiftWing. This is the first time I've realized the scale of >> what will become unavailable. >> > > This is the part that I don't get, since as mentioned before all the > models that currently run on ORES are available in both ores-legacy and > Lift Wing. What changes is that we don't expose anymore functionality that > logs clearly show are not used, and that would need to be maintained and > improved over time. We are open to improve and add any requirement that the > community needs, the only thing that we ask is to provide a valid use case > to support it. > > I do think that Lift Wing is a great improvement for the community, we > have been working with all the folks that reached out to us, without hiding > anything (including deprecation plans and path forwards). > > Thanks for following up! > > Luca > _______________________________________________ > Wikitech-l mailing list -- [email protected] > To unsubscribe send an email to [email protected] > https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
_______________________________________________ Wikitech-l mailing list -- [email protected] To unsubscribe send an email to [email protected] https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
