KaMan updated the task description. (Show Details)
CHANGES TO TASK DESCRIPTION...- [[ https://www.wikidata.org/wiki/Module:Lexeme-pl | Module:Lexeme-pl ]]
- [[ https://www.wikidata.org/wiki/User:GZWDer/newentity.js | User:GZWDer/newentity.js ]]
**Example:**...TASK
Bugreporter created this task.Bugreporter added projects: Wikidata, Lexicographical data.
TASK DESCRIPTIONTASK DETAILhttps://phabricator.wikimedia.org/T204397EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: BugreporterCc: Bugreporter, Mringgaard, Lahi, Gq86,
Krinkle added a comment.
Earlier when I was investigating with Stas, performing the slow recentchanges api request with X-Wikimedia-Debug: log shows the following in Logstash:
[DEBUG] [DBConnection] Wikimedia\Rdbms\LoadBalancer::getReaderIndex: using server db2080 for group 'api'
Which indeed
Adithyak1997 added a comment.
What I need is, I would like to remove the sentence ഫലകം:SHORTDESC: from https://ml.wikipedia.org/wiki/%E0%B4%AB%E0%B4%B2%E0%B4%95%E0%B4%82:SHORTDESC this template.TASK DETAILhttps://phabricator.wikimedia.org/T204136EMAIL
Reedy added a comment.
In T202764#4585276, @Smalyshev wrote:
I tried db2085:3318 and the result the same as other codfw host. So if that's what actual API is using, that could be the reason why it is so slow.
It should be the rc hosts it's using for those queriesTASK
Smalyshev added a comment.
I tried db2085:3318 and the result the same as other codfw host. So if that's what actual API is using, that could be the reason why it is so slow.TASK DETAILhttps://phabricator.wikimedia.org/T202764EMAIL
Smalyshev added a comment.
@Reedy I am not sure which host, I just logged in to maintenance host for eqiad and codfw. Lookups show db2082.codfw.wmnet and db1092.eqiad.wmnet.TASK DETAILhttps://phabricator.wikimedia.org/T202764EMAIL
Krinkle removed Krinkle as the assignee of this task.Krinkle edited projects, added Performance-Team (Radar); removed Performance-Team.
TASK DETAILhttps://phabricator.wikimedia.org/T202764EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: KrinkleCc: gerritbot,
Smalyshev added a project: DBA.Smalyshev added a comment.
Looks like codfw one does not use index. @jcrespo do you have any idea why that could happen?TASK DETAILhttps://phabricator.wikimedia.org/T202764EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Krinkle,
Smalyshev added a comment.
Explains:
codfw:
P7551 (An Untitled Masterwork)
eqiad:
P7552 (An Untitled Masterwork)TASK DETAILhttps://phabricator.wikimedia.org/T202764EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Krinkle, SmalyshevCc: gerritbot, Krinkle,
Smalyshev added a comment.
This query:
SELECT rc_id,rc_timestamp,rc_namespace,rc_title,rc_cur_id,rc_type,rc_deleted,rc_this_oldid,rc_last_oldid FROM `recentchanges`WHERE (rc_timestamp>='2018091411') AND rc_namespace IN ('0','120') AND rc_type IN ('0','1','3','6') ORDER BY
gerritbot added a project: Patch-For-Review.
TASK DETAILhttps://phabricator.wikimedia.org/T202764EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Krinkle, gerritbotCc: gerritbot, Krinkle, Addshore, Yurik, jcrespo, Imarlier, Ladsgroup, Lydia_Pintscher,
gerritbot added a comment.
Change 460613 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/core@master] debug: Allow the DBQuery channel to be used
https://gerrit.wikimedia.org/r/460613TASK DETAILhttps://phabricator.wikimedia.org/T202764EMAIL
thcipriani added a comment.
In T192006#4579891, @Addshore wrote:
@thcipriani just checking, but this would result in an image of whatever is currently on master right?
Depends on how it's setup. For Mathoid a container is built and pushed to
Smalyshev added a comment.
Doesn't seem to be WDQS related entirely - e.g. if I call 'https://www.wikidata.org/w/api.php?format=json=""> - i.e. try to load 100 items from start of the day today - it takes 29 seconds:
real 0m29.500s
user 0m0.023s
sys 0m0.011s
I don't think it's normal for this
Reedy closed this task as a duplicate of T204119: Some versions of ffi and rubyzip reported as vulnerable.
TASK DETAILhttps://phabricator.wikimedia.org/T204378EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: ReedyCc: Aklapper, Smalyshev, Lahi, Gq86,
Amire80 created this task.Amire80 added projects: Lexicographical data, I18n.Restricted Application added a project: Wikidata.
TASK DESCRIPTIONIf I understand correctly, this is a follow up to T185761.
There are many message keys in WikibaseLexeme that begin with "wikibaselexeme-api-error-*". If
DannyH added a comment.
@Adithyak1997, it's possible that there's a misunderstanding about what this template does. What are you hoping to do with this template?TASK DETAILhttps://phabricator.wikimedia.org/T204136EMAIL
Smalyshev created this task.Smalyshev added a project: Wikidata.Restricted Application added a subscriber: Aklapper.
TASK DESCRIPTIONI get these complaints for Wikibase:
ffi: upgrade to ~> 1.9.24
rubyzip: upgrade to ~> 1.2.2
Both from Gemfile.lock .TASK
MarcoAurelio added a comment.
Shall we wait till the wmf.22 train arrives to Wikidata to test if this worked before marking this task as resolved? Thanks.TASK DETAILhttps://phabricator.wikimedia.org/T203311EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To:
Adithyak1997 added a comment.
@DannyH and @Lydia_Pintscher . I would like to ask two questions to you both since you both have commented against the use of the magic word on malayalam wikipedia. My first question is: if English wikipedia can use a feature , then why can't it be used in other
DannyH added a comment.
We're not going to enable this magic word on any other wikis right now. People should update the short descriptions on Wikidata.TASK DETAILhttps://phabricator.wikimedia.org/T204136EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To:
Lydia_Pintscher added a comment.
I would like this magic word to not spread even more. It's bad enough on enwp for Wikidata already. Please!TASK DETAILhttps://phabricator.wikimedia.org/T204136EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lydia_PintscherCc:
Niharika added subscribers: kaldari, Tgr, Niharika.Niharika added a comment.
CC @Tgr @kaldariTASK DETAILhttps://phabricator.wikimedia.org/T204136EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: NiharikaCc: Niharika, Tgr, kaldari, Adithyak1997, Aklapper,
Ramsey-WMF triaged this task as "High" priority.
TASK DETAILhttps://phabricator.wikimedia.org/T202287EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Cparle, Ramsey-WMFCc: gerritbot, MarkTraceur, Aklapper, Ramsey-WMF, PDrouin-WMF, Cparle, Gaboe420, Versusxo,
Ramsey-WMF added a subscriber: MarkTraceur.Ramsey-WMF added a comment.
In T203496#4583848, @Prtksxna wrote:
Hey @Ramsey-WMF! I have installed the CleanChanges extension on my local mediawiki. Could you point me to the steps to follow to see it in action?
I refer this to the expertise of
Lydia_Pintscher closed this task as "Resolved".Lydia_Pintscher moved this task from Test (Verification) to Done on the Wikidata-Campsite-Iteration-∞ board.
TASK DETAILhttps://phabricator.wikimedia.org/T203067WORKBOARDhttps://phabricator.wikimedia.org/project/board/3539/EMAIL
Smalyshev added a comment.
A simple solution, I suppose, would be to completely skip prefixes for REGEX queries, which are (I believe) the most common queries we send out and never need any prefixes.
This looks like a good start.TASK DETAILhttps://phabricator.wikimedia.org/T204317EMAIL
Gehel created this task.Gehel added projects: Wikidata-Query-Service, Operations, Wikimedia-Logstash.Restricted Application added a subscriber: Aklapper.Restricted Application added a project: Wikidata.
TASK DESCRIPTIONWe recently had cases of wdqs sending >10K logs per seconds to logstash. This
gerritbot added a comment.
Change 460557 had a related patch set uploaded (by Pablo Grass (WMDE); owner: Pablo Grass (WMDE)):
[mediawiki/extensions/WikibaseQualityConstraints@master] gadget: separate functionalities
https://gerrit.wikimedia.org/r/460557TASK
Smalyshev added a comment.
All bans are temporary, so as soon as traffic returns to normal the bans will expire. It would be nice if there was a way to wbqc to respect the 429 throttling header, which will avoid bans.TASK DETAILhttps://phabricator.wikimedia.org/T204267EMAIL
gerritbot added a comment.
Change 460379 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Add html data attribute to statement groups for dom hydration
https://gerrit.wikimedia.org/r/460379TASK DETAILhttps://phabricator.wikimedia.org/T196226EMAIL
Amire80 added a comment.
Um, I'm not sure whether I'm in a conflict of interest here or not...
I'm a member of Langcom, and I speak Hebrew, and I learned it academically. The hbo code is definitely not ambiguous. Biblical Hebrew is very well-defined: it's the Hebrew as contained in the Bible. By
Tpt added a comment.
@Jonas Thank you for your feedback.
is it necessary for your tool to run the constraint checks in parallel?
No, I am going to switch to a sequential processing. Thanks for the idea!
Using WDQS instead would be a good idea, because then only your tool would get throttled.
gerritbot added a comment.
Change 460546 merged by Dbarratt:
[labs/tools/graphql@master] Add a language fallback system
https://gerrit.wikimedia.org/r/460546TASK DETAILhttps://phabricator.wikimedia.org/T173214EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To:
gerritbot added a project: Patch-For-Review.
TASK DETAILhttps://phabricator.wikimedia.org/T173214EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: gerritbotCc: gerritbot, Smalyshev, Lydia_Pintscher, Addshore, larsgw, Saerdnaer, simon04, bearND, Siznax, Tpt,
gerritbot added a comment.
Change 460546 had a related patch set uploaded (by Dbarratt; owner: Dbarratt):
[labs/tools/graphql@master] Add a language fallback system
https://gerrit.wikimedia.org/r/460546TASK DETAILhttps://phabricator.wikimedia.org/T173214EMAIL
Cparle added a parent task: T181458: Prototype new File page functionality for Structured Data on Commons.
TASK DETAILhttps://phabricator.wikimedia.org/T204264EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: CparleCc: Aklapper, Cparle, Lahi, PDrouin-WMF, Gq86,
Cparle added a subtask: T204264: Add displaying/editing statements to File page.
TASK DETAILhttps://phabricator.wikimedia.org/T181458EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: CparleCc: MarkTraceur, PDrouin-WMF, Ramsey-WMF, Aklapper, Lahi, Gq86,
Jonas added a comment.
@Tpt is it necessary for your tool to run the constraint checks in parallel?
Using WDQS instead would be a good idea, because then only your tool would get throttled.
The problem there is that just a fraction of all violations are in there at the moment.TASK
Tpt added a comment.
Sorry everyone for the troubles. I was experimenting with a tool that tries to find corrections for constraint violations.
I have modified it to send a proper User-Agent for all its requests to the Wikidata API but not restarted it.
@Wikidata team: what would be an ok request
Fnielsen added a comment.
If one switch to table view and back to timeline view with the left switcher then the timeline appears.TASK DETAILhttps://phabricator.wikimedia.org/T204233EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: FnielsenCc: Fnielsen,
ReleaseTaggerBot added a project: MW-1.32-release-notes (WMF-deploy-2018-09-18 (1.32.0-wmf.22)).
TASK DETAILhttps://phabricator.wikimedia.org/T196226EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Jakob_WMDE, ReleaseTaggerBotCc: gerritbot, Pablo-WMDE,
gerritbot added a comment.
Change 459806 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Add optional html id prefix to statement groups
https://gerrit.wikimedia.org/r/459806TASK DETAILhttps://phabricator.wikimedia.org/T196226EMAIL
gerritbot added a comment.
Change 460524 had a related patch set uploaded (by Pablo Grass (WMDE); owner: Pablo Grass (WMDE)):
[mediawiki/extensions/WikibaseQualityConstraints@master] gadget: encapsulate in a module
https://gerrit.wikimedia.org/r/460524TASK
Fnielsen added a comment.
The status code associated with embeddings where the timeline is missing is 200 (as far as I can determine). In the console I identify no relevant errors.TASK DETAILhttps://phabricator.wikimedia.org/T204233EMAIL
Prtksxna added a comment.
Hey @Ramsey-WMF! I have installed the CleanChanges extension on my local mediawiki. Could you point me to the steps to follow to see it in action?TASK DETAILhttps://phabricator.wikimedia.org/T203496EMAIL
Addshore reassigned this task from Addshore to Smalyshev.
TASK DETAILhttps://phabricator.wikimedia.org/T204267EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Smalyshev, AddshoreCc: Tpt, aborrero, Pintoch, Stashbot, Jonas, Addshore, TerraCodes, Liuxinyu970226,
Addshore lowered the priority of this task from "Unbreak Now!" to "High".Addshore added a comment.
The API request rate has returned to a normal level:
F25850152: image.png
As have the SPARQL error codes being hit by the WBQC extension.
The rate of wdqs queries being banned / throttled is also
aborrero added a subscriber: Tpt.aborrero added a comment.
We stopped now the corhist tool which belongs to @Tpt, please check the tool. Now that the tool has been terminated, the traffic in wikidata seems back to normal.TASK DETAILhttps://phabricator.wikimedia.org/T204267EMAIL
Stashbot added a comment.
Mentioned in SAL (#wikimedia-cloud) [2018-09-14T11:22:34Z] T204267 stop the corhist tool (k8s) because is hammering the wikidata APITASK DETAILhttps://phabricator.wikimedia.org/T204267EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To:
aborrero added a comment.
In T204267#4583629, @Pintoch wrote:
@aborrero thanks for the ping. I do not recognize the shape of the queries as coming from this tool though. The openrefine-wikidata tool should do relatively few SPARQL queries, whose results are cached in redis. How did you determine
Pintoch added a comment.
@aborrero thanks for the ping. I do not recognize the shape of the queries as coming from this tool though. The openrefine-wikidata tool should do relatively few SPARQL queries, whose results are cached in redis. How did you determine that this tool is the source of the
aborrero added subscribers: Pintoch, aborrero.aborrero added a comment.
Ping @Pintoch , this seems to be your tool.TASK DETAILhttps://phabricator.wikimedia.org/T204267EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Addshore, aborreroCc: aborrero, Pintoch,
Stashbot added a comment.
Mentioned in SAL (#wikimedia-cloud) [2018-09-14T10:51:35Z] T204267 stop the openrefine-wikidata tool (k8s) because is hammering the wikidata APITASK DETAILhttps://phabricator.wikimedia.org/T204267EMAIL
matej_suchanek created this task.matej_suchanek added projects: Pywikibot-core, Pywikibot-Wikidata, Wikidata.Restricted Application added subscribers: pywikibot-bugs-list, Aklapper.
TASK DESCRIPTIONimport pywikibot
repo = pywikibot.Site('wikidata', 'wikidata')
item = pywikibot.ItemPage(repo,
Lucas_Werkmeister_WMDE created this task.Lucas_Werkmeister_WMDE added projects: Wikidata, Wikibase-Quality, Wikibase-Quality-Constraints.Restricted Application added a subscriber: Aklapper.
TASK DESCRIPTIONAs @Smalyshev points out in T204267#4581546, we’re sending a bunch of prefixes with each
Addshore added a project: Cloud-Services.
TASK DETAILhttps://phabricator.wikimedia.org/T204267EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: AddshoreCc: Jonas, Addshore, TerraCodes, Liuxinyu970226, Aklapper, Smalyshev, AndyTan, Zylc, Davinaclare77, Qtn1293,
Addshore added a comment.
Looks like it is tools-worker-1021.tools.eqiad.wmflabsTASK DETAILhttps://phabricator.wikimedia.org/T204267EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: AddshoreCc: Jonas, Addshore, TerraCodes, Liuxinyu970226, Aklapper, Smalyshev,
Jonas added a comment.
I added a comment to the create command.TASK DETAILhttps://phabricator.wikimedia.org/T202265EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Matthias_Geisler_WMDE, JonasCc: Jonas, Marostegui, Ladsgroup, Addshore, Aklapper, Lahi, Gq86,
Jonas updated the task description. (Show Details)
CHANGES TO TASK DESCRIPTION...CREATE INDEX `tmp1` ON wb_terms (`term_language`,`term_type`,`term_entity_type`,`term_search_key`) COMMENT 'term search index (in use)';...TASK DETAILhttps://phabricator.wikimedia.org/T202265EMAIL
Addshore added a project: Operations.Addshore added a comment.
Tagging operations as we want to block these requestsTASK DETAILhttps://phabricator.wikimedia.org/T204267EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: AddshoreCc: Jonas, Addshore, TerraCodes,
Addshore added a comment.
Quick investigation shows that the increase in sparql requests is likely due to the 4x/5x increase in API requests (there was 10x increase at the peak of the spike).
Looking at the web requests most requests seem to be coming from a single UA / IP. so lets aim to block
Qgil moved this task from Backlog to Jul-Sep-2018 on the CommRel-Specialists-Support board.Qgil edited projects, added CommRel-Specialists-Support (Jul-Sep-2018); removed CommRel-Specialists-Support.
TASK
Addshore claimed this task.Restricted Application added a project: User-Addshore.
TASK DETAILhttps://phabricator.wikimedia.org/T204267EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: AddshoreCc: Jonas, Addshore, TerraCodes, Liuxinyu970226, Aklapper, Smalyshev,
Jonas added a comment.
Seems the #wikibase-quality-constraints extension is the source:
https://grafana.wikimedia.org/dashboard/db/wikidata-quality?refresh=10s=1=now-30d=now
daniel added a comment.
@awight It is, via T183490. https://wikifarm.wmflabs.org/mcr/index.php/Deployment_Pert_ChartTASK DETAILhttps://phabricator.wikimedia.org/T183487EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: awight, gerritbot, Tgr,
Pablo-WMDE created this task.Pablo-WMDE added projects: Wikibase-Quality-Constraints, Wikidata.Restricted Application added a subscriber: Aklapper.
TASK DESCRIPTIONApparently WikibaseQualityConstraints currently gets built on travis. Unfortunately it fails/errors on all builds.
Questions
What
awight added a comment.
Should this be a subtask of T174045: DB schema migration for MCR?TASK DETAILhttps://phabricator.wikimedia.org/T183487EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: awightCc: awight, gerritbot, Tgr, Jdforrester-WMF, Anomie, Addshore,
Smalyshev updated the task description. (Show Details)
CHANGES TO TASK DESCRIPTIONLooking at Kibana, I see a lot of banned requests marked with wbqc happening on the internal cluster. In the last 4 hours, it's 2,428,030 events (more than 300 per second!). Something is clearly not right there.
Addshore added a comment.
In T197658#4582768, @despens wrote:
So I guess the whole change history is not available in the Wikibase API after migrating the database from the previous install.
It is RecentChanges that you need to look at.
The age of the stuff in RC is configured by
Smalyshev added a comment.
So, weird thing: now that we switched data centers, wdqs2003 is showing the same anomaly. Could it be that our load balancing is not balancing the load evenly for these hosts?TASK DETAILhttps://phabricator.wikimedia.org/T200563EMAIL
Smalyshev added a project: Wikibase-Quality.Smalyshev added a subscriber: Addshore.
TASK DETAILhttps://phabricator.wikimedia.org/T204267EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: Addshore, TerraCodes, Liuxinyu970226, Aklapper, Smalyshev,
Smalyshev triaged this task as "Unbreak Now!" priority.Smalyshev added a comment.Restricted Application added subscribers: Liuxinyu970226, TerraCodes.
12,499,055 throttling events in last 24 hours. This is definitely not good.TASK DETAILhttps://phabricator.wikimedia.org/T204267EMAIL
Matthias_Geisler_WMDE updated the task description. (Show Details)
CHANGES TO TASK DESCRIPTION...[ X ] Figure out where to deploy it [wdq-ui.wmflabs.org](http://wdq-ui.wmflabs.org/)
[ X ] Deploy and update cronTASK DETAILhttps://phabricator.wikimedia.org/T203078EMAIL
despens added a comment.
Thank you @Smalyshev!
It seems like the --conceptUri switch is a part of munger. It is not accepted as a parameter for runUpdate.sh.
After checking this, I did also modify the Wikibase's LocalSettings.php to explicitely use http (without 's') for concept URLs:
76 matches
Mail list logo
