Lucas_Werkmeister_WMDE claimed this task.Lucas_Werkmeister_WMDE added a subscriber: WMDE-leszek.
TASK DETAILhttps://phabricator.wikimedia.org/T173696EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lucas_Werkmeister_WMDECc: WMDE-leszek, Krinkle, aaron,
Lucas_Werkmeister_WMDE closed subtask T173696: Cache format constraint check results as "Resolved".
TASK DETAILhttps://phabricator.wikimedia.org/T179839EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lucas_Werkmeister_WMDECc: Aklapper, Lucas_Werkmeister_WMDE,
Lucas_Werkmeister_WMDE closed this task as "Resolved".Lucas_Werkmeister_WMDE added a comment.
In T173696#3690945, @Lucas_Werkmeister_WMDE wrote:
(Perhaps we should repurpose this task to be just about regex checking, open a new one for general caching, and reshuffle the parent tasks so that this
gerritbot added a comment.
Change 389522 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[mediawiki/extensions/Wikibase@master] [DNM] Add Dispatching request parser for setclaim API that could be used by extensions
https://gerrit.wikimedia.org/r/389522TASK
Lucas_Werkmeister_WMDE renamed this task from "Cache constraint check results" to "Cache format constraint check results".Lucas_Werkmeister_WMDE updated the task description. (Show Details)
CHANGES TO TASK DESCRIPTIONBefore we can widely enable constraint checksResults of the “format” constraint,
Capt_Swing added a comment.
@debt @Ramsey-WMF I think you should create a project page in the Research namespace on Meta (and then link to it from the page you mention above). Here's an example research page:
Smalyshev added a comment.
I think for now limiting it by IP should probably work? I think IP ranges from production hosts, labs and outside are segregated?TASK DETAILhttps://phabricator.wikimedia.org/T178492EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To:
Lucas_Werkmeister_WMDE created subtask T179839: Cache constraint check results.
TASK DETAILhttps://phabricator.wikimedia.org/T103228EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lucas_Werkmeister_WMDECc: Liuxinyu970226, Aklapper, Jonaskeutel, Lahi,
Lucas_Werkmeister_WMDE created this task.Lucas_Werkmeister_WMDE added projects: Wikidata, Wikibase-Quality-Constraints, Wikibase-Quality.Herald added a subscriber: Aklapper.
TASK DESCRIPTIONBefore we can widely enable constraint checks, we need to cache constraint check results, otherwise the
Lucas_Werkmeister_WMDE created subtask T179844: Indicate that constraint check results are potentially stale.
TASK DETAILhttps://phabricator.wikimedia.org/T179839EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lucas_Werkmeister_WMDECc: Aklapper,
Lucas_Werkmeister_WMDE created this task.Lucas_Werkmeister_WMDE added projects: Wikidata, Wikibase-Quality-Constraints, Wikibase-Quality.
TASK DESCRIPTIONConstraint check results can sometimes be stale – this is already the case, due to the query service’s cache (causing T170672: Constraints
Smalyshev closed this task as "Resolved".
TASK DETAILhttps://phabricator.wikimedia.org/T178533EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: gerritbot, Gehel, Aklapper, Smalyshev, Lahi, Gq86, Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic,
Lucas_Werkmeister_WMDE removed a subtask: T173696: Cache format constraint check results.
TASK DETAILhttps://phabricator.wikimedia.org/T103228EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lucas_Werkmeister_WMDECc: Liuxinyu970226, Aklapper, Jonaskeutel,
Lucas_Werkmeister_WMDE edited parent tasks, added: T179839: Cache constraint check results; removed: T103228: Improve performance of constraint check.
TASK DETAILhttps://phabricator.wikimedia.org/T173696EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To:
Lucas_Werkmeister_WMDE created this task.Lucas_Werkmeister_WMDE added projects: Wikidata, Wikibase-Quality-Constraints, Wikibase-Quality.
TASK DESCRIPTIONCaching constraint check results per-constraint is probably too inefficient to be of any use, so we should probably cache the full results of
Lucas_Werkmeister_WMDE created subtask T179849: Cache all constraint check results per-entity.
TASK DETAILhttps://phabricator.wikimedia.org/T179839EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lucas_Werkmeister_WMDECc: Aklapper, Lucas_Werkmeister_WMDE,
Gehel added a comment.
It might make sense to add whitelisting / authentication / authorization on the "controlled" wdqs cluster to ensure the usage is under control. This might or might not be required. If we go that way, we might want to address this as a more general problem, which might make
gerritbot added a comment.
Change 389521 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[mediawiki/extensions/Wikibase@master] [DNM] Extracted code process input of wbsetclaim API to a class that could be "extended" by extension registering custom entity types
gerritbot added a project: Patch-For-Review.
TASK DETAILhttps://phabricator.wikimedia.org/T179662EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: WMDE-leszek, gerritbotCc: gerritbot, Aklapper, Lydia_Pintscher, Jonas, Jakob_WMDE, Ladsgroup, thiemowmde,
Capt_Swing added a comment.
If y'all have questions on how to create such a page, what info should go on it, etc. I'm happy to help!TASK DETAILhttps://phabricator.wikimedia.org/T179450EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Capt_SwingCc: Aklapper,
gerritbot added a comment.
Change 388696 merged by Gehel:
[operations/puppet@production] Report 429s to logstash too
https://gerrit.wikimedia.org/r/388696TASK DETAILhttps://phabricator.wikimedia.org/T178533EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To:
gerritbot added a comment.
Change 389427 had a related patch set uploaded (by Giuseppe Lavagetto; owner: Giuseppe Lavagetto):
[operations/puppet@production] jobrunner: make refreshlinks jobs low-priority
https://gerrit.wikimedia.org/r/389427TASK
Marostegui added a comment.
Makes sense to do it at the same time with T114904 as we might need to depool slaves for that task.TASK DETAILhttps://phabricator.wikimedia.org/T179793EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MarosteguiCc: Marostegui,
gerritbot added a comment.
Change 389427 merged by Giuseppe Lavagetto:
[operations/puppet@production] jobrunner: make refreshlinks jobs low-priority
https://gerrit.wikimedia.org/r/389427TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL
Sjoerddebruin created this task.Sjoerddebruin added projects: Wikidata, MediaWiki-extensions-WikibaseRepository.Herald added a subscriber: Aklapper.
TASK DESCRIPTIONIt's not possible to create abuse filters that work with item creations, according to this discussion. This creates an issue with
Marostegui added a comment.
Should be done together with T179106 as that task requires depooling servers most likely.TASK DETAILhttps://phabricator.wikimedia.org/T179793EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MarosteguiCc: Marostegui, Aklapper,
Stashbot added a comment.
Mentioned in SAL (#wikimedia-operations) [2017-11-06T09:37:49Z] <_joe_> manually running htmlCacheUpdate for commonswiki and ruwiki on terbium, T173710TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL
BBlack lowered the priority of this task from "High" to "Normal".BBlack changed the task status from "Open" to "Stalled".BBlack added a comment.
The timeout changes above will offer some insulation, and as time passes we're not seeing evidence of this problem recurring with the do_stream=false
Anomie added a comment.
In T174298#3734602, @Yurik wrote:
@Anomie - is there an easy api way to get resolved template parameters on a wiki page via a GET request?
No. That would involve some particularly deep diving into the parser internals.
You can use action=""> to get the wikitext
gerritbot added a comment.
Change 387228 merged by BBlack:
[operations/puppet@production] cache_text: reduce inter-cache backend timeouts as well
https://gerrit.wikimedia.org/r/387228TASK DETAILhttps://phabricator.wikimedia.org/T179156EMAIL
Lucas_Werkmeister_WMDE updated the task description. (Show Details)
CHANGES TO TASK DESCRIPTION...There is an `rdf2hdt` tool ([link](https://github.com/rdfhdt/hdt-cpp/tree/develop/libhdt); LGPLv2.1+) that can convert TTL dumps to HDT files. Unfortunately, it doesn’t run in a streaming fashion (it
Lucas_Werkmeister_WMDE added a comment.
I ran the conversion directly from the ttl.gz file
Interesting, I couldn’t get that to work and had to pipe gunzip output into the program.
I also tried converting the latest dump, and since I don’t have access to any system with that much RAM, I thought I
Lea_Lacroix_WMDE created this task.Lea_Lacroix_WMDE added a project: Wikidata.Herald added a subscriber: Aklapper.
TASK DESCRIPTIONCurrently in the search engine of Wikidata, it's not possible to find a paper by the name of its author. These results can be found with the Query Service but not
WMDE-leszek added a comment.
Thanks @daniel and @Smalyshev for the feedback so far.
Submitted https://github.com/wmde/WikibaseDataModel/pull/767 to get those changes made. Feel free to review and comment wherever you find applicable.TASK DETAILhttps://phabricator.wikimedia.org/T162371EMAIL
Lea_Lacroix_WMDE added a comment.
Discussion here: https://www.wikidata.org/wiki/Wikidata:Contact_the_development_team#Articles_not_searchableTASK DETAILhttps://phabricator.wikimedia.org/T179815EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To:
Arkanosis added a comment.
In T179681#3736916, @Lucas_Werkmeister_WMDE wrote:
I ran the conversion directly from the ttl.gz file
Interesting, I couldn’t get that to work and had to pipe gunzip output into the program.
Interesting, indeed… Could it be that you added the -f ttl flag afterwards?
WMDE-leszek changed the task status from "Stalled" to "Open".WMDE-leszek moved this task from Doing to Review on the Wikidata-Sprint board.WMDE-leszek added a comment.
Unstalling this. Some changes suggested in the benchmarking ticket has been made:
gerritbot added a comment.
Change 388524 merged by jenkins-bot:
[wikidata/query/gui@master] Only initialize code example dialog when shown
https://gerrit.wikimedia.org/r/388524TASK DETAILhttps://phabricator.wikimedia.org/T179535EMAIL
Addshore added a comment.
In T179681#3736044, @Addshore wrote:
@Smalyshev we discussed dumping the JNL files used by blaze graph directly at points during wikidata con.
I'm aware that isnt a HDT dump, but im wondering if this would help in any way.
Can we reliably get a consistent snapshot of
Lucas_Werkmeister_WMDE closed this task as "Resolved".Lucas_Werkmeister_WMDE moved this task from Review to Done on the Wikidata-Sprint board.Lucas_Werkmeister_WMDE added a comment.
Should be fixed (as soon as the fix is deployed).TASK
Lucas_Werkmeister_WMDE added a comment.
manually doing regex-style parsing of Wiki markup in _javascript_ is a guaranteed path to hell. Trust me on this one :)
Yes, I guessed as much :) but we would only be doing that for custom installs anyways (I’d definitely stick to Parsoid for
WMDE-leszek claimed this task.WMDE-leszek moved this task from Proposed to Doing on the Wikidata-Sprint board.
TASK DETAILhttps://phabricator.wikimedia.org/T179662WORKBOARDhttps://phabricator.wikimedia.org/project/board/2351/EMAIL
greg edited projects, added MediaWiki-Platform-Team, Release-Engineering-Team (Watching / External); removed Release-Engineering-Team.greg added a subscriber: bd808.greg added a comment.
Adding #mediawiki-platform-team and @bd808 for their advice.TASK
Legoktm added a comment.
If Wikibase uses a feature of ~2.0 that is incompatible with 1.0.0, it'll blow up in production. The CI that currently runs on mediawiki/extensions/Wikidata pulls in only "require-dev" via composer, and uses mediawiki/vendor for the rest, so if there's a mismatch that is
SandraF_WMF lowered the priority of this task from "Normal" to "Low".
TASK DETAILhttps://phabricator.wikimedia.org/T173978EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SandraF_WMFCc: Abit, Ramsey-WMF, Aklapper, SandraF_WMF, Lahi, PDrouin-WMF, E1presidente,
Smalyshev edited projects, added Technical-Debt; removed Regression.
TASK DETAILhttps://phabricator.wikimedia.org/T179091EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: Aklapper, EBernhardson, dcausse, Smalyshev, Lahi, GoranSMilovanovic, QZanden,
SandraF_WMF lowered the priority of this task from "Normal" to "Low".
TASK DETAILhttps://phabricator.wikimedia.org/T174168EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SandraF_WMFCc: Abit, PokestarFan, Elitre, Aklapper, Ramsey-WMF, SandraF_WMF, Lahi,
SandraF_WMF lowered the priority of this task from "Normal" to "Low".
TASK DETAILhttps://phabricator.wikimedia.org/T173979EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SandraF_WMFCc: PKM, Deskana, EBernhardson, Abit, El_Grafo, Yann, Keegan, PierreSelim,
SandraF_WMF lowered the priority of this task from "Normal" to "Low".
TASK DETAILhttps://phabricator.wikimedia.org/T177204EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SandraF_WMFCc: Aklapper, Sadads, PokestarFan, Elitre, Qgil, Abit, SandraF_WMF, Lahi,
SandraF_WMF raised the priority of this task from "Normal" to "High".
TASK DETAILhttps://phabricator.wikimedia.org/T175675EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SandraF_WMFCc: BVershbow_WMF, Sadads, Abit, SandraF_WMF, Aklapper, Lahi, PDrouin-WMF,
SandraF_WMF added a comment.
Done. Internal documents were collected and archived; on-wiki information that was still valid/useful was processed and reworked into the updated information and documentation pages about Structured Commons; other older on-wiki documents were archived. Relevant links:
SandraF_WMF closed this task as "Resolved".
TASK DETAILhttps://phabricator.wikimedia.org/T173924EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SandraF_WMFCc: Abit, SandraF_WMF, Aklapper, Lahi, PDrouin-WMF, E1presidente, Ramsey-WMF, GoranSMilovanovic,
SandraF_WMF added a comment.
Working on it at the moment.TASK DETAILhttps://phabricator.wikimedia.org/T175675EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SandraF_WMFCc: BVershbow_WMF, Sadads, Abit, SandraF_WMF, Aklapper, Lahi, PDrouin-WMF, E1presidente,
SandraF_WMF created subtask T179908: Clarify in FAQ how Structured Commons will work with categories on Wikimedia Commons.
TASK DETAILhttps://phabricator.wikimedia.org/T174168EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SandraF_WMFCc: Abit, PokestarFan,
SandraF_WMF created this task.SandraF_WMF added projects: Documentation, Structured-Data-Commons, Wikidata, Community-Liaisons (Oct-Dec 2017).
TASK DESCRIPTIONSome more clarification about categories on Commons is needed. Addressed by writing a more in-depth question and answer about how
Base added a comment.
While I support the request completely, the query linked can be optimised by using just one label language and dumping slow label service, I was able to get results for Wikipedia as big as German: http://tinyurl.com/ycbr7zq2TASK
bd808 added a comment.
@Legoktm nailed it. If wikibase is going to use the same process as the rest of the Foundation production deployed extensions it needs to go all in on that including in the CI chain and the library update process. The current system can probably be improved, but that is a
Ijon created this task.Ijon added a project: Wikidata-Query-Service.Herald added a subscriber: Aklapper.Herald added projects: Wikidata, Discovery.
TASK DESCRIPTIONCurrently, WDQS's timeout prevents many legitimate queries involving large datasets (e.g. various counting queries tied to large
Lucas_Werkmeister_WMDE added a comment.
TemplateSandbox doesn’t seem to be part of a default MediaWiki installation, so it wouldn’t help in @Addshore’s case. I think the parse tree is our best bet for now – there shouldn’t be any nested templates (other than the {{!}} workaround for | in queries),
Lucas_Werkmeister_WMDE claimed this task.Lucas_Werkmeister_WMDE added a project: Wikidata-Sprint.Lucas_Werkmeister_WMDE moved this task from incoming to in current sprint on the Wikidata board.Lucas_Werkmeister_WMDE added a comment.
Are there any cases where “type” constraints do make sense in
gerritbot added a comment.
Change 389489 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)):
[mediawiki/extensions/WikibaseQualityConstraints@master] Skip “type” constraints on references
https://gerrit.wikimedia.org/r/389489TASK
gerritbot added a project: Patch-For-Review.
TASK DETAILhttps://phabricator.wikimedia.org/T178147EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lucas_Werkmeister_WMDE, gerritbotCc: gerritbot, Aklapper, Lucas_Werkmeister_WMDE, Lahi, Lordiis,
Jonas added a comment.
Is this the point where my workaround becomes a feature?
Maybe this is a chance to get rid of the Wikitext parsing!
We have a property on Wikidata, but unfortunately we also have a very small size limit.
I think it would be really cool to use SPARQL for querying and
gerritbot added a comment.
Change 387225 merged by BBlack:
[operations/puppet@production] cache_text: reduce applayer timeouts to reasonable values
https://gerrit.wikimedia.org/r/387225TASK DETAILhttps://phabricator.wikimedia.org/T179156EMAIL
Lucas_Werkmeister_WMDE added a subscriber: daniel.Lucas_Werkmeister_WMDE added a comment.
@daniel pointed out that we can use action="">. This returns an XML tree like this:
=== Cats ===
\n
SPARQL2
query
=
SELECT ?item ?itemLabel \nWHERE \n{\n ?item
Franziska_Heine added a project: Wikidata-Test-Sprint.
TASK DETAILhttps://phabricator.wikimedia.org/T179650EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Franziska_HeineCc: gerritbot, thiemowmde, Ladsgroup, Jakob_WMDE, Jonas, Lydia_Pintscher, Aklapper,
Ladsgroup added a comment.
Hey,
All of them are admin in wikidata-federated project now. that project includes federated-commons.eqiad.wmflabs (http://structured-commons.wmflabs.org) and federated-wikidata.eqiad.wmflabs (http://federated-wikidata.wmflabs.org)TASK
Hjfocs updated the task description. (Show Details)
CHANGES TO TASK DESCRIPTION...`curl -F "user=${USER} -F "name=${DATASET_NAME}" -F "dataset=@${DATASET_FILE}" ${HOST}/bigdata/upload`...TASK DETAILhttps://phabricator.wikimedia.org/T170822EMAIL
Kiailandi claimed this task.
TASK DETAILhttps://phabricator.wikimedia.org/T170822EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: KiailandiCc: Aklapper, Hjfocs, Lahi, GoranSMilovanovic, Kiailandi, QZanden, dachary, Wikidata-bugs, aude, Ricordisamoa,
69 matches
Mail list logo
