[Wikidata-bugs] [Maniphest] [Updated] T141813: Add full-text search support to Query Service

2018-04-12 Thread hoo
hoo removed a parent task: T86530: Replace wb_terms table with more specialized mechanisms for terms (tracking). TASK DETAILhttps://phabricator.wikimedia.org/T141813EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: Realworldobject, jiemakel, PokestarFan

[Wikidata-bugs] [Maniphest] [Updated] T125500: [Epic] Index Wikidata labels and descriptions as separate fields in ElasticSearch

2018-04-12 Thread hoo
hoo added a parent task: T141813: Add full-text search support to Query Service. TASK DETAILhttps://phabricator.wikimedia.org/T125500EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Smalyshev, hooCc: K4-713, Nikki, Lydia_Pintscher, Lea_Lacroix_WMDE, Stashbot

[Wikidata-bugs] [Maniphest] [Updated] T141813: Add full-text search support to Query Service

2018-04-12 Thread hoo
hoo added a subtask: T125500: [Epic] Index Wikidata labels and descriptions as separate fields in ElasticSearch. TASK DETAILhttps://phabricator.wikimedia.org/T141813EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: Realworldobject, jiemakel, PokestarFan

[Wikidata-bugs] [Maniphest] [Updated] T86530: Replace wb_terms table with more specialized mechanisms for terms (tracking)

2018-04-12 Thread hoo
hoo removed a subtask: T141813: Add full-text search support to Query Service. TASK DETAILhttps://phabricator.wikimedia.org/T86530EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: jcrespo, Ricordisamoa, Lydia_Pintscher, adrianheine, thiemowmde

[Wikidata-bugs] [Maniphest] [Unblock] T88991: improve Wikidata dumps [tracking]

2018-04-12 Thread hoo
hoo closed subtask T187888: "Failed to dump Q12129 (Value must be at most 127 characters long.)" when dumping Wikidata as TTL as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T88991EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferenc

[Wikidata-bugs] [Maniphest] [Closed] T187888: "Failed to dump Q12129 (Value must be at most 127 characters long.)" when dumping Wikidata as TTL

2018-04-12 Thread hoo
hoo closed this task as "Resolved".hoo removed a project: Patch-For-Review.hoo claimed this task.hoo added a comment. From next week on it should not be possible to add such invalid data anymore.TASK DETAILhttps://phabricator.wikimedia.org/T187888EMAIL PREFERENCEShttps://phabricator.wik

[Wikidata-bugs] [Maniphest] [Unblock] T56318: Quantity datatype (tracking)

2018-04-12 Thread hoo
hoo closed subtask T155910: Erroneous digits in QuantityValue as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T56318EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: TerraCodes, jeblad, DixonD, Darkdadaah, Aklapper, Wolfvoll, Klo

[Wikidata-bugs] [Maniphest] [Closed] T155910: Erroneous digits in QuantityValue

2018-04-12 Thread hoo
hoo closed this task as "Resolved".hoo removed a project: Patch-For-Review. TASK DETAILhttps://phabricator.wikimedia.org/T155910EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: thiemowmde, hooCc: gerritbot, hoo, matej_suchanek, Lydia_Pintsch

[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata

2018-04-12 Thread hoo
hoo added a comment. I just added documentation for the key: https://www.mediawiki.org/w/index.php?diff=2757174=2675575TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: Edgars2007, Daniel_Mietchen

[Wikidata-bugs] [Maniphest] [Updated] T122350: Use scap3 to deploy DCAT-AP

2018-04-12 Thread hoo
hoo added a comment. We might also want to make this an actual MediaWiki extension (possibly after T192073: Make DCAT-AP use Purtle and add unit tests), as it interacts with MediaWiki (uses MediaWiki: pages) and that would generally fit our infrastructure well. This would potentially ease

[Wikidata-bugs] [Maniphest] [Closed] T125996: Invalidate (caching) PropertyInfoStore when rebuilding wb_property_info

2018-04-12 Thread hoo
hoo closed this task as "Invalid".hoo added a comment. The problem here was (and still is) with the cache being split between Zend and HHVM (which will eventually go away with HHVM). There's nothing really we can do about this in the software (as it doesn't know about the respective o

[Wikidata-bugs] [Maniphest] [Merged] T186286: Wikibase selenium test is flaky

2018-04-12 Thread hoo
hoo closed this task as a duplicate of T189762: selenium test for Wikibase is unstable. TASK DETAILhttps://phabricator.wikimedia.org/T186286EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: Lucas_Werkmeister_WMDE, Aklapper, Tgr, Lahi, Gq86

[Wikidata-bugs] [Maniphest] [Updated] T189762: selenium test for Wikibase is unstable

2018-04-12 Thread hoo
hoo added subscribers: Tgr, Lucas_Werkmeister_WMDE.hoo merged a task: T186286: Wikibase selenium test is flaky. TASK DETAILhttps://phabricator.wikimedia.org/T189762EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: Lucas_Werkmeister_WMDE, Tgr, Jonas, WMDE

[Wikidata-bugs] [Maniphest] [Closed] T58653: Viewing or restoring revdel'ed revisions of entities doesn't work

2018-04-12 Thread hoo
hoo closed this task as "Invalid".hoo added a comment. This works these days…TASK DETAILhttps://phabricator.wikimedia.org/T58653EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: hoo, Ricordisamoa, Aklapper, liangent, Wikidata-bugs, Adds

[Wikidata-bugs] [Maniphest] [Unblock] T65188: validate sitelinks (value and uniqueness) in ChangeOp

2018-04-12 Thread hoo
hoo closed subtask T65190: Implement batched ChangeOps for terms and sitelinks as "Declined". TASK DETAILhttps://phabricator.wikimedia.org/T65188EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: Ricordisamoa, Aklapper, Wikidata-bugs, Lydia

[Wikidata-bugs] [Maniphest] [Declined] T65190: Implement batched ChangeOps for terms and sitelinks

2018-04-12 Thread hoo
hoo closed this task as "Declined".hoo added a comment. No need for this now. If this is needed for a specific type of ChangeOp at some point, we can create a specific ticket for that.TASK DETAILhttps://phabricator.wikimedia.org/T65190EMAIL PREFERENCEShttps://phabricator.wikimedia.or

[Wikidata-bugs] [Maniphest] [Updated] T127169: The property parser function and mw.wikibase.entity.formatPropertyValues should resolve item redirects when formatting Snak values

2018-04-12 Thread hoo
hoo added a parent task: T112073: Lua in Wikibase (tracking). TASK DETAILhttps://phabricator.wikimedia.org/T127169EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: Laddo, TomT0m, aude, Aklapper, Lydia_Pintscher, StudiesWorld, hoo, Lahi, Gq86

[Wikidata-bugs] [Maniphest] [Updated] T112073: Lua in Wikibase (tracking)

2018-04-12 Thread hoo
hoo added a subtask: T127169: The property parser function and mw.wikibase.entity.formatPropertyValues should resolve item redirects when formatting Snak values. TASK DETAILhttps://phabricator.wikimedia.org/T112073EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Commented On] T191631: Add maintenance script to wipe term_search_key and term_weight columns

2018-04-12 Thread hoo
hoo added a comment. In T191631#4126290, @Lucas_Werkmeister_WMDE wrote: Also I wonder, why we even bothering clearing out term_weight… having 0.0 probably has very little/ no benefit compared to just having something in there. Hm, good point… but perhaps it could be confusing for replica db

[Wikidata-bugs] [Maniphest] [Edited] T190513: Make sure Wikidata entity dump scripts run for only about 1-2hours

2018-04-12 Thread hoo
hoo updated the task description. (Show Details) CHANGES TO TASK DESCRIPTION...[x] Wait for at least 1.31.0-wmf.28, better 1.31.0-wmf.29 (so that we're safe from branch rollbacks) to be deployed. [] Adapt the JSON dump bash-scripts for this. [] Adapt the RDF dump bash-scripts for this. Also use

[Wikidata-bugs] [Maniphest] [Closed] T192011: If Item creation fails with a label+description collision, Special:NewItem's creation form points to "NewItem"

2018-04-12 Thread hoo
hoo closed this task as "Resolved".hoo removed a project: Patch-For-Review.hoo moved this task from Needs Review to Done on the Wikidata-Ministry-Of-Magic board. TASK DETAILhttps://phabricator.wikimedia.org/T192011WORKBOARDhttps://phabricator.wikimedia.org/project/board/3273/EMAIL PREFER

[Wikidata-bugs] [Maniphest] [Commented On] T192014: Packagist is not picking up data-values/number 0.10.0

2018-04-12 Thread hoo
hoo added a comment. @thiemowmde I just signed up as https://packagist.org/users/mariushoch/ :)TASK DETAILhttps://phabricator.wikimedia.org/T192014EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: thiemowmde, hooCc: Smalyshev, daniel, Addshore, JeroenDeDauw

[Wikidata-bugs] [Maniphest] [Commented On] T132839: [RfC] Property suggester suggests human properties for non-human items

2018-04-11 Thread hoo
hoo added a comment. Just to give an idea of what I was working on last: Right now we take the correlation based on all properties that are present on an Item (just by their presence!). Also we take the the correlation based on the instance of/ subclass of values. This is when being aggregated

[Wikidata-bugs] [Maniphest] [Created] T192026: Add an API parameter analogue to "maxLag" for dispatch lag

2018-04-11 Thread hoo
hoo created this task.hoo added projects: MediaWiki-extensions-WikibaseRepository, Wikidata.Herald added a subscriber: Aklapper. TASK DESCRIPTIONAdditionally to the measure discussed in T184948, we should have parameter like "maxLag", which acts on replication lag. With that a bot c

[Wikidata-bugs] [Maniphest] [Edited] T192014: Packagist is not picking up data-values/number 0.10.0

2018-04-11 Thread hoo
hoo updated the task description. (Show Details) CHANGES TO TASK DESCRIPTIONEven though the configuration for Packagist on GitHub is there, Packagist is not picking up the new [[https://github.com/DataValues/Number/releases/tag/0.10.0|`0.10.0`]] release of the component (https://packagist.org

[Wikidata-bugs] [Maniphest] [Updated] T192014: Packagist is not picking up data-values/number 0.10.0

2018-04-11 Thread hoo
hoo added a parent task: T155910: Erroneous digits in QuantityValue. TASK DETAILhttps://phabricator.wikimedia.org/T192014EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: Addshore, JeroenDeDauw, Lydia_Pintscher, thiemowmde, Aklapper, hoo, Lahi, Gq86

[Wikidata-bugs] [Maniphest] [Updated] T155910: Erroneous digits in QuantityValue

2018-04-11 Thread hoo
hoo added a subtask: T192014: Packagist is not picking up data-values/number 0.10.0. TASK DETAILhttps://phabricator.wikimedia.org/T155910EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: thiemowmde, hooCc: hoo, matej_suchanek, Lydia_Pintscher, daniel, Smalyshev

[Wikidata-bugs] [Maniphest] [Created] T192014: Packagist is not picking up data-values/number 0.10.0

2018-04-11 Thread hoo
hoo created this task.hoo added projects: Wikidata, DataValues.Herald added a subscriber: Aklapper. TASK DESCRIPTIONEven though the configuration for Packagist on GitHub is there, Packagist is not picking up the new 0.10.0 release of the component. I already deleted the release once and re-tagged

[Wikidata-bugs] [Maniphest] [Commented On] T191631: Add maintenance script to wipe term_search_key and term_weight columns

2018-04-11 Thread hoo
hoo added a comment. In T191631#4120568, @Lucas_Werkmeister_WMDE wrote: Does the script need an option to stop after a certain amount of work has been done? So far I’m following rebuildTermSqlIndex and adding --from-id and --batch-size options, but no --limit options or anything like that. We

[Wikidata-bugs] [Maniphest] [Updated] T185165: Incomplete error message in Wikibase api wbeditentity when omitting title or site parameters

2018-04-11 Thread hoo
hoo added a project: Wikidata-Ministry-Of-Magic. TASK DETAILhttps://phabricator.wikimedia.org/T185165EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: gerritbot, Aklapper, valerio.bozzolan, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30

[Wikidata-bugs] [Maniphest] [Created] T192011: If Item creation fails with a label+description collision, Special:NewItem's creation form points to "NewItem"

2018-04-11 Thread hoo
hoo created this task.hoo added projects: MediaWiki-extensions-WikibaseRepository, Wikidata, Wikidata-Ministry-Of-Magic.Herald added a subscriber: Aklapper. TASK DESCRIPTIONSteps to reproduce: Go to Special:NewItem Enter the label and description of an existing Item (Note: the language has

[Wikidata-bugs] [Maniphest] [Claimed] T185165: Incomplete error message in Wikibase api wbeditentity when omitting title or site parameters

2018-04-11 Thread hoo
hoo claimed this task. TASK DETAILhttps://phabricator.wikimedia.org/T185165EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: gerritbot, Aklapper, valerio.bozzolan, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007

[Wikidata-bugs] [Maniphest] [Retitled] T150290: add CORS to all redirects in chain from https://www.wikidata.org/entity/{Q...}

2018-04-11 Thread hoo
hoo renamed this task from "add CORS to all redirecs in chain from https://www.wikidata.org/entity/{Q...}" to "add CORS to all redirects in chain from https://www.wikidata.org/entity/{Q...}". TASK DETAILhttps://phabricator.wikimedia.org/T150290EMAIL PREFERENCEShttps://phabr

[Wikidata-bugs] [Maniphest] [Closed] T147798: Wikibase repo UI is sometimes not properly starting up (on Firefox)

2018-04-11 Thread hoo
hoo closed this task as "Invalid". TASK DETAILhttps://phabricator.wikimedia.org/T147798EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: TerraCodes, Gilles, ori, Aklapper, aude, thiemowmde, Jonas, Lydia_Pintscher, Tobi_WMDE_SW, hoo,

[Wikidata-bugs] [Maniphest] [Commented On] T177453: Add wikibase client support for searching wikidata items

2018-04-11 Thread hoo
hoo added a comment. @Smalyshev What's the status here? Say we want to get rid of the wb_terms table…TASK DETAILhttps://phabricator.wikimedia.org/T177453EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: dcausse, hoo, Addshore, daniel, Aklapper, Smalyshev

[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata

2018-04-11 Thread hoo
hoo added a comment. Actually, it seems we could also have only specific limits that are not bypassable. While this is not documented (as far as I can tell), one can add '' => false to a specific rate limit action, like the following: > print_r($wgRateLimits); Array ( … [badoath] =&

[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata

2018-04-11 Thread hoo
hoo added a comment. Note: We currently also have dispatch problems while no one is going faster than 75 edits per minute (as far as I can tell)TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc

[Wikidata-bugs] [Maniphest] [Updated] T184948: limit page creation and edit rate on Wikidata

2018-04-11 Thread hoo
hoo added a comment. If we make sysop and bot subject to rate limits, the "user"-limits from the following apply (unless we set a higher limit for them specifically): P6982 Wikidata: Effective $wgRateLimits Potential problems I could see: move 8 per minute (although not relevant

[Wikidata-bugs] [Maniphest] [Commented On] T190457: Include checksums in https://dumps.wikimedia.org/wikidatawiki/entities/

2018-04-11 Thread hoo
hoo added a comment. JSON checksums look fine as well: hoo@snapshot1007:/mnt/dumpsdata/otherdumps/wikibase/wikidatawiki/20180409$ md5sum -c wikidata-20180409-md5sums.txt wikidata-20180409-all.json.gz: OK hoo@snapshot1007:/mnt/dumpsdata/otherdumps/wikibase/wikidatawiki/20180409$ sha1sum -c

[Wikidata-bugs] [Maniphest] [Merged] T177486: [Tracking] Wikidata entity dumpers need to cope with the immense Wikidata growth recently

2018-04-10 Thread hoo
hoo merged a task: T128875: How can we speed up the wikidata entity dumps?. TASK DETAILhttps://phabricator.wikimedia.org/T177486EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: Stashbot, Sjoerddebruin, gerritbot, thiemowmde, Aklapper, ezachte, daniel

[Wikidata-bugs] [Maniphest] [Commented On] T128875: How can we speed up the wikidata entity dumps?

2018-04-10 Thread hoo
hoo added a comment. As I understand it these take over 12 hours now and will only get worse over time. I would love for them to take 12 hours (or even 24) these days :DTASK DETAILhttps://phabricator.wikimedia.org/T128875EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel

[Wikidata-bugs] [Maniphest] [Updated] T128875: How can we speed up the wikidata entity dumps?

2018-04-10 Thread hoo
hoo closed this task as a duplicate of T177486: [Tracking] Wikidata entity dumpers need to cope with the immense Wikidata growth recently. TASK DETAILhttps://phabricator.wikimedia.org/T128875EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: ArielGlenn

[Wikidata-bugs] [Maniphest] [Unblock] T161592: Account for foreign repositories in RDF mapping

2018-04-10 Thread hoo
hoo closed subtask T162371: Benchmark RDF dump with foreign ID mapping as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T161592EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: WMDE-leszek, hooCc: PokestarFan, Lucas_Werkmeister_WMDE,

[Wikidata-bugs] [Maniphest] [Closed] T162371: Benchmark RDF dump with foreign ID mapping

2018-04-10 Thread hoo
hoo closed this task as "Resolved".hoo claimed this task.hoo added a comment. As far as I can tell this is all done.TASK DETAILhttps://phabricator.wikimedia.org/T162371EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: hoo, Ladsgroup, P

[Wikidata-bugs] [Maniphest] [Updated] T115223: Provide wikidata downloads as multiple files to make access more robust and efficient

2018-04-10 Thread hoo
hoo added a parent task: T88991: improve Wikidata dumps [tracking]. TASK DETAILhttps://phabricator.wikimedia.org/T115223EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: abian, JanZerebecki, Hydriz, hoo, Halfak, NealMcB, Aklapper, Lahi, Gq86

[Wikidata-bugs] [Maniphest] [Updated] T88991: improve Wikidata dumps [tracking]

2018-04-10 Thread hoo
hoo added a subtask: T115223: Provide wikidata downloads as multiple files to make access more robust and efficient . TASK DETAILhttps://phabricator.wikimedia.org/T88991EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: Lazhar, PokestarFan, Ricordisamoa

[Wikidata-bugs] [Maniphest] [Updated] T88991: improve Wikidata dumps [tracking]

2018-04-10 Thread hoo
hoo added a subtask: T191639: Wikidata JSON dumps do not have the 'ns' (namespace). TASK DETAILhttps://phabricator.wikimedia.org/T88991EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: Lazhar, PokestarFan, Ricordisamoa, Denis.bykov, Jimkont, JanZerebecki

[Wikidata-bugs] [Maniphest] [Updated] T191639: Wikidata JSON dumps do not have the 'ns' (namespace)

2018-04-10 Thread hoo
hoo added a parent task: T88991: improve Wikidata dumps [tracking]. TASK DETAILhttps://phabricator.wikimedia.org/T191639EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: Chicocvenancio, marcmiquel, Lahi, Gq86, GoranSMilovanovic, Lunewa, QZanden

[Wikidata-bugs] [Maniphest] [Closed] T70792: Wikidata JSON dump: filename prefix

2018-04-10 Thread hoo
hoo closed this task as "Resolved".hoo added a comment. Dumps are now (primarily) stored like https://dumps.wikimedia.org/wikidatawiki/entities/20180402/TASK DETAILhttps://phabricator.wikimedia.org/T70792EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailp

[Wikidata-bugs] [Maniphest] [Unblock] T88991: improve Wikidata dumps [tracking]

2018-04-10 Thread hoo
hoo closed subtask T70792: Wikidata JSON dump: filename prefix as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T88991EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: Lazhar, PokestarFan, Ricordisamoa, Denis.bykov, Jimkont, JanZereb

[Wikidata-bugs] [Maniphest] [Updated] T70792: Wikidata JSON dump: filename prefix

2018-04-10 Thread hoo
hoo added a parent task: T88991: improve Wikidata dumps [tracking]. TASK DETAILhttps://phabricator.wikimedia.org/T70792EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: ArielGlenn, hooCc: Wikidata-bugs, Nemo_bis, mkroetzsch, Svick, Lydia_Pintscher, hoo

[Wikidata-bugs] [Maniphest] [Updated] T88991: improve Wikidata dumps [tracking]

2018-04-10 Thread hoo
hoo added a subtask: T70792: Wikidata JSON dump: filename prefix. TASK DETAILhttps://phabricator.wikimedia.org/T88991EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: Lazhar, PokestarFan, Ricordisamoa, Denis.bykov, Jimkont, JanZerebecki, aude

[Wikidata-bugs] [Maniphest] [Updated] T155103: Create a truthy nt dump

2018-04-10 Thread hoo
hoo added a parent task: T88991: improve Wikidata dumps [tracking]. TASK DETAILhttps://phabricator.wikimedia.org/T155103EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: Lea_Lacroix_WMDE, gerritbot, Hadyelsahar, Smalyshev, Lydia_Pintscher, hoo, Lucie

[Wikidata-bugs] [Maniphest] [Updated] T88991: improve Wikidata dumps [tracking]

2018-04-10 Thread hoo
hoo added a subtask: T155103: Create a truthy nt dump. TASK DETAILhttps://phabricator.wikimedia.org/T88991EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: Lazhar, PokestarFan, Ricordisamoa, Denis.bykov, Jimkont, JanZerebecki, aude, Liuxinyu970226

[Wikidata-bugs] [Maniphest] [Updated] T151876: Consider using pigz (Zopfli) for Wikidata JSON dump

2018-04-10 Thread hoo
hoo added a parent task: T88991: improve Wikidata dumps [tracking]. TASK DETAILhttps://phabricator.wikimedia.org/T151876EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: ArielGlenn, Aklapper, hoo, Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer

[Wikidata-bugs] [Maniphest] [Updated] T88991: improve Wikidata dumps [tracking]

2018-04-10 Thread hoo
hoo added a subtask: T151876: Consider using pigz (Zopfli) for Wikidata JSON dump. TASK DETAILhttps://phabricator.wikimedia.org/T88991EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: Lazhar, PokestarFan, Ricordisamoa, Denis.bykov, Jimkont, JanZerebecki

[Wikidata-bugs] [Maniphest] [Closed] T185598: property-suggester-scripts master can't be installed on tool forge

2018-04-10 Thread hoo
hoo closed this task as "Resolved".hoo removed a project: Patch-For-Review. TASK DETAILhttps://phabricator.wikimedia.org/T185598EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Ladsgroup, hooCc: gerritbot, Aklapper, hoo, Lahi, Gq86, GoranSMilovanovi

[Wikidata-bugs] [Maniphest] [Commented On] T190513: Make sure Wikidata entity dump scripts run for only about 1-2hours

2018-04-10 Thread hoo
hoo added a comment. The dump script calls will basically look like this soon: php repo/maintenance/dumpJson.php --wiki wikidatawiki --first-page-id `expr $i \* 40 \* $shards + 1` --last-page-id `expr \( $i + 1 \) \* 40 \* $shards`TASK DETAILhttps://phabricator.wikimedia.org/T190513EMAIL

[Wikidata-bugs] [Maniphest] [Unblock] T112073: Lua in Wikibase (tracking)

2018-04-10 Thread hoo
hoo closed subtask T191576: Add mw.wikibase.entity:getId to easily get the id (serialization) of an entity as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T112073EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: PokestarFan, Liux

[Wikidata-bugs] [Maniphest] [Closed] T191576: Add mw.wikibase.entity:getId to easily get the id (serialization) of an entity

2018-04-10 Thread hoo
hoo closed this task as "Resolved".hoo moved this task from Needs Review to Done on the Wikidata-Ministry-Of-Magic board.hoo removed a project: Patch-For-Review. TASK DETAILhttps://phabricator.wikimedia.org/T191576WORKBOARDhttps://phabricator.wikimedia.org/project/board/3273/EMAIL PREFER

[Wikidata-bugs] [Maniphest] [Commented On] T190513: Make sure Wikidata entity dump scripts run for only about 1-2hours

2018-04-10 Thread hoo
hoo added a comment. I just noticed that we could also use: php maintenance/sql.php --wiki wikidatawiki --json --query 'SELECT MAX(page_id) AS max_page_id FROM page' | grep max_page_id | grep -oP '\d+' That's maybe simpler for just getting this one bit of information.TASK DETAILhttps

[Wikidata-bugs] [Maniphest] [Edited] T190513: Make sure Wikidata entity dump scripts run for only about 1-2hours

2018-04-10 Thread hoo
hoo updated the task description. (Show Details) CHANGES TO TASK DESCRIPTION...[x] Find out how many entities/pages we need to include, so that the scripts run for about 1-2 hours. -> 400,000 per run (initially) [] Wait for at least 1.31.0-wmf.28, better 1.31.0-wmf.29 (so that we're safe f

[Wikidata-bugs] [Maniphest] [Updated] T155910: Erroneous digits in QuantityValue

2018-04-10 Thread hoo
hoo removed projects: Need-volunteer, Patch-For-Review.hoo added a comment. Just needs a data-values/number release now.TASK DETAILhttps://phabricator.wikimedia.org/T155910EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: hoo, matej_suchanek

[Wikidata-bugs] [Maniphest] [Claimed] T190513: Make sure Wikidata entity dump scripts run for only about 1-2hours

2018-04-10 Thread hoo
hoo claimed this task.hoo added a project: Wikidata-Ministry-Of-Magic. TASK DETAILhttps://phabricator.wikimedia.org/T190513EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: ArielGlenn, Aklapper, hoo, Lahi, Gq86, GoranSMilovanovic, lisong, QZanden

[Wikidata-bugs] [Maniphest] [Commented On] T190513: Make sure Wikidata entity dump scripts run for only about 1-2hours

2018-04-10 Thread hoo
hoo added a comment. Velocity (taken as average from the three runs listed above): JSON: 214k entities/hour TTL: 189k entities/hour truthy-nt: 157k entities/hour Due to this, I suggest to always run roughly roughly 400k page ids per script run (considering there are possibly missing ones

[Wikidata-bugs] [Maniphest] [Commented On] T190513: Make sure Wikidata entity dump scripts run for only about 1-2hours

2018-04-10 Thread hoo
hoo added a comment. 20180405 truthy-nt dump: Each shard dumped about 8.04m entities in (very roughly) 60h. 20180328 truthy-nt dump: Each shard dumped about 7.96m entities in (very roughly) 47h. TASK DETAILhttps://phabricator.wikimedia.org/T190513EMAIL PREFERENCEShttps://phabricator.wikimedia.org

[Wikidata-bugs] [Maniphest] [Commented On] T190513: Make sure Wikidata entity dump scripts run for only about 1-2hours

2018-04-10 Thread hoo
hoo added a comment. 20180402 TTL dump: Each shard dumped about 8.01m entities in (very roughly) 49h. 20180326 TTL dump: Each shard dumped about 7.95m entities in (very roughly) 35h. 20180319 TTL dump: Each shard dumped about 7.91m entities in (very roughly) 45h. TASK DETAILhttps

[Wikidata-bugs] [Maniphest] [Commented On] T190513: Make sure Wikidata entity dump scripts run for only about 1-2hours

2018-04-10 Thread hoo
hoo added a comment. 20180402 JSON dump: Each shard dumped about 7.70m entities in (very roughly) 40h. 20180326 JSON dump: Each shard dumped about 7.65m entities in (very roughly) 35h. 20180326 JSON dump: Each shard dumped about 7.63m entities in (very roughly) 33h. TASK DETAILhttps

[Wikidata-bugs] [Maniphest] [Changed Project Column] T191082: Missing cross references to JSON datamodel from Lua function documentation

2018-04-09 Thread hoo
hoo moved this task from Needs Review to Done on the Wikidata-Ministry-Of-Magic board.hoo added a comment. It is now explained how entities / statements obtained look like, both by linking to specific documentation and by providing an example. I'm not sure what to do regarding the formatter

[Wikidata-bugs] [Maniphest] [Commented On] T189762: selenium test for Wikibase is unstable

2018-04-09 Thread hoo
hoo added a comment. With the (very high) 90s timeout, it took me quite some tries, but I managed to also hit this: https://integration.wikimedia.org/ci/job/mwext-mw-selenium-composer-jessie/9709/console 12:59:03 timed out after 90 seconds, Element was not visible in 90 seconds (Watir::Wait

[Wikidata-bugs] [Maniphest] [Commented On] T163328: Add the truthy nt dump to dcat-AP

2018-04-09 Thread hoo
hoo added a comment. In T163328#3749159, @Lokal_Profil wrote: Related to this is T154914: Add .nt to DCAT-AP for Wikidata dumps still relevant or have those been replaced by the truthy ones? It's still relevant. We might not do it anytime soon, but we might eventually.TASK DETAILhttps

[Wikidata-bugs] [Maniphest] [Commented On] T189762: selenium test for Wikibase is unstable

2018-04-09 Thread hoo
hoo added a comment. After raising the timeout from 10s to 15s, there was another failure: https://integration.wikimedia.org/ci/job/mwext-mw-selenium-composer-jessie/9702/console 11:23:11 timed out after 15 seconds, Element was not visible in 15 seconds (Watir::Wait::TimeoutError) So

[Wikidata-bugs] [Maniphest] [Commented On] T189762: selenium test for Wikibase is unstable

2018-04-09 Thread hoo
hoo added a comment. Still happening: https://integration.wikimedia.org/ci/job/mwext-mw-selenium-composer-jessie/9697/consoleTASK DETAILhttps://phabricator.wikimedia.org/T189762EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: hoo, greg, zeljkofilipin

[Wikidata-bugs] [Maniphest] [Claimed] T191082: Missing cross references to JSON datamodel from Lua function documentation

2018-04-09 Thread hoo
hoo claimed this task. TASK DETAILhttps://phabricator.wikimedia.org/T191082EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: gerritbot, Multichill, hoo, Snaevar, Aklapper, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86

[Wikidata-bugs] [Maniphest] [Updated] T191082: Missing cross references to JSON datamodel from Lua function documentation

2018-04-06 Thread hoo
hoo added a project: Wikidata-Ministry-Of-Magic. TASK DETAILhttps://phabricator.wikimedia.org/T191082EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: gerritbot, Multichill, hoo, Snaevar, Aklapper, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985

[Wikidata-bugs] [Maniphest] [Closed] T143970: In Lua modules, there is no way to test for validity of Wikidata entity IDs

2018-04-06 Thread hoo
hoo closed this task as "Resolved".hoo removed a project: Patch-For-Review.hoo added a comment. By the end of next week mw.wikibase.isValidEntityId and mw.wikibase.entityExists should be available on all Wikimedia wikis. I'll update the documentation (https://www.mediawik

[Wikidata-bugs] [Maniphest] [Unblock] T112073: Lua in Wikibase (tracking)

2018-04-06 Thread hoo
hoo closed subtask T143970: In Lua modules, there is no way to test for validity of Wikidata entity IDs as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T112073EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: PokestarFan, Liux

[Wikidata-bugs] [Maniphest] [Updated] T118379: bz2 dumps cannot be read with PHP

2018-04-06 Thread hoo
hoo added a parent task: T88991: improve Wikidata dumps [tracking]. TASK DETAILhttps://phabricator.wikimedia.org/T118379EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lydia_Pintscher, hooCc: Addshore, daniel, hoo, Aklapper, JeroenDeDauw, StudiesWorld, Lahi

[Wikidata-bugs] [Maniphest] [Updated] T88991: improve Wikidata dumps [tracking]

2018-04-06 Thread hoo
hoo added a subtask: T118379: bz2 dumps cannot be read with PHP. TASK DETAILhttps://phabricator.wikimedia.org/T88991EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: Lazhar, PokestarFan, Ricordisamoa, Denis.bykov, Jimkont, JanZerebecki, aude

[Wikidata-bugs] [Maniphest] [Commented On] T179155: Find a better solution for dewiki's Modul:Wikidata isParent

2018-04-05 Thread hoo
hoo added a comment. In T179155#4108520, @thiemowmde wrote: […] you will end up supporting a search from multiple source IDs to the target IDs anyways, because you’ll get into that situation as soon as there’s more than one link to follow anywhere in the chain. Yes, I think this will be the case

[Wikidata-bugs] [Maniphest] [Updated] T191576: Add mw.wikibase.entity:getId to easily get the id (serialization) of an entity

2018-04-05 Thread hoo
hoo added a project: Wikidata-Ministry-Of-Magic. TASK DETAILhttps://phabricator.wikimedia.org/T191576EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: gerritbot, Multichill, Aklapper, hoo, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30

[Wikidata-bugs] [Maniphest] [Created] T191576: Add mw.wikibase.entity:getId to easily get the id (serialization) of an entity

2018-04-05 Thread hoo
hoo created this task.hoo triaged this task as "Normal" priority.hoo added projects: Wikidata, MediaWiki-extensions-WikibaseClient.Herald added a subscriber: Aklapper. TASK DESCRIPTIONBoth to make it more obvious that this exists, and generally as it is a nice to have function.TASK D

[Wikidata-bugs] [Maniphest] [Edited] T177486: [Tracking] Wikidata entity dumpers need to cope with the immense Wikidata growth recently

2018-04-05 Thread hoo
hoo updated the task description. (Show Details) CHANGES TO TASK DESCRIPTION...[x] Deploy 56906993f95067ec156cf3412f2dabaefce282ad (will happen with 1.31.0-wmf.27 ~~on 2018-03-28~~ in early April)...TASK DETAILhttps://phabricator.wikimedia.org/T177486EMAIL PREFERENCEShttps

[Wikidata-bugs] [Maniphest] [Closed] T190457: Include checksums in https://dumps.wikimedia.org/wikidatawiki/entities/

2018-04-05 Thread hoo
hoo closed this task as "Resolved".hoo added a comment. First checksums are available: https://dumps.wikimedia.org/wikidatawiki/entities/20180402/wikidata-20180402-md5sums.txt and https://dumps.wikimedia.org/wikidatawiki/entities/20180402/wikidata-20180402-sha1sums.txt

[Wikidata-bugs] [Maniphest] [Updated] T179155: Find a better solution for dewiki's Modul:Wikidata isParent

2018-04-04 Thread hoo
hoo added a project: Patch-For-Review.hoo added a comment. I just spend ages trying to find an accurate name and class+function documentation, but at https://github.com/wmde/WikibaseDataModelServices/pull/193 I have a proposal for an interface for a lookup service. I tried hard to catch the use

[Wikidata-bugs] [Maniphest] [Commented On] T179155: Find a better solution for dewiki's Modul:Wikidata isParent

2018-04-04 Thread hoo
hoo added a comment. In T179155#4104857, @thiemowmde wrote: My personal remarks: Please find variable names that are more specific than fromId, toId(s), and propertyId. fromId could be named childId. toIds could be named parentIds. propertyId could be named hasParentPropertyId

[Wikidata-bugs] [Maniphest] [Changed Subscribers] T94019: Generate RDF from JSON

2018-04-04 Thread hoo
hoo added a subscriber: Smalyshev.hoo added a comment. @Smalyshev Why did you mark this Stalled?TASK DETAILhttps://phabricator.wikimedia.org/T94019EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: Smalyshev, hoo, Liuxinyu970226, mkroetzsch, Aklapper

[Wikidata-bugs] [Maniphest] [Commented On] T178047: Investigate why wikidata abstracts dumps are so large

2018-04-04 Thread hoo
hoo added a comment. In T178047#4097208, @ArielGlenn wrote: Well I don't mind a waiting period, let's agree on... one week? It will probably take longer than that for it to get merged and rolled out anyways. But we need an eta before I send the email :-) This is probably somewhere in between

[Wikidata-bugs] [Maniphest] [Commented On] T179155: Find a better solution for dewiki's Modul:Wikidata isParent

2018-04-04 Thread hoo
hoo added a comment. In T179155#4104367, @Lucas_Werkmeister_WMDE wrote: What happens when Kreuzberg has more than one “instance of” statements? Should the surrounding Lua code then call mw.wikibase.isInHierarchy / mw.wikibase.getClosestInHierarchy in a loop? I wonder if it’s necessary to support

[Wikidata-bugs] [Maniphest] [Commented On] T179155: Find a better solution for dewiki's Modul:Wikidata isParent

2018-04-04 Thread hoo
hoo added a comment. Moved this to review to gather comments about my proposal (T179155#4102708).TASK DETAILhttps://phabricator.wikimedia.org/T179155EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: Lucas_Werkmeister_WMDE, thiemowmde, Lydia_Pintscher

[Wikidata-bugs] [Maniphest] [Updated] T190457: Include checksums in https://dumps.wikimedia.org/wikidatawiki/entities/

2018-04-03 Thread hoo
hoo removed a project: Patch-For-Review. TASK DETAILhttps://phabricator.wikimedia.org/T190457EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: gerritbot, ArielGlenn, hoo, abian, Lahi, Gq86, GoranSMilovanovic, lisong, Lunewa, QZanden, LawExplorer

[Wikidata-bugs] [Maniphest] [Updated] T179155: Find a better solution for dewiki's Modul:Wikidata isParent

2018-04-03 Thread hoo
hoo added a comment. Given we don't have anything better at hands, I think for the initial version we should just get full entities (in PHP) and manually traverse the "tree" (much like the modules do it, but with less serialization overhead). For this we might also want to look in

[Wikidata-bugs] [Maniphest] [Commented On] T179155: Find a better solution for dewiki's Modul:Wikidata isParent

2018-04-03 Thread hoo
hoo added a comment. Given what I saw, I suggest the following interfaces: mw.wikibase.isInHierarchy( fromId, toId, propertyId ) -> bool or nil (returns nil in case the maximum recursion depth or some other limit was exhausted) Say we want to find out whether "Kreuzberg&q

[Wikidata-bugs] [Maniphest] [Commented On] T179155: Find a better solution for dewiki's Modul:Wikidata isParent

2018-04-03 Thread hoo
hoo added a comment. Just as a note: I looked into (ab)using the pagelinks table for this purpose. If we want to link an entity to its parent, the pages in question must also be (indirectly) page-linked together. My idea was to get the (indirect) pagelinks from a to b first, and then check

[Wikidata-bugs] [Maniphest] [Created] T191341: Introduce a service for looking up specific statements from entities

2018-04-03 Thread hoo
hoo created this task.hoo added projects: Wikidata, MediaWiki-extensions-WikibaseRepository.Herald added a subscriber: Aklapper. TASK DESCRIPTIONThere should be a service that allows retrieving Statements with a certain property id from a given entity. Initially this can be implemented on top

[Wikidata-bugs] [Maniphest] [Declined] T117534: DCAT-AP: XML produces invalid output with HHVM

2018-04-03 Thread hoo
hoo closed this task as "Declined".hoo added a comment. Per above: The script works fine with Zend (5.5 and 7), and we no longer really care for HHVM.TASK DETAILhttps://phabricator.wikimedia.org/T117534EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Updated] T112073: Lua in Wikibase (tracking)

2018-04-02 Thread hoo
hoo added a subtask: T166059: Track time spent in Wikibase Client data access functionality (parser functions/ Lua) during page renders. TASK DETAILhttps://phabricator.wikimedia.org/T112073EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: PokestarFan

[Wikidata-bugs] [Maniphest] [Updated] T166059: Track time spent in Wikibase Client data access functionality (parser functions/ Lua) during page renders

2018-04-02 Thread hoo
hoo added a parent task: T112073: Lua in Wikibase (tracking). TASK DETAILhttps://phabricator.wikimedia.org/T166059EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: daniel, aude, Lydia_Pintscher, Addshore, Aklapper, hoo, Lahi, Gq86, Darkminds3113

[Wikidata-bugs] [Maniphest] [Updated] T170554: [Wikimania doc sprint] Document the functions of arbitrary access

2018-04-02 Thread hoo
hoo added a parent task: T112073: Lua in Wikibase (tracking). TASK DETAILhttps://phabricator.wikimedia.org/T170554EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: PokestarFan, hoo, thiemowmde, daniel, johl, Lydia_Pintscher, Spinster, Trizek-WMF

[Wikidata-bugs] [Maniphest] [Updated] T112073: Lua in Wikibase (tracking)

2018-04-02 Thread hoo
hoo added a subtask: T170554: [Wikimania doc sprint] Document the functions of arbitrary access. TASK DETAILhttps://phabricator.wikimedia.org/T112073EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: PokestarFan, Liuxinyu970226, Ricordisamoa, aude

<    5   6   7   8   9   10   11   12   13   14   >