[Wikidata-bugs] [Maniphest] T260735: Stop using is_resource()
brion removed a project: TimedMediaHandler. brion added a comment. Removing TimedMediaHandler as the patch landed last month. TASK DETAIL https://phabricator.wikimedia.org/T260735 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: brion Cc: brion, Reedy, alex-mashin, Ricordisamoa, ssastry, TheDJ, Aklapper, MaxSem, G1964j, Zekwn, the0001, Invadibot, Zabe, Selby, AndreCstr, maantietaja, XeroS_SkalibuR, Akuckartz, DannyS712, Nandana, Mirahamira, lucamauri, Lahi, Gq86, Jontheil, Markhalsey, GoranSMilovanovic, Jayprakash12345, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, freephile, Wikidata-bugs, aude, Nikerabbit, Jdforrester-WMF, Addshore, Mbch331, Krenair ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] [Commented On] T95899: MediaWiki core PHPUnit tests do not pass on MediaWiki-Vagrant
brion added a comment. PHP 5 is obsolete; use PHP 7. cd /vagrant/mediawiki sudo -u www-data php tests/phpunit/phpunit.php --wiki wiki TASK DETAILhttps://phabricator.wikimedia.org/T95899EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: brionCc: brion, zeljkofilipin, Addshore, Krinkle, Aklapper, JanZerebecki, bd808, Lahi, jgleeson, Gq86, GoranSMilovanovic, QZanden, LawExplorer, Luke081515, Wikidata-bugs, aude, Mbch331, jeremyb___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T199121: RFC: Spec for representing multiple content objects per revision (MCR) in XML dumps
brion added a comment. As for role ids -- perhaps we should primarily use the names, not the numbers, in the bit. It's analogous to a page's reference (a primary identifier) not to its or (which are provided informatively if you want to repro the database exactly, but can be freely discarded when doing imports and such).TASK DETAILhttps://phabricator.wikimedia.org/T199121EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: ArielGlenn, brionCc: tstarling, awight, JAllemandou, hoo, pmiazga, Nemo_bis, brion, Tgr, Aklapper, Fjalapeno, ArielGlenn, daniel, kostajh, Lahi, PDrouin-WMF, Gq86, E1presidente, Ramsey-WMF, Cparle, Anooprao, SandraF_WMF, GoranSMilovanovic, Lunewa, QZanden, Tramullas, Acer, LawExplorer, JJMC89, Agabi10, Susannaanas, SBisson, gnosygnu, Aschroet, Jane023, Wikidata-bugs, Base, matthiasmullie, aude, GWicke, jayvdb, Ricordisamoa, fbstj, Lydia_Pintscher, Fabrice_Florin, Raymond, santhosh, Jdforrester-WMF, Steinsplitter, Mbch331, Rxy, Jay8g, Ltrlg, bd808, Legoktm___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T199121: RFC: Spec for representing multiple content objects per revision (MCR) in XML dumps
brion added a comment. Ok, proposed transitional schema looks like it imports cleanly via importDump (which uses same code path as Special:Import). The proposed final schema, however, imports a revision with empty text (and throws a notice on Undefined index: text in /vagrant/mediawiki/includes/import/WikiImporter.php on line 886). I know it feels less elegant, but I would recommend sticking with the back-compatible 'transitional' schema and not doing a second transition to minimize impact on interop issues.TASK DETAILhttps://phabricator.wikimedia.org/T199121EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: ArielGlenn, brionCc: tstarling, awight, JAllemandou, hoo, pmiazga, Nemo_bis, brion, Tgr, Aklapper, Fjalapeno, ArielGlenn, daniel, kostajh, Lahi, PDrouin-WMF, Gq86, E1presidente, Ramsey-WMF, Cparle, Anooprao, SandraF_WMF, GoranSMilovanovic, Lunewa, QZanden, Tramullas, Acer, LawExplorer, JJMC89, Agabi10, Susannaanas, SBisson, gnosygnu, Aschroet, Jane023, Wikidata-bugs, Base, matthiasmullie, aude, GWicke, jayvdb, Ricordisamoa, fbstj, Lydia_Pintscher, Fabrice_Florin, Raymond, santhosh, Jdforrester-WMF, Steinsplitter, Mbch331, Rxy, Jay8g, Ltrlg, bd808, Legoktm___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T199121: RFC: Spec for representing multiple content objects per revision (MCR) in XML dumps
brion added a comment. My concern with the two-step transition idea is that some consumers may not update on a reliable schedule, or may not be able to do so easily. For instance, if people are using Special:Export on one wiki and Special:Import'ing those pages on another that's *not* a Wikimedia-hosted site, it's more likely to be an older version of MediaWiki. Should test Special:Import on the proposed transitional schema to make sure what happens. :)TASK DETAILhttps://phabricator.wikimedia.org/T199121EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: ArielGlenn, brionCc: tstarling, awight, JAllemandou, hoo, pmiazga, Nemo_bis, brion, Tgr, Aklapper, Fjalapeno, ArielGlenn, daniel, kostajh, Lahi, PDrouin-WMF, Gq86, E1presidente, Ramsey-WMF, Cparle, Anooprao, SandraF_WMF, GoranSMilovanovic, Lunewa, QZanden, Tramullas, Acer, LawExplorer, JJMC89, Agabi10, Susannaanas, SBisson, gnosygnu, Aschroet, Jane023, Wikidata-bugs, Base, matthiasmullie, aude, GWicke, jayvdb, Ricordisamoa, fbstj, Lydia_Pintscher, Fabrice_Florin, Raymond, santhosh, Jdforrester-WMF, Steinsplitter, Mbch331, Rxy, Jay8g, Ltrlg, bd808, Legoktm___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T178047: Investigate why wikidata abstracts dumps are so large
brion added a comment. In T178047#4073991, @ArielGlenn wrote: In T178047#4073899, @brion wrote: Not sure offhand about the schema; Yahoo's old documentation seems to have vanished from the net. (Probably on the wayback machine but I can't find a URL reference) We don't have a schema in our repos anywhere that must be updated though, right? Right. I'm not sure anything needs changing in the schema though (making the 'abstract' el optional I guess? Existing code makes it optional if the revision isn't filled in but that seems unlikely to occur, so consumers may not expect that) Ideally, I think we'd want a way for the content handler to provide a text extract that can be used here. Isn't there something already for the built-in search dropdown and such? But just stubbing them out is probably fine as a preliminary measure. :) Trust me, from wikidata entities there is nothing useful that can be gotten out as a text abstract. I stuffed a sample semi-pretty-print-formatted revision text here: F15971185 There's the description field, where we could pick a language (English uber alles) and emit "Costa Rican singer". But as a user of the data I'd want the more structured data anyway, probably. :) I think it's fine to just stub them out blank for now. Should we consider retooling this dump to a more manageable... documented... schema? Would have to find out who depends on the current one though. This might be nice future work. I have no idea who relies on this dump though. We could try looking up ips of downloaders but I'm not sure what that would get us, and previous calls of "who uses this?" have fallen on deaf ears. If I were a bit more vicious I would turn them off for a run and see who complained :-P *nod* That may be what it takes. ;)TASK DETAILhttps://phabricator.wikimedia.org/T178047EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: ArielGlenn, brionCc: brion, gerritbot, hoo, ArielGlenn, Versusxo, Majesticalreaper22, Tamgue, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Lunewa, Th3d3v1ls, Ramalepe, Liugev6, QZanden, LawExplorer, Lewizho99, Maathavan, gnosygnu, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T178047: Investigate why wikidata abstracts dumps are so large
brion added a comment. Not sure offhand about the schema; Yahoo's old documentation seems to have vanished from the net. (Probably on the wayback machine but I can't find a URL reference) Ideally, I think we'd want a way for the content handler to provide a text extract that can be used here. Isn't there something already for the built-in search dropdown and such? But just stubbing them out is probably fine as a preliminary measure. :) Should we consider retooling this dump to a more manageable... documented... schema? Would have to find out who depends on the current one though.TASK DETAILhttps://phabricator.wikimedia.org/T178047EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: ArielGlenn, brionCc: brion, gerritbot, hoo, ArielGlenn, Versusxo, Majesticalreaper22, Tamgue, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Lunewa, Th3d3v1ls, Ramalepe, Liugev6, QZanden, LawExplorer, Lewizho99, Maathavan, gnosygnu, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T173346: IIIF and Structured Data on Wikimedia Commons discussion
brion added a comment. Note there's some folks interested in IIIF/Wikimedia discussion from the IIIF end; some notes started recently on this doc: https://docs.google.com/document/d/1lqtwd1rwUIck6nmetxtmQkzuTpgnwOSPREhH9YEjmHI/edit @SandraF_WMF I added a few notes on that doc what I'm interested in from my end and what I cannot guarantee at this time given need to get resources to actually implement things. :) Feel free to get in touch.TASK DETAILhttps://phabricator.wikimedia.org/T173346EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SandraF_WMF, brionCc: Keegan, Ainali, Ramsey-WMF, Swiss-National-Library, BeatEstermann, YULdigitalpreservation, brion, Sadads, Abit, SandraF_WMF, Aklapper, Lahi, PDrouin-WMF, Gq86, E1presidente, Cparle, GoranSMilovanovic, Ivana_Isadora, QZanden, Tramullas, Acer, LawExplorer, Jseddon, FloNight, Trizek-WMF, Susannaanas, Aschroet, Jane023, Wikidata-bugs, PKM, Base, matthiasmullie, aude, Ricordisamoa, Lydia_Pintscher, Fabrice_Florin, Raymond, Steinsplitter, Mbch331, Elitre, Qgil___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T105845: Page components / content widgets
brion added a comment. The idea's quite interesting but has fallen out of discussion some time ago. Shall we remove from the TechCom RFC list, or is there a party interested in taking it back on?TASK DETAILhttps://phabricator.wikimedia.org/T105845EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: brionCc: brion, leila, Reasno, SBisson, MZMcBride, Mholloway, RandomDSdevel, jmadler, Bianjiang, LikeLifer, MGChecker, -jem-, Daniel_Mietchen, StudiesWorld, Kelson, Jonas, daniel, Jhernandez, MrStradivarius, JanZerebecki, Quiddity, mobrovac, ssastry, Tgr, Ltrlg, Inez, cscott, TrevorParscal, Jdlrobson, GWicke, Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer, Jrf, Wikidata-bugs, aude, Gryllida, jayvdb, fbstj, santhosh, Arlolra, Jdforrester-WMF, Mbch331, Rxy, Jay8g, bd808, Krenair, Legoktm___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T173346: IIIF and Structured Data on Wikimedia Commons discussion
brion added a comment. @SandraF_WMF note I've been involved in the IIIF's A/V working group on extending the protocol to support audio and video, and have been at a few of the working group meetings for that. There's also a big IIIF working meeting in Toronto coming up in October; if there's serious interest I should probably pop in to that too, or else someone else from multimedia if there's interest in moving forward with stuff more directly.TASK DETAILhttps://phabricator.wikimedia.org/T173346EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SandraF_WMF, brionCc: brion, Sadads, Abit, SandraF_WMF, Aklapper, GoranSMilovanovic, QZanden, Acer, Susannaanas, Izno, Yann, Wikidata-bugs, PKM, Base, matthiasmullie, aude, Ricordisamoa, Fabrice_Florin, Raymond, Mbch331, Tgr___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T107595: [RFC] Multi-Content Revisions
brion added a comment. I wrote up some quick thoughts at https://www.mediawiki.org/wiki/User:Brion_VIBBER/MCR_alternative_thoughts Mainly exploring along two lines: what if we did a model with separate data tables for each new 'slot' instead of a common content-blob interface (possibly more line with Jaime's thoughts?, possibly different) what if we went full in on using subpages, what would it take to support that? The first would be in some ways similar to the MCR model, but with stricter typing, possible benefits in storage and schema consistency, etc but without the conveniences of the common interface for Content blobs. The second might be a much easier transition, but needs better high-level tooling and some new versioning concepts. May be worth fleshing these out or combining some ideas just to brainstorm a bit.TASK DETAILhttps://phabricator.wikimedia.org/T107595EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: brionCc: Alsee, Pppery, Florian, Liuxinyu970226, WMDE-leszek, Mholloway, Scott_WUaS, Niharika, MGChecker, LikeLifer, Elitre, Glaisher, JJMC89, RobLa-WMF, Yurik, ArielGlenn, APerson, TomT0m, Krenair, intracer, Tgr, Tobi_WMDE_SW, Addshore, Lydia_Pintscher, cscott, PleaseStand, awight, Ricordisamoa, GWicke, MarkTraceur, waldyrious, Legoktm, Aklapper, Jdforrester-WMF, Ltrlg, brion, Spage, MZMcBride, daniel, D3r1ck01, Izno, Luke081515, Wikidata-bugs, aude, jayvdb, fbstj, Mbch331, Jay8g, bd808___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T124569: RFC: Data namespace blob storage on wikidata.org
brion added a comment.Quick update prior to today's rfc irc discussion: we basically have two areas to discuss: 1) technical questions (data types, use as 'aggregation layer' for data from wikidata), and 2) the hosting question (which wiki to put it on, or whether to give it its own).TASK DETAILhttps://phabricator.wikimedia.org/T124569EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: brionCc: brion, Daniel_Mietchen, Danny_B, Thryduulf, Ricordisamoa, Bene, Micru, Yair_rand, Addshore, aude, hoo, JanZerebecki, MZMcBride, Lydia_Pintscher, tstarling, daniel, MaxSem, Tfinc, Milimetric, Aklapper, Yurik, StudiesWorld, D3r1ck01, Izno, Luke081515, Wikidata-bugs, GWicke, jayvdb, fbstj, RobLa-WMF, Mbch331, Jay8g, Ltrlg, bd808, Legoktm ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T107595: [RFC] Multi-Content Revisions
brion added a comment. In https://phabricator.wikimedia.org/T107595#2266131, @GWicke wrote: > The use case for providing metadata is so that we can use stores like RESTBase, which already provide an API keyed on title, revision & render ID. It also already deals with the complexities you mention. > > Basically, if we don't have a way to provide this key information to the backend store, then we can't access all the multi-content revision data that's already out there through this interface. As I understand it, restbase is a front-end caching proxy store, exposed to the public internet. Meanwhile the blob store is the equivalent of MediaWiki's existing text table and external storage backing system, entirely internal and containing data that is sometimes private (eg deleted or revdel'd page content). A front-end restbase could proxy access to MediaWiki revisions, backed by MediaWiki and the blob store. This would mean that slot data, metadata, titles, revision ids etc are all preserved and exposed because you'd be hooking up to the level of MediaWiki that has that information. Is that what you mean, or do you have something else in mind? TASK DETAIL https://phabricator.wikimedia.org/T107595 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: daniel, brion Cc: Glaisher, JJMC89, RobLa-WMF, Yurik, ArielGlenn, APerson, TomT0m, Krenair, intracer, Tgr, Tobi_WMDE_SW, Addshore, Lydia_Pintscher, cscott, PleaseStand, awight, Ricordisamoa, GWicke, MarkTraceur, waldyrious, Legoktm, Aklapper, Jdforrester-WMF, Ltrlg, brion, Spage, MZMcBride, daniel, D3r1ck01, Izno, Luke081515, Wikidata-bugs, aude, jayvdb, fbstj, Mbch331, Jay8g, bd808 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T107595: [RFC] Multi-Content Revisions
brion added a comment. If I understand, the case for passing more metadata to the blob store is as a hint for cross-blob data compression. For this I think we mainly want to pass the identifier of a related blob: the blob with the data from the same slot in the previous revision. If the related blob is in the same store, then the blob store can potentially optimize its actual backing storage (with diff-based storage, or by gzipping adjacent blob contents together with a window size larger than the blobs, etc). It might also be useful to specify a type for 'hey this is precompressed binary data, don't bother trying to recompress it or diff it'. But I would strongly recommend against being too clever. Revision metadata may change (yes, change -- revdel etc) and blobs are explicitly reused across multiple revisions. Revision histories can be rewritten (yes, rewritten -- import/export and delete/undelete can change ordering & adjacency, etc). And definitely don't include things like titles that are completely arbitrary and may change at any time. TASK DETAIL https://phabricator.wikimedia.org/T107595 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: daniel, brion Cc: Glaisher, JJMC89, RobLa-WMF, Yurik, ArielGlenn, APerson, TomT0m, Krenair, intracer, Tgr, Tobi_WMDE_SW, Addshore, Lydia_Pintscher, cscott, PleaseStand, awight, Ricordisamoa, GWicke, MarkTraceur, waldyrious, Legoktm, Aklapper, Jdforrester-WMF, Ltrlg, brion, Spage, MZMcBride, daniel, D3r1ck01, Izno, Luke081515, Wikidata-bugs, aude, jayvdb, fbstj, Mbch331, Jay8g, bd808 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T107595: [RFC] Multi-Content Revisions
brion added a comment.
> This assumes the BlobStore will actually talk to the (same) database. I
would like to have Transaction separate from the DB stuff, so it can be used
just as well with files, or Cassandra, or Swift, or whatever we come up with to
store blobs. We shouldn't assume that it knows about SQL at all.
Quite so...
Let's try instead:
$dbw->transact(function($trx) use ($this, $dbw) {
// $trx is a Transaction obj managed by the Database object, which will
// have its commit or rollback callbacks triggered when Database\transact
reaches its end state
// ...
$url = $blobStore->saveBlob( $data, $trx );
// ...
});
// if we reach this far, the transaction successfully committed.
// otherwise it'll have thrown an exception
TASK DETAIL
https://phabricator.wikimedia.org/T107595
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: daniel, brion
Cc: JJMC89, RobLa-WMF, Yurik, ArielGlenn, APerson, TomT0m, Krenair, intracer,
Tgr, Tobi_WMDE_SW, Addshore, Lydia_Pintscher, cscott, PleaseStand, awight,
Ricordisamoa, GWicke, MarkTraceur, waldyrious, Legoktm, Aklapper,
Jdforrester-WMF, Ltrlg, brion, Spage, MZMcBride, daniel, D3r1ck01, Izno,
Luke081515, Wikidata-bugs, aude, jayvdb, fbstj, Mbch331, Jay8g, bd808
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T107595: [RFC] Multi-Content Revisions
brion added a comment.
In https://phabricator.wikimedia.org/T107595#2264334, @daniel wrote:
> We could (optionally?) provide a transaction context to the blob store like
this:
I kinda like that, yeah. Maybe extend Database with a transactional interface
that takes a callback:
$dbw->transaction(function() use ($this, $dbw) {
// blah blah blah
});
// if we reach this far, the transaction successfully committed.
// otherwise it'll have thrown an exception
and internally in the BlobStore's save method, we add the rollback callback
straight onto the db object:
$dbw->addRollbackCallback( function() use ( $url ) { $this->delete( $url );
} )
That avoids having transaction state live separately in both the connection
and a Transaction object. Good model? Bad model?
TASK DETAIL
https://phabricator.wikimedia.org/T107595
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: daniel, brion
Cc: JJMC89, RobLa-WMF, Yurik, ArielGlenn, APerson, TomT0m, Krenair, intracer,
Tgr, Tobi_WMDE_SW, Addshore, Lydia_Pintscher, cscott, PleaseStand, awight,
Ricordisamoa, GWicke, MarkTraceur, waldyrious, Legoktm, Aklapper,
Jdforrester-WMF, Ltrlg, brion, Spage, MZMcBride, daniel, D3r1ck01, Izno,
Luke081515, Wikidata-bugs, aude, jayvdb, fbstj, Mbch331, Jay8g, bd808
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T107595: [RFC] Multi-Content Revisions
brion added a comment. (if RevisionBuilder takes a $dbw param via constructor/factory, then the question of the connection is easier) TASK DETAIL https://phabricator.wikimedia.org/T107595 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: daniel, brion Cc: JJMC89, RobLa-WMF, Yurik, ArielGlenn, APerson, TomT0m, Krenair, intracer, Tgr, Tobi_WMDE_SW, Addshore, Lydia_Pintscher, cscott, PleaseStand, awight, Ricordisamoa, GWicke, MarkTraceur, waldyrious, Legoktm, Aklapper, Jdforrester-WMF, Ltrlg, brion, Spage, MZMcBride, daniel, D3r1ck01, Izno, Luke081515, Wikidata-bugs, aude, jayvdb, fbstj, Mbch331, Jay8g, bd808 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T107595: [RFC] Multi-Content Revisions
brion added a comment. > The above code would replace much of what is in the Revision class now, in particular insertOn(). We can keep Revision around, but I'm not sure we can provide b/c for insertOn(). b/c here looks relatively straightforward to me; it creates a new revision with an updated default slot from the text content & metadata in the Revision object. This should be implementable by calling through to RevisionBuilder. The remaining questions are - whether we want to pass the $dbw parameter through (do we always go through load balancer in which case it'll be the same connection anyway? or are there cases where a separate connection may be used to insert revs for some reason?) and - whether there's any nested-transaction problems if someone tries to insert multiple revs in an explicitly larger transaction (Revision\insertOn doesn't try to manage transactions itself, and will leak external storage blobs if its text & revision updates get rolled back.) TASK DETAIL https://phabricator.wikimedia.org/T107595 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: daniel, brion Cc: JJMC89, RobLa-WMF, Yurik, ArielGlenn, APerson, TomT0m, Krenair, intracer, Tgr, Tobi_WMDE_SW, Addshore, Lydia_Pintscher, cscott, PleaseStand, awight, Ricordisamoa, GWicke, MarkTraceur, waldyrious, Legoktm, Aklapper, Jdforrester-WMF, Ltrlg, brion, Spage, MZMcBride, daniel, D3r1ck01, Izno, Luke081515, Wikidata-bugs, aude, jayvdb, fbstj, Mbch331, Jay8g, bd808 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T107595: [RFC] Multi-Content Revisions
brion added a comment. re this: $bs->deleteBlob( $dataUrl ); // dk: this goes wrong if the URL is content/hash based! I think the return from this: $dataUrl = $bs->saveBlob( $content->serialize() ); needs to signal whether a blob was created or whether an existing blob was reused. This means the blob store itself needs a transactional concept at least within the boundaries of 'does this blob exist?' followed by 'store this blob'. If two processes conflict (adding the same content at around the same time), then the second one needs to be able to detect the conflict and return the 'reference to existing' signal instead of the 'inserted new' signal. TASK DETAIL https://phabricator.wikimedia.org/T107595 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: daniel, brion Cc: JJMC89, RobLa-WMF, Yurik, ArielGlenn, APerson, TomT0m, Krenair, intracer, Tgr, Tobi_WMDE_SW, Addshore, Lydia_Pintscher, cscott, PleaseStand, awight, Ricordisamoa, GWicke, MarkTraceur, waldyrious, Legoktm, Aklapper, Jdforrester-WMF, Ltrlg, brion, Spage, MZMcBride, daniel, D3r1ck01, Izno, Luke081515, Wikidata-bugs, aude, jayvdb, fbstj, Mbch331, Jay8g, bd808 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T107595: [RFC] Multi-Content Revisions
brion added a comment.
Regarding transactional nature:
Assuming the backing blob storage continues to work on the model of the
current `text` table blobs with external storage backing, the "easy way" is to
allow extra backend blobs to leak in case of transaction rollback, and let them
be garbage-collected.
If you want to get *fancy* you can do explicit cleanup after a rollback that
happens in PHP-land (say after catching an exception, aborting the transaction,
and then re-throwing the exception). But this will fail in the case of a fatal
error that can't be caught, or the process being killed, leading to leaks again.
In MediaWiki in general we're pretty lax about allowing unused data to
accumulate in places like that, as long as its presence isn't harmful. Not the
best practice but it has plenty of precedent. :)
So an update pseudocode might look like:
$plu = $something->getPageLookupService();
$ps = $plu->getPageStore($title);
$initialRevId = $ps->getCurrentRevisionId();
$rs = $ps->getRevisionStore();
$rb = $rs->getRevisionBuilder( $initialRevId );
$rb->setUser($context->getUser());
$rb->setComment("awesome sauce");
$rb->updateSlot('main', $updatedTextContent);
$rb->updateSlot('script', $updatedScriptData);
$rs->apply( $rb );
where inside the RevisionStore\commit method there's something like:
$dbw->start();
$bs = $this->blobStore();
$addedBlobs = [];
if ($previousRevision) {
$slots = $previousRevision->getSlots();
} else {
$slots = [];
}
try {
foreach( $this->slotUpdates as $su ) {
$blob = $bs->saveDataBlob( $su->getData() );
$addedBlobs[] = $blob;
$slots[$su->getName()] = $this->saveRevisionSlot( $blob );
}
$this->saveRevisionRecord( $blah1, $blah2, $slots );
$dbw->commit();
} catch (Exception $e) {
// If update failed, clean up any newly added backing blobs, which
// may be in external databases, filesystems, or services.
try {
foreach( $addedBlobs as $blob ) {
$blob->delete();
}
} catch (Exception $e2) {
// if we can't get in to delete them, let them leak. they're safe.
}
try {
$dbw->rollback();
} catch (Exception $e3) {
// that probably means the db connection died.
}
throw $e;
}
TASK DETAIL
https://phabricator.wikimedia.org/T107595
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: daniel, brion
Cc: JJMC89, RobLa-WMF, Yurik, ArielGlenn, APerson, TomT0m, Krenair, intracer,
Tgr, Tobi_WMDE_SW, Addshore, Lydia_Pintscher, cscott, PleaseStand, awight,
Ricordisamoa, GWicke, MarkTraceur, waldyrious, Legoktm, Aklapper,
Jdforrester-WMF, Ltrlg, brion, Spage, MZMcBride, daniel, D3r1ck01, Izno,
Luke081515, Wikidata-bugs, aude, jayvdb, fbstj, Mbch331, Jay8g, bd808
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T107595: [RFC] Multi-Content Revisions
brion added a comment. Ok in that case... I will trust nothing ;) TASK DETAIL https://phabricator.wikimedia.org/T107595 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: daniel, brion Cc: RobLa-WMF, Yurik, ArielGlenn, APerson, TomT0m, Krenair, intracer, Tgr, Tobi_WMDE_SW, Addshore, Lydia_Pintscher, cscott, PleaseStand, awight, Ricordisamoa, GWicke, MarkTraceur, waldyrious, Legoktm, Aklapper, Jdforrester-WMF, Ltrlg, brion, Spage, MZMcBride, daniel, D3r1ck01, Izno, Wikidata-bugs, aude, jayvdb, fbstj, Mbch331, Jay8g, bd808 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T107595: [RFC] Multi-Content Revisions
brion added a comment. Aaa and now I see the bits in gerrit. I'll review all this tomorrow when I'm a little bit rested. Hehehe TASK DETAIL https://phabricator.wikimedia.org/T107595 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: daniel, brion Cc: RobLa-WMF, Yurik, ArielGlenn, APerson, TomT0m, Krenair, intracer, Tgr, Tobi_WMDE_SW, Addshore, Lydia_Pintscher, cscott, PleaseStand, awight, Ricordisamoa, GWicke, MarkTraceur, waldyrious, Legoktm, Aklapper, Jdforrester-WMF, Ltrlg, brion, Spage, MZMcBride, daniel, D3r1ck01, Izno, Wikidata-bugs, aude, jayvdb, fbstj, Mbch331, Jay8g, bd808 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T107595: [RFC] Multi-Content Revisions
brion added a comment. Ah great, that was mostly written before your post. ;) sounding good so far... Do you have code fleshed out enough to share or should we take that class structure and write fresh? TASK DETAIL https://phabricator.wikimedia.org/T107595 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: daniel, brion Cc: RobLa-WMF, Yurik, ArielGlenn, APerson, TomT0m, Krenair, intracer, Tgr, Tobi_WMDE_SW, Addshore, Lydia_Pintscher, cscott, PleaseStand, awight, Ricordisamoa, GWicke, MarkTraceur, waldyrious, Legoktm, Aklapper, Jdforrester-WMF, Ltrlg, brion, Spage, MZMcBride, daniel, D3r1ck01, Izno, Wikidata-bugs, aude, jayvdb, fbstj, Mbch331, Jay8g, bd808 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Updated] T107595: [RFC] Multi-Content Revisions
brion added a comment. @daniel I'd like to help write up updated RfC text on MediaWiki.org for this as its a thing a number of potential other work areas will depend on. Editing team is interested in putting more metadata in (https://phabricator.wikimedia.org/T132072) which means James is poking me about it. ;)) Let's start plotting some code arch for what this'll look like... TASK DETAIL https://phabricator.wikimedia.org/T107595 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: daniel, brion Cc: RobLa-WMF, Yurik, ArielGlenn, APerson, TomT0m, Krenair, intracer, Tgr, Tobi_WMDE_SW, Addshore, Lydia_Pintscher, cscott, PleaseStand, awight, Ricordisamoa, GWicke, MarkTraceur, waldyrious, Legoktm, Aklapper, Jdforrester-WMF, Ltrlg, brion, Spage, MZMcBride, daniel, D3r1ck01, Izno, Wikidata-bugs, aude, jayvdb, fbstj, Mbch331, Jay8g, bd808 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T120452: Allow tabular datasets on Commons (or some similar central repository) (CSV, TSV, JSON, XML)
brion added a comment. I'm a fan of "inheritMetadata" :) TASK DETAIL https://phabricator.wikimedia.org/T120452 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Yurik, brion Cc: TheDJ, Eloy, Jdforrester-WMF, brion, ThurnerRupert, intracer, TerraCodes, Pokefan95, gerritbot, -jem-, Bawolff, MZMcBride, Milimetric, Thryduulf, JEumerus, MarkTraceur, Yurik, Matanya, ekkis, matmarex, Lydia_Pintscher, Aklapper, Steinsplitter, StudiesWorld, DannyH, Riley_Huntley, D3r1ck01, Izno, JAllemandou, Wikidata-bugs, aude, El_Grafo, Ricordisamoa, Shizhao, Fabrice_Florin, Mbch331, Jay8g, Krenair, jeremyb ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T120452: Allow tabular datasets on Commons (or some similar central repository) (CSV, TSV, JSON, XML)
brion added a comment. Side note -- the referenced source data: page should get recorded as a template link in the link tables maybe? Or a file link at least. Some kind of reference. :) TASK DETAIL https://phabricator.wikimedia.org/T120452 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Yurik, brion Cc: TheDJ, Eloy, Jdforrester-WMF, brion, ThurnerRupert, intracer, TerraCodes, Pokefan95, gerritbot, -jem-, Bawolff, MZMcBride, Milimetric, Thryduulf, JEumerus, MarkTraceur, Yurik, Matanya, ekkis, matmarex, Lydia_Pintscher, Aklapper, Steinsplitter, StudiesWorld, DannyH, Riley_Huntley, D3r1ck01, Izno, JAllemandou, Wikidata-bugs, aude, El_Grafo, Ricordisamoa, Shizhao, Fabrice_Florin, Mbch331, Jay8g, Krenair, jeremyb ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T120452: Allow tabular datasets on Commons (or some similar central repository) (CSV, TSV, JSON, XML)
brion added a comment. @yurik I like that -- maybe generalize it as a metadata inheritance model; anything not filled out in the local json is taken from the referenced .tabular item. TASK DETAIL https://phabricator.wikimedia.org/T120452 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Yurik, brion Cc: TheDJ, Eloy, Jdforrester-WMF, brion, ThurnerRupert, intracer, TerraCodes, Pokefan95, gerritbot, -jem-, Bawolff, MZMcBride, Milimetric, Thryduulf, JEumerus, MarkTraceur, Yurik, Matanya, ekkis, matmarex, Lydia_Pintscher, Aklapper, Steinsplitter, StudiesWorld, DannyH, Riley_Huntley, D3r1ck01, Izno, JAllemandou, Wikidata-bugs, aude, El_Grafo, Ricordisamoa, Shizhao, Fabrice_Florin, Mbch331, Jay8g, Krenair, jeremyb ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T120452: Allow tabular datasets on Commons (or some similar central repository) (CSV, TSV, JSON, XML)
brion added a comment. @yurik: The W3C CSV on the Web working group's metadata model recommendation refers to "columns" with attributes for "name" and "titles" (plural, allowing alternates or per-language variants), with similar recommended character restrictions on "name" for ease of programmatic use: https://www.w3.org/TR/2015/REC-tabular-metadata-20151217/#dfn-column-description So following the naming model would give us either separate "column_name" and "column_title" keys containing lists of strings/localized string map objs... or following the W3C data model a little more closely would give us a "column" key containing a list of objects with their own "name" and "title"(s?) props. @Jdforrester-WMF data cube sounds awesome. :D someday later! TASK DETAIL https://phabricator.wikimedia.org/T120452 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Yurik, brion Cc: Eloy, Jdforrester-WMF, brion, ThurnerRupert, intracer, TerraCodes, Pokefan95, gerritbot, -jem-, Bawolff, MZMcBride, Alkamid, Milimetric, Thryduulf, JEumerus, MarkTraceur, Yurik, Matanya, ekkis, matmarex, Lydia_Pintscher, Aklapper, Steinsplitter, StudiesWorld, DannyH, Riley_Huntley, D3r1ck01, Izno, JAllemandou, Wikidata-bugs, aude, El_Grafo, Ricordisamoa, Shizhao, Fabrice_Florin, Mbch331, Jay8g, Krenair, jeremyb ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T120452: Allow tabular datasets on Commons (or some similar central repository) (CSV, TSV, JSON, XML)
brion added a comment. Re headers -- yeah need to distinguish between header labels (i18nable text) and column ids (identifiers for programs). As long as capability is there I don't mind the terms used, sounds like you're already working on that :) TASK DETAIL https://phabricator.wikimedia.org/T120452 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Yurik, brion Cc: Eloy, Jdforrester-WMF, brion, ThurnerRupert, intracer, TerraCodes, Pokefan95, gerritbot, -jem-, Bawolff, MZMcBride, Alkamid, Milimetric, Thryduulf, JEumerus, MarkTraceur, Yurik, Matanya, ekkis, matmarex, Lydia_Pintscher, Aklapper, Steinsplitter, StudiesWorld, DannyH, Riley_Huntley, D3r1ck01, Izno, JAllemandou, Wikidata-bugs, aude, El_Grafo, Ricordisamoa, Shizhao, Fabrice_Florin, Mbch331, Jay8g, Krenair, jeremyb ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T120452: Allow tabular datasets on Commons (or some similar central repository) (CSV, TSV, JSON, XML)
brion added a comment. Side note: headers are rejected if they contain spaces. That seems odd? TASK DETAIL https://phabricator.wikimedia.org/T120452 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Yurik, brion Cc: Jdforrester-PERSONAL, Jdforrester-WMF, brion, ThurnerRupert, intracer, TerraCodes, Pokefan95, gerritbot, -jem-, Bawolff, MZMcBride, Alkamid, Milimetric, Thryduulf, JEumerus, MarkTraceur, Yurik, Matanya, ekkis, matmarex, Lydia_Pintscher, Aklapper, Steinsplitter, StudiesWorld, DannyH, Riley_Huntley, D3r1ck01, Izno, JAllemandou, Wikidata-bugs, aude, El_Grafo, Ricordisamoa, Shizhao, Fabrice_Florin, Mbch331, Jay8g, Krenair, jeremyb ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T120452: Allow tabular datasets on Commons (or some similar central repository) (CSV, TSV, JSON, XML)
brion added a comment. In https://phabricator.wikimedia.org/T120452#2227240, @matmarex wrote: > As I understand these are stored as regular MediaWiki pages now, so they have a maximum length of 2 MB. Even naive queries pulling the whole thing into memory would be fast enough at these scales. If we want to think about performance for large data, we should first think about overcoming the length limitation :) Ah I forgot all about that. ;) That'll at least stop people from creating _super_ huge datasets for now... unless they break them into multiple files and create lua modules to stitch them back together. :) That may be acceptable however, and lets people prototype the kinds of crazy things they want until we hate it so much we decide we have to support them better. I tried generating some random tables with this PHP script: https://gist.github.com/brion/46469ac2df31a8eb0e179f50b1967d20 I find I can only successfully save a file of under a megabyte even though the max is 2 megs; a 2 meg file gets rejected complaining that it's over 4 megabytes... I notice the output that comes back to me in the edit window is pretty-printed, which means a lot of indentation and newlines bulking up the storage requirements. That might be on purpose to provide a better diff view? Seems to run reasonably fast to render locally, though it also produces a giant HTML table for the page view that's about 3.5 megs, which should at least compress ok. Editing also means resubmitting the entire data set at once with the edit form. A partial-edit submission system similar to section editing on wiki pages might be nice to reduce submission latency and bandwidth consumption editing a table, but that can probably wait until it needs to be paired with a spreadsheet-like UI. TASK DETAIL https://phabricator.wikimedia.org/T120452 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Yurik, brion Cc: Jdforrester-PERSONAL, Jdforrester-WMF, brion, ThurnerRupert, intracer, TerraCodes, Pokefan95, gerritbot, -jem-, Bawolff, MZMcBride, Alkamid, Milimetric, Thryduulf, JEumerus, MarkTraceur, Yurik, Matanya, ekkis, matmarex, Lydia_Pintscher, Aklapper, Steinsplitter, StudiesWorld, DannyH, Riley_Huntley, D3r1ck01, Izno, JAllemandou, Wikidata-bugs, aude, El_Grafo, Ricordisamoa, Shizhao, Fabrice_Florin, Mbch331, Jay8g, Krenair, jeremyb ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T120452: Allow tabular datasets on Commons (or some similar central repository) (CSV, TSV, JSON, XML)
brion added a comment. Pulling individual data items out of large lists; pulling relevant columns in order to sum them; pulling or updating a small number of cells during editing; sub setting a large data set to graph the subset; sub setting a large data set to perform operations on it Ina Lua module. For a concrete example: say I have a data set that lists the voting breakdown during US primary elections for every county. That's roughly 3000 rows, each with data points for each candidate. Not too awful perhaps, so let's make a bigger table. Population of every US census place for every 10-year census since 1790. That's probably a lot. Now add more columns for various breakdown information. Subset queries on a large data set would allow for reading in data relevant to a map area, or a particular place, or a particular breakdown column, or a particular year range, or some combination of all these things, having had to pull in only the column list and do some sort of index scan on the key rows. The alternative is to break down the giant table into lots of small tables, and have the client code in a Lua module or whatever 'stitch' together the multiple data sets when needing to cross boundaries. You might say that no one should store that much data in one table, but I'm pretty sure people will push the limits of a system like this. :) TASK DETAIL https://phabricator.wikimedia.org/T120452 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Yurik, brion Cc: Jdforrester-PERSONAL, Jdforrester-WMF, brion, ThurnerRupert, intracer, TerraCodes, Pokefan95, gerritbot, -jem-, Bawolff, MZMcBride, Alkamid, Milimetric, Thryduulf, JEumerus, MarkTraceur, Yurik, Matanya, ekkis, matmarex, Lydia_Pintscher, Aklapper, Steinsplitter, StudiesWorld, DannyH, Riley_Huntley, D3r1ck01, Izno, JAllemandou, Wikidata-bugs, aude, El_Grafo, Ricordisamoa, Shizhao, Fabrice_Florin, Mbch331, Jay8g, Krenair, jeremyb ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T120452: Allow tabular datasets on Commons (or some similar central repository) (CSV, TSV, JSON, XML)
brion added a comment. Couple quick notes: - pretty cool. :) - I worry about efficiency of storage and queries; for small tables json blobs are fine but for large data sets thisll get extremely verbose, and loading/saving small updates to a large table will get very slow. Consider providing query and update primitives that don't require the client (lua module, wiki API client, editor form, etc) to load and parse the entire blob. Eg "gimme the rows with key value in this range" or "update columns A B and C with these values in rows with key values X Y and Z". This could then be extended in future to a different storage backend, or a streaming json reader. TASK DETAIL https://phabricator.wikimedia.org/T120452 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Yurik, brion Cc: brion, ThurnerRupert, intracer, TerraCodes, Pokefan95, gerritbot, -jem-, Bawolff, MZMcBride, Alkamid, Milimetric, Thryduulf, JEumerus, MarkTraceur, Yurik, Matanya, ekkis, matmarex, Lydia_Pintscher, Aklapper, Steinsplitter, StudiesWorld, DannyH, Riley_Huntley, D3r1ck01, Izno, JAllemandou, Wikidata-bugs, aude, El_Grafo, Ricordisamoa, Shizhao, Fabrice_Florin, Mbch331, Jay8g, Krenair, jeremyb ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Changed Subscribers] T114443: EventBus MVP
brion added a subscriber: brion. TASK DETAIL https://phabricator.wikimedia.org/T114443 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Ottomata, brion Cc: brion, intracer, Smalyshev, mark, MZMcBride, Krinkle, EBernhardson, bd808, Joe, dr0ptp4kt, madhuvishy, Nuria, ori, faidon, aaron, GWicke, mobrovac, Eevans, Ottomata, Matanya, Aklapper, JAllemandou, jkroll, Hardikj, Wikidata-bugs, Jdouglas, aude, Deskana, Manybubbles, daniel, RobLa-WMF, Jay8g, Ltrlg, jeremyb, Legoktm ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Changed Subscribers] T114251: [RFC] Magic Infobox implementation
brion added a subscriber: brion. TASK DETAIL https://phabricator.wikimedia.org/T114251 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: brion Cc: brion, Krenair, Addshore, Smalyshev, Qgil, Lucie, MrStradivarius, Aklapper, Lydia_Pintscher, aude, GWicke, Jdforrester-WMF, cscott, daniel, hoo, Wikidata-bugs, Dinoguy1000, Jackmcbarn, Jay8g, Ltrlg, bd808, Legoktm ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T106523: Issues with srcset with browsers that do not properly support it
brion added a comment. But yeah that code'll have to run on both. Feel free to ping me for review etc TASK DETAIL https://phabricator.wikimedia.org/T106523 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: brion Cc: brion, gerritbot, Jdlrobson, Sumit, Aklapper, Wikidata-bugs, Lydia_Pintscher, Malyacko ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T106523: Issues with srcset with browsers that do not properly support it
brion added a comment. If http://caniuse.com/#feat=srcset is to be believed, the main problem browser on mobile is going to be Safari on iOS 8, since it supports native srcset but only for density switching, not for the size specifications. So if you use a customized polyfill, you can't rely on the basic test like img.srcset !== undefined... Probably ok to go ahead and use a customized polyfill for the banners; use data-srcset or something in place of the native srcset attribute so iOS 8 doesn't start downloading the giant image! I would probably recommend updating jquery.hidpi.js to support width/height specs rather than including a second polyfill. TASK DETAIL https://phabricator.wikimedia.org/T106523 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: brion Cc: brion, gerritbot, Jdlrobson, Sumit, Aklapper, Wikidata-bugs, Lydia_Pintscher, Malyacko ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T106523: Issues with srcset with browsers that do not properly support it
brion added a comment. Well, desktop isn't as big a worry since high-res screen usually means large screen (and you'll usually max out at 2, rather than the 3 that some phones like the iPhone 6plus reach)... :) So it should be safe there to load the 1x and then polyfill-load the native-density-except-on-small-screen. TASK DETAIL https://phabricator.wikimedia.org/T106523 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: brion Cc: brion, gerritbot, Jdlrobson, Sumit, Aklapper, Wikidata-bugs, Lydia_Pintscher, Malyacko ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T106523: Issues with srcset with browsers that do not properly support it
brion added a comment.
So if I'm reading the above comments correctly, these banners want to use the
width capping specifications ('w640') in img srcset, but Safari and our JS
polyfill only support switching on the device pixel ratio ('2x').
I suppose you might be able to rig up a fancier polyfill (there probably are
some out there)... but beware that a JS polyfill will run *after* the browser's
already started downloading the image it originally selected, which in this
case will be the giant one from the native srcset. So that may not be an ideal
solution.
Consider also using a CSS background image which would allow using a @media
selector on the screen size, and will work across more browsers...
TASK DETAIL
https://phabricator.wikimedia.org/T106523
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: brion
Cc: brion, gerritbot, Jdlrobson, Sumit, Aklapper, Wikidata-bugs,
Lydia_Pintscher, Malyacko, P.Copp
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T76827: Image thumbnail urls should be included where applicable in wikidata API response for commonsMedia
brion added a comment. (If the redirect gets cached that'd still have the roundtrip, though the first request would be cheaper once the thumbnail's created as we'd avoid hitting PHP app servers.) TASK DETAIL https://phabricator.wikimedia.org/T76827 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: brion Cc: brion, MaxSem, Addshore, JanZerebecki, aude, MZMcBride, Aklapper, Lydia_Pintscher, Jdlrobson, daniel, Wikidata-bugs ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T76827: Image thumbnail urls should be included where applicable in wikidata API response for commonsMedia
brion added a comment. Hmm, well if thumb.php redirected you'd have an extra round-trip plus the overhead of hitting the PHP app servers in the first place... might not be ideal either. :( TASK DETAIL https://phabricator.wikimedia.org/T76827 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: brion Cc: brion, MaxSem, Addshore, JanZerebecki, aude, MZMcBride, Aklapper, Lydia_Pintscher, Jdlrobson, daniel, Wikidata-bugs ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T76827: Image thumbnail urls should be included where applicable in wikidata API response for commonsMedia
brion added a subscriber: brion. brion added a comment. Note that the thumbnail size will need to be selectable through some input variable if you stuff it into one API req like this -- suitable size will depend on the device and how the client software chooses to show thumbs. (Note another workaround is to go ahead and do a prop=imageinfo request on all the given files as a second API hit. That's two round trips though which is sad, but at least you don't have to do one per image!) TASK DETAIL https://phabricator.wikimedia.org/T76827 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Lydia_Pintscher, brion Cc: brion, MaxSem, Addshore, JanZerebecki, aude, MZMcBride, Aklapper, Lydia_Pintscher, Jdlrobson, daniel, Wikidata-bugs, Jdouglas ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Changed Subscribers] T84923: Reliable publish / subscribe event bus
brion added a subscriber: brion. TASK DETAIL https://phabricator.wikimedia.org/T84923 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: brion Cc: brion, Krenair, Halfak, JanZerebecki, bd808, MZMcBride, mobrovac, GWicke, aaron, daniel, Hardikj, yuvipanda, jkroll, Smalyshev, Wikidata-bugs, Jdouglas, RobH, aude, marcoil, Manybubbles, mark, RobLa-WMF, Joe, QChris, chasemp ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
