Re: [Wikitech-l] [Engineering] Modernizing our content platform: Kick-off meeting on Tuesday
Gabriel and Wikimedians, Sorry I missed this meeting, but thanks for these followup Wikitech resources. https://phabricator.wikimedia.org/T99088 https://mail.google.com/mail/u/0/#drafts/14e0d31ef6f33ad7?projector=1 Regards, Scott On Jun 23, 2015 4:01 PM, Gabriel Wicke gwi...@wikimedia.org wrote: Vibha, sorry I missed your reply before the meeting. The discussion was mostly technical an centered around two main areas: - content storage and dependency tracking / change propagation - page content representation, content composition and editing We collected and discussed a list of use cases and their respective challenges on an etherpad [1]. In the end, we resolved to follow up with more focused work around the two main themes. I'll summarize the discussion in a task per area and post them here. Gabriel [1]: http://etherpad.wikimedia.org/p/Content_platform On Tue, Jun 23, 2015 at 1:16 PM, Adam Baso ab...@wikimedia.org wrote: Updated URLs, we're in R37 on air stream: http://youtu.be/RcE2kecrsIk Max of 15 users: https://plus.google.com/hangouts/_/hoaevent/AP36tYdub-Rs4mI_4UjTEzTgU7GKBkgjV5s0kXASoA9Tno4gJK34_Q On Tue, Jun 23, 2015 at 12:18 PM, Vibha Bamba vba...@wikimedia.org wrote: This sounds fairly dev centric with Front end/ UX implications. Will the discussion be fairly technical? Let us know if Design should attend. Vibha Bamba Senior Designer | WMF Design On Tue, Jun 23, 2015 at 11:12 AM, Gabriel Wicke gwi...@wikimedia.org wrote: Reminder: This is today! When: *Tuesday, June 23rd, 13:00 - 14:30 PT* [3] Where: * *https://plus.google.com/hangouts/_/wikimedia.org/contentplatform https://plus.google.com/hangouts/_/wikimedia.org/contentplatform* * *room 37* in the office On Fri, Jun 19, 2015 at 12:00 PM, Gabriel Wicke gwi...@wikimedia.org wrote: Hi all, a few of us have recently collected and roughly prioritized some open architectural questions [1]. The area that stood out as needing most urgent attention is adapting our content platform to long-term changes in the way users interact with our site [2]. People are using a wider range of devices, from feature phones to multi-core desktops. Many users are looking for short factoids and definitions, while others prefer to immerse themselves in detailed articles with rich multimedia content. MediaWiki is currently not very optimized to support such a diverse set of use cases. To address this, we see a need to improve our platform in the following areas: - Storage: To better separate data from presentation, we need the ability to store multiple bits of content and metadata associated with each revision. This storage needs to integrate well with edits, history views, and other features, and should be exposed via a high-performance API. - Change propagation: Edits to small bits of data need to be reliably and efficiently propagated to all content depending on it. The machinery needed to track dependencies should be easy to use. - Content composition and caching: Separate data gives us the freedom to render infoboxes, graphs or multimedia elements dynamically, depending on use case and client. For performance and flexibility, it would be desirable to assemble at least some of these renders as late as possible, at the edge or on the client. We don't expect to tackle all of this at once, but are starting to look into several areas. If you are interested in helping, then we would like to invite you to join us for a kick-off meeting: *When: Tuesday, June 23rd, 13:00 - 14:30 PT [3]* *Where: *A *hangout* link will be posted here before the meeting; room 37 in the office. If you can't attend, then please have a look at our current notes and let us know what you think [2]. Gabriel Wicke, Daniel Kinzler, Brion Vibber, Tim Starling, Roan Kattouw, Ori Livneh [1]: https://phabricator.wikimedia.org/T96903 [2]: https://phabricator.wikimedia.org/T99088 [3]: http://www.timeanddate.com/worldclock/fixedtime.html?msg=MediaWiki+content+platform+kick-offiso=20150623T13p1=224ah=1am=30 -- Gabriel Wicke Principal Engineer, Wikimedia Foundation ___ Engineering mailing list engineer...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/engineering ___ Engineering mailing list engineer...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/engineering -- Gabriel Wicke Principal Engineer, Wikimedia Foundation ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org
Re: [Wikitech-l] [Engineering] Modernizing our content platform: Kick-off meeting on Tuesday
On Tue, Jun 23, 2015 at 10:01 PM, Dan Garry dga...@wikimedia.org wrote:
My takeaway from this was that there are strong arguments both for and
against keeping the representation of an article as a single blob of
wikitext/HTML.
In the Architecture focus discussion, Krinkle, tgr, and others expressed a
more optimistic idea, that the wikitext of an article is a sea of prose
with isles of non-prose content, which could come from structured data [1].
I wasn't properly aware that the graph [2] and templatedata [3] parser
tags are examples of this already. Their content is highly structured and
VisualEditor or dedicated code can provide a specialized editor for it. If
you edit source of a wiki page containing them and garble their content,
you get a syntax warning or fail.
Wiki pages need more of these, drumroll Structure Content Blobs™ (or
sPage Components? anything but widget). Are they necessarily parser tags,
or is a parser function like {{#graph: *some parameters*}} equivalent?
To be concrete, does this mean the way forward for specifying lead images
[5] is a parser tag
leadimage{
imagepage: File:Einstein_1921_by_F_Schmutzer_-_restoration.jpg,
focalarea: {rect: [0.20, 0.20, 0.12, 0.12]}
}
/leadimage
in wikitext, with a MediaWiki API to add this to a document and a WYSIWYG
property editor in VisualEditor for humans?
AIUI, the *Architecture focus 2015* document discusses this under
Generalized transclusion [4] and comments:
Over the next months, the MediaWiki developer community and staff should
investigate how the different transclusion mechanism used with wikitext
content can be unified and extended to work with non-wikitext content.
Exciting stuff.
[1]
https://tools.wmflabs.org/meetbot/wikimedia-office/2015/wikimedia-office.2015-06-24-21.04.log.html
starting at 21:21:05
[2] e.g. https://www.mediawiki.org/wiki/Extension:Graph/Demo/Map?action=edit
[3] e.g. https://www.mediawiki.org/wiki/Template:Phabricator?action=edit
[4]
https://www.mediawiki.org/wiki/Architecture_focus_2015#General_architectural_concerns
[5] https://phabricator.wikimedia.org/T91683
--
=S Page WMF Tech writer
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] [Engineering] Modernizing our content platform: Kick-off meeting on Tuesday
Updated URLs, we're in R37 on air stream: http://youtu.be/RcE2kecrsIk Max of 15 users: https://plus.google.com/hangouts/_/hoaevent/AP36tYdub-Rs4mI_4UjTEzTgU7GKBkgjV5s0kXASoA9Tno4gJK34_Q On Tue, Jun 23, 2015 at 12:18 PM, Vibha Bamba vba...@wikimedia.org wrote: This sounds fairly dev centric with Front end/ UX implications. Will the discussion be fairly technical? Let us know if Design should attend. Vibha Bamba Senior Designer | WMF Design On Tue, Jun 23, 2015 at 11:12 AM, Gabriel Wicke gwi...@wikimedia.org wrote: Reminder: This is today! When: *Tuesday, June 23rd, 13:00 - 14:30 PT* [3] Where: * *https://plus.google.com/hangouts/_/wikimedia.org/contentplatform https://plus.google.com/hangouts/_/wikimedia.org/contentplatform* * *room 37* in the office On Fri, Jun 19, 2015 at 12:00 PM, Gabriel Wicke gwi...@wikimedia.org wrote: Hi all, a few of us have recently collected and roughly prioritized some open architectural questions [1]. The area that stood out as needing most urgent attention is adapting our content platform to long-term changes in the way users interact with our site [2]. People are using a wider range of devices, from feature phones to multi-core desktops. Many users are looking for short factoids and definitions, while others prefer to immerse themselves in detailed articles with rich multimedia content. MediaWiki is currently not very optimized to support such a diverse set of use cases. To address this, we see a need to improve our platform in the following areas: - Storage: To better separate data from presentation, we need the ability to store multiple bits of content and metadata associated with each revision. This storage needs to integrate well with edits, history views, and other features, and should be exposed via a high-performance API. - Change propagation: Edits to small bits of data need to be reliably and efficiently propagated to all content depending on it. The machinery needed to track dependencies should be easy to use. - Content composition and caching: Separate data gives us the freedom to render infoboxes, graphs or multimedia elements dynamically, depending on use case and client. For performance and flexibility, it would be desirable to assemble at least some of these renders as late as possible, at the edge or on the client. We don't expect to tackle all of this at once, but are starting to look into several areas. If you are interested in helping, then we would like to invite you to join us for a kick-off meeting: *When: Tuesday, June 23rd, 13:00 - 14:30 PT [3]* *Where: *A *hangout* link will be posted here before the meeting; room 37 in the office. If you can't attend, then please have a look at our current notes and let us know what you think [2]. Gabriel Wicke, Daniel Kinzler, Brion Vibber, Tim Starling, Roan Kattouw, Ori Livneh [1]: https://phabricator.wikimedia.org/T96903 [2]: https://phabricator.wikimedia.org/T99088 [3]: http://www.timeanddate.com/worldclock/fixedtime.html?msg=MediaWiki+content+platform+kick-offiso=20150623T13p1=224ah=1am=30 -- Gabriel Wicke Principal Engineer, Wikimedia Foundation ___ Engineering mailing list engineer...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/engineering ___ Engineering mailing list engineer...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/engineering ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] [Engineering] Modernizing our content platform: Kick-off meeting on Tuesday
Thanks for organising this discussion! My takeaway from this was that there are strong arguments both for and against keeping the representation of an article as a single blob of wikitext/HTML. The against big blob of wikitext argument is that it allows greater ease of editing article data in bite-size chunks because the data can be easily manipulated in a granular fashion over an API, and more easily allows API consumers (e.g. the mobile apps) to structure articles in whatever fashion they see fit. The for big blob of wikitext argument is that keeping it all as wikitext means we get content curation essentially for free using existing mechanisms (recent changes, watchlist, what links here, IRC feeds, etc.) and don't have to reinvent the wheel, for example Flow having to manually maintain mechanisms for hooking into these curation methods. Both approaches have merits and drawbacks. Of course, both the for and against arguments are possible to mitigate; one could structure the wikitext well enough to allow granularity and simplicity of editing in a seamless fashion, or one could hook into existing workflows for content curation like Flow has done. But, both of these would require extra work, so it seems there is no clear one true approach emerging right now. I'd be interested in participating in further sessions that aim to resolve this contention about article representation. In my opinion, it all boils down to which user workflow we, as a movement, prioritise the highest; if we can decide which workflow to prioritise, we can easily pick whichever solution lets us most easily satisfy that workflow. Thanks, Dan On 23 June 2015 at 16:01, Gabriel Wicke gwi...@wikimedia.org wrote: Vibha, sorry I missed your reply before the meeting. The discussion was mostly technical an centered around two main areas: - content storage and dependency tracking / change propagation - page content representation, content composition and editing We collected and discussed a list of use cases and their respective challenges on an etherpad [1]. In the end, we resolved to follow up with more focused work around the two main themes. I'll summarize the discussion in a task per area and post them here. Gabriel [1]: http://etherpad.wikimedia.org/p/Content_platform On Tue, Jun 23, 2015 at 1:16 PM, Adam Baso ab...@wikimedia.org wrote: Updated URLs, we're in R37 on air stream: http://youtu.be/RcE2kecrsIk Max of 15 users: https://plus.google.com/hangouts/_/hoaevent/AP36tYdub-Rs4mI_4UjTEzTgU7GKBkgjV5s0kXASoA9Tno4gJK34_Q On Tue, Jun 23, 2015 at 12:18 PM, Vibha Bamba vba...@wikimedia.org wrote: This sounds fairly dev centric with Front end/ UX implications. Will the discussion be fairly technical? Let us know if Design should attend. Vibha Bamba Senior Designer | WMF Design On Tue, Jun 23, 2015 at 11:12 AM, Gabriel Wicke gwi...@wikimedia.org wrote: Reminder: This is today! When: *Tuesday, June 23rd, 13:00 - 14:30 PT* [3] Where: * *https://plus.google.com/hangouts/_/wikimedia.org/contentplatform https://plus.google.com/hangouts/_/wikimedia.org/contentplatform* * *room 37* in the office On Fri, Jun 19, 2015 at 12:00 PM, Gabriel Wicke gwi...@wikimedia.org wrote: Hi all, a few of us have recently collected and roughly prioritized some open architectural questions [1]. The area that stood out as needing most urgent attention is adapting our content platform to long-term changes in the way users interact with our site [2]. People are using a wider range of devices, from feature phones to multi-core desktops. Many users are looking for short factoids and definitions, while others prefer to immerse themselves in detailed articles with rich multimedia content. MediaWiki is currently not very optimized to support such a diverse set of use cases. To address this, we see a need to improve our platform in the following areas: - Storage: To better separate data from presentation, we need the ability to store multiple bits of content and metadata associated with each revision. This storage needs to integrate well with edits, history views, and other features, and should be exposed via a high-performance API. - Change propagation: Edits to small bits of data need to be reliably and efficiently propagated to all content depending on it. The machinery needed to track dependencies should be easy to use. - Content composition and caching: Separate data gives us the freedom to render infoboxes, graphs or multimedia elements dynamically, depending on use case and client. For performance and flexibility, it would be desirable to assemble at least some of these renders as late as possible, at the edge or on the client. We don't expect to tackle all of this at once, but are starting to look into several areas. If you are interested in helping, then we would like to invite you to join us for a kick-off meeting: *When:
Re: [Wikitech-l] [Engineering] Modernizing our content platform: Kick-off meeting on Tuesday
Vibha, sorry I missed your reply before the meeting. The discussion was mostly technical an centered around two main areas: - content storage and dependency tracking / change propagation - page content representation, content composition and editing We collected and discussed a list of use cases and their respective challenges on an etherpad [1]. In the end, we resolved to follow up with more focused work around the two main themes. I'll summarize the discussion in a task per area and post them here. Gabriel [1]: http://etherpad.wikimedia.org/p/Content_platform On Tue, Jun 23, 2015 at 1:16 PM, Adam Baso ab...@wikimedia.org wrote: Updated URLs, we're in R37 on air stream: http://youtu.be/RcE2kecrsIk Max of 15 users: https://plus.google.com/hangouts/_/hoaevent/AP36tYdub-Rs4mI_4UjTEzTgU7GKBkgjV5s0kXASoA9Tno4gJK34_Q On Tue, Jun 23, 2015 at 12:18 PM, Vibha Bamba vba...@wikimedia.org wrote: This sounds fairly dev centric with Front end/ UX implications. Will the discussion be fairly technical? Let us know if Design should attend. Vibha Bamba Senior Designer | WMF Design On Tue, Jun 23, 2015 at 11:12 AM, Gabriel Wicke gwi...@wikimedia.org wrote: Reminder: This is today! When: *Tuesday, June 23rd, 13:00 - 14:30 PT* [3] Where: * *https://plus.google.com/hangouts/_/wikimedia.org/contentplatform https://plus.google.com/hangouts/_/wikimedia.org/contentplatform* * *room 37* in the office On Fri, Jun 19, 2015 at 12:00 PM, Gabriel Wicke gwi...@wikimedia.org wrote: Hi all, a few of us have recently collected and roughly prioritized some open architectural questions [1]. The area that stood out as needing most urgent attention is adapting our content platform to long-term changes in the way users interact with our site [2]. People are using a wider range of devices, from feature phones to multi-core desktops. Many users are looking for short factoids and definitions, while others prefer to immerse themselves in detailed articles with rich multimedia content. MediaWiki is currently not very optimized to support such a diverse set of use cases. To address this, we see a need to improve our platform in the following areas: - Storage: To better separate data from presentation, we need the ability to store multiple bits of content and metadata associated with each revision. This storage needs to integrate well with edits, history views, and other features, and should be exposed via a high-performance API. - Change propagation: Edits to small bits of data need to be reliably and efficiently propagated to all content depending on it. The machinery needed to track dependencies should be easy to use. - Content composition and caching: Separate data gives us the freedom to render infoboxes, graphs or multimedia elements dynamically, depending on use case and client. For performance and flexibility, it would be desirable to assemble at least some of these renders as late as possible, at the edge or on the client. We don't expect to tackle all of this at once, but are starting to look into several areas. If you are interested in helping, then we would like to invite you to join us for a kick-off meeting: *When: Tuesday, June 23rd, 13:00 - 14:30 PT [3]* *Where: *A *hangout* link will be posted here before the meeting; room 37 in the office. If you can't attend, then please have a look at our current notes and let us know what you think [2]. Gabriel Wicke, Daniel Kinzler, Brion Vibber, Tim Starling, Roan Kattouw, Ori Livneh [1]: https://phabricator.wikimedia.org/T96903 [2]: https://phabricator.wikimedia.org/T99088 [3]: http://www.timeanddate.com/worldclock/fixedtime.html?msg=MediaWiki+content+platform+kick-offiso=20150623T13p1=224ah=1am=30 -- Gabriel Wicke Principal Engineer, Wikimedia Foundation ___ Engineering mailing list engineer...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/engineering ___ Engineering mailing list engineer...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/engineering -- Gabriel Wicke Principal Engineer, Wikimedia Foundation ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
