Re: [Wikitech-l] Tools for dealing with citations of withdrawn academic journal articles

2015-09-17 Thread Gergő Tisza 
On Tue, Aug 18, 2015 at 2:42 AM, Pine W  wrote:

> Is there any easy way to find all of citations of specified academic
> articles on Wikipedias in all languages, and the text that is supported by
> those references, so that the citations of questionable articles can be
> removed and the article texts can be quickly reviewed for possible changes
> or removal?
>

Not right now, but Aaron and James are working on it!
http://librarybase.wmflabs.org/
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Fwd: [Tools] Kubernetes picked to provide alternative to GridEngine

2015-09-17 Thread Brian Gerstle 
Congrats on moving forward with a big decision!  I'm very optimistic about
containers, so it's exciting to see movement in this area.

Is there a larger arc of using this for our own services (Mediawiki,
RESTBase, etc.), potentially in production?


On Wed, Sep 16, 2015 at 9:28 PM, Yuvi Panda  wrote:

> FYI
>
>
> -- Forwarded message --
> From: Yuvi Panda 
> Date: Wed, Sep 16, 2015 at 6:25 PM
> Subject: [Tools] Kubernetes picked to provide alternative to GridEngine
> To: labs-annou...@lists.wikimedia.org
>
>
> We have picked kubernetes.io to provide an alternative to GridEngine
> on Tool Labs! See https://phabricator.wikimedia.org/T106475 for more
> details about the evaluation itself,
>
> https://docs.google.com/spreadsheets/d/1YkVsd8Y5wBn9fvwVQmp9Sf8K9DZCqmyJ-ew-PAOb4R4/edit?pli=1#gid=0
> for the spreadsheet with the evaluation matrix and
> https://lists.wikimedia.org/pipermail/labs-l/2015-August/003955.html
> for the previous email listing pros and cons of the others solutions
> that we have considered.
>
> == Rough Timeline ==
>
> * October: Start beta testing by opening up Kubernetes to whitelisted
> tools. Allows people to run arbitrary Docker images in Kubernetes,
> both for continuous jobs and web services. If you are interested in
> this,please add a comment to https://phabricator.wikimedia.org/T112824
> * December: Work on migrational tooling to assist in switching from
> GridEngine to Kubernetes should be in a good place. This will begin
> with a '--provider=kubernetes' parameter to the webservice command
> that will allow people to easily switch to Kubernetes for webservice.
> We will have something similar implemented for jsub / jstart.
> * January: Kubernetes cluster is opened up for all tools users. Fairly
> complete switching between GridEngine and Kubernetes for at least
> continuous jobs and webservices.
>
> == What does this give developers? ==
>
> # A software stack that is made up of widely-used and
> actively-developed software components. We are looking to dramatically
> reduce the surface of code that we write and maintain in-house.
> # (When out of beta) More stability and reliability.
> # It allows us to get rid of a lot of our customizations (the entire
> webservice setup, for example) which has proven to be a lot of work
> and flaky at times.
> # We can migrate tools that don't require NFS off of it, and since it
> has historically been one of our flakiest servies this allows tools to
> opt-out of it and get a lot more reliability.
> # Using Docker allows more user freedom in a structured way that is
> easier to support.
> # Has an active upstream / is used elsewhere as well (Google Container
> Engine, Microsoft Azure, RedHat OpenShift, AWS, etc.), so much more
> help / community available when users run into issues than we have now
> # Probably more :) It opens up a lot of possibilities and I'm sure
> developers will use it createively :)
>
> == Do I need to change anything? ==
>
> No, especially if you do not want to. We think Docker and Kubernetes
> are exciting technologies, so we would like volunteers who are
> interested in exploring these platforms to have the option of getting
> their hands dirty. At the same time, it is important for us to avoid
> complicating life for people for whom the current setup works well. If
> we are successful, the migration to Docker and Kubernetes will be
> seamless, and users of Tool Labs will not have to know what either
> Docker or Kubernetes are, let alone how to operate them.
>
> We will begin by offering arbitrary Docker image execution only.
> Eventually we will make it super-easy to switch between the current
> setup and Kubernetes, and allow people to be able to use this without
> having to know what Docker is or what Kubernetes is. That will slowly
> be built over the next few months.
>
> Absolutely nothing changes for developers or users right now.
>
> == Will GridEngine be going away? ==
>
> Not anytime soon!
>
> Thanks!
>
> --
> Yuvi Panda T
> http://yuvi.in/blog
>
>
> --
> Yuvi Panda T
> http://yuvi.in/blog
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l




-- 
EN Wikipedia user page: https://en.wikipedia.org/wiki/User:Brian.gerstle
IRC: bgerstle
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] thumb generation

2015-09-17 Thread Gabriel Wicke 
On Wed, Sep 16, 2015 at 12:51 AM, Federico Leva (Nemo) 
wrote:

> Have you looked into what mwoffliner does?
> https://sourceforge.net/p/kiwix/other/ci/master/tree/mwoffliner/mwoffliner.js


+1 for mwoffliner. It should be *very* close to what you are looking for,
and avoids the need to parse wikitext itself by fetching the HTML from the
Wikimedia REST API.


>
> Maybe you can even just extract the images from the ZIM files.
>
> Nemo
>
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>



-- 
Gabriel Wicke
Principal Engineer, Wikimedia Foundation
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] A couple videos: Parsoid with Reading Engineering; Reading Showcase 20150824

2015-09-17 Thread Adam Baso 
Hi all,

Just wanted to share a couple recent videos. Enjoy!

== Parsoid with Reading ==

https://youtu.be/3WJID_WC7BQ?t=14

C. Scott Ananian and Subbu Sastry from the Wikimedia Foundation provide an
overview of Parsoid, a Wikimedia technology that supports HTML to wikitext
(and vice versa) translation, supporting rich annotated output markup for
translation layers and other clients (e.g., Wikipedia related technology)
to more specifically query elements from wiki content.

You may have caught some of this material at Wikimania, or maybe this is
your first time. In any event, it's good stuff! This was an interactive
session with questions from Wikimedia Reading Engineering.

== Reading Showcase 20150824 ==

https://youtu.be/F9ExzUaRI0k

About every 4 weeks the Wikimedia Reading department gets together to
showcase experiments and works in progress and the like. This is the
session from 24-August-2015.

-Adam
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Saving MediaWiki:common.js

2015-09-17 Thread Legault, Phillip [ITSUS] 
Sorry it took me a while to get back to this.
[Thu Sep 17 14:42:22 2015] [error] [client 10.22.165.73] File does not exist: 
/app/mediawiki/mw/skins/chameleon/kai_stars-dev.png, referer: 
server/index.php?title=MediaWiki:Common.js=edit

I rebuilt the database from my production system that is working and I'm still 
getting this error.



-Original Message-
From: wikitech-l-boun...@lists.wikimedia.org 
[mailto:wikitech-l-boun...@lists.wikimedia.org] On Behalf Of Brian Wolff
Sent: Wednesday, August 26, 2015 5:31 PM
To: Wikimedia developers
Subject: Re: [Wikitech-l] Saving MediaWiki:common.js

Did you check your php error log (And make sure that php error logging, 
particularly of fatal errors is enabled).

Check also your web server error log (if php segfaults or something, it would 
show up there)

--
-bawolff

On 8/26/15, Legault, Phillip [ITSUS]  wrote:
> Yes it is a private site
>
> I don't get any errors, I enable debug and still nothing
>
>
>
> MediaWiki  1.23.10
>
>
>
>
>
>
>
>
>
>
>
> -Original Message-
> From: wikitech-l-boun...@lists.wikimedia.org
> [mailto:wikitech-l-boun...@lists.wikimedia.org] On Behalf Of Andre 
> Klapper
> Sent: Wednesday, August 26, 2015 4:55 PM
> To: Wikimedia developers
> Subject: Re: [Wikitech-l] Saving MediaWiki:common.js
>
>
>
> Hi Phillip,
>
>
>
> On Wed, 2015-08-26 at 19:38 +, Legault, Phillip [ITSUS] wrote:
>
>> I ran into an issue trying to save MediaWiki:Common.js I get this 
>> page
>
>> not available
>
>> (index.php?title=MediaWiki:Common.js=submit)
>
>>
>
>> MediaWiki:Common.css save fine
>
>
>
> On some Wikimedia site (which one)? Or a private wiki?
>
>
>
> Is that the complete error message displayed (if not: please make sure 
> to not post private data like your IP adress)?
>
>
>
> Thanks,
>
> andre
>
> --
>
> Andre Klapper | Wikimedia Bugwrangler
>
> http://blogs.gnome.org/aklapper/
>
>
>
>
>
>
>
> ___
>
> Wikitech-l mailing list
>
> Wikitech-l@lists.wikimedia.org
>
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Announcing the launch of Maps

2015-09-17 Thread Tyler Romeo 
I feel like there's an argument to be made that the images are a derived
work from the data itself, so it may be best to just stick with OSM's
license rather than run into any possible issues. (Also, of course, I am
biased and prefer copyleft licenses since they support free information.)


*-- *
*Tyler Romeo*
Stevens Institute of Technology, Class of 2016
Major in Computer Science

On Thu, Sep 17, 2015 at 3:49 PM, Ryan Kaldari 
wrote:

> And in case that isn't decided, I would suggest CC0 so that we don't have
> to link to a big explanation from each tile rendering :)
>
> On Thu, Sep 17, 2015 at 12:40 PM, Ryan Kaldari 
> wrote:
>
> > What is the license for the map tiles, i.e. the cartography rather than
> > the data? Open Street Maps uses CC-BY-SA, although they don't really make
> > it clear who is supposed to be attributed. It looks like our tiles are
> > different though.
> >
> > On Thu, Sep 17, 2015 at 12:26 PM, Tomasz Finc 
> wrote:
> >
> >> The Discovery Department has launched an experimental tile and static
> maps
> >> service available at https://maps.wikimedia.org.
> >>
> >> Using this service you can browse and embed map tiles into your own
> tools
> >> using OpenStreetMap data. Currently, we handle traffic from *.wmflabs
> .org
> >> and *.wikivoyage .org (referrer header must be either missing or set to
> >> these values) but we would like to open it up to Wikipedia traffic if we
> >> see enough use. Our hope is that this service fits the needs of the
> >> numerous maps developers and tool authors who have asked for a WMF
> hosted
> >> tile service with an initial focus on WikiVoyage.
> >>
> >> We'd love for you to try our new service, experiment writing tools using
> >> our tiles, and giving us feedback <
> >> https://www.mediawiki.org/wiki/Talk:Maps> .
> >> If you've built a tool using OpenStreetMap-based imagery then using our
> >> service is a simple drop-in replacement.
> >>
> >> Getting started is as easy as
> >> https://www.mediawiki.org/wiki/Maps#Getting_Started
> >>
> >> How can you help?
> >>
> >> * Adapt your labs tool to use this service - for example, use Leaflet js
> >> library and point it to https://maps.wikimedia.org
> >> * File bugs in Phabricator
> >> 
> >> * Provide us feedback to help guide future features
> >> 
> >> * Improve our map style  >
> >> * Improve our data extraction
> >> 
> >>
> >> Based on usage and your feedback, the Discovery team
> >>  will decide how to proceed.
> >>
> >> We could add more data sources (both vector and raster), work on
> >> additional
> >> services such as static maps or geosearch, work on supporting all
> >> languages, switch to client-side WebGL rendering, etc. Please help us
> >> decide what is most important.
> >>
> >> https://www.mediawiki.org/wiki/Maps has more about the project and
> >> related
> >> Maps work.
> >>
> >> == In Depth ==
> >>
> >> Tiles are served from https://maps.wikimedia.org, but can only be
> >> accessed
> >> from any subdomains of *.wmflabs .org and *.wikivoyage.org.
> Kartotherian
> >> can produce tiles as images (png), and as raw vector data (PBF Mapbox
> >> format or json):
> >>
> >> .../{source}/{zoom}/{x}/{y}[@{scale}x].{format}
> >>
> >> Additionally, Kartotherian can produce snapshot (static) images of any
> >> location, scaling, and zoom level with
> >>
> >> .../{source},{zoom},{lat},{lon},{width}x{height}[@{scale}x].{format}.
> >>
> >> For example, to get an image centered at 42,-3.14, at zoom level 4, size
> >> 800x600, use
> >> https://maps.wikimedia.org/img/osm-intl,4,42,-3.14,800x600.png
> >> (copy/paste the link, or else it might not work due to referrer
> >> restriction).
> >>
> >> Do note that the static feature is highly experimental right now.
> >>
> >> We would like to thank WMF Ops (especially Alex Kosiaris, Brandon Black,
> >> and Jaime Crespo), services team, OSM community and engineers, and the
> >> Mapnik and Mapbox teams. The project would not have completed so fast
> >> without you.
> >>
> >> Thank You
> >>
> >> --tomasz
> >> ___
> >> Wikitech-l mailing list
> >> Wikitech-l@lists.wikimedia.org
> >> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> >
> >
> >
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Announcing the launch of Maps

2015-09-17 Thread Oliver Keyes 
While the presentation of data can be copyrighted within the United
States, and database rights permit the pseudo-copyrighting of data
within the European Union, our core servers are hosted within the
United States - a jurisdiction where data /itself/ cannot be
copyrighted. Our license does not have to derive from OSM.

I agree with Ryan that CC-0 is the best approach here.

On 17 September 2015 at 16:00, Tyler Romeo  wrote:
> I feel like there's an argument to be made that the images are a derived
> work from the data itself, so it may be best to just stick with OSM's
> license rather than run into any possible issues. (Also, of course, I am
> biased and prefer copyleft licenses since they support free information.)
>
>
> *-- *
> *Tyler Romeo*
> Stevens Institute of Technology, Class of 2016
> Major in Computer Science
>
> On Thu, Sep 17, 2015 at 3:49 PM, Ryan Kaldari 
> wrote:
>
>> And in case that isn't decided, I would suggest CC0 so that we don't have
>> to link to a big explanation from each tile rendering :)
>>
>> On Thu, Sep 17, 2015 at 12:40 PM, Ryan Kaldari 
>> wrote:
>>
>> > What is the license for the map tiles, i.e. the cartography rather than
>> > the data? Open Street Maps uses CC-BY-SA, although they don't really make
>> > it clear who is supposed to be attributed. It looks like our tiles are
>> > different though.
>> >
>> > On Thu, Sep 17, 2015 at 12:26 PM, Tomasz Finc 
>> wrote:
>> >
>> >> The Discovery Department has launched an experimental tile and static
>> maps
>> >> service available at https://maps.wikimedia.org.
>> >>
>> >> Using this service you can browse and embed map tiles into your own
>> tools
>> >> using OpenStreetMap data. Currently, we handle traffic from *.wmflabs
>> .org
>> >> and *.wikivoyage .org (referrer header must be either missing or set to
>> >> these values) but we would like to open it up to Wikipedia traffic if we
>> >> see enough use. Our hope is that this service fits the needs of the
>> >> numerous maps developers and tool authors who have asked for a WMF
>> hosted
>> >> tile service with an initial focus on WikiVoyage.
>> >>
>> >> We'd love for you to try our new service, experiment writing tools using
>> >> our tiles, and giving us feedback <
>> >> https://www.mediawiki.org/wiki/Talk:Maps> .
>> >> If you've built a tool using OpenStreetMap-based imagery then using our
>> >> service is a simple drop-in replacement.
>> >>
>> >> Getting started is as easy as
>> >> https://www.mediawiki.org/wiki/Maps#Getting_Started
>> >>
>> >> How can you help?
>> >>
>> >> * Adapt your labs tool to use this service - for example, use Leaflet js
>> >> library and point it to https://maps.wikimedia.org
>> >> * File bugs in Phabricator
>> >> 
>> >> * Provide us feedback to help guide future features
>> >> 
>> >> * Improve our map style > >
>> >> * Improve our data extraction
>> >> 
>> >>
>> >> Based on usage and your feedback, the Discovery team
>> >>  will decide how to proceed.
>> >>
>> >> We could add more data sources (both vector and raster), work on
>> >> additional
>> >> services such as static maps or geosearch, work on supporting all
>> >> languages, switch to client-side WebGL rendering, etc. Please help us
>> >> decide what is most important.
>> >>
>> >> https://www.mediawiki.org/wiki/Maps has more about the project and
>> >> related
>> >> Maps work.
>> >>
>> >> == In Depth ==
>> >>
>> >> Tiles are served from https://maps.wikimedia.org, but can only be
>> >> accessed
>> >> from any subdomains of *.wmflabs .org and *.wikivoyage.org.
>> Kartotherian
>> >> can produce tiles as images (png), and as raw vector data (PBF Mapbox
>> >> format or json):
>> >>
>> >> .../{source}/{zoom}/{x}/{y}[@{scale}x].{format}
>> >>
>> >> Additionally, Kartotherian can produce snapshot (static) images of any
>> >> location, scaling, and zoom level with
>> >>
>> >> .../{source},{zoom},{lat},{lon},{width}x{height}[@{scale}x].{format}.
>> >>
>> >> For example, to get an image centered at 42,-3.14, at zoom level 4, size
>> >> 800x600, use
>> >> https://maps.wikimedia.org/img/osm-intl,4,42,-3.14,800x600.png
>> >> (copy/paste the link, or else it might not work due to referrer
>> >> restriction).
>> >>
>> >> Do note that the static feature is highly experimental right now.
>> >>
>> >> We would like to thank WMF Ops (especially Alex Kosiaris, Brandon Black,
>> >> and Jaime Crespo), services team, OSM community and engineers, and the
>> >> Mapnik and Mapbox teams. The project would not have completed so fast
>> >> without you.
>> >>
>> >> Thank You
>> >>
>> >> --tomasz
>> >> ___
>> >> Wikitech-l mailing list
>> >>

[Wikitech-l] Announcing the launch of Maps

2015-09-17 Thread Tomasz Finc 
The Discovery Department has launched an experimental tile and static maps
service available at https://maps.wikimedia.org.

Using this service you can browse and embed map tiles into your own tools
using OpenStreetMap data. Currently, we handle traffic from *.wmflabs .org
and *.wikivoyage .org (referrer header must be either missing or set to
these values) but we would like to open it up to Wikipedia traffic if we
see enough use. Our hope is that this service fits the needs of the
numerous maps developers and tool authors who have asked for a WMF hosted
tile service with an initial focus on WikiVoyage.

We'd love for you to try our new service, experiment writing tools using
our tiles, and giving us feedback  .
If you've built a tool using OpenStreetMap-based imagery then using our
service is a simple drop-in replacement.

Getting started is as easy as
https://www.mediawiki.org/wiki/Maps#Getting_Started

How can you help?

* Adapt your labs tool to use this service - for example, use Leaflet js
library and point it to https://maps.wikimedia.org
* File bugs in Phabricator

* Provide us feedback to help guide future features

* Improve our map style 
* Improve our data extraction


Based on usage and your feedback, the Discovery team
 will decide how to proceed.

We could add more data sources (both vector and raster), work on additional
services such as static maps or geosearch, work on supporting all
languages, switch to client-side WebGL rendering, etc. Please help us
decide what is most important.

https://www.mediawiki.org/wiki/Maps has more about the project and related
Maps work.

== In Depth ==

Tiles are served from https://maps.wikimedia.org, but can only be accessed
from any subdomains of *.wmflabs .org and *.wikivoyage.org.  Kartotherian
can produce tiles as images (png), and as raw vector data (PBF Mapbox
format or json):

.../{source}/{zoom}/{x}/{y}[@{scale}x].{format}

Additionally, Kartotherian can produce snapshot (static) images of any
location, scaling, and zoom level with

.../{source},{zoom},{lat},{lon},{width}x{height}[@{scale}x].{format}.

For example, to get an image centered at 42,-3.14, at zoom level 4, size
800x600, use https://maps.wikimedia.org/img/osm-intl,4,42,-3.14,800x600.png
(copy/paste the link, or else it might not work due to referrer
restriction).

Do note that the static feature is highly experimental right now.

We would like to thank WMF Ops (especially Alex Kosiaris, Brandon Black,
and Jaime Crespo), services team, OSM community and engineers, and the
Mapnik and Mapbox teams. The project would not have completed so fast
without you.

Thank You

--tomasz
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Announcing the launch of Maps

2015-09-17 Thread Max Semenik 
The underlying data is ODbL which is properly attributed per OSM
guidelines. The license for tiles themselves is a different issue, OSM
decided to CC them, we're not claiming any additional copyright so far,
until legal tells us in https://phabricator.wikimedia.org/T111224

On Thu, Sep 17, 2015 at 12:40 PM, Ryan Kaldari 
wrote:

> What is the license for the map tiles, i.e. the cartography rather than the
> data? Open Street Maps uses CC-BY-SA, although they don't really make it
> clear who is supposed to be attributed. It looks like our tiles are
> different though.
>
> On Thu, Sep 17, 2015 at 12:26 PM, Tomasz Finc  wrote:
>
> > The Discovery Department has launched an experimental tile and static
> maps
> > service available at https://maps.wikimedia.org.
> >
> > Using this service you can browse and embed map tiles into your own tools
> > using OpenStreetMap data. Currently, we handle traffic from *.wmflabs
> .org
> > and *.wikivoyage .org (referrer header must be either missing or set to
> > these values) but we would like to open it up to Wikipedia traffic if we
> > see enough use. Our hope is that this service fits the needs of the
> > numerous maps developers and tool authors who have asked for a WMF hosted
> > tile service with an initial focus on WikiVoyage.
> >
> > We'd love for you to try our new service, experiment writing tools using
> > our tiles, and giving us feedback <
> > https://www.mediawiki.org/wiki/Talk:Maps> .
> > If you've built a tool using OpenStreetMap-based imagery then using our
> > service is a simple drop-in replacement.
> >
> > Getting started is as easy as
> > https://www.mediawiki.org/wiki/Maps#Getting_Started
> >
> > How can you help?
> >
> > * Adapt your labs tool to use this service - for example, use Leaflet js
> > library and point it to https://maps.wikimedia.org
> > * File bugs in Phabricator
> > 
> > * Provide us feedback to help guide future features
> > 
> > * Improve our map style 
> > * Improve our data extraction
> > 
> >
> > Based on usage and your feedback, the Discovery team
> >  will decide how to proceed.
> >
> > We could add more data sources (both vector and raster), work on
> additional
> > services such as static maps or geosearch, work on supporting all
> > languages, switch to client-side WebGL rendering, etc. Please help us
> > decide what is most important.
> >
> > https://www.mediawiki.org/wiki/Maps has more about the project and
> related
> > Maps work.
> >
> > == In Depth ==
> >
> > Tiles are served from https://maps.wikimedia.org, but can only be
> accessed
> > from any subdomains of *.wmflabs .org and *.wikivoyage.org.
> Kartotherian
> > can produce tiles as images (png), and as raw vector data (PBF Mapbox
> > format or json):
> >
> > .../{source}/{zoom}/{x}/{y}[@{scale}x].{format}
> >
> > Additionally, Kartotherian can produce snapshot (static) images of any
> > location, scaling, and zoom level with
> >
> > .../{source},{zoom},{lat},{lon},{width}x{height}[@{scale}x].{format}.
> >
> > For example, to get an image centered at 42,-3.14, at zoom level 4, size
> > 800x600, use
> > https://maps.wikimedia.org/img/osm-intl,4,42,-3.14,800x600.png
> > (copy/paste the link, or else it might not work due to referrer
> > restriction).
> >
> > Do note that the static feature is highly experimental right now.
> >
> > We would like to thank WMF Ops (especially Alex Kosiaris, Brandon Black,
> > and Jaime Crespo), services team, OSM community and engineers, and the
> > Mapnik and Mapbox teams. The project would not have completed so fast
> > without you.
> >
> > Thank You
> >
> > --tomasz
> > ___
> > Wikitech-l mailing list
> > Wikitech-l@lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>



-- 
Best regards,
Max Semenik ([[User:MaxSem]])
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Announcing the launch of Maps

2015-09-17 Thread Ryan Kaldari 
And in case that isn't decided, I would suggest CC0 so that we don't have
to link to a big explanation from each tile rendering :)

On Thu, Sep 17, 2015 at 12:40 PM, Ryan Kaldari 
wrote:

> What is the license for the map tiles, i.e. the cartography rather than
> the data? Open Street Maps uses CC-BY-SA, although they don't really make
> it clear who is supposed to be attributed. It looks like our tiles are
> different though.
>
> On Thu, Sep 17, 2015 at 12:26 PM, Tomasz Finc  wrote:
>
>> The Discovery Department has launched an experimental tile and static maps
>> service available at https://maps.wikimedia.org.
>>
>> Using this service you can browse and embed map tiles into your own tools
>> using OpenStreetMap data. Currently, we handle traffic from *.wmflabs .org
>> and *.wikivoyage .org (referrer header must be either missing or set to
>> these values) but we would like to open it up to Wikipedia traffic if we
>> see enough use. Our hope is that this service fits the needs of the
>> numerous maps developers and tool authors who have asked for a WMF hosted
>> tile service with an initial focus on WikiVoyage.
>>
>> We'd love for you to try our new service, experiment writing tools using
>> our tiles, and giving us feedback <
>> https://www.mediawiki.org/wiki/Talk:Maps> .
>> If you've built a tool using OpenStreetMap-based imagery then using our
>> service is a simple drop-in replacement.
>>
>> Getting started is as easy as
>> https://www.mediawiki.org/wiki/Maps#Getting_Started
>>
>> How can you help?
>>
>> * Adapt your labs tool to use this service - for example, use Leaflet js
>> library and point it to https://maps.wikimedia.org
>> * File bugs in Phabricator
>> 
>> * Provide us feedback to help guide future features
>> 
>> * Improve our map style 
>> * Improve our data extraction
>> 
>>
>> Based on usage and your feedback, the Discovery team
>>  will decide how to proceed.
>>
>> We could add more data sources (both vector and raster), work on
>> additional
>> services such as static maps or geosearch, work on supporting all
>> languages, switch to client-side WebGL rendering, etc. Please help us
>> decide what is most important.
>>
>> https://www.mediawiki.org/wiki/Maps has more about the project and
>> related
>> Maps work.
>>
>> == In Depth ==
>>
>> Tiles are served from https://maps.wikimedia.org, but can only be
>> accessed
>> from any subdomains of *.wmflabs .org and *.wikivoyage.org.  Kartotherian
>> can produce tiles as images (png), and as raw vector data (PBF Mapbox
>> format or json):
>>
>> .../{source}/{zoom}/{x}/{y}[@{scale}x].{format}
>>
>> Additionally, Kartotherian can produce snapshot (static) images of any
>> location, scaling, and zoom level with
>>
>> .../{source},{zoom},{lat},{lon},{width}x{height}[@{scale}x].{format}.
>>
>> For example, to get an image centered at 42,-3.14, at zoom level 4, size
>> 800x600, use
>> https://maps.wikimedia.org/img/osm-intl,4,42,-3.14,800x600.png
>> (copy/paste the link, or else it might not work due to referrer
>> restriction).
>>
>> Do note that the static feature is highly experimental right now.
>>
>> We would like to thank WMF Ops (especially Alex Kosiaris, Brandon Black,
>> and Jaime Crespo), services team, OSM community and engineers, and the
>> Mapnik and Mapbox teams. The project would not have completed so fast
>> without you.
>>
>> Thank You
>>
>> --tomasz
>> ___
>> Wikitech-l mailing list
>> Wikitech-l@lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
>
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Announcing the launch of Maps

2015-09-17 Thread Ryan Kaldari 
ODbL (which is the license for the data) doesn't require any particular
license for maps produced from ODbL data. (And in the U.S. data isn't
copyrightable anyway.) There is a copyright on the style and presentation
of the tiles which is separate from any copyright on the data. This
copyright belongs to whoever created the styles for the tiles (or their
employer). Unless whoever that is explicitly puts it under a license, the
style of the tiles is fully copyrighted (in the U.S. at least).

On Thu, Sep 17, 2015 at 1:00 PM, Tyler Romeo  wrote:

> I feel like there's an argument to be made that the images are a derived
> work from the data itself, so it may be best to just stick with OSM's
> license rather than run into any possible issues. (Also, of course, I am
> biased and prefer copyleft licenses since they support free information.)
>
>
> *-- *
> *Tyler Romeo*
> Stevens Institute of Technology, Class of 2016
> Major in Computer Science
>
> On Thu, Sep 17, 2015 at 3:49 PM, Ryan Kaldari 
> wrote:
>
> > And in case that isn't decided, I would suggest CC0 so that we don't have
> > to link to a big explanation from each tile rendering :)
> >
> > On Thu, Sep 17, 2015 at 12:40 PM, Ryan Kaldari 
> > wrote:
> >
> > > What is the license for the map tiles, i.e. the cartography rather than
> > > the data? Open Street Maps uses CC-BY-SA, although they don't really
> make
> > > it clear who is supposed to be attributed. It looks like our tiles are
> > > different though.
> > >
> > > On Thu, Sep 17, 2015 at 12:26 PM, Tomasz Finc 
> > wrote:
> > >
> > >> The Discovery Department has launched an experimental tile and static
> > maps
> > >> service available at https://maps.wikimedia.org.
> > >>
> > >> Using this service you can browse and embed map tiles into your own
> > tools
> > >> using OpenStreetMap data. Currently, we handle traffic from *.wmflabs
> > .org
> > >> and *.wikivoyage .org (referrer header must be either missing or set
> to
> > >> these values) but we would like to open it up to Wikipedia traffic if
> we
> > >> see enough use. Our hope is that this service fits the needs of the
> > >> numerous maps developers and tool authors who have asked for a WMF
> > hosted
> > >> tile service with an initial focus on WikiVoyage.
> > >>
> > >> We'd love for you to try our new service, experiment writing tools
> using
> > >> our tiles, and giving us feedback <
> > >> https://www.mediawiki.org/wiki/Talk:Maps> .
> > >> If you've built a tool using OpenStreetMap-based imagery then using
> our
> > >> service is a simple drop-in replacement.
> > >>
> > >> Getting started is as easy as
> > >> https://www.mediawiki.org/wiki/Maps#Getting_Started
> > >>
> > >> How can you help?
> > >>
> > >> * Adapt your labs tool to use this service - for example, use Leaflet
> js
> > >> library and point it to https://maps.wikimedia.org
> > >> * File bugs in Phabricator
> > >> 
> > >> * Provide us feedback to help guide future features
> > >> 
> > >> * Improve our map style <
> https://github.com/kartotherian/osm-bright.tm2
> > >
> > >> * Improve our data extraction
> > >> 
> > >>
> > >> Based on usage and your feedback, the Discovery team
> > >>  will decide how to
> proceed.
> > >>
> > >> We could add more data sources (both vector and raster), work on
> > >> additional
> > >> services such as static maps or geosearch, work on supporting all
> > >> languages, switch to client-side WebGL rendering, etc. Please help us
> > >> decide what is most important.
> > >>
> > >> https://www.mediawiki.org/wiki/Maps has more about the project and
> > >> related
> > >> Maps work.
> > >>
> > >> == In Depth ==
> > >>
> > >> Tiles are served from https://maps.wikimedia.org, but can only be
> > >> accessed
> > >> from any subdomains of *.wmflabs .org and *.wikivoyage.org.
> > Kartotherian
> > >> can produce tiles as images (png), and as raw vector data (PBF Mapbox
> > >> format or json):
> > >>
> > >> .../{source}/{zoom}/{x}/{y}[@{scale}x].{format}
> > >>
> > >> Additionally, Kartotherian can produce snapshot (static) images of any
> > >> location, scaling, and zoom level with
> > >>
> > >> .../{source},{zoom},{lat},{lon},{width}x{height}[@{scale}x].{format}.
> > >>
> > >> For example, to get an image centered at 42,-3.14, at zoom level 4,
> size
> > >> 800x600, use
> > >> https://maps.wikimedia.org/img/osm-intl,4,42,-3.14,800x600.png
> > >> (copy/paste the link, or else it might not work due to referrer
> > >> restriction).
> > >>
> > >> Do note that the static feature is highly experimental right now.
> > >>
> > >> We would like to thank WMF Ops (especially Alex Kosiaris, Brandon
> Black,
> > >> and Jaime Crespo), services team, OSM community and engineers, and the
>

Re: [Wikitech-l] Announcing the launch of Maps

2015-09-17 Thread Ryan Kaldari 
What is the license for the map tiles, i.e. the cartography rather than the
data? Open Street Maps uses CC-BY-SA, although they don't really make it
clear who is supposed to be attributed. It looks like our tiles are
different though.

On Thu, Sep 17, 2015 at 12:26 PM, Tomasz Finc  wrote:

> The Discovery Department has launched an experimental tile and static maps
> service available at https://maps.wikimedia.org.
>
> Using this service you can browse and embed map tiles into your own tools
> using OpenStreetMap data. Currently, we handle traffic from *.wmflabs .org
> and *.wikivoyage .org (referrer header must be either missing or set to
> these values) but we would like to open it up to Wikipedia traffic if we
> see enough use. Our hope is that this service fits the needs of the
> numerous maps developers and tool authors who have asked for a WMF hosted
> tile service with an initial focus on WikiVoyage.
>
> We'd love for you to try our new service, experiment writing tools using
> our tiles, and giving us feedback <
> https://www.mediawiki.org/wiki/Talk:Maps> .
> If you've built a tool using OpenStreetMap-based imagery then using our
> service is a simple drop-in replacement.
>
> Getting started is as easy as
> https://www.mediawiki.org/wiki/Maps#Getting_Started
>
> How can you help?
>
> * Adapt your labs tool to use this service - for example, use Leaflet js
> library and point it to https://maps.wikimedia.org
> * File bugs in Phabricator
> 
> * Provide us feedback to help guide future features
> 
> * Improve our map style 
> * Improve our data extraction
> 
>
> Based on usage and your feedback, the Discovery team
>  will decide how to proceed.
>
> We could add more data sources (both vector and raster), work on additional
> services such as static maps or geosearch, work on supporting all
> languages, switch to client-side WebGL rendering, etc. Please help us
> decide what is most important.
>
> https://www.mediawiki.org/wiki/Maps has more about the project and related
> Maps work.
>
> == In Depth ==
>
> Tiles are served from https://maps.wikimedia.org, but can only be accessed
> from any subdomains of *.wmflabs .org and *.wikivoyage.org.  Kartotherian
> can produce tiles as images (png), and as raw vector data (PBF Mapbox
> format or json):
>
> .../{source}/{zoom}/{x}/{y}[@{scale}x].{format}
>
> Additionally, Kartotherian can produce snapshot (static) images of any
> location, scaling, and zoom level with
>
> .../{source},{zoom},{lat},{lon},{width}x{height}[@{scale}x].{format}.
>
> For example, to get an image centered at 42,-3.14, at zoom level 4, size
> 800x600, use
> https://maps.wikimedia.org/img/osm-intl,4,42,-3.14,800x600.png
> (copy/paste the link, or else it might not work due to referrer
> restriction).
>
> Do note that the static feature is highly experimental right now.
>
> We would like to thank WMF Ops (especially Alex Kosiaris, Brandon Black,
> and Jaime Crespo), services team, OSM community and engineers, and the
> Mapnik and Mapbox teams. The project would not have completed so fast
> without you.
>
> Thank You
>
> --tomasz
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Announcing the launch of Maps

2015-09-17 Thread Yuri Astrakhan 
The STYLE of the map is in https://github.com/kartotherian/osm-bright.tm2
which is a fork from https://github.com/mapbox/mapbox-studio-osm-bright.tm2
-- released by Mapbox under a 3 clause BSD license

(IANAL).

On Thu, Sep 17, 2015 at 11:21 PM, Max Semenik  wrote:

> The underlying data is ODbL which is properly attributed per OSM
> guidelines. The license for tiles themselves is a different issue, OSM
> decided to CC them, we're not claiming any additional copyright so far,
> until legal tells us in https://phabricator.wikimedia.org/T111224
>
> On Thu, Sep 17, 2015 at 12:40 PM, Ryan Kaldari 
> wrote:
>
> > What is the license for the map tiles, i.e. the cartography rather than
> the
> > data? Open Street Maps uses CC-BY-SA, although they don't really make it
> > clear who is supposed to be attributed. It looks like our tiles are
> > different though.
> >
> > On Thu, Sep 17, 2015 at 12:26 PM, Tomasz Finc 
> wrote:
> >
> > > The Discovery Department has launched an experimental tile and static
> > maps
> > > service available at https://maps.wikimedia.org.
> > >
> > > Using this service you can browse and embed map tiles into your own
> tools
> > > using OpenStreetMap data. Currently, we handle traffic from *.wmflabs
> > .org
> > > and *.wikivoyage .org (referrer header must be either missing or set to
> > > these values) but we would like to open it up to Wikipedia traffic if
> we
> > > see enough use. Our hope is that this service fits the needs of the
> > > numerous maps developers and tool authors who have asked for a WMF
> hosted
> > > tile service with an initial focus on WikiVoyage.
> > >
> > > We'd love for you to try our new service, experiment writing tools
> using
> > > our tiles, and giving us feedback <
> > > https://www.mediawiki.org/wiki/Talk:Maps> .
> > > If you've built a tool using OpenStreetMap-based imagery then using our
> > > service is a simple drop-in replacement.
> > >
> > > Getting started is as easy as
> > > https://www.mediawiki.org/wiki/Maps#Getting_Started
> > >
> > > How can you help?
> > >
> > > * Adapt your labs tool to use this service - for example, use Leaflet
> js
> > > library and point it to https://maps.wikimedia.org
> > > * File bugs in Phabricator
> > > 
> > > * Provide us feedback to help guide future features
> > > 
> > > * Improve our map style <
> https://github.com/kartotherian/osm-bright.tm2>
> > > * Improve our data extraction
> > > 
> > >
> > > Based on usage and your feedback, the Discovery team
> > >  will decide how to proceed.
> > >
> > > We could add more data sources (both vector and raster), work on
> > additional
> > > services such as static maps or geosearch, work on supporting all
> > > languages, switch to client-side WebGL rendering, etc. Please help us
> > > decide what is most important.
> > >
> > > https://www.mediawiki.org/wiki/Maps has more about the project and
> > related
> > > Maps work.
> > >
> > > == In Depth ==
> > >
> > > Tiles are served from https://maps.wikimedia.org, but can only be
> > accessed
> > > from any subdomains of *.wmflabs .org and *.wikivoyage.org.
> > Kartotherian
> > > can produce tiles as images (png), and as raw vector data (PBF Mapbox
> > > format or json):
> > >
> > > .../{source}/{zoom}/{x}/{y}[@{scale}x].{format}
> > >
> > > Additionally, Kartotherian can produce snapshot (static) images of any
> > > location, scaling, and zoom level with
> > >
> > > .../{source},{zoom},{lat},{lon},{width}x{height}[@{scale}x].{format}.
> > >
> > > For example, to get an image centered at 42,-3.14, at zoom level 4,
> size
> > > 800x600, use
> > > https://maps.wikimedia.org/img/osm-intl,4,42,-3.14,800x600.png
> > > (copy/paste the link, or else it might not work due to referrer
> > > restriction).
> > >
> > > Do note that the static feature is highly experimental right now.
> > >
> > > We would like to thank WMF Ops (especially Alex Kosiaris, Brandon
> Black,
> > > and Jaime Crespo), services team, OSM community and engineers, and the
> > > Mapnik and Mapbox teams. The project would not have completed so fast
> > > without you.
> > >
> > > Thank You
> > >
> > > --tomasz
> > > ___
> > > Wikitech-l mailing list
> > > Wikitech-l@lists.wikimedia.org
> > > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> > ___
> > Wikitech-l mailing list
> > Wikitech-l@lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> >
>
>
>
> --
> Best regards,
> Max Semenik ([[User:MaxSem]])
> ___
> Wikitech-l mailing list
>

Re: [Wikitech-l] Announcing the launch of Maps

2015-09-17 Thread Brion Vibber 
Awesome sauce! Congrats to everyone who's been working on this.


On Thu, Sep 17, 2015 at 12:26 PM, Tomasz Finc  wrote:

> == In Depth ==
>
> Tiles are served from https://maps.wikimedia.org, but can only be accessed
> from any subdomains of *.wmflabs .org and *.wikivoyage.org.  Kartotherian
> can produce tiles as images (png), and as raw vector data (PBF Mapbox
> format or json):
>
> .../{source}/{zoom}/{x}/{y}[@{scale}x].{format}
>

Just a note -- the scale factor is for high-density displays, which are now
standard on most phones and increasingly common on laptops (1.5x is common
on small Windows laptops with 1080p displays). However the available scale
factors are hardcoded, so you can't just take an arbitrary device scale
factor (window.devicePixelRatio in HTML5) and feed it in, as it might not
match and you'll get HTTP errors out.

I've got a provisional patch in for the example front-end which manually
brackets to the available settings:
https://github.com/kartotherian/kartotherian/pull/13

It'd be awesome to either expose the available scaling factors as a list
via JSON API, or move the bracketing logic into the server side so you can
send a query with your actual scale factor and get back one that will work.

(If available scaling factors depend on the layer source being viewed, this
might be more complicated.)


> We would like to thank WMF Ops (especially Alex Kosiaris, Brandon Black,
> and Jaime Crespo), services team, OSM community and engineers, and the
> Mapnik and Mapbox teams. The project would not have completed so fast
> without you.
>

Seconded. :)

-- brion
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Announcing the launch of Maps

2015-09-17 Thread Yuri Astrakhan 
The bracketing was introduced to reduce the DDOS attack surface and
increase the cache hit ratio. This means that we could in theory do all the
magic of bracketing in the varnish (to keep the cache rate high), or we
could force the clients to do the logic.  The problem with Varnish approach
is that it puts business-level logic into Varnish's brittle C-style
configs.

On Thu, Sep 17, 2015 at 11:58 PM, Brion Vibber 
wrote:

> Awesome sauce! Congrats to everyone who's been working on this.
>
>
> On Thu, Sep 17, 2015 at 12:26 PM, Tomasz Finc  wrote:
>
> > == In Depth ==
> >
> > Tiles are served from https://maps.wikimedia.org, but can only be
> accessed
> > from any subdomains of *.wmflabs .org and *.wikivoyage.org.
> Kartotherian
> > can produce tiles as images (png), and as raw vector data (PBF Mapbox
> > format or json):
> >
> > .../{source}/{zoom}/{x}/{y}[@{scale}x].{format}
> >
>
> Just a note -- the scale factor is for high-density displays, which are now
> standard on most phones and increasingly common on laptops (1.5x is common
> on small Windows laptops with 1080p displays). However the available scale
> factors are hardcoded, so you can't just take an arbitrary device scale
> factor (window.devicePixelRatio in HTML5) and feed it in, as it might not
> match and you'll get HTTP errors out.
>
> I've got a provisional patch in for the example front-end which manually
> brackets to the available settings:
> https://github.com/kartotherian/kartotherian/pull/13
>
> It'd be awesome to either expose the available scaling factors as a list
> via JSON API, or move the bracketing logic into the server side so you can
> send a query with your actual scale factor and get back one that will work.
>
> (If available scaling factors depend on the layer source being viewed, this
> might be more complicated.)
>
>
> > We would like to thank WMF Ops (especially Alex Kosiaris, Brandon Black,
> > and Jaime Crespo), services team, OSM community and engineers, and the
> > Mapnik and Mapbox teams. The project would not have completed so fast
> > without you.
> >
>
> Seconded. :)
>
> -- brion
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Announcing the launch of Maps

2015-09-17 Thread Matthew Flaschen 

On 09/17/2015 03:26 PM, Tomasz Finc wrote:

The Discovery Department has launched an experimental tile and static maps
service available at https://maps.wikimedia.org.


Congratulations to the whole team.  This is a huge deal, and will open 
up our projects to whole new kinds of media, editing, and interactivity.


Matt Flaschen


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Saving MediaWiki:common.js

2015-09-17 Thread Benjamin Lees 
On Thu, Sep 17, 2015 at 3:03 PM, Legault, Phillip [ITSUS]
 wrote:
> Sorry it took me a while to get back to this.
> [Thu Sep 17 14:42:22 2015] [error] [client 10.22.165.73] File does not exist: 
> /app/mediawiki/mw/skins/chameleon/kai_stars-dev.png, referer: 
> server/index.php?title=MediaWiki:Common.js=edit

That's probably unrelated.

I would guess that mod_security is the culprit.

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] A couple videos: Parsoid with Reading Engineering; Reading Showcase 20150824

2015-09-17 Thread C. Scott Ananian 
Adam: could you add a link to
https://commons.wikimedia.org/wiki/File:Doing_Cool_Things_with_Wiki_Content_(Parsoid_Power!).pdf
to the youtube description for that talk?

Thanks!
 --scott

On Thu, Sep 17, 2015 at 1:58 PM, Adam Baso  wrote:
> Hi all,
>
> Just wanted to share a couple recent videos. Enjoy!
>
> == Parsoid with Reading ==
>
> https://youtu.be/3WJID_WC7BQ?t=14
>
> C. Scott Ananian and Subbu Sastry from the Wikimedia Foundation provide an
> overview of Parsoid, a Wikimedia technology that supports HTML to wikitext
> (and vice versa) translation, supporting rich annotated output markup for
> translation layers and other clients (e.g., Wikipedia related technology)
> to more specifically query elements from wiki content.
>
> You may have caught some of this material at Wikimania, or maybe this is
> your first time. In any event, it's good stuff! This was an interactive
> session with questions from Wikimedia Reading Engineering.
>
> == Reading Showcase 20150824 ==
>
> https://youtu.be/F9ExzUaRI0k
>
> About every 4 weeks the Wikimedia Reading department gets together to
> showcase experiments and works in progress and the like. This is the
> session from 24-August-2015.
>
> -Adam
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l



-- 
(http://cscott.net)

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] I love Parsoid but it doesn't want me

2015-09-17 Thread Ricordisamoa 
Stephen Niedzielski: "it seems like, as soon as you get the HTML the 
first thing you want to do, perhaps a little bit ironically because it's 
called Parsoid, it's parse the output a little bit more"

https://www.youtube.com/watch?v=3WJID_WC7BQ=35m14s

Il 23/07/2015 22:02, C. Scott Ananian ha scritto:

HTML5+RDFa is a machine-readable format.  But I think what you are asking
for is either better documentation of the template-related stuff (did you
read through the slides inhttps://phabricator.wikimedia.org/T105175  ?) or
HTML template parameter support (https://phabricator.wikimedia.org/T52587)
which is in the codebase but not enabled by default in production.
  --scott
​
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l



___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] A couple videos: Parsoid with Reading Engineering; Reading Showcase 20150824

2015-09-17 Thread Adam Baso 
Done.

On Thu, Sep 17, 2015 at 2:05 PM, C. Scott Ananian 
wrote:

> Adam: could you add a link to
>
> https://commons.wikimedia.org/wiki/File:Doing_Cool_Things_with_Wiki_Content_(Parsoid_Power!).pdf
> to the youtube description for that talk?
>
> Thanks!
>  --scott
>
> On Thu, Sep 17, 2015 at 1:58 PM, Adam Baso  wrote:
> > Hi all,
> >
> > Just wanted to share a couple recent videos. Enjoy!
> >
> > == Parsoid with Reading ==
> >
> > https://youtu.be/3WJID_WC7BQ?t=14
> >
> > C. Scott Ananian and Subbu Sastry from the Wikimedia Foundation provide
> an
> > overview of Parsoid, a Wikimedia technology that supports HTML to
> wikitext
> > (and vice versa) translation, supporting rich annotated output markup for
> > translation layers and other clients (e.g., Wikipedia related technology)
> > to more specifically query elements from wiki content.
> >
> > You may have caught some of this material at Wikimania, or maybe this is
> > your first time. In any event, it's good stuff! This was an interactive
> > session with questions from Wikimedia Reading Engineering.
> >
> > == Reading Showcase 20150824 ==
> >
> > https://youtu.be/F9ExzUaRI0k
> >
> > About every 4 weeks the Wikimedia Reading department gets together to
> > showcase experiments and works in progress and the like. This is the
> > session from 24-August-2015.
> >
> > -Adam
> > ___
> > Wikitech-l mailing list
> > Wikitech-l@lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
>
>
> --
> (http://cscott.net)
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] I love Parsoid but it doesn't want me

2015-09-17 Thread Subramanya Sastry 

On 09/17/2015 07:44 PM, Ricordisamoa wrote:
Stephen Niedzielski: "it seems like, as soon as you get the HTML the 
first thing you want to do, perhaps a little bit ironically because 
it's called Parsoid, it's parse the output a little bit more"

https://www.youtube.com/watch?v=3WJID_WC7BQ=35m14s


That is somewhat of a misunderstanding that Scott clarified.

Parsing is involved whenever you want to convert a string format to an 
object format.


So, unless you want to figure out a way to transfer DOM objects between 
server and client, you are going to continue transferring HTML strings 
which you then parse in the browser to rebuild the DOM representation.


Subbu.

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] I love Parsoid but it doesn't want me

2015-09-17 Thread C. Scott Ananian 
And as I responded there, if I gave you a JSON string instead, the first
thing you'd need to do is parse the JSON to turn it into something you can
use.

The difference is that JSON and html5 parsers are standard components in
every programming language.  Html5 even has a standard object
representation and manipulation library (DOM) which is available in every
major programming language.
  --scott
On Sep 17, 2015 8:45 PM, "Ricordisamoa" 
wrote:

> Stephen Niedzielski: "it seems like, as soon as you get the HTML the first
> thing you want to do, perhaps a little bit ironically because it's called
> Parsoid, it's parse the output a little bit more"
> https://www.youtube.com/watch?v=3WJID_WC7BQ=35m14s
>
> Il 23/07/2015 22:02, C. Scott Ananian ha scritto:
>
>> HTML5+RDFa is a machine-readable format.  But I think what you are asking
>> for is either better documentation of the template-related stuff (did you
>> read through the slides inhttps://phabricator.wikimedia.org/T105175  ?)
>> or
>> HTML template parameter support (https://phabricator.wikimedia.org/T52587
>> )
>> which is in the codebase but not enabled by default in production.
>>   --scott
>> ​
>> ___
>> Wikitech-l mailing list
>> Wikitech-l@lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>>
>
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l