In cases of excessive resource usage we have several options. Contact the
source, throttle them, or flat out disable access depending on what each
case calls for.

I have seen the dev team to this liberally in the past when needed. If any
one person or group is exploiting us by using unproportionate amounts of
resources  thats one thing, if we are just trying to make money by selling
access to what we already have thats another. Limiting abusive sources
shouldnt be an issue, but as soon as we start selling access we loose sight
of our mission.

On Sat, Jan 16, 2016 at 9:11 PM, Denny Vrandecic <dvrande...@wikimedia.org>
wrote:

> I find it rather surprising, but I very much find myself in agreement with
> most what Andreas Kolbe said on this thread.
>
> To give a bit more thoughts: I am not terribly worried about current
> crawlers. But currently, and more in the future, I expect us to provide
> more complex and this expensive APIs: a SPARQL endpoint, parsing APIs, etc.
> These will be simply expensive to operate. Not for infrequent users - say,
> to the benefit of us 70,000 editors - but for use cases that involve tens
> or millions of requests per day. These have the potential of burning a lot
> of funds to basically support the operations of commercial companies whose
> mission might or might not be aligned with our.
>
> Is monetizing such use cases really entirely unthinkable? Even under
> restrictions like the ones suggested by Andreas, or other such restrictions
> we should discuss?
> On Jan 16, 2016 3:49 PM, "Risker" <risker...@gmail.com> wrote:
>
> > Hmm.  The majority of those crawlers are from search engines - the very
> > search engines that keep us in the top 10 of their results (and often in
> > the top 3), thus leading to the usage and donations that we need to
> > survive. If they have to pay, then they might prefer to change their
> > algorithm, or reduce the frequency of scraping (thus also failing to
> catch
> > updates to articles including removal of vandalism in the lead
> paragraphs,
> > which is historically one of the key reasons for frequently crawling the
> > same articles).  Those crawlers are what attracts people to our sites, to
> > read, to make donations, to possibly edit.  Of course there are lesser
> > crawlers, but they're not really big players.
> >
> > I'm at a loss to understand why the Wikimedia Foundation should take on
> the
> > costs and indemnities associated with hiring staff to create a for-pay
> API
> > that would have to meet the expectations of a customer (or more than one
> > customer) that hasn't even agreed to pay for access.  If they want a
> > specialized API (and we've been given no evidence that they do), let THEM
> > hire the staff, pay them, write the code in an appropriately open-source
> > way, and donate it to the WMF with the understanding that it could be
> > modified as required, and that it will be accessible to everyone.
> >
> > It is good that the WMF has studied the usage patterns.  Could a link be
> > given to the report, please?  It's public, correct?  This is exactly the
> > point of transparency.  If only the WMF has the information, then it
> gives
> > an excuse for the community's comments to be ignored "because they don't
> > know the facts".  So let's lay out all the facts on the table, please.
> >
> > Risker/Anne
> >
> >
> >
> > On 16 January 2016 at 15:06, Vituzzu <vituzzu.w...@gmail.com> wrote:
> >
> > > Thank you for sharing this but, above all, to focus on digging real
> data.
> > >
> > > IMHO we shouldn't forget our mission, so licenses must be as free as
> > > possible. Turning into something "more closed" would definitely deplete
> > one
> > > of the most valuable source (the open source world) of volunteering we
> > have.
> > >
> > > Crawlers' owner should definitely share our increasing expenses but any
> > > kind of agreement with them should include ways to improve our
> userbase.
> > > I'm wondering about an agreement with Google (or any other player) to
> add
> > > an "edit" button to knowledge graph. Sort of a "knowledge vs. users"
> > > agreement.
> > >
> > > So, we definitely need a long term strategy which the Foundation will
> > > pursue in *negotiating* with anyone who wants a big scale access to
> *our
> > > resources* (while access to our knowledge will have no limits, as
> usual).
> > >
> > > Vito
> > >
> > >
> > > Il 16/01/2016 19:21, Lila Tretikov ha scritto:
> > >
> > >> To share some context of the discussion the board had around this -- I
> > >> don't think the minutes give enough detail. APIs -- as we are freely
> and
> > >> rapidly creating them today are important, but are not quite at the
> core
> > >> of
> > >> the issue we are facing.
> > >>
> > >> Today Wikimedia is the largest internet channel for open free
> knowledge
> > in
> > >> the world. But the trends are against us. We have to face them
> together.
> > >> We
> > >> have to have the answers on how. The strategic discussion next week
> will
> > >> help guide us.
> > >>
> > >> Over the last year we looked at the trends in Wikimedia traffic,
> > internet
> > >> as a whole and user behaviors. It took a lot of research. When we
> > started
> > >> the process we have not had solid internal data about unique visitors
> or
> > >> human vs. crawler usage on the site. For a top 10 website this is a
> big
> > >> issue; it hurts our ability to make smart decisions. We've learned a
> > lot.
> > >>
> > >> We found data that supports Leigh's point -- our permissive license
> > >> supports our core value, we are (I know I am) here for free knowledge.
> > Yet
> > >> it allows others to use the content in ways that truncates, simplifies
> > and
> > >> reduces it. More importantly this type of reuse separates our readers
> > from
> > >> our site, disconnecting readers from our contributors (no edit
> buttons)
> > >> and
> > >> ultimately reduces traffic. Is this a problem? I'd like to hear if
> > people
> > >> on this list see it as such. And how we sustain contributions over
> time.
> > >>
> > >> Meanwhile estimated half of our hosting is used to support crawlers
> that
> > >> scan our content. This has an associated cost in infrastructure,
> power,
> > >> servers, employees to support some well-funded organizations. The
> > content
> > >> is used for a variety of commercial purposes, sometimes having nothing
> > to
> > >> do with putting our contributor's work in front of more readers.
> Still,
> > we
> > >> can say this is tangentially supportive of our mission.
> > >>
> > >> As these two trends increase without our intervention, our traffic
> > decline
> > >> will accelerate, our ability to grow editors, content and cover costs
> > will
> > >> decline as well.
> > >>
> > >> The first question on the upcoming consultation next week will be
> > squarely
> > >> on this. Please help us. API conversation is a consequence of this
> > >> challenge. If we were to build more for reuse: APIs are a good way to
> do
> > >> so. If we are to somehow incentivize users of SIri to come back to
> > >> Wikipedia, what would we need to do? Should we improve our site so
> more
> > >> people come to us directly as the first stop? How do we bring people
> > into
> > >> our world vs. the world of commercial knowledge out there? How do we
> > fund
> > >> this if the people moved to access our content through other
> interfaces
> > (a
> > >> trend that has been accelerating)?
> > >>
> > >> Those are the core questions we need to face. We will have to have
> some
> > >> uncomfortable, honest discussions as we test our hypothesis this year.
> > The
> > >> conversation next week is a good start to prioritize those. Please
> join
> > >> it.
> > >>
> > >> Lila
> > >>
> > >>
> > >>
> > >> On Sat, Jan 16, 2016 at 6:11 AM, Leigh Thelmadatter <
> > osama...@hotmail.com
> > >> >
> > >> wrote:
> > >>
> > >> If we are concerned about Google taking unfair advantage of Wikipedia,
> > one
> > >>> simple solution is to allow content donations with a non-commercial
> > >>> restriction. Right now, the concept of "free" include commercial use.
> > An
> > >>> added bonus to this is that we would get a lot more institutional
> > >>> donations
> > >>> of content if we allowed an non-commercial option.
> > >>> My problem with allowing for paying for "premium access" is that we
> are
> > >>> allowing Google to have a priviledged position.  There is no way
> around
> > >>> that.
> > >>> What is the impetus behind this proposal? Its not like we are lacking
> > >>> money.  And limiting growth of the Foundation is not a bad thing...
> at
> > >>> least not to the community.
> > >>>
> > >>>
> > >>> To: wikimedia-l@lists.wikimedia.org
> > >>>> From: ricordisa...@openmailbox.org
> > >>>> Date: Sat, 16 Jan 2016 14:13:06 +0100
> > >>>> Subject: Re: [Wikimedia-l] Monetizing Wikimedia APIs
> > >>>>
> > >>>> "Imagine a world in which every single human being can freemiumly
> > share
> > >>>> in the sum of all knowledge." XD
> > >>>>
> > >>>> Il 16/01/2016 10:23, Pete Forsyth ha scritto:
> > >>>>
> > >>>>> I'm interested to hear some perspectives on the following line of
> > >>>>>
> > >>>> thinking:
> > >>>
> > >>>> Lisa presented some alternative strategies for revenue needs for the
> > >>>>> Foundation, including the possibility of charging for premium
> access
> > >>>>>
> > >>>> to the
> > >>>
> > >>>> services and APIs, expanding major donor and foundation fundraising,
> > >>>>> providing specific services for a fee, or limiting the Wikimedia
> > >>>>> Foundation's growth. The Board emphasized the importance of keeping
> > >>>>>
> > >>>> free
> > >>>
> > >>>> access to the existing APIs and services, keeping operational growth
> > in
> > >>>>> line with the organization's effectiveness, providing room for
> > >>>>>
> > >>>> innovation
> > >>>
> > >>>> in the Foundation's activities, and other potential fundraising
> > >>>>>
> > >>>> strategies.
> > >>>
> > >>>> The Board asked Lila to analyze and develop some of these potential
> > >>>>> strategies for further discussion at a Board meeting in 2016.
> > >>>>> Source: https://wikimediafoundation.org/wiki/Minutes/2015-11-07
> > >>>>> -Pete[[User:Peteforsyth]]
> > >>>>> _______________________________________________
> > >>>>> Wikimedia-l mailing list, guidelines at:
> > >>>>>
> > >>>> https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines
> > >>>
> > >>>> New messages to: Wikimedia-l@lists.wikimedia.org
> > >>>>> Unsubscribe:
> > https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
> > >>>>>
> > >>>> <mailto:wikimedia-l-requ...@lists.wikimedia.org
> ?subject=unsubscribe>
> > >>>
> > >>>>
> > >>>> _______________________________________________
> > >>>> Wikimedia-l mailing list, guidelines at:
> > >>>>
> > >>> https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines
> > >>>
> > >>>> New messages to: Wikimedia-l@lists.wikimedia.org
> > >>>> Unsubscribe:
> https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
> > ,
> > >>>>
> > >>> <mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe>
> > >>>
> > >>> _______________________________________________
> > >>> Wikimedia-l mailing list, guidelines at:
> > >>> https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines
> > >>> New messages to: Wikimedia-l@lists.wikimedia.org
> > >>> Unsubscribe:
> https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
> > >>> <mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe>
> > >>>
> > >>>
> > >>
> > >>
> > >
> > > _______________________________________________
> > > Wikimedia-l mailing list, guidelines at:
> > > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines
> > > New messages to: Wikimedia-l@lists.wikimedia.org
> > > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
> > > <mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe>
> > >
> > _______________________________________________
> > Wikimedia-l mailing list, guidelines at:
> > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines
> > New messages to: Wikimedia-l@lists.wikimedia.org
> > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
> > <mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe>
> _______________________________________________
> Wikimedia-l mailing list, guidelines at:
> https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines
> New messages to: Wikimedia-l@lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
> <mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe>
>
_______________________________________________
Wikimedia-l mailing list, guidelines at: 
https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines
New messages to: Wikimedia-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, 
<mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe>

Reply via email to