Re: [Wikidata] References to newspaper articles behind paywalls like newspapers.com

[Houcemeddine A. Turki]

Dear all,
I thank you for your answer. I highly recommend you to see 
https://meta.wikimedia.org/wiki/Talk:Wikidata/Strategy/2019#Feedback_and_questions_for_the_Wikidata_as_a_platform_paper
 that deals in part with this important topic. The problem is that we are not 
sure when using a newspaper article if this article is neutral or not. This 
fact caused a lot of significant problems in Wikipedia and we do not want to 
reproduce the same thing in Wikidata.
Yours Sincerely,
Houcemeddine Turki

De : Wikidata  de la part de Samuel Klein 

Envoyé : mercredi 13 novembre 2019 14:21
À : Discussion list for the Wikidata project 
Cc : Mark Graham 
Objet : Re: [Wikidata] References to newspaper articles behind paywalls like 
newspapers.com

On Wed, Nov 13, 2019 at 4:07 AM Federico Leva (Nemo) 
mailto:nemow...@gmail.com>> wrote:
Let's not spread publisher FUD on Wikimedia lists, please.

I was about to say the same thing...  else the lists will be takenn over by 
FUD/BCH wars
  "we may also keep in mind that 'limiting the availability, sourcing, and 
reusability of a claim' may not be sustainable reliability-wise, as sources 
that impose those limits are increasingly being challenged as biased, 
unaccountable, and unreliable."

Liam Wyatt, 13/11/19 02:03:
> is there any bot that is systematically going
> around and collecting new URLs that are added with the Reference URL
> property (P854), adding them to Internet Archive,

As long as MediaWiki is not broken, and the URLs are announced on the
expected venues (I believe
<https://wikitech.wikimedia.org/wiki/Event_Platform/EventStreams#API>),
yes, the Internet Archive immediately crawls them. This is documented at
<https://www.mediawiki.org/wiki/Archived_Pages> (sort of).

How complete is the coverage now?  /S
___
Wikidata mailing list
Wikidata@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata

Re: [Wikidata] References to newspaper articles behind paywalls like newspapers.com

[Samuel Klein]

On Wed, Nov 13, 2019 at 4:07 AM Federico Leva (Nemo) 
wrote:

> Let's not spread publisher FUD on Wikimedia lists, please.
>

I was about to say the same thing...  else the lists will be takenn over by
FUD/BCH wars
  "we may also keep in mind that 'limiting the availability, sourcing, and
reusability of a claim' may not be sustainable reliability-wise, as sources
that impose those limits are increasingly being challenged as biased,
unaccountable, and unreliable."


> Liam Wyatt, 13/11/19 02:03:
> > is there any bot that is systematically going
> > around and collecting new URLs that are added with the Reference URL
> > property (P854), adding them to Internet Archive,
>
> As long as MediaWiki is not broken, and the URLs are announced on the
> expected venues (I believe
> ),
> yes, the Internet Archive immediately crawls them. This is documented at
>  (sort of).
>

How complete is the coverage now?  /S
___
Wikidata mailing list
Wikidata@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata

Re: [Wikidata] References to newspaper articles behind paywalls like newspapers.com

[Federico Leva (Nemo)]

Let's not spread publisher FUD on Wikimedia lists, please.

Liam Wyatt, 13/11/19 02:03:

is there any bot that is systematically going
around and collecting new URLs that are added with the Reference URL
property (P854), adding them to Internet Archive,


As long as MediaWiki is not broken, and the URLs are announced on the 
expected venues (I believe 
), 
yes, the Internet Archive immediately crawls them. This is documented at 
 (sort of).


Federico

___
Wikidata mailing list
Wikidata@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata

Re: [Wikidata] References to newspaper articles behind paywalls like newspapers.com

[Lucas Werkmeister]

I believe that’s what InternetArchiveBot
 is for, but
apparently at the moment it only updates references when they break; at
least, that’s how Ymblanter summarized the request for permissions
.
Also, its contributions

show no edits since October 29, I’m not sure if that’s a problem.

Cheers,
Lucas

On 13.11.19 01:03, Liam Wyatt wrote:
> To guard against link-rot of news websites when they're used as
> Reference for statements in Wikidata, is there any bot that is
> systematically going around and collecting new URLs that are added
> with the Reference URL property (P854), adding them to Internet
> Archive, copying the result, and then adding that as Archive URL
> property (P1065) + Archive Date (P2960) back into the original
> statement? 
>
> If not, would that be feasible, and a good idea?
>
> -Liam 
>
> ___
> Wikidata mailing list
> Wikidata@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikidata
___
Wikidata mailing list
Wikidata@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata

Re: [Wikidata] References to newspaper articles behind paywalls like newspapers.com

[Liam Wyatt]

To guard against link-rot of news websites when they're used as Reference
for statements in Wikidata, is there any bot that is systematically going
around and collecting new URLs that are added with the Reference URL
property (P854), adding them to Internet Archive, copying the result, and
then adding that as Archive URL property (P1065) + Archive Date (P2960)
back into the original statement?

If not, would that be feasible, and a good idea?

-Liam
___
Wikidata mailing list
Wikidata@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata

Re: [Wikidata] References to newspaper articles behind paywalls like newspapers.com

[Gerard Meijssen]

Hoi,
As more and more conditions are piled up, bias is not reduced but induced.
Thanks,
  GerardM

On Tue, 12 Nov 2019 at 09:55,  wrote:

> I see a gradual shift from open news articles to articles behind
> paywall, that I guess must be a natural adjustment of the income model
> as newspapers transition from paper to the Internet. Good article,
> suitable for sources in Wikipedia and Wikidata, will probably more and
> more likely be behind paywalls. We need to get use to that and it has
> also been the case for books - to some degree - for a long time.
>
> The Internet Archive, tools like Andy Mabbett refers to and the
> 'archive-url' parameter for (the English) Wikipedia's cite news template
> as well as Wikidata's P1065 may be a good solution, but we may also keep
> in mind that it might not be sustainable copyright-wise in the long
> term, cf. discussions around controlled lending and Society of Authors'
> concerns
>
> https://publishingperspectives.com/2019/01/copyright-battle-internet-archives-open-library-authors-guild-society-of-authors/
> (not currently behind paywall :)
>
> /Finn
>
>
> On 12/11/2019 08:17, Gerard Meijssen wrote:
> > Hoi,
> > It depends. When you approach it from a scientific point of view, you
> > write your paper and are personally responsible for what you write. The
> > bias of your paper may include the effort you take to verify sources. In
> > Wikipedia it is NOT your paper and it is NOT only your responsibility.
> > At the opposite end are the papers in the language you do not understand
> > and consequently not accepted as a source, there are the papers, books
> > that have not been
> >
> > So a Wikipedia is biased because of the limiting of sources and as it is
> > NOT a personal responsibility, there is also the notion if we should
> > accept the bias that limiting brings.
> > Thanks,
> > GerardM
> >
> > On Tue, 12 Nov 2019 at 08:03, Andra Waagmeester  > > wrote:
> >
> > I don't if I agree with "just citing" the newspaper article. Why not
> > push for resolvable citations if we have the technology? There is
> > not much value in a citation if you can't access the source to
> > verify, don't you think?
> >
> > On Tue, Nov 12, 2019 at 6:38 AM Wynand van der Walt
> > mailto:wynli...@gmail.com>> wrote:
> >
> > I like what Andy is doing, As a librarian, however, an
> > alternative would be to only cite the newspaper without the URL
> > as this analog citation would be valid, meaning just cite the
> > newspaper article.
> >
> > Regards,
> >
> > Wynand van der Walt
> > Head Librarian: Technical Services
> > Rhodes University Library
> >
> >
> >
> > On Mon, Nov 11, 2019 at 7:28 PM Andy Mabbett
> > mailto:a...@pigsonthewing.org.uk>>
> > wrote:
> >
> > On Mon, 11 Nov 2019 at 16:44, PWN
> >  > > wrote:
> >
> >  > I’m constantly encountering newspaper articles that have
> > disappeared from Google and
> >  > are no longer viewable or even discoverable via Google.
> >  > They are often the sole reference url for statements, yet
> > are behind a paywall - notably
> >  > newspapers.com .
> >  > How is the community handling the paywalling of
> > historical newspaper resources?
> >
> > Whenever I cite something on Wikidata, or Wikipedia, I
> > submit a copy
> > to the Internet Archive's Wayback Machine using the add-on
> for
> > Firefox:
> >
> > https://github.com/jonathanmccann/archive-url-firefox-addon
> >
> > --
> > Andy Mabbett
> > @pigsonthewing
> > http://pigsonthewing.org.uk
> >
> > ___
> > Wikidata mailing list
> > Wikidata@lists.wikimedia.org
> > 
> > https://lists.wikimedia.org/mailman/listinfo/wikidata
> >
> > ___
> > Wikidata mailing list
> > Wikidata@lists.wikimedia.org  Wikidata@lists.wikimedia.org>
> > https://lists.wikimedia.org/mailman/listinfo/wikidata
> >
> > ___
> > Wikidata mailing list
> > Wikidata@lists.wikimedia.org 
> > https://lists.wikimedia.org/mailman/listinfo/wikidata
> >
> >
> > ___
> > Wikidata mailing list
> > Wikidata@lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/wikidata
> >
>
> ___
> Wikidata mailing list
> Wikidata@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikidata
>

Re: [Wikidata] References to newspaper articles behind paywalls like newspapers.com

[fn]

I see a gradual shift from open news articles to articles behind 
paywall, that I guess must be a natural adjustment of the income model 
as newspapers transition from paper to the Internet. Good article, 
suitable for sources in Wikipedia and Wikidata, will probably more and 
more likely be behind paywalls. We need to get use to that and it has 
also been the case for books - to some degree - for a long time.


The Internet Archive, tools like Andy Mabbett refers to and the 
'archive-url' parameter for (the English) Wikipedia's cite news template 
as well as Wikidata's P1065 may be a good solution, but we may also keep 
in mind that it might not be sustainable copyright-wise in the long 
term, cf. discussions around controlled lending and Society of Authors' 
concerns 
https://publishingperspectives.com/2019/01/copyright-battle-internet-archives-open-library-authors-guild-society-of-authors/ 
(not currently behind paywall :)


/Finn


On 12/11/2019 08:17, Gerard Meijssen wrote:

Hoi,
It depends. When you approach it from a scientific point of view, you 
write your paper and are personally responsible for what you write. The 
bias of your paper may include the effort you take to verify sources. In 
Wikipedia it is NOT your paper and it is NOT only your responsibility. 
At the opposite end are the papers in the language you do not understand 
and consequently not accepted as a source, there are the papers, books 
that have not been


So a Wikipedia is biased because of the limiting of sources and as it is 
NOT a personal responsibility, there is also the notion if we should 
accept the bias that limiting brings.

Thanks,
        GerardM

On Tue, 12 Nov 2019 at 08:03, Andra Waagmeester > wrote:


I don't if I agree with "just citing" the newspaper article. Why not
push for resolvable citations if we have the technology? There is
not much value in a citation if you can't access the source to
verify, don't you think?

On Tue, Nov 12, 2019 at 6:38 AM Wynand van der Walt
mailto:wynli...@gmail.com>> wrote:

I like what Andy is doing, As a librarian, however, an
alternative would be to only cite the newspaper without the URL
as this analog citation would be valid, meaning just cite the
newspaper article.

Regards,

Wynand van der Walt
Head Librarian: Technical Services
Rhodes University Library



On Mon, Nov 11, 2019 at 7:28 PM Andy Mabbett
mailto:a...@pigsonthewing.org.uk>>
wrote:

On Mon, 11 Nov 2019 at 16:44, PWN
mailto:pariswritersn...@gmail.com>> wrote:

 > I’m constantly encountering newspaper articles that have
disappeared from Google and
 > are no longer viewable or even discoverable via Google.
 > They are often the sole reference url for statements, yet
are behind a paywall - notably
 > newspapers.com .
 > How is the community handling the paywalling of
historical newspaper resources?

Whenever I cite something on Wikidata, or Wikipedia, I
submit a copy
to the Internet Archive's Wayback Machine using the add-on for
Firefox:

https://github.com/jonathanmccann/archive-url-firefox-addon

-- 
Andy Mabbett

@pigsonthewing
http://pigsonthewing.org.uk

___
Wikidata mailing list
Wikidata@lists.wikimedia.org

https://lists.wikimedia.org/mailman/listinfo/wikidata

___
Wikidata mailing list
Wikidata@lists.wikimedia.org 
https://lists.wikimedia.org/mailman/listinfo/wikidata

___
Wikidata mailing list
Wikidata@lists.wikimedia.org 
https://lists.wikimedia.org/mailman/listinfo/wikidata


___
Wikidata mailing list
Wikidata@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata



___
Wikidata mailing list
Wikidata@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata

Re: [Wikidata] References to newspaper articles behind paywalls like newspapers.com

[Gerard Meijssen]

Hoi,
It depends. When you approach it from a scientific point of view, you write
your paper and are personally responsible for what you write. The bias of
your paper may include the effort you take to verify sources. In Wikipedia
it is NOT your paper and it is NOT only your responsibility. At the
opposite end are the papers in the language you do not understand and
consequently not accepted as a source, there are the papers, books that
have not been

So a Wikipedia is biased because of the limiting of sources and as it is
NOT a personal responsibility, there is also the notion if we should accept
the bias that limiting brings.
Thanks,
   GerardM

On Tue, 12 Nov 2019 at 08:03, Andra Waagmeester  wrote:

> I don't if I agree with "just citing" the newspaper article. Why not push
> for resolvable citations if we have the technology? There is not much value
> in a citation if you can't access the source to verify, don't you think?
>
> On Tue, Nov 12, 2019 at 6:38 AM Wynand van der Walt 
> wrote:
>
>> I like what Andy is doing, As a librarian, however, an alternative would
>> be to only cite the newspaper without the URL as this analog citation would
>> be valid, meaning just cite the newspaper article.
>>
>> Regards,
>>
>> Wynand van der Walt
>> Head Librarian: Technical Services
>> Rhodes University Library
>>
>>
>> On Mon, Nov 11, 2019 at 7:28 PM Andy Mabbett 
>> wrote:
>>
>>> On Mon, 11 Nov 2019 at 16:44, PWN  wrote:
>>>
>>> > I’m constantly encountering newspaper articles that have disappeared
>>> from Google and
>>> > are no longer viewable or even discoverable via Google.
>>> > They are often the sole reference url for statements, yet are behind a
>>> paywall - notably
>>> > newspapers.com.
>>> > How is the community handling the paywalling of historical newspaper
>>> resources?
>>>
>>> Whenever I cite something on Wikidata, or Wikipedia, I submit a copy
>>> to the Internet Archive's Wayback Machine using the add-on for
>>> Firefox:
>>>
>>>https://github.com/jonathanmccann/archive-url-firefox-addon
>>>
>>> --
>>> Andy Mabbett
>>> @pigsonthewing
>>> http://pigsonthewing.org.uk
>>>
>>> ___
>>> Wikidata mailing list
>>> Wikidata@lists.wikimedia.org
>>> https://lists.wikimedia.org/mailman/listinfo/wikidata
>>>
>> ___
>> Wikidata mailing list
>> Wikidata@lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/wikidata
>>
> ___
> Wikidata mailing list
> Wikidata@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikidata
>
___
Wikidata mailing list
Wikidata@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata

Re: [Wikidata] References to newspaper articles behind paywalls like newspapers.com

[Andra Waagmeester]

I don't if I agree with "just citing" the newspaper article. Why not push
for resolvable citations if we have the technology? There is not much value
in a citation if you can't access the source to verify, don't you think?

On Tue, Nov 12, 2019 at 6:38 AM Wynand van der Walt 
wrote:

> I like what Andy is doing, As a librarian, however, an alternative would
> be to only cite the newspaper without the URL as this analog citation would
> be valid, meaning just cite the newspaper article.
>
> Regards,
>
> Wynand van der Walt
> Head Librarian: Technical Services
> Rhodes University Library
>
>
> On Mon, Nov 11, 2019 at 7:28 PM Andy Mabbett 
> wrote:
>
>> On Mon, 11 Nov 2019 at 16:44, PWN  wrote:
>>
>> > I’m constantly encountering newspaper articles that have disappeared
>> from Google and
>> > are no longer viewable or even discoverable via Google.
>> > They are often the sole reference url for statements, yet are behind a
>> paywall - notably
>> > newspapers.com.
>> > How is the community handling the paywalling of historical newspaper
>> resources?
>>
>> Whenever I cite something on Wikidata, or Wikipedia, I submit a copy
>> to the Internet Archive's Wayback Machine using the add-on for
>> Firefox:
>>
>>https://github.com/jonathanmccann/archive-url-firefox-addon
>>
>> --
>> Andy Mabbett
>> @pigsonthewing
>> http://pigsonthewing.org.uk
>>
>> ___
>> Wikidata mailing list
>> Wikidata@lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/wikidata
>>
> ___
> Wikidata mailing list
> Wikidata@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikidata
>
___
Wikidata mailing list
Wikidata@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata

Re: [Wikidata] References to newspaper articles behind paywalls like newspapers.com

[Wynand van der Walt]

I like what Andy is doing, As a librarian, however, an alternative would be
to only cite the newspaper without the URL as this analog citation would be
valid, meaning just cite the newspaper article.

Regards,

Wynand van der Walt
Head Librarian: Technical Services
Rhodes University Library


On Mon, Nov 11, 2019 at 7:28 PM Andy Mabbett 
wrote:

> On Mon, 11 Nov 2019 at 16:44, PWN  wrote:
>
> > I’m constantly encountering newspaper articles that have disappeared
> from Google and
> > are no longer viewable or even discoverable via Google.
> > They are often the sole reference url for statements, yet are behind a
> paywall - notably
> > newspapers.com.
> > How is the community handling the paywalling of historical newspaper
> resources?
>
> Whenever I cite something on Wikidata, or Wikipedia, I submit a copy
> to the Internet Archive's Wayback Machine using the add-on for
> Firefox:
>
>https://github.com/jonathanmccann/archive-url-firefox-addon
>
> --
> Andy Mabbett
> @pigsonthewing
> http://pigsonthewing.org.uk
>
> ___
> Wikidata mailing list
> Wikidata@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikidata
>
___
Wikidata mailing list
Wikidata@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata

Re: [Wikidata] References to newspaper articles behind paywalls like newspapers.com

[Andy Mabbett]

On Mon, 11 Nov 2019 at 16:44, PWN  wrote:

> I’m constantly encountering newspaper articles that have disappeared from 
> Google and
> are no longer viewable or even discoverable via Google.
> They are often the sole reference url for statements, yet are behind a 
> paywall - notably
> newspapers.com.
> How is the community handling the paywalling of historical newspaper 
> resources?

Whenever I cite something on Wikidata, or Wikipedia, I submit a copy
to the Internet Archive's Wayback Machine using the add-on for
Firefox:

   https://github.com/jonathanmccann/archive-url-firefox-addon

-- 
Andy Mabbett
@pigsonthewing
http://pigsonthewing.org.uk

___
Wikidata mailing list
Wikidata@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata