Re: [Wikitech-l] Template transclusion on Wikimedia Commons
Also several times I got some sort of internal database errors when using
Special:WhatLinksHere.
That shouldn't happen. What sort of things were you doing when the error
happened?
I found how to reproduce the database errors I mentioned earlier. Both
https://commons.wikimedia.org/w/index.php?title=Special%3AWhatLinksHeretarget=Template%3ALocation%2Flayoutnamespace=10
and
https://commons.wikimedia.org/w/index.php?title=Special%3AWhatLinksHeretarget=Module%3AFallbacklistnamespace=828
gave me
Database error
A database query error has occurred. This may indicate a bug in the software.
Function: SpecialWhatLinksHere::showIndirectLinks
Error: 0
Jarek T.
(user:jarekt http://commons.wikimedia.org/wiki/User:Jarekt )
-Original Message-
From: wikitech-l-boun...@lists.wikimedia.org
[mailto:wikitech-l-boun...@lists.wikimedia.org] On Behalf Of bawolff
Sent: Monday, January 27, 2014 6:50 PM
To: Wikimedia developers
Subject: Re: [Wikitech-l] Template transclusion on Wikimedia Commons
On Mon, Jan 27, 2014 at 2:12 PM, Tuszynski, Jaroslaw W.
jaroslaw.w.tuszyn...@leidos.com mailto:jaroslaw.w.tuszyn...@leidos.com
wrote:
I lately switched Commons coordinate template
https://commons.wikimedia.org/wiki/Template:Location
https://commons.wikimedia.org/wiki/Template:Location from using dozens
of subtemplates to much cleaner Lua code. Everything works great,
except for Special:WhatLinksHere links. The no longer used
subtemplates like
https://commons.wikimedia.org/wiki/Template:Location/layout
https://commons.wikimedia.org/wiki/Template:Location/layout still show
over 3M files transcluding it and files, like
https://commons.wikimedia.org/wiki/File:Estadio_Pacaembu.jpg
https://commons.wikimedia.org/wiki/File:Estadio_Pacaembu.jpg which
still claims to transclude [[Template:Location/layout]] (Templates
used on this page: section in edit mode). The transclusion
dependencies do not correct themselves even with a Purge, but page
edit fixes it. Also several times I got some sort of internal database errors
when using Special:WhatLinksHere.
In the past when I run into this kind of trouble I waited, but in some
cases it took over half a year for the database to update itself. In
other cases those issues never clear (or I did not wait long enough),
like with Category:Pages with malformed coordinate tags
https://commons.wikimedia.org/wiki/Category:Pages_with_malformed_coor
di nate_tags , which due to a long fixed bug in {{#coordinates:
https://www.mediawiki.org/wiki/Extension:GeoData
https://www.mediawiki.org/wiki/Extension:GeoData }} was filled with
40k files which did not have any issues. Then the only solution I know
is Pywikibot/touch.py
(https://www.mediawiki.org/wiki/Manual:Pywikipediabot/touch.py
https://www.mediawiki.org/wiki/Manual:Pywikipediabot/touch.py ),
which I recently used to clear {{#coordinates:
https://www.mediawiki.org/wiki/Extension:GeoData
https://www.mediawiki.org/wiki/Extension:GeoData }} issues.
Is there a better way to synchronize database with reality?
Jarek T.
(user:jarekt http://commons.wikimedia.org/wiki/User:Jarekt
http://commons.wikimedia.org/wiki/User:Jarekt )
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org mailto:Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
It takes a lot of work to reparse 4 million pages. If it makes you feel better,
the toolserver transclusion count you linked to is about
2 pages higher than the real number of transclusions.
but in some
cases it took over half a year for the database to update itself
If something is taking 6 months, it probably means that some sort of bug
happened, and the pages aren't being refreshed at all.
The transclusion dependencies do
not correct themselves even with a Purge, but page edit fixes it.
Yes, null edits update links tables (e.g. Categories, whatlinkshere,
Special:LinkSearch, etc) purges do not.
Also
several times I got some sort of internal database errors when using
Special:WhatLinksHere.
That shouldn't happen. What sort of things were you doing when the error
happened?
--bawolff
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org mailto:Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Template transclusion on Wikimedia Commons
Le 30/01/14 14:39, Tuszynski, Jaroslaw W. a écrit : I found how to reproduce the database errors I mentioned earlier. Both https://commons.wikimedia.org/w/index.php?title=Special%3AWhatLinksHeretarget=Template%3ALocation%2Flayoutnamespace=10 and https://commons.wikimedia.org/w/index.php?title=Special%3AWhatLinksHeretarget=Module%3AFallbacklistnamespace=828 gave me Database error A database query error has occurred. This may indicate a bug in the software. Function: SpecialWhatLinksHere::showIndirectLinks Error: 0 Hello, The first URL worked for me, the second throws a database error. Thu Jan 30 14:11:19 UTC 2014 /* query */ Thu Jan 30 14:19:33 UTC 2014 /* query */ Connection lost and reconnected after 59.754s Can you please fill a bug about it? -- Antoine hashar Musso ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Template transclusion on Wikimedia Commons
Done. See https://bugzilla.wikimedia.org/show_bug.cgi?id=60618 Jarek T. (user:jarekt) -Original Message- From: wikitech-l-boun...@lists.wikimedia.org [mailto:wikitech-l-boun...@lists.wikimedia.org] On Behalf Of Antoine Musso Sent: Thursday, January 30, 2014 9:23 AM To: wikitech-l@lists.wikimedia.org Subject: Re: [Wikitech-l] Template transclusion on Wikimedia Commons Le 30/01/14 14:39, Tuszynski, Jaroslaw W. a écrit : I found how to reproduce the database errors I mentioned earlier. Both https://commons.wikimedia.org/w/index.php?title=Special%3AWhatLinksHer etarget=Template%3ALocation%2Flayoutnamespace=10 and https://commons.wikimedia.org/w/index.php?title=Special%3AWhatLinksHer etarget=Module%3AFallbacklistnamespace=828 gave me Database error A database query error has occurred. This may indicate a bug in the software. Function: SpecialWhatLinksHere::showIndirectLinks Error: 0 Hello, The first URL worked for me, the second throws a database error. Thu Jan 30 14:11:19 UTC 2014 /* query */ Thu Jan 30 14:19:33 UTC 2014 /* query */ Connection lost and reconnected after 59.754s Can you please fill a bug about it? -- Antoine hashar Musso ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Template transclusion on Wikimedia Commons
On Tue, 28 Jan 2014 17:04:23 +0100, Tuszynski, Jaroslaw W. jaroslaw.w.tuszyn...@leidos.com wrote: Also I am curious, is there some sort of process that refreshes occasionally all pages that were not refreshed for some period of time? In a way: as far as I know, pages are cached for no more than 30 days – if anybody (anonymous or registered) tries to view a page that was last viewed more than 30 days ago, it's parsed and generated from scratch for them. This doesn't happen if the page is not viewed, thought. (I might be wrong, in that case hopefully someone will correct me.) -- Matma Rex ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Template transclusion on Wikimedia Commons
But I assume that is equivalent to a Purge and does not trigger update of the links tables. Jarek -Original Message- From: wikitech-l-boun...@lists.wikimedia.org [mailto:wikitech-l-boun...@lists.wikimedia.org] On Behalf Of Bartosz Dziewonski Sent: Wednesday, January 29, 2014 9:52 AM To: Wikimedia developers Subject: Re: [Wikitech-l] Template transclusion on Wikimedia Commons On Tue, 28 Jan 2014 17:04:23 +0100, Tuszynski, Jaroslaw W. jaroslaw.w.tuszyn...@leidos.com wrote: Also I am curious, is there some sort of process that refreshes occasionally all pages that were not refreshed for some period of time? In a way: as far as I know, pages are cached for no more than 30 days – if anybody (anonymous or registered) tries to view a page that was last viewed more than 30 days ago, it's parsed and generated from scratch for them. This doesn't happen if the page is not viewed, thought. (I might be wrong, in that case hopefully someone will correct me.) -- Matma Rex ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Template transclusion on Wikimedia Commons
Yes that is correct. A parser cache expiry does does not update the links tables. -bawolff On Jan 29, 2014 11:03 AM, Tuszynski, Jaroslaw W. jaroslaw.w.tuszyn...@leidos.com wrote: But I assume that is equivalent to a Purge and does not trigger update of the links tables. Jarek -Original Message- From: wikitech-l-boun...@lists.wikimedia.org [mailto: wikitech-l-boun...@lists.wikimedia.org] On Behalf Of Bartosz Dziewonski Sent: Wednesday, January 29, 2014 9:52 AM To: Wikimedia developers Subject: Re: [Wikitech-l] Template transclusion on Wikimedia Commons On Tue, 28 Jan 2014 17:04:23 +0100, Tuszynski, Jaroslaw W. jaroslaw.w.tuszyn...@leidos.com wrote: Also I am curious, is there some sort of process that refreshes occasionally all pages that were not refreshed for some period of time? In a way: as far as I know, pages are cached for no more than 30 days - if anybody (anonymous or registered) tries to view a page that was last viewed more than 30 days ago, it's parsed and generated from scratch for them. This doesn't happen if the page is not viewed, thought. (I might be wrong, in that case hopefully someone will correct me.) -- Matma Rex ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Template transclusion on Wikimedia Commons
So I guess everything works as expected, it just takes a long time.
I guess I do not understand the logic of what triggers the update of the links
tables (e.g. Categories, whatlinkshere, Special:LinkSearch, etc) of pages:
purge does not, null edit does, but how about edits to transcluded templates or
modules? Based on many years of working with Commons high-use templates, it
seems like in case of templates with few uses, if changes to the template
changes links tables for pages transcluding it than those changes are done in a
timely manner. However, high-use templates (which up to about a year ago always
showed an error message after each save), are either much slower or might not
be triggering the update of the links tables at all. It also seems to depend on
how deep on the call-stack is the template which was edited. Finally if the
update of the links tables is due to changes in extensions, like
Extension:GeoData, etc. than changes to the extension code do not seem to
trigger the updates at all.
BTW, the internal database errors when using Special:WhatLinksHere do not
seem to be repeatable. They just happen occasionally.
Also I am curious, is there some sort of process that refreshes occasionally
all pages that were not refreshed for some period of time?
Jarek T.
(user:jarekt)
-Original Message-
From: wikitech-l-boun...@lists.wikimedia.org
[mailto:wikitech-l-boun...@lists.wikimedia.org] On Behalf Of bawolff
Sent: Monday, January 27, 2014 6:50 PM
To: Wikimedia developers
Subject: Re: [Wikitech-l] Template transclusion on Wikimedia Commons
On Mon, Jan 27, 2014 at 2:12 PM, Tuszynski, Jaroslaw W.
jaroslaw.w.tuszyn...@leidos.com wrote:
I lately switched Commons coordinate template
https://commons.wikimedia.org/wiki/Template:Location from using dozens
of subtemplates to much cleaner Lua code. Everything works great,
except for Special:WhatLinksHere links. The no longer used
subtemplates like
https://commons.wikimedia.org/wiki/Template:Location/layout still show
over 3M files transcluding it and files, like
https://commons.wikimedia.org/wiki/File:Estadio_Pacaembu.jpg which
still claims to transclude [[Template:Location/layout]] (Templates
used on this page: section in edit mode). The transclusion
dependencies do not correct themselves even with a Purge, but page
edit fixes it. Also several times I got some sort of internal database errors
when using Special:WhatLinksHere.
In the past when I run into this kind of trouble I waited, but in some
cases it took over half a year for the database to update itself. In
other cases those issues never clear (or I did not wait long enough),
like with Category:Pages with malformed coordinate tags
https://commons.wikimedia.org/wiki/Category:Pages_with_malformed_coor
di nate_tags , which due to a long fixed bug in {{#coordinates:
https://www.mediawiki.org/wiki/Extension:GeoData }} was filled with
40k files which did not have any issues. Then the only solution I know
is Pywikibot/touch.py
(https://www.mediawiki.org/wiki/Manual:Pywikipediabot/touch.py ),
which I recently used to clear {{#coordinates:
https://www.mediawiki.org/wiki/Extension:GeoData }} issues.
Is there a better way to synchronize database with reality?
Jarek T.
(user:jarekt http://commons.wikimedia.org/wiki/User:Jarekt )
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
It takes a lot of work to reparse 4 million pages. If it makes you feel better,
the toolserver transclusion count you linked to is about
2 pages higher than the real number of transclusions.
but in some
cases it took over half a year for the database to update itself
If something is taking 6 months, it probably means that some sort of bug
happened, and the pages aren't being refreshed at all.
The transclusion dependencies do
not correct themselves even with a Purge, but page edit fixes it.
Yes, null edits update links tables (e.g. Categories, whatlinkshere,
Special:LinkSearch, etc) purges do not.
Also
several times I got some sort of internal database errors when using
Special:WhatLinksHere.
That shouldn't happen. What sort of things were you doing when the error
happened?
--bawolff
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] Template transclusion on Wikimedia Commons
I lately switched Commons coordinate template
https://commons.wikimedia.org/wiki/Template:Location from using dozens
of subtemplates to much cleaner Lua code. Everything works great, except
for Special:WhatLinksHere links. The no longer used subtemplates like
https://commons.wikimedia.org/wiki/Template:Location/layout still show
over 3M files transcluding it and files, like
https://commons.wikimedia.org/wiki/File:Estadio_Pacaembu.jpg which
still claims to transclude [[Template:Location/layout]] (Templates used
on this page: section in edit mode). The transclusion dependencies do
not correct themselves even with a Purge, but page edit fixes it. Also
several times I got some sort of internal database errors when using
Special:WhatLinksHere.
In the past when I run into this kind of trouble I waited, but in some
cases it took over half a year for the database to update itself. In
other cases those issues never clear (or I did not wait long enough),
like with Category:Pages with malformed coordinate tags
https://commons.wikimedia.org/wiki/Category:Pages_with_malformed_coordi
nate_tags , which due to a long fixed bug in {{#coordinates:
https://www.mediawiki.org/wiki/Extension:GeoData }} was filled with
40k files which did not have any issues. Then the only solution I know
is Pywikibot/touch.py
(https://www.mediawiki.org/wiki/Manual:Pywikipediabot/touch.py ), which
I recently used to clear {{#coordinates:
https://www.mediawiki.org/wiki/Extension:GeoData }} issues.
Is there a better way to synchronize database with reality?
Jarek T.
(user:jarekt http://commons.wikimedia.org/wiki/User:Jarekt )
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Template transclusion on Wikimedia Commons
On Mon, Jan 27, 2014 at 2:12 PM, Tuszynski, Jaroslaw W.
jaroslaw.w.tuszyn...@leidos.com wrote:
I lately switched Commons coordinate template
https://commons.wikimedia.org/wiki/Template:Location from using dozens
of subtemplates to much cleaner Lua code. Everything works great, except
for Special:WhatLinksHere links. The no longer used subtemplates like
https://commons.wikimedia.org/wiki/Template:Location/layout still show
over 3M files transcluding it and files, like
https://commons.wikimedia.org/wiki/File:Estadio_Pacaembu.jpg which
still claims to transclude [[Template:Location/layout]] (Templates used
on this page: section in edit mode). The transclusion dependencies do
not correct themselves even with a Purge, but page edit fixes it. Also
several times I got some sort of internal database errors when using
Special:WhatLinksHere.
In the past when I run into this kind of trouble I waited, but in some
cases it took over half a year for the database to update itself. In
other cases those issues never clear (or I did not wait long enough),
like with Category:Pages with malformed coordinate tags
https://commons.wikimedia.org/wiki/Category:Pages_with_malformed_coordi
nate_tags , which due to a long fixed bug in {{#coordinates:
https://www.mediawiki.org/wiki/Extension:GeoData }} was filled with
40k files which did not have any issues. Then the only solution I know
is Pywikibot/touch.py
(https://www.mediawiki.org/wiki/Manual:Pywikipediabot/touch.py ), which
I recently used to clear {{#coordinates:
https://www.mediawiki.org/wiki/Extension:GeoData }} issues.
Is there a better way to synchronize database with reality?
Jarek T.
(user:jarekt http://commons.wikimedia.org/wiki/User:Jarekt )
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
It takes a lot of work to reparse 4 million pages. If it makes you
feel better, the toolserver transclusion count you linked to is about
2 pages higher than the real number of transclusions.
but in some
cases it took over half a year for the database to update itself
If something is taking 6 months, it probably means that some sort of
bug happened, and the pages aren't being refreshed at all.
The transclusion dependencies do
not correct themselves even with a Purge, but page edit fixes it.
Yes, null edits update links tables (e.g. Categories, whatlinkshere,
Special:LinkSearch, etc) purges do not.
Also
several times I got some sort of internal database errors when using
Special:WhatLinksHere.
That shouldn't happen. What sort of things were you doing when the
error happened?
--bawolff
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
