Re: [Wikitech-l] Template transclusion on Wikimedia Commons
Also several times I got some sort of internal database errors when using
Special:WhatLinksHere.
That shouldn't happen. What sort of things were you doing when the error
happened?
I found how to reproduce the database errors I mentioned earlier. Both
https://commons.wikimedia.org/w/index.php?title=Special%3AWhatLinksHeretarget=Template%3ALocation%2Flayoutnamespace=10
and
https://commons.wikimedia.org/w/index.php?title=Special%3AWhatLinksHeretarget=Module%3AFallbacklistnamespace=828
gave me
Database error
A database query error has occurred. This may indicate a bug in the software.
Function: SpecialWhatLinksHere::showIndirectLinks
Error: 0
Jarek T.
(user:jarekt http://commons.wikimedia.org/wiki/User:Jarekt )
-Original Message-
From: wikitech-l-boun...@lists.wikimedia.org
[mailto:wikitech-l-boun...@lists.wikimedia.org] On Behalf Of bawolff
Sent: Monday, January 27, 2014 6:50 PM
To: Wikimedia developers
Subject: Re: [Wikitech-l] Template transclusion on Wikimedia Commons
On Mon, Jan 27, 2014 at 2:12 PM, Tuszynski, Jaroslaw W.
jaroslaw.w.tuszyn...@leidos.com mailto:jaroslaw.w.tuszyn...@leidos.com
wrote:
I lately switched Commons coordinate template
https://commons.wikimedia.org/wiki/Template:Location
https://commons.wikimedia.org/wiki/Template:Location from using dozens
of subtemplates to much cleaner Lua code. Everything works great,
except for Special:WhatLinksHere links. The no longer used
subtemplates like
https://commons.wikimedia.org/wiki/Template:Location/layout
https://commons.wikimedia.org/wiki/Template:Location/layout still show
over 3M files transcluding it and files, like
https://commons.wikimedia.org/wiki/File:Estadio_Pacaembu.jpg
https://commons.wikimedia.org/wiki/File:Estadio_Pacaembu.jpg which
still claims to transclude [[Template:Location/layout]] (Templates
used on this page: section in edit mode). The transclusion
dependencies do not correct themselves even with a Purge, but page
edit fixes it. Also several times I got some sort of internal database errors
when using Special:WhatLinksHere.
In the past when I run into this kind of trouble I waited, but in some
cases it took over half a year for the database to update itself. In
other cases those issues never clear (or I did not wait long enough),
like with Category:Pages with malformed coordinate tags
https://commons.wikimedia.org/wiki/Category:Pages_with_malformed_coor
di nate_tags , which due to a long fixed bug in {{#coordinates:
https://www.mediawiki.org/wiki/Extension:GeoData
https://www.mediawiki.org/wiki/Extension:GeoData }} was filled with
40k files which did not have any issues. Then the only solution I know
is Pywikibot/touch.py
(https://www.mediawiki.org/wiki/Manual:Pywikipediabot/touch.py
https://www.mediawiki.org/wiki/Manual:Pywikipediabot/touch.py ),
which I recently used to clear {{#coordinates:
https://www.mediawiki.org/wiki/Extension:GeoData
https://www.mediawiki.org/wiki/Extension:GeoData }} issues.
Is there a better way to synchronize database with reality?
Jarek T.
(user:jarekt http://commons.wikimedia.org/wiki/User:Jarekt
http://commons.wikimedia.org/wiki/User:Jarekt )
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org mailto:Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
It takes a lot of work to reparse 4 million pages. If it makes you feel better,
the toolserver transclusion count you linked to is about
2 pages higher than the real number of transclusions.
but in some
cases it took over half a year for the database to update itself
If something is taking 6 months, it probably means that some sort of bug
happened, and the pages aren't being refreshed at all.
The transclusion dependencies do
not correct themselves even with a Purge, but page edit fixes it.
Yes, null edits update links tables (e.g. Categories, whatlinkshere,
Special:LinkSearch, etc) purges do not.
Also
several times I got some sort of internal database errors when using
Special:WhatLinksHere.
That shouldn't happen. What sort of things were you doing when the error
happened?
--bawolff
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org mailto:Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Template transclusion on Wikimedia Commons
Le 30/01/14 14:39, Tuszynski, Jaroslaw W. a écrit : I found how to reproduce the database errors I mentioned earlier. Both https://commons.wikimedia.org/w/index.php?title=Special%3AWhatLinksHeretarget=Template%3ALocation%2Flayoutnamespace=10 and https://commons.wikimedia.org/w/index.php?title=Special%3AWhatLinksHeretarget=Module%3AFallbacklistnamespace=828 gave me Database error A database query error has occurred. This may indicate a bug in the software. Function: SpecialWhatLinksHere::showIndirectLinks Error: 0 Hello, The first URL worked for me, the second throws a database error. Thu Jan 30 14:11:19 UTC 2014 /* query */ Thu Jan 30 14:19:33 UTC 2014 /* query */ Connection lost and reconnected after 59.754s Can you please fill a bug about it? -- Antoine hashar Musso ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Template transclusion on Wikimedia Commons
Done. See https://bugzilla.wikimedia.org/show_bug.cgi?id=60618 Jarek T. (user:jarekt) -Original Message- From: wikitech-l-boun...@lists.wikimedia.org [mailto:wikitech-l-boun...@lists.wikimedia.org] On Behalf Of Antoine Musso Sent: Thursday, January 30, 2014 9:23 AM To: wikitech-l@lists.wikimedia.org Subject: Re: [Wikitech-l] Template transclusion on Wikimedia Commons Le 30/01/14 14:39, Tuszynski, Jaroslaw W. a écrit : I found how to reproduce the database errors I mentioned earlier. Both https://commons.wikimedia.org/w/index.php?title=Special%3AWhatLinksHer etarget=Template%3ALocation%2Flayoutnamespace=10 and https://commons.wikimedia.org/w/index.php?title=Special%3AWhatLinksHer etarget=Module%3AFallbacklistnamespace=828 gave me Database error A database query error has occurred. This may indicate a bug in the software. Function: SpecialWhatLinksHere::showIndirectLinks Error: 0 Hello, The first URL worked for me, the second throws a database error. Thu Jan 30 14:11:19 UTC 2014 /* query */ Thu Jan 30 14:19:33 UTC 2014 /* query */ Connection lost and reconnected after 59.754s Can you please fill a bug about it? -- Antoine hashar Musso ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] WikiConference USA May 30-June 1, scholarships available
You can now submit tutorial or talk proposals for WikiConference USA, and you can register and ask for a scholarship for your travel expenses (more information below and at http://wikiconferenceusa.org/wiki/Scholarships ). If it'll be hard for you to get to the Zurich or London hackathons this year, consider meeting up at WikiConference USA. Sumana Harihareswara Engineering Community Manager Wikimedia Foundation Date: Tue, 28 Jan 2014 16:56:55 -0500 From: Pharos pharosofalexand...@gmail.com To: wikimediaannounc...@lists.wikimedia.org Subject: [Wikimedia Announcements] WikiConference USA Announcement Message-ID: cajcrdm5+hquzyayerjd_4sf4zj2oktq8pmyr78lj23dzykj...@mail.gmail.com Content-Type: text/plain; charset=iso-8859-1 I am very pleased to announce that Wikimedia NYC and Wikimedia DC are working in collaboration to host the first national Wikimedia conference in the United States! Here are the details for the conference: Dates: Friday, May 30, 2014 - Sunday, June 1, 2014 Location: New York Law School (185 West Broadway, New York, NY 10013) Website: http://wikiconferenceusa.org Email: wiki...@wikimedianyc.org Registration: http://wikiconusa.eventbrite.org/ For more information, please review our official press release below! We hope you will join us and help us spread the word! https://commons.wikimedia.org/wiki/File:WikiCon_USA_2014_Press_Release_v1.pdf Thanks, Richard (User:Pharos) Wikimedia NYC ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Dynamic search results as category pages
On Sun, Jan 26, 2014 at 1:10 AM, Lydia Pintscher lydia.pintsc...@wikimedia.de wrote: On Fri, Jan 24, 2014 at 10:53 AM, Yuri Astrakhan yastrak...@wikimedia.org wrote: Hi, I am thinking of implementing a #CATQUERY query magic keyword for the category pages. When this keyword is present, the category page would execute a query against the search backend instead of normal category behavior and show result as if those pages were actually marked with this category. For example, this would allow Greek Philosophers category page to be quickly redefined as a cross-section of greeks philosophers categories: #CATQUERY incategory:Greek incategory:Philosopher Obviously the community will be able to define much more elaborate queries, including the ordering (will be supported by the new search backend) In the future this will be possible using Wikidata and queries. My dream is that categories will to a very large extend go away then and be replaced with better Wikidata-based tools. I'd _really_ like us not to introduce more incentives to use categories like this one. Also it'd be quite a source for confusion to have both. Indeed. I think it would be very shortsighted to introduce any sort of new syntax here for this right now as proposed. There's lots of exciting query opportunities coming with Wikidata as Lydia says (which will very likely end up using Elasticsearch in some manner). On Sun, Jan 26, 2014 at 10:21 AM, Brian Wolff bawo...@gmail.com wrote: Category intersection proposals seem to appear on this list about once every 2 years as far as i remember. Usually they dont succede because nobody ends up writing any efficient code to implement it. Yep, we've been here before. Because category intersection isn't easy with the schema we've got. However, it's incredibly trivial to do in Elasticsearch. So I imagine if we invest more effort in helping Wikidata expand its syntax and uses we'll likely end up with what Yuri's wanting--if maybe in a different form. I'll also note we've already solved category intersection in Cirrus ;-) https://en.wikipedia.org/w/index.php?search=incategory%3A%221941+births%22+incategory%3A%221995+deaths%22title=Special%3ASearch -Chad ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] Lsearch and MWSearch: how to turn on morphology for Russian
Hi guys! I've installed MWSearch and Lucene Search extensions but I can see that the search engine doesn't understand the morphology of Russian (doesn't recognize word forms). How can I turn the morphological analyzer on? How it's done in Russian Wikipedia? Cheers, - Yury Katkov, WikiVote ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Lsearch and MWSearch: how to turn on morphology for Russian
I hate to say this after all you went through setting up Lucene Search but it is end of life and not receiving any real support. We're in the process of replacing it with the combination of CirrusSearchhttps://www.mediawiki.org/wiki/Extension:CirrusSearch /Elasticsearch http://www.elasticsearch.org/ which work pretty much the same way the MWSearch/Lucene Search combination does. CirrusSearch has to be smarter than MWSearch because Elasticsearch doesn't have any Mediawiki knowledge but because it links into Mediawiki it can do things like expand templates. I like it but I'm biased. That aside, it looks like Lucene Search is supposed to read InitializeSettings which is kind of wmf specific thing. You might be able to trick it into doing it by putting a file called InitializeSettings.php in the conf directory with the contents 'wgLanguageCode' = array( 'your $wgDBname' = 'ru', ), CirrusSearch, if you care to try it, reads the language code from wgLanguageCode. Nik On Thu, Jan 30, 2014 at 3:39 PM, Yury Katkov katkov.ju...@gmail.com wrote: Hi guys! I've installed MWSearch and Lucene Search extensions but I can see that the search engine doesn't understand the morphology of Russian (doesn't recognize word forms). How can I turn the morphological analyzer on? How it's done in Russian Wikipedia? Cheers, - Yury Katkov, WikiVote ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] [WikimediaMobile] Serverside Events not being logged for Echo
+wikitech On Jan 29, 2014 12:15 PM, Jon Robson jdlrob...@gmail.com wrote: In the creation of this graph [1] - I noticed that for some reason no events for Thanks are being recorded to the database anymore. I raised a bug [2] but according to another bug [3] events on the server side are being logged to a file but no the database itself. According to the graph this happened January 9th - any ideas to why? [1] http://mobile-reportcard.wmflabs.org/#other-graphs-tab [2] https://bugzilla.wikimedia.org/show_bug.cgi?id=60550 [3] https://bugzilla.wikimedia.org/show_bug.cgi?id=60555 ___ Mobile-l mailing list mobil...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] [WikimediaMobile] Serverside Events not being logged for Echo
See https://bugzilla.wikimedia.org/show_bug.cgi?id=60550 On Thu, Jan 30, 2014 at 2:49 PM, Arthur Richards aricha...@wikimedia.org wrote: +wikitech On Jan 29, 2014 12:15 PM, Jon Robson jdlrob...@gmail.com wrote: In the creation of this graph [1] - I noticed that for some reason no events for Thanks are being recorded to the database anymore. I raised a bug [2] but according to another bug [3] events on the server side are being logged to a file but no the database itself. According to the graph this happened January 9th - any ideas to why? [1] http://mobile-reportcard.wmflabs.org/#other-graphs-tab [2] https://bugzilla.wikimedia.org/show_bug.cgi?id=60550 [3] https://bugzilla.wikimedia.org/show_bug.cgi?id=60555 ___ Mobile-l mailing list mobil...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l -- Jon Robson * http://jonrobson.me.uk * https://www.facebook.com/jonrobson * @rakugojon ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Lsearch and MWSearch: how to turn on morphology for Russian
Hi! I'll definitely try Cirrus, but still it's interesting to see Lucene
working. Besides everynew extension by WMF typically requires very fresh
MediaWiki version which can be a burden for 3rd parties.
I tried to add InitializeSettings.php, run ./build and ./lsearchd again.
Still no good, when I search the word банк, I expect Lucene to find also
банков, банки, банке, etc., and I can see that these word forms are
presented in a file
LuceneSearch.jar/uzip://org/apache/lucene/analysis/ru/stemsUnicode.txt
and words.Unicode.txt.
Still when I search for банк, I only get банк and the following log:
18409 [pool-2-thread-1] INFO org.wikimedia.lsearch.search.SearchEngine -
Using FilterWrapper wrap: {} []
18414 [pool-2-thread-1] INFO org.wikimedia.lsearch.search.SearchEngine -
search wikivote: query=[банк] parsed=[custom(+contents:банк^0.2 relevance
([((P contents:банк) (P sections:банк^0.25))^2.0], (P
alttitle:банк~20^2.5) (P related:банк^12.0)) (P alttitle:банк~20))]
hit=[0] in 7ms using IndexSearcherMul:1391088160991
18439 [pool-2-thread-1] INFO org.wikimedia.lsearch.spell.Suggest -
wikivote for original=[банк] suggest: [банк] using=[] in 18 ms
24262 [pool-2-thread-2] INFO org.wikimedia.lsearch.frontend.HttpHandler -
query:/search/wikivote/%D0%B1%D0%B0%D0%BD%D0%BA?namespaces=0%2C1%2C2%2C3%2C4%2C5%2C6%2C7%2C8%2C9%2C10%2C11%2C12%2C13%2C14%2C15%2C90%2C91%2C92%2C93%2C102%2C103%2C106%2C107%2C108%2C109%2C170%2C171offset=0limit=20version=2.1iwlimit=10searchall=1
what:search dbname:wikivote term:банк
24263 [pool-2-thread-2] INFO org.wikimedia.lsearch.search.SearchEngine -
Using FilterWrapper wrap: {} []
-
Yury Katkov, WikiVote
On Fri, Jan 31, 2014 at 1:02 AM, Nikolas Everett never...@wikimedia.orgwrote:
I hate to say this after all you went through setting up Lucene Search but
it is end of life and not receiving any real support. We're in the process
of replacing it with the combination of
CirrusSearchhttps://www.mediawiki.org/wiki/Extension:CirrusSearch
/Elasticsearch http://www.elasticsearch.org/ which work pretty much the
same way the MWSearch/Lucene Search combination does. CirrusSearch has to
be smarter than MWSearch because Elasticsearch doesn't have any Mediawiki
knowledge but because it links into Mediawiki it can do things like expand
templates. I like it but I'm biased.
That aside, it looks like Lucene Search is supposed to read
InitializeSettings which is kind of wmf specific thing. You might be able
to trick it into doing it by putting a file called InitializeSettings.php
in the conf directory with the contents
'wgLanguageCode' = array(
'your $wgDBname' = 'ru',
),
CirrusSearch, if you care to try it, reads the language code from
wgLanguageCode.
Nik
On Thu, Jan 30, 2014 at 3:39 PM, Yury Katkov katkov.ju...@gmail.com
wrote:
Hi guys!
I've installed MWSearch and Lucene Search extensions but I can see that
the
search engine doesn't understand the morphology of Russian (doesn't
recognize word forms). How can I turn the morphological analyzer on? How
it's done in Russian Wikipedia?
Cheers,
-
Yury Katkov, WikiVote
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
