[Wikitech-l] Wikimedia-Categories Issue T147762

[Shanika Ediriweera]

Hi,

My name is Shanika Ediriweera and I am an undergraduate of University of
Moratuwa. I am new to the wikimedia community.

I would like to tackle this issue https://phabricator.wikimedia.org/T147762.

Since I am new to the community could someone guide me in reproducing the
issue?
Where is the code related to this issue? Is it in the core or in an
extension.

Thank you.

Best Regards,

Shanika Ediriweera
Undergraduate
Dept. of Computer Science & Eng.
University of Moratuwa
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] [Analytics] Monthly page view stats that can now be queried via Pageview API.

[Leon Ziemba]

Pageviews Analysis now has support for the monthly granularity:

http://tools.wmflabs.org/pageviews/?project=en.wikipedia.org=2015-07=2016-12=Barack_Obama|Donald_Trump|Hillary_Clinton|Bernie_Sanders

...which is much more decipherable than the same range with daily
granularity!:

http://tools.wmflabs.org/pageviews/?project=en.wikipedia.org=2015-07-01=2016-12-31=Barack_Obama|Donald_Trump|Hillary_Clinton|Bernie_Sanders


Thanks to the Analytics team and GCI! :)

~MA

On Wed, Jan 25, 2017 at 12:49 PM, Nick Wilson (Quiddity) <
nwil...@wikimedia.org> wrote:

> (Fixed link, for wikitech-l)
>
> On Tue, Jan 24, 2017 at 8:18 PM, Nuria Ruiz  wrote:
> >
> > Hello!
> >
> > The Analytics team would like to announce that the Pageview API is able
> to
> > return monthly pageview stats as of this week.
> >
> >
> > For example, the request below will get you a monthly count of pageviews
> > for de.wikipedia's article Barack_Obama for the year 2016 (note 'monthly'
> > parameter on url)
> >
>
> https://wikimedia.org/api/rest_v1/metrics/pageviews/per-
> article/de.wikipedia/all-access/all-agents/Barack_
> Obama/monthly/2016010100/2016123100
>
> (automatic richtext conversion on wikitech-l had added *s around the
> bolded "monthly". :)
>
> >
> >
> > More documentation and examples can be found on our quickstart guide:
> > https://wikitech.wikimedia.org/wiki/Analytics/PageviewAPI#Quick_Start
>
> The interface given in that first link is also really interesting! (
> https://wikimedia.org/api/rest_v1/?doc ). Thanks, Analytics.
>
> >
> > Thanks,
> >
> > Nuria
> > ___
> > Wikitech-l mailing list
> > Wikitech-l@lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
> --
> Nick Wilson (Quiddity)
> Community Liaison, WMF
>
> ___
> Analytics mailing list
> analyt...@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/analytics
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] CirrusSearch index incompleteness problem

[Aran]

I can't see any errors in the debug log... one page had five occurrences
of a search term and only showed up after rebuilding the index many
times. I kept running it because I knew the page should be showing up in
the results and eventually it got included.


On 26/01/17 17:36, Erik Bernhardson wrote:
> On Thu, Jan 26, 2017 at 11:30 AM, Aran  wrote:
>
>> Hello,
>>
>> I'm managing some mediawiki 1.27.1's running CirrusSearch 0.2 with
>> Elasticsearch 1.7.5. I been noticing that there are often search results
>> missing so I started running the forceSearchIndex.php script each night
>> on a cron job.
>>
>> But I'm still finding results missing. Today I re-ran the script
>> manually and then found that one of the missing results showed up and
>> that the result count for that term had increased from 18 to 23. I ran
>> the script again and it increased more to 37. I ran more times but the
>> result count did not increase any more.
>>
>> The commands I've been doing are:
>> forceSearchIndex.php --skipLinks --indexOnSkip
>> forceSearchIndex.php --skipParse
>>
>> Is this the correct way to do a full index rebuild? is there some
>> parameter that can ensure that no pages get missed?
>>
>>
> This is the correct way to rebuild documents in place. It sounds like
> something is running into errors while building documents though. Could you
> check your logs for errors related to CirrusSearch?
>
>
>> Thanks,
>> Aran
>>
>>
>> ___
>> Wikitech-l mailing list
>> Wikitech-l@lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] +2 request for yurik in mediawiki and maps-dev

[Legoktm]

Hi,

On 01/26/2017 08:03 AM, Quim Gil wrote:
> For the record: Yuri got his permissions restored 52 minutes after the task
> was filed, without much discussion. The task is Resolved since then. Can we
> resolve this thread with this title as well, please?

Yes, thanks. I respectfully ask people not further respond to this
thread, if you have questions please ask me offlist.

-- Legoktm

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] CirrusSearch index incompleteness problem

[Erik Bernhardson]

On Thu, Jan 26, 2017 at 11:30 AM, Aran  wrote:

> Hello,
>
> I'm managing some mediawiki 1.27.1's running CirrusSearch 0.2 with
> Elasticsearch 1.7.5. I been noticing that there are often search results
> missing so I started running the forceSearchIndex.php script each night
> on a cron job.
>
> But I'm still finding results missing. Today I re-ran the script
> manually and then found that one of the missing results showed up and
> that the result count for that term had increased from 18 to 23. I ran
> the script again and it increased more to 37. I ran more times but the
> result count did not increase any more.
>
> The commands I've been doing are:
> forceSearchIndex.php --skipLinks --indexOnSkip
> forceSearchIndex.php --skipParse
>
> Is this the correct way to do a full index rebuild? is there some
> parameter that can ensure that no pages get missed?
>
>
This is the correct way to rebuild documents in place. It sounds like
something is running into errors while building documents though. Could you
check your logs for errors related to CirrusSearch?


> Thanks,
> Aran
>
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] CirrusSearch index incompleteness problem

[Aran]

Hello,

I'm managing some mediawiki 1.27.1's running CirrusSearch 0.2 with
Elasticsearch 1.7.5. I been noticing that there are often search results
missing so I started running the forceSearchIndex.php script each night
on a cron job.

But I'm still finding results missing. Today I re-ran the script
manually and then found that one of the missing results showed up and
that the result count for that term had increased from 18 to 23. I ran
the script again and it increased more to 37. I ran more times but the
result count did not increase any more.

The commands I've been doing are:
forceSearchIndex.php --skipLinks --indexOnSkip
forceSearchIndex.php --skipParse

Is this the correct way to do a full index rebuild? is there some
parameter that can ensure that no pages get missed?

Thanks,
Aran


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] +2 request for yurik in mediawiki and maps-dev

[Alex Monk]

Update: I demanded an explanation for Yurik's removal, nobody would give a
good one, I revoked ops' administrative rights, and now I've been removed
as a gerrit administrator with ops re-added.
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Deprecation logging in production

[Chad]

On Thu, Jan 26, 2017 at 10:19 AM Daniel Kinzler 
wrote:

> * Extensions and skins bundled with the MediaWiki tarball MUST NOT trigger
> hard
> deprecation warnings and MUST be updated to use the new code.
>
>
> But that's just for bundeled extensions, not deployed extensions. There is
> this:
>
>
> * As one of the principles of MediaWiki, developers should ensure any
> removals
> will not cause issues in the Wikimedia setup and extensions deployed
> there. If
> they do, developers should expect to be reverted by Wikimedia system
> administrators.
>
>
As the person to changed the tarball criteria to MUST NOT, I would
be totally in favor of extending that to deployed extensions.

Removing code that's still used in production is embarrassing/sad
and offenders should be ashamed ;-)

-Chad
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] +2 request for yurik in mediawiki and maps-dev

[Neil P. Quinn]

On Wed, Jan 25, 2017 at 2:55 PM, Alex Monk  wrote:

> On 25 January 2017 at 19:48, Kevin Smith  wrote:
>
> > I support restoring the rights in this case, but I'm not sure it should
> be
> > automatic in all cases. If having +2 rights is relatively harmless and
> > reversible, then an automatic (but announced) restoration of the rights
> > should be fine.
> >
> > The issue is that someone who leaves the foundation could do so under
> > unfriendly terms, possibly affecting their ability to do good work. I
> know
> > we don't want to think that a previously productive volunteer could later
> > cause problems, but it is possible.
> >
> > That's why I think there should be some form of check, so we have
> > confidence that this person still has good intentions. For example, their
> > manager and/or someone from Talent & Culture could be consulted, or
> trusted
> > people who still have close contact with the person so know their state
> of
> > mind. It could be quick and lightweight, in almost all cases, but
> skipping
> > that step entirely seems risky to me. Unless, as I said, having +2 really
> > isn't a big deal.
> >
> >
> >
> >
> > Kevin Smith
> > Agile Coach, Wikimedia Foundation
> >
>
> You don't appear to be a developer. Even if you were, Brian said 'similar
> situation' - i.e., where someone had been granted rights before becoming
> WMF staff.
>

Someone's familiarity with the topic is certainly relevant, but let's not
fall into the trap of "you're not a core developer, therefore you shouldn't
be posting to this list." In this specific case, Kevin is an expert in the
social dynamics of technical teams (enough so that he's employed by the WMF
for that expertise). That doesn't mean you have to agree with him, but it
does mean you should at least consider his input (not to mention respect
him as you would any community member making good faith suggestions).

-- 
Neil P. Quinn ,
product analyst
Wikimedia Foundation
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Deprecation logging in production

[Daniel Kinzler]

Am 26.01.2017 um 10:36 schrieb Antoine Musso:
> We have a strict deprecation policy which ask for extensions to be updated 
> when
> a code is @deprecated.  And removing code/hook must only be made when we have
> verified that all extensions we host have been migrated.
> https://www.mediawiki.org/wiki/Deprecation

Not all extensions. The relevant bits of the (new, as of two weeks ago!) policy:

* Developers SHOULD update any extension or skin bundled with the MediaWiki
tarball when soft deprecating, and MAY update popular extensions

* If they don't submit patches, developers MUST file bugs about bundled
extensions/skins using deprecated functions so their maintainers can work on
updating them.

* Extensions and skins bundled with the MediaWiki tarball MUST NOT trigger hard
deprecation warnings and MUST be updated to use the new code.


But that's just for bundeled extensions, not deployed extensions. There is this:


* As one of the principles of MediaWiki, developers should ensure any removals
will not cause issues in the Wikimedia setup and extensions deployed there. If
they do, developers should expect to be reverted by Wikimedia system 
administrators.

But that's only removals, not deprecation.

-- 
Daniel Kinzler
Principal Platform Engineer

Wikimedia Deutschland
Gesellschaft zur Förderung Freien Wissens e.V.

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] +2 request for yurik in mediawiki and maps-dev

[Alex Monk]

On 26 January 2017 at 16:03, Quim Gil  wrote:

> Can we
> resolve this thread with this title as well, please?
>
We could rename the thread.

On 26 January 2017 at 16:03, Quim Gil  wrote:

> The cause for the removal was simple: standard procedure for off-boarding
> Wikimedia Foundation employees.
>

Our +2 policy does not allow for that to happen in this way as the 'you
have no intent to continue contributing' part does not appear to have been
satisfied - and even if it were, the rights were granted as a volunteer. In
any case you are not the person who removed the rights, we need to hear
from Brandon.
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] ArchCom Status & Meeting Minutes, WD4

[Daniel Kinzler]

Hi all!

Here are the minutes from this week's ArchCom meeting. You can also find the
minutes at .

See also the ArchCom status page at
 and the RFC board
.

Here are the minutes, for your convenience:

* Gabriel is considering use cases for storage of HTML for old revision:
mw:Wikimedia_Services/Revision_storage_for_HTML_and_structured_data:_Use_cases
* Continuing meta-discussion about ArchCom, touching on the following topics:
** Encouraging more RFC discussion on Phabricator
** Using IRC meetings as a broader office hour
** ArchCom’s involvement with strategy making and resource planning
At the IRC meeting, we discussed MZMcBride’s proposal for Accessing page
properties from wiki pages. We explored use cases, identified potential issues
and formulated requirements. A summary can be found at T154738#2972889.

Active RFCs, please have a look:
* T66214: Define an official thumb API. Effort to define a client-controlled API
for retrieving thumbnails with various options.
* T128351: Notifications in core.
* T124752: Expiring watchlist entries.
* T145604: Future of magic links.

Last Call:
[RFC] Image and oldimage tables
Please review the updated RFC page
 and send any final comments
here on Wikitech-l or comment on T589 by 2017-02-01.

Next week on IRC:
Discuss the planned schema changes for DB realignment & MCR (pending
confirmation from Brion). See:
* mw:Wikimedia_Developer_Summit/2017/Scaling_the_Wikimedia_database_schema
* mw:User:Brion_VIBBER/Compacting_the_revision_table
* mw:Multi-Content_Revisions/Content_Meta-Data#Database_Schema
* mw:Requests_for_comment/Content_model_storage

-- 
Daniel Kinzler
Principal Platform Engineer

Wikimedia Deutschland
Gesellschaft zur Förderung Freien Wissens e.V.

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] +2 request for yurik in mediawiki and maps-dev

[Quim Gil]

On Wed, Jan 25, 2017 at 4:44 AM, Legoktm 
wrote:

> Hi,
>
> After speaking with Yurik, I've filed
>  on his behalf to restore his
> membership in the mediawiki and maps-dev groups.
>
> I would appreciate guidance in whether these rights can be summarily
> granted since he used to have them, or if it needs to go through the
> full process.
>

For the record: Yuri got his permissions restored 52 minutes after the task
was filed, without much discussion. The task is Resolved since then. Can we
resolve this thread with this title as well, please?

The cause for the removal was simple: standard procedure for off-boarding
Wikimedia Foundation employees. If this off-boarding process can be
improved, please propose improvements in a new thread or in a Phabricator
task. Considering how often we have ended in this situation and how quickly
the problem has been resolved, it looks like we can have a constructive
discussion without pressure or blame.

-- 
Quim Gil
Engineering Community Manager @ Wikimedia Foundation
http://www.mediawiki.org/wiki/User:Qgil
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] +2 request for yurik in mediawiki and maps-dev

[Zppix]

Why did they got removed?

> On Jan 24, 2017, at 9:44 PM, Legoktm  wrote:
> 
> Hi,
> 
> After speaking with Yurik, I've filed
>  on his behalf to restore his
> membership in the mediawiki and maps-dev groups.
> 
> I would appreciate guidance in whether these rights can be summarily
> granted since he used to have them, or if it needs to go through the
> full process.
> 
> -- Legoktm
> 
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] +2 request for yurik in mediawiki and maps-dev

[MZMcBride]

Antoine Musso wrote:
>The staff period can have changed the state of mind of the person, even
>had it been a volunteer before employment. If leaving in bad terms with
>an eventual envy of retaliation against the project, then code get
>screwed.
>
>A very basic exit interview would clarify the future ground and as Kevin
>said in 99% of case it will be a non issue. But we might catch the 1%
>lone wolf that will eventually cause havoc.  For the price of a few
>minutes interview, I think it is worth it.

Lone wolf? Wow. In the history of both MediaWiki and Wikimedia
development, Wikimedia Foundation staff have created significantly more
havoc and disruption than volunteers. Please don't forget that.

MZMcBride



___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Code of Conduct "Creation and renewal of the Committee" text

[Zppix]

Mind if i proofread and edit it a bit if i see fit?

> On Jan 24, 2017, at 1:14 PM, Matthew Flaschen  wrote:
> 
> Please participate in the discussion about the "Creation and renewal of the 
> Committee" section.  This is not to approve it yet, just a discussion:
> 
> https://www.mediawiki.org/wiki/Talk:Code_of_Conduct/Draft#.22Creation_and_renewal_of_the_Committee.22_section
> 
> Most of this text has been around for a while, but I made a few changes to 
> the draft last week, so I wanted to give people an opportunity to comment, 
> suggest changes, etc.
> 
> I'll give it a few more days, then start the formal approval discussion.
> 
> Thanks,
> 
> Matt Flaschen
> 
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] +2 request for yurik in mediawiki and maps-dev

[Antoine Musso]

On 25/01/17 22:03, Alex Monk wrote:

We are talking about exactly the same situation: Someone was a productive
volunteer, then staff, then no longer staff. 99+% of the time, they should
retain their rights, or get them back shortly after leaving. But there may
be cases where someone would be disgruntled, and if rights are automatic in
all cases, such a person could do damage. I'm just suggesting that there
should be a quick and simple check to be sure that's not the case. Being
without rights for a couple days doesn't seem like a small price to pay for
the safety of the project.


If they were added as a volunteer that means they passed a vote to be
there, we're not going to start adding such extra checks and requirements.


Hello,

That is quite a bold statement Alex. Being staff or contractor to the 
foundation is quite a different scope than acting as volunteer.  There 
are deadlines, work to be done that you would prefer not, some internal 
drama and politics.  And for staff potentially clash with management and 
colleague.


The staff period can have changed the state of mind of the person, even 
had it been a volunteer before employment. If leaving in bad terms with 
an eventual envy of retaliation against the project, then code get screwed.


A very basic exit interview would clarify the future ground and as Kevin 
said in 99% of case it will be a non issue. But we might catch the 1% 
lone wolf that will eventually cause havoc.  For the price of a few 
minutes interview, I think it is worth it.


Imho, when staff / contractor finish their relationship with the 
foundation, all rights should be removed and the process should start 
over (eg: make sure the volunteer as a NDA signed and is only granted 
appropriate rights).


cheers,

--
Antoine "hashar" Musso




___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Deprecation logging in production

[Antoine Musso]

On 26/01/17 01:18, Erik Bernhardson wrote:

Noticed earlier today, because some code that had been deprecated since
1.21 was removed and starting causing fatals, that we don't log deprecation
notices in production. This has now been fixed.

Please check logstash for `channel:deprecated` and see if anything you are
reposinsible for is using hard-deprecated methods and fix them.

Some culprits which have each generated hundreds of messages in the ~5
minutes it's been turned on:

Use of SpecialRecentChangesQuery hook (used in
FlaggedRevsUIHooks::modifyRecentChangesQuery) was deprecated in MediaWiki
1.23. [Called from SpecialRecentChanges::runMainQueryHook in
/srv/mediawiki/php-1.29.0-wmf.8/includes/specials/SpecialRecentchanges.ph



Thanks for the logging channel, that is going to be quite helpful. To 
anyone spotting them, please fill corresponding tasks against the 
appropriate extension/skin and add in #wikimedia-log-errors to the task.



There are a few things that puzzle me though:

We have CI running with $wgDevelopmentWarnings = true; which fails the 
test whenever wfDeprecated() is invoked.  But I guess lot of extension 
code lack tests for those hooks.


We have a strict deprecation policy which ask for extensions to be 
updated when a code is @deprecated.  And removing code/hook must only be 
made when we have verified that all extensions we host have been migrated.

https://www.mediawiki.org/wiki/Deprecation

Then mistakes happen :(

--
Antoine "hashar" Musso





___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Deprecation logging in production

[Antoine Musso]

On 26/01/17 01:48, Gergo Tisza wrote:


Here is a simple dashboard:
https://logstash.wikimedia.org/app/kibana#/dashboard/mediawiki-deprecated


Thanks quite helpful.

The equivalent for beta which is publicly available:
https://logstash-beta.wmflabs.org/app/kibana#/dashboard/mediawiki-deprecated


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l