Re: [Wikitech-l] Previous mediawiki version test wiki

2017-09-21 Thread Niharika Kohli 
>
> When filing a phab task with some new bug,
> you always want to know - is it really new, or I just did not pay attention
> to it before?

What's the purpose of this information? If it's a bug, new or not, a ticket
needs to be filed.

And when I do know it's a new bug, I can open both versions
> in the same time, and compare the behaviour for this bug. And also, compare
> the console results - what exactly changed in html, in css, in js commands
> reactions.

I agree that information will save some developer time but at the same time
this information is not so easy to gather. This is helpful when the users
have some working knowledge of how developer tools work and how to compare
file changes. Usually in each version there are a lot of new changes. Often
it's not easy for developers even to find out what could be causing the
bug.

I can easily imagine such a wiki quickly falling into disuse.


On Thu, Sep 21, 2017 at 7:29 PM, יגאל חיטרון  wrote:

> It can work. But another Monday. I mean, if Tue-Wed-Thu there is a
> deployment of version 5, a day before, Mon there is a deployment of
> version 4, so starting from tomorrow, group 0 will get a way  to see
> both version, exactly from the beginning, but not until the end, for 6
> days, group 1 for 5 days, and group 2 for 4 days. And from Monday to the
> deployment, 1-2-3 days, there will not be use of this. I'll be very glad if
> it will be decided to do this, and if so, it will be a good thing to add to
> the text of how to report a bug in phabricator help, something about, you
> can check if it is a regression, the last version "falt", by comparing with
> this new wiki. I can thing about many dozens of tasks I wrote and read
> where this information could be useful, if added at the first place. Hope
> you decide this indeed. Thank you very much,
> Igal
>
>
> On Sep 22, 2017 05:17, "Chad"  wrote:
>
> > No non-emergency deployments on Fridays, Saturdays or Sundays.
> > Monday could work.
> >
> > -Chad
> >
> > ‪On Thu, Sep 21, 2017 at 7:15 PM ‫יגאל חיטרון‬‎ 
> > wrote:‬
> >
> > > I glad you say so. What about Friday?
> > > Igal
> > >
> > >
> > > On Sep 22, 2017 05:07, "Chad"  wrote:
> > >
> > > > It wouldn't be hard to do at all, technically. I imagine it'd be
> > > something
> > > > like a test3wiki.
> > > >
> > > > Main thing to know is when do we cycle off of the old version? When
> the
> > > > version goes out on Tuesdays? That day's already pretty loaded for
> > > software
> > > > moving about...
> > > >
> > > > -Chad
> > > >
> > > > On Thu, Sep 21, 2017 at 1:25 PM Brian Wolff 
> wrote:
> > > >
> > > > > Making your case here is probably best. The release engineering
> team
> > > are
> > > > > the people you probably have to convince, although of course anyone
> > > could
> > > > > potentially create such a wiki, in an unofficial way.
> > > > >
> > > > > Keep in mind that keeping an older version of the software running
> > does
> > > > > introduce a maintinance burden, so you will probably have to
> convince
> > > > > people that it would be regularly useful and not just useful this
> one
> > > > time.
> > > > >
> > > > > --
> > > > > bawolff
> > > > >
> > > > > On Thursday, September 21, 2017, יגאל חיטרון 
> > > wrote:
> > > > > > Thank you. Sorry to hear this. Is there some place I can suggest
> > this
> > > > and
> > > > > > explain why do I think it can be very helpful?
> > > > > > Igal
> > > > > >
> > > > > > On Sep 21, 2017 22:12, "Brian Wolff"  wrote:
> > > > > >
> > > > > >> On Thursday, September 21, 2017, יגאל חיטרון <
> > > khit...@post.bgu.ac.il>
> > > > > >> wrote:
> > > > > >> > Hi. Sometimes after the week deployment I need to compare the
> > new
> > > > > version
> > > > > >> > with the previous one, in some aspect. Is there a test wiki
> that
> > > > > always
> > > > > >> has
> > > > > >> > one version before the current?
> > > > > >> > Thank you.
> > > > > >> > Igal (User:IKhitron)
> > > > > >> > ___
> > > > > >> > Wikitech-l mailing list
> > > > > >> > Wikitech-l@lists.wikimedia.org
> > > > > >> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> > > > > >>
> > > > > >> No there is not. You can of course download old versions of the
> > > > software
> > > > > >> and setup your own wiki but that is a lot of effort.
> > > > > >>
> > > > > >> --
> > > > > >> bawolff
> > > > > >> ___
> > > > > >> Wikitech-l mailing list
> > > > > >> Wikitech-l@lists.wikimedia.org
> > > > > >> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> > > > > > ___
> > > > > > Wikitech-l mailing list
> > > > > > Wikitech-l@lists.wikimedia.org
> > > > > > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> > > > > ___
> > > > > Wikitech-l mailing list
> > > > > Wikitech-l@lists.wikimedia.org
> > > > > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> > >

Re: [Wikitech-l] Previous mediawiki version test wiki

2017-09-21 Thread יגאל חיטרון 
It can work. But another Monday. I mean, if Tue-Wed-Thu there is a
deployment of version 5, a day before, Mon there is a deployment of
version 4, so starting from tomorrow, group 0 will get a way  to see
both version, exactly from the beginning, but not until the end, for 6
days, group 1 for 5 days, and group 2 for 4 days. And from Monday to the
deployment, 1-2-3 days, there will not be use of this. I'll be very glad if
it will be decided to do this, and if so, it will be a good thing to add to
the text of how to report a bug in phabricator help, something about, you
can check if it is a regression, the last version "falt", by comparing with
this new wiki. I can thing about many dozens of tasks I wrote and read
where this information could be useful, if added at the first place. Hope
you decide this indeed. Thank you very much,
Igal


On Sep 22, 2017 05:17, "Chad"  wrote:

> No non-emergency deployments on Fridays, Saturdays or Sundays.
> Monday could work.
>
> -Chad
>
> ‪On Thu, Sep 21, 2017 at 7:15 PM ‫יגאל חיטרון‬‎ 
> wrote:‬
>
> > I glad you say so. What about Friday?
> > Igal
> >
> >
> > On Sep 22, 2017 05:07, "Chad"  wrote:
> >
> > > It wouldn't be hard to do at all, technically. I imagine it'd be
> > something
> > > like a test3wiki.
> > >
> > > Main thing to know is when do we cycle off of the old version? When the
> > > version goes out on Tuesdays? That day's already pretty loaded for
> > software
> > > moving about...
> > >
> > > -Chad
> > >
> > > On Thu, Sep 21, 2017 at 1:25 PM Brian Wolff  wrote:
> > >
> > > > Making your case here is probably best. The release engineering team
> > are
> > > > the people you probably have to convince, although of course anyone
> > could
> > > > potentially create such a wiki, in an unofficial way.
> > > >
> > > > Keep in mind that keeping an older version of the software running
> does
> > > > introduce a maintinance burden, so you will probably have to convince
> > > > people that it would be regularly useful and not just useful this one
> > > time.
> > > >
> > > > --
> > > > bawolff
> > > >
> > > > On Thursday, September 21, 2017, יגאל חיטרון 
> > wrote:
> > > > > Thank you. Sorry to hear this. Is there some place I can suggest
> this
> > > and
> > > > > explain why do I think it can be very helpful?
> > > > > Igal
> > > > >
> > > > > On Sep 21, 2017 22:12, "Brian Wolff"  wrote:
> > > > >
> > > > >> On Thursday, September 21, 2017, יגאל חיטרון <
> > khit...@post.bgu.ac.il>
> > > > >> wrote:
> > > > >> > Hi. Sometimes after the week deployment I need to compare the
> new
> > > > version
> > > > >> > with the previous one, in some aspect. Is there a test wiki that
> > > > always
> > > > >> has
> > > > >> > one version before the current?
> > > > >> > Thank you.
> > > > >> > Igal (User:IKhitron)
> > > > >> > ___
> > > > >> > Wikitech-l mailing list
> > > > >> > Wikitech-l@lists.wikimedia.org
> > > > >> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> > > > >>
> > > > >> No there is not. You can of course download old versions of the
> > > software
> > > > >> and setup your own wiki but that is a lot of effort.
> > > > >>
> > > > >> --
> > > > >> bawolff
> > > > >> ___
> > > > >> Wikitech-l mailing list
> > > > >> Wikitech-l@lists.wikimedia.org
> > > > >> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> > > > > ___
> > > > > Wikitech-l mailing list
> > > > > Wikitech-l@lists.wikimedia.org
> > > > > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> > > > ___
> > > > Wikitech-l mailing list
> > > > Wikitech-l@lists.wikimedia.org
> > > > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> > > ___
> > > Wikitech-l mailing list
> > > Wikitech-l@lists.wikimedia.org
> > > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> > ___
> > Wikitech-l mailing list
> > Wikitech-l@lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Previous mediawiki version test wiki

2017-09-21 Thread Chad 
No non-emergency deployments on Fridays, Saturdays or Sundays.
Monday could work.

-Chad

‪On Thu, Sep 21, 2017 at 7:15 PM ‫יגאל חיטרון‬‎  wrote:‬

> I glad you say so. What about Friday?
> Igal
>
>
> On Sep 22, 2017 05:07, "Chad"  wrote:
>
> > It wouldn't be hard to do at all, technically. I imagine it'd be
> something
> > like a test3wiki.
> >
> > Main thing to know is when do we cycle off of the old version? When the
> > version goes out on Tuesdays? That day's already pretty loaded for
> software
> > moving about...
> >
> > -Chad
> >
> > On Thu, Sep 21, 2017 at 1:25 PM Brian Wolff  wrote:
> >
> > > Making your case here is probably best. The release engineering team
> are
> > > the people you probably have to convince, although of course anyone
> could
> > > potentially create such a wiki, in an unofficial way.
> > >
> > > Keep in mind that keeping an older version of the software running does
> > > introduce a maintinance burden, so you will probably have to convince
> > > people that it would be regularly useful and not just useful this one
> > time.
> > >
> > > --
> > > bawolff
> > >
> > > On Thursday, September 21, 2017, יגאל חיטרון 
> wrote:
> > > > Thank you. Sorry to hear this. Is there some place I can suggest this
> > and
> > > > explain why do I think it can be very helpful?
> > > > Igal
> > > >
> > > > On Sep 21, 2017 22:12, "Brian Wolff"  wrote:
> > > >
> > > >> On Thursday, September 21, 2017, יגאל חיטרון <
> khit...@post.bgu.ac.il>
> > > >> wrote:
> > > >> > Hi. Sometimes after the week deployment I need to compare the new
> > > version
> > > >> > with the previous one, in some aspect. Is there a test wiki that
> > > always
> > > >> has
> > > >> > one version before the current?
> > > >> > Thank you.
> > > >> > Igal (User:IKhitron)
> > > >> > ___
> > > >> > Wikitech-l mailing list
> > > >> > Wikitech-l@lists.wikimedia.org
> > > >> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> > > >>
> > > >> No there is not. You can of course download old versions of the
> > software
> > > >> and setup your own wiki but that is a lot of effort.
> > > >>
> > > >> --
> > > >> bawolff
> > > >> ___
> > > >> Wikitech-l mailing list
> > > >> Wikitech-l@lists.wikimedia.org
> > > >> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> > > > ___
> > > > Wikitech-l mailing list
> > > > Wikitech-l@lists.wikimedia.org
> > > > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> > > ___
> > > Wikitech-l mailing list
> > > Wikitech-l@lists.wikimedia.org
> > > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> > ___
> > Wikitech-l mailing list
> > Wikitech-l@lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Previous mediawiki version test wiki

2017-09-21 Thread יגאל חיטרון 
I glad you say so. What about Friday?
Igal


On Sep 22, 2017 05:07, "Chad"  wrote:

> It wouldn't be hard to do at all, technically. I imagine it'd be something
> like a test3wiki.
>
> Main thing to know is when do we cycle off of the old version? When the
> version goes out on Tuesdays? That day's already pretty loaded for software
> moving about...
>
> -Chad
>
> On Thu, Sep 21, 2017 at 1:25 PM Brian Wolff  wrote:
>
> > Making your case here is probably best. The release engineering team are
> > the people you probably have to convince, although of course anyone could
> > potentially create such a wiki, in an unofficial way.
> >
> > Keep in mind that keeping an older version of the software running does
> > introduce a maintinance burden, so you will probably have to convince
> > people that it would be regularly useful and not just useful this one
> time.
> >
> > --
> > bawolff
> >
> > On Thursday, September 21, 2017, יגאל חיטרון  wrote:
> > > Thank you. Sorry to hear this. Is there some place I can suggest this
> and
> > > explain why do I think it can be very helpful?
> > > Igal
> > >
> > > On Sep 21, 2017 22:12, "Brian Wolff"  wrote:
> > >
> > >> On Thursday, September 21, 2017, יגאל חיטרון 
> > >> wrote:
> > >> > Hi. Sometimes after the week deployment I need to compare the new
> > version
> > >> > with the previous one, in some aspect. Is there a test wiki that
> > always
> > >> has
> > >> > one version before the current?
> > >> > Thank you.
> > >> > Igal (User:IKhitron)
> > >> > ___
> > >> > Wikitech-l mailing list
> > >> > Wikitech-l@lists.wikimedia.org
> > >> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> > >>
> > >> No there is not. You can of course download old versions of the
> software
> > >> and setup your own wiki but that is a lot of effort.
> > >>
> > >> --
> > >> bawolff
> > >> ___
> > >> Wikitech-l mailing list
> > >> Wikitech-l@lists.wikimedia.org
> > >> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> > > ___
> > > Wikitech-l mailing list
> > > Wikitech-l@lists.wikimedia.org
> > > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> > ___
> > Wikitech-l mailing list
> > Wikitech-l@lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Previous mediawiki version test wiki

2017-09-21 Thread Chad 
It wouldn't be hard to do at all, technically. I imagine it'd be something
like a test3wiki.

Main thing to know is when do we cycle off of the old version? When the
version goes out on Tuesdays? That day's already pretty loaded for software
moving about...

-Chad

On Thu, Sep 21, 2017 at 1:25 PM Brian Wolff  wrote:

> Making your case here is probably best. The release engineering team are
> the people you probably have to convince, although of course anyone could
> potentially create such a wiki, in an unofficial way.
>
> Keep in mind that keeping an older version of the software running does
> introduce a maintinance burden, so you will probably have to convince
> people that it would be regularly useful and not just useful this one time.
>
> --
> bawolff
>
> On Thursday, September 21, 2017, יגאל חיטרון  wrote:
> > Thank you. Sorry to hear this. Is there some place I can suggest this and
> > explain why do I think it can be very helpful?
> > Igal
> >
> > On Sep 21, 2017 22:12, "Brian Wolff"  wrote:
> >
> >> On Thursday, September 21, 2017, יגאל חיטרון 
> >> wrote:
> >> > Hi. Sometimes after the week deployment I need to compare the new
> version
> >> > with the previous one, in some aspect. Is there a test wiki that
> always
> >> has
> >> > one version before the current?
> >> > Thank you.
> >> > Igal (User:IKhitron)
> >> > ___
> >> > Wikitech-l mailing list
> >> > Wikitech-l@lists.wikimedia.org
> >> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> >>
> >> No there is not. You can of course download old versions of the software
> >> and setup your own wiki but that is a lot of effort.
> >>
> >> --
> >> bawolff
> >> ___
> >> Wikitech-l mailing list
> >> Wikitech-l@lists.wikimedia.org
> >> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> > ___
> > Wikitech-l mailing list
> > Wikitech-l@lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Previous mediawiki version test wiki

2017-09-21 Thread יגאל חיטרון 
I know this, of course. Our wiki is in group 1, and we reported a lot of
bugs in the first day. I'm talking about the case when you should file the
bug after Thursday.
Igal


On Sep 22, 2017 02:14, "Bryan Davis"  wrote:

On Thu, Sep 21, 2017 at 2:52 PM, יגאל חיטרון  wrote:
> Very well, thank you. And I thought it will not be a problem at all. But
> I'll try at least.
> -
> Hello, people! I believe you should think about a possibility of creation
a
> new test wiki, that will be a screenshot of deployments with one week
> delay. So, every Thursday, at the moment of group 2 deployment of version
> 5 in place of 4, this wiki will get 4 in place of 3. It's
> group 2, because you'll not create three new wikis, of course, so it
should
> help to most wikis. Other groups users will wait a day or two.
> Why do I think it is helpful? When filing a phab task with some new bug,
> you always want to know - is it really new, or I just did not pay
attention
> to it before? And when I do know it's a new bug, I can open both versions
> in the same time, and compare the behaviour for this bug. And also,
compare
> the console results - what exactly changed in html, in css, in js commands
> reactions. IMHO it can be a powerful helping tool for better description
of
> new bugs in phabricator. Not for developers, who remember all the changes
> in last deployment, but for users that find the bugs. Even it is heavy for
> maintenance, I think the profit is much better. Could you think about
this,
> please?
> Thank you in advance,
> Igal (User:IKhitron)

This use case is exactly the reason that group0 (testwiki, test2wiki,
testwikidatawiki, medaiwikiwiki) and group1 (non-wikipedia wikis) get
code before group2 (all wikipedias). The order of operations is just
reversed from what you are asking for. Group0 gets new release on
Tuesday, then group1 on Wednesday, and finally group2 on Thursday.

Why in this order? We want people to test and find bugs *before* they
hit the larger wikis.

Bryan
--
Bryan Davis  Wikimedia Foundation
[[m:User:BDavis_(WMF)]] Manager, Cloud Services  Boise, ID USA
irc: bd808v:415.839.6885 x6855

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Be bold with code changes

2017-09-21 Thread Brian Wolff 
Code review latency is a complicated issue which a lot has been said about.
It was a subject of a session at the last 2 dev summits (im not sure if the
last one was recorded, but there is definitely a video of the session from
the dev summit two years ago, on commons). It was an issue i used to care
very much about when it highly affected me, but now that my role shifted so
im less affected, i notice it much less and thus tend to forget about it
(unfortunately). My personal opinion is it is a structural problem of both
the wikimedia foundation and the mediawiki community, and it wont change
unless deep changes are made in our processes and expectations.

In any case, as far as (4) goes, id hope that is already the case. I
usually just do a quick glance and rapid approve if the change does not
involve code changes. If you are aware of any trivial (non code changes)
that are languishing in review, please mention them on #mediawiki or
#wikimedia-dev and they should be taken care of right away.

--
bawolff

On Thursday, September 21, 2017, Tom Bishop, Wenlin Institute <
tan...@wenlin.com> wrote:
> Wikipedia is the greatest encyclopedia in the world, due in large part to
its openness, including its "be bold" policy:
>
> https://en.wikipedia.org/wiki/Wikipedia:Be_bold
>
> The MediaWiki code is also great, and has a lot of room for improvement,
which will happen faster if code changes are treated more like wiki content
changes. While anyone can edit Wikipedia content, only a relatively small
group has authorization to approve MediaWiki code changes. This difference
may be partly justified, supposing that code changes can do more harm than
content changes. Still, there appears to be a problematic bottleneck.
Efforts to improve the code are wasted when users submit patches that just
sit and rot without being approved or rejected. I've seen this: a bug is
found and reported; a patch is submitted; nothing happens for a year and a
half; the patch gets out of date, then gets updated and submitted by
another programmer; several months later nothing has happened. Unless this
is a rare exception, problems like it seem bound to result in a lot of
wasted effort, and in programmers deciding not to risk wasting further
effort.
>
> Maybe the authorized group is too small and can't keep up with the
patches. Maybe the group is too cautious, either to authorize or reject
patches, or to invite more people to join it.
>
> How about a policy to be bolder with code changes? Possible ways: (1)
enlarge the authorized group; (2) encourage the group to approve more
changes; (3) enable more kinds of approval, such as "I haven't had time to
test this patch or study it very carefully, but at least I've read it and
I'm fairly sure it's not malicious or a security risk"; (4) be especially
quick to approve patches that only change comments or variable names,
without affecting code execution, since their potential harm is minimal,
analogous to editing wiki content.
>
> Understandably, nobody wants to be blamed for letting someone into the
authorized group who turns out to have bad judgment. If the group
nevertheless really needs to be enlarged, how can it become bolder about
letting people in? Similarly, nobody wants to be blamed for approving a
patch that turns out to cause a new bug, especially if the penalty for one
such mistake might outweigh the rewards for approving many beneficial
patches. How about increasing the rewards and decreasing the penalties?
What signs of appreciation do the gatekeepers receive for the number of
patches, or new members, they approve? Approving a bad patch, or an
inappropriate group member, once in a while but not too often, could be
viewed positively as an indication of boldness rather than recklessness.
>
> I hope these suggestions are helpful. They're offered in the spirit of
"being bold", even though my understanding of the organizational problem,
and my experience with submitting patches, are very limited.
>
> Best wishes,
>
> Tom
>
> Wenlin Institute, Inc. SPC (a Social Purpose Corporation)
> 文林研究所社会目的公司
> Software for Learning Chinese
> E-mail: wen...@wenlin.com Web: http://www.wenlin.com
> Telephone: 1-877-4-WENLIN (1-877-493-6546)
> ☯
>
>
>
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Previous mediawiki version test wiki

2017-09-21 Thread Bryan Davis 
On Thu, Sep 21, 2017 at 2:52 PM, יגאל חיטרון  wrote:
> Very well, thank you. And I thought it will not be a problem at all. But
> I'll try at least.
> -
> Hello, people! I believe you should think about a possibility of creation a
> new test wiki, that will be a screenshot of deployments with one week
> delay. So, every Thursday, at the moment of group 2 deployment of version
> 5 in place of 4, this wiki will get 4 in place of 3. It's
> group 2, because you'll not create three new wikis, of course, so it should
> help to most wikis. Other groups users will wait a day or two.
> Why do I think it is helpful? When filing a phab task with some new bug,
> you always want to know - is it really new, or I just did not pay attention
> to it before? And when I do know it's a new bug, I can open both versions
> in the same time, and compare the behaviour for this bug. And also, compare
> the console results - what exactly changed in html, in css, in js commands
> reactions. IMHO it can be a powerful helping tool for better description of
> new bugs in phabricator. Not for developers, who remember all the changes
> in last deployment, but for users that find the bugs. Even it is heavy for
> maintenance, I think the profit is much better. Could you think about this,
> please?
> Thank you in advance,
> Igal (User:IKhitron)

This use case is exactly the reason that group0 (testwiki, test2wiki,
testwikidatawiki, medaiwikiwiki) and group1 (non-wikipedia wikis) get
code before group2 (all wikipedias). The order of operations is just
reversed from what you are asking for. Group0 gets new release on
Tuesday, then group1 on Wednesday, and finally group2 on Thursday.

Why in this order? We want people to test and find bugs *before* they
hit the larger wikis.

Bryan
-- 
Bryan Davis  Wikimedia Foundation
[[m:User:BDavis_(WMF)]] Manager, Cloud Services  Boise, ID USA
irc: bd808v:415.839.6885 x6855

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Be bold with code changes

2017-09-21 Thread Tom Bishop, Wenlin Institute 
Wikipedia is the greatest encyclopedia in the world, due in large part to its 
openness, including its "be bold" policy:

https://en.wikipedia.org/wiki/Wikipedia:Be_bold

The MediaWiki code is also great, and has a lot of room for improvement, which 
will happen faster if code changes are treated more like wiki content changes. 
While anyone can edit Wikipedia content, only a relatively small group has 
authorization to approve MediaWiki code changes. This difference may be partly 
justified, supposing that code changes can do more harm than content changes. 
Still, there appears to be a problematic bottleneck. Efforts to improve the 
code are wasted when users submit patches that just sit and rot without being 
approved or rejected. I've seen this: a bug is found and reported; a patch is 
submitted; nothing happens for a year and a half; the patch gets out of date, 
then gets updated and submitted by another programmer; several months later 
nothing has happened. Unless this is a rare exception, problems like it seem 
bound to result in a lot of wasted effort, and in programmers deciding not to 
risk wasting further effort.

Maybe the authorized group is too small and can't keep up with the patches. 
Maybe the group is too cautious, either to authorize or reject patches, or to 
invite more people to join it.

How about a policy to be bolder with code changes? Possible ways: (1) enlarge 
the authorized group; (2) encourage the group to approve more changes; (3) 
enable more kinds of approval, such as "I haven't had time to test this patch 
or study it very carefully, but at least I've read it and I'm fairly sure it's 
not malicious or a security risk"; (4) be especially quick to approve patches 
that only change comments or variable names, without affecting code execution, 
since their potential harm is minimal, analogous to editing wiki content.

Understandably, nobody wants to be blamed for letting someone into the 
authorized group who turns out to have bad judgment. If the group nevertheless 
really needs to be enlarged, how can it become bolder about letting people in? 
Similarly, nobody wants to be blamed for approving a patch that turns out to 
cause a new bug, especially if the penalty for one such mistake might outweigh 
the rewards for approving many beneficial patches. How about increasing the 
rewards and decreasing the penalties? What signs of appreciation do the 
gatekeepers receive for the number of patches, or new members, they approve? 
Approving a bad patch, or an inappropriate group member, once in a while but 
not too often, could be viewed positively as an indication of boldness rather 
than recklessness.

I hope these suggestions are helpful. They're offered in the spirit of "being 
bold", even though my understanding of the organizational problem, and my 
experience with submitting patches, are very limited.

Best wishes,

Tom

Wenlin Institute, Inc. SPC (a Social Purpose Corporation)
文林研究所社会目的公司
Software for Learning Chinese
E-mail: wen...@wenlin.com Web: http://www.wenlin.com
Telephone: 1-877-4-WENLIN (1-877-493-6546)
☯




___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Wikidata revisions mail

2017-09-21 Thread יגאל חיטרון 
Thank you very much for your help.
Igal


On Sep 21, 2017 21:23, "Lydia Pintscher" 
wrote:

> On Sep 19, 2017 12:01, "יגאל חיטרון"  wrote:
>
> Hello.
> Since yesterday, I started to get a lot of letters about unexisting
> revisions in ruwiki articles. I did not changed something relevant in
> preferences, I think. A little investigation gave me the cause - these are
> edits of items that are connected by the the item of my watchlist articles
> in Wikidata. I know this is a probem in special:watchlist, this is why I do
> not turn on the wikidata in watchlist. But it's the first time I can see
> them in emails. How can I turn off these, and only these letters, please? I
> couldn't find something fit in preferences. At least, good it's on ruwiki
> only, I almost do not work there, have about a dozen of pages in watchlist,
> so I get these mails once an hour. If it would happen in my home wiki with
> thousands of pages in watchlist, it can be a letter every second. What can
> I do?
>
>
> Sorry about this. You should not be getting these emails. We are
> investigating this in https://phabricator.wikimedia.org/T174794
>
>
> Cheers
> Lydia
>
>
> Thank you,
> Igal (User:Ikhitron)
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Previous mediawiki version test wiki

2017-09-21 Thread יגאל חיטרון 
Very well, thank you. And I thought it will not be a problem at all. But
I'll try at least.
-
Hello, people! I believe you should think about a possibility of creation a
new test wiki, that will be a screenshot of deployments with one week
delay. So, every Thursday, at the moment of group 2 deployment of version
5 in place of 4, this wiki will get 4 in place of 3. It's
group 2, because you'll not create three new wikis, of course, so it should
help to most wikis. Other groups users will wait a day or two.
Why do I think it is helpful? When filing a phab task with some new bug,
you always want to know - is it really new, or I just did not pay attention
to it before? And when I do know it's a new bug, I can open both versions
in the same time, and compare the behaviour for this bug. And also, compare
the console results - what exactly changed in html, in css, in js commands
reactions. IMHO it can be a powerful helping tool for better description of
new bugs in phabricator. Not for developers, who remember all the changes
in last deployment, but for users that find the bugs. Even it is heavy for
maintenance, I think the profit is much better. Could you think about this,
please?
Thank you in advance,
Igal (User:IKhitron)


On Sep 21, 2017 23:25, "Brian Wolff"  wrote:

> Making your case here is probably best. The release engineering team are
> the people you probably have to convince, although of course anyone could
> potentially create such a wiki, in an unofficial way.
>
> Keep in mind that keeping an older version of the software running does
> introduce a maintinance burden, so you will probably have to convince
> people that it would be regularly useful and not just useful this one time.
>
> --
> bawolff
>
> On Thursday, September 21, 2017, יגאל חיטרון  wrote:
> > Thank you. Sorry to hear this. Is there some place I can suggest this and
> > explain why do I think it can be very helpful?
> > Igal
> >
> > On Sep 21, 2017 22:12, "Brian Wolff"  wrote:
> >
> >> On Thursday, September 21, 2017, יגאל חיטרון 
> >> wrote:
> >> > Hi. Sometimes after the week deployment I need to compare the new
> version
> >> > with the previous one, in some aspect. Is there a test wiki that
> always
> >> has
> >> > one version before the current?
> >> > Thank you.
> >> > Igal (User:IKhitron)
> >> > ___
> >> > Wikitech-l mailing list
> >> > Wikitech-l@lists.wikimedia.org
> >> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> >>
> >> No there is not. You can of course download old versions of the software
> >> and setup your own wiki but that is a lot of effort.
> >>
> >> --
> >> bawolff
> >> ___
> >> Wikitech-l mailing list
> >> Wikitech-l@lists.wikimedia.org
> >> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> > ___
> > Wikitech-l mailing list
> > Wikitech-l@lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Previous mediawiki version test wiki

2017-09-21 Thread Brian Wolff 
Making your case here is probably best. The release engineering team are
the people you probably have to convince, although of course anyone could
potentially create such a wiki, in an unofficial way.

Keep in mind that keeping an older version of the software running does
introduce a maintinance burden, so you will probably have to convince
people that it would be regularly useful and not just useful this one time.

--
bawolff

On Thursday, September 21, 2017, יגאל חיטרון  wrote:
> Thank you. Sorry to hear this. Is there some place I can suggest this and
> explain why do I think it can be very helpful?
> Igal
>
> On Sep 21, 2017 22:12, "Brian Wolff"  wrote:
>
>> On Thursday, September 21, 2017, יגאל חיטרון 
>> wrote:
>> > Hi. Sometimes after the week deployment I need to compare the new
version
>> > with the previous one, in some aspect. Is there a test wiki that always
>> has
>> > one version before the current?
>> > Thank you.
>> > Igal (User:IKhitron)
>> > ___
>> > Wikitech-l mailing list
>> > Wikitech-l@lists.wikimedia.org
>> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>>
>> No there is not. You can of course download old versions of the software
>> and setup your own wiki but that is a lot of effort.
>>
>> --
>> bawolff
>> ___
>> Wikitech-l mailing list
>> Wikitech-l@lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Previous mediawiki version test wiki

2017-09-21 Thread יגאל חיטרון 
Thank you. Sorry to hear this. Is there some place I can suggest this and
explain why do I think it can be very helpful?
Igal

On Sep 21, 2017 22:12, "Brian Wolff"  wrote:

> On Thursday, September 21, 2017, יגאל חיטרון 
> wrote:
> > Hi. Sometimes after the week deployment I need to compare the new version
> > with the previous one, in some aspect. Is there a test wiki that always
> has
> > one version before the current?
> > Thank you.
> > Igal (User:IKhitron)
> > ___
> > Wikitech-l mailing list
> > Wikitech-l@lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
> No there is not. You can of course download old versions of the software
> and setup your own wiki but that is a lot of effort.
>
> --
> bawolff
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Can we drop revision hashes (rev_sha1)?

2017-09-21 Thread Gergo Tisza 
On Thu, Sep 21, 2017 at 6:10 AM, Daniel Kinzler  wrote:

> Yes, we could put it into a separate table. But that table would be
> exactly as
> tall as the content table, and would be keyed to it. I see no advantage.


The advantage is that MediaWiki almost would never need to use the hash
table. It would need to add the hash for a new revision there, but table
size is not much of an issue on INSERT; other than that, only slow
operations like export and API requests which explicitly ask for the hash
would need to join on that table.
Or this primarily a disk space concern?

> Also, since content is supposed to be deduplicated (so two revisions with
> > the exact same content will have the same content_address), cannot that
> > replace content_sha1 for revert detection purposes?
>
> Only if we could detect and track "manual" reverts. And the only reliable
> way to
> do this right now is by looking at the sha1.


The content table points to a blob store which is content-addressible and
has its own deduplication mechanism, right? So you just send it the content
to store, and get an address back, and in the case of a manual revert, that
address will be one that has already been used in other content rows. Or do
you need to detect the revert before saving it?

SHA1 is not that slow.
>

For the API/Special:Export definitely not. Maybe for generating the
official dump files it might be significant? A single sha1 operation on a
modern CPU should not take more than a microsecond: there are a few hundred
operations in a decently implemented sha1 and processors are in the GHz
range. PHP benchmarks [1] also give similar values. With the 64-byte block
size, that's something like 5 hours/TB - not sure how that compares to the
dump process itself (also it's probably running on lots of cores in
parallel).


[1] http://www.spudsdesign.com/benchmark/index.php?t=hash1
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Previous mediawiki version test wiki

2017-09-21 Thread Brian Wolff 
On Thursday, September 21, 2017, יגאל חיטרון  wrote:
> Hi. Sometimes after the week deployment I need to compare the new version
> with the previous one, in some aspect. Is there a test wiki that always
has
> one version before the current?
> Thank you.
> Igal (User:IKhitron)
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

No there is not. You can of course download old versions of the software
and setup your own wiki but that is a lot of effort.

--
bawolff
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Wikidata revisions mail

2017-09-21 Thread Lydia Pintscher 
On Sep 19, 2017 12:01, "יגאל חיטרון"  wrote:

Hello.
Since yesterday, I started to get a lot of letters about unexisting
revisions in ruwiki articles. I did not changed something relevant in
preferences, I think. A little investigation gave me the cause - these are
edits of items that are connected by the the item of my watchlist articles
in Wikidata. I know this is a probem in special:watchlist, this is why I do
not turn on the wikidata in watchlist. But it's the first time I can see
them in emails. How can I turn off these, and only these letters, please? I
couldn't find something fit in preferences. At least, good it's on ruwiki
only, I almost do not work there, have about a dozen of pages in watchlist,
so I get these mails once an hour. If it would happen in my home wiki with
thousands of pages in watchlist, it can be a letter every second. What can
I do?


Sorry about this. You should not be getting these emails. We are
investigating this in https://phabricator.wikimedia.org/T174794


Cheers
Lydia


Thank you,
Igal (User:Ikhitron)
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Previous mediawiki version test wiki

2017-09-21 Thread יגאל חיטרון 
Hi. Sometimes after the week deployment I need to compare the new version
with the previous one, in some aspect. Is there a test wiki that always has
one version before the current?
Thank you.
Igal (User:IKhitron)
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] TechCom Radar, 2017-09-20

2017-09-21 Thread Daniel Kinzler 
Hello all!

Find below the minutes of the last meeting of the Technical Committee.


* TechCom Radar Newsletter: if you would like to be notified about the TechCom
radar on-wiki, subscribe at
!

* Last Call: Introduce InterruptMutexManager. While the intent seems
uncontroversial, some details may need further discussion.
.

Should no pertinent concerns remain unaddressed by October 5, this RFC will be
approved for implementation.

* Deadline for applications for the Developer Summit is approaching:


* New Special Interest Group launched: Technical Debt:


* New RFC about safely executing user-provided regular expressions. Lots of
options to discuss 

* New RFC about consolidating the rpc/RunJobs.php entry point with MW core


* Last week’s discussion about the JobQueue turned up some interesting points
particularly about de-duplication and scheduling, in particular with respect to
the upcoming switch to a EventBus/Kafka based job queue.


* A draft of the database schema for MCR is up for discussion:
 and


* Next week’s IRC discussion: Open discussion about how long we need/want to
keep support for HHVM. Thread on wikitech:
. RFC about bumping the PHP version
(for context) 

As always, the discussion will take place in the IRC channel
#wikimedia-office on Wednesday 21:00 UTC (2pm PDT, 23:00 CEST).


You can also find our meeting minutes at


See also the TechCom RFC board
.

-- 
Daniel Kinzler
Principal Platform Engineer

Wikimedia Deutschland
Gesellschaft zur Förderung Freien Wissens e.V.

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Can we drop revision hashes (rev_sha1)?

2017-09-21 Thread Daniel Kinzler 
Am 21.09.2017 um 17:18 schrieb Federico Leva:
> (Offlist)
> 
> Daniel Kinzler, 21/09/2017 17:24:
>> Hashing is a lot faster than loading the content. Since Special:Export needs 
>> to
>> load the content anyway, the extra cost of hashing is negligible.
> 
> I trust you, but really? Even when exporting 5000 revisions?

Exporting 5000 revisions is likely to time out due to the time it takes to even
load all the data. If we can load the data, we can probably also hash it in
time. SHA1 is not that slow. Hashing all 1269 PHP files in the includes
directory takes half a second of CPU time on my system (about 2 seconds wall
clock time).

Hashing does put considerable load on the CPU though (on an otherwise I/O bound
operation), so it may cause problems if a lot of people do it. But since we have
a lot more edits than exports, and every edit needs hashing, I don't think thiat
makes much of a difference either.

-- 
Daniel Kinzler
Principal Platform Engineer

Wikimedia Deutschland
Gesellschaft zur Förderung Freien Wissens e.V.

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Can we drop revision hashes (rev_sha1)?

2017-09-21 Thread Daniel Kinzler 
Am 21.09.2017 um 11:24 schrieb Federico Leva (Nemo):
> The revision hashes are also supposed to be used by at least some of the 
> import
> tools for XML dumps. The dumps would be less valuable without some way to 
> check
> their content.

While this is a typical use cacse for hashes in theory, i have never heard of
any MediaWiki related tool actually doing this.

> Generating hashes on the fly is surely not an option given
> exports can also need to happen within the time of a PHP request 
> (Special:Export
> for instance).

Hashing is a lot faster than loading the content. Since Special:Export needs to
load the content anyway, the extra cost of hashing is negligible.


If we only need the hashes in contexts where we also need the full content,
generating on the fly should work fine.

But if we need revision hashes in a list of 500 revisions returned from the API,
*that* we can't calculate on the fly. Similarly, database queries that need the
hashes to detect revisions with the same content can't use on-the-fly hashes.

-- 
Daniel Kinzler
Principal Platform Engineer

Wikimedia Deutschland
Gesellschaft zur Förderung Freien Wissens e.V.

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Mapping Hiragana and Katakana

2017-09-21 Thread Trey Jones 
>
> Well, I would expect "phonetic:" would bind with something like IPA, but
> the concept of keyword is interesting.


Finding good names for keywords is also an art. "phonetic:" came to mind
because the algorithms used to index words by pronunciation are
collectively called phonetic algorithms[1]. You could conceivably also map
to IPA, but in general the algorithms are much less detailed than IPA,
because they are trying to find a balance between inclusivity and
exclusivity in grouping similar words (a lot drop non-initial vowels, for
example), while IPA is usually much more specific.

Mapping IPA into such a system would be interesting. Say I heard someone
talk about someone named /ɡədɑfi/—hopefully that would allow me to find
Gaddafi (with his famously a hard-to-spell name). Amusingly, the character
folding on English Wikipedia maps ɡədɑfi to gadafi, which is a redirect to
Gaddafi—so IPA sometimes works now! But it wouldn't work for George /kluni/.

We have gone far afield now, but there are Phabricator tickets for advanced
search in general[2] and phonetic search specifically[3] if anyone wants to
follow up there.

[1] https://en.wikipedia.org/wiki/Phonetic_algorithm
[2] https://phabricator.wikimedia.org/T174064
[3] https://phabricator.wikimedia.org/T174705


On Thu, Sep 21, 2017 at 6:50 AM, mathieu stumpf guntz <
psychosl...@culture-libre.org> wrote:

>
>
> Le 20/09/2017 à 03:40, Trey Jones a écrit :
>
> Anyway, would it be a big deal to show the transliterated results with
>> less weight in ranking?
>
>
> Doing any special weighting would be more difficult, but they would
> already be naturally ranked lower for not being exact matches. (You can see
> this at work if you compare the results for *resume, resumé,* and *résumé*
> on English Wikipedia, for example.)
>
> Interesting to know. Thank you.
>
>
> Actually, add an option button in advanced search in any case, and just
>> limit discussion about should it be opt-in or opt-out.
>
>
> There are longer term plans for revamping advanced search capabilities, so
> if we want to go that route, it's doable, but it would definitely be on
> hold for a while. Options that have been mentioned include a special case
> keyword like "kana:オオカミ", or a more generic keyword like "phonetic:オオカミ"
> that was smart enough to know what to do with kana, but might do something
> different with other characters... but that's all at the vague ideation
> stage right now.
>
> Well, I would expect "phonetic:" would bind with something like IPA, but
> the concept of keyword is interesting.
>
>
> Thanks!
>
>
> Trey Jones
> Sr. Software Engineer, Search Platform
> Wikimedia Foundation
>
> On Tue, Sep 19, 2017 at 8:29 PM, mathieu stumpf guntz <
> psychosl...@culture-libre.org> wrote:
>
>>
>>
>> Le 19/09/2017 à 23:47, Trey Jones a écrit :
>>
>> We recently got a suggestion via Phabricator[1] to automatically map
>> between hiragana and katakana when searching on English Wikipedia and other
>> wiki projects. As an always-on feature, this isn't difficult to implement,
>> but major commercial search engines (Google.jp, Bing, Yahoo Japan,
>> DuckDuckGo, Goo) don't do that. They give different results when searching
>> for hiragana/katakana forms (for example, オオカミ/おおかみ "wolf"). They also give
>> different *numbers* of results, seeming to indicate that it's not just
>> re-ordering the same results (say, so that results in the same script are
>> ranked higher).[2] I want to know what they know that I don't!
>>
>> Does anyone have any thoughts on whether this would be useful (seems that
>> it would) and whether it would cause any problems (it must, or otherwise
>> all the other search engines would do it, right?).
>>
>> Well, maybe. Or not. Look how Duckduckgo continue to only give a
>> "country" option to filter *languages*. Now both might be complementary,
>> but personally I'm generally more interested with the later. All the more
>> when
>> I'm using a language which have no country using it as official language.
>> :)
>>
>> Anyway, would it be a big deal to show the transliterated results with
>> less
>> weight in ranking? Actually, add an option button in advanced search in
>> any
>> case, and just limit discussion about should it be opt-in or opt-out.
>>
>> Any idea why it might be different between a Japanese-language wiki and a
>> non-Japanese-language wiki? We often are more aggressive in matching
>> between characters that are not native to a given language--for example,
>> accents on Latin characters are generally ignored on English-language
>> wikis. So it might make sense to merge hiragana and katakana on
>> English-language wikis but not Japanese-language wikis.
>>
>> Thanks very much for any suggestions or information!
>> —Trey
>>
>>
>> どういたしました。
>>
>>
>>
>> [1] https://phabricator.wikimedia.org/T176197
>> [2] Details of my tests at https://phabricator.wikimedia.org/T173650#3580309
>>
>> Trey Jones
>> Sr. Software Engineer, Search Platform
>> Wikimedia Foundation
>> _

Re: [Wikitech-l] Can we drop revision hashes (rev_sha1)?

2017-09-21 Thread Daniel Kinzler 
Am 19.09.2017 um 20:48 schrieb Gergo Tisza:
> On Tue, Sep 19, 2017 at 6:42 AM, Daniel Kinzler  Can't you just split it into a separate table? Core would only need to
> touch it on insert/update, so that should resolve the performance concerns.

Yes, we could put it into a separate table. But that table would be exactly as
tall as the content table, and would be keyed to it. I see no advantage. But if
DBAs prefer a separate table with a 1:1 relation to the content table, that's
fine with me.

Note that the content table is indeed touched a lot less than the revision 
table.

> Also, since content is supposed to be deduplicated (so two revisions with
> the exact same content will have the same content_address), cannot that
> replace content_sha1 for revert detection purposes?

Only if we could detect and track "manual" reverts. And the only reliable way to
do this right now is by looking at the sha1.

-- 
Daniel Kinzler
Principal Platform Engineer

Wikimedia Deutschland
Gesellschaft zur Förderung Freien Wissens e.V.

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Mapping Hiragana and Katakana

2017-09-21 Thread mathieu stumpf guntz 



Le 20/09/2017 à 03:40, Trey Jones a écrit :


Anyway, would it be a big deal to show the transliterated results
with less weight in ranking? 



Doing any special weighting would be more difficult, but they would 
already be naturally ranked lower for not being exact matches. (You 
can see this at work if you compare the results for /resume, resumé,/ 
and /résumé/ on English Wikipedia, for example.)

Interesting to know. Thank you.


Actually, add an option button in advanced search in any case, and
just limit discussion about should it be opt-in or opt-out.


There are longer term plans for revamping advanced search 
capabilities, so if we want to go that route, it's doable, but it 
would definitely be on hold for a while. Options that have been 
mentioned include a special case keyword like "kana:オオカミ", or a more 
generic keyword like "phonetic:オオカミ" that was smart enough to know 
what to do with kana, but might do something different with other 
characters... but that's all at the vague ideation stage right now.
Well, I would expect "phonetic:" would bind with something like IPA, but 
the concept of keyword is interesting.


Thanks!


Trey Jones
Sr. Software Engineer, Search Platform
Wikimedia Foundation

On Tue, Sep 19, 2017 at 8:29 PM, mathieu stumpf guntz 
mailto:psychosl...@culture-libre.org>> 
wrote:




Le 19/09/2017 à 23:47, Trey Jones a écrit :

We recently got a suggestion via Phabricator[1] to automatically map
between hiragana and katakana when searching on English Wikipedia and other
wiki projects. As an always-on feature, this isn't difficult to implement,
but major commercial search engines (Google.jp, Bing, Yahoo Japan,
DuckDuckGo, Goo) don't do that. They give different results when searching
for hiragana/katakana forms (for example, オオカミ/おおかみ "wolf"). They also give
different *numbers* of results, seeming to indicate that it's not just
re-ordering the same results (say, so that results in the same script are
ranked higher).[2] I want to know what they know that I don't!

Does anyone have any thoughts on whether this would be useful (seems that
it would) and whether it would cause any problems (it must, or otherwise
all the other search engines would do it, right?).

Well, maybe. Or not. Look how Duckduckgo continue to only give a
"country" option to filter *languages*. Now both might be
complementary,
but personally I'm generally more interested with the later. All
the more when
I'm using a language which have no country using it as official
language. :)

Anyway, would it be a big deal to show the transliterated results
with less
weight in ranking? Actually, add an option button in advanced
search in any
case, and just limit discussion about should it be opt-in or opt-out.


Any idea why it might be different between a Japanese-language wiki and a
non-Japanese-language wiki? We often are more aggressive in matching
between characters that are not native to a given language--for example,
accents on Latin characters are generally ignored on English-language
wikis. So it might make sense to merge hiragana and katakana on
English-language wikis but not Japanese-language wikis.

Thanks very much for any suggestions or information!
—Trey


どういたしました。

[1] https://phabricator.wikimedia.org/T176197
 [2] Details of my
tests at https://phabricator.wikimedia.org/T173650#3580309


Trey Jones
Sr. Software Engineer, Search Platform
Wikimedia Foundation
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org

https://lists.wikimedia.org/mailman/listinfo/wikitech-l






___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Can we drop revision hashes (rev_sha1)?

2017-09-21 Thread Federico Leva (Nemo) 
The revision hashes are also supposed to be used by at least some of the 
import tools for XML dumps. The dumps would be less valuable without 
some way to check their content. Generating hashes on the fly is surely 
not an option given exports can also need to happen within the time of a 
PHP request (Special:Export for instance).


Nemo

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] HHVM vs. Zend divergence

2017-09-21 Thread Giuseppe Lavagetto 
On Wed, Sep 20, 2017 at 10:56 PM, Brion Vibber 
wrote:

> On Mon, Sep 18, 2017 at 1:58 PM, Max Semenik 
> wrote:
>
> >
> > 3) Revert WMF to Zend and forget about HHVM. This will result in
> > performance degradation, however it will not be that dramatic: when we
> > upgraded, we switched to HHVM from PHP 5.3 which was really outdated,
> while
> > 5.6 and 7 provided nice performance improvements.
> >
>
> Migrating WMF's implementation to PHP 7 is probably the way to go. I leave
> it up to ops to figure out how to make the change. :)
>


I think this is the more viable option too, mostly not to cause huge issues
to non-Wikimedia users. The ease of installation and operation of PHP on
most platforms compared to HHVM is incomparable. Just to make an example, I
don't remember having to check the code of the VM to understand what an ini
setting does with PHP (or even PHP-fpm), while with HHVM that has been a
recurring pain, as options come and go without any warning between
versions, not to mention the online docs that can even be plainly
misleading. At the moment, there is no doubt PHP is a much friendlier
environment for any third-party wiki.


But I think there is other value in going the PHP7 way; I had actually
pitched the idea of switching to PHP7 for some time now, for various
reasons, namely:
- We use HHVM differently than what Facebook does. For instance, we use the
fcgi server mode, and not the pure http one, and we don't run in repoAuth
mode. This has brought us new bugs to solve every time we upgrade, as there
is no battle testing in production for those code patterns at scale before
we use it.
- The recent, prolonged difficulty of interaction with the FLOSS community,
although acknowledged by the HHVM team as something they're willing to fix,
is worrying in itself.
- PHP7 has shown performance comparable to HHVM for most PHP shops that
migrated. So the single most compelling reason for which we migrated
(performance) might not be a factor anymore. Using a runtime readily
available (and security-patched) by the upstream distribution would make
the ops team lives easier as well.

As for the actual migration:

I don't think there is any need to panic or rush to a decision, but the
timeline is pretty set: by the end of 2018, when official support for HHVM
3.24 will end, any migration should be well underway within the WMF
infrastructure. I expect a migration from HHVM to PHP 7 to be a less
formidable undertaking than the switch from PHP 5.3 to HHVM  - we did repay
a good deal t of the tech debt in the Wikimedia Foundation installation
back then, and we won't have to change radically the way we serve
MediaWiki, as PHP 7 works as a FastCGI server as well. Still, it will take
time and resources, and it needs to be planned in advance.

One important consequence of the announcement for Wikimedia is that we
won't be able to use any version of MediaWiki not compatible with PHP 5.x
until we transition to PHP7 (unless we decide to support both HHVM and
PHP7). This might be important in steering the timing of the change in
MediaWiki itself.

Cheers,

Giuseppe

-- 
Giuseppe Lavagetto, Ph.d.
Senior Technical Operations Engineer, Wikimedia Foundation
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l