Re: [Wikitech-l] HHVM vs. Zend divergence

[Tim Starling]

On 19/09/17 12:30, Legoktm wrote:
> Hi,
> 
> On 09/18/2017 05:13 PM, Tim Starling wrote:
>> * The plan to also drop PHP 5 compatibility, on a short timeline (1 year).
>> * Rather than "drifting away" from PHP, their top priority plans
>> include removing core language features like references and destructors.
> 
> On Reddit[1], a member of the HHVM team clarified they plan on dropping
> support for destructors *from Hack* soon. (Not that I think it really
> makes any difference in what our long-term plan should be.)

It's unclear how much difference that will make, since they are clear
about wanting to make HHVM be purely a Hack runtime. They'll carry on
supporting Composer and PHPUnit, but only until Hack has "its own
ecosystem of core frameworks".

They "will not be targeting PHP software beyond such libraries after
the 3.24 release", which presumably means that they will no longer run
MediaWiki or PHP unit tests against HHVM.

Also, they said that they want to remove destructors in order to
eliminate the performance overhead of reference counting, and I don't
think it is possible to get that performance benefit unless you remove
reference counts from the VM entirely. Maybe removing them from the
Hack language will be a first step, but we can't expect them to keep
them in the VM in the longer term.

-- Tim Starling


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] HHVM vs. Zend divergence

[Legoktm]

Hi,

On 09/18/2017 05:13 PM, Tim Starling wrote:
> * The plan to also drop PHP 5 compatibility, on a short timeline (1 year).
> * Rather than "drifting away" from PHP, their top priority plans
> include removing core language features like references and destructors.

On Reddit[1], a member of the HHVM team clarified they plan on dropping
support for destructors *from Hack* soon. (Not that I think it really
makes any difference in what our long-term plan should be.)

[1] https://www.reddit.com/r/PHP/comments/70wtky/the_future_of_hhvm/dn6skdn/

-- Legoktm

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] HHVM vs. Zend divergence

[Chad]

On Mon, Sep 18, 2017 at 5:44 PM Stas Malyshev 
wrote:

> Hi!
>
> > * Rather than "drifting away" from PHP, their top priority plans
> > include removing core language features like references and destructors.
>
> Wow. I can see why they're doing it (those are sources of most
> complications ans security issues in the language, references being
> especially weird and tricky). But dropping those would certainly mean
> very heavy incompatibility with PHP, by which point it'd be completely
> separate language. Which probably excludes Max's #2 from consideration
> altogether.
>
> > Actually, I think a year is a pretty short time for ops to switch to
> > PHP 7. I think we need to decide on this pretty much immediately.
>
> Should it be on the TechCom agenda and should we have some public
> discussion on IRC in RFC format for this soon?
>
>
I see zero reason for us to go through all the formalities, unless we want
to really. I have yet to see anyone (on list, or on IRC anywhere at all
today) where anyone suggested (2) was a good idea at all. It's a
horrifically bad idea. I don't consider it remotely viable and would do
everything possible to veto such a move.

(1) is impossible as a long-term goal.

So this basically means we're going the route of (3) which is the only way
we can actually expect people to use MediaWiki outside of Wikimedia. And
considering we never really implemented any HHVM/Hack specific features (as
far as I know) it means there's no reason we have to continue to support
HHVM at all once WMF has moved off of it.

It's been a fun experiment for the past couple of years, but it's time to
move on.

-Chad
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] HHVM vs. Zend divergence

[Stas Malyshev]

Hi!

> * Rather than "drifting away" from PHP, their top priority plans
> include removing core language features like references and destructors.

Wow. I can see why they're doing it (those are sources of most
complications ans security issues in the language, references being
especially weird and tricky). But dropping those would certainly mean
very heavy incompatibility with PHP, by which point it'd be completely
separate language. Which probably excludes Max's #2 from consideration
altogether.

> Actually, I think a year is a pretty short time for ops to switch to
> PHP 7. I think we need to decide on this pretty much immediately.

Should it be on the TechCom agenda and should we have some public
discussion on IRC in RFC format for this soon?

-- 
Stas Malyshev
smalys...@wikimedia.org

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Categories in RDF/WDQS

[Stas Malyshev]

Hi!

I'd like to announce that the category tree of certain wikis is now
available as RDF dump and in Wikidata Query Service.

More documentation is at:
https://www.mediawiki.org/wiki/Wikidata_query_service/Categories
which I will summarize shortly below.

The dumps are located at
https://dumps.wikimedia.org/other/categoriesrdf/. You can use these
dumps any way you wish, data format is described at the link above[1].

The same dump is loaded into "categories" namespace in WDQS, which can
be queried by
https://query.wikidata.org/bigdata/namespace/categories/sparql?query=SPARQL.
Sorry, no GUI support yet (probably will happen later). See example in
the docs[2].

These datasets are not updated automatically yet, so they'll be up to
date roughly for the date of the latest dump. Hopefully soon it will be
automated and then the datasets will be updated daily.

The list of currently supported wikis is here:
https://noc.wikimedia.org/conf/categories-rdf.dblist - these are
basically all 1M+ wikis and couple more that I added for various
reasons. If you have a good candidate wiki to add, please tell me or
write on the talk page for the document above.

Please note this is only the first step for the project, so there might
still be some rough edges. I am announcing it early since I think it
would be useful for people to look at the dumps and SPARQL endpoint and
see if something is missing or does not work properly, and share ideas
on how it can be used.

We plan eventually to use it for search improvement[3] - this work is
still in progress.

As always, we welcome any comments and suggestions.

[1]
https://www.mediawiki.org/wiki/Wikidata_query_service/Categories#Data_format
[2]
https://www.mediawiki.org/wiki/Wikidata_query_service/Categories#Accessing_the_data
[3] https://phabricator.wikimedia.org/T165982

Thanks,
-- 
Stas Malyshev
smalys...@wikimedia.org

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] HHVM vs. Zend divergence

[Tim Starling]

On 19/09/17 06:58, Max Semenik wrote:
> Today, the HHVM developers made an announcement[1] that they have plans of
> ceasing to maintain 100% PHP7 compatibility and concentrating on Hack
> instead.

The HHVM team did tell us privately that they were planning on
changing their strategy, basically as you describe it above. The
surprising things for me in this announcement were:

* The plan to also drop PHP 5 compatibility, on a short timeline (1 year).
* Rather than "drifting away" from PHP, their top priority plans
include removing core language features like references and destructors.

> While this does not mean that we need to take an action immediately,
> eventually we will have to decide something. 

Actually, I think a year is a pretty short time for ops to switch to
PHP 7. I think we need to decide on this pretty much immediately.

> 3) Revert WMF to Zend and forget about HHVM. This will result in
> performance degradation, however it will not be that dramatic: when we
> upgraded, we switched to HHVM from PHP 5.3 which was really outdated, while
> 5.6 and 7 provided nice performance improvements.
> 
> I personally think that 3) is the only viable option in the long run. What
> do you think?

Yes, I think it's the only viable option.

I'll run a benchmark, but I don't see how it could influence the
decision. It'll be more for capacity planning.

-- Tim Starling


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] HHVM vs. Zend divergence

[Jeroen De Dauw]

Hey,

Going with option 2 would be a rather big f***-*** to the non-WMF part of
the MediaWiki community. Furthermore, it would limit usage of common PHP
libraries, which sooner or later will end up using features not available
in HHVM. This, together with the already outlined reasons, makes me concur
with HHVM not being a viable long term option.

If only WMF was running PHP 7.1 already... guess I'll have more luck asking
for a pony though :)

Cheers

--
Jeroen De Dauw | https://entropywins.wtf | https://keybase.io/jeroendedauw
Software craftsmanship advocate
~=[,,_,,]:3
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Is it possible to edit scribunto data module content through template edit popup of visual editor?

[mathieu stumpf guntz]

Well, I have investigated a bit, and so far the only reference where I 
found a description of saving/retrieving data from a Scribunto module is 
SemanticScribunto.


https://github.com/SemanticMediaWiki/SemanticScribunto
https://upload.wikimedia.org/wikipedia/mediawiki/7/75/EMWCon_Spring_2017_-_Introducing_SemanticScribunto_Extension.pdf

It's build on Semantic Mediawiki, well, I don't know much about that. 
All that seems interesting, although a far more large than what I was 
looking for, at least to begin with. What I would like is a way to 
quickly prototype and drop some trials. This extension seems fine but it 
already requires to install extension, so it would be more difficult to 
put that on an existing wiki so I can have feedback on prototypes. Maybe 
the least resistance path would be to install a mediawiki instance on 
toolforge…



Le 17/09/2017 à 14:05, mathieu stumpf guntz a écrit :

Saluton ĉiuj kundisvolvantoj,

I think the subject summarize it all, so here are more details on what 
I'm trying to do and what I'm looking for.


# Context

You might skip this section if you are not interested in contextual 
verbiage. If you would like to react to anything stated in this 
section, please change the email subject to reflect that.


So I'm currently meditating ways to improve factorization of knowledge 
stored in Wiktionary.


I'm taking a multi-approach experimentation there. On the one hand, I 
just began a Wikiversity project 
 
(in French) to establish a specification of how a DBMS should be 
structured to be useful for Wiktionaries. It mainly emerged from my 
point of view that the current data model 
 
proposed for the wikidata for wiktionary 
 does not fit needs 
of Wiktionary contributors. I did made some alternative proposals 
, 
and tried to gather a first feedback from the French wiktionary 
 
on this model too, which led me to the creation Wikiversity research 
project because I was pointed to the lack of "specify extensively the 
needs before you model".


Now, on an other hand, I'm also trying to factorize some data within 
the Wikitionary with current available tools. One driving topic for 
that is fixing gender gap 
, 
and more broadly inflection-form gap. That is a feminine form will 
generally be summarized in a laconic "feminine form of *some-term*", 
rather than being treated as an entry of it's own. That's all the more 
problematic in cases where a word only share a subset of relevant 
definitions depending on which gender(/inflection-form) it applies to.


# What I'm trying to do

I am trying to factorize data which pertains to several 
inflection-forms. This way each form can use it to build a stand-alone 
article about a term. The current approach tends to be gathering 
everything under a single lemme, although some statements will only 
pertains to some specific forms.


So far I experimented with transclusion of subpages to share 
definitions, examples and so on between inflection-forms. Well, from a 
consultation point of view it works. But from an editing point of 
view, it's all but fine.


What I would think interesting, is to store this data in a scribunto 
data module (at least for now), and enable user to change them while 
editing an lexical entry article. That might be, when using the visual 
editor, through something like a model popup. Wikitext editors will 
probably be skilled enough to edit the relevant module, but for the 
sake of convenience, it might be interesting to allow to give a 
parameter to the model, which would at publishing time modify the data 
module and remove the parameter from the wikitext generated.


Let's take an example to make a bit clearer. Let's take the French 
pair "contributeur/contributrice". In both article, I would like that 
the definition could be generated from transclusion with something 
like 
{{definition|vocable=contributrice|lang=French|gloss=contributor}}. 
Note that this template might, by default, take into account the name 
of the calling page, thus avoiding the "vocable" parameter. Also, the 
lang would be required in contributrice, as this is a vocable which 
exist in at least in French and Italian. But it would not be required 
in "contributeur", nor "contributore". Finaly gloss is a string whose 
purpose is to distinguish a given term in case of homonymy. When no 
homonym exist, it might be skiped. So in "contributeur", one might 
simply use 

Re: [Wikitech-l] HHVM vs. Zend divergence

[Stas Malyshev]

Hi!

> I can say that PHP 7 locally runs unit tests significantly faster than PHP
> 5.6, although that's not really a representative workload for running a
> website.

PHP 7 is faster than 5.6, and probably be on almost any workload, from
my experience (the degree of speedup would vary of course). As for
comparison with hhvm, I heard various reports, but I think spending some
time on seriously testing it (I mean creating proper production setup,
and directing either captured/replayed or real traffic to it) is the
best way to go.
-- 
Stas Malyshev
smalys...@wikimedia.org

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] HHVM vs. Zend divergence

[Stas Malyshev]

Hi!

> Other tools we are using, such as Phabricator, will also be following HHVM
> to Hack (presumably).  

I am not sure this is the case. Mozilla recently declared they want to
use Phabricator[1], but I heard no mention about HHVM. Which makes me
think that one stays on PHP. Also, Phabricator is now independent from
Facebook, afaik, since it's developers have separate company, Phacility.

[1] https://wiki.mozilla.org/Phabricator

-- 
Stas Malyshev
smalys...@wikimedia.org

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] HHVM vs. Zend divergence

[Stas Malyshev]

Hi!

> 1) Continue requiring that MediaWiki uses a common set of HHVM and Zend
> PHP. This, however, is a dead end and will make things progressively harder
> as the implementations will diverge and various Composer libraries we use
> will start requiring some Zend-specific features.

This will probably ultimately happen, but given the PHP version stats,
e.g. here:
https://seld.be/notes/php-versions-stats-2017-1-edition

I think we have several years at least before that starts becoming an
issue. Realistically, if you write distributable PHP code now, targeting
7.1 gets you only 17% of the users, so you'd do either 7.0 (gets you
about half) or more likely even 5.6. Extending this trend (I know,
dangerous, but let's assume) if 7.2 is released somewhere around Dec
2017-Jan 2018, 7.3 would probably not happen before around 2019. If that
would have features not supported in HHVM, that means we'd have to worry
around 2021 when people would start releasing components targeting it.
So we have about 3 years to get the solution - *if* 7.3 has features not
supported by HHVM.

Note that this statistics is for Composer users, which means it is
probably skewed towards modern versions, since people using PHP 5.3
probably don't use Composer too much in general. OTOH, since we do use
Composer, that appears to be appropriate for our case.

> 2) Declare our loyalty to HHVM. This will result in most of our current
> users being unable to upgrade, eventually producing what amounts to a
> WMF-only product and lots of installations with outdated MediaWiki having
> security holes. At least we will be able to convert to Hack eventually.
> This is a very clean-cut case of vendor lock-in though, and if Facebook
> decides to switch their code base to something shinier, we'll be deep in
> trouble.

I don't think this is a good idea, for reasons that seem obvious to me
(but I can elaborate if necessary).

> 3) Revert WMF to Zend and forget about HHVM. This will result in
> performance degradation, however it will not be that dramatic: when we
> upgraded, we switched to HHVM from PHP 5.3 which was really outdated, while
> 5.6 and 7 provided nice performance improvements.

I think we should evaluate 7.1 or 7.2 (provided we don't have any
runtime issues with them) and see how performance looks like ASAP (with
opcache, of course). Of there's some help needed, or there are some
specific issues that are blockers, I think Zend team would be glad to
talk to us. If needed, I could probably help with establishing the
contacts.
-- 
Stas Malyshev
smalys...@wikimedia.org

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] HHVM vs. Zend divergence

[Max Semenik]

On Mon, Sep 18, 2017 at 2:33 PM, C. Scott Ananian 
wrote:

> Other tools we are using, such as Phabricator, will also be following HHVM
> to Hack (presumably).


To the contrary, Phabricator has never supported HHVM.

-- 
Best regards,
Max Semenik ([[User:MaxSem]])
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] HHVM vs. Zend divergence

[C. Scott Ananian]

Other tools we are using, such as Phabricator, will also be following HHVM
to Hack (presumably).  Facebook, as the largest (by engineering budget)
production user of PHP, will certainly have an outsized influence on the
direction of the platform and surrounding ecosystem.  If we follow path #3
we're probably also committing to supporting Zend development long-term as
the primary production user.
  --scott

On Mon, Sep 18, 2017 at 5:26 PM, Brian Wolff  wrote:

> On Monday, September 18, 2017, Max Semenik  wrote:
> > Today, the HHVM developers made an announcement[1] that they have plans
> of
> > ceasing to maintain 100% PHP7 compatibility and concentrating on Hack
> > instead.
> >
> > While this does not mean that we need to take an action immediately,
> > eventually we will have to decide something. As I see it, our options
> are:
> >
> > 1) Continue requiring that MediaWiki uses a common set of HHVM and Zend
> > PHP. This, however, is a dead end and will make things progressively
> harder
> > as the implementations will diverge and various Composer libraries we use
> > will start requiring some Zend-specific features.
> >
> > 2) Declare our loyalty to HHVM. This will result in most of our current
> > users being unable to upgrade, eventually producing what amounts to a
> > WMF-only product and lots of installations with outdated MediaWiki having
> > security holes. At least we will be able to convert to Hack eventually.
> > This is a very clean-cut case of vendor lock-in though, and if Facebook
> > decides to switch their code base to something shinier, we'll be deep in
> > trouble.
> >
> > 3) Revert WMF to Zend and forget about HHVM. This will result in
> > performance degradation, however it will not be that dramatic: when we
> > upgraded, we switched to HHVM from PHP 5.3 which was really outdated,
> while
> > 5.6 and 7 provided nice performance improvements.
> >
> > I personally think that 3) is the only viable option in the long run.
> What
> > do you think?
> >
> > 
> > [1] http://hhvm.com/blog/2017/09/18/the-future-of-hhvm.html
> >
> > --
> > Best regards,
> > Max Semenik ([[User:MaxSem]])
> > ___
> > Wikitech-l mailing list
> > Wikitech-l@lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
> Well i agree that 3 seems likely the best long term option, we will
> probably be doing 1 in the short term. However I think it would be prudent
> to wait and see how things turn out in the short term before comitting to
> any path. The landscape can still shift quite a lot before the time comes
> when its impractical to continue doing 1.
>
> --
> bawolff
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>



-- 
(http://cscott.net)
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] HHVM vs. Zend divergence

[Brian Wolff]

On Monday, September 18, 2017, Max Semenik  wrote:
> Today, the HHVM developers made an announcement[1] that they have plans of
> ceasing to maintain 100% PHP7 compatibility and concentrating on Hack
> instead.
>
> While this does not mean that we need to take an action immediately,
> eventually we will have to decide something. As I see it, our options are:
>
> 1) Continue requiring that MediaWiki uses a common set of HHVM and Zend
> PHP. This, however, is a dead end and will make things progressively
harder
> as the implementations will diverge and various Composer libraries we use
> will start requiring some Zend-specific features.
>
> 2) Declare our loyalty to HHVM. This will result in most of our current
> users being unable to upgrade, eventually producing what amounts to a
> WMF-only product and lots of installations with outdated MediaWiki having
> security holes. At least we will be able to convert to Hack eventually.
> This is a very clean-cut case of vendor lock-in though, and if Facebook
> decides to switch their code base to something shinier, we'll be deep in
> trouble.
>
> 3) Revert WMF to Zend and forget about HHVM. This will result in
> performance degradation, however it will not be that dramatic: when we
> upgraded, we switched to HHVM from PHP 5.3 which was really outdated,
while
> 5.6 and 7 provided nice performance improvements.
>
> I personally think that 3) is the only viable option in the long run. What
> do you think?
>
> 
> [1] http://hhvm.com/blog/2017/09/18/the-future-of-hhvm.html
>
> --
> Best regards,
> Max Semenik ([[User:MaxSem]])
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Well i agree that 3 seems likely the best long term option, we will
probably be doing 1 in the short term. However I think it would be prudent
to wait and see how things turn out in the short term before comitting to
any path. The landscape can still shift quite a lot before the time comes
when its impractical to continue doing 1.

--
bawolff
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Discovery Weekly Update for the week starting 2017-09-11

[Chris Koerner]

Hello,
Here are the updates from the Discovery team for last week. As always,
feedback and questions are welcome.

Reminder: There is a new way to follow these weekly updates.You can
subscribe to recieve on-wiki (or opt-in email) notifications of the
Discovery weekly update. Subscribe to be notified.

https://www.mediawiki.org/wiki/Newsletter:Discovery_Weekly

==Discussions==

=== Search ===
* Gehel did quite a bit of work to implement the new version of
Logstash on our servers, thanks for the help, RobH! [0]
* David, Gehel and Moritz updated the elasticsearch deployment plugins
to use debian packages instead of salt [1]
* Stas and David fixed an issue where Wikidata Elastic search drops
results with matches with different language labels (e.g., you search
in English but Spanish label matches) [2]
* The explore similar test for language links A/B test was turned on
Sep 14 and will run for a week. [3] [4]

=== Analysis  ===
* Mikhail finished up a new dashboard metric for dwell time on SERP [5] [6]
* Chelsy finished up the final analyzation of the results of the
swap2and3 search test and it's up on Commons. [7] [8]

[0] https://phabricator.wikimedia.org/T175045
[1] https://phabricator.wikimedia.org/T158560
[2] https://phabricator.wikimedia.org/T173231
[3] https://phabricator.wikimedia.org/T175647
[4] https://phabricator.wikimedia.org/T175648
[5] https://discovery.wmflabs.org/metrics/#spr_surv
[6] https://phabricator.wikimedia.org/T170468
[7] https://phabricator.wikimedia.org/T136017
[8] https://commons.wikimedia.org/wiki/File:Swap2and3_Search_Test_Analysis.pdf



The archive of all past updates can be found on MediaWiki.org:

https://www.mediawiki.org/wiki/Discovery/Status_updates

Interested in getting involved? See tasks marked as "Easy" or
"Volunteer needed" in Phabricator.

[1] https://phabricator.wikimedia.org/maniphest/query/qW51XhCCd8.7/#R
[2] https://phabricator.wikimedia.org/maniphest/query/5KEPuEJh9TPS/#R


Yours,
Chris Koerner
Community Liaison
Wikimedia Foundation

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] HHVM vs. Zend divergence

[Brad Jorsch (Anomie)]

On Mon, Sep 18, 2017 at 4:58 PM, Max Semenik  wrote:

> 3) Revert WMF to Zend and forget about HHVM. This will result in
> performance degradation, however it will not be that dramatic: when we
> upgraded, we switched to HHVM from PHP 5.3 which was really outdated,
> while 5.6 and 7 provided nice performance improvements.
>

In particular, I've heard good things about PHP 7 performance. Someone less
lazy than I am at 5pm might want to do some research on that though.

I can say that PHP 7 locally runs unit tests significantly faster than PHP
5.6, although that's not really a representative workload for running a
website.


-- 
Brad Jorsch (Anomie)
Senior Software Engineer
Wikimedia Foundation
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] HHVM vs. Zend divergence

[C. Scott Ananian]

On Mon, Sep 18, 2017 at 4:58 PM, Max Semenik  wrote:

> security holes. At least we will be able to convert to Hack eventually.
> This is a very clean-cut case of vendor lock-in though, and if Facebook
> decides to switch their code base to something shinier, we'll be deep in
> trouble.
>

I actually predicted this and wrote up my suggestions here:
https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2018/Writing_Tips/Examples#Example_of_a_.22medium_level.22_position_statement

I will *not* be making this my position statement at the dev summit -- only
one per person, so many positions to take, this one didn't make the cut! --
but I would like to strongly encourage someone to submit a position
statement relating to this issue (could be any of max's three positions) to
the dev summit.  I think it is an important issue for our foundation and
community to discuss; certainly very relevant to the next 15 years of the
project.
  --scott
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] HHVM vs. Zend divergence

[Paladox]

 I think option 2 will be impossible. As that will decrease the user base of 
mediawiki by a lot.
I think we should go with option 3 as php 7 has had increased in performances.  
  On Monday, 18 September 2017, 22:03:20 BST, James Hare  
wrote:  
 
 On Sep 18, 2017, at 1:58 PM, Max Semenik  wrote:
> 
> Today, the HHVM developers made an announcement[1] that they have plans of
> ceasing to maintain 100% PHP7 compatibility and concentrating on Hack
> instead.
> 
> While this does not mean that we need to take an action immediately,
> eventually we will have to decide something. As I see it, our options are:
> 
> 1) Continue requiring that MediaWiki uses a common set of HHVM and Zend
> PHP. This, however, is a dead end and will make things progressively harder
> as the implementations will diverge and various Composer libraries we use
> will start requiring some Zend-specific features.
> 
> 2) Declare our loyalty to HHVM. This will result in most of our current
> users being unable to upgrade, eventually producing what amounts to a
> WMF-only product and lots of installations with outdated MediaWiki having
> security holes. At least we will be able to convert to Hack eventually.
> This is a very clean-cut case of vendor lock-in though, and if Facebook
> decides to switch their code base to something shinier, we'll be deep in
> trouble.
> 
> 3) Revert WMF to Zend and forget about HHVM. This will result in
> performance degradation, however it will not be that dramatic: when we
> upgraded, we switched to HHVM from PHP 5.3 which was really outdated, while
> 5.6 and 7 provided nice performance improvements.

Do we have performance benchmarks on 5.3 vs. HHVM vs. 5.6 vs. 7?

> 
> I personally think that 3) is the only viable option in the long run. What
> do you think?
> 
> 
> [1] http://hhvm.com/blog/2017/09/18/the-future-of-hhvm.html
> 
> -- 
> Best regards,
> Max Semenik ([[User:MaxSem]])
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] HHVM vs. Zend divergence

[James Hare]

On Sep 18, 2017, at 1:58 PM, Max Semenik  wrote:
> 
> Today, the HHVM developers made an announcement[1] that they have plans of
> ceasing to maintain 100% PHP7 compatibility and concentrating on Hack
> instead.
> 
> While this does not mean that we need to take an action immediately,
> eventually we will have to decide something. As I see it, our options are:
> 
> 1) Continue requiring that MediaWiki uses a common set of HHVM and Zend
> PHP. This, however, is a dead end and will make things progressively harder
> as the implementations will diverge and various Composer libraries we use
> will start requiring some Zend-specific features.
> 
> 2) Declare our loyalty to HHVM. This will result in most of our current
> users being unable to upgrade, eventually producing what amounts to a
> WMF-only product and lots of installations with outdated MediaWiki having
> security holes. At least we will be able to convert to Hack eventually.
> This is a very clean-cut case of vendor lock-in though, and if Facebook
> decides to switch their code base to something shinier, we'll be deep in
> trouble.
> 
> 3) Revert WMF to Zend and forget about HHVM. This will result in
> performance degradation, however it will not be that dramatic: when we
> upgraded, we switched to HHVM from PHP 5.3 which was really outdated, while
> 5.6 and 7 provided nice performance improvements.

Do we have performance benchmarks on 5.3 vs. HHVM vs. 5.6 vs. 7?

> 
> I personally think that 3) is the only viable option in the long run. What
> do you think?
> 
> 
> [1] http://hhvm.com/blog/2017/09/18/the-future-of-hhvm.html
> 
> -- 
> Best regards,
> Max Semenik ([[User:MaxSem]])
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] HHVM vs. Zend divergence

[Max Semenik]

Today, the HHVM developers made an announcement[1] that they have plans of
ceasing to maintain 100% PHP7 compatibility and concentrating on Hack
instead.

While this does not mean that we need to take an action immediately,
eventually we will have to decide something. As I see it, our options are:

1) Continue requiring that MediaWiki uses a common set of HHVM and Zend
PHP. This, however, is a dead end and will make things progressively harder
as the implementations will diverge and various Composer libraries we use
will start requiring some Zend-specific features.

2) Declare our loyalty to HHVM. This will result in most of our current
users being unable to upgrade, eventually producing what amounts to a
WMF-only product and lots of installations with outdated MediaWiki having
security holes. At least we will be able to convert to Hack eventually.
This is a very clean-cut case of vendor lock-in though, and if Facebook
decides to switch their code base to something shinier, we'll be deep in
trouble.

3) Revert WMF to Zend and forget about HHVM. This will result in
performance degradation, however it will not be that dramatic: when we
upgraded, we switched to HHVM from PHP 5.3 which was really outdated, while
5.6 and 7 provided nice performance improvements.

I personally think that 3) is the only viable option in the long run. What
do you think?


[1] http://hhvm.com/blog/2017/09/18/the-future-of-hhvm.html

-- 
Best regards,
Max Semenik ([[User:MaxSem]])
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Can we drop revision hashes (rev_sha1)?

[Danny B.]

-- Původní e-mail --
Od: Dan Andreescu 
Komu: Wikimedia developers 
Datum: 18. 9. 2017 16:26:18
Předmět: Re: [Wikitech-l] Can we drop revision hashes (rev_sha1)? 
"So, as things stand, rev_sha1 in the database is used for:

1. the XML dumps process and all the researchers depending on the XML dumps
(probably just for revert detection)
2. revert detection for libraries like python-mwreverts [1]
3. revert detection in mediawiki history reconstruction processes in Hadoop
(Wikistats 2.0)
4. revert detection in Wikistats 1.0
5. revert detection for tools that run on labs, like Wikimetrics
?. I think Aaron also uses rev_sha1 in ORES, but I can't seem to find the
latest code for that service

If you think about this list above as a flow of data, you'll see that
rev_sha1 is replicated to xml, labs databases, hadoop, ML models, etc. So
removing it and adding it back downstream from the main mediawiki database
somewhere, like in XML, cuts off the other places that need it. That means
it must be available either in the mediawiki database or in some other
central database which all those other consumers can pull from.
"



I use rev_sha1 on replicas to check the consistency of modules, templates or
other pages (typically help) which should be same between projects (either 
within one language or even crosslanguage, if the page is not language 
dependent). In other words to detect possible changes in them and syncing 
them.




Also, I haven't noticed it mentioned in the thread: Flow also notices users 
on reverts, but IDK whether it uses rev_sha1 or not. So I'm rather 
mentioning it.







Kind regards







Danny B.


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Content Translation office hour and online meeting on September 20, 2017 (Wednesday) at 1300 UTC

[Runa Bhattacharjee]

On Mon, Sep 18, 2017 at 2:51 PM, mathieu stumpf guntz <
psychosl...@culture-libre.org> wrote:
>
> Did you consider using an other solution, such as the jisti
>  FLOSS?
>

Hello,

We have gotten some mixed reviews about this, and unfortunately did not
have enough time to evaluate it for the upcoming session. We will
definitely explore it for the next session.

Thanks for the suggestion.

regards
Runa

-- 
Engineering Manager, Global Collaboration (Contributors)
Wikimedia Foundation
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] How to add custom styles to a page?

[Kaartic Sivaraam]

On Monday 18 September 2017 08:54 PM, Brad Jorsch (Anomie) wrote:

On Sat, Sep 16, 2017 at 5:24 PM, Gergo Tisza  wrote:


You could also just put it into User:Kaartic/common.css and wrap it in a
.page-User_Kaartic selector, that is less restrictive.


Although that will only apply the CSS for you while you're logged in, no
one else will see it.

Yeah, that was the reason I wanted to know if there was some other way 
to add
custom styles to a page. I find it useless to have styles (e.g., styles 
needed by

layout-grid) that make things look nice only to me but ugly for others.

---
Kaartic

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] How to add custom styles to a page?

[Kaartic Sivaraam]

On Sat, 2017-09-16 at 10:46 -0400, Brian Wolff wrote:

> Im assuming here that you cannot just edit mediawiki:common.css
> 
> 

That's right.


> What you are looking for is TemplateStyles. This is not enabled on english
> wikipedia yet, but should be coming there soon (I think). It would allow
> you to use @media styles for your pages. See
> https://www.mediawiki.org/wiki/Help:TemplateStyles  for more details.
> 
> Templatestyles does not let you use css variables (the properties starting

> with -- and the var() css function), but the example css you used could
> easily be rewritten to not use that feature.
> 
Thanks, that sounds useful. IIRC, I have to create a template for the 
'layout grid' and add custom styles to that using TemplateStyles and use 
that template on my user page. Sounds like a lot of work.


--

Kaartic
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] How to add custom styles to a page?

[Brad Jorsch (Anomie)]

On Sat, Sep 16, 2017 at 5:24 PM, Gergo Tisza  wrote:

> On Sat, Sep 16, 2017 at 7:46 AM, Brian Wolff  wrote:
>
> > What you are looking for is TemplateStyles. This is not enabled on
> english
> > wikipedia yet, but should be coming there soon (I think). It would allow
> > you to use @media styles for your pages. See
> > https://www.mediawiki.org/wiki/Help:TemplateStyles for more details.
> >
> > Templatestyles does not let you use css variables (the properties
> starting
> > with -- and the var() css function), but the example css you used could
> > easily be rewritten to not use that feature.
> >
>
> You could also just put it into User:Kaartic/common.css and wrap it in a
> .page-User_Kaartic selector, that is less restrictive.
>

Although that will only apply the CSS for you while you're logged in, no
one else will see it.

-- 
Brad Jorsch (Anomie)
Senior Software Engineer
Wikimedia Foundation
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Can we drop revision hashes (rev_sha1)?

[Dan Andreescu]

So, as things stand, rev_sha1 in the database is used for:

1. the XML dumps process and all the researchers depending on the XML dumps
(probably just for revert detection)
2. revert detection for libraries like python-mwreverts [1]
3. revert detection in mediawiki history reconstruction processes in Hadoop
(Wikistats 2.0)
4. revert detection in Wikistats 1.0
5. revert detection for tools that run on labs, like Wikimetrics
?. I think Aaron also uses rev_sha1 in ORES, but I can't seem to find the
latest code for that service

If you think about this list above as a flow of data, you'll see that
rev_sha1 is replicated to xml, labs databases, hadoop, ML models, etc.  So
removing it and adding it back downstream from the main mediawiki database
somewhere, like in XML, cuts off the other places that need it.  That means
it must be available either in the mediawiki database or in some other
central database which all those other consumers can pull from.

I defer to your expertise when you say it's expensive to keep in the db,
and I can see how that would get much worse with MCR.  I'm sure we can
figure something out, though.  Right now it seems like our options are, as
others have pointed out:

* compute async and store in DB or somewhere else that's central and easy
to access from all the branches I mentioned
* update how we detect reverts and keep a revert database with good
references to wiki_db, rev_id so it can be brought back in context.

Personally, I would love to get better revert detection, using sha1 exact
matches doesn't really get to the heart of the issue.  Important phenomena
like revert wars, bullying, and stalking are hiding behind bad revert
detection.  I'm happy to brainstorm ways we can use Analytics
infrastructure to do this.  We definitely have the tools necessary, but not
so much the man-power.  That said, please don't strip out rev_sha1 until
we've accounted for all its "data customers".

So, put another way, I think it's totally fine if we say ok everyone, from
date XYZ, you will no longer have rev_sha1 in the database, but if you want
to know whether an edit reverts a previous edit or a series of edits, go
*HERE*.  That's fine.  And just for context, here's how we do our revert
detection in Hadoop (it's pretty fancy) [2].


[1] https://github.com/mediawiki-utilities/python-mwreverts
[2]
https://github.com/wikimedia/analytics-refinery-source/blob/1d38b8e4acfd10dc811279826ffdff236e8b0f2d/refinery-job/src/main/scala/org/wikimedia/analytics/refinery/job/mediawikihistory/denormalized/DenormalizedRevisionsBuilder.scala#L174-L317

On Mon, Sep 18, 2017 at 9:19 AM, Daniel Kinzler  wrote:

> Am 16.09.2017 um 01:22 schrieb Matthew Flaschen:
> > On 09/15/2017 06:51 AM, Daniel Kinzler wrote:
> >> Also, I believe Roan is currently looking for a better mechanism for
> tracking
> >> all kinds of reverts directly.
> >
> > Let's see if we want to use rev_sha1 for that better solution (a way to
> track
> > reverts within MW itself) before we drop it.
>
>
> The problem is that if we don't drop is, we have to *introduce* it for the
> new
> content table for MCR. I'd like to avoid that.
>
> I guess we can define the field and just null it, but... well. I'd like to
> avoid
> that.
>
>
> --
> Daniel Kinzler
> Principal Platform Engineer
>
> Wikimedia Deutschland
> Gesellschaft zur Förderung Freien Wissens e.V.
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Can we drop revision hashes (rev_sha1)?

[Daniel Kinzler]

Am 16.09.2017 um 01:22 schrieb Matthew Flaschen:
> On 09/15/2017 06:51 AM, Daniel Kinzler wrote:
>> Also, I believe Roan is currently looking for a better mechanism for tracking
>> all kinds of reverts directly.
> 
> Let's see if we want to use rev_sha1 for that better solution (a way to track
> reverts within MW itself) before we drop it.


The problem is that if we don't drop is, we have to *introduce* it for the new
content table for MCR. I'd like to avoid that.

I guess we can define the field and just null it, but... well. I'd like to avoid
that.


-- 
Daniel Kinzler
Principal Platform Engineer

Wikimedia Deutschland
Gesellschaft zur Förderung Freien Wissens e.V.

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Content Translation office hour and online meeting on September 20, 2017 (Wednesday) at 1300 UTC

[mathieu stumpf guntz]

Hello Runa,

Thank you for the information.


Le 18/09/2017 à 11:06, Runa Bhattacharjee a écrit :


This session is going to be an online discussion over Google
Hangouts/Youtube with a simultaneous IRC conversation. Due to the
limitation of Google Hangouts, only a limited number of participation slots
are available. Hence, do please let us know in advance if you would like to
join in the Hangout. The IRC channel will be open for interactions during
the session.
Did you consider using an other solution, such as the jisti 
 FLOSS?

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Content Translation office hour and online meeting on September 20, 2017 (Wednesday) at 1300 UTC

[Runa Bhattacharjee]

[x-posted announcement]

Hello,

Wikimedia Foundation’s development team working on Content Translation -
the tool that helps you create new Wikipedia articles by translating from
another article, would like to invite you for an online office hour session
scheduled for Wednesday, September 20th, 2017 at 13:00 UTC. This will be an
open session to talk about recent changes to Content Translation and
ongoing development.

You can know more about Content Translation on the project page
 or even watch a short
video . We have discussed
Content Translation in the past as part of the office hours hosted by WMF
Language team. The last was in June and you can watch the recording at:
https://www.youtube.com/watch?v=8Euhu4Q7HF4 .

This session is going to be an online discussion over Google
Hangouts/Youtube with a simultaneous IRC conversation. Due to the
limitation of Google Hangouts, only a limited number of participation slots
are available. Hence, do please let us know in advance if you would like to
join in the Hangout. The IRC channel will be open for interactions during
the session.

Please read below for the event details, including local time, youtube
session links and do let us know if you have any questions.

Thank you
Runa

== Details ==

# Event: Content Translation office hour session

# When: September 20th, 2017 (Wednesday) at 13:00 UTC (check local time
http://www.timeanddate.com/worldclock/fixedtime.html?iso=20170920T1300)

# Where:On IRC #wikimedia-office (Freenode) and
https://www.youtube.com/watch?v=MD-BKoSj-oY


# Agenda:
Recent updates about Content Translation, and Q & A.


-- 
Engineering Manager, Global Collaboration (Contributors)
Wikimedia Foundation
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l