[Wikitech-l] Building tools, services and datasets that respect deleted and suppressed revisions

[Lucas Dixon]

Hello,

Yiqing (cc'd) and I have been working on tools and corpora that transform
wiki revision dumps into structured conversations; as part of this, we want
to make sure any down-stream services and corpora that we develop respect
the deleted (and suppressed) revisions; namely that we remove any copies we
have of things deleted on Wikipedia.

For that we need a way to:
1. get all revisions IDs that were deleted or suppressed (or all
non-deleted and non-suppressed ones)
2. have a way to get new deletions or suppressions so that we can remove
any copies that we have.

What's the right infrastructure/APIs to use for this?

Thanks!
lucas
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] wgEmergencyContact - n...@wikipedia.org

[Daniel Zahn]

n...@wikipedia.org is a real working address. It is an alias for
n...@wikimedia.org and that is an alias for root@ and that is an alias for
all people with root on the cluster. You would reach real people there.
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Phabricator monthly statistics - 2018-05

[communitymetrics]

Hi Community Metrics team,

This is your automatic monthly Phabricator statistics mail.

Accounts created in (2018-05): 312
Active Maniphest users (any activity) in (2018-05): 999
Task authors in (2018-05): 517
Users who have closed tasks in (2018-05): 284

Projects which had at least one task moved from one column to another on
their workboard in (2018-05): 288

Tasks created in (2018-05): 2670
Tasks closed in (2018-05): 2055
Open and stalled tasks in total: 38640

Median age in days of open tasks by priority:

Unbreak now: 20
Needs Triage: 389
High: 685
Normal: 947
Low: 1211
Lowest: 1177

(How long tasks have been open, not how long they have had that priority)

Active Differential users (any activity) in (2018-05): 19

TODO: Numbers which refer to closed tasks might not be correct, as
described in https://phabricator.wikimedia.org/T1003 .

Yours sincerely,
Fab Rick Aytor

(via community_metrics.sh on phab1001 at Fri Jun  1 00:00:20 UTC 2018)

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Your feedback requested - Beta Cluster Survey...

[Jean-Rene Branaa]

The Wikimedia Foundation Technology department is seeking feedback
regarding your current and future use of the Beta Cluster[0].

The anonymized results of this survey will be used by the Wikimedia
Foundation Technology department to inform future decisions.

This survey will be conducted via a third-party service, which may subject
it to additional terms. For more information on privacy and data-handling,
see the survey privacy statement[1].

Please help us improve the Beta Cluster by filling out this quick survey:

https://goo.gl/forms/XgIxXiSi1G5eVHbp2

This survey will be open until June 15th, 2018.

Thanks!

Jean-René Branaa
Wikimedia Foundation - Technology


[0] https://www.mediawiki.org/wiki/Beta_Cluster
[1]
https://wikimediafoundation.org/wiki/Beta_Cluster_Survey_Privacy_Statement

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] TechCom Radar 2018-05-30

[Kate Chapman]

Hi All,

Here are the minutes from this week's TechCom meeting:

* TechCom will be announcing a call for nominations to expand the
committee next week

* Discussion and community input on MediaWiki Platform Architecture
Principles Document continues:


* Public IRC Discussion of RFC next week 2018-06-06 in the
#wikimedia-office channel at 2pm PST(22:00 UTC, 23:00 CET): Use
ar_page_id to determine the parent IDs for undeleted revisions



You can also find our meeting minutes at


See also the TechCom RFC board
.

-- 
Kate Chapman
TechCom Facilitator (Contractor)


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Planet Wikimedia using new software "rawdog"

[Daniel Zahn]

On Wed, May 30, 2018 at 9:58 PM, Brion Vibber  wrote:

> > I would probably recommend googling software names for common
> alternate meanings


Fair enough. yea, i wasn't really aware / noticed this.
Though i think we might have a Barbara-Streisand-effect and now everybody
looked it up.


> before choosing to put them in a public-facing role,
>

The end-user isn't really exposed to it besides a small "powered by" link
at the bottom. Also not the name of the puppet module,
just a package name in Debian.  Not like we are running rawdog.wikimedia or
anything.


On Thu, May 31, 2018 at 2:08 AM, David Cuenca Tudela 
wrote:

> I like the new design

:) It's from Planet KDE and customized.  Paladox did that.  Files are here:
https://gerrit.wikimedia.org/r/#/c/435327/

> missing some information about the source of each post.

Please see the screenshot Paladox just sent. Is that what you meant?

>  could the Planet be linked from the Wikimedia blog

Sure, i'm all for that. As Andre pointed out this would be something for
people maintaining the blog though. That is hosted outside our
infrastructure and i don't personally have access to that.

-- 
Daniel Zahn 
Operations Engineer
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Planet Wikimedia using new software "rawdog"

[Paladox]

 https://phabricator.wikimedia.org/F18632742




On Thursday, 31 May 2018, 14:45:45 BST, Paladox 
 wrote:  
 
  Hi, something like ⇪ Screen Shot 2018-05-31 at 14.45.05.png ?

| 
| 
|  | 
⇪ Screen Shot 2018-05-31 at 14.45.05.png


 |

 |

 |




On Thursday, 31 May 2018, 14:12:58 BST, David Cuenca Tudela 
 wrote:  
 
 On Thu, May 31, 2018 at 12:08 PM, Andre Klapper 
wrote:

> The source website is linked from the date header.
>

I'm aware of that, but in my opinion that is not visible enough.


> That's a question for the blog maintainers that you could file at
> https://phabricator.wikimedia.org/maniphest/task/edit/form/
> 1/?projects=Wikimedia-Blog


 Thanks for the pointer. Done:
https://phabricator.wikimedia.org/T196069

Regards,
Micru
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Planet Wikimedia using new software "rawdog"

[Paladox]

 Hi, something like ⇪ Screen Shot 2018-05-31 at 14.45.05.png ?

| 
| 
|  | 
⇪ Screen Shot 2018-05-31 at 14.45.05.png


 |

 |

 |




On Thursday, 31 May 2018, 14:12:58 BST, David Cuenca Tudela 
 wrote:  
 
 On Thu, May 31, 2018 at 12:08 PM, Andre Klapper 
wrote:

> The source website is linked from the date header.
>

I'm aware of that, but in my opinion that is not visible enough.


> That's a question for the blog maintainers that you could file at
> https://phabricator.wikimedia.org/maniphest/task/edit/form/
> 1/?projects=Wikimedia-Blog


 Thanks for the pointer. Done:
https://phabricator.wikimedia.org/T196069

Regards,
Micru
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l  
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Planet Wikimedia using new software "rawdog"

[David Cuenca Tudela]

On Thu, May 31, 2018 at 12:08 PM, Andre Klapper 
wrote:

> The source website is linked from the date header.
>

I'm aware of that, but in my opinion that is not visible enough.


> That's a question for the blog maintainers that you could file at
> https://phabricator.wikimedia.org/maniphest/task/edit/form/
> 1/?projects=Wikimedia-Blog


 Thanks for the pointer. Done:
https://phabricator.wikimedia.org/T196069

Regards,
Micru
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] change to output file numbering of big wikis

[Ariel Glenn WMF]

TL;DR:
Scripts that reply on xml files numbered 1 through 4 should be updated to
check for 1 through 6.

Explanation:

A number of wikis have stubs and page content files generated 4 parts at a
time, with the appropriate number added to the filename. I'm going to be
increasing that thi month to 6.

The reason for the increase is that near the end of the run there are
usually just a few big wikis taking their time at completing. If they run
with 6 processes at once, they'll finish up a bit sooner.

If you have scripts that rely on the number 4, just increase it to 6 and
you're done.

This will go into effect for the June 1 run and all runs afterwards.

Thanks!
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Planet Wikimedia using new software "rawdog"

[Andre Klapper]

Hi,

On Thu, 2018-05-31 at 11:08 +0200, David Cuenca Tudela wrote:
> I like the new design, however I am missing some information about the
> source of each post. Would it be possible to add the source website to each
> post?

The source website is linked from the date header.

> And another more general question I have is, could the Planet be linked
> from the Wikimedia blog? I feel that it is quite hidden now, so by linking
> it from the blog maybe it would gain visibility.

That's a question for the blog maintainers that you could file at
https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=Wikimedia-Blog

Cheers,
andre
-- 
Andre Klapper | Wikimedia Bugwrangler
https://blogs.gnome.org/aklapper/


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Planet Wikimedia using new software "rawdog"

[David Cuenca Tudela]

Hi Daniel,

I like the new design, however I am missing some information about the
source of each post. Would it be possible to add the source website to each
post?

And another more general question I have is, could the Planet be linked
from the Wikimedia blog? I feel that it is quite hidden now, so by linking
it from the blog maybe it would gain visibility.

Regards,
Micru

On Thu, May 31, 2018 at 3:33 AM, Daniel Zahn  wrote:

> Hi,
>
> this is an announcement to let you know that the service ($lang).
> planet.wikimedia.org, an RSS feed aggregator for all Wikimedia related
> blogs, has switched software.
>
> You can find the English version at  https://en.planet.wikimedia.org/
>
> Other existing languages are listed on
> https://wikitech.wikimedia.org/wiki/Planet.wikimedia.org#
> Which_languages_exist
> ?
>
> Today we moved away from planet-venus and to a newer package called
> "rawdog" that does the same thing as before, fetching a bunch of RSS feed
> and combining them into a single page and feed.
>
> The reason is that planet-venus has been dropped in Debian stable (stretch)
> because it was unmaintained, so we had to find an alternative to be able to
> upgrade the underlying servers to a current OS version.
>
> If you never heard of planet, here you can find more info:
>
> https://wikitech.wikimedia.org/wiki/Planet.wikimedia.org
>
> If you already use it but just subscribe to the "feed of feeds" then
> nothing should change for you.
>
> (Though note that we support RSS 2.0 but not a separate Atom feed anymore.
> We are redirecting the old atom.xml URL to the new (and old) URL
> rss20.xml.)
>
> If you already use it and look at the web UI, enjoy the new theme that
> Paladox imported from KDE to make it look about 150% better than before.
> (thanks to him for that theming work!)
>
> We also applied patches to make it look more like our former planet for a
> smooth transition.  A "wmf1" package has been built and uploaded at
> https://apt.wikimedia.org/wikimedia/pool/main/r/rawdog/
>
> If you want to know more about "rawdog":
>
> https://offog.org/code/rawdog/
> https://packages.debian.org/stretch/rawdog
>
> If you want to add your blog feed, feel free to upload changes or just drop
> me a mail.
>
> Bugs can be reported here:
> https://phabricator.wikimedia.org/project/view/413/
>
> Tickets are:  https://phabricator.wikimedia.org/T180498 ,
> https://phabricator.wikimedia.org/T168490
>
> Cheers,
>
> Daniel
>
> --
> Daniel Zahn 
> Operations Engineer
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l




-- 
Etiamsi omnes, ego non
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] New feature deployed: Finding text changes in moved paragraphs in diffs

[Victoria Coleman]

Very excited to see this get deployed widely. Very timely actually! Congrats to 
the team!

Best,

Victoria 

> On May 30, 2018, at 6:34 AM, Birgit Müller  
> wrote:
> 
> Hi all,
> 
> 
> over the past 1,5 years WMDE’s Technical Wishes team was working on a major
> improvement of wikidiff2 [1] - the engine behind the wikitext diff view.
> The major outcome of our improvements got deployed to most wikis just now:
> It is now easier to find moved paragraphs and text changes inside them. [2]
> 
> 
> Why we worked on this:
> 
> The idea for the project arose from the wishlist of the German-speaking
> communities: Show text changes in moved paragraphs. Before we worked on
> this, moved paragraphs were shown in the diff as a deleted paragraph and
> then as an added paragraph. This meant that you had to parse visually
> through the paragraphs to find any potential changes - a fact that vandals
> liked to make use of. [3]
> 
> 
> Desktop version, mobile version
> 
> The desktop version of the feature already got deployed to Mediawiki.org
> and deWP in December 2017. After further improvements of the diff
> algorithm, the change could now get deployed to most wikis. The languages
> unified Han, Thai and Japanese that don’t use spaces as delimiters will
> follow soon.
> The mobile version is in the making and will follow in the next weeks. The
> backend work has been done by the WMDE team, the WMF’s mobile team is
> taking care of the design work for the user interface of the mobile version.
> 
> Improving the algorithm of wikidiff2: More than a new feature
> 
> The work that has been done goes further than the original wish: First of
> all, we not only highlighted text changes within a moved paragraph, but
> also made it easier to find moved paragraphs in the first place: Clickable
> arrows allow to follow the move of the paragraph.
> Second, we generally improved the diff algorithm: For example, we could
> find an old bug in the diff engine: Sometimes two paragraphs which are not
> related at all were considered a changed paragraph. The bug fix was already
> deployed in October 2017. [4]
> 
> If you are interested in learning more about the work that has been done,
> we're going to publish the lessons learned by the dev team within the next
> days (so stay tuned :-) !.
> 
> 
> Thank you!
> 
> We would like to thank everyone who supported us in the work around
> wikidiff2: People from different wikis looked into diffs and gave us
> valuable feedback. Moritz Mühlenhoff supported us by updating the library,
> and deploying it. Tim Starling and Max Semenik supported us with code
> review and consultation during the Vienna Hackathon. The Desktop and Mobile
> Web team is going to add a user friendly design to the mobile version. Your
> advice, feedback and support is much appreciated :-)
> 
> 
> Further feedback is very welcome!
> 
> Birgit (for the technical wishes team)
> 
> 
> [1] Extension manual: https://www.mediawiki.org/wiki/Extension:Wikidiff2
> 
> [2] Deployment ticket: https://phabricator.wikimedia.org/T195375
> 
> [3] Main project page on meta:
> https://meta.wikimedia.org/wiki/WMDE_Technical_Wishes/Show_text_changes_when_moving_text_chunks
> 
> [4] Patch on gerrit for the bug fix:
> https://gerrit.wikimedia.org/r/#/c/356582/
> 
> 
> -- 
> Birgit Müller
> Community Communications Manager
> Software Development and Engineering
> 
> 
> 
> 
> Wikimedia Deutschland e.V. | Tempelhofer Ufer 23-24 | 10963 Berlin
> Tel. (030) 219 158 26-0
> http://wikimedia.de
> 
> Stellen Sie sich eine Welt vor, in der jeder Mensch an der Menge allen
> Wissens frei teilhaben kann. Helfen Sie uns dabei!
> http://spenden.wikimedia.de/
> 
> Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
> Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
> der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
> Körperschaften I Berlin, Steuernummer 27/681/51985.
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l