Re: [Wikitech-l] Bulk link rewrites for HTTP -> HTTPS migration?

[Max Semenik]

Fix them with a bot, for example AWB
.

On Wed, Jan 13, 2016 at 9:09 AM, Chris Adams  wrote:

> I've been working with a number of colleagues getting ready to turn HTTPS
> on by default for various loc.gov domains. This has been fairly successful
> and we're working through the old legacy apps now.
>
> When that work completes, we'll have somewhere around half a million links
> which differ only in the URL scheme. What would be the best way to rewrite
> all of those URLs? I'd like to reduce the window during which users transit
> from HTTPS -> HTTP -> HTTPS.
>
> If anyone's curious, I've been collecting the links for a few dozen wikis
> in a somewhat oversized Git repo:
>
> https://github.com/acdha/lc-wikipedia-links
>
> The first site which has completely migrated is the much smaller World
> Digital Library which has just under four thousand links:
> https://gist.github.com/acdha/f785b22b356a9842439e
>
> Thanks,
> Chris
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l




-- 
Best regards,
Max Semenik ([[User:MaxSem]])
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Video: Loading Barack Obama on 2G from Spain on a Nexus 5

[Bernardo Sulzbach]

Thanks, Joaquin. I image how tiresome waiting 3 minutes of loading
(for a web page!) must be.

I just hope that no one decides that this makes a good reason for
"splitting" the Obama article. I would expect more improvements from
the browser itself, more on-the-fly rendering of the page as you get
data.

I've seen forum threads (think about GitHub issues) reach gargantuan
sizes (way more than a mebibyte). I think it is the browser's job to
handle these scenarios.

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Video: Loading Barack Obama on 2G from Spain on a Nexus 5

[Joaquin Oltra Hernandez]

Sure Bernardo, happy to share.

This was the best take I got from around 4 runs. I have other video that I
haven't uploaded that goes on for 4 minutes and tries to close chrome 3
times (it is painful and boring to watch).

With high speed connections I haven't experienced this behaviour, the worst
I've experienced have been around 1s-2s of locked browser (not being able
to interact at all) but it quickly recovers and it doesn't show the
unresponsive app dialog.

I tried diagnosing using chrome developer tools connected to the device but
the timeline profiles I recorded didn't show anything really evident (and
it should, since it spent 20s blocked). It was pretty confusing, and my
best guess is that the sheer size of the HTML (ungzipping, parsing, laying
it out, etc) (+1mb of html for B. Obama), and the 100+ images that are
being fetched (both the network requests, and the multiple (slow) streams
of image data that need to be decoded by the phone) are somehow creating a
huge bottleneck on the phone's CPU.

I'm thinking of repeating the experiment disabling images and see what
difference it makes, to find out if it is the multiple images being decoded
in parallel really slowly (because of network speed).

You're right Bernardo in that Nexus5 is supposed to be a good phone (not
the best, but "modern" for sure), that's why I asked if more people could
try doing the same thing and reporting back.

I'll also try to run the same experiment with different devices (got a few
of old ones for testing), that should yield interesting results.

As far as my conclusions go, we need more data from real tests on real
devices, and to really care more about this, given the industry numbers and
predictions around 2G (
https://commons.wikimedia.org/w/index.php?title=File:Paradigm.pdf=8)

I'll report back when I get around to testing more!

On Tue, Jan 12, 2016 at 10:24 AM, Brian Wolff  wrote:

> My apologies. I guess I read the original email too fast.
>
> On Tuesday, January 12, 2016, Bernardo Sulzbach  >
> wrote:
> > Brian, you seem off. Do you know that we are talking about loading the
> > Obama page, right?
> >
> > ___
> > Wikitech-l mailing list
> > Wikitech-l@lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Bulk link rewrites for HTTP -> HTTPS migration?

[Chris Adams]

On Wed, Jan 13, 2016 at 1:49 PM, Risker  wrote:

> Before properly answering this question, it's important to know how many
> links we're talking about.  If it's 5000, the fallout is probably
> manageable; but if it's in the hundreds of thousands on any project (most
> likely enwiki) there will be renting of garments and gnashing of teeth.
> All those changes show up on people's watchlists, after all.
>

Yes, that's exactly what I'd like to avoid. The first batch of URLs which
is ready to go is small (~4K) but the full list is significantly larger and
many of those are used on multiple pages so the edit churn would be
non-trivial.


> Please also ensure that if you're changing the URL, it's not just a http
> --> https swap, but that the new URL is tested to verify it lands on a real
> page.  There are no doubt plenty of bad links in amongst all those URLs -
> even government websites rearrange themselves periodically -  and replacing
> a bad link with a more secure bad link is not really helpful.
>

Yes – part of this project on our side is setting permanent redirects not
just for the protocol but also for pages which have moved into a different
application. This is the other side of what Oliver Keyes was asking about
where there are a mix of legacy applications which are non-trivial to
rewrite but also many thousands of URLs where a simple regex could handle
both the protocol change and switching to the canonical item page in the
modern unified app instead of continuing to use a long-deprecated legacy
view. Internally we've been working to chunk that list of URLs into
patterns by application / project so they can be reviewed and tested in a
reasonable amount of time.

Chris
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Bulk link rewrites for HTTP -> HTTPS migration?

[Risker]

Before properly answering this question, it's important to know how many
links we're talking about.  If it's 5000, the fallout is probably
manageable; but if it's in the hundreds of thousands on any project (most
likely enwiki) there will be renting of garments and gnashing of teeth.
All those changes show up on people's watchlists, after all.

Please also ensure that if you're changing the URL, it's not just a http
--> https swap, but that the new URL is tested to verify it lands on a real
page.  There are no doubt plenty of bad links in amongst all those URLs -
even government websites rearrange themselves periodically -  and replacing
a bad link with a more secure bad link is not really helpful.

Risker/Anne

On 13 January 2016 at 13:32, Max Semenik  wrote:

> Fix them with a bot, for example AWB
> .
>
> On Wed, Jan 13, 2016 at 9:09 AM, Chris Adams  wrote:
>
> > I've been working with a number of colleagues getting ready to turn HTTPS
> > on by default for various loc.gov domains. This has been fairly
> successful
> > and we're working through the old legacy apps now.
> >
> > When that work completes, we'll have somewhere around half a million
> links
> > which differ only in the URL scheme. What would be the best way to
> rewrite
> > all of those URLs? I'd like to reduce the window during which users
> transit
> > from HTTPS -> HTTP -> HTTPS.
> >
> > If anyone's curious, I've been collecting the links for a few dozen wikis
> > in a somewhat oversized Git repo:
> >
> > https://github.com/acdha/lc-wikipedia-links
> >
> > The first site which has completely migrated is the much smaller World
> > Digital Library which has just under four thousand links:
> > https://gist.github.com/acdha/f785b22b356a9842439e
> >
> > Thanks,
> > Chris
> > ___
> > Wikitech-l mailing list
> > Wikitech-l@lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
>
>
>
> --
> Best regards,
> Max Semenik ([[User:MaxSem]])
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Bulk link rewrites for HTTP -> HTTPS migration?

[Oliver Keyes]

Question; are LOC links handled in a standardised way using a
template? Because if so this could be one change, not hundreds of
thousands.

(If it's not I'd really suggest using the same edit sets and
opportunity to restructure them that way, if LOC links are consistent
enough for it to be done. That way you'll both avoid this problem in
the future if something goes kooky - only have to make one edit! - and
have a much easier way of identifying how many there are and where
they live)

On 13 January 2016 at 10:49, Risker  wrote:
> Before properly answering this question, it's important to know how many
> links we're talking about.  If it's 5000, the fallout is probably
> manageable; but if it's in the hundreds of thousands on any project (most
> likely enwiki) there will be renting of garments and gnashing of teeth.
> All those changes show up on people's watchlists, after all.
>
> Please also ensure that if you're changing the URL, it's not just a http
> --> https swap, but that the new URL is tested to verify it lands on a real
> page.  There are no doubt plenty of bad links in amongst all those URLs -
> even government websites rearrange themselves periodically -  and replacing
> a bad link with a more secure bad link is not really helpful.
>
> Risker/Anne
>
> On 13 January 2016 at 13:32, Max Semenik  wrote:
>
>> Fix them with a bot, for example AWB
>> .
>>
>> On Wed, Jan 13, 2016 at 9:09 AM, Chris Adams  wrote:
>>
>> > I've been working with a number of colleagues getting ready to turn HTTPS
>> > on by default for various loc.gov domains. This has been fairly
>> successful
>> > and we're working through the old legacy apps now.
>> >
>> > When that work completes, we'll have somewhere around half a million
>> links
>> > which differ only in the URL scheme. What would be the best way to
>> rewrite
>> > all of those URLs? I'd like to reduce the window during which users
>> transit
>> > from HTTPS -> HTTP -> HTTPS.
>> >
>> > If anyone's curious, I've been collecting the links for a few dozen wikis
>> > in a somewhat oversized Git repo:
>> >
>> > https://github.com/acdha/lc-wikipedia-links
>> >
>> > The first site which has completely migrated is the much smaller World
>> > Digital Library which has just under four thousand links:
>> > https://gist.github.com/acdha/f785b22b356a9842439e
>> >
>> > Thanks,
>> > Chris
>> > ___
>> > Wikitech-l mailing list
>> > Wikitech-l@lists.wikimedia.org
>> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>>
>>
>>
>>
>> --
>> Best regards,
>> Max Semenik ([[User:MaxSem]])
>> ___
>> Wikitech-l mailing list
>> Wikitech-l@lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l



-- 
Oliver Keyes
Count Logula
Wikimedia Foundation

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Bulk link rewrites for HTTP -> HTTPS migration?

[Chris Adams]

On Wed, Jan 13, 2016 at 12:47 PM, Legoktm 
wrote:

> You can use Pywikbot's replace.py[1], which lets you provide regex
> find/replace and can get a list of pages from the API equivalent of
> Special:LinkSearch.
>

Thanks - I'll look into that as we get various batches of URLs ready for
testing.


> You should also consider setting up HSTS[2] so regardless if users click
> on an HTTP link, they'll be sent to the HTTPS version of the site.
>

Yes – that's on the plan as soon as we finishing remediating the older
legacy content. I've been using lists from Wikipedia, a sampling of web
access logs, etc. to feed a script[1] to find cases where someone used an
absolute URL in a 

Re: [Wikitech-l] Accuracy review on tool labs?

[John]

It might be due to DMCA safe harbor restrictions

On Wednesday, January 13, 2016, Brian Wolff  wrote:

> On Wednesday, January 13, 2016, James Salsman  > wrote:
> > John,
> >
> > Are you sure it makes sense to run accuracy review on wikimedia servers?
> >
> > Best regards,
> > Jim
> > ___
> > Wikitech-l mailing list
> > Wikitech-l@lists.wikimedia.org 
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
> When changing the subject headline, please include (was old subject
> headline) so we can follow the context.
>
> And yes it absolutely makes sense to run tools (of all sorts) on tool labs.
> That is literally the primary purpose of tool labs - to run tools. There
> are certain things that arent allowed in the tool labs rules, but this
> hardly seems like one of then.
>
> --
> bawolff
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org 
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] 2015-01-13 Scrum of Scrums meeting notes

[Grace Gellerman]

https://www.mediawiki.org/wiki/Scrum_of_scrums/2016-01-13

= 2015-01-13 =
== Reading ==
=== Android ===
*v2.1.137 beta published. Includes a new native article toolbar and lower
memory usage.

=== Reading Infrastructure ===
* Nothing blocked this week.
* SessionManager and bot passwords should be going out next week.

== Discovery ==
* Working on load test for Dallas cluster
* Load spikes on weekend for ES getting worse, cause not clear yet,
investigating ( https://grafana.wikimedia.org/dashboard/db/elasticsearch )
** Did some brainstorming, discovered queries generating a lot of ES calls
to handle variants, with very low success rate. Will investigate removing
them.
* Finalizing data models for TextCat language detection library, security
review next
* Testing for upgrading WDQS to Blazegraph 2.0 once it's released
* No blockers

== Community Tech ==
* Working on assessment report for Community Wishlist Top 10 results:
https://meta.wikimedia.org/wiki/2015_Community_Wishlist_Survey/Top_10/Status_draft
** Would like feedback from Wikidata, Multimedia, Analytics, and
Collaboration before publishing
* Working on dead link rescuing bot - talking with Tool Labs and IA
* Waiting on feedback from Kunal on Gadgets 2.0
* Wrapping up work on PageAssessments extension:
https://www.mediawiki.org/wiki/Extension:PageAssessments

== Infrastructure ==

=== RelEng ===
*Blocking: nothing
*Blocked on: nothing
*Updates:
** Differential migration discussion at WikiDev16 went really well, very
positive, no roadblocks placed
** Scap3 discussion at WikiDev16 also really positive, looking for more
early adopters

=== Technical Operations ===
*Blocking: nothing
*Blocked on: nothing
*Updates:
** Started work on the new Quarter goals

=== Services ===
* moved to Node 4.2 - RESTBase, Mobile Content Service, Mathoid
:* in progress: Graphoid and Citoid
* working on enabling pre-generation for Mobile Content Service

=== Security ===
* Should get to Kartographer this week
* Several patches to deploy this week
* Schedule your reviews!

=== Fundraising Tech ===
* cleaning up after December
* updating CiviCRM extensions to use new core features

== Editing ==

=== VisualEditor ===
*Blocking: nothing
*Blocked on: nothing
*Updates:
** Single Edit Tab work nearing completion. Planned to release to one wiki
next week if all goes well for validation before larger rollout.
** Note the OOUI breaking release from yesterday (in -wmf.11); no known
impacts.

=== Parsing ===
*Blocking: nothing
*Blocked on: nothing
*Updates
** Hoping to finish up (this week) puppetizing testing infrastructure on
ruthenium
** Coordinating with reading and services to reduce size of HTML payload.

=== Multimedia ===
*Blocking: nothing
*Blocked on: nothing
*Updates:
** Christmas A/B test for cross-wiki upload tool inconclusive. Not sure
what's next happening.

=== Language ===
*Blocking: nothing
*Blocked on: nothing

=== Collaboration ===
* Mostly Echo stuff.  Cross-wiki notification stuff is getting closer.
* MediaWiki-Vagrant code for cross-wiki notifications is implemented and
almost merged.  This will also facilitate other local multi-wiki testing,
e.g. Flow.
* Flow Nuke fix merged.
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Tech Talk: Creating Useful Dashboards with Grafana: January 14

[Rachel Farrand]

Please join for the following tech talk:

*Tech Talk**:* Creating Useful Dashboards with Grafana
*Presenter:* Timo Tijhof
*Date:* January 13, 2016
*Time: *21:30 UTC

Link to live YouTube stream 
*IRC channel for questions/discussion:* #wikimedia-office

*Summary: *Over the past two years Timo has worked a lot with our metric
infrastructure and the various visualisation tools in use at Wikimedia. The
aim of this talk is to help you create better dashboards with Grafana. Timo
will discuss the various metric types we have, how they are gathered in our
software environments, and how you can use Graphite to query this data. He
will also share lessons learned in the Performance Team using real world
examples.
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Video: Loading Barack Obama on 2G from Spain on a Nexus 5

[Joaquin Oltra Hernandez]

Can you clarify what you consider "splitting"? I'm interested in knowing
more.

I'm of the opinion that there's only so much that browsers can do, there's
ways of serving the content like lazy loading that still get you the full
content but help the browser do less work.

I also think that it's responsible to use those techniques where required
(low end network conditions on underpowered devices like phones, for
example) as long as the experience is not broken (which needs careful
implementation and consideration!).

Whatever we need to reach as much people as we can! (and make google not
redirect wikipedia to googleweblight on certain countries)

On Wed, Jan 13, 2016 at 12:29 PM, Bernardo Sulzbach <
mafagafogiga...@gmail.com> wrote:

> Thanks, Joaquin. I image how tiresome waiting 3 minutes of loading
> (for a web page!) must be.
>
> I just hope that no one decides that this makes a good reason for
> "splitting" the Obama article. I would expect more improvements from
> the browser itself, more on-the-fly rendering of the page as you get
> data.
>
> I've seen forum threads (think about GitHub issues) reach gargantuan
> sizes (way more than a mebibyte). I think it is the browser's job to
> handle these scenarios.
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Video: Loading Barack Obama on 2G from Spain on a Nexus 5

[Bernardo Sulzbach]

A section that looks only loosely related to the article could get an
article of its own to make the article shorter. Literally, splitting
it into multiple articles. However, I think that does not make sense
in this case.

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Video: Loading Barack Obama on 2G from Spain on a Nexus 5

[Joaquin Oltra Hernandez]

I understand, thanks for the response!

I don't think in any case any implementation would mean splitting article
content in the sense you mentioned here. As you said, it doesn't make sense.

Cheers!

On Wed, Jan 13, 2016 at 2:35 PM, Bernardo Sulzbach <
mafagafogiga...@gmail.com> wrote:

> A section that looks only loosely related to the article could get an
> article of its own to make the article shorter. Literally, splitting
> it into multiple articles. However, I think that does not make sense
> in this case.
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Tech Talk: Creating Useful Dashboards with Grafana: January 14

[Rachel Farrand]

This has a typo - the correct date for this talk is *Tomorrow January 14th.*
Not today January 13th. Thanks!

On Wed, Jan 13, 2016 at 1:38 PM, Rachel Farrand 
wrote:

> Please join for the following tech talk:
>
> *Tech Talk**:* Creating Useful Dashboards with Grafana
> *Presenter:* Timo Tijhof
> *Date:* January 13, 2016
> *Time: *21:30 UTC
> 
> Link to live YouTube stream 
> *IRC channel for questions/discussion:* #wikimedia-office
>
> *Summary: *Over the past two years Timo has worked a lot with our metric
> infrastructure and the various visualisation tools in use at Wikimedia. The
> aim of this talk is to help you create better dashboards with Grafana. Timo
> will discuss the various metric types we have, how they are gathered in our
> software environments, and how you can use Graphite to query this data. He
> will also share lessons learned in the Performance Team using real world
> examples.
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Tech Talk: Creating Useful Dashboards with Grafana: January 14

[Jamison Lofthouse]

Quick question: Is the YouTube stream usually available for replay after
the event has passed? I know I've seen a few tech talks where the broadcast
disappears after it ends. Is there a way to make them available? This tech
talk happens to fall on an unavailable time for me and also has me very
interested.

Thanks,
Negative24

On Wed, Jan 13, 2016 at 9:11 PM Jamison Lofthouse <
jamison.loftho...@gmail.com> wrote:

> The correct time and date link:
> http://www.timeanddate.com/worldclock/fixedtime.html?msg=Timo+Tech+Talk=20160114T2130=1440=1
>
> Negative24
>
> On Wed, Jan 13, 2016 at 4:18 PM Rachel Farrand 
> wrote:
>
>> This has a typo - the correct date for this talk is *Tomorrow January
>> 14th.*
>> Not today January 13th. Thanks!
>>
>> On Wed, Jan 13, 2016 at 1:38 PM, Rachel Farrand 
>> wrote:
>>
>> > Please join for the following tech talk:
>> >
>> > *Tech Talk**:* Creating Useful Dashboards with Grafana
>> > *Presenter:* Timo Tijhof
>> > *Date:* January 13, 2016
>> > *Time: *21:30 UTC
>> > <
>> http://www.timeanddate.com/worldclock/fixedtime.html?msg=Timo+Tech+Talk=20160113T2130=1440=1
>> >
>> > Link to live YouTube stream > >
>> > *IRC channel for questions/discussion:* #wikimedia-office
>> >
>> > *Summary: *Over the past two years Timo has worked a lot with our metric
>> > infrastructure and the various visualisation tools in use at Wikimedia.
>> The
>> > aim of this talk is to help you create better dashboards with Grafana.
>> Timo
>> > will discuss the various metric types we have, how they are gathered in
>> our
>> > software environments, and how you can use Graphite to query this data.
>> He
>> > will also share lessons learned in the Performance Team using real world
>> > examples.
>> >
>> ___
>> Wikitech-l mailing list
>> Wikitech-l@lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Tech Talk: Creating Useful Dashboards with Grafana: January 14

[Jamison Lofthouse]

The correct time and date link:
http://www.timeanddate.com/worldclock/fixedtime.html?msg=Timo+Tech+Talk=20160114T2130=1440=1

Negative24

On Wed, Jan 13, 2016 at 4:18 PM Rachel Farrand 
wrote:

> This has a typo - the correct date for this talk is *Tomorrow January
> 14th.*
> Not today January 13th. Thanks!
>
> On Wed, Jan 13, 2016 at 1:38 PM, Rachel Farrand 
> wrote:
>
> > Please join for the following tech talk:
> >
> > *Tech Talk**:* Creating Useful Dashboards with Grafana
> > *Presenter:* Timo Tijhof
> > *Date:* January 13, 2016
> > *Time: *21:30 UTC
> > <
> http://www.timeanddate.com/worldclock/fixedtime.html?msg=Timo+Tech+Talk=20160113T2130=1440=1
> >
> > Link to live YouTube stream 
> > *IRC channel for questions/discussion:* #wikimedia-office
> >
> > *Summary: *Over the past two years Timo has worked a lot with our metric
> > infrastructure and the various visualisation tools in use at Wikimedia.
> The
> > aim of this talk is to help you create better dashboards with Grafana.
> Timo
> > will discuss the various metric types we have, how they are gathered in
> our
> > software environments, and how you can use Graphite to query this data.
> He
> > will also share lessons learned in the Performance Team using real world
> > examples.
> >
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Accuracy review on tool labs?

[James Salsman]

John,

Are you sure it makes sense to run accuracy review on wikimedia servers?

Best regards,
Jim
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Bulk link rewrites for HTTP -> HTTPS migration?

[Chris Adams]

I've been working with a number of colleagues getting ready to turn HTTPS
on by default for various loc.gov domains. This has been fairly successful
and we're working through the old legacy apps now.

When that work completes, we'll have somewhere around half a million links
which differ only in the URL scheme. What would be the best way to rewrite
all of those URLs? I'd like to reduce the window during which users transit
from HTTPS -> HTTP -> HTTPS.

If anyone's curious, I've been collecting the links for a few dozen wikis
in a somewhat oversized Git repo:

https://github.com/acdha/lc-wikipedia-links

The first site which has completely migrated is the much smaller World
Digital Library which has just under four thousand links:
https://gist.github.com/acdha/f785b22b356a9842439e

Thanks,
Chris
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Bulk link rewrites for HTTP -> HTTPS migration?

[Legoktm]

On 01/13/2016 09:09 AM, Chris Adams wrote:
> I've been working with a number of colleagues getting ready to turn HTTPS
> on by default for various loc.gov domains. This has been fairly successful
> and we're working through the old legacy apps now.

Awesome!

> When that work completes, we'll have somewhere around half a million links
> which differ only in the URL scheme. What would be the best way to rewrite
> all of those URLs? I'd like to reduce the window during which users transit
> from HTTPS -> HTTP -> HTTPS.

You can use Pywikbot's replace.py[1], which lets you provide regex
find/replace and can get a list of pages from the API equivalent of
Special:LinkSearch.

You should also consider setting up HSTS[2] so regardless if users click
on an HTTP link, they'll be sent to the HTTPS version of the site.

[1] https://www.mediawiki.org/wiki/Manual:Pywikibot/replace.py
[2] https://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security

-- Legoktm

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Bulk link rewrites for HTTP -> HTTPS migration?

[P. Josepherum]

If you use Apache, a rewrite rule is the simplest approach and instructions
can be found by searching for "rewrite http to https Apache". A similar
process will work with nginx.

On Wed, 13 Jan 2016, 17:09 Chris Adams  wrote:

> I've been working with a number of colleagues getting ready to turn HTTPS
> on by default for various loc.gov domains. This has been fairly successful
> and we're working through the old legacy apps now.
>
> When that work completes, we'll have somewhere around half a million links
> which differ only in the URL scheme. What would be the best way to rewrite
> all of those URLs? I'd like to reduce the window during which users transit
> from HTTPS -> HTTP -> HTTPS.
>
> If anyone's curious, I've been collecting the links for a few dozen wikis
> in a somewhat oversized Git repo:
>
> https://github.com/acdha/lc-wikipedia-links
>
> The first site which has completely migrated is the much smaller World
> Digital Library which has just under four thousand links:
> https://gist.github.com/acdha/f785b22b356a9842439e
>
> Thanks,
> Chris
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

-- 

Love and waffles,
PJosepherum
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Accuracy review on tool labs?

[Brian Wolff]

On Wednesday, January 13, 2016, James Salsman  wrote:
> John,
>
> Are you sure it makes sense to run accuracy review on wikimedia servers?
>
> Best regards,
> Jim
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

When changing the subject headline, please include (was old subject
headline) so we can follow the context.

And yes it absolutely makes sense to run tools (of all sorts) on tool labs.
That is literally the primary purpose of tool labs - to run tools. There
are certain things that arent allowed in the tool labs rules, but this
hardly seems like one of then.

--
bawolff
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l